Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 81
Posts: 81   Pages: 9   [ Previous Page | 1 2 3 4 5 6 7 8 9 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 326638 times and has 80 replies Next Thread
Bryn Mawr
Senior Cruncher
Joined: Dec 26, 2018
Post Count: 342
Status: Recently Active
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

Purging of MCM1 results have stopped completely again. No MCM1 results from my results list has been purged for about 2 days. The only purging that works as it should, is for OPNG results.

But, it's not the end of the world. There are plenty of other things going on around us, that are more important.


Strange, my queue has dropped by 1,000 in the past day from 23,3nn to 22,2nn from a peak of 29,8nn
[Mar 10, 2024 11:35:01 AM]   Link   Report threatening or abusive post: please login first  Go to top 
TPCBF
Master Cruncher
USA
Joined: Jan 2, 2011
Post Count: 1944
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

Well, maybe there's finally some light at the end of the tunnel (let's hope it's not an oncoming freight train).
For the first time in weeks, I was able to refresh the "Results" page with only one try and I could notice that the number of still not purged valid WUs had dropped to 48,500, down from a peak of about 63,000...

And I seem to have gotten at least some non-resend MCM1 WUs between all of my hosts.


Ralf
----------------------------------------

[Mar 10, 2024 6:45:16 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7633
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

Purging of MCM1 results have stopped completely again. No MCM1 results from my results list has been purged for about 2 days. The only purging that works as it should, is for OPNG results.

But, it's not the end of the world. There are plenty of other things going on around us, that are more important.


Strange, my queue has dropped by 1,000 in the past day from 23,3nn to 22,2nn from a peak of 29,8nn

I'm down about 2,500 from yesterday, from about 29,000 to about 26,500.

Update March 13, 02:20 UTC: Now down to 22,300 and oldest date is Dec. 18, 2023

Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
----------------------------------------
[Edit 1 times, last edit by Sgt.Joe at Mar 13, 2024 2:25:04 AM]
[Mar 11, 2024 2:36:30 AM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2145
Status: Recently Active
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

Dear volunteers.

By observing that the number of my Valid MCM1-tasks is dropping quite consistently, a careful forecast can already be made. According to my tatest calculations, my prediction is that the backlog of unpurged Valid MCM1-workunits — without any major hiccups —will be completely processed in 11 days [*1] from now.

(*1) The number of my Valid MCM1-tasks was 54 hours ago 74,906 and is currently 62,081. That means that that number is dropping at a rate of 237.5 per hour, that's 5,700 per day. So, eleven days are needed to reach 62,700, which just exceeds my current number of Valids.

Adri
[Mar 13, 2024 6:38:38 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Grumpy Swede
Master Cruncher
Svíþjóð
Joined: Apr 10, 2020
Post Count: 2138
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

Yes, finally my Valid MCM1 tasks is dropping off pretty fast. I will soon have no tasks left to purge for 2023.
[Mar 13, 2024 11:51:21 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Blount
Senior Cruncher
Joined: Aug 19, 2005
Post Count: 439
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

Interesting on Jan 31 i had 213,322 Valids. That was the last time a results query on the web worked. This evening I am at 175,779. :)
[Mar 14, 2024 1:49:44 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Bryn Mawr
Senior Cruncher
Joined: Dec 26, 2018
Post Count: 342
Status: Recently Active
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

I get the feeling that as the database frees up the assimilators are accelerating.

No science behind that, just a feeling.
[Mar 16, 2024 12:04:21 PM]   Link   Report threatening or abusive post: please login first  Go to top 
adriverhoef
Master Cruncher
The Netherlands
Joined: Apr 3, 2009
Post Count: 2145
Status: Recently Active
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

$ sed 's/.*<Result>//' /var/tmp/wcgresults.2025-02-25T09\:59\:00.3285539.0 | perl -w00ne \
'print if /<FileDeleteState>2/' | sed -n 's/.*\(..\)<\/WorkunitId>/\1/p' | sort | uniq -c | pr -t -6
This command will select all results with FileDeleteState = 2 initially, after which it will select two characters from the WorkunitId-field. Those two characters are the last two digits from the Workunit-ID-field and each unique value (00 - 99) should then be counted.

You would then expect about - but not more than - 100 unique values (00 to 99). Let's have a look.

The output should show two pairs of columns (evenly divided by 'pr' across 6 columns):
- to the right, the last two digits of a workunit-ID;
- to the left, the number of unique values for each resulting last two digits of a workunit-ID.

And this is the output:

6 01 5 18 6 35 5 53 4 69 3 85
4 02 6 19 3 37 6 54 6 70 8 86
5 03 6 21 8 38 6 55 6 71 5 87
6 05 8 22 6 39 5 57 8 73 2 89
7 06 1 23 2 41 4 58 5 74 5 90
4 07 4 25 9 42 4 59 6 75 4 91
9 09 3 26 4 43 4 61 7 77 5 93
2 10 3 27 2 45 7 62 6 78 4 94
6 11 3 29 9 46 7 63 8 79 5 95
8 13 4 30 3 47 2 65 4 81 5 97
3 14 6 31 3 49 2 66 9 82 6 98
4 15 7 33 5 50 6 67 5 83 2 99
7 17 5 34 7 51
You can easily see that the values for 00, 04, 08, 12, 16, 20, 24, 28, etc. are not represented in this table, there are only (exactly) 75 values. So this output would show that IDs modulo 4 = 0 are missing and not following the path of FileDeleteState = 2.

So, 25% of the workunit(-ID)s are not being assimilated at the moment.

This is in line with what I'm seeing by looking at the total number of my pending tasks at the moment of this measurement: 660.
--- This total number (Pending Validation/Verification) varies from day to day as follows:
****************************************************************** (660) [today]
***************** (177) [yesterday]
***************** (174) [2 days ago]
******************* (195) [3 days ago]

Adri
[Feb 25, 2025 10:18:15 AM]   Link   Report threatening or abusive post: please login first  Go to top 
alanb1951
Veteran Cruncher
Joined: Jan 20, 2006
Post Count: 929
Status: Recently Active
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

Adri, they aren't even validating, let alone assimilating.

I have only just finished wading through the current state of my pending and recently validated MCM1 tasks, and I can report that every even-numbered WU divisible by 4 that gets enough results to go Pending Validation ends up in PVal jail; some odd-numbered WUs also end up in PVal jail, but many validate (no pattern I can discern at a quick glance), and all even-numbered WUs not divisible by 4 validate promptly!

Given that I'm also seeing that the [rare] ARP1 tasks I get go to PVal jail if the WU number is divisible by 4, and that the occasional "remainder 3" WU also ends up jailed, I wonder exactly where the issue might lie, as the validators for the projects are separate but the effects are similar!

Cheers - Al.
[Feb 25, 2025 11:42:06 AM]   Link   Report threatening or abusive post: please login first  Go to top 
hchc
Veteran Cruncher
USA
Joined: Aug 15, 2006
Post Count: 787
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Are all MCM1 assimilators running?

I assume Jurisica lab only has 1 part-time technical sys admin who probably works on other duties at Krembil, explaining the lack of incident resolution and lack of communication.

If F@h can get corporate sponsorships, so can WCG,which is almost just as old and just as reputable. I wonder if they're even sending out requests.

These kinds of things keep recurring, and when they do recur, they last multiple days. This points to an understaffing issue, from experience.
----------------------------------------
  • i5-7500 (Kaby Lake, 4C/4T) @ 3.4 GHz
  • i5-4590 (Haswell, 4C/4T) @ 3.3 GHz
  • i5-3570 (Broadwell, 4C/4T) @ 3.4 GHz

[Feb 25, 2025 7:46:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 81   Pages: 9   [ Previous Page | 1 2 3 4 5 6 7 8 9 | Next Page ]
[ Jump to Last Post ]
Post new Thread