| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Active Research
Forum: Mapping Cancer Markers Forum
Thread: Are all MCM1 assimilators running? |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 81
|
|
| Author |
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 1944 Status: Offline Project Badges: 
|
Nothing purged for weeks. for months... Mike   |
||
|
|
Grumpy Swede
Master Cruncher Svíþjóð Joined: Apr 10, 2020 Post Count: 2138 Status: Offline Project Badges: 
|
Hi everyone, I forwarded your analyses and insights to the tech team and they provided the following update: Correct, as deduced by alanb1951 and adriverhoef the workunits assigned to an instance of the MCM1 assimilator are determined by applying the modulo operator to the workunit ID, so 3/4 assimilators for MCM1 are not currently running due to problems with the workunits in their assigned ranges triggering a non-zero exit condition for the assimilator. After resolving the initial issue, we encountered another issue that triggers a different non-zero exit condition in the MCM1 assimilator code. We have tested a modified version of the assimilator that is able to handle these errors and continue processing workunits that are not affected in the assigned range. The new assimilator will mark the workunits that flagged as problematic in assimilation with an appropriate error_mask, so that they can be handled separately as a group, and once we get through them all we should be in the clear. In the interim, we should be able to process the vast majority of workunits that pass the checks in the assimilator by catching these two error conditions, but allowing the assimlator to continue. The new version of the assimilator should be up and running Monday, and the expected outcome is that we will return to the "flurry of assimilations" from early January. If all goes well, we should be able to double the rate again, and then again, essentially by starting more assimilators with a higher modulo, after allowing 48h in between to monitor. Well, this solution posted by TigerLily, on Jan 26, 2024 9:28:17 PM, obviously did not work. I still have MCM1 tasks from back in October 31, 2023, in my Results list, and no MCM1 results has been purged for a long time. |
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2145 Status: Recently Active Project Badges: 
|
It has been a long time indeed, Grumpy Swede: the last time MCM1-results got purged, was on 14 February, as can be read in post 694228, that's two weeks ago (and I still have MCM1-results dating from 31 October lingering, too).
Adri |
||
|
|
Speedy51
Veteran Cruncher New Zealand Joined: Nov 4, 2005 Post Count: 1271 Status: Offline Project Badges: 
|
My oldest return time for MCM is 2023-10-31 07:12:01 UTC
---------------------------------------- |
||
|
|
zdnko
Senior Cruncher Joined: Dec 1, 2005 Post Count: 225 Status: Offline Project Badges: 
|
the same
|
||
|
|
bfmorse
Senior Cruncher US Joined: Jul 26, 2009 Post Count: 295 Status: Offline Project Badges: 
|
My current scorecard of MCM1 WU's: In Progress: 227 items Error: 8 items (described previously in this thread and are still being displayed) Pending Verification: 0 items Pending Validation: 3686 items (2024-02-10 to 2024-02-19) Valid: 157532 items (2023-10-31 to 2023-11-01) In Progress: 228 items (2024-02-14 to 2024-02-22) Note: all dates in () refer to date only in "Sent times" column. All my systems are Windows based. ETA: As of 2/27/2024: Valid: 168452 items (Net increase of 10920) Pending Validation: 4804 items In Progress: 245 items ---------------------------------------- [Edit 1 times, last edit by bfmorse at Feb 27, 2024 5:52:21 PM] My current VALID WU's awaiting assimilation is175093 items. The backlog continues to GROW! |
||
|
|
alanb1951
Veteran Cruncher Joined: Jan 20, 2006 Post Count: 929 Status: Recently Active Project Badges:
|
I've just got round to looking at the Valid task status data I collected at 02:00 this morning, and there are [finally] signs of movement again...
At the time of collection, I had 17815 items stuck at "Valid", and 931 of them have moved on to file delete state 2 (so they should be gone by 02:00 tomorrow...) All [4?] assimilators seemed to be running, and my tasks were modified between 06:43 on the 29th and 02:05 on 1st March, a longer interval than any of the previous restarts. (Times are UTC) It appears that it is working its way upwards through WU numbers (as one might expect) -- by 02:00 all my remaining tasks returned before 2023-11-05 had been through the assimilator (and it had swept up a fair number from 2023-11-05 to 2023-11-07 as well) It is good to see it is tackling the oldest work-units again -- that suggests there may be a solution at last. Here's hoping I haven't jinxed it by mentioning it :-) Cheers - Al. P.S. I've just done another less detailed sweep (at about 11:40 UTC) and it still seems to be chipping away at the backlog. Also, my number of tasks waiting at Valid has gone down so the first items assimilated are now being purged. |
||
|
|
Grumpy Swede
Master Cruncher Svíþjóð Joined: Apr 10, 2020 Post Count: 2138 Status: Offline Project Badges:
|
Very good. My last remaining result from 2023-10-31 is now gone. That's the only one purged though, this time. Next in line to be purged, are results from 2023-11-07 and newer.
----------------------------------------[Edit 1 times, last edit by Grumpy Swede at Mar 1, 2024 1:54:03 PM] |
||
|
|
AgrFan
Senior Cruncher USA Joined: Apr 17, 2008 Post Count: 374 Status: Offline Project Badges: 
|
No change here. My valid results are still growing.
----------------------------------------14728 units, oldest 2023-11-01 16:26:22 UTC [Edit 5 times, last edit by AgrFan at Mar 5, 2024 2:35:44 AM] |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 1944 Status: Offline Project Badges:
|
Well, after more than 4 months now, it should be possible to have a solution for the ever increasing number of validated MCM1 WUs. It's currently at 62,000 for me and about 2 out of 3 times I try to check the result page, those obnoxious progress bars never stop moving.
----------------------------------------Though I did a quick check and the oldest WUs are now from 2023-11-02 (instead of 2023-10-31), so maybe they are (very) slowly being purged...  Ralf |
||
|
|
|