World Community Grid - View Thread - WORKUNIT (2 tasks) Pending Validation for 6+ days

Hi, Dr Who Fan - welcome to the [current] world of ARP1 at WCG...

Is there a problem with the validators for ARP tasks?

As Mark said, yes! I suspect that there are various interlocked reasons for the current blockage, but it's nothing new [see below :-)] and the WCG team seems to on the Red Queen's race track[1] so we don't get frequent feedback...

In the many years I have been doing WCG I have never seen such a long backlog to validate a completed workunit.

I presume that means you've not been a frequent runner of ARP1 tasks over the last 9 months or so :-) -- there have been frequent occasions where quantities of tasks have ended up in "PVal jail", and on at least one occasion validated tasks weren't being fully assimilated (the latter probably indicating that the file deleter wasn't active at the time...)

I've been using the WCG APIs to track what happens to all the tasks for each ARP1 workunit I process since the "migration" so I've got lots of data (some of it not in particularly "user friendly" formats!) and have been watching tasks that take far too long to get a quorum and noting [most of the] tasks stuck in PVal jail[2]... I may not know why things are happening, but I sure as heck can see them happening :-)

Looking as far back as the end of August 2022, quite a few WUs were stuck in PVal jail for well over a week... Here's one of my notes at the time:

ARP1_0032608_114 is an "Extreme" unit, and although all three results were returned within 24 hours (well inside the 36-hour deadline) they ended up in PVal jail and were still in that state 11+ days later...

This was a persistent theme through September as well; it seemed to have calmed down by early October...

Since then there have been long periods where no ARP1 work has been available for one reason or another (some explained, some unexplained...), and I didn't see any more PVal issues severe enough to bother noting them until after the March/Aprll 2023 outage...

But now, I've started recording PVal issues again, and it seems to be at least as bad as any previous such event. Some WUs do get validated, but in my case the occasional retries I get to run are actually increasing the PVal count despite those escapees...

Cheers - Al.

[1] In Lewis Carrol’s Through the Looking Glass, the Red Queen tells Alice, “Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!” -- I prefer the shortened version, "running like **** just to stand still...", and I had times like that before I retired :-)

[2] Unfortunately, working out exactly when a work-unit validated is made complicated by the nature of those APIs -- one only lets me track my own results, but if I get lucky I can deduce the validation time; the other is the API underpinning the displays on the new-format web site so it doesn't return as much information about each result.