Regarding the continued build-up of Validated tasks that won't go away...

As promised earlier, I have been digging in to the data I collected from WCG to get statistics on credit and wingman performance. What I have found so far is somewhat confusing, so I need to dig some more. However, one thing is clear-cut...

The long-term "validated but not purged" tasks seem to fall into two sub-groups; one lot don't seem to have gone through the file-deletion phase of assimilation (so can't be purged), the other lot seem to have gone through file deletion but haven't been purged! Of the 1000+ results I currently have stacked up for assimilation, all the ones that seem to have deleted files are fairly recent; all the ones that are weeks overdue don't have the "files deleted" flag set...

Now, it could be that the API is misinforming me about the status of the results, but I think it unlikely; I need to dig some more because I think I spotted some real oddities (but that might just have been down to tiredness.)

I'll try to get back to it after some sleep :-)

Cheers - Al.

You are assuming they don't just think "Back seat drivers..." :-)

There's some possible confusion if one looks at BOINC documentation , because they use Project to mean the entire site and Application to mean what we typically call projects here! In the below I'll call them sub-projects :-)

A lot of the support server-side applications such as work-unit generators and validators are "per sub-project" (or possibly, even per application version within sub-project...)

Some other processes such as the transitioner, file-deleter and database purge are single applications, and there are the user-facing aspects such as the scheduler (which runs as a CGI app [using Apache server?])

A lot of the processes can run multiple copies (by having each one run on a different subset of work unit numbers); if there's enough database power and available processors, this can help with throughput. However, unless there have been some changes to the database purge mechanism, running multiples of that disables the ability to produce an archive (which might, amongst other things, impede the abiilty to recover lost statistics...)

Hope that's of interest - Al.

[Edit: added file deleter to the list...]

I have only 1871 PVs. (4 32 threaded machines, one 12 threads). Seems like a reasonable number considering about 2500 WUs processed per day.

There are over 30,000 Valid's. The number seems large. Last two in the results page, completed with ERROR. That maybe hanging up the cleanup to the results page of valids.

Since multiple users confirm now new MCM work getting out either, I fear there's more getting stuck. The dreaded Canadian Friday Meltdown again, I think.

Can you pls check MCM validators ?
Some WU passed validation in seconds. Other waits for hours.

Example where is delay almost one day.
https://www.worldcommunitygrid.org/contribution/workunit/488567108

Result name Status Time due/ Return time
MCM1_0214223_4397_0 Pending Validation 2024-03-01 23:18:08 UTC
MCM1_0214223_4397_1 Pending Validation 2024-03-03 13:23:38 UTC