| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Active Research
Forum: Smash Childhood Cancer
Thread: validation vs verification |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 13
|
|
| Author |
|
|
dondee
Advanced Cruncher Joined: Jan 16, 2006 Post Count: 100 Status: Offline Project Badges: 
|
I am sure this is not specific only to scc, but I can't the answer anywhere I have been looking.
What is the difference between pending "validation" and pending 'verification"? Thanks for the answer. dondee |
||
|
|
Speedy51
Veteran Cruncher New Zealand Joined: Nov 4, 2005 Post Count: 1265 Status: Recently Active Project Badges: 
|
Pending "validation" means the same job has been sent to another computer pending "verification" means it is checking to see if your machine is "reliable". Generally once you have returned "10 tasks" your computer is classed as "reliable" then once every now and then your computer will get send a few tasks to check it is reliable
---------------------------------------- |
||
|
|
dondee
Advanced Cruncher Joined: Jan 16, 2006 Post Count: 100 Status: Offline Project Badges:
|
Thanks for the info.
----------------------------------------So, validation means we are done and verification means a second time around"? Is the xxxxx_1 a way the verification is marked? Appreciate your answering my question Speedy51. [Edit 1 times, last edit by dondee at Aug 26, 2023 11:38:13 PM] |
||
|
|
gb009761
Master Cruncher Scotland Joined: Apr 6, 2005 Post Count: 2977 Status: Offline Project Badges: 
|
Basically yes dondee - the _n indicates the number of versions that have been sent out (starting with _0).
----------------------------------------I can't just remember, but I think the highest this would ever go, is either _6 or _7, depending on whether an earlier result (or more) have been either returned late/not at all/aborted or erroneously etc.  [Edit 1 times, last edit by gb009761 at Aug 27, 2023 1:36:14 AM] |
||
|
|
Speedy51
Veteran Cruncher New Zealand Joined: Nov 4, 2005 Post Count: 1265 Status: Recently Active Project Badges:
|
I would say it like this "verification" is another result been sent out to make sure that your computer is reliable where "validation" is you getting sent a 2nd result to check that the computer that returned the 1st result is reliable. In both instances if the result is returned "valid" by both computers the result is a success.
----------------------------------------More often than not you will only receive results with "_0" on the end of them because this project doesn't require a wingperson for the majority of results [Edit 2 times, last edit by Speedy51 at Aug 27, 2023 3:15:11 AM] |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 1932 Status: Offline Project Badges: 
|
I get now dozens of "Pending for Verification" WUs on hosts than have returned tens of thousands of WUs, some probably more than a couple of hundred thousands. So this is not a "10 and done" kind of thing, and it started to increase for certainly reliable hosts a couple of weeks ago or so...
----------------------------------------Ralf  |
||
|
|
Bryn Mawr
Senior Cruncher Joined: Dec 26, 2018 Post Count: 337 Status: Offline Project Badges:
|
Another take on the question - the system sends out two copies of a task, the first copy back is pending validation until the second copy comes back. If the second copy gives the same result as the first then they are both valid. If, on the other hand, the second copy gives a different but equally possible result then the two copies are set to pending verification and a third copy is sent out .
----------------------------------------When the third copy comes back, if it matches one of the first two then they are set to valid and the third to invalid, if we now have three different results they are raised on an exception result for the researcher to deal with. This is my understanding of the process - that doesn’t mean that it’s right and I do not guarantee I’m correct. [Edit 1 times, last edit by Bryn Mawr at Aug 27, 2023 8:44:14 AM] |
||
|
|
alanb1951
Veteran Cruncher Joined: Jan 20, 2006 Post Count: 873 Status: Offline Project Badges: 
|
There's a lot of good information above :-) -- so just a little filling in...
At present SCC1 and the quiescent OPN1/OPNG both use Adaptive Replication, where a "reliable host" usually gets a chance to validate without needing a wingman. (It doesn't necessarily mean one needs to have task_0, by the way - there are situations where retries qualify for AR...) A WU is marked as not acceptable if it either hits a pre-set number of errors or a pre-set number of total tasks without validation. A typical BOINC set-up has 3 errors or 6 tasks as the limit, but I think WCG may have been using slightly more open-ended settings for some of its projects. How WU rejection works out might depend on whether certain types of failure (No Reply, Not Started by Deadline, download error?) always count against the notional total tasks. Whatever the case, it should be possible to have quite a few Pending Verification as long as there aren't any totally invalid tasks. Some of us will recall watching ARP1 WUs heading up to (and sometimes beyond) 6 total tasks when there was that group of cells that needed to be shepherded past a difficult data situation...) Cheers - Al. |
||
|
|
TPCBF
Master Cruncher USA Joined: Jan 2, 2011 Post Count: 1932 Status: Offline Project Badges:
|
How WU rejection works out might depend on whether certain types of failure (No Reply, Not Started by Deadline, download error?) always count against the notional total tasks. Whatever the case, it should be possible to have quite a few Pending Verification as long as there aren't any totally invalid tasks.. Sorry Al, but that is not what I am observing recently (it is just extremely hard to communicate about this thanks to WCG Towers). In years past, even when WCG came back up in Toronto, there was always a ratio of no more than a dozen PVe WUs out of roughly 2500 WUs total in the results (ignoring here the higher total number of WUs on some "shorty" projects, though this did never significantly increase the number of WUs in PVe jail). So for that, your assumption could be kind of valid.But as I tried to communicate for the last two weeks or so, possibly since the last system dive 33 days ago, the ratio of PVe WUs has significantly increased. Right now, with a lot of the shorter SCC1 WUs of recent batches, I have currently 218 WUs in PVe jail (plus +450 in PVa) out of about 4200 WUs total on the Results page. That is roughly a 10x increase, from about 0.5% to about 5%. And none of that is certainly due to any of my hosts fault. And those PVe WUs are were I see most often that "Waiting to be sent" status for more than a week... Ralf [Edit 1 times, last edit by TPCBF at Aug 28, 2023 3:02:38 PM] |
||
|
|
dondee
Advanced Cruncher Joined: Jan 16, 2006 Post Count: 100 Status: Offline Project Badges:
|
Thanks all for your replies, I get the jest of the marking system now.
dondee |
||
|
|
|