| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Active Research
Forum: OpenPandemics - COVID-19 Project
Thread: Invalid GPU work units |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 171
|
|
| Author |
|
|
zdnko
Senior Cruncher Joined: Dec 1, 2005 Post Count: 225 Status: Offline Project Badges: 
|
Someone can explain me how works the Too Late status? [...] After a few seconds the wu was sent 2 more times. What Causes Too Late? If my Return Time is Too Late why was the wu sent again? That usually happens when the server has sent the "server abort" command to a task that has already started. The command from the server was "Too Late" to abort the task. Ok, but why the wu was resent 2 more times after a server abort? |
||
|
|
PMH_UK
Veteran Cruncher UK Joined: Apr 26, 2007 Post Count: 769 Status: Offline Project Badges: 
|
Too late is set when a unit is returned but max Invalids already returned.
----------------------------------------Also when others validated OK then unit returned so too late to validate. Paul.
Paul.
|
||
|
|
PMH_UK
Veteran Cruncher UK Joined: Apr 26, 2007 Post Count: 769 Status: Offline Project Badges:
|
Valid is per project so need to return valids before send rrate increases.
----------------------------------------Paul.
Paul.
|
||
|
|
zdnko
Senior Cruncher Joined: Dec 1, 2005 Post Count: 225 Status: Offline Project Badges:
|
Too late is set when a unit is returned but max Invalids already returned. Also when others validated OK then unit returned so too late to validate. That doesn't answer my question. Before my Too Late: _0 User Aborted _1 Invalid No one validated OK If "max Invalids already returned" why the wu was send again? After my Too Late why the wu was send again 2 more times? |
||
|
|
sam6861
Advanced Cruncher Joined: Mar 31, 2020 Post Count: 107 Status: Offline Project Badges: 
|
If "max Invalids already returned" why the wu was send again? A Question I don't know. Other Projects like ARP1 resend only 1 more time when invalid happens. Unsure why OPNG resend more then 1 extra tasks on invalid, then server abort an extra from "too late" or too many invalids. In the end, like to see a fix to very frequent invalid, server abort, and "too late" from too many other invalids. |
||
|
|
adriverhoef
Master Cruncher The Netherlands Joined: Apr 3, 2009 Post Count: 2155 Status: Offline Project Badges: 
|
Someone can explain me how works the Too Late status? (reformatted for easier reading)https://www.worldcommunitygrid.org/ms/device/...s.do?workunitId=620639614 Result Name AVN Status Sent Time Due / Return Time CPUh Claimed/Granted0002331_ 00101_ 2 Sent Time: 15:41:12 Return Time: 15:48:03 Less of 7 minutes! After a few seconds the wu was sent 2 more times. What Causes Too Late? If my Return Time is Too Late why was the wu sent again? WU _0 was user aborted and was immediately sent out in two copies as _1 and _2 (15:41:06 and 15:41:12). WU _1 ended up as Invalid when returned at 15:43:12. When WU _2 was returned, it didn't turn out as Valid yet, so two more copies were sent out as _3 and _4 (at 15:48:11). Both _3 and _4 ended up Invalid. That's where the generation of new WU copies ends, because it stops at _4. So there was no possibility to check the validity of _2 against another WU, it was Too Late to validate. |
||
|
|
zdnko
Senior Cruncher Joined: Dec 1, 2005 Post Count: 225 Status: Offline Project Badges:
|
That's where the generation of new WU copies ends, because it stops at _4. So there was no possibility to check the validity of _2 against another WU, it was Too Late to validate. So it was Too Late to validate, not returned Too Late. Ok, thanks for explanation [Edit 1 times, last edit by zdnko at Apr 16, 2021 10:13:10 PM] |
||
|
|
goben_2003
Advanced Cruncher Joined: Jun 16, 2006 Post Count: 145 Status: Offline Project Badges:
|
But it prompted a couple of thoughts. (1) Every single one of these tasks that I spot-checked was paired with other Linux machines. I spot-checked a Windows machine, and every single wingmate there was - another Windows machine. We know (Uplinger has told us) that a task requiring confirmation is flagged to need the same class of GPU: it appears that applies to OS, as well. Coming from the SETI stable, where we struggled mightily to ensure that every version - CPU or GP; NV, AMD or intel; Windows, Linux or Mac; stock, optimised, or third-party - produced compatible and validateable results, these isolated 'bubbles' of validation feel very strange. I do hope that cross-bubble verification is being performed elsewhere in the system. It seems strange to me that 2 results for the same work unit can be different but both be valid. That makes it seem like results are not being verified even within the same bubble. Perhaps I am missing something with how validation works though. I know Uplinger said that credit is granted based upon % of expected calculations that were actually completed. So I would expect wingmates to get the same amount of credit since I would expect they should be running the same calculations. However, this is not the case, they (almost?) always do a different amount of calculations and thus get a different amount of credit. Note: I am not complaining about credit(points), I do not care about credit(points). I do care about science being valid and verifiable.  |
||
|
|
ca05065
Senior Cruncher Joined: Dec 4, 2007 Post Count: 325 Status: Offline Project Badges: 
|
My understanding is that docking programs are trying to filter 30 million possible chemicals down to a few hundred or thousand which can interact with the target ligand. WCG is rare in validating work units against a wingman. Processing using different CPU chips, operating systems, compilers (and now GPU chips) can produce a different numerical result when compared on a bit to bit basis, however providing the results are within a certain tolerance (say less than a fraction of a percent) the filtering can be achieved.
The next stage of more precise and time consuming docking programs can then proceed. |
||
|
|
goben_2003
Advanced Cruncher Joined: Jun 16, 2006 Post Count: 145 Status: Offline Project Badges:
|
I might understand that more if we were talking about differing precision causing slight differences with rounding on different GPUs.
----------------------------------------However, I had one that was the same OS and GPU, but did a significantly different % of calculations. I should have saved the details because it has fallen off the work unit history, so I get an error when I try the viewWorkunitStatus for it. |
||
|
|
|