Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 13
Posts: 13   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 1865 times and has 12 replies Next Thread
keithhenry
Ace Cruncher
Senile old farts of the world ....uh.....uh..... nevermind
Joined: Nov 18, 2004
Post Count: 18667
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Validation delay???

Has there been a pretty recent change in the validation processing?? I'm noticing that on WU's where I'm the third or fourth result returned, instead of validating within a matter of minutes, it's taking more than an hour. If mine is the third result, this is when all three get marked valid normally. If mine is the fourth, the other three are usually already marked valid.
----------------------------------------
Join/Website/IMODB



[May 4, 2006 10:40:35 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Dirk Gently
Senior Cruncher
England
Joined: Mar 1, 2005
Post Count: 153
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Validation delay???

How do you find out where you are in the validation order?

I was starting to think that all validation was taking longer recently - I currently have about 12 units awaiting vaildation, from 1 dual processor machine and 1 laptop. The bargraphs of my results have a peak every 2-3 days now instead of a fairly constant level. Any one else noticed anything?
----------------------------------------
[May 4, 2006 3:23:13 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Validation delay???

Go to the results status page and click on the work unit name.

Clicking on the status gives the output log.
[May 4, 2006 4:01:08 PM]   Link   Report threatening or abusive post: please login first  Go to top 
keithhenry
Ace Cruncher
Senile old farts of the world ....uh.....uh..... nevermind
Joined: Nov 18, 2004
Post Count: 18667
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Validation delay???

Right, the work unit name on the results status page is a link to that info. I am suspecting that the WCG admins have changed how often the validation software runs on their servers. If so, I'm guessing at this point that it runs every two hours and validates what work it's gotten since the last run. Six of one, half dozen of the other but if you've just finished crunching a WU that's going to put you over some milestone and it's close to the scheduled stats run, you may have to wait a few more hours until the following stats run to see your milestone occur.
----------------------------------------
Join/Website/IMODB



[May 4, 2006 4:38:00 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Validation delay???

Seeing same here, but a phenomena exclusive to BOINC. Even when 3 have been validated, #4 still has to wait a while....several hours observed. Also because of the option to allow comms delays, the quorum of 3 final validation is pure luck. Observed WU's that had a single result returned, then waiting for days before a second or third coming thru.....at the end it all balances, but the daily and 6 hourly stats do get a little distorted...in the eyes beholder. wink

Also consider this....when i joint WCG back in July 05, there were less than 100000 devices....we're growing accilerated now well of 300000. Here the updates are run every 6 hours and back then took 10 minutes or less, but now including the pre-stat routines close to 40/45 minutes.....larger community will surely require adapted processes.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at May 4, 2006 6:15:45 PM]
[May 4, 2006 6:08:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Validation delay???

Has there been a pretty recent change in the validation processing?? I'm noticing that on WU's where I'm the third or fourth result returned, instead of validating within a matter of minutes, it's taking more than an hour. If mine is the third result, this is when all three get marked valid normally. If mine is the fourth, the other three are usually already marked valid.


There are a couple of backend end processes that occur (such as loading additional work onto the grid and online database backups) where we will suspend the validation processes for a short period. If it isn't validated within a few minutes (after the quorum has been reached) then you will know that it is within this period.

The validators are given plenty of time to validate and catch up prior to the stats update so you will not be behind when the stats are run.

Kevin
[May 4, 2006 6:48:24 PM]   Link   Report threatening or abusive post: please login first  Go to top 
keithhenry
Ace Cruncher
Senile old farts of the world ....uh.....uh..... nevermind
Joined: Nov 18, 2004
Post Count: 18667
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Validation delay???

There are a couple of backend end processes that occur (such as loading additional work onto the grid and online database backups) where we will suspend the validation processes for a short period. If it isn't validated within a few minutes (after the quorum has been reached) then you will know that it is within this period.

The validators are given plenty of time to validate and catch up prior to the stats update so you will not be behind when the stats are run.

Kevin


Thanks Kevin! I must have hit a string with just the right timing as I was seeing this on the last few WU's I've returned. Hmm, I might just want to go buy some lottery tickets........

laughing
----------------------------------------
Join/Website/IMODB



[May 4, 2006 7:28:04 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Validation delay???

With respect to KnReed's comments and backend processes causing delay, i've observed more than a few times the commented "several hours delay". To case in point: Send 10:02 UTC 7 may 06 the below completing a quorum of 3.

Workunit Name Status Sent Time Time Due /
Return Time CPU Time (hours) Claimed/ Granted BOINC Credit
faah0452_ d019cb929_ x1hpv_ 00 Pending Validation 05/05/2006 21:12:56 05/06/2006 17:42:08 15.58 67 / 0
faah0452_ d019cb929_ x1hpv_ 00 In Progress 05/05/2006 21:11:27 05/26/2006 21:11:27 0.00 0 / 0
faah0452_ d019cb929_ x1hpv_ 00 Pending Validation 05/05/2006 21:04:06 05/07/2006 10:22:17 10.02 78 / 0
faah0452_ d019cb929_ x1hpv_ 00 Pending Validation 05/05/2006 20:55:20 05/06/2006 11:36:29 13.09 65 / 0

Facts or Fiction or Figments of to quote "The validators are given plenty of time to validate and catch up prior to the stats update so you will not be behind when the stats are run.
" .... this WU did not get validated before or during the 6 hourly stats-update run and as at this post 12:33 pm UTC i.e. at least 2.5 hours delay and longer been seen. During the day its not so 'critical', but at days end when the teamstats are rolled up, the 'competition' gets effected.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 2 times, last edit by Sekerob at May 7, 2006 12:43:17 PM]
[May 7, 2006 12:40:39 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Validation delay???

The validators are stopped at the following times each day:

1:00 to about 1:30 UTC
7:00 to about 11:00 UTC
13:00 to about 13:30 UTC

Stats are run at 0:06, 6:06, 12:06, 18:06 each day.

I took a look at your workunit and noticed that it wasn't actually validated until 13:30 UTC so I went digging into why. What I found is that BOINC has a feature built into it that detects if it is 'behind' (too many results coming in to keep up). This feature assumes that the validators are running continuously. Since we periodically stop the validators, this mechanism kicks in. This results in the validation for each workunit being performed at a time twice the time it was determined to be behind.

What this means here is that if the workunit was ready for validation at 9:45 UTC and the validators were turned back on at 11:00 UTC, then that workunit would not be actually validated until 13:30 UTC. In the extreme case, if a workunit was ready for validation at 7:00 UTC (just after the validators were stopped) and the validators were started again at 11:00 UTC then actual validation would be performed at 19:00 UTC.

This means that you might not have been awarded all of your points in time for the 12 UTC stats update (for results returned between 7:00 UTC and about 11:00 UTC) and in the worse case it would miss the 18 UTC. However, the max delay as a result of this is that by 19:00 UTC the system would be 'caught up' and all end of day processing (including team stats) would include the full day stats.

Ok - having written all of that above and not wishing to have to explain it again (and of course wanting members to receive their points quickly), I will investigate if we can leave the validators on during the 7:00 - 11:00 window. It isn't strickly necessary and was only done to reduce contention.

Kevin
[May 8, 2006 2:43:45 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Validation delay???

Could the Validators be down?
Workunit Name Status Sent Time Time Due / Return Time CPU Time (hours) Claimed/ Granted BOINC Credit
faah0680_ d475cb087_ x1hpv_ 00 Pending Validation 07/04/2006 10:15:31 07/07/2006 18:05:51 8.30 77 / 0
faah0680_ d475cb087_ x1hpv_ 00 Pending Validation 07/04/2006 10:13:45 07/05/2006 02:56:25 9.62 59 / 0
faah0680_ d475cb087_ x1hpv_ 00 Pending Validation 07/04/2006 10:13:35 07/04/2006 18:56:18 5.03 48 / 0

Been sitting since 7.7.06 18:05 UTC, which is outside the times of close in previous post. Could be coincidence but rounded the receipt time was 18:06 UTC right when the 3rd validation of the day started.

thx
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at Jul 7, 2006 8:38:05 PM]
[Jul 7, 2006 8:36:26 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 13   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread