World Community Grid - View Thread - Statistics Update, Exports & Scheduled Maintenance Process

There are several types of regular, scheduled activity that will stop access or sign-in to the websites Global Statistics, My Grid, Forum or interrupt the retrieval of new work / send back results and interrupt validation. Those interruptions show messages like the next samples:

The scheduled events are:
The 2 times a day Global/Project & Individual Statistics:
How Late How Long (estim)
00:06 UTC 90-120+ minutes, Ends approx. 01:30-02:00 UTC-. (Incl. Daily Team & Challenge Statistics updates)
12:06 UTC 45-60+ minutes. Only member and global statistics are updated with continued access to the My Contributions web section.
Sometime during the 00:06 UTC starting statistics update a message will appear on the My Statistics and Global Statistics pages in example:
Message

The page you requested is currently unavailable because the statistics are being updated.

The statistics update will finish in about 59 minutes 20 seconds.

Further notes on time variations, most notably that the actual Statistics and My Grid access blocking can start later. knreed wrote on this January 21, 2008:
A couple of quick notes:

1) The 'start' of the stats update process is always 6 minutes after 0:00, 12:00. However, the site is not marked 'unavailable' until it has finished extracting the data from the BOINC grid database. In particular, extract the last result time from the BOINC database takes awhile to extract. This causes a variable delay in when the site is marked unavailable.

2) The time that the website is unavailable due to running stats is a little over 45 minutes for the mid-day update and about an 1:40 hours for the end of day stats update. Time should improve once the system has been migrated late 2008 to a new location, hardware and database version

Respective 1/2) above,, the unavailable time period only applies to the 00:06 UTC stats. My Contributions remains open during the 12:06 UTC statistics update.

The BOINC validators (see point 4) are allowed pre-update to complete all checks on the results
received with full quorum prior to 00:00, ~~06:00~~, 12:00 and ~~18:00~~. Any work who's minimum
quorum completed and was validated between 00:00 UTC cut off and the actual stats compilation run at around
00:06 UTC, is included in next days statistics. It shows in the global statistics as a
spill-over value e.g.
06/17/2007 0:000:00:39:40 60 1 < Individual Stats spill-over
06/16/2007 0:004:11:32:37 6,445 90
or
06/17/2007 0:285:09:34:15 266,748 1,026 < Global Stats spill-over
06/16/2007 164:224:19:45:03 59,118,835 228,380

Several times a day, the Validators for BOINC results are paused
How Late How Long - A Quote from a post by knreed on Nov.11, 2008
There are a set of scripts that run 4 times a day. These scripts load new work into BOINC to send, archive results into their proper directories, update status for workunits that errored, notify us of these errors, etc.

There is one step that is very db intensive that takes about 20-30 minutes to run. There is another step that is moderately db intensive that takes about 40 minutes to run.

When the 'very intensive' step runs, we stop the backend daemons (transitioner, validators, asssimilators) to minimize load on the db. If we let these keep running, we saw transaction timeouts and clients being able to request work. However, about a month ago (due to the increasing load) we needed to also stop the daemons while the moderately db intensive step ran. This means that 4 times a day, we stop those backend processes for about 1-1.5 hours. It then takes another 1-1.5 hours for the transitioner to catch up.

This is being done to keep feeding work to the clients while these back-end processes run.

Bi-Daily Backups
A back up of all the databases is performed on Tuesday, Thursday, Saturday, Sunday at around 07:00 UTC. This may intermittently impact users ability to access different parts of the website.

Sign-In can be problematic during these operations. The workaround is to go to the Forum, hit the
Index button, then Sign-In. This can take
a bit of time, or even cause time out. Try again later. During activity point 2., portions of the Fora may not be accessible at all.

These times were taken from observation during May.08
See Primary Source of schedule: http://www.worldcommunitygrid.org/forums/wcg/viewthread?thread=1534
It is recommended to set the website device profile or local preference "Additional Work Buffer" to 0.3 days (7.2 hours). This ensures that enough work is stored on the client to bridge any outages during scheduled and non-scheduled maintenance and most all other outages of the servers/feeders/schedulers/validators/transitioner. For client versions prior to 5.10.x the setting in the My Grid > Device Manager > Device profile to update is "Connect to network about every 0.3 days".

Historic Information:

The 06:06 and 18:06 UTC statistics updates were dropped from the schedule on permanent basis to increase the accessibility of the Global + Projects Statistics and My Grid section for personal settings maintenance & statistics consultation. At the same time the remaining 12:06 & 00:06 UTC updates have several times been optimized by adding more indices to minimize interruption of service.