| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Website Support
Thread: SLOOOW updates? |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 15
|
|
| Author |
|
|
David_L6
Senior Cruncher USA Joined: Aug 24, 2006 Post Count: 296 Status: Offline Project Badges: 
|
Why are the updates taking so long again? (They used to take almost FOREVER, then some changes were made and they were quicker. Now they are SLOW again.)
---------------------------------------- |
|||
|
||||
|
David_L6
Senior Cruncher USA Joined: Aug 24, 2006 Post Count: 296 Status: Offline Project Badges:
|
Still wondering why the updates are so slow again.......
---------------------------------------- |
|||
|
||||
|
JmBoullier
Former Community Advisor Normandy - France Joined: Jan 26, 2007 Post Count: 3716 Status: Offline Project Badges: 
|
What have you really experienced?
----------------------------------------Is it a later availability of the new stats and/or of those transactions affected by the stats run? Or is it really longer periods of unavailability of these transactions? Personally I have not been really affected by one or the other, simply because I have got used to doing something else at those times. But sometimes recently I have forgotten which time it was and I asked for something which should not have come back at this time (e.g. a profile update or a Results Status) and it did come back... because the stats update had not yet started! The latest start I have experienced was half an hour on a midday run. At 12:30 UTC the stats had not started yet! Hence my question: are the stats run really slower of are they only starting later?  |
||
|
|
nasher
Veteran Cruncher USA Joined: Dec 2, 2005 Post Count: 1423 Status: Offline Project Badges:
|
honestly i normaly just know when the status updates are happening and just dont try to connect to the server that hour.
----------------------------------------prior to the updates they were taking 1 hour +... imedatly folloing the update they took i think 20-30 min... and yes now i have seen when i forget the time times upwards of 40 min... also notice dits normaly the end of day ones that take longer than the midday run... not a big problem in my opinion but something i have minorly been wondering about also. it could also be slower do to server loading.. (like a DOS attack or a spam attack)  |
||
|
|
David_L6
Senior Cruncher USA Joined: Aug 24, 2006 Post Count: 296 Status: Offline Project Badges:
|
Yes, the daily updates are definitely taking longer again. I have also noticed that they don't always start when they a "supposed" to. That's not it though. Using my local times for this example. The updates usually start around 19:15. Usually it's 20:30, and sometimes it's as late as 21:00, before I can access "My Grid" again. That's about the same amount of time it was before the updates. Right after the updates were made it took maybe 30 minutes.
----------------------------------------I'm just trying to understand WHY? What happened to make the updates slow down again? |
|||
|
||||
|
JmBoullier
Former Community Advisor Normandy - France Joined: Jan 26, 2007 Post Count: 3716 Status: Offline Project Badges:
|
For the record, at the time of this post (almost 01:00 UTC) the stats run has not yet started....
-------------------------------------------------------------------------------- [Edit 1 times, last edit by JmBoullier at Sep 20, 2009 1:00:02 AM] |
||
|
|
nasher
Veteran Cruncher USA Joined: Dec 2, 2005 Post Count: 1423 Status: Offline Project Badges:
|
well they are running now and taking 55+ min acording to them (dont know exactly when they started but i tried today to hit the update during the update and it said 55 min so... yes they are slow again
---------------------------------------- |
||
|
|
Johnny Cool
Ace Cruncher USA Joined: Jul 28, 2005 Post Count: 8621 Status: Offline Project Badges: 
|
'tis true, the updates are taking longer no matter what anyone says.
----------------------------------------Yet, I don't mind if it allows IBM to make this all stable (so many Teams now). One only has to be reminded of what used to happen at the old UD.org. For those of you that crunched there (like we did) when it was still in existence, you know.  Remember, the WCG is a non-profit organization, and it does not have the money or resources to purchase expensive SCM (Configuration Management) software or spend money on uber servers. This is something that I do know about. SCM (basically software that serves the large Enterprise) is my area of specialization as an IT Recruiter. Just check out the IBM portfolio at NASDAQ. Oh yes, and EMC as well.  EMC does rock as good as IBM. Both are World Class and will continue to be.  ---------------------------------------- [Edit 1 times, last edit by Johnny Cool at Sep 20, 2009 3:08:31 AM] |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
The end of day stats run is taking between 1 hour 10 minutes to run and 1 hour 25 minutes (step 4 below). The variation is due to load on the db while they are running.
The stats work as follows: 1) Export data from the BOINC database 2) Run some pre-processing steps to prepare to load the data 3) Make the website unavailable 4) Load the data 5) Make the website available 6) Run some scripts to populate caches in the web application The duration of steps 1 and 2 effect the time that members perceive the when the stats 'start'. There appears to be an issue that can cause one of the exports in step #1 above to jump from taking 6-7 minutes to taking 40-50 minutes. We are taking a look at what is causing this periodic increase in time. I expect that we will find one of our other processes that puts a heavy load on the database can sometimes overlap and cause this delay. |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
It appears that the conflict was caused by the process that exports the stats for the external stats sites. We are moving that a little later to avoid the conflict.
|
||
|
|
|