Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 4
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 4939 times and has 3 replies Next Thread
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Unexpected Outage - started at 0:30 UTC on Jan 29 [Resolved]

We experienced a serious unexpected outage last night starting at 0:30 UTC. We are starting to bring some elements of the system back up and have just now been able to get the website and forums back online.

You will currently not be able to upload or download work.
----------------------------------------
[Edit 1 times, last edit by knreed at Jan 30, 2013 12:15:18 AM]
[Jan 29, 2013 1:36:40 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unexpected Outage - started at 0:30 UTC on Jan 29 and is continuing

You should be able to upload and download work now.

Some of our backend storage is currently being checked for errors. This process will continue for another few hours.

We have a certain amount of space available on the storage that is available to start letting work be uploaded and sending work out. We are going to use that to make the system functional again. However, we will be watching our disk usage carefully. If we use up our available space too quickly or if the process of checking the other storage takes longer than expected, then we might have to stop the system again. We will keep you updated.

Thank you for your patience.
----------------------------------------
[Edit 1 times, last edit by knreed at Jan 29, 2013 7:57:01 PM]
[Jan 29, 2013 7:51:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unexpected Outage - started at 0:30 UTC on Jan 29 and is continuing

In case anyone was wondering what the surge looks like after we start up after an outage:

We have received in the past 30 minutes:
209,132 jobs which is 116 results/second and ~240 file uploads per second from 14,054 different computers

We have sent in the past 30 minutes:
131,409 results which is 73 results/second to 16,037 different computers
[Jan 29, 2013 8:18:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Unexpected Outage - started at 0:30 UTC on Jan 29 and is continuing

We have completed our recovery operations and the system is now operating normally. We apologize for the disruptions and appreciate your patience.
[Jan 30, 2013 12:16:19 AM]   Link   Report threatening or abusive post: please login first  Go to top 
[ Jump to Last Post ]
Post new Thread