Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 53
Posts: 53   Pages: 6   [ Previous Page | 1 2 3 4 5 6 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 12321 times and has 52 replies Next Thread
uplinger
Former World Community Grid Tech
Joined: May 23, 2005
Post Count: 3952
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

Thanks for letting us know, are you able to elaborate on the server/storage upgrades e.g. are you changing to SSD's (solid state drives) & the size old drives of verse new drives & what server hardware is changing? Thanks for working on the weekend. All the best with the upgrade


Sure, We did some work about 2 months ago to migrate the data to larger SSD drives that were about 3 years newer. This left us with the old drives still on the devices which we were paying for. The old drives when migrating the data from them to the new drives reported 1.6GB of data being still on those drives and we spent the past 2 months trying to figure out where that was coming from. It was caused by an incorrect reporting of values in that migration tool but we spent many days attempting to confirm the data was all migrated. Now since the drives have been removed from the cluster file system, we are physically removing them. But to be extra paranoid on my end, after they are removed, I am rebooting the machine they are rebooted from. This is to confirm that during system updates that happen about once a month, we don't encounter data issues. However to do these steps, takes about 17 hours from start to completion.

We will be performing these actions for the next few days.

Thanks,
-Uplinger
[Oct 26, 2020 1:27:18 PM]   Link   Report threatening or abusive post: please login first  Go to top 
uplinger
Former World Community Grid Tech
Joined: May 23, 2005
Post Count: 3952
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

I'm not sure if this is related to your maintenance, but a number of machines in my fleet are hung up on downloading new work units.

The BOINC client log reports a "transient HTTP error", and when I update the project, I receive the log message: "Not requesting tasks: some download is stalled".

I can clear the stalled downloads by restarting the BOINC client service, but as I'm sure you can imagine, having clients stall and run out of work until someone manually restarts it is not ideal.


With the updates, we put extra load on the systems which seems to have caused some load issues but cleared on their own. I will be monitoring this as we continue the changes. Thanks for reporting! Also, thanks for your patience.

-Uplinger
[Oct 26, 2020 1:28:28 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7846
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

Looks like the validators are down sometime between 13:07 and 13:34 UTC across the board.
Edit: Looks like somebody gave them a kick. I just had several pages of pending validation units change to valid.
Thanks
Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
----------------------------------------
[Edit 1 times, last edit by Sgt.Joe at Oct 26, 2020 2:21:02 PM]
[Oct 26, 2020 2:13:23 PM]   Link   Report threatening or abusive post: please login first  Go to top 
uplinger
Former World Community Grid Tech
Joined: May 23, 2005
Post Count: 3952
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

Yeah, the backend processes are something I shut down to help migrate them off the host that is being manipulated. However, we are still in the middle of the changes and uploads/downloads may be disrupted due to this.

Thanks,
-Uplinger
[Oct 26, 2020 2:43:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
BladeD
Ace Cruncher
USA
Joined: Nov 17, 2004
Post Count: 28976
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

So, is this why the stats are late tonight?
----------------------------------------
[Oct 27, 2020 1:07:12 AM]   Link   Report threatening or abusive post: please login first  Go to top 
uplinger
Former World Community Grid Tech
Joined: May 23, 2005
Post Count: 3952
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

Thanks for the heads up. You are correct, stats are late.

I will look into starting them.

Thanks,
-Uplinger
[Oct 27, 2020 1:13:59 AM]   Link   Report threatening or abusive post: please login first  Go to top 
uplinger
Former World Community Grid Tech
Joined: May 23, 2005
Post Count: 3952
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

Stats have been started.

Thanks again for the heads up.
-Uplinger
[Oct 27, 2020 1:24:52 AM]   Link   Report threatening or abusive post: please login first  Go to top 
BladeD
Ace Cruncher
USA
Joined: Nov 17, 2004
Post Count: 28976
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

You're welcome! Thanks for the quick response!
----------------------------------------
[Oct 27, 2020 1:31:12 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Mike.Gibson
Ace Cruncher
England
Joined: Aug 23, 2007
Post Count: 12594
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

Keith

We seem to have skipped about 1500 batches of opn1. Is this anything to do with the server changes?

Mike
[Oct 27, 2020 4:20:24 PM]   Link   Report threatening or abusive post: please login first  Go to top 
TPCBF
Master Cruncher
USA
Joined: Jan 2, 2011
Post Count: 2173
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Server updates and changes

Stats have been started.

Thanks again for the heads up.
-Uplinger
Looks like the generation of the external stats files needs a kick in the buns as well...

Ralf
[Oct 27, 2020 4:42:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 53   Pages: 6   [ Previous Page | 1 2 3 4 5 6 | Next Page ]
[ Jump to Last Post ]
Post new Thread