| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: BOINC Agent Support
Thread: World Community Grid Moves to IBM Cloud |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 182
|
|
| Author |
|
|
asdavid
Veteran Cruncher FRANCE Joined: Nov 18, 2004 Post Count: 521 Status: Offline Project Badges: 
|
Thanks for the details, Kevin
----------------------------------------
Anne-Sophie
 |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
So this is a very exciting set of changes for the team, but it won't have an immediate impact on the volunteers. However, we will be in a position to handle much higher participation and we will be able to do more with what we have. These are very good things for the long term health of the program. Sorry Kevin, but I'm hoping that the above statement is incorrect. I'm hoping it will have an immediate impact on the volunteers by eliminating the messages indicating resource shortages, fixing the ever present and ongoing upload/download issues documented in many forums, and maybe even quicker stats runs so that we don't get messages about the stats being run and they will end in 30 to 50 minutes and in the meantime stats aren't available just to name a few of the ongoing issues. If there is no change to these issues after the move then your statement about handling higher participation may be categorized more as wishful thinking than fact based prognostication. |
||
|
|
TonyEllis
Senior Cruncher Australia Joined: Jul 9, 2008 Post Count: 286 Status: Offline Project Badges: 
|
As I understand it the forums will not be available during the migration commencing May 15. Will the WCG team be supplying reports on how well the migration is progressing? If so, by what means?
----------------------------------------
Run Time Stats https://grassmere-productions.no-ip.biz/
|
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
IIRC, last time there were big works there was a redirect page which kept members/visitors abreast.
|
||
|
|
SNURK
Veteran Cruncher The Netherlands Joined: Nov 26, 2007 Post Count: 1217 Status: Offline Project Badges: 
|
Kevin, thumbs up for this interesting reading material. Good luck with the big migration!
----------------------------------------Thanks, SNURK |
||
|
|
Richard Mitnick
Veteran Cruncher USA Joined: Feb 28, 2007 Post Count: 583 Status: Offline Project Badges: 
|
Kevin-
----------------------------------------Thanks for your well crafted post. |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges: 
|
So this is a very exciting set of changes for the team, but it won't have an immediate impact on the volunteers. However, we will be in a position to handle much higher participation and we will be able to do more with what we have. These are very good things for the long term health of the program. Sorry Kevin, but I'm hoping that the above statement is incorrect. I'm hoping it will have an immediate impact on the volunteers by eliminating the messages indicating resource shortages, fixing the ever present and ongoing upload/download issues documented in many forums, and maybe even quicker stats runs so that we don't get messages about the stats being run and they will end in 30 to 50 minutes and in the meantime stats aren't available just to name a few of the ongoing issues. If there is no change to these issues after the move then your statement about handling higher participation may be categorized more as wishful thinking than fact based prognostication. The stats are not running significantly faster on the new hardware than on the old hardware. One of the reasons is that right now the stats is a serial process and single core performance hasn't improved dramatically so the time to complete the stats isn't decreasing significantly. However, the newer version of the database allows us to use new import/export tools which will allow us to rewrite the stats scripts so that we don't get the deadlocks we were getting before (which is why the site gets shut out). Additionally, since the DB servers will have far more cores, we will be able to break the stats process into parallel execution for some parts and shorten the time it takes for them to run. Issues triggered by load on the servers should drop dramatically so I was perhaps a little careless in not acknowledging that. For example, the scheduler and website are both served from an instance of Apache HTTP running on a server with 16GB of RAM and running on a 4 core Intel Xeon X5450. The scheduler will get its own server with 64GB of RAM running on server with two 6-core Intel Xeon E5-2620 v3. This should address many of the issues where a request is made and the scheduler responds with a message stating that the load is too high. However, there are better improvements coming in the future. Post migration, we will still be running the scheduler on a single server (with the ability to failover to another server). One of our tasks later this year is updating our version of the BOINC client and BOINC server. Once we have done that we are looking to make some modifications to the code. One of the ones I most want to do is to make a change the will allow the scheduler to run on multiple servers effectively. While there are some ways to do this now they have problems and don't work well. Fixing this so that we can run the scheduler across multiple servers effectively will let us take much better advantage of horizontal scaling rather than having to do vertical scaling like we are doing at first in this migration. As far as upload/download issues - those will be spread across more servers and should experience a reduction in issues caused by contention as well (this is already able to horizontally scaled and is therefore more cloud friendly) You are correct in that these issues should see an improvement post migration. [Edit 1 times, last edit by knreed at May 5, 2017 12:57:10 PM] |
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
As I understand it the forums will not be available during the migration commencing May 15. Will the WCG team be supplying reports on how well the migration is progressing? If so, by what means? Facebook and Twitter will be where any update messages are posted while the forums are offline. Due to the relative size of database backups and filesystems, the website and forums will be the first things to come online at the new hosting location and once they are back online we will be able to use the forums to discuss the progress. We are finalizing our cutover plan today and we will see if we can produce a meaningful list of milestones that we can share that will give you an indication of the progress we are making. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Thanks you for confirming, through your elaboration, the performance concerns were on topic :O)
|
||
|
|
knreed
Former World Community Grid Tech Joined: Nov 8, 2004 Post Count: 4504 Status: Offline Project Badges:
|
From how I read it. it's just a hardware change. I think currently WCS had there own hardware and are now are moving to "cloud" Right. Who takes care of the hardware now? After moving operations to the cloud there wouldn't be hardware to take care of. Cheers  We have always had layers of teams involved in working on the infrastructure. At both the old hosting environment and the new hosting environment the way to understand this is to think about four layers of the application:
The immediate WCG team has always been focused on the application and some parts of the middleware and some aspects of the operating system and we have utilized support teams within IBM to support the middleware, operating system and infrastructure that we don't manage. This will continue to be true in the new environment as well. IBM has been doing a lot of reorganization over the past few years and in particular making changes that allow teams much more flexibility and ability to move quicker and with more agility. You can see a talk on this here: https://www.youtube.com/watch?v=3zduSKajK64 and an article about it here: https://medium.com/@BillHiggins/tools-as-a-ca...lture-change-f012b2c0b527 (the material is a year or two old but in a company as big as IBM it takes awhile to roll out significant changes like this at scale as well as for teams to be able to take advantage of the changes). Part of this has been a new model of engagement for teams that need infrastructure to run their applications. Our migration is taking advantage of the new structures that exist internally and I have been very pleased with the way that it has been working so far. |
||
|
|
|