Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
World Community Grid Forums
Category: Support Forum: Suggestions / Feedback Thread: Suggestion – Setup Power Crunchers Group on server |
No member browsing this thread |
Thread Status: Active Total posts in this thread: 9
|
Author |
|
Mamajuanauk
Master Cruncher United Kingdom Joined: Dec 15, 2012 Post Count: 1900 Status: Offline Project Badges: |
Suggestion – Setup Power Crunchers Group
----------------------------------------Purpose – Issue WU’s to Power Cruncher group devices Aim - Speed flow of WU’s & Reduce storage requirements Benefits – Reduced requirements for storing results & better flow of WU’s Logic It struck me that myself and several/many others known as 'Power Crunchers' have large numbers of WU's in PV prison, awaiting wingmen to validate. This slows the flow of WU's and increases the storage requirement for results. To alleviate this, if a group for power crunchers was created on the servers, in the same way as WU’s are allocate by OS to the wingman, if both the initial xxx_0 and the xxx_1 WU’s was sent to a Power Crunchers, they would validate each other much quicker. This would increase the flow of WU’s and reduce storage requirements. There would have to be some criteria met to 'Qualify' for the Power Crunchers group, may be some or all of these possibly others too:
Number of active devices Number of RT days per day Devices must be classed as 'reliable' The initial group could be setup with known power crunchers, anyone returning more than, say, 500 Gflops or 500 results per day, with anyone dropping below a defined level for more than a couple of days dropping out of the group, returning as and when they reach the requirements again. While in the group WU’s would be issued to the group members to crunch, anyone falling out of the group would get ‘normal’ WU’s. While anyone falling out of the group may reduce the flow slightly, over all it would still be an increase. This would work in exactly the same way for anyone else who is currently (at the time of initial group setup) under the entry criteria for the group. This suggestion may need some work around the detail, however, I believe the principle is sound. Whether or not it's possible is another matter... Your comments/thoughts/suggestions would be appreciated…
Mamajuanauk is the Name! Crunching is the Game!
|
||
|
OldChap
Veteran Cruncher UK Joined: Jun 5, 2009 Post Count: 978 Status: Offline Project Badges: |
Sounds like a plan. but consideration for cache length might be a factor.
----------------------------------------I believe all who were to participate might have to run similar cache sizes for this to work the way you describe |
||
|
OldChap
Veteran Cruncher UK Joined: Jun 5, 2009 Post Count: 978 Status: Offline Project Badges: |
It's time you merged those accounts btw. :)
---------------------------------------- |
||
|
ca05065
Senior Cruncher Joined: Dec 4, 2007 Post Count: 325 Status: Recently Active Project Badges: |
The only requirement for a 'fast turnaround' group is the time it takes to return units. It does not matter if only a few work units are returned per day or if hundreds are.
|
||
|
Mamajuanauk
Master Cruncher United Kingdom Joined: Dec 15, 2012 Post Count: 1900 Status: Offline Project Badges: |
@OC - Yep, details... and the group members would have to agree on that cache size, which may be fun in it's self...
----------------------------------------@OC - Yep, been trying to find a way, asked SNURK, but no reply to date. May be I'll ask WCG guys... @ca05065 - the fast turnaround would benefit the scientists with quicker results. How much benefit would depend on how soon they look at the results. However, it has been mentioned many times within the forum about the more than occaisional shortage of storage space, which is why some projects have 7 day deadlines Thanks to both for your comments...
Mamajuanauk is the Name! Crunching is the Game!
|
||
|
Crystal Pellet
Veteran Cruncher Joined: May 21, 2008 Post Count: 1316 Status: Offline Project Badges: |
The only requirement for a 'fast turnaround' group is the time it takes to return units. It does not matter if only a few work units are returned per day or if hundreds are. This is the best answer No need for a power group. It's not about people, but about hosts.A host of a 'power cruncher' could belong in the fast group and another host from the same cruncher in a 'normal' group. In the DB for each host is a field "Average turnaround time". One could make 3 groups: 1 - Fast - Returners with an average turnaround time < 1 day 2 - Normal - Returners with an average turnaround time ≥ 1 day and < 4 days 3 - Slow - All other hosts This could be fully automated. A host will switch when it reduces it cache size. Less Pending Validation - Faster purging of the Database possible. |
||
|
Mamajuanauk
Master Cruncher United Kingdom Joined: Dec 15, 2012 Post Count: 1900 Status: Offline Project Badges: |
@CP - interesting twist... and a very good point, it is the host that would determine it's group. I was only thinking of/on the lines of crunching time against online time. as some of my machines only go online 7pm-7am but run 24/7. Those in slower turnaround groups may not crunch 24/7 or even 12/24, but that is primarily what and why Grid computing came about!
----------------------------------------The key is to have those with a faster turnaround in the same group, not so much how it's implemented or what it's called. I see the biggest benefit of this would be the reduced storage requirement. I think this requires more thought and discussion... Thanks for your comments. It would be appreciated if one of the admin team at WCG were to add some comment or input...
Mamajuanauk is the Name! Crunching is the Game!
|
||
|
SNURK
Veteran Cruncher The Netherlands Joined: Nov 26, 2007 Post Count: 1217 Status: Offline Project Badges: |
@OC - Yep, been trying to find a way, asked SNURK, but no reply to date. May be I'll ask WCG guys... I did reply. The answer is in the other thread somewhere. Maybe a bit lost between posts about the new project. |
||
|
Mamajuanauk
Master Cruncher United Kingdom Joined: Dec 15, 2012 Post Count: 1900 Status: Offline Project Badges: |
@OC - Yep, been trying to find a way, asked SNURK, but no reply to date. May be I'll ask WCG guys... I did reply. The answer is in the other thread somewhere. Maybe a bit lost between posts about the new project.
Mamajuanauk is the Name! Crunching is the Game!
|
||
|
|