Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 9
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 1511 times and has 8 replies Next Thread
Mamajuanauk
Master Cruncher
United Kingdom
Joined: Dec 15, 2012
Post Count: 1900
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Suggestion – Setup Power Crunchers Group on server

Suggestion – Setup Power Crunchers Group
Purpose – Issue WU’s to Power Cruncher group devices
Aim - Speed flow of WU’s & Reduce storage requirements
Benefits – Reduced requirements for storing results & better flow of WU’s


Logic
It struck me that myself and several/many others known as 'Power Crunchers' have large numbers of WU's in PV prison, awaiting wingmen to validate.

This slows the flow of WU's and increases the storage requirement for results.

To alleviate this, if a group for power crunchers was created on the servers, in the same way as WU’s are allocate by OS to the wingman, if both the initial xxx_0 and the xxx_1 WU’s was sent to a Power Crunchers, they would validate each other much quicker. This would increase the flow of WU’s and reduce storage requirements.

There would have to be some criteria met to 'Qualify' for the Power Crunchers group, may be some or all of these possibly others too:
    Number of results returned in a given period
    Number of active devices
    Number of RT days per day
    Devices must be classed as 'reliable'
How will it work?

The initial group could be setup with known power crunchers, anyone returning more than, say, 500 Gflops or 500 results per day, with anyone dropping below a defined level for more than a couple of days dropping out of the group, returning as and when they reach the requirements again.

While in the group WU’s would be issued to the group members to crunch, anyone falling out of the group would get ‘normal’ WU’s. While anyone falling out of the group may reduce the flow slightly, over all it would still be an increase.

This would work in exactly the same way for anyone else who is currently (at the time of initial group setup) under the entry criteria for the group.

This suggestion may need some work around the detail, however, I believe the principle is sound. Whether or not it's possible is another matter...

Your comments/thoughts/suggestions would be appreciated…
----------------------------------------
Mamajuanauk is the Name! Crunching is the Game!



[Oct 18, 2014 9:38:54 AM]   Link   Report threatening or abusive post: please login first  Go to top 
OldChap
Veteran Cruncher
UK
Joined: Jun 5, 2009
Post Count: 978
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Suggestion – Setup Power Crunchers Group on server

Sounds like a plan. but consideration for cache length might be a factor.

I believe all who were to participate might have to run similar cache sizes for this to work the way you describe
----------------------------------------

[Oct 18, 2014 12:17:18 PM]   Link   Report threatening or abusive post: please login first  Go to top 
OldChap
Veteran Cruncher
UK
Joined: Jun 5, 2009
Post Count: 978
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Suggestion – Setup Power Crunchers Group on server

It's time you merged those accounts btw. :)
----------------------------------------

[Oct 18, 2014 12:18:52 PM]   Link   Report threatening or abusive post: please login first  Go to top 
ca05065
Senior Cruncher
Joined: Dec 4, 2007
Post Count: 325
Status: Recently Active
Project Badges:
Reply to this Post  Reply with Quote 
Re: Suggestion – Setup Power Crunchers Group on server

The only requirement for a 'fast turnaround' group is the time it takes to return units. It does not matter if only a few work units are returned per day or if hundreds are.
[Oct 18, 2014 12:48:46 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mamajuanauk
Master Cruncher
United Kingdom
Joined: Dec 15, 2012
Post Count: 1900
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Suggestion – Setup Power Crunchers Group on server

@OC - Yep, details... and the group members would have to agree on that cache size, which may be fun in it's self...

@OC - Yep, been trying to find a way, asked SNURK, but no reply to date. May be I'll ask WCG guys...

@ca05065 - the fast turnaround would benefit the scientists with quicker results. How much benefit would depend on how soon they look at the results.

However, it has been mentioned many times within the forum about the more than occaisional shortage of storage space, which is why some projects have 7 day deadlines

Thanks to both for your comments...
----------------------------------------
Mamajuanauk is the Name! Crunching is the Game!



[Oct 18, 2014 1:08:53 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Crystal Pellet
Veteran Cruncher
Joined: May 21, 2008
Post Count: 1316
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Suggestion – Setup Power Crunchers Group on server

The only requirement for a 'fast turnaround' group is the time it takes to return units. It does not matter if only a few work units are returned per day or if hundreds are.
This is the best answer applause No need for a power group. It's not about people, but about hosts.
A host of a 'power cruncher' could belong in the fast group and another host from the same cruncher in a 'normal' group.
In the DB for each host is a field "Average turnaround time".
One could make 3 groups:
1 - Fast - Returners with an average turnaround time < 1 day
2 - Normal - Returners with an average turnaround time ≥ 1 day and < 4 days
3 - Slow - All other hosts

This could be fully automated.
A host will switch when it reduces it cache size.
Less Pending Validation - Faster purging of the Database possible.
----------------------------------------

[Oct 18, 2014 1:46:28 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mamajuanauk
Master Cruncher
United Kingdom
Joined: Dec 15, 2012
Post Count: 1900
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Suggestion – Setup Power Crunchers Group on server

@CP - interesting twist... and a very good point, it is the host that would determine it's group. I was only thinking of/on the lines of crunching time against online time. as some of my machines only go online 7pm-7am but run 24/7. Those in slower turnaround groups may not crunch 24/7 or even 12/24, but that is primarily what and why Grid computing came about!

The key is to have those with a faster turnaround in the same group, not so much how it's implemented or what it's called. I see the biggest benefit of this would be the reduced storage requirement.

I think this requires more thought and discussion...

Thanks for your comments. It would be appreciated if one of the admin team at WCG were to add some comment or input...
----------------------------------------
Mamajuanauk is the Name! Crunching is the Game!



[Oct 18, 2014 2:19:35 PM]   Link   Report threatening or abusive post: please login first  Go to top 
SNURK
Veteran Cruncher
The Netherlands
Joined: Nov 26, 2007
Post Count: 1217
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Suggestion – Setup Power Crunchers Group on server

@OC - Yep, been trying to find a way, asked SNURK, but no reply to date. May be I'll ask WCG guys...

I did reply. The answer is in the other thread somewhere. Maybe a bit lost between posts about the new project. smile
----------------------------------------
[Oct 18, 2014 2:46:17 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Mamajuanauk
Master Cruncher
United Kingdom
Joined: Dec 15, 2012
Post Count: 1900
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Suggestion – Setup Power Crunchers Group on server

@OC - Yep, been trying to find a way, asked SNURK, but no reply to date. May be I'll ask WCG guys...

I did reply. The answer is in the other thread somewhere. Maybe a bit lost between posts about the new project. smile
Thanks SNURK and apologies, I'll search it out...
----------------------------------------
Mamajuanauk is the Name! Crunching is the Game!



[Oct 18, 2014 3:49:18 PM]   Link   Report threatening or abusive post: please login first  Go to top 
[ Jump to Last Post ]
Post new Thread