| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
| World Community Grid Forums
|
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 62
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Let it Rip!
|
||
|
|
Mysteron347
Senior Cruncher Australia Joined: Apr 28, 2007 Post Count: 179 Status: Offline Project Badges:
|
I believe that a proper analysis should not begin with leaping to an obvious conclusion.
As the system stands, we have three termination conditions for a pair of tasks: 1) Both run to completion. 2) One completes, but the other hits a barrier. 3) Both hit barriers. Only the second two of these situations are of concern because the differential processing is wasted. In general, if one processor completes X and the other Y positions of a Z-position task then the processing of (Y-X) positions is wasted because two new tasks will be generated to complete X+1..Z [or four or more, depending on ((Z-X)/X)] The closer X and Y are, the less is wasted, hence the improvement reported when tighter matching was implemented. What I would propose rather than the let-it-rip scenario is this: 1. The returned result unit of Y positions be split into 2 separate returns, X and X+1..Y. 2. a single task be generated for the pair to X+1..Y (both scenarios 2 and 3) and two tasks for Y+1..Z (scenario 3 only) 3. Replace the processor-classification matrix with a simple (total-number-of-positions-recently-processed/total-number-of-minutes-required) calculation for the individual processor. Allot tasks to this processor which have a maximum of (400 minutes' runtime) positions. If the returns were processed in this way, then the enemy, wasted processing, would be defeated. It would no longer matter what the differential between processors was as each would home in on its preferred task-size. Certainly, the splitting operation would increase server-side load, but this would be balanced by the reduction in later-generation production (as processors home in on their 6hr run-time tasksize) reduction in tasks despatched (no re-processing) and no need for processor-classification logic. The very slowest processors could then be given the very useful task of cleaning up the very small tasks that could result rather than attempting to swallow parent units intended for the ever-increasing power of the "average" machine. And since it appears from the WU numbering that tasks of arbitrary size can be generated, it might even be possible to automate the sizing of tasks, dynamically creating bigger ones for the newer, faster machines as they become available. And this would even let the old plodder P2/200s in the print-server retirement homes put the final touches once the speed-demons have done the bulk of the work. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
The object of this thread was to determine which folk would be okay with an all-positions, no matter how long, as by discretion of the technicians, so all can truly run to the end even it takes a week, as a KISS solution with max return with the least effort [my guess] on the part of the 100% resource booked WCG staff. As you read, even that is not as simple to implement, us guessing what the back-end mechanics are. The rest of the optimization approaches have already been discussed in other threads, which I wanted to keep out of this one!
----------------------------------------So, who does not flinch to run multi day jobs on a routine basis? Say Aye! edit: @rembertw, the Rosetta approach to e.g. HPF2 has been mentioned a few times, 2,4,8,12,24 hours... not the object of this thread, as this one aims to get an as easy as possible for techs result efficiency improvement for HCMD2, which 2,4,8 hour selectable stretches would not.
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 1 times, last edit by Sekerob at Feb 23, 2010 8:25:47 AM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Say Aye
|
||
|
|
KerSamson
Master Cruncher Switzerland Joined: Jan 29, 2007 Post Count: 1684 Status: Offline Project Badges:
|
Let it rip ? why not !
----------------------------------------I would support and join every initiative which can improve crunching efficiency ! Cheers, Yves |
||
|
|
rembertw
Senior Cruncher Belgium Joined: Nov 21, 2005 Post Count: 275 Status: Offline Project Badges:
|
Ok, 100% back to topic then.
![]() If it's a matter of choice for the user: then yes, of course. Let the techs look at the feasability, the possible gain they can make. Taking in account weekends, smaller and larger holidays is something for the techs. I'm most of the time "Pro-choice". Keep it simple for the mass, give options for the interested. |
||
|
|
Col323
Senior Cruncher Joined: Nov 4, 2008 Post Count: 372 Status: Offline Project Badges:
|
Let it Rip! +1 |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I like Mysteron's solution.
|
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I may have missed the point. What is the benefit of having my little P4 crunch a job for 3-8 days, taking on the risk of losing all the credit if the box gets turned off, the power goes out or the WU crashes at 95%?
Why do we need to consider a system change? What are the benefit(s) to us or to the scientists? Is it broken now so we need to fix it or are we going to fix it so it gets broken? Just asking........ |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I may have missed the point. What is the benefit of having my little P4 crunch a job for 3-8 days, taking on the risk of losing all the credit if the box gets turned off, the power goes out or the WU crashes at 95%? Maybe you've missed the word option in the title? ![]() |
||
|
|
|