Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 62
Posts: 62   Pages: 7   [ Previous Page | 1 2 3 4 5 6 7 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 12845 times and has 61 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

Let it Rip!
[Feb 23, 2010 12:25:36 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Mysteron347
Senior Cruncher
Australia
Joined: Apr 28, 2007
Post Count: 179
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

I believe that a proper analysis should not begin with leaping to an obvious conclusion.

As the system stands, we have three termination conditions for a pair of tasks:

1) Both run to completion.
2) One completes, but the other hits a barrier.
3) Both hit barriers.

Only the second two of these situations are of concern because the differential processing is wasted. In general, if one processor completes X and the other Y positions of a Z-position task then the processing of (Y-X) positions is wasted because two new tasks will be generated to complete X+1..Z [or four or more, depending on ((Z-X)/X)]

The closer X and Y are, the less is wasted, hence the improvement reported when tighter matching was implemented.

What I would propose rather than the let-it-rip scenario is this:

1. The returned result unit of Y positions be split into 2 separate returns, X and X+1..Y.
2. a single task be generated for the pair to X+1..Y (both scenarios 2 and 3) and two tasks for Y+1..Z (scenario 3 only)
3. Replace the processor-classification matrix with a simple (total-number-of-positions-recently-processed/total-number-of-minutes-required) calculation for the individual processor. Allot tasks to this processor which have a maximum of (400 minutes' runtime) positions.

If the returns were processed in this way, then the enemy, wasted processing, would be defeated. It would no longer matter what the differential between processors was as each would home in on its preferred task-size.

Certainly, the splitting operation would increase server-side load, but this would be balanced by the reduction in later-generation production (as processors home in on their 6hr run-time tasksize) reduction in tasks despatched (no re-processing) and no need for processor-classification logic.

The very slowest processors could then be given the very useful task of cleaning up the very small tasks that could result rather than attempting to swallow parent units intended for the ever-increasing power of the "average" machine.

And since it appears from the WU numbering that tasks of arbitrary size can be generated, it might even be possible to automate the sizing of tasks, dynamically creating bigger ones for the newer, faster machines as they become available.

And this would even let the old plodder P2/200s in the print-server retirement homes put the final touches once the speed-demons have done the bulk of the work.
[Feb 23, 2010 3:38:10 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

The object of this thread was to determine which folk would be okay with an all-positions, no matter how long, as by discretion of the technicians, so all can truly run to the end even it takes a week, as a KISS solution with max return with the least effort [my guess] on the part of the 100% resource booked WCG staff. As you read, even that is not as simple to implement, us guessing what the back-end mechanics are. The rest of the optimization approaches have already been discussed in other threads, which I wanted to keep out of this one!

So, who does not flinch to run multi day jobs on a routine basis? Say Aye!

edit:

@rembertw, the Rosetta approach to e.g. HPF2 has been mentioned a few times, 2,4,8,12,24 hours... not the object of this thread, as this one aims to get an as easy as possible for techs result efficiency improvement for HCMD2, which 2,4,8 hour selectable stretches would not.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at Feb 23, 2010 8:25:47 AM]
[Feb 23, 2010 8:15:42 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

Say Aye
[Feb 23, 2010 11:38:32 AM]   Link   Report threatening or abusive post: please login first  Go to top 
KerSamson
Master Cruncher
Switzerland
Joined: Jan 29, 2007
Post Count: 1684
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

Let it rip ? why not !
I would support and join every initiative which can improve crunching efficiency !
Cheers,
Yves
----------------------------------------
[Feb 23, 2010 12:39:47 PM]   Link   Report threatening or abusive post: please login first  Go to top 
rembertw
Senior Cruncher
Belgium
Joined: Nov 21, 2005
Post Count: 275
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

Ok, 100% back to topic then. smile

If it's a matter of choice for the user: then yes, of course. Let the techs look at the feasability, the possible gain they can make. Taking in account weekends, smaller and larger holidays is something for the techs.

I'm most of the time "Pro-choice". Keep it simple for the mass, give options for the interested.
[Feb 23, 2010 12:41:14 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Col323
Senior Cruncher
Joined: Nov 4, 2008
Post Count: 372
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

Let it Rip!

+1
[Feb 23, 2010 4:49:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

I like Mysteron's solution.
[Feb 23, 2010 9:57:01 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

I may have missed the point. What is the benefit of having my little P4 crunch a job for 3-8 days, taking on the risk of losing all the credit if the box gets turned off, the power goes out or the WU crashes at 95%?

Why do we need to consider a system change? What are the benefit(s) to us or to the scientists? Is it broken now so we need to fix it or are we going to fix it so it gets broken? Just asking........
[Feb 23, 2010 11:01:29 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: How many would be OK with a "Let it Rip" option on HCMD2?

I may have missed the point. What is the benefit of having my little P4 crunch a job for 3-8 days, taking on the risk of losing all the credit if the box gets turned off, the power goes out or the WU crashes at 95%?

Maybe you've missed the word option in the title? tongue
[Feb 24, 2010 5:16:48 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 62   Pages: 7   [ Previous Page | 1 2 3 4 5 6 7 | Next Page ]
[ Jump to Last Post ]
Post new Thread