Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 28
Posts: 28   Pages: 3   [ Previous Page | 1 2 3 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 3944 times and has 27 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: deadlines way too short

Your information is just out of date.

Nothing wrong with feedback, but aborting tasks is wasting your time more than anyone else's.
[Jun 24, 2007 1:57:36 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: deadlines way too short

Ironically, the whole issue is probably rather pointless, since WCG appears to be filtering the beta participants rather rigorously for only fast-turnaround hosts anyway.

I've currently got 3 systems crunching, 2 with no extra work buffered, and one set for 3 hours of buffered work due to intermittent internet connection issues. The 2 with no extra work buffered are getting beta WUs, and the one with 3 hours of work buffered isn't, even though it's not appreciably slower than the others. Presumably, the 3 hour work buffer pushes the total turnaround time out too far and makes the system undesirable for beta WUs.

If you're already filtering for fast-turnaround systems anyway, it doesn't really matter if you force EDF mode or not. The beta WUs will be processed quickly just like the normal WUs have been, even if they don't jump to the very front of the queue.

I'll also mention that generally, you want hands-on, involved and savvy people in your beta program, not the ones who just sign up and then never pay any attention to what's happening. These are precisely the people who'll notice when WUs aren't going to finish in time, and to know that they'll therefore be reissued (and it's not just about the bad time estimator; if you're halfway done after 4 hours, you ain't gonna make the deadline). Many will then naturally feel that if the WU is doomed to be reissued no matter what anyway, there's no point in finishing this copy, and being the good hands-on beta types they are, will tend to take action.

However, being the good involved beta types they are, they probably also would've noticed and read a simple announcement in the forums that WCG is purposely sending out WUs which can't be completed in time by many hosts, and that this is intentional, the WUs are still wanted, and please don't abort them. Would've saved a lot of hassle and wasted crunch time.

On the other hand, those aborting the overdue WUs should also realize that just because they'll be reissued, that doesn't make your copy worthless. Even if you turn it in hours late, it may still help reach the desired quorum of results, and may help avoid sending yet more reissues from other, later hosts who also aren't meeting their (later) deadlines.

There's obviously diminishing returns involved; a result turned in 2 hours late has a fairly good chance of still being useful, but one turned in 20 hours late is probably well after the quorum was reached. But a system that slow probably wouldn't have been issued a beta WU in the first place. Chances are, if your system is fast enough to get a beta WU, it'll finish it quickly enough to still be useful, over deadline or not.
[Jun 24, 2007 2:43:47 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: deadlines way too short

Really, you're all overthinking this.

Quite frankly, micromanagers are the bane of my existence. It's like my Dad always says: "You can't make it better by fiddling with it. Leave it alone!"

All beta participants are expected to do is run the work and report any problems. You will only need to take action if there is a serious problem, and in that case, there will definitely be a message in the Beta Announce forum, explaining the problem and the required action.

And remember: non deterministic computing. Nobody knows how long the work units will take, least of all the scheduler which only has a couple of hints to go by. I seem to be saying this a lot today: the time to completion is an estimate. Okay, it's a complete fabrication. It's made up. Pay it no heed! I could go into more detail, but it's dull, complicated, and I would bore you to death. Besides, you can all research this fascinating topic yourself. Be sure to ignore the documentation, it's probably out of date.
[Jun 24, 2007 3:05:37 AM]   Link   Report threatening or abusive post: please login first  Go to top 
BobCat13
Senior Cruncher
Joined: Oct 29, 2005
Post Count: 295
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: deadlines way too short

Well, just for interest's sake, I ran a BETA task on a box I knew would not finish in 6 hours. It started 14 mintues after download as a Superlink task had to finish first, and took 6:29 to finish and reported immediately. Here is the result of that experiment (my result in bold):

BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 10-- Too Late 06/24/2007 02:05:49 06/24/2007 08:50:33 6.49 76.8 / 0.0
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 9-- No Reply 06/24/2007 02:00:52 06/24/2007 08:00:52 0.00 0.0 / 0.0
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 8-- Too Late 06/23/2007 20:05:25 06/24/2007 10:00:30 5.03 52.1 / 0.0
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 7-- Valid 06/23/2007 19:57:56 06/24/2007 03:24:46 7.24 65.9 / 53.7
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 6-- Too Late 06/23/2007 18:55:53 06/24/2007 04:26:08 5.62 48.4 / 0.0
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 5-- Valid 06/23/2007 15:20:34 06/23/2007 19:15:52 3.61 52.1 / 53.7
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 1-- Valid 06/23/2007 13:52:54 06/23/2007 20:18:14 4.77 43.1 / 53.7
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 4-- Too Late 06/23/2007 13:30:12 06/24/2007 06:32:55 8.28 87.7 / 0.0
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 3-- Error 06/23/2007 12:01:06 06/23/2007 14:13:01 1.33 15.3 / 0.0
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 0-- No Reply 06/23/2007 11:42:13 06/23/2007 17:42:13 0.00 0.0 / 0.0
BETA_ faah1790_ d256n089_ x2AZ8_ 00_ 2-- No Reply 06/23/2007 11:01:30 06/23/2007 11:41:39 0.00 0.0 / 0.0

Now the question is, was my result deemed Too Late because it exceeded the Deadline date or was it because quorum had already been reached? Seeing as the 3rd valid result took 7:24 to run, which means it exceeded it's deadline, I am going to say mine was marked Too Late due to quorum being reached.

So, it appears to me that WCG is wasting volunteers' CPU cycles by creating a 6 hour deadline, immediately reissue tasks when someone exceeds that 6 hour deadline, but then rejecting those reissued tasks if they don't return it within the first 3 valid results.

A 12 hour deadline should be sufficient to make the beta tasks jump to the front of the queue without causing all of the reissues that happen with the 6 hour deadline.

Even better, look at the bottom task in that list (in italics). Unless I am reading that incorrectly, it was give 40m 9s for a deadline.
[Jun 24, 2007 2:29:06 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: deadlines way too short

I want to deeply apologize to all of you - this whole problem stems from a missing zero. The deadline should have been 216000 seconds not 21600 seconds. That is the difference between 60 hours and 6 hours.

We do set short deadlines for the workunits on beta tests - but not 6 hour deadlines.
[Jun 25, 2007 4:50:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: deadlines way too short

....and in a future WCG recommended release (not there yet) feedback to the user will be improved to indicate a rush state.
25 June 2007
- Manager: if a job is running EDF, show "high priority" in status!

Added: Link to Acronym List post in the FAQ (Start Here) forum
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at Jun 26, 2007 2:36:22 AM]
[Jun 26, 2007 2:14:37 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: deadlines way too short

Acronyms are always our undoing. This particular one causes more headaches than most because many people use it to refer to the logic behind panic scheduling, which has totally changed (and has changed a few times in the past). The actual code barely uses the term "EDF" at all.

As JM7 said: "The idea of an EDF mode is not gone, but it has been modified."

Anyway, this thread drifted waaaay off topic, so I think I will summarise:

Please do not abort beta work until you have confirmed that it is necessary. Sometimes failed beta work units help the techs get a better picture of the problem.
[Jun 26, 2007 2:28:07 AM]   Link   Report threatening or abusive post: please login first  Go to top 
retsof
Former Community Advisor
USA
Joined: Jul 31, 2005
Post Count: 6824
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: deadlines way too short

There's some new FAAH betas today. Deadline seems to be 3 days.
----------------------------------------
SUPPORT ADVISOR
Work+GPU i7 8700 12threads
School i7 4770 8threads
Default+GPU Ryzen 7 3700X 16threads
Ryzen 7 3800X 16 threads
Ryzen 9 3900X 24threads
Home i7 3540M 4threads50%
[Jul 9, 2007 10:14:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 28   Pages: 3   [ Previous Page | 1 2 3 ]
[ Jump to Last Post ]
Post new Thread