| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Suggestions / Feedback
Thread: Task Prioritization by Deadline |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 15
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hi, I've been doing WCG and BOINC for years and really appreciate the opportunity to contribute to all of these projects.
I've noticed that a lot of WCG work units have deadlines that are often less than a week away. For example, today is Oct. 14th and I have work units that have an Oct. 17 deadline at 9:48am. If it were to work on and finish these work units today, then the work it does on these units would certainly have a guaranteed upload opportunity before their deadline. Instead, though, it seems to be working on other work units with a later deadline, while not having made any progress on those with a sooner deadline. I don't know if it determines the order of work unit processing by download order or what, but experience has shown me that it will not process these short-deadline work units until it's at risk of not completing them on time. I notice this happens often, where work units with sooner deadlines aren't processed until the last minute. I don't run any other BOINC projects on this specific computer, so it isn't any sort of cross-BOINC prioritization conflict. In that case, that would be the user/BOINC over-committing their computer's processing capability, but in this scenario it's a project that is causing its own work units to miss their deadlines. I know I could manually suspend other work units until it processes these work units, but that would mean you're depending on users to manage the work unit processing themselves; on its own, it's potentially wasting processing time by consistently completing work units near or past their deadline. tldr: Work units aren't being processed until they're at risk for not meeting their deadline. Deadline should take priority over download order or whatever other factor determines the task prioritization, within a single BOINC project. |
||
|
|
retsof
Former Community Advisor USA Joined: Jul 31, 2005 Post Count: 6824 Status: Offline Project Badges: 
|
Tiebreaker workunits are usually the ones sent with short deadlines, since your computer has been reliable in the past.
----------------------------------------For projects with normal workunits ending in -0 or -1, tiebreakers have been seen ending in -2 with the short due dates you have noticed. If time becomes critical, BOINC will go into a panic mode and move up the workunits so the ones with close dates can be completed before the due date. If you don't want to work on a few of this type, simply abort them before they start and they will be resent elsewhere. Excessive aborts can kick you into unreliability, so don't go wild here. When going out of town, I stop accepting new workunits for a few days so that the queue will be whittled down to nothing and I can finish what I have. My reserve queue is only 1/2 day or less before I leave.
SUPPORT ADVISOR
----------------------------------------Work+GPU i7 8700 12threads School i7 4770 8threads Default+GPU Ryzen 7 3700X 16threads Ryzen 7 3800X 16 threads Ryzen 9 3900X 24threads Home i7 3540M 4threads50% [Edit 3 times, last edit by retsof at Oct 14, 2011 6:59:00 PM] |
||
|
|
anhhai
Veteran Cruncher Joined: Mar 22, 2005 Post Count: 839 Status: Offline Project Badges: 
|
smeggysmeg,
----------------------------------------Even though, the once with earlier deadlines aren't started, you shouldn't worry. Boinc would of started them if it thinks there is a chance that it wouldn't finish on time based on past performances of your system. However, if you wish to make boinc give higher priority to WU that are only a few days out, you can change it under perferences (details below). Click Advance->Pereferences. A window should open up, there is a field labeled "Switch between applications every". If you increase the value to 7200 minutes, it will give higher priority to those WU that are due in a few days  |
||
|
|
retsof
Former Community Advisor USA Joined: Jul 31, 2005 Post Count: 6824 Status: Offline Project Badges:
|
smeggysmeg, I think that's more for between major BOINC applications, like between WCG and SETI.Even though, the once with earlier deadlines aren't started, you shouldn't worry. Boinc would of started them if it thinks there is a chance that it wouldn't finish on time based on past performances of your system. However, if you wish to make boinc give higher priority to WU that are only a few days out, you can change it under perferences (details below). Click Advance->Pereferences. A window should open up, there is a field labeled "Switch between applications every". If you increase the value to 7200 minutes, it will give higher priority to those WU that are due in a few days All of the little WCG subprojects are really one big application, as far as BOINC is concerned, since it is one URL. My own is set to switch every 180 minutes, but since it is currently running WCG only, it doesn't really do anything. If you are switching applications, don't set that number too low or you can get blindsided by checkpoints and waste a lot of time. I do that every now and then when running other things. I *DO* find it useful to set the parameter to keep applications in memory when things are switching around. They don't have to be saved and reloaded every time the application is switched. It is also helpful when BOINC is in panic mode and running all sorts of things out of sequence.
SUPPORT ADVISOR
----------------------------------------Work+GPU i7 8700 12threads School i7 4770 8threads Default+GPU Ryzen 7 3700X 16threads Ryzen 7 3800X 16 threads Ryzen 9 3900X 24threads Home i7 3540M 4threads50% [Edit 1 times, last edit by retsof at Oct 14, 2011 10:37:38 PM] |
||
|
|
anhhai
Veteran Cruncher Joined: Mar 22, 2005 Post Count: 839 Status: Offline Project Badges:
|
restof, please just try it and see what happens. many people use this method to make sure repair jobs and beta are crunched first. I was not the person who figure this method out, and I original had the same doubts at you, but it works even if you only run WCG
---------------------------------------- |
||
|
|
gb009761
Master Cruncher Scotland Joined: Apr 6, 2005 Post Count: 3010 Status: Offline Project Badges:
|
SekeRob has created a new FAQ relating to this topic - see BOINC: High Priority / Earliest Deadline First (EDF) task processing.
---------------------------------------- |
||
|
|
Mysteron347
Senior Cruncher Australia Joined: Apr 28, 2007 Post Count: 179 Status: Offline Project Badges: 
|
I'd agree with the original thrust of this thread - that prioritisation strategy seems odd, but I gather from various comments in the many similar threads over the past few months that there are moves afoot.
I've read Sekerob's FAQ, which seems fair enough - although I feel that there's a trace of a personal aversion to large caches. Personally, I prefer a large cache - it's invaluable when there's a catastrophe that takes more than a few hours to fix. I gain the impression that the scheduling algorithm is badly showing its age. It doesn't seem to suit multiprocessors very well - an increasing "problem" as more quads and hexes are being brought in. Add in the complexities of GPUs - even multi-GPUs, and BOINC seems to lose the plot rather easily. For instance, I currently have 62 CPU-hours' (estimated) work queued on my dual-core, yet it's running two 6-hr jobs due on 29/10 - a fortnight away - at high priority (ie BOINCpanic) while it only estimates 20 minutes left on a job due in 3 days. Setting switch-between-jobs to such massive figures seems to be using brute force to overcome some fundamental issue with the scheduling. Mine is set to 750 minutes, so switching should only take place every 12 hours. It's certainly switching more rapidly than that... Giving an already-running job only just enough time to run to completion by starting others seems to be a dangerous policy. The usurped job's only got to run into a bit of headwind to have both jobs miss the deadline. We're all aware of the unpredictability of run-times, even on machines BOINCing 24/7 on a permanent connection. I'd suggest that multicores and permanent connections have crept in unannounced since the scheduler was conceived. Time for a review. |
||
|
|
retsof
Former Community Advisor USA Joined: Jul 31, 2005 Post Count: 6824 Status: Offline Project Badges:
|
restof, please just try it and see what happens. many people use this method to make sure repair jobs and beta are crunched first. I was not the person who figure this method out, and I original had the same doubts at you, but it works even if you only run WCG OK. I will run at switch 7200 minutes for awhile and watch repair jobs and any betas. It would certainly make for a better queue. One machine here has BOINC 6.12.10 and the other BOINC 6.10.58.
SUPPORT ADVISOR
Work+GPU i7 8700 12threads School i7 4770 8threads Default+GPU Ryzen 7 3700X 16threads Ryzen 7 3800X 16 threads Ryzen 9 3900X 24threads Home i7 3540M 4threads50% |
||
|
|
KWSN - A Shrubbery
Master Cruncher Joined: Jan 8, 2006 Post Count: 1585 Status: Offline |
That deserves a definite +1 Mysteron. I've watched the scheduler go through increasingly complex contortions to make its job more difficult than it needs to be for some time now.
---------------------------------------- Distributed computing volunteer since September 27, 2000 |
||
|
|
retsof
Former Community Advisor USA Joined: Jul 31, 2005 Post Count: 6824 Status: Offline Project Badges:
|
I changed the buffer on a 2 core computer from 0.5 days to two days. This one is running only HCC -0 or -1. Two jobs at the end changed to high priority and started running, with 30 other jobs also with 10/22 due dates ahead of them waiting. The two previously running jobs at the top due 10/22 are now waiting for the two at the end of the queue due 10/22 to finish. I did not expect this behavior.
----------------------------------------
SUPPORT ADVISOR
----------------------------------------Work+GPU i7 8700 12threads School i7 4770 8threads Default+GPU Ryzen 7 3700X 16threads Ryzen 7 3800X 16 threads Ryzen 9 3900X 24threads Home i7 3540M 4threads50% [Edit 5 times, last edit by retsof at Oct 16, 2011 4:03:10 AM] |
||
|
|
|