Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 15
Posts: 15   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 2321 times and has 14 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Project selection priority....

Remember that you cannot compare frequency of WU per project against the Run Time per project. If you get a 4 hour WU from one project twice as often as an 8 hour WU from another project, then the scheduler thinks you are getting the same time.
[Oct 5, 2009 6:38:20 PM]   Link   Report threatening or abusive post: please login first  Go to top 
DGG
Cruncher
United States
Joined: Mar 10, 2009
Post Count: 7
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Project selection priority....

Oh...OK, the Clean Energy Project and Influenza Project is done? They are still showing as selectable active projects and that's why I thought they were still giving out work units. I would think that they would no longer be available for sleection is they have no work units. But that does explain those two now although since March when I joined the only way to get them was to turn off all the others. Also I was trying to get Nutritious Rice which only works when I turn off all the others. I've been running with only the three projects mentioned above selected.

Makes more sense to me now. Thanks.........
[Oct 5, 2009 6:44:22 PM]   Link   Report threatening or abusive post: please login first  Go to top 
gb009761
Master Cruncher
Scotland
Joined: Apr 6, 2005
Post Count: 3010
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Project selection priority....

WCG won't actually set CEP and Flu to Complete (or in the case of flu, Intermittent), until all the WU's that are currently being crunched have been returned. CEP is nearly there, whilst there'll still be some Flu WU's floating around the system for a couple of weeks (WU's that have passed their due date, errored out or become inconclusive etc.).

Meanwhile, CEP was always set to a lower priority than the other projects - basically due to the size of file they sent back (they were HUGE), thus, causing an issue with disc space. As to Flu, I believe that this was also a project 'restricted' in size - due to similar issues.
----------------------------------------

[Oct 5, 2009 6:52:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Project selection priority....

No, there is no prejudice! As said, there are supply limitations, size variabilities, overall server load considerations.


Need to add one more consideration to this: Client load impact if running on more/all cores concurrently... yes, they are a factor... picture 4 ACAH on one quad low on memory and you'll get it.

A fifth one is more of a marketing nature, to which you'll never here hear an official comment, just something I would be doing if I were in charge. Spread projects out if they are not too big [so all who want can get their bronze] or their follow up phase is longer way away... programming learning curves, extended analysis of phase 1 results e.g.

edit: omophones
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 2 times, last edit by Sekerob at Oct 5, 2009 7:03:56 PM]
[Oct 5, 2009 7:02:20 PM]   Link   Report threatening or abusive post: please login first  Go to top 
knreed
Former World Community Grid Tech
Joined: Nov 8, 2004
Post Count: 4504
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Project selection priority....

There are a few more complexities to assigning the logic that matches work to computers:

1) BOINC doesn't store your history of contribution to a specific research application. As a result, this cannot be factored into the match-making logic. We plan to add this someday, but that is at some point in the future.

2) There is a cache of jobs ready to be matched with computers. Only the jobs in the cache are checked for suitability to send to your computer. The only control we have on distribution is the relative proportion of results ready to send. Since the projects all have varying duration and the duration varies over time, this ratio has to be periodically adjusted as average runtimes vary to keep things 'fair'.

The cache also 'interleaves' the results in an array. Thus the array would contain:

faah
dddt
hcmd2
hpf2
rice
cep1
....
faah
dddt
hcmd2
....

the interleaving accounts for the different weights (i.e. it might go hcmd2, faah, dddt, hcmd2) to account for the higher number of results sent for hcmd2

3) For a particular user, as a request comes in the following logic is run:


  • If the host is a reliable host, then scan for jobs that need to be processed by a reliable host
  • If the host is willing to accept beta work, then scan for jobs that are beta jobs
  • Look for jobs that have been marked as infeasible by scheduler request from another computer (i.e. another computer couldn't run it because it didn't have enough memory or disk space or because the hr class on the job was set to a different platform)
  • Look for any available job
  • If no work was assigned, and the user will accept work from non-preferred projects, then look for work from any project


Note that in all cases (except the last) only projects that the user indicated they want will be considered.

The biggest one here is the infeasible check. Projects like Help Conquer Cancer that run with redundancy and use the homogeneous redundancy logic enabled for the project have a lot more jobs ready to send that are already allocated to a specific hr_class. This means that they are more likely to be assigned to a computer then a job from a project like Nutritious Rice that doesn't use homogeneous redundancy. As a result, Rice has to be given a higher weight to make up for this effect.

The overall thing I'm attempting to explain, is that there are a lot of factors in addition to user preferences that affects the overall runtime that a project gets. As a result, we have to weight the projects to keep things on an even basis.

At some point I would like to query the database and look at the user preferences and measure that against their contribution and be able to deduce the desired overall contribution ratio. At the moment though, this data is locked up in XML documents stored in varchar fields. At some point we will switch that so that we can use xquery to get at the data, but for now we cannot.

I hope this helps folks understand some additional things about how the BOINC scheduler works.
[Oct 7, 2009 7:44:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 15   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread