Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 152
Posts: 152   Pages: 16   [ Previous Page | 5 6 7 8 9 10 11 12 13 14 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 497209 times and has 151 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

Dear Bearcat and SekeRob,
glad to hear that CEP2 is running smoother now than it used to in the beginning.
[Mar 3, 2011 12:02:25 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Rickjb
Veteran Cruncher
Australia
Joined: Sep 17, 2006
Post Count: 666
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

cleanenergy: "You can select the number of CEP2 wus you want to crunch at a time by adjusting the value in ‘Number of workunits per host for The Clean Energy Project - Phase 2?’ (no complicated micro-management necessary)" ...
Thanks, but this is not the same as the no of CEP2 WUs to run simultaneously. This sets the total no of CEP2 WUs sent out but not yet returned & reported to the WCG server. I don't think the server cares what you are doing with the the WUs at a given moment.
Ballpark calculation for this CEP2 WUs setting (C):
s = no of Simultaneous CEP2 WUs to run
D = no of CEP2 WUs processed per Day per CPU core
Q = "Additional work buffer ... (days)" (Queue) setting in BOINC >> Advanced >> Preferences
C = s x D x Q
Without active management, the actual no of CEP2 WUs that run simultaneously will vary because the spacing of the CEP2s in the cache and the runtimes of these and WUs from other projects will vary. I have FAAH in some profiles, and FAAH runtimes are currently varying by over 2:1, and active queue management is needed to keep 's' constant.

Behaviour will be better if cache length is kept short so that a C setting of (2 x s) or (3 x s) can be used. However, some of us are old enough to remember the Great Boulder Colorado Christmas Blizzard way back in '06, when present Harvard scientists were nothing more than glints in their daddies' eyes. A WCG server in Boulder crashed near the start of the holiday period, people were on leave, and no-one could get access to the hardware anyway, because of the snow & ice. The server was down for about 4-5 days. Some of us like to run long-ish caches. 3 days is reported to be standard at XS.
[Mar 3, 2011 2:22:00 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

T.S. Garp had no glint as far as my reading was back then and memory does change over time... the outage was about 48 hours and partial at that looking at the global history for X-mas period 2006 and http://www.worldcommunitygrid.org/forums/wcg/viewthread_thread,10703
[Mar 3, 2011 2:52:51 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

...However, some of us are old enough to remember the Great Boulder Colorado Christmas Blizzard way back in '06, when present Harvard scientists were nothing more than glints in their daddies' eyes...

lol...so Harvard's got a bunch of these running around:



...or did you mean 1906?

laughing
[Mar 3, 2011 3:17:20 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

Dear Rickjb,
You may be right on this one. In this case we don’t have a good answer – maybe IBM has an idea. You could increase the ‘Number of workunits per host…’ so that it is less likely to fall below 2 – or cut back on other projects biggrin . Sorry that we cannot be of more help.

Best wishes,
Your Harvard CEP team
[Mar 3, 2011 3:41:45 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

@SMTB1963 & Rickjb: Well, we essentially all look like this - LOL. Theoreticians in particular run around all day with an overfull beaker of CuSO4 solution, gigantic goggles, and an XXL sized labcoat. And yes, some of us could even already walk in 2006 biggrin (although that particular skill tends to get lost by spending too much time on the computer).
[Mar 3, 2011 3:49:26 PM]   Link   Report threatening or abusive post: please login first  Go to top 
uplinger
Former World Community Grid Tech
Joined: May 23, 2005
Post Count: 3952
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

cleanenergy: "You can select the number of CEP2 wus you want to crunch at a time by adjusting the value in ‘Number of workunits per host for The Clean Energy Project - Phase 2?’ (no complicated micro-management necessary)" ...
Thanks, but this is not the same as the no of CEP2 WUs to run simultaneously. This sets the total no of CEP2 WUs sent out but not yet returned & reported to the WCG server. I don't think the server cares what you are doing with the the WUs at a given moment.
Ballpark calculation for this CEP2 WUs setting (C):
s = no of Simultaneous CEP2 WUs to run
D = no of CEP2 WUs processed per Day per CPU core
Q = "Additional work buffer ... (days)" (Queue) setting in BOINC >> Advanced >> Preferences
C = s x D x Q
Without active management, the actual no of CEP2 WUs that run simultaneously will vary because the spacing of the CEP2s in the cache and the runtimes of these and WUs from other projects will vary. I have FAAH in some profiles, and FAAH runtimes are currently varying by over 2:1, and active queue management is needed to keep 's' constant.

Behaviour will be better if cache length is kept short so that a C setting of (2 x s) or (3 x s) can be used. However, some of us are old enough to remember the Great Boulder Colorado Christmas Blizzard way back in '06, when present Harvard scientists were nothing more than glints in their daddies' eyes. A WCG server in Boulder crashed near the start of the holiday period, people were on leave, and no-one could get access to the hardware anyway, because of the snow & ice. The server was down for about 4-5 days. Some of us like to run long-ish caches. 3 days is reported to be standard at XS.


Rickjb,

You are correct in the assumption that the server has no idea what your plans are with the work units once you download them. The only way to guarentee just one cep2 work unit run on your machine at a given time is set the number to the default of 1.

The only way to keep just one CEP2 running at a given time on a machine but download more than one would be to change the scheduler on the client. This however is not going to happen because it needs to work for all BOINC projects.

One suggestion I would make to get a better 1:1 ratio for CEP2 is to run DDDT-2 as your other project. It has a low priority in the feeder (which sends the workunits out). Also, set your additional work buffer down to 0.01. This will cause your agent to download a new work unit once one completes. Chances of getting a CEP2 work unit over DDDT-2 are very high. Also, keep your setting to 1 for workunits of CEP2 per host.

-Uplinger
[Mar 3, 2011 4:22:22 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Rickjb
Veteran Cruncher
Australia
Joined: Sep 17, 2006
Post Count: 666
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

Thanks, uplinger, but as I pointed out, the WCG servers have frequent outages and with a work cache setting of only 0.01 days, I would often run out of work. About 10 hours ago, the WCG WU despatching server went out of action, complaining "can't attach shared memory" and I'd have run out of work.
The availability of DDDT2 WUs is currently very low, eg zero. I tried getting some exclusively only yesterday & failed to get any. Crunching only 1 CEP2 on a quad leaving 3 empty cores would be a 75% processing-time loss.
DDDT2 performs heavy disc I/O and in the past has caused one of my machines to lock up so that WCG programs stop for 30+ seconds and BOINC kills them. CEP2 is also doing this. Running the 2 projects together would not be a good idea. See also Re: work units not finishing , BOINC Agent Support > exited with zero st...o 'finished' file , Beta Test Support Forum > Re: BETA Clean ...ject phase 2 version 6.35 , My thread @ XS > WCG - WCG tasks die &quot...;finished' file"
Setting the number of outstanding CEP2 WUs to 1 almost guarantees that I receive no CEP2 work at all because WCG's bandwidth tests think that my connection is too slow, even though it can upload at around 100kBps and download at 10-20 times that.
----------------------------------------
[Edit 1 times, last edit by Rickjb at Mar 3, 2011 6:48:31 PM]
[Mar 3, 2011 6:24:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

hmmm "the WCG servers have frequent outages" ... really?

Rickjb, maybe step back for a while, for the summing of your problems is NOT the general experience of most crunchers and certainly not mine.

--//--
[Mar 3, 2011 6:48:37 PM]   Link   Report threatening or abusive post: please login first  Go to top 
deltavee
Ace Cruncher
Texas Hill Country
Joined: Nov 17, 2004
Post Count: 4894
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

..with a work cache setting of only 0.01 days, I would often run out of work.

The availability of DDDT2 WUs is currently very low, eg zero. I tried getting some exclusively only yesterday & failed to get any.


I currently run DDDT2 exclusively with a cache of 0.4 days (soon to switch back to CEP2). Sometimes I go for hours without picking up any new WUs and this assures that I can keep crunching during those times. It certainly helped this morning with the outage.
----------------------------------------
[Edit 1 times, last edit by deltavee at Mar 3, 2011 7:43:12 PM]
[Mar 3, 2011 7:36:14 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 152   Pages: 16   [ Previous Page | 5 6 7 8 9 10 11 12 13 14 | Next Page ]
[ Jump to Last Post ]
Post new Thread