Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 2370
Posts: 2370   Pages: 237   [ Previous Page | 135 136 137 138 139 140 141 142 143 144 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 2443724 times and has 2369 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

Welllll, now that you know, don't go bonkers and keep that setting 2-3 days below the standard deadline, at least. ;D.

Cache more than 2 days worth and you wont get the A type or repair/make up jobs in the run-out. Also due the substantial variability of DDDT2's dozen different WU sub-types, if caching too much [5+ days or even less], the client may off and on switch in panic/high priority processing mode, which could eat so much memory whilst switching around jobs, that the machine could freeze or crawl. A panic state stops work fetching... the price for the caching fun.

The ones not being served with all this caching are WCG (storage inflation) and the scientists. If everyone would run with low cache, more would be able to participate without all the expert knobbing, and more batches would complete quicker for return to the scientist, so they can assess their validity, so they can decide on what more work to generate.

Don't feel guilty though :P

--//--
[May 3, 2011 10:11:30 AM]   Link   Report threatening or abusive post: please login first  Go to top 
sk..
Master Cruncher
http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif
Joined: Mar 22, 2007
Post Count: 2324
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

A mere 12h of dddt2 cache will not move me to far towards blue, but the 5days of PV should inch me forward, over the next ten days or so. While my orientation has changed back towards points (the right place), I would still like to see this one out.
----------------------------------------
[Edit 1 times, last edit by skgiven at May 3, 2011 11:21:52 AM]
[May 3, 2011 11:21:09 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Allen008
Senior Cruncher
USA
Joined: Sep 22, 2009
Post Count: 244
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

Cache more than 2 days worth and you wont get the A type or repair/make up jobs in the run-out. Also due the substantial variability of DDDT2's dozen different WU sub-types, if caching too much [5+ days or even less], the client may off and on switch in panic/high priority processing mode, which could eat so much memory whilst switching around jobs, that the machine could freeze or crawl. A panic state stops work fetching... the price for the caching fun.

The ones not being served with all this caching are WCG (storage inflation) and the scientists. If everyone would run with low cache, more would be able to participate without all the expert knobbing, and more batches would complete quicker for return to the scientist, so they can assess their validity, so they can decide on what more work to generate.


Thanks, SekeRob, for your suggestions about caching. I especially like the comment about everyone running with low cache. Unfortunately, many run with large caches. The range for the network workspace is 10 days or less. I wonder why BOINC has that range is so large? Is the network workspace on my client or on the server? I'm assuming the cache is on my client.
----------------------------------------

[May 3, 2011 2:38:40 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

There are projects outside WCG that have much longer deadlines... 14-30-60 days...1-2 years but also shorter 1-2-3 days, so it was capped at 10 days as else this would/could have tremendous scheduling issues both for the client as well as for the projects for those doing multi grid crunching more so... particular those grids with large daily turnover in WU and storage needs.

If you have a cache of 10 days and you'd select to include HCC which has 7 day deadline, you be seeing permanent panic whenever a HCC is in cache and since the client would schedule these ahead continuously, the servers would think your large cache machine is still returning fast, to a point that chaos will rule for the client. Too complex for it to handle elegantly, so the recommendation is to at least set the cache lower than the shortest deadline ever occurring for any assigned work for a smoothest possible ride.

The simplified description :D

--//--
[May 3, 2011 2:51:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Allen008
Senior Cruncher
USA
Joined: Sep 22, 2009
Post Count: 244
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

OK, let me see, SekeRob, if I understand what you're saying.

1. Keep the cache and network work buffer small enough that the client never goes into priority mode.

2. If I want Type A or repair WU, I should set my cache and buffer to 2 days or less.

3. Smaller caches will help the scientists get results sooner, which is, after all, the real reason why we're doing this thing.

Right now, I'm running only DDDT2 and my deadlines are about 7 days out. I should, therefore, keep my cache & network work buffer less than 7 days. However, I need a margin to allow for things that might slow down my processing of WU, so I really should set my cache & buffer to maybe half or so of the time to deadlines. Less if I want Type A or repair WU.

Have I missed anything important?
----------------------------------------

[May 3, 2011 3:44:45 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

You've got it down near perfect. If your deadlines are presently showing 7 days out and assuming there's presently only C type in circulation, your cache is likely over 3 days, which knocks you out of the point 2 eligibility. B Type is 7 days deadline too together with HCC and Beta is 4 days, though latter will be send to anyone enrolled in the testing program, regardless of the cache size.

If your client consistently runs with return times longer than 7 days, you'd not be getting the HCC+B type as the client and server would agree that timely return is implausible **. So, if running a cache of > 7days for longer and switching from DDDT2 to HCC, this would very probably cause a message to be send to say that the device is tardy. When then the "If there is no work for....send anything else" is not selected, the chance of an idle device mounts, if only crunching at WCG, which of course we like all member to do ;)

Happy crunching.

--//--

edit: ** Then the client message/event log show: No Work Sent (Won't Finish In Time)
----------------------------------------
[Edit 1 times, last edit by Former Member at May 8, 2011 8:33:15 AM]
[May 3, 2011 4:19:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

I have followed this thread as it progressed. One thing about BOINC is that you cannot know too much about what impact the various settings have.

I have my buffer set at 2 days now and I am conscious of how that impacts the validation of WU. I find it interesting that I now have 18 20 pages of PV which has increased from 15 pages since I upped my buffer from 1 to 2 days. Logic should dictate the PV go down not up. Doesn't really matter to me though. cool
----------------------------------------
[Edit 1 times, last edit by Former Member at May 4, 2011 4:55:10 PM]
[May 3, 2011 9:32:58 PM]   Link   Report threatening or abusive post: please login first  Go to top 
nasher
Veteran Cruncher
USA
Joined: Dec 2, 2005
Post Count: 1423
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

I am still wondering how many more days we have of new work here.. its definatly a nice rainshower...


I am only sitting at about 3 pages of PV but i also dont have a large cache.
----------------------------------------

[May 4, 2011 5:10:45 PM]   Link   Report threatening or abusive post: please login first  Go to top 
genes
Advanced Cruncher
USA
Joined: Jan 28, 2006
Post Count: 132
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

I would venture to say that how much you have in PV is not a function of YOUR cache size, but is a function of everyone else's cache size, as the project waits for these results to return. If you have a large cache, then chances are that by the time you return a result, it is immediately validated against someone else's PV result.

Edit: Of course, it is also a function of how many cores you have running and how fast they are. The more you return, the more ends up in PV, that should be obvious.
----------------------------------------
[Edit 1 times, last edit by genes at May 5, 2011 1:30:16 PM]
[May 5, 2011 1:26:35 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: New thread for It's raining Dengue, Hallelujah!...

I am still wondering how many more days we have of new work here.. its definatly a nice rainshower...


I am only sitting at about 3 pages of PV but i also dont have a large cache.

470,000 have validated since last release starting about April 19. Just had a d434 queing up. If this continues to the full, e500, then we could be hitting 700-750k tasks when C-Type dg02 is done.

Taking 22 million for 18 targets, this means about 1.2 million of C-Type to complete for each, but that depends on A and B type... are those all complete for dg02? If not maybe more C is imminent in this round.

Think we're now at a point where it becomes tedious for the techs to keep calling out detail work totals for this science. It's a frequent hands on for DDDT2, so really no automated process to make it easy to know how much there's coming through at any one time, pre- or in-feeder ... at any rate, started to work seriously on getting year 2 on the road, for me badge... i.e. there's as of now more suction ;>)

Still enjoying the hunt.

--//--
[May 5, 2011 1:45:16 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 2370   Pages: 237   [ Previous Page | 135 136 137 138 139 140 141 142 143 144 | Next Page ]
[ Jump to Last Post ]
Post new Thread