| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: BOINC Agent Support
Thread: BOINC: v7 Minimum work buffer / Max additional work buffer & v6 Connect about every xx days / Additional work buffer operation |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 40
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Draft FAQ:
----------------------------------------Up to client version 6.12, the "Additional buffer" (AB) operated exactly as the name implies, maintaining an extra amount of work queue on the host. With a default of "Connect about every 0.10 days" (CE) and an "Additional work buffer 0.25 days", the client had in effect always about 0.35 days worth of cached work [the remaining time on running tasks + the estimated time on tasks "Ready to Start"]. Soon as the "Total time to complete" (TTC) balance per active core/thread on a client would drop below this value, the client asks for more work to top up the cache. One of the purposes is, to facilitate a device in continuing computing for 0.35 days at least, in case the internet connection to the project servers would go down. New behavior in BOINC version 7 is that work requesting is no longer done when the total of CE and AB goes below the sum balance. Rather, work will be requested when the total TTC of the cache, per active core/thread, goes below the CE value. Commensurate with this behavioral change [how the client scheduler operates], the two functions are renamed respectively from "Connect about every" to "Minimum work buffer" [MinB] and from "Additional buffer" to "Max additional work buffer" [MaxAB] (see screenshot below with commentary in red.).  This change is not a problem for new / clean installations with version 7, but with upgrades from version 6.12 or earlier, the preference values of past will remain in place. Anyone who had been running with a version 6 or earlier CE of 0.0 days and an AB of for instance 1.0 days, will mostly ** see the cache to be backfilled any time the client computes there's less than 1.000000 days of work left to do... trickle work requesting. With the new version 7 way, work will not be requested until the buffer balance [including time left of running work] has dropped below the MinB value. Then a fetch will be issued and honored [at WCG] by up to 30 work units in 1 request, but then work fetching will stop again until the first core/thread finishes the last task, threatens to go idle again, with a MinB of 0.0 days. Those who don't feel save with this behavior, ISP connectivity uncertain, mobility of host working offline etc., are suggested to increase the MinB value to the lowest balance that would not result in clients whole or part idling. When the MaxAB value is zero, the client will resume its old method of back-filling cache, the trickle down way. If the MaxAB value is set greater than zero, it will act as a maximum value *adding* to the MinB value. In the example, with a MinB of 1.0 and an MaxAB of 0.5, the minimum cached work will be 1.0 days, below which new work will automatically be request for the shortfall, and the maximum will be about 1.5 days. A maximum request will then be 0.5 days + the seconds the buffer has fallen below the MinB value. The grand objective is, that the number of work requests to the project scheduler are reduced. Particularly for larger devices with 4-6-8-12 cores and more there will be no more trickle requests of 1 or 2 tasks, but rather just 1 periodic request which is honored by up to the current set maximum tasks per call [for WCG that is 30 as at February 2012]. Those who choose to run with a MinB / MaxAB of zero days will see that a work fetch is attempted about 5 minutes before any processor thread will go idle and not as was in past, fetch a task when a processor did go idle. ** If the scheduler decides to clear the Ready to Report lines [for instance older than 24 hours] or the Update button is operated, this is combined with an early work fetch request, if there buffer has gone below the MinB+MaxAB value. Questions? --//-- NB. This is how it is [should and musts to pass to the developers]. If not liking this and 6.10 or 6.12 works well for you, stay away from 7.xx and ignore the upgrade notices [a suggestion!] you will eventually see in the client message/event log [if you are not on the WCG standard client 6.10.58 which checks with WCG, not at Berkeley developers]. As of last word, WCG is not going to need an upgrade to run their upcoming GPU project! A sample how a task queue builds on a multi-core device with a 1.2 day CE [Now Minimum work buffer] and an additional buffer of 0.55 days [Now called "Maximum "additional" work buffer", running only GFAM science. Typically a work fetch is honored by 7 tasks. World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0004965_0126_0 - (-) 0,00 0,000 11:11:37 08d,20:54:05 5-2-2012 8:41:20 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0004965_0037_0 - (-) 0,00 0,000 11:11:37 08d,20:54:05 5-2-2012 8:41:20 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0004966_0070_1 - (-) 0,00 0,000 11:11:37 08d,20:54:05 5-2-2012 8:41:20 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0004966_0136_0 - (-) 0,00 0,000 11:11:37 08d,20:54:05 5-2-2012 8:41:20 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0004965_0063_1 - (-) 0,00 0,000 11:11:37 08d,20:54:05 5-2-2012 8:41:20 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0004964_0130_1 - (-) 0,00 0,000 11:11:37 08d,20:54:05 5-2-2012 8:41:20 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0004966_0128_0 - (-) 0,00 0,000 11:11:37 08d,20:54:05 5-2-2012 8:41:20 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0005004_0021_0 - (-) 0,00 0,000 11:11:37 09d,02:30:12 5-2-2012 14:17:27 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0005004_0063_1 - (-) 0,00 0,000 11:11:37 09d,02:30:13 5-2-2012 14:17:27 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0005004_0072_1 - (-) 0,00 0,000 11:11:37 09d,02:30:13 5-2-2012 14:17:27 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0005002_0140_0 - (-) 0,00 0,000 11:11:37 09d,02:30:13 5-2-2012 14:17:27 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2aqk_TB_ENRmutS94A_0005004_0135_1 - (-) 0,00 0,000 11:11:37 09d,02:30:13 5-2-2012 14:17:27 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005060_0148_1 - (-) 0,00 0,000 10:50:32 09d,10:45:56 5-2-2012 22:33:12 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005059_0170_1 - (-) 0,00 0,000 10:50:32 09d,10:45:56 5-2-2012 22:33:12 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005059_0125_1 - (-) 0,00 0,000 10:50:32 09d,10:45:56 5-2-2012 22:33:12 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005060_0082_0 - (-) 0,00 0,000 10:50:32 09d,10:45:56 5-2-2012 22:33:12 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005060_0075_0 - (-) 0,00 0,000 10:50:32 09d,10:45:56 5-2-2012 22:33:12 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005059_0002_1 - (-) 0,00 0,000 10:50:32 09d,10:45:56 5-2-2012 22:33:12 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005060_0067_0 - (-) 0,00 0,000 10:50:32 09d,10:45:56 5-2-2012 22:33:12 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005147_0014_1 - (-) 0,00 0,000 10:50:32 09d,23:04:39 6-2-2012 10:51:57 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005147_0118_1 - (-) 0,00 0,000 10:50:32 09d,23:04:39 6-2-2012 10:51:57 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005146_0097_0 - (-) 0,00 0,000 10:50:32 09d,23:04:39 6-2-2012 10:51:57 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005147_0078_1 - (-) 0,00 0,000 10:50:32 09d,23:04:39 6-2-2012 10:51:57 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005147_0111_0 - (-) 0,00 0,000 10:50:32 09d,23:04:39 6-2-2012 10:51:57 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005147_0123_0 - (-) 0,00 0,000 10:50:32 09d,23:04:39 6-2-2012 10:51:57 Ready to start - 0.00 World Community Grid 6.11 gfam GFAM_x2oos_PfENR_0005147_0048_1 - (-) 0,00 0,000 10:50:32 09d,23:04:39 6-2-2012 10:51:57 Ready to start - 0.00 [Edit 8 times, last edit by Former Member at May 5, 2012 7:41:48 AM] |
||
|
|
mikey
Veteran Cruncher Joined: May 10, 2009 Post Count: 826 Status: Offline Project Badges: 
|
Draft FAQ: Up to client version 6.12, the "Additional buffer" (AB) operated exactly as the name implies, maintaining an extra amount of work queue on the host. With a default of "Connect every 0.10 days" and an "Additional Buffer 0.25 days", the client had in effect always about 0.35 days worth of cached work [the remaining time on running tasks + the estimated time on tasks "Ready to Start"]. Soon as the "Total time to complete" (TTC) balance per active core/thread on a client would drop below this value, the client asks for more work to top up the cache. One of the purposes is, to facilitate a device in continuing computing for 0.35 days at least, in case the internet connection to the project servers would go down. New behavior in BOINC version 7 is that work requesting is no longer done when the total of CE and AB goes below the sum balance. Rather, work will be requested when the total TTC of the cache, per active core/thread, goes below the CE value. Commensurate with this behavioral change [how the client scheduler operates], the two functions are renamed to "Minimum work buffer" and "Max additional work buffer" (see screenshot below). This change is not a problem for new / clean installations with version 7, but with upgrades from version 6.12 or earlier, the preferences of past will remain in place. Anyone who had been running with a CE of 0.0 days and an AB of for instance 1.0 days, will see the cache to be completely depleted till the first core/thread finishes the last cached task, at which point a work fetch is issued for the 1.0 days. This will be honored [at WCG] by up to 30 work units in 1 request, but then work fetching will stop again until the first core/thread finishes the last task, threatens to go idle. Those who don't feel save with this behavior, ISP connectivity uncertain, mobility of host working offline etc., are suggested to increase the CE value to the lowest balance that would not result in clients whole or part idling. When the AB value is zero, the client will resume it's old method of back-filling cache, the trickle way. If the AB value is set greater than zero, it will act as a maximum value *adding* to the CE value. In the example with a CE of 1.0 and an AB of 0.5, the minimum cache work will be 1.0 and the maximum will be about 1.5 days. The grand object is, that the number of work requests to the project scheduler are reduced. Particularly for larger devices with 4-6-8-12 cores and more there will be no more trickle requests of 1 or 2 tasks, but rather just 1 request which is honored by up to 30 tasks per call. Questions? Very clear to me, THANKS for laying this out!! I LIKE to micro-manage my pc's and this will make it better for me now that I understand what is going on! THANKS AGAIN!!   |
||
|
|
nanoprobe
Master Cruncher Classified Joined: Aug 29, 2008 Post Count: 2998 Status: Offline Project Badges: 
|
Is this the new behavior for WCG only or for all BOINC projects? The reason I ask is I'm currently running version 7.0.8 and the preferences window looks the same. CE is still CE and AB is still AB.
---------------------------------------- Now as for the way it fetches tasks I see no difference but that may be due to the fact I keep a very low AC and CE value.  On another note, those of you who will be doing GPU crunching when WCG implements it will need to upgrade to version 7.x.x for it to work. AKAIK only those versions will support OpenCL.
In 1969 I took an oath to defend and protect the U S Constitution against all enemies, both foreign and Domestic. There was no expiration date.
----------------------------------------  [Edit 1 times, last edit by nanoprobe at Feb 6, 2012 5:12:47 PM] |
||
|
|
marvey11
Advanced Cruncher Germany Joined: Apr 2, 2011 Post Count: 89 Status: Offline Project Badges: 
|
Is this the new behavior for WCG only or for all BOINC projects? The reason I ask is I'm currently running version 7.0.8 and the preferences window looks the same. CE is still CE and AB is still AB. The strings in my preferences window (BOINC 7.0.7) are also the same as in 6.12 with "Connect about every" but the behaviour is definitely the new one. My settings are 0.1 and 0.5 days, respectively, and the cache is only refilled if the work queue size falls below approximately 2:20 h (which is nearly 0.1 days or 2.4 hours). I like this new behaviour very much. Much more logical. When I started using BOINC with version 6 I played around with the two values and finally gave up trying to understand why it was necessary to have two values in the first place where one would have sufficed...  |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Is this the new behavior for WCG only or for all BOINC projects? The reason I ask is I'm currently running version 7.0.8 and the preferences window looks the same. CE is still CE and AB is still AB. Now as for the way it fetches tasks I see no difference but that may be due to the fact I keep a very AC and CE value. On another note, those of you who will be doing GPU crunching when WCG implements it will need to upgrade to version 7.x.x for it to work. AKAIK only those versions will support OpenCL. Let me repeat: You do NOT need upgrading! There's no need for OpenCL detection by BOINC as the apps themselves will do that. As you can see in the screenshot the prefs have been adapted... being a few steps ahead. WCG has recently increased the fetch limit from 15 to 30 per call to cater for bigger devices and this new client behavior [see discussion with knreed]. What other projects do is their business... BOINC clients works same, everywhere, that's for sure. --//-- [Edit 1 times, last edit by Former Member at Feb 6, 2012 3:47:54 PM] |
||
|
|
kashie
Cruncher Joined: Mar 7, 2007 Post Count: 46 Status: Offline Project Badges: 
|
Yes the 2 preferences for buffer size have been renamed as per the OP. This should show in more recent 7.0.xx versions.
http://boinc.berkeley.edu/trac/changeset/25168/boinc |
||
|
|
Dataman
Ace Cruncher Joined: Nov 16, 2004 Post Count: 4865 Status: Offline Project Badges: 
|
I have not put a 7.0.x version up on the test machine yet. So I think I will do that today. I'm on 64-bit and wondering why the 32-bit version is 7.95 meg and the 64-bit version 9.25 meg. Seems like a large difference. I plan to run it on the machine that runs CPDN/GPUGrid. Anything to look out for?
----------------------------------------  |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
7 through 7.0.14 is not self-starting after installation. You'll have to do that yourself through the BM or manually starting the service.
----------------------------------------Oh, and a major download issue for gzb task files just surfaced with this version the MD5 failing, so beware... 7 is still ALPHA! --//-- edit: Developers confirm that 7.0.14 and .12 / .13 are broken v.v. use on WCG. Cannot properly handle .gzb files. [Edit 1 times, last edit by Former Member at Feb 6, 2012 10:42:32 PM] |
||
|
|
Dataman
Ace Cruncher Joined: Nov 16, 2004 Post Count: 4865 Status: Offline Project Badges:
|
Yes the 2 preferences for buffer size have been renamed as per the OP. This should show in more recent 7.0.xx versions. http://boinc.berkeley.edu/trac/changeset/25168/boinc Thank you for the link kashie (my team mate). That is exactly what I was looking for!!! I get lost sometimes when wandering around Berekley.    [Edit 1 times, last edit by Dataman at Feb 6, 2012 4:16:24 PM] |
||
|
|
kashie
Cruncher Joined: Mar 7, 2007 Post Count: 46 Status: Offline Project Badges:
|
I have not put a 7.0.x version up on the test machine yet. So I think I will do that today. I'm on 64-bit and wondering why the 32-bit version is 7.95 meg and the 64-bit version 9.25 meg. Seems like a large difference. I plan to run it on the machine that runs CPDN/GPUGrid. Anything to look out for? Unless you really need it such as for POEM or Albert OpenCL applications, I would steer clear myself. Wouldn't touch it with a barge pole. In my experience the changed work buffer policies do not work well for GPU projects that have limited/intermittent availability of work and there may be other bugs as sometimes found in development versions. If you set a high Minimum work buffer with return results immediately in order to get POEM tasks, you then need to manually update your CPU projects every day after resetting the buffers lower. Otherwise you will end up with inconsiderately large caches on quorum CPU projects. It's only once a day so not really a trouble but with the release versions this extra fiddling is not necessary. If POEM had better work availability, I would rather just run 2 instances of Windows BOINC using a release BOINC version for CPU projects or use my VirtualBox Linux BOINC for CPU projects. However I prefer to use freed up CPU cores for CPU projects if POEM GPU runs out of work. Does GPUGrid still have a bonus for early return of tasks? If so and you need to set a higher Minimum work buffer for any reason you may need to use return results immediately or an AutoUpdate script on GPUGrid or perhaps miss out on the bonus. |
||
|
|
|