| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
| World Community Grid Forums
|
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 38
|
|
| Author |
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Hi again. It's 3:40 (AM), .... Sooo funny. BOINC still reports disk usage as less than 40 MB (with WCG and SETI@Home together). Hi again hoanqui, with respect to the 40mb you see used in those pie charts of BOINC, those are the sizes of the project subdirectories where the jobs to do / not completed are stored. BOINC does not report usage for: 1. The portions used by the tasks from the Swapfile/Virtual Memory. It only reports this at start up of what's available. 2. The Temporary store used by the files after each checkpoint. 3. AFAIK, the portions used in the progress slot subdirectories (anyone knows about this.... slot0 slot1 etc?) If you monitor the process manager and compare to the Table provided, you are observing at time 300-450mb Swapfile use. With HPF2 the actual needs are about 200mb swapfile, PER JOB, so the wall can be hit if you have concurrent jobs going. If you've set the value % of what's allowed to be used from RAM, VM or storage, you will hit the wall. There are some internal values in a job e.g. FAAH has itself limited to 500mb (last i heard). If it hits that limit it will hang and report this in the message log. As yet i do not discover which exact BOINC version you run (assume it to be 5.10 considering the Local pref ref.), but the 5.10 parameter management is, to put it mildly 'up dung creek" depending on the point release used. Not even what you see in local prefs thru a remote connection can be trusted to be correct at this time of writing thru 5.10.18. WCG has refrained from declaring any 5.10.x version recommended and continues with 5.8 offering on it's download page. WCG also asked Berkeley to hold back on sending out new release recommendations. Both 5.10.7 and 5.10.13 are premature.... they work for many, but not in sufficient hi numbers for others due all it's parm complications and bugs. So let me tell you what works for me on a C2D a. Use at most 5gb disk storage (it never will unless you get into HDC jobs on a quad where sizes can be 1gb+ per job) b. Leave at least 1gb free on disk (i got plenty more though, but swapfiles can grow) c. Use at most 75% of total disk space. (can get finicky if real space is tight) d. Use at most 75% of swap file (allow VM to freely expand in the OS settings itself) I've not had but 1 issue with any present project, but the strange none % progress on HPF2, which in 99% of the cases on a suspend/release of the WCG project (in the project tab) makes it hop across the barrier it sat looping before. If there are no messages on HPF2, then that is a possibility of happening. don't get upset.... we'll get thru it for a good cause, but accept that 5.10 is not for the greater crunching populous yet. cheers
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 1 times, last edit by Sekerob at Aug 15, 2007 8:56:16 AM] |
||
|
|
hoanqui
Cruncher Joined: Aug 14, 2007 Post Count: 15 Status: Offline |
Lawrence,
I mention HD usage because Sekerob's response about the problem was to check the minimum requirements of the projects, specifically the HD limit I had set in my prefs (200 MB). So I upped the limit to 2 GB. (as you see, it didn't help) I'm using the latest recommended BOINC version for OS X (5.10.7). As I said in my first post, I'm using local prefs (defined in the Simple View of BOINCManager), and the limits are right now 2 GB HD, 30% CPU, don't work when on battery, work after 1 minute idle. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Suggest you switch to Advance View and the Advance Menu, Local Prefs for greater detail settings and following my recommendation in my post earlier. The Simple view I think only controls the storage and not things like the VM settings.
----------------------------------------ttyl
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
hoanqui
Cruncher Joined: Aug 14, 2007 Post Count: 15 Status: Offline |
with respect to the 40mb you see used in those pie charts of BOINC, those are the sizes of the project subdirectories where the jobs to do / not completed are stored. BOINC does not report usage for: 1. The portions used by the tasks from the Swapfile/Virtual Memory. It only reports this at start up of what's available. 2. The Temporary store used by the files after each checkpoint. 3. AFAIK, the portions used in the progress slot subdirectories (anyone knows about this.... slot0 slot1 etc?) Anyway, HD free space has never decreased noticeably. When the problem reappeared last night, I had more than 2.7 GB free, and they are still free now. So I can't find a link with the problem. If you monitor the process manager and compare to the Table provided, you are observing at time 300-450mb Swapfile use. With HPF2 the actual needs are about 200mb swapfile, PER JOB, so the wall can be hit if you have concurrent jobs going. I think having a preferences-set limit of 2 GB, and 2.7 GB free on the HD, discards that possiblity. If you've set the value % of what's allowed to be used from RAM, VM or storage, you will hit the wall. Just for completeness, here I recap once again my prefs, local and otherwise: Local: 2 GB, 30% CPU, 1 minute idle, no working while on battery Web (are they used? who knows? not the normal user, that's for sure): no more than 50% CPU, no more than 75% of total mem, 3 mins idle, at most 1 CPU, switch apps every 120 mins, no more than .3 GB HD, at least .5 GB free HD, no more than 75% HD, write to disk at most every 99, stay in memory, connect to network every 3 days. There are some internal values in a job e.g. FAAH has itself limited to 500mb (last i heard). If it hits that limit it will hang and report this in the message log. The log never reported more than the oh-so-useful "zero status, no file, may need to reset". As yet i do not discover which exact BOINC version you run (assume it to be 5.10 considering the Local pref ref.), 5.10.7 (latest recommended for OS X) Not even what you see in local prefs thru a remote connection can be trusted to be correct at this time of writing thru 5.10.18. WCG has refrained from declaring any 5.10.x version recommended and continues with 5.8 offering on it's download page. Interesting. The problem is, I already was using BOINC when I found WCG months ago, so I didn't even realize that they had a different BOINC recommendation. The other (and I feel much bigger) part of the problem, of course, is the lack of any sanity checking - or at least, the lack of any reporting of the results of whatever sanity checking. "Hey, user, we'll try to run but you should be aware that this BOINC release is not an officially supported one by WCG; if something goes wrong, you should yadda yadda yadda". EVEN IF I had gotten into BOINC thru WCG, I would have surely updated my BOINC installation as soon as I had discovered that a new version was available. Which is what normal users do, and in fact is what they are usually told to do. So sanity checking AND REPORTING gets mandatory - and its lack, crippling. Which is the case. WCG also asked Berkeley to hold back on sending out new release recommendations. Both 5.10.7 and 5.10.13 are premature.... they work for many, but not in sufficient hi numbers for others due all it's parm complications and bugs. Shame on them. And shame on WCG for not letting me know without me having to go out of my way. So let me tell you what works for me on a C2D a. Use at most 5gb disk storage (it never will unless you get into HDC jobs on a quad where sizes can be 1gb+ per job) As said, 2.7 GB free, 2 GB prefs-limited. So: check. b. Leave at least 1gb free on disk (i got plenty more though, but swapfiles can grow) Check. c. Use at most 75% of total disk space. (can get finicky if real space is tight) Check. d. Use at most 75% of swap file (allow VM to freely expand in the OS settings itself) There is no VM setting in OS X that I can think of, and anyway, I set 75% of memory in the web's prefs page. So, check. I've not had but 1 issue with any present project, but the strange none % progress on HPF2, which in 99% of the cases on a suspend/release of the WCG project (in the project tab) makes it hop across the barrier it sat looping before. If there are no messages on HPF2, then that is a possibility of happening. I have tried that, and anyway I guess that the effects of that are a subset of what a project reset does, which I did four times yesterday. So... check, and Not Working For Me. don't get upset.... we'll get thru it for a good cause, but accept that 5.10 is not for the greater crunching populous yet. I do feel that this problem asks for some upsetting :P. Someone, somewhere, is not doing his/her homework. Just for the record: I'm keeping WCG suspended on my computer, as nothing new has been suggested that could improve the situation, and WCG-induced CPU maxing-out has managed to blaze a direct trail for getting into my nerves :P. (and I'm not downgrading BOINC, since anyway good old SETI@Home is happily crunching away). |
||
|
|
hoanqui
Cruncher Joined: Aug 14, 2007 Post Count: 15 Status: Offline |
Suggest you switch to Advance View and the Advance Menu, Local Prefs for greater detail settings and following my recommendation in my post earlier. The Simple view I think only controls the storage and not things like the VM settings. ttyl Nice. The last I knew was that local prefs were only available through the Simple View - that was the only reason I used to switch out of the Advanced View. So I recap the prefs, as seen from Advanced View: CPU:Do work after 1 min idle, not on batts, not while in use; switch apps every 60 mins, at most 16 processors, at most 30% CPU Network:Connect about ever .1 days, additional work buffer .25 days; everything else off Disk: use at most 2 GB, leave free at least .1 GB, at most 50% total disk space, write to disk at most every 60 secs, at most 75% swap file Memory: at most 50% when in use, at most 90% when idle, do not leave in memory Those are not the prefs I would most like to be in effect (the network values are downright weird), but anyway they still don't seem to explain anything. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
From the early days of 5.10 CA's (Community Advisors) have been vocal on WCG not recommending 5.10. The WCG download page now explicitly states the version it offers:
----------------------------------------We can't win them all, even if going out of our way to help users going. Someone with advanced MAC experience will be able to tell how to address the VM side. Here's a oogled post of someone even putting his swapfile on a dedicated partition and commenting on a 80mb default: http://www.xlr8yourmac.com/OSX/OS_x_swapfile_sizes.html http://text.dslreports.com/faq/allthingsmac/6_OSX strange thing is, i think you are the first to report issues that are linked to disk space on a Mac (I'm not sure now if it is the problem, but we can only work from eliminating possibilities). That OS with it's strong GUI is probably less needing looking under the hood.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
hoanqui, 5.10.7 has numerous preference related bugs. In particular, the Advanced View DOES NOT SHOW THE CORRECT PREFERENCES. Worse, if you then click "Okay", it saves these incorrect preferences, overwriting anything you set elsewhere. And that's why 5.10 isn't ready for release yet.
Please don't reset WCG any more. Clearly that isn't fixing your problem, so there's no point. Instead, please will you try just letting the work unit continue. You should be aware that there are periods when the BOINC throttle is not applied - in particular during checkpointing. The throttle works by suspending and resuming the application very frequently, and the application can't be suspended during checkpointing. If checkpointing is taking a very long time, then that may need looking into. |
||
|
|
hoanqui
Cruncher Joined: Aug 14, 2007 Post Count: 15 Status: Offline |
From the early days of 5.10 CA's (Community Advisors) have been vocal on WCG not recommending 5.10. The WCG download page now explicitly states the version it offers: I see that WCG, in its downloads page, offers a 5.8.x version. I can't see anywhere else (except in these forums, that is) any remote hint that the user SHOULD NOT update to the latest officially recommended BOINC version, as stated in the official BOINC homepage. How exactly am I supposed to know that? This should be made EXPLICIT, in-your-face, multiple times and in many ways: in the WCG info web pages, the WCG prefs web pages, and the BOINC logs. We can't win them all, even if going out of our way to help users going. Someone with advanced MAC experience will be able to tell how to address the VM side. Here's a oogled post of someone even putting his swapfile on a dedicated partition and commenting on a 80mb default: http://www.xlr8yourmac.com/OSX/OS_x_swapfile_sizes.html http://text.dslreports.com/faq/allthingsmac/6_OSX Oh, please. Just for the record, in case some Mac OS X user comes here and sees these posts: THERE IS NO REASON to touch your VM, except in VERY specific cases. In the earliest versions of OS X, it could SOMETIMES make sense. But it's been a long time (OS X 10.2 era) since I last saw anyone talking about swap partitions and the like for OS X. By the way, the first page you link to confirm that: they refer to OS X 10.1 (circa 2001). (the second link doesn't even mention virtual memory) In case you want some more context, my OS X with it standard VM settings (standard because there is no official way to change them) is using right now a 3GB swap file, of which more than 2.5 GB are reported as in use by the OS. OS X grows and shrinks its swap files as needed; it starts with few MB at boot and I've seen it grow past 5 GB and back under 2GB as the OS sees fit. For completeness sake again, I have 3 GB RAM, in a Macbook Core2Duo. I have right now 2.7 GB of free HD space. When freshly booted, the HD free space is about 6 GB; the VM grows in that free space. strange thing is, i think you are the first to report issues that are linked to disk space on a Mac (I'm not sure now if it is the problem, but we can only work from eliminating possibilities). That's it, we indeed can't be sure if that is the problem. And since there is absolutely no hint about what could exactly be the problem, any user with remotely similar symptoms could be having it. And as I said, I have been suffering this problem for weeks before coming here to look for some solution. Other users could just be suffering the CPU peaks and don't realize, or don't bother to look for the reason, or don't know how to! Because, as said, there is no hint, even in BOINC's logs, that HPF2 is causing CPU extreme usage. Only that "it exited with no files". Useful! |
||
|
|
hoanqui
Cruncher Joined: Aug 14, 2007 Post Count: 15 Status: Offline |
hoanqui, 5.10.7 has numerous preference related bugs. In particular, the Advanced View DOES NOT SHOW THE CORRECT PREFERENCES. Worse, if you then click "Okay", it saves these incorrect preferences, overwriting anything you set elsewhere. And that's why 5.10 isn't ready for release yet. OK, just in case I'm deactivating those local preferences and going back to the webpage ones. Please don't reset WCG any more. Clearly that isn't fixing your problem, so there's no point. Instead, please will you try just letting the work unit continue. Yes, I noticed that. But again, I was just following the terribly limited information I had about the problem: "the project may need to be reset". Yeah. About "letting the work unit continue": that means spurious CPU usage peaks that last minutes. That means impeded computer usage, and fan blowing hard for minutes. That means the noise waking me up in the night because of some problem that I have no information to solve. So that means WCG / HPF2 (haven't decided yet) are off my computer. You should be aware that there are periods when the BOINC throttle is not applied - in particular during checkpointing. The throttle works by suspending and resuming the application very frequently, and the application can't be suspended during checkpointing. If checkpointing is taking a very long time, then that may need looking into. I can report lots of instances of >1 minute periods of 100% CPU usage, which seemingly always end up with that "no file, may need reset" log message. The happening of those peaks seems random. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
The BOINC CPU throttle of 30% means it runs 3 seconds at 100% and pauses for 7 seconds.... on all projects, both 5.8 and 5.10. At some point the throttling itself caused problems in the science, with the only way avoiding offered was to set it at 100%.
----------------------------------------Like Didactylos already mentioned, during checkpointing the throttle is ignored as disk i/o takes priority. As for the original message: Task lf432_00028_11 exited with zero status but no 'finished' file One known cause is the system clock being synchronized by the system. If that happens middle of the science, particularly backward, BOINC considers a timekeeping problem and tries to return a project to the last checkpoint. Just wonder if the throttling alternation 3:7 and the system time keeping might have something to do with this. Very few i've heard of set it to 30%, considering that the processes use pure idsle time and are at lowest priority i.e. even at 100% should hardly impair use. Full tilt is known to cause the fan to run all the time, but another Mac user reported that 90% was enough for that not to happen.
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 2 times, last edit by Sekerob at Aug 15, 2007 1:25:21 PM] |
||
|
|
|