| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
| World Community Grid Forums
|
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 13
|
|
| Author |
|
|
Mgruben
Advanced Cruncher Joined: May 26, 2013 Post Count: 94 Status: Offline Project Badges:
|
Hey all,
----------------------------------------Trying to figure out how much space a CEP2 WU can possibly take up while it is crunching, since I've personally seen one go from 400MB to 850MB in a matter of minutes (then shoot back down), and I've also seen one crunching away at 1.1GB. It's relevant for me since I'm hoping to run these units on (read: I've mounted my /slots directory to) a RAMdisk, but I need some worst-case scenario figures so I avoid running out of space and crashing/aborting the WU. Tl;dr: do CEP2 WUs get bigger than 1.1GB when crunching, or is that their max size (final form, for DBZ fans)? Edit: spelling better, one edit at a time. ![]() [Edit 1 times, last edit by Mgruben at Aug 20, 2013 8:41:11 PM] |
||
|
|
Paul Schlaffer
Senior Cruncher USA Joined: Jun 12, 2005 Post Count: 279 Status: Offline Project Badges:
|
The system requirements are 2gb per work unit.
----------------------------------------In my case I bought a used 50nm Intel X25-E slc SSD on line. It's much cheaper than ram, and is handling 16 simultaneous work units. With a 2000 terabyte endurance rating it will be good for more than 8 years.
“Where an excess of power prevails, property of no sort is duly respected. No man is safe in his opinions, his person, his faculties, or his possessions.” – James Madison (1792)
|
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Here are the official system requirements - 1 GB RAM for CEP2 and 2 GB disk : https://secure.worldcommunitygrid.org/help/viewTopic.do?shortName=minimumreq
----------------------------------------Some important background information : The researchers run large molecules that use up to 8 GB RAM on their own linux grid at the lab. They are picking small molecules to send to us that can run on 32-bit compiled code. An improved version of the code is being developed. The current talk is that we will continue to have a 32-bit small version of the improved program but will also have the option of running a 64-bit version for large (8 GB) jobs. So what does our current code require? I am assuming that molecule picking is not an exact science so some assigned molecules probably exceed the limits. I have no idea how the code enforces limits. If I were doing the coding, I would simply set a 1 GB limit on the largest array and return an error as soon as a molecule tried to exceed it. This would be a loose interpretation of a 1 GB limit since the RAM would include smaller arrays, program code, etc. So a max usage of 1.1 GB would be optimistic but 2 GB seems unreasonably large. If testing showed more than 1.25 GB, I would reduce the max allowed array size. I do not know how the actual programmers implemented the limit, but if they are like me the limit is probably 1GB plus a bit. After all, virtual memory is supposed to make life easier for programmers by allowing us some slop compared to the really tight coding necessary in the 1960's. Lawrence [Edit 1 times, last edit by Former Member at Aug 21, 2013 5:17:48 AM] |
||
|
|
Eric_Kaiser
Veteran Cruncher Germany (Hessen) Joined: May 7, 2013 Post Count: 1047 Status: Offline Project Badges:
|
I appreciate a 64-bit version for large jobs. Please improve checkpointing too.
----------------------------------------![]() |
||
|
|
CandymanWCG
Senior Cruncher Romania Joined: Dec 20, 2010 Post Count: 421 Status: Offline Project Badges:
|
hi Mcgruber,
----------------------------------------Here are my 2 cents on the matter: I have also successfully been running CEP2 on a standard (free) RAMDisk partition of just 4GB, with 2 concurrent tasks plus 2 WUs from the other projects. Never ran into trouble. From time to time I would even let S@H and S@H Beta run wild with a WU (a 5th) and still no issues. I hope this helps. Cheers! Knowledge is limited. Imagination encircles the world! - Albert Einstein ![]() |
||
|
|
Mgruben
Advanced Cruncher Joined: May 26, 2013 Post Count: 94 Status: Offline Project Badges:
|
All great information everyone, thanks for the replies.
----------------------------------------I did notice the 2048MB Disk Space and 1024MB RAM space per CEP2 WU under system requirements; I'm fine with allowing (1024x4)MB RAM for 4 concurrent CEP2 WUs, though I had sort of assumed that at least some of the 2048MB Disk Space per WU resided outside of the /slots directory while crunching, and hence I would only need somewhat less than (2048x4)MB RAMdisk for 4 concurrent CEP2 WUs. I guess if I want to cut it close and only allocate an 8GB RAMdisk to hold 4 concurrent CEP2 WUs, that's my call haha, but I've been warned. <aside> I bought 4GB ram when I first built my crunching rig, before I had aspirations of RAMdisking. After I discovered the ability to RAMdisk, I quicky found out that I'd need more RAM sticks if I wanted to run multiple CEP2s, but could only find my RAM flavor in 8GB units. In sum, as it happens I'll only have 12GB RAM unless I want to replace the 3-week-old 4GB RAM stick with an 8GB unit, so it looks like I'll have the bare minimum needed to RAMdisk 4 concurrent CEP2 WUs (4GB RAM, 8GB RAMdisk). I'm sort of ignoring system overhead and other things that might be going on in the system that would tip me over the crucial RAM usage figures, so my plot for 4 CEP2 WUs may be ultimately ill-fated, but we'll see how things work out. </aside> Thanks again all ![]() |
||
|
|
Mgruben
Advanced Cruncher Joined: May 26, 2013 Post Count: 94 Status: Offline Project Badges:
|
I have also successfully been running CEP2 on a standard (free) RAMDisk partition of just 4GB, with 2 concurrent tasks plus 2 WUs from the other projects. Never ran into trouble. Are there any other WUs which use more than a few MBs of disk space while crunching? Every time I check the slots folder size of FAAH or FAAHv or SN2S I usually see maybe 10MB or something, and so I usually ignore these as negligible. Reading your statement, I assume you're tapping 2.2GB or so with the (CEP2)x2 in a worst-case scenario, to which the other WUs add maybe 0.02GB. Given that I don't know exactly the working size max of the CEP2 WUs though, it could also be the case that your CEP2 WUs peg 3.9GB, for a total of maybe 3.95GB conservatively (this seems way too high, though logically still possible). I suppose this could always change in the future though. ![]() |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
My plot for 4 CEP2 WUs may be ultimately ill-fated, but we'll see how things work out. I think you'll be fine. Most of the CEP processes I run seem to hover around 200-300MB.. they may spike here and there but I'd think that the likelihood of exceeding max RAM would be pretty low. However, if you've configured BOINC to consume max 50% while in use (or idle for that matter), you might run into some WU stoppage:) I might be wrong, but I don't think the min. requirements listed actually take those configuration values into consideration. |
||
|
|
Mgruben
Advanced Cruncher Joined: May 26, 2013 Post Count: 94 Status: Offline Project Badges:
|
Update:
----------------------------------------I configured a script to record my RAMdisk's usage (my /slots folder resides on my RAMdisk) every 15 minutes for 10 hours while one CEP2 (and 3 other different WCG Projects) WU was running. The highest I got (unfortunately the WU lasted 11:40, while I was only measure for 10:00 of that) was 1.3GB used. Assuming that this usage mostly if not all attributable to the single CEP2 WU, it looks like 1.3GB may be the functional max, so that 1.5GB is a conservative CEP2 WU estimate while crunching. That said, I'm extrapolating from 40 polling points over a single WU without controlling for the other WCG projects running on my box, so while I'll be running with the 1.5GB number, I caution others against doing so without checking /slot size for yourself. ![]() [Edit 1 times, last edit by Mgruben at Aug 23, 2013 12:52:20 PM] |
||
|
|
Jim1348
Veteran Cruncher USA Joined: Jul 13, 2009 Post Count: 1066 Status: Offline Project Badges:
|
I guess if I want to cut it close and only allocate an 8GB RAMdisk to hold 4 concurrent CEP2 WUs, that's my call haha, but I've been warned. I ran 4 concurrent jobs on a quad-core with a 5120 MB ramdisk for a year without problems. On rare occasion (maybe once a month as I recall) I would get a message that a work unit would not download because of insufficient disk space, but that did not interfere with anything running. This was with 8 GB total memory, so the rest was for the running of the applications. Now I run 7 concurrent jobs on an i7-3770 with a 12 GB ramdisk with no problems or messages (16 GB total memory). If you have problems, I doubt that it will be due to insufficient disk space, though it is of course possible that the work units and their working requirements get larger over time. But I have now bought an additional 16 GB of memory for the new work units and am awaiting their arrival. |
||
|
|
|