| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: Help Cure Muscular Dystrophy - Phase 2 Forum
Thread: HCMD2 disk space requirements |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 19
|
|
| Author |
|
|
martin64
Senior Cruncher Germany Joined: May 11, 2009 Post Count: 445 Status: Offline Project Badges: 
|
It looks like there is a swap of a bit more than 100 MB on sdb5. I did a swapoff, and it increased the available memory by exactly that size, so it looks like the swap file is in memory rather than on disk. It can't be on the memory stick anyway, as the space there is too limited and there is no reduction in available space. If you want a df report, I can do it next week.
----------------------------------------I guess swap space is kind of mandatory in linux, and therefore a bit of the 2 gig gets taken away from available memory, just to give it back as swap space. Sounds silly. I still wonder why there should be any restrictions on swap space for the WCG projects. If there is a computer with zillions of Gigabytes of memory, and the owner decides this is enough, don't need and allow to swap, then he cannot crunch on WCG? [edit]I did an experiment on my windows machine: set the available disk space for boinc to the current usage + a bit more than 150 MB. Here is what happens: 22.05.2010 09:15:00 World Community Grid Message from server: Help Cure Muscular Dystrophy - Phase 2 needs 243.14MB more disk space. etc. Plenty of swap space, it's just the disk limit.[/edit] Regards, Martin  [Edit 2 times, last edit by martin64 at May 22, 2010 7:35:55 AM] |
||
|
|
Marc Andre Wyss
Advanced Cruncher Switzerland Joined: Jun 4, 2009 Post Count: 57 Status: Offline Project Badges: 
|
Swap is not mandatory for Linux. Let's say you have very less memory (compaired to the system needs), then a swap partition can help to run the system (instead of "crashing" it), but it will be horrible slow, because the systems moves the memories content to the swap and vice versa. This can lead into a "deadlock", when the system uses all it's resources to swap in and swap out, because all other processes are blocked meanwhile. In this case you should upgrade your memory.
----------------------------------------Normally a swap partition is a good solution to handle memory peaks, but it is a bad solution to increase your memory constantly. On some systems the swap partition is used for hibernation too i.e. Ubuntu. Still have to check that Linux USB stick builder. It is possible that the builder creates a swap partition on the disk, but that would be a very slow solution (,because most sticks are very slow compared to harddisks). On the other hand, a swap partition in the memory itself is theoretically possible, but total crap. ---------------------------------------- [Edit 1 times, last edit by Marc Andre Wyss at May 22, 2010 7:19:12 AM] |
||
|
|
Marc Andre Wyss
Advanced Cruncher Switzerland Joined: Jun 4, 2009 Post Count: 57 Status: Offline Project Badges:
|
I guess one solution is to use a lighter Linux distribution than Knoppix for the stick...
---------------------------------------- |
||
|
|
martin64
Senior Cruncher Germany Joined: May 11, 2009 Post Count: 445 Status: Offline Project Badges:
|
I guess one solution is to use a lighter Linux distribution than Knoppix for the stick... Yes, there are a lot of potential solutions to that: I can use my larger, newer stick - however, there is the issue with the limited numbers of writes, so I don't care about that with the old 1Gig stick(s), but I do for the newer ones. I could put a smaller linux on the stick, but knoppix is so convenient to crunch and browse. Or WCG could, if the memory requirements in the tables are correct, apply those numbers to the servers.  So the issue is not so much how I get a solution (I can crunch HCC, anyway), but I would like to understand the reason for the mismatch between the official disk space requirements and the server responses. Why are there official minimum requirements for each of the projects if they do not apply? Regards, Martin |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Is LAIM (Leave Application in Memory when pre-empted) On or Off? When Off switch it On and try again, with Swap-file off of course.
----------------------------------------Presume this to be a diskless system. "Looks like" have you actually inspected the usb stick for a dead file of the old swap file? Having my reservations whether it resided in RAM. Several solutions are on offer. See for instance the total BOINC included out of the box Dotch/UX which is also listed in the FAQ index. There was another one that created a RAMDISK (DCLinux) and only stored data every 6 hours and whenever the jobs finished i.e. very low usb mem write: http://www.worldcommunitygrid.org/forums/wcg/...ead,29073_offset,0#279890
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
martin64
Senior Cruncher Germany Joined: May 11, 2009 Post Count: 445 Status: Offline Project Badges:
|
Is LAIM (Leave Application in Memory when pre-empted) On or Off? I tried both - no change. Presume this to be a diskless system. "Looks like" have you actually inspected the usb stick for a dead file of the old swap file? Having my reservations whether it resided in RAM. There is no swap on the USB. Remember, when I do swapoff, the amount of physical memory available increases by the size of the swap file, which I take as an indication that it resides in RAM. Forget about a solution for my "problem" - I will find one. It's more that I would like to understand the memory requirements philosophy in WCG and the difference between the requirements stated on the website and the ones the servers seem to apply. Regards, Martin |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
On the official SR page it used to be that "Memory" was qualified as the sum of what the RAM+Swapfile, suspecting there to be an input error given that HCC with swapfile on is not using anywhere near 250Mb. From observation with swapfile on, a max of 57Mb ram and 94Mb VM were used, so the SR spec is with a big safety margin. (Old HCC version, not having checked what it does with v 6.08)
----------------------------------------Dual core btw, you need to double the requirements of course. Try setting it to 1 core for the test. Anyway, given you qualified it as "my problem", we will forward on treat it as such. have a nice weekend.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
martin64
Senior Cruncher Germany Joined: May 11, 2009 Post Count: 445 Status: Offline Project Badges:
|
On the official SR page it used to be that "Memory" was qualified as the sum of what the RAM+Swapfile, suspecting there to be an input error given that HCC with swapfile on is not using anywhere near 250Mb. From observation with swapfile on, a max of 57Mb ram and 94Mb VM were used, so the SR spec is with a big safety margin. Disk space was the issue, not memory. The servers didn't complain about the memory, and 2 Gig should be enough. Of course the total memory required is physical + swap, but the server complaint is about the lack of disk space. Regards, Martin |
||
|
|
Ingleside
Veteran Cruncher Norway Joined: Nov 19, 2005 Post Count: 974 Status: Offline Project Badges: 
|
It's more that I would like to understand the memory requirements philosophy in WCG and the difference between the requirements stated on the website and the ones the servers seem to apply. The system requirement-page being wrong is either due to bad input in the 1st. place, for a couple of the numbers it can actually look like they've switched disk & memory. Or, after release some wu's has generated lots of logging-info for one reason and another, something that has used more disk-space than expected, and to accomodate this the requirement on wu-generation has been increased, but the system requirement-page hasn't been updated with the new numbers. In any case, the requirements of work is currently: 300 disk - 120 memory - aids 48 disk - 75 memory - hcc 400 disk - 64 memory - hcmd2 300 disk - 120 memory - hfcc 191 disk - 172 memory - hpf2 All numbers is MB, and is rounded-up to whole MB.  "I make so many mistakes. But then just think of all the mistakes I don't make, although I might." |
||
|
|
|