| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Suggestions / Feedback
Thread: Memory Allocation |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 28
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hi Thomas W. Fry, Don't expect them to shrink much. I read a recent prediction that all the molecular docking programs will switch to all-atom models by 2010. wonderful, 2010 is only, what, 4 years away? By then we'll all have several TB of memory and solid state drives, so 300MB or even 3GB of mem usage wont be a big deal. but since we're living in the now of 2006 where MOST normal computers only have between 128 and 512MB of memory, 300MB is serious overkill and is going to hinder adoption. Many of my friends/colleagues who are running UD have asked me if they should switch, but i've told them all to hold off unless they've got at least a gig of mem... in my opinion there should be 2 versions or modes: 1) for geeks with massive system memory (not including Pagefile size of course) allow them to burn up 300mb pagefile + 50-100 physical. 2) for the rest of the world, keep that memory footprint down < 50MB, however it must be done... i dont care if we're crunching algorithms to create planets, to every problem there are ALWAYS several solution... ok so maybe the WCG coders aren't getting paid a lot of money (if anything) and nobody is paying to use the software, so there probably isn't a huge motivation to fix this.... not to mention 99.999% of WCG users arent computer savvy and naturally won't complain about this because they 1) have NO clue how much memory they have 2) confuse memory with disk space .. "oh but 300MB is nothing, i have 40GB!" 3) have no clue WCG is consuming so much of their system resources etc etc Ignorance is bliss? |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
The minimum requirement is 120 MB of physical memory.
You need to trust your OS to handle the paging elegantly. Nearly all of my team members have had no problems. That's a lot of geeks happily crunching away. When problems with paging are noticable, it is usually WCG highlighting an existing problem. For example, a fragmented disc will cause noticable thrashing. Unfortunately, the science won't wait for computing technology. Grid technology allows more computing power than ever before to be brought to bear, but problems have other constraints. I don't understand your UD comment. And remember, some problems have NO solutions. CS101 is -> that way. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
hi, "ignorant cruncher" here...
---------------------------------------- 1. i've never had a noticeable problem on any of 3 machines in this regard. 2. it's my understanding windows will automatically adjust the memory. 3. i was with UD (grid.org) before, it turned into a disaster, i just had another look at the forums there and it is still a disaster. [Edit 1 times, last edit by Former Member at Feb 15, 2006 5:34:37 AM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
The minimum requirement is 120 MB of physical memory. most people i believe will interpret that as: your PC should have at least 120MB of memory, period, i.e. not 120MB free. You need to trust your OS to handle the paging elegantly. Nearly all of my team members have had no problems. That's a lot of geeks happily crunching away. When problems with paging are noticable, it is usually WCG highlighting an existing problem. For example, a fragmented disc will cause noticable thrashing. can you seriously trust Windows to do anything elegantly? How many non-techies do you know who run a disk defrag tool every month? I know some, but only because i put it in their scheduled tasks  Unfortunately, the science won't wait for computing technology. Grid technology allows more computing power than ever before to be brought to bear, but problems have other constraints. good point, constraints are just another part of life and science won't wait, but i still believe something can be done to make the app not suck up so much mem  Don't get me wrong here, im not complaining about the technology or lack thereof, im complaining about the implementation not taking into consideration the average user with 128-256MB of mem. I'd happily run the app on a whole buncha boxes, but alas due to exhorbant footprints and mem usage i'll have to keep it off and not recommend people switch from UD unless they've got lots of mem to spare.I don't understand your UD comment. And remember, some problems have NO solutions. CS101 is -> that way. sorry musta skipped that class  and i can't seriously believe this problem couldn't be solved. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hi glenneroo.com,
The minimum requirement is 120 MB of physical memory. most people i believe will interpret that as: your PC should have at least 120MB of memory, period, i.e. not 120MB free. Most people will be right, then. 120MB is the minimum requirement for HPF, which uses about 25 MB of RAM. FAAH uses about 96 MB at most, so the minimum requirement is for 250 MB of RAM. Earlier, you said that you thought that most people would think that the 300 MB of Virtual Memory used by FAAH would use the hard disk. Once again, most people would be right. Virtual Memory DOES use the hard disk.  I think that there is something subtly wrong in the way you are thinking about this, because you keep stating facts as though they are popular misconceptions. |
||
|
|
Alther
Former World Community Grid Tech United States of America Joined: Sep 30, 2004 Post Count: 414 Status: Offline Project Badges: 
|
Science applications tend to use a lot of RAM. Note, however, there is a big difference between actual RAM used and virtual RAM. This is not to say that we take this issue lightly. The virtual RAM requirements of these apps can cause low RAM computers to slow down significantly due to paging. On most operating systems (including Windows and Linux), the OS will see that there is some allocated VM that hasn't been touched and will permanently page this out. At this point, the additional VM footprint has no more impact on the computer (other than taking up paging space on the hard drive).
----------------------------------------Before you start making blanket assumptions that we're all idiots here and have no clue what we're doing, you really should step back and think about where these science applications come from. Many science apps (Rosetta and AutoDock included) have been around for many years and often were originally programmed in FORTRAN. Over the years, these programs typically aren't rewritten from scratch, but simply evolve over time. Also remember that before these were put on the grid, these were running happily on supercomputers or clusters where memory was not an issue and the scientists were getting the results they wanted. The reason you see a large difference between the virutal memory and the real memory these apps use is a direct result of their FORTRAN legacy. FORTRAN didn't allow dynamic memory allocation (easily) and it was easier to simply allocate all you might need at the beginning of the application. To ease the transition to C/C++, these static memory allocations were kept in. Is there a solution to the large VM utilization? Of course there is. Go through the code and change all static memory allocations to dynamic ones. It's not as simple as you think and takes a lot of work. As always, it's a tradeoff between the time and effort it takes to do this and the expected benefit from it. Our mission is not to rewrite each science application, which would take months. Our mission is to board their applicaiton and make the necessary changes to make it run on the grid. In the process of boarding, if we see some quick changes to help this situation, then we'll do them and hand them back to the sponsoring organization.
Rick Alther
Former World Community Grid Developer |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
i wasnt calling anyone an idiot, i know in order to write an app of this complexity must require some serious brainwork, not just any VB/html monkey could put WCG together. i too write code for a living which is exactly why i wasnt accepting "thats just the way it is", "it cant be fixed" or "just deal with it" type responses.
but i thank you dearly Alther for the detailed information.. finally someone with info as to WHY the memory allocation is as it is! :) i knew there was a solution, however complicated and time-consuming it may be, it is possible and maybe someday will enable the rest of the non geek crowd to join in the fun, regardless of how much RAM (virtual or physical) we've got  seriousness aside, static -> dynamic allocations in C++ usually requires just using new and delete keywords and if you port the app to C# you can do away with the deletes  hehe not saying its just as easy as that, i'd guess the port aint pretty and has nasty weeds coming out of all the cracks.... and heeey there hasnt been a new WCG app version in quite some time, those code monkeys should be dilligently workin on allocating some of that there RAM a bit more conservatively for the next release |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
As I recall, the FAAH version of AutoDock was updated very recently, and the techs had managed to trim the VM requirements slightly.
I know there is a Rosetta project, focussed solely on improving the software - is there a similar AutoDock project? |
||
|
|
|