Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 15
Posts: 15   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 2018 times and has 14 replies Next Thread
Viktors
Former World Community Grid Tech
Joined: Sep 20, 2004
Post Count: 653
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Computers unusable on HPF2 project

There are two reasons we set the limits higher on HPF2. First, the program uses about 3 times the real memory (working set size even though the virtual memory allocation is smaller) compared to HPF1. This leaves quite a bit less for other applications on smaller memory machines. When there is not enough real memory on the machine, and if you try to use the machine for other work which needs more real memory than the total you are using, counting grid work, then paging activity increases and this slows down memory hungry programs. Even with the old minimums for HPF1, we would occasionally get comments from members about it slowing their work. Thus we set the minimum memory higher to give a sufficient margin of available real memory for other work on the machine.

Second, we set the CPU speed requirement higher because the checkpoints on HPF2 can be considerably further apart in time than in HPF1. This means that on much slower machines, this time could be as long as several hours depending on settings, and would likely cause a lot of lost work if the member did not keep his machine running continuously.

Having said all of that, we think some of our future projects will be able to run on the smaller and slower machines... so stay tuned.
[Jun 25, 2006 2:46:37 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Dirk Gently
Senior Cruncher
England
Joined: Mar 1, 2005
Post Count: 153
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Computers unusable on HPF2 project

I think there is a slow subtle shift of emphasis happening in Grid Computing. When I first heard about it, the essential principles were:-

1) The grid task taps into the unused spare computing capacity of a machine while the user is using it anyway.
2) Thousands or even millions of machines as in (1) share the required computing load.
3) Because of (2), the power or degree of availablility of each individual machine does not really matter, it is the SUM that matters.

So, by the original principle, while I am writing this email, my spare computing power (most of it at the moment) is being used to crunch WCG's worthy workunits. When I finish this, if I have no other work to do I shut my PC down, having done my work and made a useful contribution to important medical research at the same time. Most people would find it hard to find reasons NOT to help in this way!

Increasingly, grid computing does not work like this. Many of us leave machines on when not actually using them, or even all the time, because we believe in the causes. Some crunchers are even prepared to upgrade machines (memory etc) to meet requirements. This is fine, because in effect the costs of the extra computing effort becomes a donation to charity. A grid composed entirely of such "Premium" machines would be a grid scientist's dream. It would be so easy to control and use efficiently - like the best supercomputer!

Obviously it is diffcult to program and use a grid according to the strict principles, and in practice compromises and constraints are necessary. But it worries me that the trend may be moving too far. Didactylos mentioned 10 hours as an average completion time (I know that the absolute time limit is much longer). As the limits get tighter, I can see that the cruncher who chooses to contribute using the "strict" principles, may eventually be squeezed out. At the moment, we even have an option to "Do work only when computer is Idle", for those who want reassurance that their computer performance is unnaffected. This option, and the option to shut down a computer may become unusable (even for fast machines) if the power hungry trend continues. The situation gets worse for Laptops - which are on average less powerful and powered up for shorter periods. Laptops are very popular!

I think we should strive therefore to treat the original principles as an important design consideration for WCG workunits, and make them as small as practicable. I know that there are limits on how much a task can be broken down. What about the solution they adopted at Climate Prediction? Here, the workunits were necessarily HUGE, but they had a system of INCREMENTAL reporting and crediting during the processing of a WU.

David Hobbs - Thanks for your contributions, and please do not be discouraged. You might like to look at another worthy project for your "little" machines - http://www.malariacontrol.net/ - I don't know what the min requirements are but as they are in Beta at the moment they are releasing VERY SMALL WORKUNITS which process like lightning.
----------------------------------------
[Jun 25, 2006 9:00:57 AM]   Link   Report threatening or abusive post: please login first  Go to top 
davidhobbs
Senior Cruncher
England
Joined: Dec 30, 2004
Post Count: 151
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Computers unusable on HPF2 project

Thanks Victors for that explanation, and I do understand the point you raise. In my own case, however, I have over 30 computers dedicated solely to running WCG so the issue of conflict with other tasks doesn't arise.

I spent most of yesterday swapping memory sticks between machines to get as many as possible configured suitably for HPF2. I think I will be able to set 15 in action for you today. The remander, all too slow and now with only 128MB RAM I will set to work elsewhere, although I will keep a lookout for future suitable WCG projects.
[Jun 25, 2006 9:58:48 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Computers unusable on HPF2 project

Thanks Victors for that explanation, and I do understand the point you raise. In my own case, however, I have over 30 computers dedicated solely to running WCG so the issue of conflict with other tasks doesn't arise.

I spent most of yesterday swapping memory sticks between machines to get as many as possible configured suitably for HPF2. I think I will be able to set 15 in action for you today. The remander, all too slow and now with only 128MB RAM I will set to work elsewhere, although I will keep a lookout for future suitable WCG projects.
nice work smile
[Jun 25, 2006 12:47:25 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Computers unusable on HPF2 project

united we are.......in squashing the bugs.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Jun 25, 2006 1:07:02 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 15   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread