| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: The Clean Energy Project - Phase 2 Forum
Thread: To Project Scientists: Clean Energy Project Phase 2 - Limited Cruncher Participation |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 84
|
|
| Author |
|
|
Bearcat
Master Cruncher USA Joined: Jan 6, 2007 Post Count: 2803 Status: Offline Project Badges: 
|
I have two servers with 8 cores each. When more than about 5 or so cores are running and a new CEP2 work units starts it creates some sort of an issue and all the 8 currently running processes are restarted from their last check point. This costs only a few seconds or minutes on most projects, but the CEP2 project can lose several hours, apparently. This was happening so much that I was losing sometimes 20 CPU hours per day, so I had to turn on the restriction to only have one CEP2 work unit at a time. It didn't seem right to waste so much valuable CPU time when it could be used on the other projects. This is a shame. I like the idea behind this project. I have one of my 2 octo's running this project. Using E5420's with 8gb ram. I noticed lost time but didn't dig deep to see if my losses were as big as yours. I did add clean water to mix this hoping not all 8 cores would crunch at the same time. Will have to look in the morning to see. Did you try something like this to reduce the loss? Curious on your 8 core setup to compare to mine to give me an idea of what I missed. I hope I am not losing this much time. Glad I didn't commit my dual quad westmere box. To cleanenergy, if this is the case, is there any way possible to program in to allow only half the avaiable cores to crunch, ie if 8 then allow 4, or something like this? I'm more of a hardware person in IT than software so don't know if this is possible. 
Crunching for humanity since 2007!
 |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
...The fact that CEP2 (both in terms of upload times as well as errors) runs perfectly smooth for some users and makes a lot of trouble for others suggests platform dependent issues. Getting a hold of these is very tough because of the extreme heterogeneity of WCG... Best wishes from Cambridge, MA Your CEP team  I want to let you know that I have 4 PCs running 24x7 with different operating systems.1. One is a Lenovo ThinkPad duo-core using Windows XP. 2. One is an HP desktop duo-core using Vista. 3. One is an HP quad using Vista to be upgraded to Windows 7. 4. One is a Lenovo quad using Windows 7. I have not had any download or upload problems once I got a problem with the DSL line resolved. I have seen the problem with reaching the 12 hour maximum time and everything operates as described in other posts. The ThinkPad appears to be the PC most subject to reaching the 12 hour maximum.  |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
... I did add clean water to mix this hoping not all 8 cores would crunch at the same time. Will have to look in the morning to see. I have CEP2 and HCC selected. Since the project weight was increased I receive nearly exclusive CEP2 WUs. In the last 24 hours there was 1 (one!) HCC WU on two machines. (Sekerob had a similar observation earlier in this thread.) Guess there are too less crunchers with CEP2 marked. So to get the correct balance CEP2 WUs are prioritized over all other checked projects... |
||
|
|
nasher
Veteran Cruncher USA Joined: Dec 2, 2005 Post Count: 1423 Status: Offline Project Badges: 
|
curently i have my 3 computers (all old) crunching 1 per computer. 2 of them have 2 cores but the 3rd is a single core... See told ya they were old
----------------------------------------my newest is running Vista the other two are xp machines. I have found out if i try to add the 2nd processor it gives problems but as long as i am on single CEP-2 per computer i dont have problems. i am happy with this project it is just going to take me a while to get a high level badge and i may decide to stop at gold or ruby depending on how the project looks when i get there. At this time I am looking toward about 1-2 cpu days per day. Thanks for all the updates from the scientists its great to hear from them. the only download/upload problem i have is caused by Comcast... i am sure anyone who uses Comcast has similar problems do to bandwith throttleing or such yes it may be a tough cookie but the reward is great. only critisism i could pass on is that i would like the cut-off time extended to say 16 hours/WU <from 12> i just know some of us who have slower computers could complete the last little bit if we had a little more time... almost all my jobs complete to end of job but there are slower ones now and then that i know are almost there... please keep us up to date... oh one thing i do ask from a badge hunting perspective... when you think the project has say less than 1 year left less than 6 months left can you tell us please (if it goes longer no one will complain) the reason for us who like hunting the badges i do not want to be 363 days crunching (still ruby) and the jobs run out. i would prefer to know when the project end is approching so i can decide if i can make the next level or if i should let others who are also trying to get the next badge level work on there badge. if the project runs long enough i will put in 2+ years into it if not i will try for emerald at 1 year if the project wont make a year i will definatly shoot for gold or ruby but will work on other projects instead. and if you havent seen people grab for every last work unit you havent seen the badge hunters at the end of a project trying to get the next badge level. happy crunching and i look forward to reading your other reports  |
||
|
|
JSYKES
Senior Cruncher Joined: Apr 28, 2007 Post Count: 200 Status: Offline Project Badges:
|
The probability of getting WU's is perplexing - I get CEP2s on 4 no. i7 W7 64 PC's with regularity, I get the odd WU on an older dual core PC, but only irregular WU's on an i7 W7 64 that did Betas for CEP but which does not now seem to receive them!!
----------------------------------------Yes, it is a tough cookie to crunch, and I like the 1 unit at a time routine to avoid too many problems with I/O - (having run multiple units in Beta, it was a bit wild to see the HD led lit almost continuously). The one unit rule does make amassing reasonable tallies of completed units a much longer process - at best I have crunched 6 units in a day....and then waited for another batch to arrive sometimes a couple of days later!! I certainly have not been seeing a unit every log-in, which is a bit frustrating when there are piles of every other job under the sun in cache and not a single CEP..... only recently passed bronze and some way to go for silver - the chance of sapphire at this rate is in dreamland..... Thanks to the scientists for the explanations - it all helps.  |
||
|
|
JSYKES
Senior Cruncher Joined: Apr 28, 2007 Post Count: 200 Status: Offline Project Badges:
|
Dataman's comment about the credit return values got me thinking, with the following (quite crudely derived from total credits received divided by total WU's) results:
----------------------------------------C4CW - 493.8 points/WU CEP2 - 1100.4 points/WU HCMD2 - 650.5 points/WU HFCC - 750.9 points/WU HCC - 351.8 points/WU HPF - 560.9 points/WU FAAH - 640.4 points/WU DDDT2 - 591.8 points/WU IADS - 645.3 points/WU* Beta - 842.8 points/WU NR4W - 991.4 points/WU* HCMD - 724.7 points/WU* GC - 157.8 points/WU* * - Now finished/dormant I thought it made interesting reading especially if you then contrast with the average time to complete each unit....this I'll leave to you as every machine (and many WU's) will be wildly different.... [Edit 1 times, last edit by JSYKES at Dec 2, 2010 9:14:29 PM] |
||
|
|
nanoprobe
Master Cruncher Classified Joined: Aug 29, 2008 Post Count: 2998 Status: Offline Project Badges: 
|
I thought it made interesting reading especially if you then contrast with the average time to complete each unit....this I'll leave to you as every machine (and many WU's) will be wildly different... Wildly different indeed. I divided my CEP2 out and it came to 325.5 point per WU. FWIW, they were all done in Linux which seems to always award less points. The time spent crunching each unit has a lot to do with it. Faster machine, less PPWU. That's my take. YMMV.
In 1969 I took an oath to defend and protect the U S Constitution against all enemies, both foreign and Domestic. There was no expiration date.
----------------------------------------  [Edit 1 times, last edit by nanoprobe at Dec 2, 2010 11:53:36 PM] |
||
|
|
sem@ok.cz
Cruncher Joined: Apr 14, 2006 Post Count: 3 Status: Offline Project Badges:
|
2 cleanenergy: Hello. Just one question to the capacities. If your site is "wide open for business", why it takes so much tries before WU is uploaded. On any of my networks on any of computers, on any of locations - it takes 20-30 tries before WU is uploaded. (locations are not interconnected and does not share same provider) Connectivity on my locations ranging from 4 to 8Mbits/s - most of the time free for use. There are no traffic spikes during the time when BOINC is trying to upload CEP2 units - I checked that several times. I tried to trace couple of Harvard sites and I didn't see any obstacles along the way. Problem is that I got lot of time upload message which ends with HTTP error and after lots and lots of tries, the block goes finally through. If it is uploaded by client itself, it takes usually 1.5-2 days - if I try occasionally by hand, roughly 30 tries. I was thinking about this from different sides, but the only thing that stayed after all was, that there may be some max_concurrent_upload or max_concurrent_upload_request cap set in your system? I hope it will improve a little. I really don't care about uploading big blocks, but it looks problem IS really on server side, which would be understandable with amount of data that is pushed there. I'm quite happy that this project generates a lot less block errors than former CEP, which I considered real nightmare and quit couple of months after start. Sorry for imperfect use of english language, I'm not native speaker. I'm not trying to assault your team position or work, I'm just trying to help improve responsiveness of CEP2 servers, which I feel is not as good as it could be. (I lost also around 15 block which was not uploaded at all. Just several days later I found out, that blocks which server does not "take" automatically could be uploaded by lots of tries by hand - which is, of course, quite unfortunate.) There is other idea, that hit me during writing of this msg. Could U please tell us, (if U have that statistics) how many concurrent uploads is usually running on servers? How many request for upload it has that are not served, let's say per minute or 5 minutes? VTS I am not certain your user community views the additional complexity of third-party servers as a benefit to them. WCG's servers maintain world class standards in the percent of availability. Yes, I guess we haven't explained this topic properly: The decision to buy our own servers and storage arrays was due to the limited capacity of the WCG servers. Since our results are bigger than usual, WCG could have only accommodated an upload of 400 results per day which would have rendered the whole project pointless. Our setup is at least in part modeled after the one of WCG and our research computing department is in close contact with our IBM friends on this. To give you some numbers: our servers have a load capacity of 200MB/sec (with some more in reserve) and the actual load is currently at about 10MB/sec (with the 40% throttle). So we are wide open for business .A positive side effect of uploading directly here to Harvard is that we save the additional traffic of the detour via IBM. . . . Best wishes The CEP Team ---------------------------------------- [Edit 1 times, last edit by sem@ok.cz at Dec 3, 2010 4:25:50 PM] |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Let the stats speak for themselves, 20% 40% 80%, it's obvious that those who wanted a fair share in the mix profile multi-core devise did no buy into the golden goose being funnel fed method and AFAICD, this worked like a boomerang... it did for me... contribution dropped to 74% of 1 core.
----------------------------------------Pretty plz WCG techs, why not a little tête-à-tête with cleanenergy to review and set the weight back down. Those who have no reservations and selected CEP2 as their preferred project will have done so weeks ago and those who want a regular stream and not running for their May-West, will have closed the dam gauge to a 1 core trickle... and I'm sure that was not your objective... but maybe it was ;>)
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
another solution - if it is possible to set only 1 CEP2 task per computer it must be possible to allow for 2 or 3 or any other number of tasks per computer so everybody that wants to will be able to set this in their profile and not to loose crunchers that otherwise will limit their participation to 1 task only
|
||
|
|
|