| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: The Clean Energy Project - Phase 2 Forum
Thread: Questions about participation drive for CEP2 |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 152
|
|
| Author |
|
|
martianmoons
Cruncher USA Joined: Nov 29, 2006 Post Count: 49 Status: Offline Project Badges: 
|
I just started Clean Energy up again after about a 2-year absence from WCG, because I read CE needed more recruits.
I subsequently successfully set up a 6-core AMD with 4GB ram, a 4-core hyperthreaded i-7 (8 effective cores) with 4GB ram, and a dual-core at work with 3GB ram. I am running all cores. So far no problems, it has all been "set and forget" for me. (Have received a few "inclusive" results, but assume that will work out and they will be validated. But I have not had to tend my machines.) |
||
|
|
Bearcat
Master Cruncher USA Joined: Jan 6, 2007 Post Count: 2803 Status: Offline Project Badges: 
|
The opt in and selecting the number to crunch was a good start. But I moved on to other projects after 1 year due to the differences between CPU time and actual time crunched being so large on a multicore system. Very wasteful. I can see 15 minutes or less but when I started to see around an hour difference, that was enough for me. This was on harpertown and westmere processors using 7200 RPM hard drives and plenty of ram. As was mentioned, this is a high IO project. I gave it a shot. Sorry, but unless these wu's could maybe be broke in half or some other way so not to waste so much time, I would gladly come back.
----------------------------------------
Crunching for humanity since 2007!
 |
||
|
|
sk..
Master Cruncher http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif Joined: Mar 22, 2007 Post Count: 2324 Status: Offline Project Badges: 
|
martianmoons, I'm sure the team appreciate your additional 16 tasks at a time contribution. If more people chipped in it would be a big boost to the project. I will slowly increase my contribution to this project to about 10tasks at a time 24/7 as and when I reach my other targets and finish upgrades. It would be more if I could afford additional RAM, but I have already spent >£100 on RAM upgrades alone this month.
----------------------------------------Bearcat, when many of these tasks are run together the IO issues can become a problem, but this issue is not as bad when only running a few tasks per high end system. So you could configure your systems to just run one or two CEP2 tasks each, and still contribute without big losses due to IO issues - some of which have already been dealt with by keeping the task in RAM (I think it's less of a problem now than it was initially). Anyway, a mixed bag of tasks performs better overall (they require different calculations and therefore reduce cache latency and can better expliot HT). Perhaps a TimeLine of this projects changes would be a useful reference; to highlight project changes and how this effects crunching. Lots of people are just not aware that some of the initial problems have been dealt with, at least to some extent. Would it be possible for WCG to assess bandwidth capable systems on a CPU/RAM ratio basis and move this to a non semi opt-in only project? For those systems capable, automatically allocate a low task number for the non-thread reading masses who have selected to run new projects. Should their failure rate increase, reduce the number - right down to zero if need be. [Edit 3 times, last edit by skgiven at Jan 14, 2011 12:41:20 PM] |
||
|
|
David_L6
Senior Cruncher USA Joined: Aug 24, 2006 Post Count: 296 Status: Offline Project Badges:
|
David_L6, Have you got any new observations beyond repeating long expired experiences you had in a previous decade? It would help to know your specifics so that if fixable and significant, they can be addressed by the techs and the Harvard team. FAIK the only ''regular'' problem left and being tackled in a current beta is the upload issue. Beyond that, yes this project is demanding and therefor it's opt-in. cheers PS: One member quite recently reported to run CEP2 on a P4 with HT... 2 concurrently, whilst his spouse is using the computer, without complaints reaching his ears... what more testimony does one need? And your post was helpful in what way? My post was to let the CEP2 scientists know why I am not crunching this project and a possible reason that others aren't since they are looking to increase participation. ** **edited to appropriate forum language**TKH  [Edit 1 times, last edit by TKH at Jan 17, 2011 1:28:53 PM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Plz refer to the reply by cleanenergy, re-quoted below which in different wording appears to concur with mine.
And please refrain from acronyms as just used. thx Hi David_L6, Many of the early problems of CEP2 got ironed out by now (and there is no software overlap with CEP1). So you may want to give it another shot - if it doesn't work out for you it's perfectly fine to donate your time to other projects. As SekeRob said - we appreciate all reports of concrete problems, so that we can improve the project. Best wishes, Your Harvard CEP team |
||
|
|
sk..
Master Cruncher http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif Joined: Mar 22, 2007 Post Count: 2324 Status: Offline Project Badges:
|
Inherent in the problem of increasing participation of infrequent thread readers is their absence of knowledge.
A concise, non-acronymic list of project related changes and adaptations from the original release might quell such vexed anger, and facilitate improved participation amongst this group. For example: Started to use more RAM during runtime - Date. Effect on project/tasks - possible reduction in participation due to increase in RAM requirements, reduced I/O times and related task failures. Effect on crunchers, less system slowdowns and less failures, slightly more RAM required, slight improvement in performance. Addition of options to select number of tasks to run - Date. Effect on project/tasks - additional work for WCG, increase in participation and increase on server demands. Effect on crunchers, allows more crunchers to participate, slight improvement on task times for multiple project crunchers. Addition of option to participate and ignore bandwidth limitations - Date. Effect on project/tasks - increased participation, increased server demands, possible increase in average return times. Effect on crunchers, more people can participate, potential slowdown in Internet. Future plans - approximate dates. Example, Increase average RAM usage - over 3 years. Additional notes, might be offset by task allocation using computer matching or other methods. ... Good Luck  |
||
|
|
Bearcat
Master Cruncher USA Joined: Jan 6, 2007 Post Count: 2803 Status: Offline Project Badges:
|
Bearcat, when many of these tasks are run together the IO issues can become a problem, but this issue is not as bad when only running a few tasks per high end system. So you could configure your systems to just run one or two CEP2 tasks each, and still contribute without big losses due to IO issues - some of which have already been dealt with by keeping the task in RAM (I think it's less of a problem now than it was initially). Anyway, a mixed bag of tasks performs better overall (they require different calculations and therefore reduce cache latency and can better expliot HT). I did try reducing to one on all my multi core machines. I would check each periodically to see any changes. Seems the higher the cores, the larger the difference with just one of these along with other projects. One thought I had was if boinc could read from one disk and write to another to maybe speed things up, this could reduce the constant reading and writing on just one disk. Anyone know if this is possible? If so, I could have cep2 write to a ssd drive.
Crunching for humanity since 2007!
|
||
|
|
Hypernova
Master Cruncher Audaces Fortuna Juvat ! Vaud - Switzerland Joined: Dec 16, 2008 Post Count: 1908 Status: Offline Project Badges:
|
I did crunch exactly 3 years and 139 days of runtime. I recognize that this is the lowest crunching effort I have devoted to a project (without a limitation due to the ending of the project). I had planned at least a minimum of 5 years.
----------------------------------------The reason is that many of my devices would simply error out all the CEP2 WU's that I received. When the CEP2 started on Windows I opted-in all my machines. I have three days of cache. When the CEP2 WU's started to be crunched I had in the cache 3 days worth of WU's. But this erratic erroring disrupted the flow of crunching because after a certain number of repeated errors the server would block saying that a daily limit was reached and refusing to send uny new WU until the next day. The corresponding machine then was sitting idle. I had to test each device (I have 20) and see what was the behaviour before allowing it to run this project. This is time consuming, finally about five or six machines would run CEP2 without too much trouble, except that very often the uplink would block and be deferred, ending with hundred of megabytes waiting to be uploaded, which I did many times manually, so finally I gave up. I will give it a try again next week when a new machine I am building will start crunching. I will post the result.  |
||
|
|
Hypernova
Master Cruncher Audaces Fortuna Juvat ! Vaud - Switzerland Joined: Dec 16, 2008 Post Count: 1908 Status: Offline Project Badges:
|
I checked the numbers for CEP2 and indeed they are very low.
----------------------------------------The average across all active projects (excluded DDDT2 which I do not consider an active project) if we look at the results returned per day is of 71'480. CEP2 stands at 8'813 as an average which is just 12.3 % of what could be expected. There is indeed a very large gap. The best way is to look at the success of HCC with an average of 109'072 (152%) and try to design the CEP2 WU to compare with the HCC WU's. I know that the projects are very different but with HCC you have the recipe of success. And to be frank HCC has run an all my machines without a hitch. As a result I crunched 27 years runtime for this project and still counting. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hypernova,
Set the associated device profile "custom" part to 1 - Default, so your client will only cache 1, and run mix with HCMD2 on the side which I've found produces the highest efficiency on my quad. Then increase the number to 2 and slowly increment with each completed / successful task... 3-4-5. With a 1.2 day cache and good intermixing, the Quad never seems to be running more than 2 CEP2 jobs at any one time, and that gives about 95% efficiency for CEP2 and HCMD2 near max. The Cache setting for CEP2 I have now at 4 (5 cached was too much over longer time), then only one is fetched when one is reported, and in between all other sciences selected get fetched to achieve the optimal spacing, to include that rarely there being CEP2 tasks running concurrently in the same progress phase. This is a curse to many tweakers, but I run stock, and no GPU crunching on side, the success rate continues to be 99.999% for this science on duo (profile set to 3, with 1.3 days cache) and quad. -- SekeRob |
||
|
|
|