Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 152
Posts: 152   Pages: 16   [ Previous Page | 1 2 3 4 5 6 7 8 9 10 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 497094 times and has 151 replies Next Thread
rwremote
Cruncher
U.K.
Joined: Aug 27, 2009
Post Count: 36
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

Interesting thread,

@ David_L6 : I too had great reservations about running this project, (see https://secure.worldcommunitygrid.org/forums/wcg/viewthread_thread,30618 ) but have opted-in (clicked the box), left everything at default, and 1 in 4 of my WU's are a CEP2. I checked back after 7 days to ensure everything was OK . . . and it was.

I have no time to be tweaking PC's to the the nth degree to get better efficiency, more points per WU, restarting/removing stuck WU's or whatever. In that sense I think WCG is correct to have this as an "opt in", if only for the bandwidth requirement. I had no idea if my bandwidth service would be up to the task, so one Sunday I devoted 15 minutes to looking at this WCG Project, setting it up and it all appeared to work. - since then I've left it alone . . . .
I would take a guess that 98% of crunchers just go with default values and crunch away quite happily.

I would think that until the BOINC software is more "intelligent" about its host and available resources, then WCG probably are wise to keep this as an op-in.

That said, (there's always one of those . . . . ) If it were not for threads like this, crunchers taking time out to tweak settings, try new devices, report faults and the like, then, I'm absolutely sure that WCG would not run as smoothly as it does.

I take my hat off to all you Tweakers, Scientists, Admins, and CA's, you've made WCG is very nice BOINC project to crunch.
----------------------------------------

[Jan 16, 2011 11:51:26 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Ingleside
Veteran Cruncher
Norway
Joined: Nov 19, 2005
Post Count: 974
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

One thought I had was if boinc could read from one disk and write to another to maybe speed things up, this could reduce the constant reading and writing on just one disk. Anyone know if this is possible? If so, I could have cep2 write to a ssd drive.

Don't know specifically for SSD, but if it shows-up as a "normal" hd, this will be possible, since AFAIK the biggest "bottleneck" is the (unneccessary) expansion of qcaux and it's 278 MB worth of files for each and every task.

Note: For most projects except CEP2, there's no point of moving the slots-directory to a different hd, since appart for stderr.txt that often is empty, the slots-directory often includes only links to the actual files that is residing under the project-directory.

Under Windows, you'll need linkd.exe (or similar). If this is included or not AFAIK depends on windows-version you're using. If assumes BOINC data-directory is c:\programdata\boinc and you'll want to use d:\boinc as the 2nd. hd, it will be like this:

0: You must use NTFS on the hd's.
1: Exit BOINC-client.
2: To guard against problems, it can be an idea to backup the BOINC data-directory initially, in case you'll later-on runs into problems...
3: If neccessary, create d:\boinc
4: Navigate to c:\programdata\boinc, select the "Slots"-directory, and move this directory (including sub-directories) to d:\boinc
5: Open-up a cmd-window (just select start, run, and type cmd (enter)), and type following commands:

c:
cd \programdata\boinc
mkdir slots
linkd slots d:\boinc\slots
dir slots (This should now show the contents of the slots-directory residing under d:\boinc\slots, if it's not, something's wrong...)

Most common error, you'll get a message about linkd isn't a known program. If this is the case, search for linkd on the internet and download a version.

6: When successfully moved & linked slots-directory, uninstall BOINC & re-install BOINC.
7: Startup BOINC (while offline), and verify everything's working as it should...


If your current BOINC data-directory is located anywhere else, just replace as neccessary.
----------------------------------------


"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."
----------------------------------------
[Edit 1 times, last edit by Ingleside at Jan 16, 2011 12:55:33 PM]
[Jan 16, 2011 12:41:17 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

Ingleside,

If this works and provides improved efficiency, great, but we tried similar on Linux to link files x-slot and it failed, corrupting results. knreed commented that it was in contravention of BOINC slot-isolation design.

Looking forward to hands-on reports. Anything that also could reduce the ''used'' and ''reported'' BOINC dir data store could possibly also help the improve the RAMDisk approach.

cheers
[Jan 16, 2011 1:24:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
KerSamson
Master Cruncher
Switzerland
Joined: Jan 29, 2007
Post Count: 1684
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

Hello everybody,
During 4 weeks in November and December 2010, I ran CEP2 only. The performance decreasing was more than significant - around 20% in comparison to HCMD2 or C4W - and the IO rate was really high.
Because I wished to test SSDs, I installed 3 weeks ago SSD for the boincdata directory. At the same time I took benefit of the new options for CEP2. I defined a limit of one WU per host for quad core with conventional HD and of two WUs for hosts using SSD (in this case: two Phenom II x6). Indeed the performance decreasing seems to be less significant with SSD. Nonetheless, even if SSD are used, Windows based hosts generate much more IO than Linux based hosts (Ubuntu 10.04 64 bits server).
The next step will be to limit CEP2 to Linux based hosts only, in order to treat the (expensive) SSDs with care. However, I have still the feeling that the CEP2 performance could (should) be improved to become more efficient and finally to complete more work per CPU hours.
My problem is not the earned credits - since I am crunching only for WCG and no real "competition" could be done against other projects, in particular with those using GPUs - but much more to consider earned credits as a representative performance indicator.
Briefly summarized, the question is not only to win more participants but to use the available computation performance more efficiently.
Cheers,
Yves
---
PS: @hypernova: for my-self, I did not experience many problems with errored WUs (excepted at the project start); are you maybe overclocking your hosts?
----------------------------------------
[Jan 16, 2011 1:44:35 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Hypernova
Master Cruncher
Audaces Fortuna Juvat ! Vaud - Switzerland
Joined: Dec 16, 2008
Post Count: 1908
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

PS: @hypernova: for my-self, I did not experience many problems with errored WUs (excepted at the project start); are you maybe overclocking your hosts?


Indeed Yves. All my hosts are overclocked around 4-4.2 Ghz. They run 980X in HT means 12 thread. CPU's are very well cooled so that core temps stay under 70C. Memory DDR3 is also fast. I use sticks certified from 1600 Mhz up to 2100 Mhz depending on host. But memory is not overclocked. On certain hosts I have 3 sticks of 2Gb (total 6 Gb)and on other hosts 2 sticks of 2 Gb (tot 4Gb). I always run the memory to a speed a little under the certified speed.
----------------------------------------

[Jan 16, 2011 6:10:27 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Bearcat
Master Cruncher
USA
Joined: Jan 6, 2007
Post Count: 2803
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

Ingleside, I think doing it your way would cause issues as sekerob pointed out. Boinc would need to be programed to allow this option to work. Similar to how video editing, ( input to one disk, output to another), kind of like a scratch disk. I think this would work great. I thought about using a ssd but presume it would die allot sooner than normal use. One thing I gave noticed is even crunching one wu along with other projects, it effects the other projects CPU time verses actual time. For now I will crunch other projects but will check here periodically to see if a better solution becomes available.
----------------------------------------
Crunching for humanity since 2007!

[Jan 16, 2011 6:10:29 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

Dear bearcat,
As skgiven wrote, you may see a substantial improvement if you only run a few CEP2 wus simultaneously and fill up your remaining cores with other projects which are less I/O intensive.

Dear Hypernova,
We are sorry to hear about the problems you experienced. We had a few server glitches in the beginning which we fixed back then, so your upload problem may already be ironed out by now. If not, please try our current beta test, where we replace HTTP with HTTPS.
It would be helpful for us and the IBM team is you could specify your ‘simply error out’ (in case it still persists) – maybe there is something we can do about or at least learn from it.

Best wishes,

Your Harvard CEP team
[Jan 17, 2011 5:42:06 PM]   Link   Report threatening or abusive post: please login first  Go to top 
sk..
Master Cruncher
http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif
Joined: Mar 22, 2007
Post Count: 2324
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

For tripple channel RAM to be used in tripple channel mode, either 3 modules or 6 modules need to be installed. Using 2 X 2GB means you are running in dual channel mode which has 33% less memory bandwidth. I'm just saying as having tripple channel improves the IO performance, of importance for CEP2.
[Jan 17, 2011 7:09:06 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Bearcat
Master Cruncher
USA
Joined: Jan 6, 2007
Post Count: 2803
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

Dear bearcat,
As skgiven wrote, you may see a substantial improvement if you only run a few CEP2 wus simultaneously and fill up your remaining cores with other projects which are less I/O intensive.
Best wishes,

Your Harvard CEP team


Thanks for the reply. As I mentioned on my 1st post, on multicore systems (I have 4 core no HT, 8 core no HT and a 8 core w/HT -16 threads), when crunching just one along with other projects, have noticed the CPU time vs actual time grew more on the other tasks so it affects the other projects. I just started using my Mac mini, dual core only, and haven't test cep2 on this one but will see if I get the same kind of results as I do on my other pc's. Would be interesting to see if anyone with a SB chip has this kind of an issue. But it does seem the higher the core count, the larger the difference in CPU time vs actual time. Will let you know how the mini performs. Thanks again.
----------------------------------------
Crunching for humanity since 2007!

[Jan 18, 2011 5:47:47 AM]   Link   Report threatening or abusive post: please login first  Go to top 
sk..
Master Cruncher
http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif
Joined: Mar 22, 2007
Post Count: 2324
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Questions about participation drive for CEP2

While running just one CEP2 task initially caused other tasks on an 8core or 16core system to slow down due to the CPU time - actual time delta, it should be less now than it was when the project was first released (due to some I/O fixes). It's also likely to be less if you use tripple channel RAM, Linux and/or a fast hard drive system but dependent on what other projects you are running. HCC has been reported by some to be the best match for CEP2. I don't know how this would scale, for running multiple CEP2 tasks but I'm sure it would depend on the amount of RAM you have as well as the memory bandwidth and disk r/w speeds.

Have you noticed any improvements lately?
[Jan 18, 2011 7:30:06 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 152   Pages: 16   [ Previous Page | 1 2 3 4 5 6 7 8 9 10 | Next Page ]
[ Jump to Last Post ]
Post new Thread