| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Completed Research
Forum: The Clean Energy Project - Phase 2 Forum
Thread: Reducing the gap from elapsed time to CPU time |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 37
|
|
| Author |
|
|
Mamajuanauk
Master Cruncher United Kingdom Joined: Dec 15, 2012 Post Count: 1900 Status: Offline Project Badges: 
|
As it says above, I'm working on reducing the gap, which should give more output?
----------------------------------------I'm running CEP2 on my 4P/64th machines with nothing else running. There is always 1 - 1.5 hours difference between elapsed and cpu time, in an attempt to reduce this I've restricted the concurrent threads to 62. I understand the gap is where the thread is waiting for processing time, however, this is an AMD machine with 16 cores per CPU, no hyperthreading, so should have 1:1 with wu:core shouldn't it?. They also have 48Gb of RAM per machine 12Gb per CPU So how can I further reduce this gap? If you need info on the spec of the machine check out this thread
Mamajuanauk is the Name! Crunching is the Game!
  |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Look at any 'bloatware' that is also on the system, in your case unlikely as you built it yourself, but I know I managed to reduce the gap by getting rid, esp on a lappy
 |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
The problem is simple: Way too many to ever achieve an efficiency of 97-98% or better. knreed requested a staggered starting facility for apps such as cep2. The key problem is i/o access bottlenecking and even worse during uploading part _4. He raised a ticket with developers. The ticket remains untouched, unactioned, the 'lets find arguments to not do' approach so often observed.
----------------------------------------How to address: Hit the books and set up multiple clients which can run concurrent, each configured to use a different storage unit. Raid won't do this, might actually worsen the situation. With a provisional september end, based on known work, 3 more months and the project could be gone, which makes the problem go away by itself. [Edit 1 times, last edit by Former Member at Jun 15, 2014 3:30:52 PM] |
||
|
|
Yarensc
Advanced Cruncher USA Joined: Sep 24, 2011 Post Count: 136 Status: Offline Project Badges: 
|
As said above its almost definitely because two (or more) tasks are waiting to use the hard drive at the same time, causing at least one to not be crunching. Even though you have an SSD its still just one disk, so only one task can read/write at once. If you have another drive you could instal it and dedicate it to BOINC (you would probably want to split the tasks among the drives).
Also for the potential upload slowdowns, you can change the number of tasks that can upload at once in cc_config.xml. Never tested this, but if you set it so only one could upload at once it might help your problem a little bit since your disk wouldn't be reading two tasks. <max_file_xfers>x</max_file_xfers> or <max_file_xfers_per_project>x</max_file_xfers_per_project> Lavaflow, I believe the scientists said the September date was just for this set of work and they'll be continuing the project for the foreseeable future. |
||
|
|
captainjack
Advanced Cruncher Joined: Apr 14, 2008 Post Count: 147 Status: Offline Project Badges: 
|
Mamajuanauk,
Let me know if you want any help setting up multiple clients on one machine. I am running multiple clients on Windows 7 and on Ubuntu 14.04. I would be glad to share my setup with you. captainjack |
||
|
|
Mamajuanauk
Master Cruncher United Kingdom Joined: Dec 15, 2012 Post Count: 1900 Status: Offline Project Badges:
|
Look at any 'bloatware' that is also on the system, in your case unlikely as you built it yourself, but I know I managed to reduce the gap by getting rid, esp on a lappy Thanks Alan, I just suspended Boinc and monitored the cpu useage, on one machine (the older build) it was 0%-4% on the other it was only around 0-2% 'bloatware' not likely as I was connecting to the 2 machines via VNC which was probably using at least some of the CPU... However, I do run the standard 'Ubuntu-Desktop' which likely takes up the rest of the CPU, although I don't think I've ever seen an 'Idle' pc runing at zero%... BTW - I'm currently running Ubuntu 12.04 LTS server
Mamajuanauk is the Name! Crunching is the Game!
|
||
|
|
Mamajuanauk
Master Cruncher United Kingdom Joined: Dec 15, 2012 Post Count: 1900 Status: Offline Project Badges:
|
The problem is simple: Way too many to ever achieve an efficiency of 97-98% or better. knreed requested a staggered starting facility for apps such as cep2. The key problem is i/o access bottlenecking and even worse during uploading part _4. He raised a ticket with developers. The ticket remains untouched, unactioned, the 'lets find arguments to not do' approach so often observed. Many thanks Lavaflow, this make some sense...How to address: Hit the books and set up multiple clients which can run concurrent, each configured to use a different storage unit. Raid won't do this, might actually worsen the situation. With a provisional september end, based on known work, 3 more months and the project could be gone, which makes the problem go away by itself. I say 'Some' as CEP2 is known for being the worst project for 'Checkpointing' so while it makes sense, it also conflicts with the known issue of bad checkpointing... However, with little else to go on, I'm going to set one machine to 'No new tasks' and run it down. Then add some additional drives and take CaptainJack up on his offer to 'assist with setting up multiple clients on one machine' This should be a challenge as I don't have much knowledge or experience with Ubuntu...
Mamajuanauk is the Name! Crunching is the Game!
|
||
|
|
branjo
Master Cruncher Slovakia Joined: Jun 29, 2012 Post Count: 1892 Status: Offline Project Badges:
|
4% is huge mamajuana-both-uk-and-blank-one while doing nothing |(i.e. being idle). This is what my Mac OS X uses when I am surfing, listening music, checking e-mails, monitoring PC activities and such stupidities (including running anti virus ON MAC!)
---------------------------------------- IDK if it is some bloatware, malware or simple CEP2ware, but you have definitely to find out what it is, because it is not normal. When I have exclusively run CEP2 on my Win7 rig (i7-3770 octo-thread, 8-giga RAM only, almost full SSD, MS Ess. in background, some GPGPU tasks on the top), its utilization has never dropped under 98-99%. ON WINDOZE! Cheers and  ETA: run MCM (and/or FAAH/FAHV) on affected machine only and check what is the utilization with this sub-project(s)  Crunching@Home since January 13 2000. Shrubbing@Home since January 5 2006  [Edit 4 times, last edit by branjo at Jun 15, 2014 5:27:57 PM] |
||
|
|
Mamajuanauk
Master Cruncher United Kingdom Joined: Dec 15, 2012 Post Count: 1900 Status: Offline Project Badges:
|
Mamajuanauk, Yes, please, but first I have to sort out the additional HDD's, i.e. making then visible and the how to install the boinc client to use the specific disk.Let me know if you want any help setting up multiple clients on one machine. I am running multiple clients on Windows 7 and on Ubuntu 14.04. I would be glad to share my setup with you. captainjack Any guidance is most appreciated...
Mamajuanauk is the Name! Crunching is the Game!
|
||
|
|
Mamajuanauk
Master Cruncher United Kingdom Joined: Dec 15, 2012 Post Count: 1900 Status: Offline Project Badges:
|
4% is huge mamajuana-both-uk-and-blank-one while doing nothing |(i.e. being idle). This is what my Mac OS X uses when I am surfing, listening music, checking e-mails, monitoring PC activities and such stupidities (including running anti virus ON MAC!) IDK if it is some bloatware, malware or simple CEP2ware, but you have definitely to find out what it is, because it is not normal. When I have exclusively run CEP2 on my Win7 rig (i7-3770 octo-thread, 8-giga RAM only, almost full SSD, MS Ess. in background, some GPGPU tasks on the top), its utilization has never dropped under 98-99%. ON WINDOZE! Cheers and ETA: run MCM (and/or FAAH/FAHV) on affected machine only and check what is the utilization with this sub-project(s) Many thanks branjo, with nothing to compare it to, I was unaware... I'll look into it.
Mamajuanauk is the Name! Crunching is the Game!
|
||
|
|
|