Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
![]() |
World Community Grid Forums
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
No member browsing this thread |
Thread Status: Active Total posts in this thread: 156
|
![]() |
Author |
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
We're not talking the lost time, but the reason described and long discussed of the time that goes towards IO and why WCG changed the default work distribution for CEP2: https://secure.worldcommunitygrid.org/forums/...ad,30161_offset,50#300323
----------------------------------------
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
sk..
Master Cruncher http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif Joined: Mar 22, 2007 Post Count: 2324 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Well that explains it; you are discussing something about USB IO and why the WCG changed the default work distribution for CEP2 in the Poor crunching on this project thread, instead of in your link to the Clean Energy Project Phase 2 Scheduling change or one of the many other similar or related threads that spawned from other CEP2 topics - not that I stick rigidly to topics, especially those with such poorly named titles.
|
||
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
No, I'm talking about the 30-45 minutes most everyone experiences with CEP2 on Linux. The USB IO is another fish to fry... covered in the next post, nearly ready to post... waiting on completion of more C4CW tasks and an HFCC.
----------------------------------------Sticking to the topic would be a major breath of fresh air... this one is about poor crunching (efficiency).
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
sk..
Master Cruncher http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif Joined: Mar 22, 2007 Post Count: 2324 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
OK, I will try to confine things to IO slowness, observed by most people for all tasks and especially CEP2 by some. I wish I still had my RAM drive to test this. Mind you the OCZ 80GB RevoDrive PCI-E SSD (OCZSSDPX-1RVD0080) would be much faster than the one I had - better chance of noticing any difference. If I had a 12 thread system and wanted just to crunch this project I might want one or two of these; could possibly save 12h runtime per day.
I guess triple channel RAM systems should fare better than dual channel, so it would be interesting to compare an i7 LGA 1366’s losses to an i7 LGA 1156. Using a different hard drive for Virtual Memory would be better than using your primary drive and RAID systems should also be slightly faster. Is there any way of creating a Virtual Drive from system memory? Could you setup a VM only in RAM and run this solely from the VM? Some Linux versions allow you to run them without installing them, would be interesting to see how these perform (if it is possible to run Boinc this way). |
||
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Intermediate data is building... I'm hopeful to bring all sciences back to the 97-99%+ and CEP2 to 95% instead of 90-92%. The WTD extending helped already quite allot.
----------------------------------------VM and RAM disk are from what I've read no big issue.... you may want to research that and add that to the pool of knowledge... substantially, something needs to back up what's in there on a periodic basis or booting becomes quite costly in progress loss... even for 24/7 crunchers. Though all avenues on the swap file angle has not paid any dividend for crunching, it has made my quad system more user-responsive not having any, but that is a Linux thing. For Windows a VM is almost a must (W7 and maybe Vista and earlier) since MS uses strategies such as writing portions of RAM retained data to VM, so it can literally unmap it from RAM instantenously and point it to the VM mirror when RAM is actually needed for something else.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
sk..
Master Cruncher http://s17.rimg.info/ccb5d62bd3e856cc0d1df9b0ee2f7f6a.gif Joined: Mar 22, 2007 Post Count: 2324 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Sorry, I meant Virtual Machine, not Virtual Memory.
In Windows, if you have a high amount of memory, if you match your Virtual Memory to physical memory, it should speed things up. |
||
|
X-Files 27
Senior Cruncher Canada Joined: May 21, 2007 Post Count: 391 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Intermediate data is building... I'm hopeful to bring all sciences back to the 97-99%+ and CEP2 to 95% instead of 90-92%. The WTD extending helped already quite allot. My CEP2 usage: I7 with 6GB = 95%-97% Q6600 with 2GB = 90-92% Sek, what's the amount ram on your rig? With your numbers and mine it seems more ram means higher efficiency. ![]() ![]() |
||
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Usable by BOINC is 2.9GB on my Q6600. It's using 1.4 kind of constantly. The USB stick has all effectively 6.3GB allocated to BOINC of which 20% is used according BOINC stats tab... 1.27GB
----------------------------------------More RAM for crunching has always been a speedupper. 90-92% pretty much is dictated by the disk IO and that's what I'm working to eliminate as much as possible... will doc a USB mem-stick trick that may help many. Not heard back of that BFS kernel patch if it did anything for RaymondFO.
WCG
----------------------------------------Please help to make the Forums an enjoyable experience for All! [Edit 1 times, last edit by Sekerob at Oct 28, 2010 6:15:28 PM] |
||
|
codes
Advanced Cruncher Joined: Oct 20, 2009 Post Count: 142 Status: Offline |
Some more "sysstat" performance data. No significant disk I/O bottlenecks detected.
Last CEP2 job completed on this device: CPU time: 12 Hours Elapsed time: 12.4 Hours <result> <name>E200486_431_A.25.C21H13NOS2.127.3.set1d06_0</name> <final_cpu_time>43200.006800</final_cpu_time> <final_elapsed_time>44937.790886</final_elapsed_time> <exit_status>0</exit_status> <state>5</state> <platform>i686-pc-linux-gnu</platform> <version_num>619</version_num> <stderr_out> Job starts: [07:26:42] Starting job 0,CPU time has been restored to 0.000000. [07:29:07] Starting job 1,CPU time has been restored to 116.610272. [07:36:38] Starting job 2,CPU time has been restored to 537.081350. [10:47:09] Starting job 3,CPU time has been restored to 11198.435577. [10:55:36] Starting job 4,CPU time has been restored to 11667.995193. [11:03:05] Starting job 5,CPU time has been restored to 12091.264846. [11:10:58] Starting job 6,CPU time has been restored to 12535.115370. [11:18:36] Starting job 7,CPU time has been restored to 12967.189684. [11:29:58] Starting job 8,CPU time has been restored to 13621.074278. [11:37:24] Starting job 9,CPU time has been restored to 14046.812555. [11:45:53] Starting job 10,CPU time has been restored to 14528.093389. [12:06:55] Starting job 11,CPU time has been restored to 15741.876865. [12:17:13] Starting job 12,CPU time has been restored to 16320.548893. [13:06:41] Starting job 13,CPU time has been restored to 19164.929481. [15:30:07] Starting job 14,CPU time has been restored to 27527.013251. [17:39:39] Starting job 15,CPU time has been restored to 35087.966811. Killing job because cpu time has been exceeded. Subjob start time = -266494131, Subjob current time = 1088496126 [19:55:25] Finished Job #15 Performance log entries (same time or very close to same) for job start times listed above: # sar -u 07:17:01 AM CPU %user %nice %system %iowait %steal %idle The below averages are for all the performance recorded entries since midnight: 08:07:01 PM CPU %user %nice %system %iowait %steal %idle Swap pages in/out: # sar -W 08:07:02 PM pswpin/s pswpout/s Disk I/O performance: # sar -d 08:07:02 PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util Memory performance: # sar -r 08:07:02 PM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit Interesting, %memused (68.82%) for all system/user processes is less than the 75% max memory setting in the BOINC Manager while in use. The CEP2 system requirements say "1,024 MB Memory Available". I wonder if I reduce system RAM from 2 GB to 1.5 GB what effect that would have on SWAP usage and any increased disk I/O activity bottleneck. |
||
|
mikaok
Senior Cruncher Finland Joined: Aug 8, 2006 Post Count: 489 Status: Offline Project Badges: ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
How come I only can have one CEP2 wu in progress at a time? Server refuses to send more, although when the one in progress finishes, the next one is downloaded
----------------------------------------![]()
to infinity and beyond
![]() |
||
|
|
![]() |