| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
| World Community Grid Forums
|
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 11
|
|
| Author |
|
|
jsobry@gmail.com
Cruncher Canada Joined: Jan 11, 2008 Post Count: 31 Status: Offline Project Badges:
|
Running with 2 GB of real storage and ample free storage available I find wgc_hpf2_rosetta_6.03_windows_intelx86 ... paging excessively as reported by Windows Task Manager under Processes. I see PF Deltas of 3000+ and the task is the highest pager after Windows explorer.
----------------------------------------Real memory useage and peak memory useage seem normal compared to other tasks. Even VM size seems normal. I'm running two BOINC tasks simultaneously and the other task hardly pages at all . I also allow up tp 50% of memory usage and keep the tasks in storage while idle. Heavy paging consumes cpu unnecessarily and it could be used by the task itself. What is wrong with this task? [Edit 1 times, last edit by jsobry at Oct 2, 2008 10:15:02 PM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hello jsobry,
We probably won't be able to give you an explanation. Last year Uplinger spent a lot of time on HCC paging and finally tracked the problem down to interactions between several different standard libraries - not the application code but the standard functions called by it. After a lot of experimentation, he was able to select functions that did the job without all the page faults. The growing OS bloat is starting to impact the application programs compiled to run on these over-complicated monsters. At least, that is my opinion. I am still steamed by an article I read earlier this evening praising Microsoft's upcoming OS developments, which faulted users for still using 32-bit systems with only 3 GB of memory. 'Of course it seems slow' the article stated. It is all the fault of the users. Argh-h-h, Lawrence |
||
|
|
jsobry@gmail.com
Cruncher Canada Joined: Jan 11, 2008 Post Count: 31 Status: Offline Project Badges:
|
I may have jumped the gun a little here. Of course the lack of proper and valuable information about BG's OS does not help. Not even their monitor provides the really valuable data one needs to properly evaluate a system's performance.
Depending on how they implement the Windows paging algorithm this may not be as bad as it seems. I claimed that the system was paging heavily where in fact I cannot monitor paging activity as such, I can only see the page fault rate. They may use an algorithm that flags pages to be paged out at the next cycle should a shortage of real storage develop. The real storage manager (RSM) then only turns on a flag indicating that this page should be paged out. This marks the page as invalid. However the page is not actually paged out and moved to disk. If I remember correctly the whole thing was called 'preemptive paging' in the mainframe world from which I retired many years ago. When the processor needs to access the page the RSM only has to turn off the flag rather than paging it in from disk. This takes very little cpu time indeed. It is possible that this happens because BOINC runs at a low priority (actually the lowest priority) and pages of low priority tasks become the first candidates for paging out. One mystery remains i.e. why this BOINC task and not the others. Today I am running two new tasks and there is virtually no page faulting although the other memory indicators are largely the same as yesterday. By the way today's processes are HCC (help conquer cancer) and yesterday's process in question was HPF2 (Human protein folding 2). Believe me when I say that I understand and share your feelings of frustration and exasperation with the Windows OS. I have 2 GB of real storage and run with 2 users on my system. When I logoff the other user the whole system freezes for up to 20 seconds. Compare that to a mainframe system with lower clock speed and less real storage (1GB) supporting up to 5000 users with sub-second response time no matter how many users were logging off or on. It is beyond my comprehension that the system would slow down at all just to logoff a user. In any event I can probably proof if my assumptions are right by increasing the priority of the next HPF2 task that seems to have a lot of page faults. I'll update this post should I be able to do that. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
I think with HPF2 being the oldest, no science code changed, just the graphics part upgraded reflected in the version change from 5.18 to 6.03 (for BOINC 6), it has the highest in terms of PF & PF Delta. Some noted that the points per hour are the lowest for this project ON Average, and it is, which suggests to confirm it's not totally optimal for all running, due BG's overarching engine "standards".
----------------------------------------If you want to see the going rates, visit the graphs link in my sig and find the "Points per Hour" chart.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
jsobry, you aren't jumping the gun. On the contrary, you are late to the party :-)
The whole paging issue was examined in great detail. For most people, soft faults are now reduced to a reasonable amount. However, if you have insufficient L2 cache, you may see more page faults than most people. These are soft faults, not hard faults. The disk is not accessed. Actually, you don't give Bill Gates enough credit. The way the Windows memory manager works is well documented. It is instrumented, too - you can track soft and hard page faults for a process using Performance Monitor. |
||
|
|
jsobry@gmail.com
Cruncher Canada Joined: Jan 11, 2008 Post Count: 31 Status: Offline Project Badges:
|
I have 2 MB of L2 that should be more than adequate.
I did point out that I can see page faults in the Performance and Process windows of the Windows Task Manager monitor. That is how I noticed this odd behaviour in the first place. What you cannot see in the monitor is the actual paging I/O to disk. I am no longer concerned as what you call 'soft faults' do not really use much cpu. I'm only curious and interested now. As to giving BG credit I think I'm safe to say he is one of the few people who does not need any credit during the current financial crisis. |
||
|
|
jsobry@gmail.com
Cruncher Canada Joined: Jan 11, 2008 Post Count: 31 Status: Offline Project Badges:
|
Thank you for the updates good people. I'm curious to know what you, Sekerob, mean by BG's overarching engine "standards".
Also you do seem to imply that this is somehow the application's fault. Do we think the page faulting is caused because we do not touch certain code pages often enough? |
||
|
|
JmBoullier
Former Community Advisor Normandy - France Joined: Jan 26, 2007 Post Count: 3716 Status: Offline Project Badges:
|
am no longer concerned as what you call 'soft faults' do not really use much cpu. The soft page fault itself may be cheap, that already depends on how many instructions the system grinds before marking it as a soft page fault. But in the case mentioned earlier it was also an indication that something in the libraries was not used in the most efficient way, and that's what was costing several percents of CPU. So when the total number of soft page faults for a WU is in the billions range (as it was) it is worth some deeper investigation. Cheers. Jean. |
||
|
|
mreuter80
Advanced Cruncher Joined: Oct 2, 2006 Post Count: 83 Status: Offline Project Badges:
|
Hi jsobry,
I had/have the same issue (if you wanna call it that way). Usually that happens when I used some specific Microsoft program (sorry don't' wanna point the finger). But this is OK, since you still crunch - especially when you finished with your work and close the other application it should return to normal (and even if not, you still support the good thing). cheers. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Thank you for the updates good people. I'm curious to know what you, Sekerob, mean by BG's overarching engine "standards". Also you do seem to imply that this is somehow the application's fault. Do we think the page faulting is caused because we do not touch certain code pages often enough? Was referring to lawrencehardin's observation of library calls and bloat-ware. HPF2 does not yet use the latest programming "insides" to minimise the PF occurrences as was explained being the achievement for e.g. HCC. There's nearly 18 months between the 2 coming on stream. PF's for this science never been a particular bother and with HPF2 having had it's difficulties to get it working on all platforms it's running on now, not a priority. It runs fine on my Quad, and is was a dog on my P4 (retired). How I rate: Claim v Granted credit are aligned on the 4 core.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
|