Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 11
Posts: 11   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 1212 times and has 10 replies Next Thread
jsobry@gmail.com
Cruncher
Canada
Joined: Jan 11, 2008
Post Count: 31
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
High paging

Running with 2 GB of real storage and ample free storage available I find wgc_hpf2_rosetta_6.03_windows_intelx86 ... paging excessively as reported by Windows Task Manager under Processes. I see PF Deltas of 3000+ and the task is the highest pager after Windows explorer.
Real memory useage and peak memory useage seem normal compared to other tasks.
Even VM size seems normal.
I'm running two BOINC tasks simultaneously and the other task hardly pages at all .
I also allow up tp 50% of memory usage and keep the tasks in storage while idle.
Heavy paging consumes cpu unnecessarily and it could be used by the task itself.

What is wrong with this task?
----------------------------------------
[Edit 1 times, last edit by jsobry at Oct 2, 2008 10:15:02 PM]
[Oct 2, 2008 10:08:26 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: High paging

Hello jsobry,
We probably won't be able to give you an explanation. Last year Uplinger spent a lot of time on HCC paging and finally tracked the problem down to interactions between several different standard libraries - not the application code but the standard functions called by it. After a lot of experimentation, he was able to select functions that did the job without all the page faults.

The growing OS bloat is starting to impact the application programs compiled to run on these over-complicated monsters.

At least, that is my opinion. I am still steamed by an article I read earlier this evening praising Microsoft's upcoming OS developments, which faulted users for still using 32-bit systems with only 3 GB of memory. 'Of course it seems slow' the article stated. It is all the fault of the users.

Argh-h-h,
Lawrence
[Oct 3, 2008 12:00:23 AM]   Link   Report threatening or abusive post: please login first  Go to top 
jsobry@gmail.com
Cruncher
Canada
Joined: Jan 11, 2008
Post Count: 31
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: High paging

I may have jumped the gun a little here. Of course the lack of proper and valuable information about BG's OS does not help. Not even their monitor provides the really valuable data one needs to properly evaluate a system's performance.
Depending on how they implement the Windows paging algorithm this may not be as bad as it seems.
I claimed that the system was paging heavily where in fact I cannot monitor paging activity as such, I can only see the page fault rate.
They may use an algorithm that flags pages to be paged out at the next cycle should a shortage of real storage develop. The real storage manager (RSM) then only turns on a flag indicating that this page should be paged out. This marks the page as invalid. However the page is not actually paged out and moved to disk. If I remember correctly the whole thing was called 'preemptive paging' in the mainframe world from which I retired many years ago.
When the processor needs to access the page the RSM only has to turn off the flag rather than paging it in from disk. This takes very little cpu time indeed.
It is possible that this happens because BOINC runs at a low priority (actually the lowest priority) and pages of low priority tasks become the first candidates for paging out.
One mystery remains i.e. why this BOINC task and not the others. Today I am running two new tasks and there is virtually no page faulting although the other memory indicators are largely the same as yesterday.
By the way today's processes are HCC (help conquer cancer) and yesterday's process in question was HPF2 (Human protein folding 2).
Believe me when I say that I understand and share your feelings of frustration and exasperation with the Windows OS. I have 2 GB of real storage and run with 2 users on my system. When I logoff the other user the whole system freezes for up to 20 seconds. Compare that to a mainframe system with lower clock speed and less real storage (1GB) supporting up to 5000 users with sub-second response time no matter how many users were logging off or on. It is beyond my comprehension that the system would slow down at all just to logoff a user.
In any event I can probably proof if my assumptions are right by increasing the priority of the next HPF2 task that seems to have a lot of page faults. I'll update this post should I be able to do that.
[Oct 3, 2008 5:24:49 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: High paging

I think with HPF2 being the oldest, no science code changed, just the graphics part upgraded reflected in the version change from 5.18 to 6.03 (for BOINC 6), it has the highest in terms of PF & PF Delta. Some noted that the points per hour are the lowest for this project ON Average, and it is, which suggests to confirm it's not totally optimal for all running, due BG's overarching engine "standards".

If you want to see the going rates, visit the graphs link in my sig and find the "Points per Hour" chart.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Oct 3, 2008 5:46:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: High paging

jsobry, you aren't jumping the gun. On the contrary, you are late to the party :-)

The whole paging issue was examined in great detail. For most people, soft faults are now reduced to a reasonable amount. However, if you have insufficient L2 cache, you may see more page faults than most people.

These are soft faults, not hard faults. The disk is not accessed.

Actually, you don't give Bill Gates enough credit. The way the Windows memory manager works is well documented. It is instrumented, too - you can track soft and hard page faults for a process using Performance Monitor.
[Oct 3, 2008 6:00:52 PM]   Link   Report threatening or abusive post: please login first  Go to top 
jsobry@gmail.com
Cruncher
Canada
Joined: Jan 11, 2008
Post Count: 31
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: High paging

I have 2 MB of L2 that should be more than adequate.
I did point out that I can see page faults in the Performance and Process windows of the Windows Task Manager monitor. That is how I noticed this odd behaviour in the first place. What you cannot see in the monitor is the actual paging I/O to disk.
I am no longer concerned as what you call 'soft faults' do not really use much cpu.
I'm only curious and interested now.
As to giving BG credit I think I'm safe to say he is one of the few people who does not need any credit during the current financial crisis.
[Oct 3, 2008 10:39:21 PM]   Link   Report threatening or abusive post: please login first  Go to top 
jsobry@gmail.com
Cruncher
Canada
Joined: Jan 11, 2008
Post Count: 31
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: High paging

Thank you for the updates good people. I'm curious to know what you, Sekerob, mean by BG's overarching engine "standards".
Also you do seem to imply that this is somehow the application's fault. Do we think the page faulting is caused because we do not touch certain code pages often enough?
[Oct 3, 2008 11:09:01 PM]   Link   Report threatening or abusive post: please login first  Go to top 
JmBoullier
Former Community Advisor
Normandy - France
Joined: Jan 26, 2007
Post Count: 3716
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: High paging

am no longer concerned as what you call 'soft faults' do not really use much cpu.

The soft page fault itself may be cheap, that already depends on how many instructions the system grinds before marking it as a soft page fault.
But in the case mentioned earlier it was also an indication that something in the libraries was not used in the most efficient way, and that's what was costing several percents of CPU.
So when the total number of soft page faults for a WU is in the billions range (as it was) it is worth some deeper investigation. smile

Cheers. Jean.
----------------------------------------
Team--> Decrypthon -->Statistics/Join -->Thread
[Oct 4, 2008 1:34:42 AM]   Link   Report threatening or abusive post: please login first  Go to top 
mreuter80
Advanced Cruncher
Joined: Oct 2, 2006
Post Count: 83
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: High paging

Hi jsobry,

I had/have the same issue (if you wanna call it that way). Usually that happens when I used some specific Microsoft program (sorry don't' wanna point the finger). But this is OK, since you still crunch - especially when you finished with your work and close the other application it should return to normal (and even if not, you still support the good thing).

cheers.
[Oct 4, 2008 5:22:31 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: High paging

Thank you for the updates good people. I'm curious to know what you, Sekerob, mean by BG's overarching engine "standards".
Also you do seem to imply that this is somehow the application's fault. Do we think the page faulting is caused because we do not touch certain code pages often enough?

Was referring to lawrencehardin's observation of library calls and bloat-ware. HPF2 does not yet use the latest programming "insides" to minimise the PF occurrences as was explained being the achievement for e.g. HCC. There's nearly 18 months between the 2 coming on stream. PF's for this science never been a particular bother and with HPF2 having had it's difficulties to get it working on all platforms it's running on now, not a priority. It runs fine on my Quad, and is was a dog on my P4 (retired). How I rate: Claim v Granted credit are aligned on the 4 core.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Oct 4, 2008 7:43:54 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 11   Pages: 2   [ 1 2 | Next Page ]
[ Jump to Last Post ]
Post new Thread