Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
World Community Grid Forums
Category: Completed Research Forum: Help Defeat Cancer Thread: CPU/Disk Load to 100% every 15 minutes |
No member browsing this thread |
Thread Status: Active Total posts in this thread: 29
|
Author |
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hello retsof,
HDC spikes up to 750 Mb Rick Alther said that HDC occasionally spiked up to a bit more than 500 MB. So 750 MB should allow a normal XP system to run light programs such as a browser even while the HDC program is spiking without excessive paging. Of course, if you run hyperthreading and use BOINC to allow 2 threads to run, then you should have at least 1.25 GB to avoid excess paging. Lawrence |
||
|
Alther
Former World Community Grid Tech United States of America Joined: Sep 30, 2004 Post Count: 414 Status: Offline Project Badges: |
Since I've been receiving work units for Help Defeat Cancer, my PC goes nuts every 15 minutes (more or less). Something is writing or reading from disk to such an extent that all other tasks slowdown. Since this PC is my media PC, this is a problem. Is there a workaround for this? What you are experiencing is the UD Agent checkpointing a large file. Help Defeat Cancer can write some very large files (130MB or more). This causes a problem with UD Agents because when we checkpoint with UD, it compresses and stream encrypts the files to be checkpointed to backup files. This causes the I/O to take a relatively long time to complete (it's about 7-9 times slower than straight I/O). This is the I/O lag you are seeing. Knowing this, we "throttle" the I/O for the program. Instead of writing a lot of data at once, we write out the data in chunks and pause slightly between the chunks so that our application doesn't impact the user's machine with I/O. This also saves some memory. This makes a much larger difference in UD where all I/O is stream encrypted. While this I/O "throttle" (not to be confused in any way with the CPU throttle) certainly helps, there is nothing we can do when we checkpoint and UD backs the files up. That code is out of our hands. What happens under the covers is this: * The science app writes out files as it progresses through the code. All I/O in UD is stream encrypted and thus slow. The I/O read/written by the science app is "throttled" as described above. * We reach a point in the code where we can checkpoint. For UD, we call their checkpoint routine. All it basically does is look at the list of files that are considered checkpoint files and backs them up. This is where you are seeing the I/O hit. For BOINC, we call our own checkpoint class which backs up the files for us. Under BOINC this is very quick compared to UD, so it should barely be noticable. After the files are copied and the state updated, we tell BOINC we've completed our checkpoint. This answer is much longer than I anticpiated it would be, so I hope it's clear.
Rick Alther
Former World Community Grid Developer |
||
|
BlackSmurf68
Cruncher Joined: Nov 18, 2004 Post Count: 4 Status: Offline Project Badges: |
Defrag didn't help. The pagefile is already in one fragment.
I'm running UD client and BeyondTV 4.3 only. Sometimes I pop open Firefox to do a little surfing. I noticed that when the disk goes nuts, the CPU usage drops from 60% to 20%. The pagefile used stays the same (around 630MB). I have a 7600gt gfx card so there are no shared memory issues. Is it possible to store the checkpoint files in RAM or on a flash disk? If I could configure UD to store on another drive, that would probably fix my problem. |
||
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
i hope it's not one of those 'anti-virus' problems like with mcafee.
that virus scanner used to conflict with WCG a lot. i only found out about it by opening task manager and figuring out what else was taking CPU power. |
||
|
retsof
Former Community Advisor USA Joined: Jul 31, 2005 Post Count: 6824 Status: Offline Project Badges: |
Is it possible to store the checkpoint files in RAM or on a flash disk? If I could configure UD to store on another drive, that would probably fix my problem. The most gain will be made by putting the Windows swap file on a separate drive from the UD files. That topic comes up now and then.It seemed to be possible to run the UD interface on a flash drive and even swap it from computer to computer. There's a "sneakernet" topic on http://forum.grid.org concerning moving files from to the hard drive on one computer to the hard drive on another, advantageous when only one computer has a connection to the internet, and several others do not, but could be used for crunching. It works easiest when you have UDMonitor to manage the multiple slots on the internet computer. I tried it with BOINC but it was bad news. It was more closely tied to the timestamp in the registry on one computer, and things became corrupted when it was on the other. It couldn't quite understand why I couldn't finish the workunit by a deadline of 1900.
SUPPORT ADVISOR
----------------------------------------Work+GPU i7 8700 12threads School i7 4770 8threads Default+GPU Ryzen 7 3700X 16threads Ryzen 7 3800X 16 threads Ryzen 9 3900X 24threads Home i7 3540M 4threads50% [Edit 3 times, last edit by retsof at Jul 26, 2006 12:54:12 PM] |
||
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Retsof, when desperate one looks at anything.....the timestamp point very very valid, to the extend that i disabled the WinXp clock sync function as it corrupted twice a WU in BOINC (latest official 5.4.9).
----------------------------------------In UD that behaviour might be a bit more friendly...restart the WU, but could the clock synching be a factor? The 33.33% is an observed one. Similar to HPF2, these present batches of FAAH are split in very large segments. Put a post out, seeing types existing of 6, 3, 2 and one resetting for even 90 to zero, suggesting a no segment at all ????? What ever is said about the frequent checkpointing, i wait with closing until it passes these percent hurdles just to be totally save......nothing so annoying than loosing 4 / 5 or more hours of crunch. The Porting to a different computer...zip....copy....move pc ..unzip been successfully deployed by DPC members a few times without hearing of troubles.....lucked out. Swapfile on a 2nd drive helped total system performance quite a bit. Also it sits in its private partion there of 2gb, so never has issues of fragmentation when on autosizing. PageDefrag it once a while, but usually it says its already in 1 space.
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All! |
||
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
It may or may not help, but in the WCG BOINC profile there is an option to save your work every 60 second (default). I've not figured the function, since the checkpoints of some project Work Units is sometimes hours apart and a restart would throw u back to the last checkpoint saved. Anyway, subjectively, things improved on my machine, setting that default to 600 seconds.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All! |
||
|
AnimeStore.pl
Cruncher Joined: Dec 26, 2004 Post Count: 2 Status: Offline |
HDC seems to send the disk into a frenzy every 15 minutes or so and I can't understand why. For now, I'll just turn off HDC unless/until I hear the issue has been solved. Same here. It's impossible to work on my computer with something like that. I'm ignoring this project. |
||
|
BlackSmurf68
Cruncher Joined: Nov 18, 2004 Post Count: 4 Status: Offline Project Badges: |
The machine I'm running this on is my media PC. I still wanted to run WCG because contributing to help defeat diseases is important to me. I decided to buy a 1GB flash card and install the software there thinking that would resolve the disk "freeze" issue. The flash card is interfaced to the IDE bus via a card reader. I then tried another HDC project. I still have a problem. All other tasks are frozen for a split second every so often. If the computer is running a game, for example, the game will "hiccup" about every 5 seconds.
I'm turning off this project since I'm out of ideas on how to resolve this issue. I don't have any problem with other projects. |
||
|
Alther
Former World Community Grid Tech United States of America Joined: Sep 30, 2004 Post Count: 414 Status: Offline Project Badges: |
ALL: Please read my earlier post on 25 July. It appears no one has read it. The problem with the I/O hit should only be happening with the UD Agent, as I described in my earlier post. If the I/O is affecting you, I would recommend running the BOINC client (at least for the HDC project), which doesn't have the I/O overhead that UD does.
----------------------------------------
Rick Alther
Former World Community Grid Developer |
||
|
|