kate

in my case i run sudo screen because i can't get default perms to play nicely as install by dpkg so i have to manually fire it up on my boxen.

so pasting the two lines starting with 'sudo screen'

into konsole is exactly what i do.

screen provides a terminal session that can run in the background once you close the konsole window.

'sudo screen', is a heavy handed _root_ terminal session, so be careful with fdupes and everything else inside screen.

I forgot to mention that if root writes files (fdupes probably does not write any files) the boinc user will be unable to do housecleaning on the slots, so either run boinc as root or run screen as user: boinc, or manually, ymmv

run in what order?

ideally this goes in run_client script before startup as a background process in the distro, if it proves valuable.

the fdupes will not change the disk blocks read alredy by the queue before fdupes merges the data, and it will not change the executable image that is run prior to fdupes merging the executable images.

but if its run continuously, most of those things stuck in memory will release the handles to old data/blocks and use the new normalized blocks between loops and each time a file is opened.

so order is 'concurrently', and if left alone, it should be a profittable tradeoff of parasitic housekeeping overhead to reduce the amount of redundant kernel buffers to perform the same results.

when a job loads the first time, it willl probably load an isolated, redundant image freshly copied from the network into a file. this executable RAM lives in a kernel buffer and sticks around for the life of the process or subprocess.

after the 10 minute sleep interval the fdupes results will be hardlinking the image that once loaded the process against the normalized hardlink image. this will not be a shared executable until the next time it is started.

i do not know how long boinc keeps processes alive, and how often disk is updated for results, but the goal here is to avoid hardlinking results files and to avoid not linking executables and data files brought in from the net.

as near as i can tell this is a pretty handy trick even if it;s just to save a few gigs of free space let alone reduce page-in calls.

i will probably not be doing a formal validation of the method, but if others see a graph jump the same as me, I won't have to.

Huh, I just looked at the graph in my BOINC manager, and it's trended up over 200 points in the last month without my using fdupes. I'm trying to think what has changed. I've been running CEP2 or CEP2 betas almost exclusively the whole time -- mix in a single HPF2 and HFCC on very rare occasions, and had 3 DDDT2s.

The Linux version of CEP2 *has* changed recently. I wonder if that could be it. And I wonder whether that could account for any
of the improvement sgggas saw recently.

In any case, I'll report back in a couple days on effect of running fdupes on a Core 2 Duo.

Rather than adding a "while do done", have created a crontab line for user/group "boinc" in /etc/crontab:

10 * * * * boinc fdupes -Lrnq /var/lib/boinc-client

This setup will though not execute first time until 10 minutes after boot... but then, when was the last boot again?

Have had so far 2 outright HFCC error jobs, though the result logs and client messages are completely clean (puzzling). Here the last 15 results. BOINCTasks is not logging improved efficiency... still trundling between 93% when system is used to 96+% while system is idle.

c4cw_ target02_ 061045146_ 0-- 1292373 Valid 11/26/10 04:16:12 11/27/10 10:02:26 2.63 71.1 / 49.1
c4cw_ target02_ 060997149_ 0-- 1292373 Valid 11/26/10 03:00:07 11/27/10 09:33:42 2.64 72.0 / 49.0
E200639_ 658_ A.26.C19H13N3S2Si2.9.2.set1d06_ 1-- 1292373 Pending Validation 11/26/10 19:33:26 11/27/10 08:52:43 9.90 269.9 / 0.0
c4cw_ target02_ 060981973_ 0-- 1292373 Inconclusive 11/26/10 02:22:21 11/27/10 08:34:38 2.65 67.8 / 0.0
E200639_ 681_ A.26.C20H13NOS3Si.118.2.set1d06_ 1-- 1292373 Valid 11/26/10 19:33:06 11/27/10 07:16:11 8.49 217.3 / 222.3
c4cw_ target02_ 060977955_ 0-- 1292373 Valid 11/26/10 02:02:48 11/27/10 06:49:22 2.66 68.6 / 49.6
c4cw_ target02_ 060959324_ 0-- 1292373 Valid 11/26/10 01:11:46 11/27/10 05:53:00 2.70 71.0 / 49.8
c4cw_ target02_ 060897156_ 0-- 1292373 Valid 11/25/10 23:18:09 11/27/10 04:06:33 2.64 69.9 / 48.9
c4cw_ target02_ 060886344_ 0-- 1292373 Valid 11/25/10 23:07:56 11/27/10 03:06:22 2.64 65.0 / 49.3
HFCC_ L3_ 00899273_ L3_ 0001_ 0-- 1292373 Valid 11/25/10 22:41:23 11/27/10 01:25:42 5.73 146.2 / 90.4
HFCC_ L3_ 00892093_ L3_ 0001_ 0-- 1292373 Valid 11/25/10 21:25:07 11/27/10 00:48:57 5.29 138.2 / 81.7
E200636_ 593_ A.27.C19H10N4S4.236.2.set1d06_ 0-- 1292373 Valid 11/26/10 12:18:44 11/26/10 22:17:45 9.09 237.4 / 223.7
E200636_ 503_ A.27.C20H10N2OS4.177.0.set1d06_ 0-- 1292373 Pending Validation 11/26/10 12:18:26 11/26/10 22:02:38 8.93 233.1 / 0.0
c4cw_ target02_ 060839450_ 0-- 1292373 Valid 11/25/10 21:24:45 11/26/10 19:33:26 2.72 71.5 / 49.7
HFCC_ L3_ 00884098_ L3_ 0001_ 0-- 1292373 Error 11/25/10 19:50:56 11/26/10 19:22:49 5.03 132.0 / 0.0

The version fduped 1.50-PR2-3 seems to run considerably faster than 1.50-PR2-bld 1 as installed under Ubuntu 10.04.1. (10.04.2 expected in January).

BOINC Manager Disk Tab does not report a single byte of change (1.77GB used on 2 CEP, 2 C4CW concurrent) though the "du sh" command suggests that there's only 973MB GB. Not figured why. Maybe BOINC counts the files are were they still in each slot?

wrt disk space its nice that each boincmgr slot's tally is one-dimensional and doesn't discount symlinks. du -sh is probably the accurate reading of disk bytes used.


as i say the jobs may start without being hardlinked files, and then fdupes will actually unlink the files that are in memory leaving them as orphans preserved until the process ends and releases the executable and data pages of the old unlinked files.

sekerob when i first started out testing fdupes i had seen errors results while also tuning clockspeed/heat. i also did not use -n (ignore empty files) and had worries that new results files all start out lookinging the same, causing a process to write to a phantom disk buffer that is discarded when released, yet performing the correct code.

the improvement to this method is for boinc to mark files that are safe with some kind of bit, so that find + fdupes can be used to omit the pobbile results files before they diverge in content.

/s/ggg/qqq/g :)

thanks for giving it a shot. i am surprised there's a negative benefit though if you are swimming in RAM or if you are starving for RAM there might be from roiling the buffercaches with fdupes in contention with sharing between 2 cores vs. many more.

cron might kick off too much other libc/unix processing during the job sending results and processing mail queues from captured output, some other daemon would be desirable.


i checked out something else related to HT, do jobs have different heat requirements? resounding yes. CMD2 runs remarkably cooler (therefore different hardware paths) than CEP2. the efficiency jump could be related to instruction diversification + free hardware context switch on greater cache hits.


running 12 CMD2 jobs side by side is 58 degrees solid
running 12 CEP2 jobs 68 degrees solid.
running 6/6 jobs: 66-67 degrees
running 1 CMD2: 34 degrees
running 1 CEP2: q-fan cyclical 40 and down to 36 degrees

running cpu bench on 1 core :44 degrees
running cpu bench on 12 core :67.5 degrees

the benchmarks are not going to give any instruction soup insights but here's the numbers below.




ation


at 4ghz (via BCLCK, 24x multiplier + bclk boost)