Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 20
Posts: 20   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 2552 times and has 19 replies Next Thread
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

The long term improvement to the benchmarks /credit scoring system is going to becoming in the future - possibly the 5.12 BOINC client (the 5.10 client will be a relatively small release that largely only includes an auto-update feature). The general gist of the change is that instead of just running a general benchmark - everyone will actually run a short workunit using the actual science application. Thus your computer will be measured on how long it takes for you to run the actual science project - which is the most accurate way to measure your computers performance for running that particular science project.


That is not an improvement. It's just a new way of committing the same mistake, the mistake of allowing host computers to evaluate their own performance. That system will always be cheated.

Furthermore, if a project can devise a test WU that will way of predict the performance of all hosts when those hosts run the real WUs, then surely the project has enough knowledge of its science app and WUs to just assign fixed credits on the host, as many projects are doing today.

Any system that allows hosts to evaluate their own performance will be cheated and anybody who believes otherwise is delusional. You can add encryption, hashes, whatever you want and they will be broken too. Modern encryption and hashing is secure when there are zero to very few clues into the encryption/hash. Unfortunately, the exchange of WUs and results will provide cheaters with more than enough clues to break whatever security is built into this supposed "improvement". It might take a lot of computing power to break it but such power is easily obtained today... just setup a BOINC project, advertise it, hand out ridiculously high credits and wait... soon you will have thousands of fools merrily crunching whatever you ask them to crunch, no questions asked, just wash away any curiosity with rich lather of credits, propaganda and censored forums. Sheeples will do anything when you know how to use them and how to herd them.

Also - BOINC is open source. The above feature is on the schedule to be implemented - but if you want it done sooner - than dig in! Look at the code, figure out how it works, start working on the change. Check out: http://boinc.berkeley.edu/boinc_dev.php to learn more.


Do convicts design and build the prisons that hold them? Ridiculous!! Open source security, yeah right!! Has Merck/Frosst created a naivite pill?
.
[Feb 20, 2007 3:48:24 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

Hi David_L6 et al

The Benchmark has a reason and compared to the UD agent that runs it every WU, every 5 days is fine for most hardly changing machines and is intended to measure if the client 'environment' has significantly changed. This is why the divisor 480 ** is used for BOINC so that small changes have hardly no impact. Also with the quorum 3 for all but the HPF2 project, the claims are summed and divided by 3. Statistically as your machine will fluctuate a bit, others will too and compensate.

You need to tell us the reason for not running it regularly so we can understand the motivation. If it causes Work Units to go back to last checkpoint, activate the 'leave in memory' option in the profile or upgrade to 5.8.14 (the latest dev.version), which waits for checkpoint saving or hold the WU in memory during housekeeping activities.

Far as i'm concerned given that BOINC processes from buffered work, thus never stops opposed to UD agent, it should be run frequently AND randomly. Would become a semi full time job for state file editors..... corrupt the file and you'd loose more work than the shore-up will gain.

Mostly, we're glad that teams do not condone amplifying the hourly claims resulting from the changed benchmark values, but for the real benchmark coming up higher due pure hardware optimization. The word spreads quickly around the Distributed Computing community..... works as a self purging.

We row with the oars that we got and compute along to get the projects to completion ASAP.

** Divisor 480 means that if the sum of Whet- + Dhry-stone adds to e.g. 4,800, the hourly claim is computed as 10 per CPU hour. On a 3 CPU hour, per core job, that would call for a claim of 30 and when running dual or more cores, the number of cores times 10 i.e. on a quad 40 per CPU hour. The 30 per job would translate to 30 x 7 = 210 WCG points.

Edit: inserted "for most hardly changing machines" to clarify!
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at Feb 20, 2007 6:03:48 PM]
[Feb 20, 2007 4:59:25 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

You need to tell us the reason for not running it regularly so we can understand the motivation. If it causes Work Units to go back to last checkpoint, activate the 'leave in memory' option in the profile or upgrade to 5.8.14 (the latest dev.version), which waits for checkpoint saving or hold the WU in memory during housekeeping activities.


Read the OP's post. That's not the problem.

Far as i'm concerned given that BOINC processes from buffered work, thus never stops opposed to UD agent, it should be run frequently AND randomly. Would become a semi full time job for state file editors..... corrupt the file and you'd loose more work than the shore-up will gain.


Illogical and just plain fear mongering and FUD slinging.

We row with the oars that we got and compute along to get the projects to completion ASAP.


If you're happy with the oars you got then row. Those who get broken oars will fix them, if they choose, using advice that travels fast around the 'Net, like a FUD purging.
.
[Feb 20, 2007 5:33:55 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

Would it be OK to just change the <p_calculated> part so that it doesn't run benchmarks every 5 days?

I have 3 machines with Intel P4's with HyperThreading. You probably already know what kind of problems they cause.....


No, i run an HT and have no problem, so still would like to know what the problem is with running it as scheduled.

On postponing the benchmark, a fix was just been put in to achieve the opposite: The benchmark will be forced, if a modification is detected.
David 21 Feb 2007
- core client: if benchmark time is in the future (due to user tweak)
always run benchmarks

----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Feb 21, 2007 5:49:26 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Dirk Gently
Senior Cruncher
England
Joined: Mar 1, 2005
Post Count: 153
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

Aplogies for resurrecting this thread after so long, but I just noticed something.

Just now I looked at my pending credit and noticed that the claimed credits all seemed a higher than I am used to seeing (almost x2). I looked at the benchmark scores currently stored in Client_State.xml and these seemed to be almost double what I remember. I kept a copy of Client_State.xml and re ran the benchmarks. This produced figures more in line with what I usually see.

Pasted below is the erroneous High benchmark:
<p_fpops>4675852066.715012</p_fpops>
<p_iops>8225251508.120034</p_iops>

This is the more usual score:
<p_fpops>2041665416.451956</p_fpops>
<p_iops>3181556128.139779</p_iops>

I originally assumed that the variation in Benchmark was only one way - ie sometimes lower than the "true" figure due to processor being interrupted during the benchmark run by critical windows system components. Now either this was a one off error, or the variation can be a double edged sword - penalising or over rewarding. I have re run the benchmarks 5 times now though, and I get the "expected" (lower) results consistently. It seems the fates are only generous once in a blue moon!

I agree with knreed that adopting ones own benchmark scores would be a questionable practice - even if done using fair figures. Would it be fair for me to use my occasional high score? No. In any case, I should see my batch of high credit claims ironed out by my quorum companions. Usually my claimed and granted scores match well.

I only started looking at benchmarks because I had forked out £80 for a new processor - from a P4D 830 3GHz to a P4D 950 3.4GHz. The clock speed increase is small, but I was assuming that increased cache of 2x2MB from 2x1MB would also make a big difference. It did not. The increase I got was only about 3.4/3 = 13%

13 % - Pah. So, how to make the lazy little Intel g*t work harder then? Overclock! I increased FSB from 800 to 957 which puts core speed up to 4.07Ghz. So increase now is 4.07/3 = 36%. Thats better!

But my CPU temp went up 10degC !! Reduced Processor Voltage by 1 notch - 0.0125V and it went back down. Now at 55-60degC.

System has now run like this 24/7 for about 6 weeks without a single crash and without a single errored workunit.

I think I will stick there. I have heard tales of Overclockers really turning the wick up on Intels and plumbing their processors into their central heating etc - but I am sure the little chap will only sulk if I push him any further.
----------------------------------------
[Apr 2, 2007 11:41:43 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

Dirk, you're not telling: Either you put hand an a optimized copy, don't think so, or you upgraded on a Linux OS from the BOINC 5.4 or early 5.8 clients to 5.8.15 or 5.8.17. Those benchmarks should now be in approximation to the Windows benchmarks. Though technically inflated, WOZ clients are the reference point holding 80% plus of the work generation across the crunching community.

In past the Linux clients generated benchmarks, to much distress of the Linux crunchers, that were like 55% of the Windows equivalent. Going from 55% to 100% would be a near doubling of score.

If the machine generates 6 weeks of error free work, adopt the adage: "If now not broken, dont mess with it".

PS: The benchmarks of 4675/8225 are hi, but clueless if it fits the processor you have.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
----------------------------------------
[Edit 1 times, last edit by Sekerob at Apr 3, 2007 8:50:42 AM]
[Apr 3, 2007 8:48:23 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Dirk Gently
Senior Cruncher
England
Joined: Mar 1, 2005
Post Count: 153
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

Dirk, you're not telling: Either you put hand an a optimized copy, don't think so, or you upgraded on a Linux OS from the BOINC 5.4 or early 5.8 clients to 5.8.15 or 5.8.17. .....

PS: The benchmarks of 4675/8225 are hi, but clueless if it fits the processor you have.



I have never used an "optimized copy" (whatever that is), nor have I ever manually intervened in the Benchmark scores. I get official BOINC updates directly from the website. I dont run Linux.

I only ever saw 4675/8225 once, having first noticed the high credit claims. A Benchmark of 2032/3179 is typical now (after proc upgrade).
SiSoft Sandra gives 18148 MIPS and 14983 MFLOPS, which bear no resemblance to the BOINC figures, although this does not matter.

My last BOINC upgrade was 5.8.11 to 5.8.15. This blip in benchmark score happened at some point after this. I was just curious if it was a rarely seen bug (relating to dual core processors) in BOINC and if any one else had noticed anything. I am still trying to get it to do it again.
----------------------------------------
[Apr 3, 2007 3:54:21 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

.... did not think you would as what i wrote, but there are some wild distributions in circulation.

a few times in windows i've seen the taskmanager show cpu allocations as if there was running just one core..... a benchmark glitch like that would cause your client to become an Outlier/high claimer in a quorum. That would default your credits to just being reduced to the mean of the remaining claims.

If you capture that situation again any info would help, so it can be passed to the developers.

Office 2007 is supposed to be able to use 2 cores, but i have yet to observe it doing it....maybe only on Vista?
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Apr 3, 2007 4:16:22 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Dirk Gently
Senior Cruncher
England
Joined: Mar 1, 2005
Post Count: 153
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

I have just installed the new BOINC version 5.8.16 (from 5.8.15), in the hope that software updates improve things laughing

Back a step! Whereas I was getting benchmarks of 2032/3179 fairly consistently with v5.8.16, after the upgrade I now get mostly figures of 2047/1773, and OCCASIONALY (1 run in 5) I get 2048/3225 - similar to what I used to get.

At least the Floating Point figure is consistent, and maybe it is only this that matters?? confused
----------------------------------------
[Apr 5, 2007 3:48:35 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Variation in BOINC Benchmark score

Well i'll be darned..... So see a post on Vista and the benchmark topic, then this comments drops the jaw:
Message 9378 - Posted 5 Apr 2007 17:08:35 UTC - in response to Message ID 9377.

BOINC 5.8.16 had some Benchmarking issues on Vista. It is less than 1/2. The Core 2 Duo T7200 (2.0G 4M L2) chip used to have a 3.9GFlOPs per core now has a 1.53GFlOPs per core. XP is fine, but Vista is not. The FPU speed is OK. The E6400 (2.13G 2M L2) is 4.07GlOPs per core.

Just wanted to bring this to someone's attention.

Did they try rerunning benchmarks a bunch of times? There's a bug somewhere where dual cores will drop one of the benchmarks by about half seemingly randomly.

On a Pentium D, I ran them maybe 6 or 7 times before I got an accurate one on more than one occasion.

----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Apr 6, 2007 7:02:56 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 20   Pages: 2   [ Previous Page | 1 2 ]
[ Jump to Last Post ]
Post new Thread