Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 10
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 2196 times and has 9 replies Next Thread
keithhenry
Ace Cruncher
Senile old farts of the world ....uh.....uh..... nevermind
Joined: Nov 18, 2004
Post Count: 18667
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Is fairly comparing project scoring possible?

I typically crunch on any and all projects WCG has running under BOINC. What I have noticed is that when I have a run of FAAH WUs, my stats seem to grow some degree better that when I have more of a good mix of different project WUs. This is all antedotal of course. Plus, the last this I was to do is ignite a storm over how the projects are scored. Offhand, it would seem that crunching project A for X amount of time and project B for the same amount of time would yield pretty much the same amount of points. However, there are bound to be complicating factors - how your benchmarks vary over time for example. So, at this point I'm just wondering - IS there a FAIR way to apples to apples compare the points you get crunching on project versus another? What do you need to allow for to make such a comparasion?
----------------------------------------
Join/Website/IMODB



[Jan 8, 2007 2:42:25 AM]   Link   Report threatening or abusive post: please login first  Go to top 
olympic
Senior Cruncher
Joined: Jun 12, 2005
Post Count: 156
Status: Offline
Reply to this Post  Reply with Quote 
Re: Is fairly comparing project scoring possible?

The points you receive have nothing to do with the type of WU being crunched. Points are awarded based on the CPU benchmark and CPU time. So theoretically, a computer should earn the same amount of points/day regardless of what project it participates in.
----------------------------------------

[Jan 8, 2007 5:17:27 AM]   Link   Report threatening or abusive post: please login first  Go to top 
zombie67 [MM]
Senior Cruncher
USA
Joined: May 26, 2006
Post Count: 228
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Is fairly comparing project scoring possible?

Benchmarks are meaningless, as they are easily modified with a simple text editor.
----------------------------------------

[Jan 8, 2007 7:40:39 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Is fairly comparing project scoring possible?

The points you receive have nothing to do with the type of WU being crunched. Points are awarded based on the CPU benchmark and CPU time. So theoretically, a computer should earn the same amount of points/day regardless of what project it participates in.

In theory this is true. In the real world, it's not even close.

If the theory held up every machine would claim the same score for the same work unit. A quick look at the quorum scores shows that this doesn't happen.

Essentially, ignoring text editors, the differences come down to CPU architecture and the way each handles individual tasks. More modern machines, for example, are far more efficient than older ones and outperform their benchmark expectations. Due to this they complete task more quickly and claim fewer points per work unit. This also varies depending on how efficient your CPU is at performing the task being asked of it. HDC will suit one architecture best while FA@H will suit another.

This can be seen in the quorums. One will find that their computer may be consistently the high or low claiming machine in one project and yet will be right in the middle on others. It follows then that if you are a consistent low claimer for a particular project and your computer claims a constant number of points per day, that your overall score will be better on that project because other machines in the quorum will drag your average score up. The opposite is also true.

Benchmarks therefore are increasingly meaningless, as more and different architecture enters the market, and the comparisons Keith asked about are impossible.

Cheers. ozylynx smile
[Jan 8, 2007 9:49:43 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Is fairly comparing project scoring possible?

Subjectively, HDC seems to generate the highest median per hour and GC the lowest. The sighted architecture but also the choice of project......people with power-boxes electing the heavier work plays a role.

With FA@H, the large majority of work done on that project, there is no telling. Sometimes my 10.2 stock per hour is the lowest, sometimes close to median, sometimes way up. Pure chance of PC mix meeting up in the quorum. Having tracked it for a while (to learn understand the discussions), just know that the overall median seems to credit me slightly more than is claimed. Opposed, there will be, that overall get less....such is life which we try to save with all our crunching.

Once HPF2 starts on BOINC, it will be a different ballgame. Groupings of 25 Work Units, with credits awarded / median computed from the 15th WU returned....16-25 will get that median value as I understand. We'll see.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Jan 8, 2007 10:42:39 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
love struck Is fairly comparing project scoring possible?

Subjectively, HDC seems to generate the highest median per hour and GC the lowest. The sighted architecture but also the choice of project......people with power-boxes electing the heavier work plays a role.

With FA@H, the large majority of work done on that project, there is no telling. Sometimes my 10.2 stock per hour is the lowest, sometimes close to median, sometimes way up. Pure chance of PC mix meeting up in the quorum. Having tracked it for a while (to learn understand the discussions), just know that the overall median seems to credit me slightly more than is claimed. Opposed, there will be, that overall get less....such is life which we try to save with all our crunching.

Once HPF2 starts on BOINC, it will be a different ballgame. Groupings of 25 Work Units, with credits awarded / median computed from the 15th WU returned....16-25 will get that median value as I understand. We'll see.

Fot those of you (me) that don't get this.. Just keep Crunching!
We will all get there at the same time!
(Uh....... where ever that is!)


drooling
[Jan 8, 2007 11:29:32 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Is fairly comparing project scoring possible?

I guess that the definitive test for Keith would be to look at his FA@H quorum results and see where he ranks in the claims as a median. If this is compared to His median claim for other projects and it is observed that his FA@H claim is lower than the others, it would be fair to say that his average score would indeed be higher when working on FA@H.

It works like this... Ones computer is benchmarked to produce say 10.6 PPH. Regardless of the project one works on the computer claims 10.6 PPH. If the computer is better than average at doing a project it will complete the assignment quicker than would otherwise be expected. i.e. If the benchmark indicates completion in 10 hours but is completed in 9, instead of claiming 106 points it will claim only 95.4 points. When this score is placed into a quorum it will be awarded (106+106+95.4)/3 = 102.46 points.

This gives the low scoring machine a 7% increase in points raising the effective PPH from 10.6 to 11.34 PPH and one would observe an overall increase in production when crunching this project from an expected 254.4 PPD to 272.1. Not a lot these figures are conservative. If a high claiming machine were to be included in the example the impact would become far more dramatic.

Those who cheat in order to claim artificially high scores benefit the low claimer far more than they do themselves, weird justice in a way.

Cheers. ozylynx smile
[Jan 8, 2007 12:06:05 PM]   Link   Report threatening or abusive post: please login first  Go to top 
keithhenry
Ace Cruncher
Senile old farts of the world ....uh.....uh..... nevermind
Joined: Nov 18, 2004
Post Count: 18667
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Is fairly comparing project scoring possible?

Well, actually, I'm trying to take an even simpler look at this. Let's ignore the points I actually get along with everyone else in the quorum for the WU. Let's look only at what points I claim for the different projects I crunch. At a very simplified level, CPU time would seem to be CPU time regardless of which project is using it. If I crunch a project A WU for 1 CPU hour and claim 10 points, shouldn't I expect to claim 20 points for a project B WU that runs 2 CPU hours?
----------------------------------------
Join/Website/IMODB



[Jan 8, 2007 2:05:47 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Is fairly comparing project scoring possible?

Hi Keith

Yes your claimed points should be constant/hour regardless of the project. At least between benchmark updates.

Cheers. ozylynx smile
[Jan 8, 2007 2:39:30 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Sekerob
Ace Cruncher
Joined: Jul 24, 2005
Post Count: 20043
Status: Offline
Reply to this Post  Reply with Quote 
Re: Is fairly comparing project scoring possible?

BOINC benchmark is automatic and presently runs on versions 5.8.1 and earlier every 5x24 hours Wall clock. Also runs immediately upon install, upgrade or downgrade of the agent.
----------------------------------------
WCG Global & Research > Make Proposal Help: Start Here!
Please help to make the Forums an enjoyable experience for All!
[Jan 8, 2007 3:10:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
[ Jump to Last Post ]
Post new Thread