| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: BOINC Agent Support
Thread: BOINC Points post in Member News |
| No member browsing this thread |
|
Thread Status: Locked Total posts in this thread: 277
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
ozylynx, part of the problem is the different compiler optimisations used in different builds of the benchmark. For the benchmark to operate as designed, it shouldn't really be optimised at all, since a benchmark consists of a load of work for the processor that doesn't actually serve any purpose - clever optimisers spot this and remove a lot of the actual benchmark. The other problem is that the benchmark only accesses a small amount of memory, so while the benchmark is running, this will be cached on the processor. This cache memory is a lot (lot!) faster than accessing main memory, so the benchmark again fails to match real world performance.
One of the options being considered by the BOINC developers is a memory benchmark, that will attempt to factor memory performance into the total benchmark and thus the credit calculation. The final problem is these benchmarks are old. They have been around for decades. It is hard to know exactly how well they represent the power of a modern computer. Other benchmark suites may be more relevant to measuring crunching performance. |
||
|
|
zombie67 [MM]
Senior Cruncher USA Joined: May 26, 2006 Post Count: 228 Status: Offline Project Badges: 
|
How about this: Within each WU, include a mini-WU with a known value. The machines run this as part of the WU. Upon reporting, the performance running the mini-WU is used to gauge total credit for the WU. Sure, it would add a little more run time to the WU. But it would be a way to remove the benchmarks and guessing.
---------------------------------------- |
||
|
|
Movieman
Veteran Cruncher Joined: Sep 9, 2006 Post Count: 1042 Status: Offline |
Lets cut to the chase:
----------------------------------------The current system is very flawed.We all agree on that. How can we fix it? Who needs to be contacted and what are their names and email addresses? To provide a public database of what machine will do what with a given WU, is it possible for me to get a sample known WU that I can take and run on the PC's at XS? With this I can get you an exact breakdown from a huge variety of machines from the norm to the highest state of the art machines. If one can be sent to me I'll get this ball rolling and we will all know and not think what the facts are.I can arrange this so that an exact breakdown is possible. You will then know what factors give what results. Please let me know on this. The clock is ticking. Lets fix this problem and not discuss it to death. Thanks.  |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
My 2c. worth...
Not willing to prolong this debate - but trying to 'fix' the benchmarks could be troublesome in the extreme. While WCG should be applauded for their efforts to date, they are trying to fix something that is fundamentally flawed IMHO, so a different solution should be found. I, for one, would be in favour of a fixed award per WU, per project. So each faah wu gets x points, hpf2 y, and hdc z etc. Let WCG fix the values of x,y and z according to bechmarks on a standard system. This way those with faster systems will get more multiples of x, y or z (whatever they are cruching). I'm aware that some wus can be highly variable in terms of execution time (hdc runtimes have tripled on my systems recently for example), but there would be nothing to prevent sampling of the wu stream on occassion and fixing scores accordingly. It is slighly irksome that a highly o/c'd, phase cooled X6800 claims little more per hour than a stock FX60. Just hope those QX's turn up soon  Jonathan |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
You already have the WCG contact details. The BOINC contact details are on their website.
zombie67, the mini-WU idea is an interesting one. BOINC already has a pluggable benchmark feature. Really, the only thing to check is that the WU reflects the characteristics of the real WU. That shouldn't be a problem if it uses exactly the same software. However, it's a costly thing to implement, in terms of coding time and deployment. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
OK. Cache performance and size, was the next thing on my list of possibles.
Interesting idea. You couldn't switch cache out as that would cause a different error. You'd need to swamp the cache, but by how much? Even HDD caching could have an effect. Too many unknowns and variables IMHO. Cheers. ozylynx  |
||
|
|
Movieman
Veteran Cruncher Joined: Sep 9, 2006 Post Count: 1042 Status: Offline |
You already have the WCG contact details. The BOINC contact details are on their website. zombie67, the mini-WU idea is an interesting one. BOINC already has a pluggable benchmark feature. Really, the only thing to check is that the WU reflects the characteristics of the real WU. That shouldn't be a problem if it uses exactly the same software. However, it's a costly thing to implement, in terms of coding time and deployment. yes, I have emailed that address, but no answer as of yet which is why I asked for a specific address or telephone #. I also like the idea of a mini WU but even if it was doable with no cost to implement, unless it took a decent amount of time to run could it bring back a "good" number? then add in the factor of all that cpu time run for no scientific work being generated. I was always taught never to bring a problem to upper management without also bringing a solution so here is one, at least for the interim: Roll back your last changes whatever they were leaving only the 1/2 factor for "outliers". There were other changes beside that outlier issue that have totally skewed the credit system. Do that and you then buy yourself the time to attack the main problem. Thanks. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
The mean calculation seems to be providing a far more stable system than the median. You still have to demonstrate that a problem exists. Tempting though it is to argue from a single quorum or from a few quorums you have observed, doing that misses the bigger picture.
You can't have failed to consider that WCG have been monitoring the new credit system, and also the bigger statistics view. Certainly there is no massive problem with the new system, or that would have been spotted in the beta test analysis. Frankly, I barely understand your argument, since it seems to be based on uneven quorums giving an advantage or disadvantage somehow. Statistically, that shouldn't be an issue, since quorum composition is essentially random, and any minor disadvantage will even out over the long term. I suspect any tiny overall drop in points you may have detected can be attributed directly to the removal of the overclaimers from consideration. |
||
|
|
Movieman
Veteran Cruncher Joined: Sep 9, 2006 Post Count: 1042 Status: Offline |
The mean calculation seems to be providing a far more stable system than the median. You still have to demonstrate that a problem exists. Tempting though it is to argue from a single quorum or from a few quorums you have observed, doing that misses the bigger picture. You can't have failed to consider that WCG have been monitoring the new credit system, and also the bigger statistics view. Certainly there is no massive problem with the new system, or that would have been spotted in the beta test analysis. Frankly, I barely understand your argument, since it seems to be based on uneven quorums giving an advantage or disadvantage somehow. Statistically, that shouldn't be an issue, since quorum composition is essentially random, and any minor disadvantage will even out over the long term. I suspect any tiny overall drop in points you may have detected can be attributed directly to the removal of the overclaimers from consideration. How about that you and I just agree that the 2 of us have vastly different viewpoints on this and let it go at that. Your not answering my questions but tossing in your viewpoint. I didn't ask for your viewpoint but for information. Thanks. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Sorry, I can't give you direct contact information. WCG will deal with your inquiry promptly, but please bear in mind that credit issues do not rate highly on their list of priorities. It is more important to do the science, and keep the vast majority of members satisfied. WCG cannot spend a disproportionate amount of time catering for the needs of supercrunchers. They simply lack the resources. I trust you understand this.
|
||
|
|
|