| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Support
Forum: Suggestions / Feedback
Thread: The Claims 'fArce Majeure'! |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 36
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Most of us admit the credit system is important and arguable. Some of us think the system here at WCG is good enough and that a system like R@H's new system will not bring lasting benefits. Some of us feel there are much more important issues that need to be addressed first. From R@H we will learn that the new system stopped the cheating for a while but eventually the system was cracked and the cheating started all over again. Show me a credit system that cannot be cheated and I will support it 100%. ja, it's the vicious circle  As long as WCG keeps quorum three, it's relatively no problem.On the R@H board appeared an idea that said the server should throw out the credit claims from 10 to 20/30 percents higher and lower than the standard deviation. This way needs, however, great amount of the same units. If not, even the credits claimed by very low spec's computers will be thrown out. Anyway, thinking the better system requires us to have known which factors are introduced into the system. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I was thinking a similar idea... to throw out claims that are 50 to 60% higher. That would allow larger differences in work units. But where do we find the programmers to turn that idea into code? Also, some honest claims would be thrown out simply because they are outside the standard deviation. It's a difficult problem but I am glad we can discuss it
 |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
We can provide a better solution than R@H, since we still require a quorum for scientific reasons, not merely to confirm points.
This means that if the science application code is adapted to keep track of the number of times each iteration is performed, it should be possible to calculate a fair measure of floating ops for each work unit. Since the flop count should be reported identically for each valid work unit in the quorum, we can consider the flop count to be reliable. There is still potential for attempting to cheat, but it would be much harder, since a fraudulent claim would only need one genuine claim to enable the fraud to be tracked down. The reason we don't have this setup already is because it will take major work to insert instrumentation into each science app. Counting operations sounds simple, but to do it without slowing down the science and also getting a totally accurate count is a lot harder. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Thanks for your explanation, didactylos
 The reason we don't have this setup already is because it will take major work to insert instrumentation into each science app. Counting operations sounds simple, but to do it without slowing down the science and also getting a totally accurate count is a lot harder. It makes sense, although it's mortifying that modified credits can be validated  This issue should be come out when it'll become major. It's minor one now. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Throwing out the low and high values already removes most of the false claims.
But we can do better even with the current system. If one of the claimed values is significantly different from the average claimed credit for a single work unit, then we can treat the high claims as suspicious. This will work even if two inflated claims coincide, since the low value will be exceptionally low compared to the other two. Then we can do two things: we can mark the host as untrusted, and avoid sending the same work unit to more than one untrusted host, and we can calculate a "penalty factor" to scale future claims from the bogus host. The problem with this plan is that BOINC claims vary by such a large amount even in the normal running of things. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Then we can do two things: we can mark the host as untrusted, and avoid sending the same work unit to more than one untrusted host, and we can calculate a "penalty factor" to scale future claims from the bogus host. The problem with this plan is that BOINC claims vary by such a large amount even in the normal running of things. What if we used that scheme with a minor modification? Rather than calculate a "penalty factor" which the machine automatically applies, why not increment a "bogus host" flag each time a host submits a ridiculously high claim. When the "bogus host" flag reaches a certain value then a human is notified. The human can assess the situation to determine if the host is truly bogus and take appropriate action. For repeat offenders who repeatedly refuse to rectify their ridiculous claims I would define "appropriate action" as stripping them of all their points and deleting their membership. Harsh yes, but refusing to stop cheating when politely asked to do so cannot be tolerated. Make examples of a few offenders and other offenders will stop cheating. Or maybe 1000 cheaters will quit WCG. No problem. 300 new members join every day, on average, the cheaters will not be missed. The problem with my suggestion is that it might consume precious manpower. For example, the alarm from the "bogus host" flag might go off 2,000 times per day. Well, that could indicate there are lots of cheaters who need to be dealt with OR it could indicate the system is too sensitive. The latter case could be dealt with by tuning a variable in the "bogus detector" which wouldn't require a lot of manpower. The former may require lots of manpower at first but once word got around that WCG doesn't tolerate cheating the manpower requirement would diminish. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
For repeat offenders who repeatedly refuse to rectify their ridiculous claims I would define "appropriate action" as stripping them of all their points and deleting their membership. Harsh yes, but refusing to stop cheating when politely asked to do so cannot be tolerated. Make examples of a few offenders and other offenders will stop cheating. Or maybe 1000 cheaters will quit WCG. No problem. 300 new members join every day, on average, the cheaters will not be missed. Your brave comment is worthy of praise  The problem with my suggestion is that it might consume precious manpower. For example, the alarm from the "bogus host" flag might go off 2,000 times per day. Well, that could indicate there are lots of cheaters who need to be dealt with OR it could indicate the system is too sensitive. The latter case could be dealt with by tuning a variable in the "bogus detector" which wouldn't require a lot of manpower. The former may require lots of manpower at first but once word got around that WCG doesn't tolerate cheating the manpower requirement would diminish. I think the latter case is better. Manpower at first should be consumed for developing technologies for sciences of this grid. It were okay if a plenty of manpower existed. Probably using automatic detector is effective system. The next problem will be that which factors have to be included into the detector, which can find out the bogus hosts from tremendous number of hosts. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Hello,
How will this project treat the result send from Truxoft core client? The home page says that it calibrate credits claimed, it can claim fair credits. If detector will be made, will you developers flag even the results from the hosts which use the client bogus? |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
I'm seeing in previous posts returning ideas, phrased in different ways, which is good.....consensus slowly building. Here some more comments from Ralph http://ralph.bakerlab.org/forum_thread.php?id=233#1954 where they're test-bedding different solutions, with emphasis that the localized credit system must abide by the global BOINC concept....apparently 100000 credits per teraflop.... if only there was a way to quickly determine these flops for each WU up front or in arrears in a fast manner...that would weed out the cheaters and simply remove any need for benchmarking
---------------------------------------- wonder if the '# number of errors per computer rule' has been re-activated...cherry picking hi yield WU's is brought up somewhere lower in that thread....what people do....it's like the Tour de France or Italian Serie A. PS Suguruhirahara, complements on your English.....re-reading Graham Greene's 'The Human Factor' at the moment....in the original, the only way to appreciate foreign language books...too much lost in translation ;>)
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I think Truxoft does not cheat if you set it up the way the author intends. I made a mistake in the config file (truxoft_prefs.xml) when I setup Truxoft and that caused it to claim too much credit. I think WCG will always treat the results from Truxoft clent the same as any other client.
|
||
|
|
|