| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Community
Forum: Chat Room
Thread: POINTS: KISS method |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 15
|
|
| Author |
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
For those who don't know KISS stands for Keep It Simple Stupid.
I've read the other thread (now closed), about suggestions for a points system. What a flame fest! Although there are some excellent ideas and arguements put forward, the quest for the "perfect" points system becomes somewhat bogged down with complexity. Here's my idea: First let me say that the perfect point system will most likely never be achieved and the best one can realistically hope for is a fair points system. Cheats will always invent ways to cheat and ultimately cheat only themselves as they know that their points have no real meaning. They have no idea what their efforts are worth in the real world. Forget them they are of no consequence. There is no lottery win or trophy for the mantlepiece here. The only prize on offer is the benefit to humanity our work can help to achieve. Observation tells me that the scientists and programmers at WCG have a good idea of roughly how much work is involved in doing a particular type of work unit. Given this, it should be a relatively easy task to assign a fixed point score for 'x' amount of work and group similar work units into categories based on the average CPU demand for each categoryand award points based on this work. Thus slow and fast machines and machines with different Operating systems would receive the same points for the same work regardless of time taken to return the WU(Work Unit). Yes some of the Work will take longer than others due to changing and unpredictable energies within the structure of the protein or the chemicals being studied, likewise some will be shorter. Chaos theory and the law of averages will balance this effect provided a reasonable mean value has been set at the start. If this is not the case, that will be evident quite quickly and at this stage there should be more than enough data on record to make a very good estimate right now. Thus points are awarded for WUs completed. Fast computers complete more WU/Day than slow machines and are commensurately rewarded for doing so. That's simple!! The Trap: If the mean points per day(PPD) is significantly lower than those on offer by other worthy and not so worthy, projects on BOINC there is a risk that those who are more interested in points than the science, like it or not they are out there, may abandon the project to chase the higher PPD averages. This will require a good deal of "tweaking" and a watchful eye by the programmers at WCG. Sorry guys but it has to be better than the intensive MFLOP counting suggested elsewhere. Well that's it. No rewards or penalties for CPU type, HDD size, OS choice, or the brand of RAM and HSF you use, other than the increase or decrease produced in the form of WU/Day. The only way to gain points is to complete a WU That's my suggestion. What are your thoughts? Flames are not thoughts they are the opposite. Cheers. ozylynx  |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Actually, there are 2 different completely fair methods, but they are not in conflict with each other. One method is to measure effort donated (sum of all effective CPU times) and the other is to measure the actual work (mainly floating point operations) done for each work unit and assign points in proportion. Ignoring cheating and error conditions, this second method is still very difficult because to do it well is likely to require a great deal of coding in each project in order to board it, with many opportunities for bugs and much work expended for no scientific gain.
Observation tells me that the scientists and programmers at WCG have a good idea of roughly how much work is involved in doing a particular type of work unit. You are right. After running a large sample of work units, each project is able to estimate average work required per work unit with astounding accuracy. Unfortunately, any particular work unit might vary by almost an order of magnitude from the statistical average. This is a contentious subject that many people hold strong opinions about. Having tossed in my 2 bits, I will go back to lurking. Lawrence |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
ozylynx, you have kept your method simple by ignoring the realities of the project. Averages are all well and good, but even the most trivial analysis reveals it to be unfair.
I'm afraid complexity is here to stay. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
I too would like to keep it simple ozylynx, but I'm afraid Didactylos is right - to get every circumstance covered is going to require alot more complex solutions.
Personally the whole issue bores me somewhat and I would rather just get on with the crunching - frankly anymore flaming about this subject and I am likely to leave. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
The various solutions were discussed at what's referred to as the '2nd Pan-Galactic Workshop', so there might be coming an all encompassing system revision from BOINC, that also recognise special CPU features and eventually the capacity of the graphics processor....goal...x-project equity. Participating in 4 projects, find that Rosetta who is basing their credit system on structures / real computing effort and still tweaking it, is getting closer and closer.....the last 10 now just 12% off (more) from stock claim (got 8.1 per CPU hour)...... If a change happens, new discussion will break loose, but that's with any change..... the quorum 3 solution is still the second best available......SIMAP has just switched to quorum 2, BOINC rule being that the credit for all is the lowest, TANPAKU is a (no comment)....first come, first served.
----------------------------------------I totally agree with Ady.....next ad-homimem flame war allowed to pass, it will be suspend WCG project in BOINC........Think the WCG commission of wise man is still out. They promised a solution, should not wait too long and then present.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
to get every circumstance covered is going to require alot more complex solutions. Further explaination: First and foremost I have worked under a system like the one that I am proposing and except for a reluctance, on the part of those administering it, to admit when they got it wrong and take steps to correct errors, combined with an introduction of complexities over time, such as optimisations for specific OSs and bonus point systems etc., it worked exceptionally well. Adywebb and Didactylos my whole point revolves around not addressing all of the minutae. The reasons are manifold and include having clock cycles used for crunching work and not for counting and confirming that work. The science to me is more important. Also, the simpler the system the more time the scientists have to do their work and not get bogged down with points systems. Although a consistant effort here will be necessary to keep the donors happy. Here is an example: I have used the latest 'valid' results for my Athlon XP3000+ at a standard 2.1Ghz. Kind of midrange I'm sure you will agree. This system is dedicated 24/7 and except for some data server duties is undisturbed, crunching for WCG with a BOINC client. 5.44 Hrs 60/56 5.58 Hrs 62/49 Oh Dear 5.58 Hrs 62/62 5.58 Hrs 62/62 5.64 Hrs 62/62 5.65 Hrs 62/57 5.70 Hrs 63/54 Hmmm 5.72 Hrs 63/81 Bonus Time? 5.73 Hrs 63/65 5.74 Hrs 63/63 5.81 Hrs 64/84 Bonus again? 5.82 Hrs 64/55 Hmmm Again 5.95 Hrs 66/67 6.05 Hrs 67/64 6.10 Hrs 67/57 6.26 Hrs 69/65 The last of these is a "mx1k17_dry_01' the only one of its kind in the list and as such can be discarded as a seperate project, probably deserving of a different point score to the remainder. The first is also a probable anomoly but I will leave it in place for the excercise. Our range then is 0.66 Hrs or 40 minutes from slowest to fastest and a midrange of 5.77 Hrs/WU for this machine. If this machine were to be used as a benchmark for the excercise, the maximum deviation would be only 20 minutes and if the points allocated for this work were 63 the results would yield a range within 4 not the current 35 point deviation. A significant improvement I'm sure. No, it isn't perfect, but it is simple and it is fair. It would be worth noting that under the current system this machine is yielding 63.1 points per work unit of this type while my PIII 598Mhz machine is receiving 77.2 points/WU for similar work and in fact is claiming much more. The question here is too obvious. In summary: Perfection is always desireable but rarely acheivable. (I know because I am a perfectionist) Complexity, by its nature, opens loopholes and is too hard to administer, look no further for evidence of this than the ever expanding laws under which we all live. "Every word added to a law provides three opportunities to circumvent that law"(unknown) Hope that's food for further, constructive, discussion. Cheers. ozylynx [Edit 1 times, last edit by Former Member at Sep 27, 2006 9:27:15 AM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
For every problem, there is an answer that is simple, elegant - and wrong.
Local variation is, as you have observed, minimal. I expect, if you plotted the time taken by work units over a week, you woud see a simple normal distribution. But you need to understand how the project works to see the flaws. The scientists send the work to WCG in batches. Each batch tends to consist of similar work units - or sometimes, work units with wildly varying performances. My point is, if you performed this same experiment every couple of months, you would get very different results. Certainly, the current system has (major) flaws, but your proposal is even more naive than the Rosetta project's credit method. You have the right basic idea, though. A work unit should get the same credit no matter what computer it runs on, or how long it takes. What your solution ignores is the variation in work unit size that is so hard to measure. |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
Observation tells me that the scientists and programmers at WCG have a good idea of roughly how much work is involved in doing a particular type of work unit. You are right. After running a large sample of work units, each project is able to estimate average work required per work unit with astounding accuracy. Lawrence I believe that this forms the basis of the answer to that problem. Cheers. ozylynx [Edit 1 times, last edit by Former Member at Sep 27, 2006 10:25:12 AM] |
||
|
|
Former Member
Cruncher Joined: May 22, 2018 Post Count: 0 Status: Offline |
*sigh*
You ignored everything I said, didn't you? Well, you're welcome to do that. Just don't be insulted if your "solution" is ignored. |
||
|
|
Sekerob
Ace Cruncher Joined: Jul 24, 2005 Post Count: 20043 Status: Offline |
Personally, i don't think at all that the WCG system is (major) flawed......it's the method of BOINC benchmark control that is seriously flawed **, which coming from an open source software is made possible. The benchmark could be encapsulated in a closed routine which is not large and could be send up to the clients on a periodic basis in an encrypted way with a signature and version number encoded. Each WU returned could carry that benchmark code with it to signify its validity.....by torrenting this and making it to come from a single controlled website/source, it would put an end to the x-project imbalances atop.
----------------------------------------There are now close to 800,000 BOINC users world wide....the funding of BOINC development as understood from Rom is secured for years.....BOINC is the vehicle of choice in the DC community, so WCG as i think has got cloud, why not exercise that a little? ** By simply changing the BOINC.exe priority from normal to high one already gets significantly higher benchmark results. By making the benchmark running at high priority for all, an even playing field is created, where the CPU time for a WU that comes out at the end times benchmark value is than an equal claim however slow or fast that CPU time was accumulated.....1st K.I.S.S. Running a benchmark ad random (every 5 days) and make 5 days of result dependent on a momentary overall system load test is dubious.
WCG
Please help to make the Forums an enjoyable experience for All! |
||
|
|
|