Hi Mysteron347
Thanks for adding your thoughts on this topic, and for keeping the discussion alive. I was starting to think that I was the only person who cared.

Your proposed scheme would be ideal, but I think knreed has said that it would be too difficult.

With my proposed modification #1, instead of sending out parent WUs with your 20000 structures, they would send out only 4000, or whatever number the vast majority of the crunchers selected to do parent WUs could run to completion. In your example, they would both complete the 4000 and not hit the 60%/6h barrier at 5500 and 5000, and there would be no wastage. The crunching times and credit claims (and BOINC benchmark scores, if available) from these parent WUs would enable the amount of work per structure for the job to be calculated, and the remaining 16000 structures would be divided up accordingly.

Reading, just not injecting assumptions and presumptions or misunderstanding, my observation atm is that the device matching is not as fine grained as I'd expected it to be. Certainly on a result done few days ago if my wingman runs 6 hours, not reaching 60% and mine doing 9.74 hours doing the full, in linear, the 60% was reached at 5.84 hours so I know that 9.74 - 5.84 = 3.9 hours were binned, at least. For the slower machine that equates to at minimum an equivalent of 6.33 hours. Does not convey a happy feeling.

The 75% having no credit differential is open to interpretation. Could be high claim granted to both, low claim granted to both, could be pure quorum 2 claim mean and still have positions passed into a child.

Anyway, I'm fine with whatever other solutions such as:

A. Have volunteers opt to get WU's compiled to run the full length no matter what.
B. Assume the positions that had no match, the overage of 1 result in the quorum to be considered 100% okay when such a device has had a perfect score on the last 30 results (or any other number that gives scientists a high enough rate of confidence).

Right now at present pace and project growth we'll be going on till mid next decade, unless the positions truly become much lighter than they presently are or the scientists start culling the targets of lesser interest, or improve efficiency somehow. On that duration, any 1% redundancy reduction has a tremendous effect towards reaching project end.

PS, if the logs would actually print the positions completed we'd have real visible numbers for I don't consider credit in any form or fashion a useful part in computing waste. Credit claiming is just plain too iffy, one in quorum having run on a real Q6600 core and the other on a I7 hyperthreaded virtual core comes too mind as a possible quorum combo.

Make a *short* CMD2 and a *long* CMD2 subprojects.
Target the 6 hours for the long but let them run till completion.
Target 2 hours for the short and let them run to completion.
No fancy calculations, no parents, children, odd formulas for multiple cutoff levels ... run them like every other project (except RICE where I hope there is a better formula).

I believe that the difficulties which concern knreed is a matter of precisely how the 'fill-in' unit returns would be used, and could be a simple matter of interpretation.

If, given the 5,000/5,500 example a fill-in of (5,001-5,500) was created, then to verify 1-5,500, a minimum of THREE result units would have to be interpreted - the original two results for the parent and the fill-in. This complication would be compounded by the possibility that the fill-in is sent to an AWOL processor, or it is itself split into child workunits or that verification may fail and a _2 unit be generated, waited-for and then added to the co-ordination scheme. A nightmare scenario.

The key to my proposal is that the returned unit is partitioned, and the effective units generated are
parent : 1-5,000
Workunit A: 5,001-5,500
Workunit B: 5,501-9,125
Workunit C: 9,126 -12,750
Workunit D: 12,751-16,375
Workunit E: 16,376-20,000

There are EXACTLY two result returns for each of these units, and the existing mechanism (modified as I propose) may be used to deal with AWOL processors or fail-to-verify instances. The parent may be done and dusted for weeks before Workunit A, for instance - in the same way as the current mechanism works and has been proved.

=========

Moving the wall out is not desired as it will increase average run-time, contrary to lawrencehardin's user-comfort note.

Making the wall lower will similarly increase average run-time, as more processors will cross the Rubicon.

Making the wall higher will cause more processors to hit it. Only the very fastest will go over, and the faster (but not fastEST) processors simply hit higher than their companions. As noted, this leads to the waste we're discussing.

((Quite what it is that's hitting/clearing the wall is a mental image I'm having difficulty expunging..))

The issue once again comes down to the SLOPE of the line - back to pure processing speed. The closer the match in the slope, the less the splash from the wastage.

I believe that considering turnaround time in the cruncher-matching mechanism is not productive. Turnaround time would seem to affect Pending-Validation but not credit-granted (which in turn measures work done) I'm sure that crunchers would prefer to see improved efficiency at the cost of PV time - and improved efficiency should also act to decrease PV time...

Tightening the speed differential allowed would seem to be the way to go - perhaps even insisting that the processor given the second replica be of EXACTLY the same speed-class (relax after 24 or 48 hours to prevent "stuck" units) might give a worthwhile improvement.


I'm confident that if my return-result partitioning proposal was implemented, we'd see an improvement - and if that could be proved, the next improvement proposal just might be a little surprising...

I think we have a translation issue regarding my comments about RICE. I was not saying the algorythm was bad ... I was saying that I hoped it was better than CMD2 ... and you have now confirmed for me that it is