After painstakingly trawling through the arguments about CPU matching (pairing) on the above mentioned thread, I have now come to the conclusion that I have been sucked into a separate discussion of work in progress. Yes, this is excellent work but in fact it does not actually tackle the potential I am talking about. It does something different; basically it tackles the problems with the WCG Servers system of task dispensing, but at the level of WCG sub-project (HCMD2).
The above formula will not identify a 64bit platforms productivity advantages and encourage it to run one WCG task over another. It does not deal with this potential to optimise at this one-project level. It cant because it merely calculates CPU abilities of systems on one project and only affects that project, by CPU-pairing for it, and that is based on Whetstone benchmarking – not Operating system and not actual CPU capabilities or actual performances. I’m not knocking what you are doing here, just saying that it is different to what I am suggesting and does not optimise systemically.
Yes, there will be a rise in performance on that project, but that is not what I am getting at. Whetstone performance does not tell you that a certain CPU is naturally 5% more efficient at running HCC than FAAH or that it is on a 64bit platform that actually performs an additional 20% better on some tasks due to the operating system.
Any benefit to other projects using CPU-Pairing will only be as a side effect of expediting the throughput of that sub-project (HCMD2). I am talking about a system that benefits the whole WCG project, and all its sub-projects, all at once.

For what I am talking about, tables and algorithms for dispensing work units in one project are at the wrong level. Although I don’t want to presume any method, what I suggest would be like creating a table (or database) of system efficiencies that can be used to select computers to do tasks based on those computers specific abilities. Then selectively allocate project tasks, specifically based on these sub-project optimisation factors. This would be a WCG project and not a Boinc project.
You could still keep an equal weight for all projects approach by setting preference levels for each computers factor level.
The system I am thinking about would not just increase throughput for one project and as a result free up a small amount of time for the others, it will inherently speed up all projects at the same time; acceleration towards a new speed.

One simple way to do this would be to put together a system measurement task that runs a sample task for each sub-project for a small amount of time and is then returned and used to calculate system efficiency factors WRT each WCG sub-project. This would give a base line approximation of system performance for each sub-project. Then when the completed tasks come home, run the performance results through some efficiency matrix and update the sub-project efficiencies (productivity factors) accordingly. Then continue to use it to allocate tasks.

This way someone with 2 or 3 systems on the WCG would have their systems optimised for points, which equals work. For example my Phenom II might run HCC most of the time and my Q9400 might run HPF2 most of the time, rather than 50/50. The result could be 5 or 10% more work done.

The badge collectors don’t have to opt in, or might choose to after say getting bronze, or gold... Most of those with large numbers of computers attached to several projects would be happy to participate. They would actually get more points and the same number of badges. The order in which they reach the badges might change. Such a system would still allow people to select sub-project preferences, but could also facilitate greater control of the WCG projects by the WCG team. Something that might one day make or break a science group; if the project does not finish on time, you don’t get published, you don’t get a grant, the research takes a big knock and there is less work for the WCG.

rembertw, lots of good suggestions and interesting points.

I am all for choice and on the face of it, your €Optimise the Projects I Participate in €™ button would do the trick. However the reality is that we all participate in the WGC Project, and the WCG has a rule to balance projects; so when someone opts out of one project the people that opt in to all projects end up doing more of the tasks the first person chose not to participate in.

As an asside, I think to some extent the badge system has facilitated this balancing, as it encourages everyone to participate in each project. Unfortunately it facilitates balancing at both the cost of real choice and optimisation for everyone.

So, given this balancing rule is set to continue, it makes more sense that people with systems inherently optimised to do one task actually be encouraged to do that task, even if it is not their preferred choice of task; because with balancing in place, their preferred choice will benefit by others doing more work on their preferred sub-project!

It is worth noting that even if someone has reasons to opt out of a WCG sub-project, their mere participation in the WCG will, as a result of balancing, mean that the tasks they opt out of get more work done!
It is a false choice.

I would be happy to support all the present WCG projects and I expect this to continue, given the vetting system. With project support balancing in place, perhaps users should simply be given the choice to participate or not. There is a lot to be said about simplification. It would certainly help the credit system and perhaps the badge system has got a bit out of hand too!

It might be best to leave optimisation up to the techs, as if we do it ourselves, we wont be as effective, and it might even prevent them from doing it!

Credit should accurately reflect the amount of work actually done and could well be important if it allows a method to calculate optimisation. It is also to some extent important in that it allows people to set goals and targets, gives people something to aim for, sort of like the badge system but overall. It outlasts projects.

Perhaps it would be better for optimisation if there were no more badges (purple for 4 years, or whatever) as this might take away from would be attempts at optimisation?

That said I dont have any Blue badges, yet.

Let me comment about the 'tech throttle'.

First a bit of explanation about how the scheduler works. When a request arrives at the server, the scheduler starts at a random spot in an array of results that are ready to send. It scans this array multiple times doing the following in order:

• If the host is 'reliable', then find any results that need to be sent to a reliable host. If found, assign to that host.
  
• If the host is willing to accept beta work, then scan the array for beta results. If found, assign to that host.
  
• Scan the array for results that were marked 'infeasible' by another computer (this would be because of hr_class, memory requirements, etc). If found, assign to that host.
  
• Scan the array for any results. Assign as needed

In all cases, if the user has said they don't want work from that application then they will not receive work from that application. If the host does not belong to the required hr_class for the result, then it will not be assigned. If the host does not meet the requirements for the result, then it will not be assigned.

The only control we have over the distribution of results is controlling the number of spots in the array each project gets.

As a result of these rules, if we were to assign the same number of slots to each project then the number of results distributed would not be even. Help Cure Muscular Dystrophy and Help Conquer Cancer would have the most results assigned because as soon as the first result of the pair was assigned, the other would distributed at a higher priority because it would now have an hr_class. Next would come FightAIDS@Home and Help Fight Childhood Cancer. When 2nd copies are generated for these, those 2nd copies would belong to an hr_class and would therefore be sent our at a higher priority. The last would be Human Proteome Folding Phase 2 and Rice. This do not use any hr_classes.

The next item is that everyone looks at the amount of run-time between the projects, not the rate at which workunits are distributed. The run-time for each result varies between each project and over time. As a result, the distribution of runtime varies over time. If we set a specific 'weight' and left it alone, then the distribution of runtime per project would vary due to this.

We currently do not have a way to determine what the users 'want'. The files that represent which projects users have opted to run on their machines are stored in either BLOB columns in the databases. We need to migrate them to XML columns. This would then allow us to run a query and find out what people have opted to run. Unfortunately, we cannot do that now without writing a custom app.

Even if we could determine those values, we have had issues in the past with the process of 'opting-in' people who want to be opted in to new projects. There are a couple of projects where it appears that as much as a third of the population did not get opted in as they had indicated. We have improved the process significantly, but I'm still not sure that it works 100%. As a result, taking what is stored in the database as an accurate reflection of what people want to run is a bit dicey.

Finally, there is a constant stream of issues minor issues with each of the projects. Disks fill up, people get sick, work generation falls behind. We are frequently adjusting the weights to smooth all of this out to keep work flowing for all projects even if the amount of work for the projects adjusts over time.

This is why there is a 'tech throttle' and why we have to adjust it over time.