Hi Dr Bonneau

Good to hear from you again, even better to hear that we are progressing so well; and thanks for the appreciation

awesome!

it's also good to feel honestly informed about what's going on. thanks.

I cannot speculate about how UD runs grid.org. However, I can clear up a few points about what we do here.

First, the average work unit run time is tunable by us. For example, the set of work units needed to process a given gene can be tuned in length and quantity. For example 20 10-hour-work-units could be used to process the same amount of total work as 10 20-hour-work-units. The overall result is the same, but members get more frequent feedback when shorter running work units are used. However, our servers have to handle more network traffic running shorter work units. For now, we are quite happy with the 10-hour average. At the start of the project our average was 20 hours and because of the wide variation in difficulty of particular work units (the part of the estimation we cannot predict), those work units were sometimes getting in the 200-300 hour run-time range. So, by cutting the average in half, the really long ones got cut in half on average (we still see some in the 100+ hour range) and the really short ones are as just a couple hours or so. So the trade-off is network bandwidth and average work unit run time. We will continue to keep them shorter unless we find that for some reason they need to be lengthened (or shortened).

We do not run easier genes or anything like that. I have found relatively little variation in the overall time to process any given batch of work. A batch of work consists of approximately 1000 genes. Once in a while we see that a few genes create monster run times, but that is just part of the normal variation and unpredictability of how hard it is to fold a particular protein.

As for redundancy, currently each work unit is sent to a minimum of 6 machines. However, machines sometimes lose work for whatever reasons and come back for new work, so not all of the assigned work gets finished and returned. We send out more of these straggler work units to additional machines. And as time grows longer we also relax the redundancy compare requirement somewhat for the last small percent of the work units. Without these measures, our disk storage would run out before the oldest batch gets finished. We would probably have to reprocess the all the batches under way from a clean slate, because it would be hard to patch everything up after running out of space. We process the returned results around the clock so that we can free up space used by finished work units. Even so, we sometimes get dangerously close to running out of disk space, so we have to keep a close eye on everything. In fact, we are planning a large disk space increase to better accommodate this and new projects. We will soon be announcing a several hour outage to let us restructure our storage layout.

We have set the minimum physical memory requirement at 128MB. We wanted as many machines as possible that could reasonably contribute, so we didn't want to set this value too high. However, we do depend very much on virtual memory. The Rosetta program allocates approximately 200MB of memory, which obviously does not fit in 128MB. However, once the computation gets under way, the working set size is only about 25MB, so the 128MB machines can typically handle this, albeit with some extra paging activity. We do find that quite a few member machines have their maximum virtual memory size set too low or have other applications consuming most of the virtual memory allotment. This crashes Rosetta and that work unit ends up not being finished by that machine. Some of the future projects may have much higher hardware requirements and might not be suitable for all of the member machines. We hope to keep a mix of projects going, some of which require less resources so that as many member machines as possible can contribute. Also, in the future we hope to develop the ability to assign shorter work units to slower machines, so that their run times are not so long.

FWIW, that is my long winded 2 cents on the subject.

We don't keep sending out repeated work until 6 are received. This would cause considerable inefficiency, so we usually send out at least 6 so that we are very likely to get 5 back. We don't have precise control over how many get sent out because of the nature of the scheduler, so we have found ways to work around that to achieve acceptable efficiency. Our compares start at 5 and for a small percentage of the work units the compare count is lowered as I mentioned above.

I find the lack of interest/knowledge in and of one of ones partners rather disturbing.