Foundation,

Can you increase your bandwidth?

Reminder: there was a mistake in this sentence. It is the upload which takes time. The download is peanuts.

Dear Cliff,

first of all: thanks for supporting our project! Your participation in the CEP2 is greatly appreciated.

Indeed, the size of the results is one of our biggest concerns as well (although the 80MB are somewhat of a worst case scenario, the average is so far 14MB and the maximum 68MB).

We put a lot of effort in designing the work units and tailoring them to the limitations of the WCG. The binary molecular orbital file is responsible for 90-99% of the data volume. Unfortunately, it is central to our analysis and subsequent calculations, so we cannot drop it. Regeneration is also not an option, since it would require nearly as much time as the original calculation on the grid. We still use a number of tricks to reduce the result size, e.g., we convert all binary numbers from double to single precision and recover the full accuracy in house. This reduces the overall size by a factor of 2 and the scaling by 4. Otherwise, our outputs are already stripped as bare as possible without compromising their scientific use.

Thanks for your suggestions anyways and best wishes

Johannes

(Harvard CEP2 Team)

Johannes,
Thanks for the reply. I've read your response and the web pages about the science and admit that I still don't have a complete picture of what exactly you are calculating. But this is the internet, so I can't let a lack knowledge stand in the way of offering an opinion!

The critical ratio for distributed computing is the amount of calculation per unit of network transfer. So the first thing I would look at is whether more work could be done in a single calculation. Make the calculations run 10 times as long for the amount of network traffic, and I could put ten times the number of machines on this project before it chokes my DSL. With intermittently available computational resources, there would be some inefficiency due to interrupted/lost work units, but perhaps you have or could derive an empirical model for this effect, and could find an optimal value. (Or perhaps you already have?)

The next question is whether you could use parametric continuity in some fashion to reduce the transfered data volume. As an example, we often use ab initio QM to calculate reaction systems over a trajectory from one (meta-)stable configuration through a transition state to another. To determine the trajectory accurately, hundreds of points along the path are calculated. We evaluate a few physical quantities at each of these points, then store only the nuclear configuration and only at a handful of points over the trajectory. If we subsequently need to calculate additional physical quantities, we do a single-point calculation at the trajectory configuration of interest (usually only at the stable configurations or the transition state). This example of a reaction coordinate as a continuous parameter is where we achieve the factor of 1000 compression I mentioned before. It probably doesn't exactly apply to your calculations, but you might think of something parametric in your problem space and use a similar trick.

Finally, there is the question of whether this project really is (or should be) a search for candidate materials. There are always reasons to store the results from all work performed, the most obvious being to avoid redundant calculations. Also, scientific insight can be gained from looking at the results from both promising and non-promising materials. And it is possible that you are using the detailed results from all materials to guide subsequent material choices, in a directed and dynamic search process within a large dimensional material composition/configuration space. However, I'm guessing that is not the case and rather the researchers have simply defined a large number of materials they wish to evaluate. Therefore, if the goal is to find materials with the right physical properties, it seems a scalar functional "material value" could be formed. Once the "value" of a material has been defined, there are two obvious approaches to be considered. The first is that the detailed results (14-80MB) would be returned only for those materials with sufficiently high "value". The second is that only the material "value" and very limited results be returned from each work unit, with the intent that high "value" materials will be fully investigated in subsequent work units either on your own local machines or at institutions with large bandwidth.

An obvious counter to this "material value" approach would be if the detailed results from a current work unit are not sufficient to evaluate the materials behavior. But then I would point to my first suggestion to do more work in a single work unit.

If you could direct me to an available reference which explains the details of the calculations you are doing, I would be interested and might come back with more specific (and useful) suggestions. At the very least, it would keep me busy for a while.

I hope you see this discussion as constructive and not simply as having to deal with a crank. I currently have 16 of my 75 active cores devoted to this project. I have another 200 cores which could be brought online, but I face power, cooling and now bandwidth limitations. Rest assured, my interest is driven by wanting to make as much of this as possible available to you.
Regards, Cliff

The good news is, the average task run times keep rising. It's now 6 hours and my quad is now starting to take 8+ CPU hours for the current work. Taken it off-line to see for when finished if there is a proportional reduction from the 15Mb sizes I've seen for the 5 hour tasks.