I'd like to throw in a few comments into this discussion:

1) Use Ubuntu Server, not Desktop
- PRO: fewer cycles used up for the flashy UI and many background desktop processes, therefore more cycles for BOINC to use. Also 4-6GB ram seems to be plenty.

- PRO: use the cheapest Video board you can get your hands on. Spend the money you saved on the CPUs.

You're only going to use it for a few hours until everything is installed. Then disconnect your new machine from your keyboard, mouse and monitor and set Ubuntu free.

- CON: Ubuntu Server is a little more linux-y to work with because it's the command line all the time. You will need to work with a command line editor like vi or emacs. You will have to install ssh and figure out ssh-keygen, etc. but it's boilerplate work, i.e. use a script.
- CON: a little more work having to configure BOINC using XML files and boinccmd. But not terribly difficult either, see http://boinc.berkeley.edu/trac/wiki/PrefsOverride
and http://boinc.berkeley.edu/wiki/Boinccmd_tool. This is all mitigated by using scripts and backing the config files up on another machine.

2) get script-happy and nearly the whole Ubuntu server installation can fit into a series of scripts.

This is very nice to have when the machine breaks down (and it will eventually). You can be back up and running in about 2 hours, not counting building the hardware. For almost all of that time it's installing things automatically for you while you watch TV.

And a set of scripts keeps all the boxes the same. Don't underestimate the benefits of conformity. I am assimilated.

3) Use Solid State Drives; 32GB is plenty
- the machine will be on all the time. My experience with BOINC/WCG over the last many years is that the hard drives take a bad mechanical beating. I've had a hard drive failure once or twice each year in the past.

SSDs seem to run cooler and hold up longer. Also if the SSD runs cooler, then the system fan runs quieter with less power.

So far no hard drive failed this year, fingers crossed.

4) The next component to fail is the power supply (I've had two fail).
- Don't skimp too much here, aim for middle of the road in price. Going overboard costs unnecessary money.
- If a power supply does fail, a new one is $60-$80. One that costs $120 or more may last twice as long as a cheaper one, but the $60 one could very well outlast the other components in the system anyway.
- When a power supply does fail, the replacement is fast and easy. Generally speaking no OS reinstall is necessary or other activity. Compare that with a hard drive failure.

5) go for many cores but, to save money, go for slightly slower GHz.
- Going from 4 cores to 6 cores is a bigger bang than going from 3.0Ghz to 3.3 Ghz.

A lot of folks are willing to pay a bragging-rights premium and vendors very much understand the "value" of chest-puffing, so you will pay through the nose for an extra few MHz that don't give you much more in WCG results or points.

- Yes it will still cost you more for 6 cores (roughly $170 vs $100 for quads). It's an extra $70 for the CPU but the cost of the other machine components are "amortized" over the total number of cores in the CPU. In other words, what would be the additional cost of another machine to hold the extra two cores? A: a lot more than $70.

6) Put the machines in a cold room, with lots of air circulation.
- the fan will run slightly slower, saving power.

- get the air circulation from the outside air, not from the inside of your house, otherwise you will pay for air conditioning as well as running the machine's fans. If you're lucky, in winter the fans might not run much at all.

- You can try using one of the those huge CPU cooling fans (copper, tons of cooling fins, looks futuristic or like an old V8 - take your choice). They seem to help but I'm not 100% convinced they are cost effective.

7) Put the machines in a room that you can close off.
- 4 machines all with fans running near 100% will be loud. You might not want to put the machines in your home office next to your bedroom (ask me how I know!)
- This contradicts the previous point in that a closed off room will be hotter. So it's a trade-off.

Summary: I have built a 6-core cpu using AMD PhenomII X6 1090T under Ubuntu Server 10.01 for around $700. It's the most powerful machine I have producing the most out of the 10 machines running BOINC right now.

For you, at $700 each and a budget of $2500, that's easily 3 machines (18 cores) plus some left over for an ethernet hub, cables, a nice big power bar and other odds and ends that tend to accumulate.

If you shop around a bit more than I did, each machine could be slightly less say $625. That's 4 machines (24 cores), but you'd be out of pocket for the other stuff you need.

John

Well yeah, I was thinking of Canada when I wrote it. I was raised there for most of my life and now I'm in San Diego. One is cool, one is hot. :)

Hi,


This post

http://www.worldcommunitygrid.org/forums/wcg/...31821_lastpage,yes#343202

might be of interest to those folks building new machines.

John

It's interesting to see opinions on 'the best machine' for crunshing. I think there are too many variables; each has its merits, but the main thrust seems to advise adding more cores than boxes. That means GPUs.

Also, chewing the most numbers in the shortest possible time means zip if the machine(s) are out for downtime, so reliability must presumably be a factor. Should some of the budget be spent on a service contract? Or can the OP pull machines apart when they die?

Cooling seems to be a major issue too, and then we get to future-proofing and the forecast of WCG GPU crunching.

There is presumably, a hint in the CPU benchmarks below in which direction the rather large budget could be spent.

I go with the suggestion of ATX mainboards with two or more graphics card slots and, probably i7 cpus. Add 2 + x GPU's to suit. Chuck in some Linux, spend money on cooling and good cases.

http://www.cpubenchmark.net/high_end_cpus.html

University of Antwerp built a 12 teraflop machine on this principle. Full build details are on the FASTRA site.

http://www.bit-tech.net/news/bits/2009/12/15/...ds-13-gpu-supercomputer/1

I find that leaving crunching boxes "naked" helps the most. So, if looks are not considered in the over all "cost", then that is an obvious advantage to the heat debate. Also, location and weather is another huge factor. GPU's definitely add up heat quick, but that is partially offset by locating them in different rooms and under windows. Now my thermostat can be set lower during the winter. There are several boards now that have more then just a couple PCI Express X16 slots in them. I have been looking at some that have 5+ of them. If and when WCG gets GPU work, those will out perform most CPU's and so, cost-wise may be smarter to just use a hex core AMD chip. We wont know until that day comes because WCG may never get GPU. arrizza mentions fanless cooling options and their costs, however if one sees cost beyond dollar signs, noise reduction may also be a factor. I know a few of my towers are so loud, I can't watch TV in the same room without turning the volume to obnoxious levels.