What member? I'm confused. ericinboston last posted 8 days ago and I have no idea where this person is from, so still worth looking into the link I posted.

I'm stepping a bit late into this, but as someone who has crunched the numbers many times, and who has set up similar "farms" (though I'd never own anything that large) I would like to point out a couple of other things as well:

when designing such a network, efficiency is critical. I would highly recommend a blade type system, or other turn key solution. By the time you factor in the cost of ownership, over the course of a year or two the turnkey solutions can offer real advantages. Contact the major vendors, they have sales people who deal daily with small business and can explain to you the advantages/disadvantages of using the systems. Your best bet will likely be to drop a 220V feed into the basement and go with a blade system.

flops for dollars, the really expensive, fast 4 and 6 core Xeons are a bargain compared to the cheaper counterparts when projected over the usable life of the system. If you sit down and do the math, you will find that Intel has already done it when pricing their processors, and that each step up will save you more.


To quantify 100 Mac mini's

The current draw stated is 110W max for the CPU, but that is actually a bit low, by the time you add in peripherals, drives, and switches, but we will assume that we can figure on 110W because the CPU will typically perform a little better than that. So 100 x 110 = 11,000 With a 85% efficient power supply, we will be around 12,500W

with average cost of electricity in the U.S. at about $0.14 kw/h this bring the cost of operation to roughly $1.75 per hr for electricity.

$1.75 / hr * 24 hours = $42.00 / day * 365 =

$15,330 / year to run the systems under full load, just for electrical costs @ $0.14/kwhr.

@ $0.15/kwh the total is about $16,425 / year


Unless you live near the poles, heat will be a huge factor

I've run 40+ cores in similar sized rooms, and the cost to cool the rooms has typically been roughly equal to the cost of running the machines, so go ahead and double those numbers to take into account cooling costs.

These electrical costs can be greatly reduced with turnkey systems, so despite the higher up front cost, by the end of a year or two, you can get far more performance, with far less energy, and at a lower cost. Also, you should consider leasing the equipment, especially if this is for a business venture, since you can write off the cost as a business expense annually rather than the depreciated value over time. Usually equipment gets replaced long before it's purchased price is written off.

For networking, just buy a couple rolls of cat-5 and wire them with some commercial grade switches, anything less will be really really disappointing.

Hi eric,
I wonder if you could share with us what you will do about this project, and maybe your thougths by now, specially after the lot of advises, alerts and tips posted here to help you on that.

The space is not that important in some areas, in fact you do need some space to ensure good air flow, and the more heat you generate the more important cycling the air becomes. It really depends on environmental conditions. Someone in Arizona will have far different concerns than someone in Alaska. Usually when comparing systems that seem like a bargain, once projected over 2 or 3 years cost, the $2500 CPU's win out over the $90 CPUS in total output per $ spent by fairly large margins. Similarly things like properly sizing the power supplies (200% of actual load), and getting the more expensive but more efficient power supplies, usually pay for themselves within the first year. If however you live in an area that maintains cold temperatures for extended periods of the year, you may only have to allow for some ventilation to the outside rather than expensive cooling costs, so for some it is cheaper to go with multiple cheap CPUs. Air conditioning is nice because it removes humidity from the air, but there are other ways.

It all comes down to budget, but ultimately, getting the best is usually also the most cost effective (outside of things like black box CPUs or off brands). The form factor will depend on necessity for the hardware, but if one were to simply look for a lot of crunching power, a diskless stack of very high end server CPU's on multi-proc boards, or a blade system usually win out in raw power per $.

Obviously, because of the noise and heat, most people would not want something like this in their home. So it's also about personal preferences and quite often spousal opinion. My home workspace ended up looking like this (sorry for the poor image, my camera phone erm.. sucks): Right now a little over half of that rack is unused, and will probably be for some time. I had it about 70% filled prior to the economy taking a dump. A couple machines in there are just for screwing around with (Sparc / Irix) but the rest are there for work and crunching. Pulled a dual quad Xeon server out of it when I lost my job to make ends meet, and recently pulled older 2.66GHz & 3.06GHz Prestonia core Xeons out of it because of the cost to keep them running. Once things pick up again, I'll be filling it up again thougn.