Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
World Community Grid Forums
Category: Community Forum: Hardware Chat Room Thread: My setup |
No member browsing this thread |
Thread Status: Active Total posts in this thread: 49
|
Author |
|
ericinboston
Senior Cruncher Joined: Jan 12, 2010 Post Count: 251 Status: Offline Project Badges: |
Hi everyone. @hchc (not sure if he will get the alert with that tag) asked me to describe my setup since I have about 2300 years of WCG crunching.
----------------------------------------I would like to start off that I have been with the project 13+ years (maybe longer with an older, accidentally deleted account) and have always been Cancer-centric with my crunching other than some long periods of time when there was no Cancer work and I attached to some other projects. There are a lot of ways that people can "donate" in this world and for me, donating to Cancer research has never been about donating cash, it's always been about computer time and actual research. I would rather use my cash to buy machines and actually see results and feel part of a community. But everyone has their way of giving and I totally respect those ways. For a long time I worked for a great company that gave fantastic discounts on brand new machines as well as pretty good discounts on used, older employee machines. So I spent, to my wife's rolling eyes, a lot of my time and money looking for machines to buy over a period of 10+ years. Now on to the hardware. For the past 4-5 years, I've basically been running dozens of machines in my house: • 10 Lenovo P330 desktops with Intel Core i9-9900 vPro (3.10GHz, up to 5.0GHz with Turbo Boost, 8 Cores) • 10 Lenovo M710 desktops with Intel Core i7-7700 Processor ( 8 Threads 3.6GHz ) • A handful of random Wintel laptops running 8 or 16 cores • 2 Intel iMacs from 2019 I believe • 1 Mac Mini M1 • 4 Mac Mini M2 • 2 Macbook Air M1 • 1 Macbook Air M2 • A handful of older i7 machines from ~2018 that I have given to family and WCG is still running on them. Outside my home: • 1 Macbook Air M2 that I bought for my dad • A handful of Wintel desktops and laptops that I donated to friends that still run WCG I would have to guesstimate my home electric bill is about $150/month just to run all these computers. While IBM/Krembil took a year to migrate the project, I powered everything off. But yes, I think about $150/month (I live in New England and electricity is very expensive if you ask me!). The nice thing is that 1)I run these devices in the basement where the temp is still 67 at the highest with them all running and 2)for ~6 months of the year I can open the window and let in plenty of cold air ranging from 10 degrees (F) in the cold winter to 40 (F) in Fall/Winter months. I certainly do not mind the heat these machines generate for our winter season! What I am finding is that the new Mac Mini M1 and M2 machines are about 80% the crunching power of the P330 machines HOWEVER use far less electricity, generate far less heat, and are dead silent. I love my P330s but man do these Mac Minis rock. I bought the P330s for about $720 each with my steep discount but these days the Mac Mini M2s are $500 on everyday sale (I buy the base) so I think I may lean more towards Macs going forward. I also am hopes in that as the years pass, I will recoup some of my money by selling the machines in bulk to local schools or similar. For example, if I bought the P300s for $720 with my steep discount (they were valued over $1100 at the time), that means right now they should be valued at about $500 (and sounds about right with CPU comparison to Mac Mini M2)...the new owner shouldn't know/care what I paid for them...just what they are valued at now. And boy do these machines scream as well as have fantastic upgrade ability. I will add that although I am no way a Mac Fanboy, the Macs are sooo much easier to maintain because they don't harass me every 3-4 weeks to upgrade or complete some stupid Windows 10 reboot wizard. That's really, really, really annoying and it began with Win10. If and when the Macs reboot (power outage or if it does apply a patch) never ever ever does it prevent BOINC from running while it waits for me to click OK or Skip for some dumb option. I have no idea how or why Microsoft thinks it's ok to reboot my machine AND prevent it from doing a normal startup/login so it can harass me to check out/setup some seemingly cool new feature that's about as hip as Clippy (if you remember that horrific thing). At this point people may be wondering about how much money I've spent on devices just in my basement and crunch 24x7. I would say probably $7500 every 4 years beginning in 2014 I think. I am currently buying about 10 more Mac Mini M2 machines. I really want to stress that I am not at all trying to brag about spending this kind of money for machines that do nothing but crunch data. I am super passionate about helping to find some kind of cure(s), medication for Cancer. I'm also a very, very techie person (yet I have given up on following GPU technology since about 1999) and believe so much good can come from distributed computing and even brute force computing. Again, I don't think it's worth my money to donate $7500 in cold cash to <name one of 10,000 Cancer foundations>. Nor would I have any actual real world understanding of where my donation went if I did. There are folks on non-BOINC projects that have awesome GPU-based machines and some of those rigs are $3500+ due to their GPU cards. I wish WCG would leverage the GPUs to some extent. Even if the GPUs can only crunch at 10% efficiency, it's still better than 0! Maybe there are too many variables with GPU programming but it just seems like a missed opportunity...especially with my setup with so many flavors of Wintel and Mac machines...NO GPUs can be used at all to any extent? Really? A machine sitting running for 24 hours can't let the GPU complete 8 WUs? The GPUs just need to sit there and stare at the motherboard? :) On average my 30 Wintel and Mac desktops are completing ~150 WUs a day EACH. They can't EACH add 8 more WUs that were processed on the GPU? I find that hard to believe that a desktop CPU (CPUs have been very generic for 30+ years now while GPUs are extremely focused) could process 150 but it's so inefficient for a GPU to even crunch 8 so the programmers don't bother to write the code. On a side note, I am finding the Mac Mini M2 is only about 10% faster than the Mac Mini M1. I wish I could find the Mac Mini M1 for like $400 (M2 is $500) in bulk but I can't find a better price than $489 which is nuts. Why would I buy a 3+ year old (technology) machine to literally save $10? I'll buy the M2 version and let it hold its value. I know I rambled a bit but hope this was useful and maybe even helpful information. If you have any questions I would be happy to answer...whether technical, simply about me and my background, or my approach. [Edit 2 times, last edit by ericinboston at Dec 5, 2023 2:04:00 PM] |
||
|
hchc
Veteran Cruncher USA Joined: Aug 15, 2006 Post Count: 747 Status: Offline Project Badges: |
Thanks Eric. I read the whole thing and appreciate the details, and I relate to the rambling, so no worries. That's a nice little way to heat the house (especially in New England) and keep the pipes from freezing lol.
----------------------------------------I've bought half a dozen machines from Dell Refurbished dot com. A couple from their clearance sales, and the rest from the main section. They have 50% off coupons for Black Friday and back to school, and 40% off coupons every couple months. I've gotten some rigs for $99 + tax or $150 or $200 or so. I plan on using one of the micro form factor machines as a Home Assistant server (way way more powerful than a Raspberry Pi but only idles at 3 Watts or so!). I'd go completely crazy managing a farm like that, especially if they were running some flavor of Windows (Home, Pro, whatever) because Windows is constantly phoning home 24/7 and the data is pretty vulgar. I've been meaning to do a very detailed, step-by-step guide on WCG on how to install a very lean, headless, Linux BOINC machine for beginners so will do that in 2024 hopefully once I spin up a new Linux box. Also in 2024 I'll try to buy a 24U server rack or cabinet and clean up the basement "datacenter" so it doesn't look like a rat's nest of a home lab. I'm pretty partial to density, so if I had thousands burning a hole in my pocket, I'd get some EPYC or Threadripper machines and build a 4U rackmount server loaded with 64, 96, or 128 cores and some discrete GPUs and use that as the crunchbox. It might consume 400-1200 Watts by itself though. $150 electric bill -- is that for just the computers or is that the entire household??? Either way that's not bad at all, and it's justified in the cold months up North. Thanks for the post. I'm the same way re: cancer research. Thing is, I don't want to put all my computing eggs in one basket, so it's nice to donate computing power to multiple research labs, since research moves at such a snail's pace and it may take 10-20 or more years before anything is useful. I just wish millions more people around the country/world knew about distributed computing. There's so much computing power just doing nothing, and it's hard winning people's hearts and minds to projects like these because they still think they need to understand computers or science to contribute. Definitely need more publicity and how-to guides for the common layperson.
[Edit 1 times, last edit by hchc at Dec 5, 2023 6:32:42 AM] |
||
|
ericinboston
Senior Cruncher Joined: Jan 12, 2010 Post Count: 251 Status: Offline Project Badges: |
I just wish millions more people around the country/world knew about distributed computing. There's so much computing power just doing nothing, and it's hard winning people's hearts and minds to projects like these because they still think they need to understand computers or science to contribute. Definitely need more publicity and how-to guides for the common layperson. I've been pushing the same feelings for 10+ years. I don't understand how/why WCG can't better promote themselves. Please, WCG, don't tell me it's because you run a shoestring staff that are 100% volunteer and only work a total of 11 hours a week combined across all employees. There are people (like me and others on this forum) that would love to help promote WCG from an employee-type role. Meaning, I want a dedicated staff member (volunteer or paid) who will commit to promoting WCG (Social Media, digital print ads, digital magazines, an article in Time Magazine, etc.) like promoting any other business. When WCG got started with IBM, the first few years there was some promotion, but then it seemed to completely stop. It's truly sad that in a world with over 1 billion humans that use at least 1 computer every day, WCG has barely 100,000 volunteers. 1% of a billion would be 10 million volunteers...we can't get 10 million volunteers to contribute? Not even 1 million? The answer in my eyes is 2 fold: 1)WCG is simply not promoting itself. Period. 2)Although the BOINC app is relatively easy to use (at least for us techies), it can be better for the completely tech newbie that wants to help but not want to "figure it out". I know WCG doesn't write the BOINC app but it's feedback for WCG to share with Berkeley: a)There could be a few bundled choices of Computer Preferences such as "Run all the time 100%" at the end of the installation wizard b)There could be better explanations of errors/problems in the Notices tab or other areas of the GUI c)There could be a few more options in Computer Preferences such as "Run at 25% cpu power while on batteries" d)a much better Statistics area (I prefer the Advanced View of BOINC) rather than 1 boring old stat on how many WUs the host is crunching...why not basically replicate what's on the website Overview page? EDIT (I forgot to add this earlier): Also, why not allow users to crunch anonymously? I bet there is a high abandon rate by newbies who want to contribute but find out that they a)need to create a login just to download the app, b)then they need to AGAIN enter that login info during the BOINC setup, c)then then need to YET AGAIN enter that login info every few days just to see their stats. I can easily see lots of people saying to themselves "dude, this is nuts". WCG/BOINC should allow people to just download the darn tool, start crunching anonymously, and if they want to associate their work with a particular WCG volunteer (maybe a friend that referred them) they can type in that volunteer's ID number (and save them even more time than joining a Team). If they don't want to associate the work with a volunteer then fine...have 1 big, fun stats page for Anonymous User that maybe breaks down all the work by region, CPU style, OS, etc. #1 is the biggest blocker in my opinion. #2 would be a bit more of a nice-to-have. Heck, #2 could simply be implemented with a 3 minute video on the WCG website AND a step-by-step script with numerous screenshots on the WCG website. I'm sure some of these exist in various locations of the internet, but why do we expect someone to go Google how to use a tool that is pitched as a screensaver utility? Anyway, we're off topic a bit for this thread but I did want to reply. [Edit 1 times, last edit by ericinboston at Dec 6, 2023 3:43:24 AM] |
||
|
Boca Raton Community HS
Advanced Cruncher Joined: Aug 27, 2021 Post Count: 114 Status: Offline Project Badges: |
Thanks for sharing the details of your set up- I also would go crazy maintaining all of these, but at the same time, most of us enjoy (somewhat) that aspect as well. Really awesome that you have contributed so much time, effort and finances into this project!
|
||
|
ericinboston
Senior Cruncher Joined: Jan 12, 2010 Post Count: 251 Status: Offline Project Badges: |
Thanks for sharing the details of your set up- I also would go crazy maintaining all of these, but at the same time, most of us enjoy (somewhat) that aspect as well. Really awesome that you have contributed so much time, effort and finances into this project! Thank you. Yes, I have to say I believe going forward for WCG I'm going to dump Windows. It just constantly reboots and waits for user input before it will start BOINC. I'm done with Microsoft's pestering me to press the "skip" button. Apple, at least so far, plays so much nicer on reboots. |
||
|
bikeaddict
Cruncher Joined: Apr 11, 2020 Post Count: 32 Status: Offline Project Badges: |
Distributed computing wasn't something that interested me until the start of the pandemic in March 2020. I heard about Folding@Home on Reddit and decided to put a couple i7 quad core machines to work. I later also contributed to Rosetta@Home, DreamLab, WCG, QuChemPedIA@home and TN-Grid. A couple old phones were also put to work for a while.
After buying a couple more used quad core boxes, I upgraded a Xeon machine to a used 8-core CPU off eBay. I quickly learned that multi-core CPUs are the only way to really get a lot of computing power with fewer boxes to maintain and lower power consumption. Then I got a couple used HP Z420 workstations and put 8 and 10-core Xeons in them. Then I found a Z440 and put a 14-core Xeon in. Then a couple more identical Z440s. One was upgraded to a 16-core CPU. A lot of computing power could be bought for just $300-500 on eBay. Now I've upgraded to Ryzen 9 machines that are faster and more power efficient. An all core 32-thread load uses about 220-250W without even trying to undervolt or tweak much. It's about 3-4 times as fast as the old Xeons. The GPU journey was similar with a GTX 1650 Super to start, then moving to more and more powerful models. You should reconsider the strategy of using tons of smaller machines. A small cluster of Ryzen 9 machines should be more energy efficient and lower maintenance. |
||
|
ericinboston
Senior Cruncher Joined: Jan 12, 2010 Post Count: 251 Status: Offline Project Badges: |
After buying a couple more used quad core boxes, I upgraded a Xeon machine to a used 8-core CPU off eBay. I quickly learned that multi-core CPUs are the only way to really get a lot of computing power with fewer boxes to maintain and lower power consumption. Then I got a couple used HP Z420 workstations and put 8 and 10-core Xeons in them. Then I found a Z440 and put a 14-core Xeon in. Then a couple more identical Z440s. One was upgraded to a 16-core CPU. A lot of computing power could be bought for just $300-500 on eBay. Now I've upgraded to Ryzen 9 machines that are faster and more power efficient. An all core 32-thread load uses about 220-250W without even trying to undervolt or tweak much. It's about 3-4 times as fast as the old Xeons. The GPU journey was similar with a GTX 1650 Super to start, then moving to more and more powerful models. You should reconsider the strategy of using tons of smaller machines. A small cluster of Ryzen 9 machines should be more energy efficient and lower maintenance. Thank you for the info and advice. A few points that maybe you can elaborate on: 1)I really don't want to be building or configuring physical machines. I loved doing that 30 years ago when I was in my 20s but I simply do not have the time for it with family and stuff. 2)As for buying machines on eBay or elsewhere, I actually posted a new thread this morning about if such a Sticky Thread is somewhere on these forums to help us see great deals and go buy the machine easily. 3)I agree on not having tons of "smaller" machines...but I'm not sure what you mean. Physically or core wise smaller? Physically, yes, the Mac Minis are extremely small and that is super helpful for storing them all in a house (as well as 0 noise and almost 0 heat and very lower power usage). Some of the Wintels are Small Form Factors and the other Wintels are actually decently thin so I'm good all around for the physical aspects of the machines. From a core/thread perspective, all the Wintel machines are 16 cores and the Mac Minis are 8 cores. The only way I'm going to get more than 16 cores in the Wintel world is to start buying physically larger machines that are servers or workstations and I don't want to have large machines all over the house. Also, I agree that there is a balance of having tons of machines that a)require power cords and b)require network setup/management. I don't want to have 100 machines on my router/switch and I don't want power strips all over the house. I am more than welcome to hear about $300-$500 machines you and others can point me to to purchase that would crunch more than my $500 Mac Mini M2s. But I'm also a Linux novice so that's basically a no-go. And I've about had it with Windows 10 constantly nagging me after a reboot. So any machines I buy would truly need minimal setup and just run 24x7 forever without babysitting. Lastly, from what I understand, WCG does not utilize the GPU at all (other BOINC projects do). And I'm also only contributing to the Cancer projects on WCG. |
||
|
bikeaddict
Cruncher Joined: Apr 11, 2020 Post Count: 32 Status: Offline Project Badges: |
These are what I was using in past years. These aren't as efficient as more modern hardware, but for someone with cheap power and wants a space heater, a good deal.
- Barebones HP Z440 workstation, $250 a couple years ago, now $134 https://www.ebay.com/itm/384284924158?ViewItem=&item=384284924158 - Add a 14-core Xeon E5-2690-v4 ($250 a couple years ago, now $25) or 16-core Xeon E5-2697A-v4 ($50) https://www.ebay.com/sch/i.html?_from=R40&...w=e5-2690+v4&_sacat=0 https://www.ebay.com/sch/i.html?_from=R40&...=e5-2690+v4&_osacat=0 Add any cheap SATA SSD. Add a high power GPU to do GPU tasks, or a cheap Quadro NVS 295 or similar for just video out. My machines were built from components, but I do have the free time and knowledge to do that after becoming interested in computer hardware again in 2020 after losing interest back in the late 90s and just buying laptops and later a HP Z240 prebuilt. I can't give much for recommendations on prebuilts, but maybe look at mini-PCs with Ryzen from Minisforum or Beelink. It's hard to compare performance between Mac Mini and PCs. There's Geekbench, which shows an all-core score of 9748. https://browser.geekbench.com/macs/mac-mini-2023-8c-cpu That's about like an old Ryzen 5700X or 5900 and way behind the latest Ryzen 9 models over 19,000. But the Macs may run cooler and quieter. You just need a bunch of them to match the performance. All my machines run Fedora Linux and it's not that hard to install and maintain, but I also have a lot years of Linux experience. One advantage of PC is having an upgrade path. I will be able to upgrade to the next couple generations of CPU with my current motherboard. And I can swap in any more powerful GPU that comes out. |
||
|
ericinboston
Senior Cruncher Joined: Jan 12, 2010 Post Count: 251 Status: Offline Project Badges: |
Just as an FYI...I ran a kill-a-watt tool today and found the following data for my systems that run MCM on WCG 24x7 full throttle:
----------------------------------------Mac Mini M1 machines use 27 watts Mac Mini M2 machines use 27 watts Lenovo P330s I-9900 vPro use ~100 watts as it fluctuates from 96 to 102. Now, both the Mac Mini M1 and M2 machines essentially use 1/4 the electricity of the P330s, yet yield about 75% of what the P330s crunch. Given other benefits of the Mac Minis (physical size, lower amount of heat, dead silent, and far easier to manage from a WCG point of view regarding reboots and OS patch reboots), I would say the Macs win hands down overall. The P330s were $741 each 4+ years ago with my super duper employee discount (regular sale price would have been about $850). The Mac Mini M2s are easily found for $500. So it's really comparing $850 vs $500, too. So in about 4 years, Mac Minis are almost as fast (for WCG at least) as a 4 year old high end Intel machine yet at 25% the energy consumption. I'm not sure what today's Intel chip would match the M2 crunching power. Are the Mac Minis going to crunch the most WUs per day compared to some Wintel machines? No. But given the $500 price tag and all the benefits I listed above, I would give the Mac Mini a very high recommendation. I love my P330s but I'm very much considering figuring out how to donate/sell them asap so I can replace them with Mac Minis. The P330 machines are each worth about $550 but I would likely sell them for $450...they are extremely expandable, have a dvd drive, USB 3.1 Gen2, Wifi, M.2, NVME, and more so they would make great non-WCG machines. [Edit 3 times, last edit by ericinboston at Dec 7, 2023 2:25:25 PM] |
||
|
Sgt.Joe
Ace Cruncher USA Joined: Jul 4, 2006 Post Count: 7581 Status: Recently Active Project Badges: |
75% of the throughput versus 25% of the energy use seems like a definite win.
----------------------------------------Cheers
Sgt. Joe
*Minnesota Crunchers* |
||
|
|