Just get a 12 core+ Ryzen and a bunch of RAM (64GB+). It'll be more efficient/faster GHz than a bunch of old servers and more than capable for running three node clusters.
Honestly, depending on what you're doing? Not even that.
* Don't underestimate the number of services you can having running in containers on a Raspberry Pi 4B with 8GB of RAM, sipping just 2.1W (yes, that little!) at idle over PoE or as low as 1.35W over WiFi powered by USB-C. With distributions like DietPi you can have a mere 10 processes at startup (using just 44MiB of RAM), but still all the flexibility in the world.
* Also don't forget you can cluster them. Want Kubernetes experience? Perfect opportunity to have a cluster than pulls at most like 17W under extreme loads.
* Have an old laptop? Fantastic. Great second life for it. You get the bonus of an integrated keyboard, UPS and display if and when you need it, even if it's normally headless.
* Don't forget Mac Minis and 1L form factors. I have an 1L form factor (that weighs about 1.3kg) 8-core Zen 3 Ryzen 7 PRO APU that boosts to 4.6GHz, with 64GB of memory, a 1TB SSD and 2.5GbE that sips a mere 10W at idle, and pulls maybe 45W at full bore. Its silent. takes up as much space as like 25 paper napkins, and I can pass the GPU cores through to VMs.
* Don't forget hypervisors as well. For truly ephemeral stuff you can just spin up virtual machines on your main desktop or laptop -- just stuff more RAM in it.
-- ----- -----
What's great is you can put the power sipping stuff basically ANYWHERE. It can be easily accessible, but entirely hidden from view. It makes no noise. It's not spewing heat or demanding cooling. There's no blinking lights. You're not driving up the electricity bill. Yet you can do very real work on them.
For cost? Raspberry Pis aside (sorry, use rpilocator!) check eBay or other marketplace completed listings to establish a price floor, and just be patient and only grab quality stuff that approaches the price floor. Nothing better than getting new (but open box) or barely used modern hardware for peanuts (60-70% off) because the seller is impatient or ignorant.
The Deskmini x300 is good. Takes everything from 2400g up to 4750g (or whatever it is). 2x nvme ssd and can fit 2x 2.5 drives. You can also buy barebones for ~$150 (everything but storage, RAM, and CPU).
Had x300 for quite a few years but have started playing games again, so decided to trade in for a desktop...but will probably go back if RDNA 2 is good/x300 gets updated for it (imo, the iGPU on the 4750g is terrible...RDNA 2 is way better). Great for running VMs on though, can take a Noctua NH-L9 so is quiet, far less space-intensive.
> * Don't underestimate the number of services you can having running in containers on a Raspberry Pi 4B with 8GB of RAM, sipping just 2.1W (yes, that little!) at idle over PoE or as low as 1.35W over WiFi powered by USB-C. With distributions like DietPi you can have a mere 10 processes at startup (using just 44MiB of RAM), but still all the flexibility in the world.
I've been disappointed in the general reliability, terrible IO, and heat/power of the RPI4. I'm getting rid of them in favor of second hand thin clients.
> Also don't forget you can cluster them. Want Kubernetes experience?
Just don't be thinking about mixing ARM and x86 in the same Kubernetes cluster. Like even if you technically can run a multi-arch kubernetes cluster, it's a major footgun. Docker's multi-architecture support leaves a lot to be desired and is a real pain to work with.
HP ProDesk 405 G8 Mini, or HP EliteDesk 805 G8 Mini.
The ProDesk 405 G8 can take a 35W RYZEN 7 PRO 5750GE. This includes AMD DASH (IPMI), and the Ryzen Pro has a few extra security features from EPYC/Threadripper. I got mine shipped for just $665 including tax, and did the SSD (1TB 980 Pro) and memory (64GB DDR4-3200 CL16) myself for another $300ish.
If you get three of them, you can have a pretty fantastic Spark cluster that consumes only about 100-120W! But yes, they do XCP-NG or Proxmox clusters extremely well.
The EliteDesk 805 G8 can have the 65W Ryzen 7 PRO 5750G, has a copper heatsink (instead of aluminum), has a 2nd NVMe slot, and can be shipped with a 2.5GbE port (you can order the part for the ProDesk 405 G8). Though if you get the 35W 5750GE, you can have an NVIDIA GeForce 1660Ti 6GB installed as well.
Since both use Ryzen PRO processors, they actually support ECC memory as well. I'd argue they may be the best overall 1L form factor systems on the market, especially in terms of performance per watt. The only other thing I'd ever want is 10GbE instead of 2.5GbE -- though in the EliteDesk 805, you COULD get an NVMe to 10GbE Ethernet and install it, just need to make a faceplate to properly mount the port on the back of the chassis.
Did you get it yet? Multiple reports of 1-month-plus shipping delay notifications after the order is placed. I ordered one of these myself and canceled after the delay. Still plan on ordering one albeit with a different config.
Originally I opted for the 5700GE (apparently slightly faster than the 5750GE) thinking I wanted the most power possible in the box. But my new thinking is I really want a 5300GE to pull out and put in an AM4-based silent HTPC for the living room, then upgrade the HP with a separately purchased 5700GE. QuietPC seems the only vendor selling these "GE" 35W models -- but not any 53X0GE parts. In other words buying one of these non-vendor-locked HP minis and ripping them out might be the only way on Earth right now to get one's grubby hands on any 53X0GE.
Also, are you sure there are two M.2 slots on the EliteDesk G8's? I know they are on the G6 but haven't found confirmation the G8's continue the tradition.
Look at the TMM series at servethehome.com. They go over vendors, models, specs of the 1L segment as well as the new/used market for them.
It's generally a really good form factor for a small server. The only things most people sometimes wish they had in them is 2.5-10GBE instead of the stock 1GB, and formal ECC support.
I went the "prosumer" route too. Threadripper maxed with 256G memory.
It runs esxi that auto boots my "workstation" vm on startup. This vm has my GPU, nvme, and usb passthru.
This way I can boot my workstation "on metal" if needed, but the vm works great. I then manage my lab infra running on the same machine via vCenter html5 ui.
Why ESXi just out of curiousity? Seems like most businesses I work with are ripping it out (some with glee) and going acropolis/kubevirt/QEMU or cloud.
* Best support for PCIe passthrough, easily toggle PCIe devices from UI without reboot.
* Best networking with virtual distrubuted switches - more intuitive to manage and use
* vSAN integration with k8s for block volumes is great, including volume snapshots. Also supports RWX (nfs) volumes
* vSphere api is a first class citizen with cloud native tooling such as terraform, ansible and packer. It also has a great golang-based cli via govc.
With all that said I actually still experiment with kvm (kubevirt, harvester etc) by running it nested inside esxi which works exactly how you would expect. Much prefer vSphere as the base of my lab though.
For really modern stuff sure, but a lot of enterprises are not using containers, they're still Hyper-V or VMware based. The only reason to go Hyper-V is because it's cheaper, so if you can use ESXi free... you choose ESXi.
ESXi remains one of the most ridiculously and stupidly stable platforms of all time. Someday someone will find an ESXi host with a twenty year uptime. Not because you should, but because it can.
Pretty much spot on, though the ceiling is a lot lower, at least in the US.
-- -----
A 12-core Ryzen 9 5900X is now $395 everywhere in the US, though a patient soul can get one on new eBay for $370-375 before sales tax (but including shipping), best $350 used.
DDR4-3200 CL16 from top brands isr $3.50-$3.65/GB new, or under $250 for 64GB. Again, a patient soul can save more still and get it closer to $200. Want 32GB? Cut those prices in half.
So ~$530-650 new with sales tax and shipping, with a floor of maybe $455-575. For a tremendously capable processor and 32-64GB of memory.
I guess that's pretty cheap. I've also wondered about a Mac Mini running a bunch of Linux VM's. I've to this date never seen a piece of hardware as reliable as the Mac Mini, based on my own and anecdotal evidence.
