Introducing the Hyperconverged Home Lab!

I was asked the other day why I’ve never written a blog post about my home lab, which I use to test out my scripts and record demos etc, and my answer was “I honestly don’t know?!”. So, I present to you my beloved hyperconverged home lab:

I first unveiled it to the public at the inaugural ZertoCON Boston 2016 and it was certainly a hit. I answered more questions about the lab than I did on Zerto! Questions such as “that looks awesome, what the hell is it?” and “I want one, what’s the build spec?”. At this point you are probably asking the same things and wondering what makes it hyperconverged? Here goes:

• 4 motherboards in 1 m-ATX case totaling 24 logical CPUs, 96GB RAM, 6 NICs and 4.7TB storage
• 4 x 2.5” disk built-in NAS
• 2 x integrated gigabit switches (1 front, 1 back, both USB powered)
• 2 x 1TB HDD and 2 x 512GB SSDs
• 2 x 120mm silent blue LED fans to cool everything (does a surprisingly good job)
• 1 400w silent  PSU
• 1 external power and ethernet cable for the whole lab

Cool huh? My favorite part has to be the Win-D frame m-ATX case. I first saw it while walking through Akihabara, Tokyo in September 2013 (an amazing 30th birthday present!) and had a Wayne’s World moment of “you will be mine, oh yes, you will be mine”. It’s not cheap, but it certainly looks good and turned out to be very flexible too. Soon after buying the case, with nothing to put in it yet, all the open space gave me the idea. It seemed a bit of a waste to simply put 1 motherboard in there considering I needed multiple ESXi hosts and vCenters to demo disaster recovery. Why not put 2 in and use a Y splitter cable from the PSU to power them both? To do this I glued together 2 L shaped brackets from a DIY store then used multiple motherboard mount screws to elevate and stagger the 2 motherboards on top of one another. Simple, crude, yet effective:

After installing 2 motherboards I still had an awful lot of space leftover so I thought to myself, why stop there? I had a few Intel NUCs from my previous lab lying around so I decided to take 2 of them, de-case them, then add 8 more mounts on top of the existing motherboard. I also managed to squeeze in the network switches and a 4 disk NAS to give them all shared storage and voila:

The 4-motherboard hyperconverged lab was born! One of the NUCs is powered using a standard 4-pin molex, the second has its own power supply for a very good reason. It runs my VPNServer VM (using SoftEther, can’t recommend this enough) which allows me to remote in, reboot the main motherboards and never lose connectivity.

I also have 3 standalone Intel NUCs, each with 16GB RAM and a 256GB mSATA SSD, with custom built all anodized aluminum Zerto red cases:

This further builds out the lab to a total:

• 7 ESXi hosts
• 36 logical processors, 144GB RAM, 9 NICs
• 2.3TB flash storage, 4.7TB spinning disk

Not too shabby for a lab running on 2 small shelves in the corner of the room operating at a barely audible whisper! Many of these components are getting a bit long in the tooth as I built this back in late 2014. I would certainly look to up-level the majority if building it today and I’ve been reading bad things about the Intel C2750 CPUs in the ASRock motherboards which has me worried! However, to help you choose the up to date equivalent, here is what I used to build it:

• Win-D frame m-ATX case:
  https://www.newegg.com/Product/Product.aspx?Item=N82E16811108133
• Seasonic 400w Platinum Fanless PSU:
  https://www.newegg.com/Product/Product.aspx?Item=N82E16817151097
• 2 x m-ATX motherboards
  https://www.newegg.com/Product/Product.aspx?Item=N82E16813157475
• 1 x Kingston 64GB RAM kit:
  https://www.amazon.com/dp/B00NLZANJO/ref=psdc_172500_t1_B00KUSMVVS
• 2 x 2.5” 512GB SSD:
  https://www.newegg.com/Product/Product.aspx?Item=N82E16820148821
• 2 x 2.5” 1TB HDD (for the 2 m-ATX motherboards):
  https://www.newegg.com/Product/Product.aspx?Item=N82E16822145881&cm_re=HTS721010A9E630-_-22-145-881-_-Product
• 1 x Synology DS414 Slim NAS:
  https://www.amazon.com/Synology-Station-Network-Attached-DS414slim/dp/B00KXQD7WY
• 4 x 1TB HDD for the NAS:
  https://www.newegg.com/Product/Product.aspx?Item=N82E16822178340
• 1 x Corsair Air series 120mm twin pack fan:
  https://www.newegg.com/Product/Product.aspx?Item=N82E16835181072
• 1 x Power Supply Y Splitter:
  https://www.moddiy.com/products/Power-Supply-ATX-20%252dPin%7B47%7D24%252dPin-Cable-Y-Splitter.html
• 8 port gigabit switch:
  http://www.tp-link.com/en/products/details/TL-SG1008D.html#specifications
• 5 x Intel NUCs (can be bought on eBay, newer models available, each also needs SODIMM and mSATA):
  http://www.intel.com/content/www/us/en/nuc/nuc-kit-dc3217iye-board-d33217gke.html
• Custom Intel NUC anodized aluminum fanless cases created by the excellent Tranquil PC in Manchester (minimum order of 10, FYI I have 2 spare!):
  http://www.tranquilpc.co.uk/
• Check out their cool NUC story here:
  http://www.tranquilpc.co.uk/nuc_story.html

So, what was the total build cost for the hyperconverged home lab? I try not to make an exact figure as I’d feel guilty, but it was certainly around $4,000 mark and significantly more when you factor in all the additional NUCs. I’m sure you could build the same thing for less today, especially if you don’t spend $1,500 on the custom NUC cases!

I hope you found this interesting and the next time I post a script at least you will know what I tested it on. Happy scripting (or lab building),

Joshua