At the start of the year I implemented what I had hoped was the ultimate home server solution for my needs. Of course that wasn’t going to last forever, it was inevitable that I’d start messing with it eventually. Unfortunately, I had to start the messing part sooner than expected.
As a brief reminder, the setup consisted of a Optiplex micro thin client with built-in 500GB NVMe and 1TB SSD, 32 GB of DDR4 memory and an attached 2x2TB mirrored external drive. That was running a Proxmox with several instances for various purposes. All of my MVs were stored on the NVMe drive while the 1TB one was passed through to one of them as a big storage place. The idea was that in case of hypervisor failures I can just take the drive, plug it to any other linux machine and have data there. The idea, in my opinion, was still good, but I misjudged the ability of other things to go wrong.
Well… the external drives failed first, which forced me to invest into a separate, dedicated NAS device, but I wrote about it in a separate article. Long story short now most of the storage was handled by a separate box. With the NAS part outsourced, I discovered that the most problematic part remaining on the Optiplex server was the janky passthrough. While it worked just fine most of the time, I was storing all the data on the VM with the additional 1TB drive which included: databases, docker images, shared directories, web root dirs for websites, home dirs and Nextcloud storage – all of which was shared to proper VMs over NFS. That sometimes caused one VM to get overwhelmed and crash even with 16Gigs of RAM I gave it. On top of that, the NVMe drive Proxmox and all of its children stored wasn’t really designed to be working 24/7 for nearly a year now and is clearly degrading faster than I’d like.
To fix that, I gradually emptied the 1TB drive and added virtual disks to VMs with the data they needed and everything else was moved to the NAS box. Next step on my to-do list is to migrate databases to a separate VM and leave only docker stuff on that overworked server. The now empty 1TB SSD will be reassigned in Proxmox to store virtual disks giving me more space to move things around the NVMe and hopefully prepare it for cloning to a more server-friendly medium some time next year.
As for the home lab in general, I have the other Proxmox machine with TrueNAS instance on it, which, while not the most power-efficient, can work as a backup hypervisor, but I haven’t tested it in this role yet.
I’m also considering a little investment in 2.5Gb networking, at least between gaming PCs and the NAS, but since my ISP doesn’t provide anything near those speeds, I’m not in too much of a hurry on that front.
