I came in last week to a UPS indicating that it had zero runtime. Everything was up including a test server that wasn't connected to the UPS so it wasn't a power hit. I hit the test switch which *should* have beeped, gone from line power to battery power and then back again but instead rebooted everything attached to the device!
Not good. After a few minutes the UPS started indicating that it had a charge again but I was past trusting it. So I ordered a new UPS and hoped that power wouldn't flicker until it got here.
So tonight I downed everything and moved all of knitebane.net to the new UPS. It's a smarter UPS and I can do more monitoring of via its USB monitoring connection than the old UPS bu that required running some USB cable and removing the old serial cables. I took the opportunity to do a bit of housekeeping by pulling some dead systems and unused wiring out of the rack and after a couple of little bumps we're back in business.
I'll probably order new batteries for the old UPS and replace them. It's quite likely that it's just a bad battery but I'd rather have a new, reliable system with the old one as a spare than wonder if every time the power flickers everything is going to reboot.
With all the systems I've virtualized I'm down to just a couple of physical systems so using a KVM system now makes more sense than a serial console server so that box got retired. I've got a few more infrastructure changes coming up like trunking VLANs across both the external and internal switches and de-cabling the desktops in the office in favor of 802.11n wireless cards but I have some hardware purchases to make before that happens.
So far I'm enjoying the freedom that the virtual machines gives me. I can spin up a test system in minutes rather than build a hardware-based box and making it a live production system is a matter of a software configuration change rather than a re-cabling and re-racking job. This has made the migration away from Google services much easier once I got past having to build a new VM server to handle the additional load.
It's almost there. The new mail/calendar/chat/cloud server is up and running and sometime in the next week or so I'll migrate all of the mail from the old system to the new. That's the hardest and most tedious part but once it's done I'll not need to touch it again for a few years. The hardest part is remembering how to use the tools to migrate the email without losing anything. The old email server has been running since 2009. You kind of forget what you did the last time when it's three years.