Well, it’s 7:13 PM CST at the moment, and I’m shacked up in DFW due to some server upgrades for Goo at the moment. Unfortunately, we’re also recovering from our worst loss yet, a full wipe and deletion of all files stored on Goo’s main file server.
So, what happened? Well, based on what I can tell, when I loaded the the OST into the main MDS for Lustre, it caused a race which ended up blanking out our MDS, requiring a full wipe and reload of both OST’s, and the MDS itself. For those not familiar with the terms, OST is the object storage, or file storage, and MDS is the metadata server. Files are two parts, metadata, and the actual physical file. Unfortunately, when you lose the metadata, you lose all reference to the location on the drive where the physical file is stored. Due to this, the loss was pretty much catastrophic, wiping out every developer’s /home/ drive.
Good news though!
Have all gone into production (Well, not the switch yet, more on that later.) These boxes will be doing as their names suggest, handling the web frontend, distribution of files, and additional storage space (10+tb now, for anyone asking….) This will take load off the current servers, and help to ensure a good experience overall.
Now, for the switch.
Lgswitch is sexy, and fun, and a blast. But unfortunately, we didn’t receive the switchmodule needed to plug our SFP’s into for the 10gbps fiber drop seen in the pictures below (It’s the yellow loop, if you’re curious.) So we’re stuck on 1gbit for the next two weeks, as I’ll be on call this coming week, and won’t be able to drive to Dallas, however, the following week, I plan on driving up and installing the module. Tuesday if I can wangle the day off, if not sooner, because I WANT MOAR SPEED!