Lustre, Pain, And Goo.

By Snipa on August 12, 2012

Well, it’s 7:13 PM CST at the moment, and I’m shacked up in DFW due to some server upgrades for Goo at the moment. Unfortunately, we’re also recovering from our worst loss yet, a full wipe and deletion of all files stored on Goo’s main file server.

So, what happened? Well, based on what I can tell, when I loaded the the OST into the main MDS for Lustre, it caused a race which ended up blanking out our MDS, requiring a full wipe and reload of both OST’s, and the MDS itself. For those not familiar with the terms, OST is the object storage, or file storage, and MDS is the metadata server. Files are two parts, metadata, and the actual physical file. Unfortunately, when you lose the metadata, you lose all reference to the location on the drive where the physical file is stored. Due to this, the loss was pretty much catastrophic, wiping out every developer’s /home/ drive.

Good news though!
Web01
Distro03
Distro04
Storage02
lgswitch

Have all gone into production (Well, not the switch yet, more on that later.) These boxes will be doing as their names suggest, handling the web frontend, distribution of files, and additional storage space (10+tb now, for anyone asking….) This will take load off the current servers, and help to ensure a good experience overall.

Now, for the switch.
Lgswitch is sexy, and fun, and a blast. But unfortunately, we didn’t receive the switchmodule needed to plug our SFP’s into for the 10gbps fiber drop seen in the pictures below (It’s the yellow loop, if you’re curious.) So we’re stuck on 1gbit for the next two weeks, as I’ll be on call this coming week, and won’t be able to drive to Dallas, however, the following week, I plan on driving up and installing the module. Tuesday if I can wangle the day off, if not sooner, because I WANT MOAR SPEED!

Pictures!

Start:

In-Progress:

Done with current servers:

Done with all installs:

Posted in | 15 Responses

Batch 4

By Snipa on August 5, 2012

Type: Beer
Style: Hefenweizen
Base: Wheat Beer — Coopers
Batch Size: 5 Gal.

Additions:

  1. Additional Wheat Malt – Approx 2lb

Hops:

  1. 1oz German Hallertau – 30 min boil, 4.1% Alpha Acids
  2. 1oz Cascade – 5 min boil, 8.9% Alpha Acids
  3. 1oz Centennial – Added during cooling, approx 10 minutes, 10.3% Alpha Acids

Yeast:

  1. White Labs WLP300
  2. 1 pkt Bru Vigor

Data:

  • SG at Pri Ferment: 1.047
  • SG at Sec Ferment: 1.010
  • SG at Bottling:
  • Primary Ferment: 08/04/2012
  • Secondary Ferment: 08/10/2012
  • Bottled:

Notes:
Wort cooled using copper immersion chiller, inserted at 5 minutes prior to flame out, at the time of the Cascade Addition. Managed to avoid any boil overs. Yeast prepared as part of a starter in 1.040 gravity LME wort, 24h in advance, allowed to chill 3 hours to settle, spare LME wort poured off to clarify the yeast. Pitched at 78 deg F, yeast went right to work. Sitting in fridge at 65 deg f, good solid krausen with a heavy, stable foam. Bubbling approximately 1 bubble every 1-2 seconds. Bit of a sulphur scent, but that’s not unusual according to what I’ve read. No fermaid-K or other additives mixed in, does not appear to be required for something this low of gravity.

Posted in | Leave a response

Batch 3

By Snipa on July 29, 2012

Type: Infusions

General:

  • Bottling Date: 07/25/2012
  • Clean Date: 08/08/2012

Infusion 1:
Alcohol: Vodka, White Gold
Additives:

  • Vanilla

Status:
Settling and clearing

Infusion 2:
Alcohol: Vodka, White Gold
Additives:

  • Saffron

Status:
Settling and clearing

Infusion 3:
Alcohol: Rum, Ron Barcelo Blanco
Additives:

  • Vanilla
  • Oak Cubes

Status:
Settling and clearing

Infusion 4:
Alcohol: Rum, Bacardi Gold
Additives:

  • Vanilla

Status:
Settling and clearing

Infusion 5:
Alcohol: Bourbon, Makers Mark
Additives:

  • Vanilla

Status:
Settling and clearing

Posted in | Leave a response

Batch 2

By Snipa on July 25, 2012

Type: Mead
Style: Mixed, 3 part
Base: Tupelo Honey, 9 Lbs
Batch Size: 3 Gal, split into 1 Gal at tertiary ferment

Additions:

  1. Peaches
  2. Vanilla/Oak
  3. None

Yeast:

  1. Wyeast Dry Mead 4632
  2. Lalvin EC-1118

Data:

  • SG at Primary: 1.178
  • SG at Secondary:1.074
  • SG at Tertiary:
  • PH at Primary: 4.7
  • PH at Seconary: 3.7
  • PH at Tertiary:
  • Primary – 7/24/2012
  • Secondary – 8/4/2012

Fermentation Information:
PID unit setup for primary ferment, setup to run at 65 deg f w/ a 2deg +- for sorting out the compressor run times.
Update 7/26/2012 – Adjusted temp to 63 deg f, as fermentation is progressing nicely at the cooler temp
Update 8/4/2012 – Adjusted temp to 65 deg f, due to the high level of sugars left.

Notes:
Tupelo honey is delicious, and joyous to work with.  Very thin, mixed easily with water.  No boiling involved as there was an excellent floral quality I didn’t wish to lose.  Batch is slated to be split into 3 1 gal batches in 4-6 weeks after secondary clarification.  Batch is heavily oxygenated, 30-45 minutes on the air stone, heavy bubbling during the time, so it seems quite handy to keep the oxygen levels high for primary.  Yeast pack had been unusually warm during shipment, however, it bloated properly, and appears to be fine.

Update 8/4/2012 – Still quite cloudly, racked off primary into 3 gal + 1 gal container to help it settle again. Temperature raised to help ferment out some of the residual sugar. All have received a second dose of Fermaid-K to help assist with the slow fermentation.

Plans:
1 Gal – Peach based Mead, will probably cause an early Tertiary, assuming that the SG drops as planned.
1 Gal – Vanilla/Oak, light roast oak for this, due to the lightness of the honey.
1 Gal – Plain, will be split in 2 at bottling for sparkling vs non sparkling.

Update 08/12/2012:
Mead has been split into 4 1Gal batches at this time, Peaches infused for a week and pulled.  Vanilla/oak still ticking over.  2Gal of plain mead.  Each has received an injection of Lalvin EC-1118 at this time to help take care of the last of the sugars.

Pictures!

Posted in | Tagged , | Leave a response

Batch 1

By Snipa on July 25, 2012

Type: Beer
Style: Chocolate Stout
Base: Irish Stout — Coopers
Batch Size: 5 Gal.

Additions:

  1. Additional Dark Malt – Approx 2lb
  2. Ghirardelli Baking Cocoa — 10 Oz. – 5 min boil

Hops:

  1. ¾ oz US Goldings – 30 min boil
  2. ¼ oz US Goldings – 5 min boil

Yeast:

  1. Provided w/ Irish Stout
  2. 1 pkt Bru Vigor
  3. 5g Go-Ferm
  4. 6.25g Fermaid K

Data:

  • SG at Pri Ferment: 1.083
  • SG at Sec Ferment: Didn’t verify, last tested at 1.020
  • SG at Bottling: 1.010
  • Primary Ferment: 07/21/2012
  • Secondary Ferment: 07/24/2012
  • Bottled: 07/28/2012

Notes:
At time of transfer from brew kettle to primary fermenter (6 Gal bucket), had a very heady chocolate aroma with little scent of hops/malt.  Aerated via dropping into bucket, did not provide a huge amount of air, however, it should be sufficient.  Yeast pitched a bit cold, allowing to warm up in the brew room and get started.  Very active at time of addition.

Posted in | Tagged , | Leave a response