Building The Perfect Beast

Imagine you are a moderately-successful but unknown pipe organ builder in early 18th-century Leipzig, with far more ego than brains, and a mouth bigger than both.  One evening, you go to a neighboring parish to listen to a recital by that famous organist and composer J. S. Bach.  The music is lovely and captivating, but you can’t stand the sound of the instrument he’s playing on.  Right in the middle of Herr Bach’s performance, you stand up, loudly cry, “I could build a more suitable organ from my son’s discarded tin whistles!”, throw a handful of small coins into the offering basket, and finish with “Here – get yourself a better one!

The next morning, a footman delivers a note from the gentleman, asking if you would be interested in building him such a musical instrument.  Cold sweat breaks out on your brow, and your stomach twists itself in knots around your breakfast.  Me – build a new organ – for Bach?!  The honor would be immense.  The cost of failure would be catastrophic.

Transpose this tale to the early 21st century and the key of hacking, and you can see how chutzpah seduced me into offering to build a new high-end computer workstation for Eric S. Raymond.  We’re friendly, from afar, but I’m not fit to even clean his keyboard and screens, much less attempt to give him a tool that would meet his exacting standards and huge requirements.

This is not to say that I’m completely unsuited to the task.  I’ve been a Unix user and programmer for more than 30 years, and I’ve had ‘root’ for all but a couple of those years.  I’ve supported more than 8 different versions of Unix professionally (and that counts every Linux distro I’ve ever touched as only one of those versions), often several variants simultaneously, for communities of up to 1000 users.  I use Linux (Ubuntu 14.04 LTS) exclusively on my personal system.  I know what I’m talking about, and I’ve done nearly as much doing as talking.

But still – Eric.  S.  Raymond.  *gulp*

To be precise, I didn’t build up the hardware of the “Great Beast of Malvern”.  That honor and responsibility went to Wendell and Logan of, an online PC components retailer targeting the high-end gamer and modder market.  I was just liable for getting the system software installed and configured, and assisting Eric in getting his home directory and other environment moved off his current desktop and onto this fire-breathing monster.

So everything converged on the first week of December.  My wife was out of town to visit her mother.  That freed me for all of each evening to work with Wendell to install the system.  Eric had already run a short script I had supplied him, which gave me an outline of the configuration of his old system, and a list (several thousand items long!) of the packages he had installed.  I knew he also ran Ubuntu, so we agreed that 14.04 LTS would be most suitable.  Given the huge disks going into his new box, I recommended configuring his filesystems under LVM, to give him resizing and snapshot flexibility.  Wendell had a motherboard that was almost identical to the one going into the new box, but it had an IPMI configuration so that I could get a remote console, for booting and adjusting the BIOS.  (He’s in Kentucky, I’m in northwest Ohio.)  And the other HDD wouldn’t be available, so I would have to format and partition it after the fact.  He also had a flight booked to Eric’s place for Saturday morning, so the box had to be finished by then.

No sweat.  Easy-peasy.

(Cue ominous music….)

Tuesday night Wendell and I started a Google video Hangout after work, and just chatted about my background and the background of this project.  Next thing I knew, nearly four hours had passed, and I had to wrap it up.  Well, this might be a little more involved that I first thought.

Wednesday, we tried to get me connected to the IPMI.  Hm, this needs Java and the Java Web Start packages.  And it wanted to use non-standard ports, so Wendell’s firewall had to be adjusted.  And re-adjusted.  And Google Chromium didn’t want to run javaws, so I had to tweak Firefox into submission.  And the video of the GUI was laggy as hell, which made the feedback of a mouse click or keystroke agonizing.  And…  And…  Five hours later, I was beginning to sweat.  I hadn’t added a single package to the box yet, hadn’t partitioned one disk, hadn’t done a damned thing towards the actual project.  And I couldn’t work on it Thursday night, because an unavoidable piece of Real Life got in the way.  It would all come down to what I could get done on Friday.

Friday evening, I put on my lucky “LISA in Chicago 1996” T-shirt, and Wendell and I started for the last time after we each finished work.  He had the installer ISO mounted on the machine.  I  c a r e f u l l y  typed the command to zero out the SSD (wouldn’t do to smash your installer!), and rebooted to the installer image.  Now the Ubuntu 14.04 installer won’t let you set up the disk with LVM, so I used these directions to hack the installation process.  Hours melted away as I checked and double-checked each command – no time for errors now.  After I got the basic install done, I copied a script onto the box which contained better than 600 lines of “apt-get” commands for that long list of packages Eric had previously been using.  No problem, I thought, “apt-get” won’t error out if you try to install a package that’s already been installed.  And it drags in all the requirements for a package when it installs it, so the order of installation doesn’t make any difference.  Thus, I had written the script’s hashbang line with “-xe” to give me a trace and to die if something else broke.

Something else broke.

Seems Eric’s old box had some obsolete packages installed, which weren’t available anymore.  So I cursed, and edited the script to take out every line up to the broken one, and took out the name of the missing package.  And ran it again.

(Betcha you can guess what happened.)

Something else broke.

This time, it was some package (don’t remember – inn2?) that needed some install-time configuration I hadn’t previously captured.  Interrupt the script, copy it, delete everything down to the broken line.  Make a note to Eric that he would have to do this one himself. Lather, rinse, repeat.

Then something else broke.

By the 10th iteration of this nightmare, sometime more than 6 hours later, I just threw caution to the winds, removed that pesky “-e” from the hashbang line of the package installer script, and let it rip one last time.  It thrashed and stumbled to the “exit 0” at its end.

Whiskey Tango Foxtrot.  Did everything install? (Obliviously not.)  Did enough install?  (Hope so!)  I shut the system down for the last time, Wendell and I called it a wrap and stopped the video recording, and I went home, worrying about how this would all look in Malvern some 19 hours later.  Fortunately, Herr Raymond likes the sound of his new organ.

Oh, by the way – if you ever hear me muttering about “tin whistles”  or “bitty boxes” under my breath again –

Kindly beat some sense into me!


Edited (2015-01-01) to add:  Here’s the first video from TekSyndicate.


This entry was posted in about hacking, geeky friends. Bookmark the permalink.

One Response to Building The Perfect Beast

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.