LISA14 - Day Four

Submitted by Hugh Brown on November 13, 2014 - 8:10 am

Day Three Four at LISA began with good news from outer space: the Philae spaceprobe successfully landed on Comet 67P! I wasn't the only one scampering around the hotel lobby, giddy with joy...

Back in conference-land, LISA14 Chair Nicole Forsgren Valesquez gave the opening address as the papers and presentations part of LISA14 started. "Good morning!" [mumbles from the audience] "...Okay, let's try that again. GOOD MORNING!" Audience: "Good morning, Ms Forsgren." Turns out, in case you're wondering, there are 1100 of our closest friends in attendance, and that she would really REALLY like us to fill in the survey and give feedback.

I stayed for Ken Pratchett's talk and had my mind pretty much blown by the scale of things at Facebook. But not everyone who can afford to redesign their own data centre, and we're a long way off from being able to ship our Open Compute-certified hardware to an Open Compute-certified data centre. I'm waiting impatiently.

One other thing: I happened to be sitting next to Tom Limoncelli, who clapped enthusiastically when the speaker complained about IPMI. I asked Tom what he hated about it, and it was pretty what I expected: the UI sucks, and it's a security nightmare. It's an embedded Linux system, with full access to EVERYTHING in your server, and between clumsy patching processes or a lack of love/care/maintenance from the vendor, it never, ever gets patched. It sucks but we need the functionality it provides; there's nothing else that does it and there's nothing else on the horizon. My personal conclusion: IPMI is a big ball of pain.

Next up was Wade Minter's talk, "You Code Like A Sysadmin!" It was fun: a lot of entertaining anecdotes about how he can't code to save his life...but the code he *does* write is useful. If it's useful, it lasts and he'll get an opportunity to rewrite later -- in one case, by founding a company on that code grew profitable enough to hire proper programmers to do it. "Did those devs like us? No. But could we have hired them if we'd taken the time to do this properly? I argue no." After that, he invited members from the audience to testify about the crappy code they'd written, and among many others up came Tobi Oetiker. "I wrote MRTG. I still get emails about it." "But it works!" The feel-good moment of LISA14 was in that room, I tell you what.

Up next was Tom's talk about his new book, and what cloud computing changes for sysadmins. I'm not going to quote much here because a) go grab the talk and b) go throw money at Tom and buy the book. But I will relate this: a big point of his talk was that things fail all the time, and we need to be prepared for this; we sysadmins are hired to be awesome in the face of failure.

At lunch I struck up a conversation with a sysadmin from the East Coast who's at his first LISA -- he's wanted to come for a long time, and it's finally working out. We talked about Puppet and Beaker, and how to bolt code tests onto an existing code base, and how and when to test code that takes a long time to test. He's using Docker to do full tests to help speed things up -- and that's a refrain I've heard a *lot* at LISA. Neither one of us have the answers, but we're both doing distributed research by asking everyone we meet.

After that the vendor exhibit, where I mostly avoided the t-shirts (got a lot), but *did* throw money at the EFF table and got this awesome picture of Kurt Opsahl; he's a USENIX board member, Deputy General Counsel for the EFF and lead attorney on the Coder's Rights Project:

If you're not a member, find them in the Vendor's exhibit and join today.

The afternoon was spent drifting between different sections, picking up what I could. From the Google presentation on running services came the aphorism that small-scale stuff fails when scaled up, but large-scale practices scale down quite well. In the LISA Labs I talked with Matt Simmons about a blog post he's drafting ("I just dropped the mic, linguistically speaking") about education and training in our profession (probably to be posted here, so watch this space). I also got to talk with Adam Moskowitz about the history of the Advanced Topics Workshop at LISA, and the terribly knotty problem of maintaining a good culture in a company that grows beyond Dunbar's number.

Avleen and Caroline's talk on avoiding distractions was great. This is something I struggle with, and it was really good to hear a lot of practical tips on dealing with it. I scribbled notes furiously and resisted the temptation to get out my phone and tweet all the funny quotes. Well, mostly:

Supper was spent at Elysian Brewing's restaurant a short walk away; I went there with a couple of friends I've met at previous LISAs.We talked about best practices and why so many places avoid them...but also about the joys of being a new parent, getting fun new jobs and living in places with lots of good craft breweries.

I got back in time for the Puppet vendor BoF, where I took a lot of notes. After that was the BoF I signed up/created/wrote down, on testing infrastructure code: what works, what doesn't, and what we'd love to have that we don't know ("Imagine a unicorn dropped down from outer space and handed you the perfect tool...") It was a really, really productive session; again, nobody had all the answers, but we're starting to feel our way around the edges. I'm going to write up all the notes from both sessions for my own blog.

And then...then the mandatory staying up too late, trying to settle down from all this brain thinking. It's tough, and I'm pretty sure I'm going to need to sleep in for a week afterward.

Tags: 
lisa14