The Adventures of Greg DevOps Travel Make Lifehack

Managing your information diet

How many emails do you have waiting for you each morning? 10? 50? 500?

Managing your information diet is going to give you amazing gains in getting productivity back, AND improve the response time to the really important messages you get.

I believe too much information is worse then not enough information. If your pager didn’t go off, you suffer an outage and you get an irate phone call from your boss. If your pager is going off every 5 minutes because your monitoring sucks you are going to be extremely frazzled AND you are going to miss that one alert that REALLY means your site is down and you STILL get that irate call.

So what can you do to manage the emails you are getting?

First thing, EVERY monitoring alert should be followed up by an action. Either you are fixing a broken server, or you are tuning the false alert. You have to be vicious about chasing down and fixing false alerts. Don’t let yourself become numb to them.

Next, SPLIT your HIGH traffic messages from your LOW traffic messages. While you may want to follow every Subversion check-in in engineering or know every time your ISP replaces a router line-card in Azerbaijani, be sure these aren’t also going to your group’s discussion email list. It may be nice to be able to refer back to them, but they should not be polluting your INBOX.

Get adept at the mail filtering tools you have at your disposal I’m partial to procmail, just because it is what I’d grown up with. Do your best to only have ACTIONABLE email land in your INBOX, everything else should get filtered into a folders that you can reference at your leisure.

Finally, do something SMART with your system mail. If you have hundreds of nodes and they all send you a note every time they rotate logs or burp on a cron job that is bad.

I built a NNTP server for all system mail. All of that mail gets thrown into an NNTP news-group. So I don’t have to look at it, but when I need to investigate a problem we can always refer back to several days of history.

Managing your communications, particularly your email, is one of the keys to going from being a good sysadmin to a great one. A little effort on the front end will reap great gains. The better you manage your incoming email, the better your piece of mind, AND the faster you’ll be able to respond when a serious incident occurs.