Sean Kilgore

logikal is : black shirt ninja

A Month to Myself

“What are you going to do with all that time off?” is the first thing everyone asks me.

I’m taking a month to myself before I start at SendGrid.

It’s not a vacation.

I’m considering this a part-time job, working for myself. There’s three major things I’m going to do this month:

Recover

If you’ve worked in gaming or startups, you know about crunch time. We did almost 5 months of crunching at Beachhead and got a two week vacation at the end of it. I haven’t done any of my best work since November of last year, nor have I been very motivated.

I haven’t been able to find a lot of data on the time-to-recovery after long-term overtime. Anecdotal evidence suggests it’s pretty lengthy. Two weeks wasn’t enough for me. I want to get to SendGrid ready and motivated to do awesome work again.

Learn

I’ve wanted to learn more Ruby for a couple of years now (I’ve never needed it on the job). This is my chance to really dig down and immerse myself in Ruby, not “just enough Ruby for Chef”.

Heck, maybe I’ll dabble in some JS, too. Write some coffeescript for hubot. Maybe even try out some Puppet! It’ll be an adventure.

I joke about always re-evaluating your toolset, but it’s actually a really good rule to live by. Since I don’t have anything “in production”, augmenting my toolset now should be a low-cost, low-friction change.

Repair Relationships

I’ve been commuting 86 miles (about 2 hours between Orange County and Los Angeles) a day for the last year, in addition to 10AM-7PM hours and crunch time. This left me with a schedule that made it very difficult to hang out with friends, see my family, or even spend time with my wife.

No more. Relationships are an important contributor to our long-term happiness and health, and I intend to start treating them like that.

Testing Chef 11 With Vagrant

Chef 11 has come out, and possibly broken a bunch of your carefully crafted cookbooks. How are you going to test them, though? Here’s one way, using chef-solo via Vagrant.

Requirements

  • RVM (or rbenv, but this will use RVM)
    • You’re using RVM or rbenv to manage your rubies, right? If not, you should be.
  • Virtualbox
  • some cookbooks you wrote and want to test

Set it up

We don’t want to mess up our carefully crafted Chef 10.x environment, right? We’ll use rvm gemsets to make a disposable set of gems. If something goes wrong, just close the terminal you’re in, or run rvm gemset use default. You’ll drop back to the default gemset.

DO NOT install RVM if you already use rbenv. They’re incompatible, and you don’t want to mess with that.

1
2
3
4
cd $HOMEBASE
rvm gemset create chef11
rvm gemset use chef11
gem install chef vagrant

NOTE: update the homebase = to be the full path to your homebase.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
cat <<VAGRANTFILE > Vagrantfile
homebase = "/path/to/homebase"
Vagrant::Config.run do |config|
  config.vm.box = "opscode-ubuntu-12.04-chef11"
  config.vm.box_url = "https://opscode-vm.s3.amazonaws.com/vagrant/opscode_ubuntu-12.04_chef-11.2.0.box" # from https://github.com/opscode/bento
  config.vm.network :hostonly, "10.111.222.33"
  config.vm.customize ["modifyvm", :id, "--memory", 2048]
  config.vm.provision :chef_solo do |chef|
    # chef.log_level = :debug
    chef.cookbooks_path = "#{homebase}/cookbooks"
    chef.data_bags_path = "#{homebase}/data_bags"
    chef.roles_path = "#{homebase}/roles"
    # Replace the following lines with your roles/recipes
    chef.add_recipe "apt::default"
    #chef.add_recipe "foo::bar"
    # How to add a role:
    # chef.add_role "foo"
    # How to set node attributes (like an environment)
    chef.json = {
      :load_limit => 42,
      :chunky_bacon => true
    }
  end
end
VAGRANTFILE

Do some magic

vagrant up

At this point, you should have an ubuntu 12.04 machine starting up, using Chef 11.2.0. It’ll run the apt::default recipe (assuming you have it), and throw errors if there are any.

Notes

Adding Recipes/Roles

  • chef.add_recipe "foo::bar" to add a recipe
  • chef.add_role "baz" to add a role

Environments/Node Attributes

If you’re used to environments, you can run knife environment show <envname> --format json to get an environment as json from your chef-server. You can also do knife node show <nodename> -a node -f j to get all of the attributes from a particular node you already have. This gist can be adapted to get your json into the ruby ‘hashrockets’ syntax. Then you’ll need to update the chef.json in your Vagrantfile with your attributes.

What about test-kitchen?

As of this writing test-kitchen’s master branch doesn’t support Chef 11. Once it does, you should definitely use it.

Further reading

Cleaner Chef Environment Files With Ruby

The problem

The syntax used in all the examples of chef environment files have something similar to the following:

dirty environment attributes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
"default_attributes": {
"apache": {
  "prefork": {
    "startservers": "30",
    "minspareservers": "5",
    "maxspareservers": "30",
    "serverlimit": "65",
    "maxclients": "60",
    "maxrequestsperchild": "1000"
  }
}
},
"override_attributes": {
}

This gets really really unwieldy, really quickly when you have over a thousand environment variables. It gets even worse when you have an attribute like this:

json format looooong attribute example
1
2
3
4
5
6
7
8
9
10
11
12
13
default_attributes": {
  "application": {
    "partner": {
      "service": {
        "api": {
          "platform": {
            "stat": "foo"
          }
        }
      }
    }
  }
}

That’s equivalent to:

1
node['application']['partner']['service']['api']['platform']['stat'] = "foo"

This format should look more familiar to you. Doesn’t this look like one of your cookbook attribute files?

default attribute files
1
2
3
4
5
6
7
# Prefork Attributes
default['apache']['prefork']['startservers'] = 16
default['apache']['prefork']['minspareservers'] = 16
default['apache']['prefork']['maxspareservers'] = 32
default['apache']['prefork']['serverlimit'] = 400
default['apache']['prefork']['maxclients'] = 400
default['apache']['prefork']['maxrequestsperchild'] = 10000

So why do it the ugly default way?

Don’t. Clean it up with a little Ruby. Put this in your environment file:

how to use ruby to make your chef environment files cleaner (this is a bad example)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$environment = Hash.new{|h,k| h[k]=Hash.new(&h.default_proc) }
$override = Hash.new{|h,k| h[k]=Hash.new(&h.default_proc) }

$environment['apache']['prefork']['startservers'] = 16
$environment['apache']['prefork']['minspareservers'] = 16
$environment['apache']['prefork']['maxspareservers'] = 32
$environment['apache']['prefork']['serverlimit'] = 400
$environment['apache']['prefork']['maxclients'] = 400
$environment['apache']['prefork']['maxrequestsperchild'] = 10000

$override['apache']['prefork']['startservers'] = 16
$override['apache']['prefork']['minspareservers'] = 24
$override['apache']['prefork']['maxspareservers'] = 24
$override['apache']['prefork']['serverlimit'] = 400
$override['apache']['prefork']['maxclients'] = 400
$override['apache']['prefork']['maxrequestsperchild'] = 10000

default_attributes(Chef::Mixin::DeepMerge.merge($_default_environment, $environment))
override_attributes($override)

It’s better, but still pretty dirty. Make it even better.

Take all those default environment settings and put them in $HOMEBASE/environments/common/apache.rb, then use Ruby to include them.

use includes in chef environments!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
$environment = Hash.new{|h,k| h[k]=Hash.new(&h.default_proc) }
$override = Hash.new{|h,k| h[k]=Hash.new(&h.default_proc) }

require File.expand_path("./common/apache.rb", File.dirname(__FILE__))

$override['apache']['prefork']['startservers'] = 16
$override['apache']['prefork']['minspareservers'] = 24
$override['apache']['prefork']['maxspareservers'] = 24
$override['apache']['prefork']['serverlimit'] = 400
$override['apache']['prefork']['maxclients'] = 400
$override['apache']['prefork']['maxrequestsperchild'] = 10000

default_attributes(Chef::Mixin::DeepMerge.merge($_default_environment, $environment))
override_attributes($override)

Wow, that’s a lot cleaner, anything else?

This arrangement allows you to keep things that are the same across all environments managed in common files, separate from your environment files. We use this kind of arrangement to test things in certain environments, especially for new features or load testing new perfomance tuning variables. Once we know what we want, it makes it much easier to copy the $override attributes to the common files as defaults, instead of using the json format.

Chef Environment Files That Are Easy on the Eye

The problem

The syntax used in all the examples of chef environemnt files have something similar to the following: Ever get tired of looking at chef environment files with default attributes that look like this?

dirty environment attributes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
"default_attributes": {
"apache": {
  "prefork": {
    "startservers": "30",
    "minspareservers": "5",
    "maxspareservers": "30",
    "serverlimit": "65",
    "maxclients": "60",
    "maxrequestsperchild": "1000"
  }
}
},
"override_attributes": {
}

This gets really really unwieldy, really quickly when you have over a thousand environment variables. It gets even worse when you have an attribute like this:

json format looooong attribute example
1
2
3
4
5
6
7
8
9
10
11
12
13
default_attributes": {
  "application": {
    "partner": {
      "service": {
        "api": {
          "platform": {
            "stat": "foo"
          }
        }
      }
    }
  }
}

That’s equivalent to:

1
node['application']['partner']['service']['api']['platform']['stat'] = "foo"

This format should look more familiar to you. Doesn’t this look like one of your cookbook attribute files?

default attribute files
1
2
3
4
5
6
7
# Prefork Attributes
default['apache']['prefork']['startservers'] = 16
default['apache']['prefork']['minspareservers'] = 16
default['apache']['prefork']['maxspareservers'] = 32
default['apache']['prefork']['serverlimit'] = 400
default['apache']['prefork']['maxclients'] = 400
default['apache']['prefork']['maxrequestsperchild'] = 10000

So why do it the ugly default way?

Don’t. Clean it up with a little Ruby. Put this in your environment file:

how to use ruby to make your chef environment files cleaner (this is a bad example)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$environment = Hash.new{|h,k| h[k]=Hash.new(&h.default_proc) }
$override = Hash.new{|h,k| h[k]=Hash.new(&h.default_proc) }

$environment['apache']['prefork']['startservers'] = 16
$environment['apache']['prefork']['minspareservers'] = 16
$environment['apache']['prefork']['maxspareservers'] = 32
$environment['apache']['prefork']['serverlimit'] = 400
$environment['apache']['prefork']['maxclients'] = 400
$environment['apache']['prefork']['maxrequestsperchild'] = 10000

$override['apache']['prefork']['startservers'] = 16
$override['apache']['prefork']['minspareservers'] = 24
$override['apache']['prefork']['maxspareservers'] = 24
$override['apache']['prefork']['serverlimit'] = 400
$override['apache']['prefork']['maxclients'] = 400
$override['apache']['prefork']['maxrequestsperchild'] = 10000

default_attributes(Chef::Mixin::DeepMerge.merge($_default_environment, $environment))
override_attributes($override)

It’s better, but still pretty dirty. Make it even better.

Take all those default environment settings and put them in $HOMEBASE/environments/common/apache.rb, then use Ruby to include them.

use includes in chef environments!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
$environment = Hash.new{|h,k| h[k]=Hash.new(&h.default_proc) }
$override = Hash.new{|h,k| h[k]=Hash.new(&h.default_proc) }

require File.expand_path("./common/apache.rb", File.dirname(__FILE__))

$override['apache']['prefork']['startservers'] = 16
$override['apache']['prefork']['minspareservers'] = 24
$override['apache']['prefork']['maxspareservers'] = 24
$override['apache']['prefork']['serverlimit'] = 400
$override['apache']['prefork']['maxclients'] = 400
$override['apache']['prefork']['maxrequestsperchild'] = 10000

default_attributes(Chef::Mixin::DeepMerge.merge($_default_environment, $environment))
override_attributes($override)

Wow, that’s a lot cleaner, anything else?

This arrangement allows you to keep things that are the same across all environments managed in common files, separate from your environment files. We use this kind of arrangement to test things in certain environments, especially for new features or load testing new perfomance tuning variables. Once we know what we want, it makes it much easier to copy the $override attributes to the common files as defaults, instead of using the json format.

Who Pays for Your Conferences?

I find myself paying for conferences I want to attend, as my employer’s training budget got cut this year. Some people are offended that my company isn’t scrambling to send me. They say I shouldn’t even go if the company won’t pay for it.

I find myself paying for my conferences just to make sure that I get a ticket, more than anything else. Some of these events are sold out before I even hear back from my boss.

Even if the company wasn’t offering to pitch in (they are), I’m willing to spend my own money to attend these conferences. It makes sense; I’m likely to be the person the education benefits most.

If you’re going to Monitorama, MtnWestRubyConf DevOps Day, or DEFCON, let me know. We should hang out and hack.

Open Source Is Awesome

I recently needed to be able to mount EBS volumes during a chef run. I figured someone had already solved that problem by now. I found this cookbook by titanous which did the job well. The only problem was that the assumption titanous had made regarding how to handle authentication didn’t work for me. I have two AWS accounts (dev and prod) and they don’t share keys. I live dangerously and only have one chef server, though.

Because this is open source we’re talking about, I made it work for me. The beauty was that I gave my changes back to titanous. I’d made a mistake that he caught, and then he merged my changes in. Now anyone who follows the same path I did, and has a setup like mine will having a working system out of the box.

Awesome.