Setting Up to Develop with Chef Data and Cookbooks

Think of the chef repo as your snowflake, or the data that makes your organization unique.

One of the hardest challenges, at this stage, is keeping the unique pieces of your data contained in such a way that it supports the principles we touched on at the beginning of this series.

When you work with a cookbook for an organization you need to do it in an agnostic way that separates your data from the structure. Each organization should have a private chef repository and a set of publicly supported cookbooks. The practice of working in a public space helps developers pay attention to the code they are writing. It helps identify if code (or configuration) is a structural definition that can be shared or data to protect or expose for customization. If public work is performed properly, it can be done safely and increase the benefit of your organizations efforts.

Thinking of the chef server as your systems meta data, while not entirely accurate, is a great way to start. Let’s walk through the scenario of a developer interested in making a modification to an existing cookbook. The developer is tasked with creating a file on a server that contains the companies slogan. The developer is also tasked with doing this in a test driven fashion that is reusable.

The first step is to pull down the chef cookbook that is being worked with. The cookbook should pass all of it’s tests before work commences.

For the purposes of this example we will use the nmdbase cookbook.

git clone [email protected]:newmediadenver/nmdbase ~/nmdbase

After the cookbook has been verified the developer associates it with their organizations data by cloning the chef repository. This should be a private repository.

git clone [email protected]:newmediadenver/nmdchef ~/nmdchef

The cloned chef repository should contain the roles and data bags most developers will be interested in.

Connecting Cookbooks with Chef Data

An effective method for keeping sensitive data out of code is proper use of variables. Each cookbook that you work with should have a .kitchen file (like the one in nmdbase) that allows you to launch/test a server with that recipe applied to it. In order for our test kitchen to read the chef repositories data we need to set an environment variable. In this example, we are setting the data bag path for the nmdbase cookbook to the data_bags directory in the nmdchef repository. When we do this, it gives the developer the extra benefit of working with data in the same context of the server it will be deployed on.

export NMDBASE_DATA_BAGS_PATH=$HOME/nmdchef/data_bags

It is a good practice to keep in mind that we could switch this environment variable to another chef repository. This may seem like an unnecessary item to be aware of it, but it is important. Let’s say that you have cookbook Foo. You need to support client A and client B. These two clients require the same technology, but configured in different ways. You can easily isolate the structural changes in the Foo repository and then validate the entire testing suite against client A, or client B’s data. It’s another form of checking that helps make sure we are creating reusable infrastructure.

Understanding the Layout.

Let’s take a quick look at some of the common components you will find in a chef cookbook. Our unit tests live in nmdbase/spec while our integration tests live in nmdbase/test

├── .gitignore                # Tells git what files to ignore.
├── .kitchen.yml              # Contains our kitchen configurations and test suites
├── .rubocop.yml              # Configures the Rubocop Linter
├── .travis.yml               # Configures integration with TravisCI ex. https://travis-ci.org/NEWMEDIAdenver/nmdbase
├── Berksfile                 # This only ever contains a reference to metadata which is generally in metadata.rb
├── Berksfile.lock            # Locks Berksfile dependencies to versions.
├── Gemfile                   # Manages runtime and development dependencies.
├── Gemfile.lock              # Locks Gemfile dependencies to versions.
├── Rakefile                  # This is our control center.
├── attributes                # attributes are generic pieces of data that can change ex: a path.
│   └── default.rb
├── chefignore                # Tells chef which files to ignore.
├── coverage                  # The recipe may be using something like SimpleCov for test coverage metrics.
│   ├── .last_run.json
│   └── .resultset.json
├── metadata.rb               # Contains all metadata about the cookbook and it's dependencies.
├── recipes                   # Recipes contain the code that performs the provisioning.
│   ├── default.rb
│   ├── ldap.rb
│   ├── ssl.rb
│   └── yubico.rb
├── script                    # This creates a caching layer in s3 so we don't kill TravisCI.
│   ├── cached-bundle
│   └── s3-put
├── spec                      # Our first level of Unit tests go here.
│   ├── default_spec.rb
│   ├── ldap_spec.rb
│   ├── spec_helper.rb
│   ├── ssl_spec.rb
│   └── yubico_spec.rb
├── templates                 # Templates that are used by the recipes.
│   └── default
│       ├── generic.erb
│       ├── ldap.secret.erb
│       └── readme.md.erb
├── test                      # Our Integration kitchen tests go here.
│   └── integration
│       ├── default
│       │   └── serverspec
│       │       └── default_spec.rb
│       └── yubico
│           └── serverspec
│               └── yubico_spec.rb
└── vendor                    # This is where we can download vendor packages.
    └── bundle
        └── .gitignore

Up next we will continue the developers task and write our first test.