Using Knife and Knife.rb for Managing Databags

Securely deploying sensitive data with Chef can be achieved through the use of encrypted databags, which can be managed easily with a properly configured knife.rb file.

If you’re using Chef as your configuration management solution, you are probably familiar with it’s powerful command line utility, knife, which allows one to interact with the chef server to manage its cookbooks, nodes, roles, and databags. For the purpose of this article, we will be focusing specifically on using knife to read and write databags (both encrypted and non-encrypted), which are typically used to store and deploy sensitive data (such as passwords or SSL certificates) outside of a version control system. In addition to the basic commands, we will also demonstrate how to leverage environment variables within a user agnostic knife.rb.

Prerequisites

This article assumes you have a working Chef server. It also assumes you have generated and downloaded the server’s validation_key and client_key. Please see the Public and Private Keys Opscode documentation if you need assistance in generating these two files.

Creating a User Agnostic Knife.rb

As stated by Opscode: “A knife.rb file is used to specify the chef-repo-specific configuration details for Knife.” And by placing this file directly within a repo (Expected location ~/.chef/knife.rb), you allow anyone to clone the repository, cd into the repo directory, and immediately begin interacting with the chef server if they have the appropriate credentials.

In order to achieve this, the knife.rb file must contain environment variables.

# See http://docs.opscode.com/config_rb_knife.html for more information on knife configuration options
current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "#{ENV['NMDCHEF_NODE_NAME']}"
client_key               "#{ENV['NMDCHEF_CLIENT_KEY']}"
validation_client_name   "#{ENV['NMDCHEF_VALIDATION_CLIENT_NAME']}"
validation_key           "#{ENV['NMDCHEF_VALIDATION_KEY']}"
chef_server_url          "#{ENV['NMDCHEF_SERVER_URL']}"
cache_type               'BasicFile'
cache_options( :path =>  "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{current_dir}/../cookbooks"]

Notice! We’ve namespaced the environment variables with the NMDCHEF_ prefix to target a specific repo. This is recommended because it allows one to easily work with multiple chef servers.

The next step is for the individual using this chef repo to add these environment variables to their shell configuration files. Using zsh, one could add the following to the ~/.zsrch file.

export NMDCHEF_NODE_NAME=example_username
export NMDCHEF_CLIENT_KEY=$HOME/.ssh/example_username.pem
export NMDCHEF_VALIDATION_CLIENT_NAME=example_clientname-validator
export NMDCHEF_VALIDATION_KEY=$HOME/.ssh/example_clientname-validator.pem
export NMDCHEF_SERVER_URL=https://api.opscode.com/organizations/example_organization_name
export NMDCHEF_SECRET_FILE=$HOME/.ssh/example_encrypted_data_bag_secret

If done correctly, you should be able to cd into the main repository folder, run knife status, and get a response from the chef server.

Creating, Updating, and Reading Unencrypted Data bags

Creating data bags is a straightforward process. For the purposes of this article, we will create two: example_encrypted and example_unencrypted. For testing purposes, we will use the following json file.

{
  "id": "example_data",
  "_default": {
    "db_user": "example_database_username",
    "db_password": "example_database_password",
    "admin_user": "example_username",
    "admin_pass": "example_password"
  }
}

We’ll create the unencrypted databag first using the following command:

knife data bag create unencrypted

Next, we will populate that data bag with the sample json data listed above.

knife data bag from file unencrypted example.json

To verify this worked, we can verify the databag exists…

knife data bag list

…and then view its contents…

knife data bag show unencrypted

Now your cookbooks can use this information during a chef run!

Creating, Updating, and Reading Encrypted Data bags

You’ll note that, in the previous example, simply having access to the chef server gave us access to all the data bag contents without any additional access control checking or granularity. If there are many users within this chef server, this might not be desirable. The solution is to use a secret file. The process of interacting with the databags is almost identical with the exception of referencing the secret file used to encrypt/decrypt the data bag.

First let’s create a new data bag entry.

knife data bag create encrypted

Next let’s use the same json with a reference to the secret file’s environment variable.

knife data bag from file encrypted example.json --secret-file $NMDCHEF_SECRET_FILE

Now if you use knife data bag show encrypted example, you will see the following:

_default:
  cipher:         aes-256-cbc
  encrypted_data: zDE61IUD97ZK7O6Eq11lkJ+5EhBFv7g7vvWfi50hU3pGZPYGnZ35pIWFsdUE
  lYeeRJd7OYpNwhfu3WGaLnL5ZXl+S7qm018JI7F5Cc4LTDMBpuzxozc3n8p6
  OFswlpoagRLNQFs0t4oYqWok4qQpdg==
  iv:             1w+bQ46evg8jZWBs0MZW6A==
  version:        1
id:       example

If you then specify the secret-file on the command line, you’ll get the original values back.

_default:
  admin_pass:  admin
  admin_user:  example_admin
  db_password: drupal
  db_user:     drupal
id:       example_data

Using this approach, it is possible to store sensitive data directly in the repo without exposing those credentials to the wrong individuals.

Summary

We’ve demonstrated how to create a user agnostic knife.rb for an arbitrary chef repo to allow users to quickly begin interacting with a chef server. We then demonstrated how to use this setup to begin reading and writing encrypted databags to the chef server to protect any sensitive data necessary for a chef deployment. For additional information on knife, please visit the Opscode documentation.