A challenge faced by users of Open Source lies in understanding how to properly evaluate and use a given piece of code.
Let’s walk through evaluating and getting set up to work with an opscode chef cookbook in the context of our ideal system.
These examples will focus on our nmdbase cookbook from a development team perspective. The steps outlined are applicable to any cookbook.
The first step is to clone the cookbook so that we can evaluate it locally.
$ git clone https://github.com/newmediadenver/nmdbase.git ~/nmdbase
$ cd ~/nmdbaseAny time we work with a cookbook we do it in a completely isolated fashion. This helps reinforce the agnostic nature of the structure we are defining. This means that dependencies are managed per cookbook.
$ bundle install
Using rake 10.3.0
Using i18n 0.6.9
Using multi_json 1.9.2
Using activesupport 3.2.17
Using addressable 2.3.6
Using builder 3.2.2
Using gyoku 1.1.1
Using nokogiri 1.5.11
Using akami 1.2.1
Using ast 1.1.0
Using backports 3.6.0
Using buff-ruby_engine 0.1.0
Using buff-shell_out 0.1.1
Using hashie 2.1.1
Using chozo 0.6.1
Using multipart-post 1.2.0
Using faraday 0.8.9
Using minitar 0.5.4
Using rbzip2 0.2.0
Using retryable 1.3.5
Using buff-extensions 0.5.0
Using varia_model 0.3.2
Using buff-config 0.4.0
Using buff-ignore 1.1.1
Using hitimes 1.2.1
Using timers 2.0.0
Using celluloid 0.14.1
Using nio4r 1.0.0
Using celluloid-io 0.14.1
Using erubis 2.7.0
Using json 1.8.1
Using mixlib-log 1.6.0
Using mixlib-authentication 1.3.0
Using net-http-persistent 2.9.4
Using net-ssh 2.8.0
Using solve 0.8.2
Using ffi 1.9.3
Using gssapi 1.0.3
Using httpclient 2.3.4.1
Using little-plugger 1.1.3
Using logging 1.8.2
Using rubyntlm 0.1.1
Using rack 1.5.2
Using httpi 0.9.7
Using nori 1.1.5
Using wasabi 1.0.0
Using savon 0.9.5
Using uuidtools 2.1.4
Using winrm 1.1.3
Using ridley 1.5.3
Using thor 0.18.1
Using berkshelf 2.0.15
Using chef-zero 2.0.2
Using diff-lcs 1.2.5
Using highline 1.6.21
Using mime-types 1.25.1
Using mixlib-cli 1.4.0
Using mixlib-config 2.1.0
Using mixlib-shellout 1.4.0
Using net-ssh-gateway 1.2.0
Using net-ssh-multi 1.2.0
Using ipaddress 0.8.0
Using systemu 2.5.2
Using yajl-ruby 1.1.0
Using ohai 7.0.2
Using coderay 1.1.0
Using method_source 0.8.2
Using slop 3.5.0
Using pry 0.9.12.6
Using rest-client 1.6.7
Using chef 11.12.2
Using fauxhai 2.1.0
Using rspec-core 2.14.8
Using rspec-expectations 2.14.5
Using rspec-mocks 2.14.6
Using rspec 2.14.1
Using chefspec 3.4.0
Using docile 1.1.3
Using simplecov-html 0.8.0
Using simplecov 0.8.2
Using tins 1.1.0
Using term-ansicolor 1.3.0
Using coveralls 0.7.0
Using ethon 0.7.0
Using faraday_middleware 0.9.0
Using gherkin 2.11.8
Using polyglot 0.3.4
Using treetop 1.4.15
Using foodcritic 3.0.3
Using net-http-pipeline 1.0.1
Using gh 0.13.2
Using net-scp 1.2.0
Using safe_yaml 1.0.2
Using test-kitchen 1.2.1
Using kitchen-vagrant 0.14.0
Using launchy 2.4.2
Using net-ldap 0.6.1
Using parser 2.1.7
Using powerpack 0.0.9
Using websocket 1.1.2
Using pusher-client 0.5.0
Using rainbow 2.0.0
Using ruby-progressbar 1.4.2
Using rubocop 0.20.1
Using typhoeus 0.6.8
Using travis 1.6.9
Using vagrant-wrapper 1.2.1.1
Using websocket-native 1.0.0
Using bundler 1.6.2
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.Rake – A Task Executor
Once the dependencies are met we need to see what tasks have been exposed to us. Note that we begin wrapping all of our commands in bundle exec; savy users may create an alias when they get tired of typing it.
$ bundle exec rake -T
rake foodcritic # Lint Chef cookbooks
rake integration # Alias for kitchen:all
rake kitchen:all # Run all test instances
rake kitchen:default-ubuntu-1204 # Run default-ubuntu-1204 test instance
rake kitchen:ldap-ubuntu-1204 # Run ldap-ubuntu-1204 test instance
rake kitchen:yubico-ldap-ubuntu-1204 # Run yubico-ldap-ubuntu-1204 test instance
rake kitchen:yubico-ubuntu-1204 # Run yubico-ubuntu-1204 test instance
rake readme # Generate the Readme.md file
rake rubocop # Run RuboCop style and lint checks
rake spec # Run ChefSpec examples
rake test # Run all testsThere are a fair number of tasks available to us and it is important that we understand how to use them. We can begin by breaking them into two categories: testing and utility. The primary task that should be run by continuous integration systems like JenkinsCI or TravisCI is ‘rake test,’ but let’s go through each of the types.
- rake foodcritic – Foodcritic is a lint tool for Chef cookbooks.
- rake integration – This is an alias to run all of our integration tests for this cookbook. Integration tests are normally driven by test kitchen and are tied into custom data.
- rake kitchen:default-ubuntu-1204 – Examples like this task speak to test kitchen. The naming convention states that we are running the default recipes integration tests against ubuntu-1204. When you invoke the integration test in this fashion, the resulting instance is destroyed when the test run is complete.
- rake readme – This task is used to generate the README.md file. This serves the purpose of documentation in addition to reinforcing proper utilization of meta data in the cookbook.
- rake rubocop – Rubocop is something that I have a love hate relationship with. I love it because it makes my code a thing of beauty. I hate it because it points out repeatedly that I seldom write it that way the first time. It is an invaluable lint tool for Ruby.
- rake spec – This runs the RSpec testing suite. This suite is an in memory set of tests that can simulate extraordinary things without requiring a virtual machine. This is your first layer of testing.
At this point in the evaluation, everything is looking good. The next step is to validate that there is reasonable test coverage in place. At this juncture, it could take considerable time to run through all tests because kitchen will create and destroy virtual machines. Let’s see what this cookbook can do without involving any virtual machines.
$ bundle exec rake spec
/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby -S rspec ./spec/default_spec.rb ./spec/ldap_spec.rb ./spec/ssl_spec.rb ./spec/yubico_spec.rb
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's default settings.
nmdbase::default
Includes the fail2ban recipe.
Configures this instance as an LDAP client.
Configures this instance as an Yubico API client.
Configures this instance as a chef client.
Configures the chef-client service.
nmdbase::ldap
Installs the LDAP package to set this instance up as a client.
Installs LDAP command line utilities.
Configures the LDAP connection for this client.
Installs the LDAP secret authentication content.
Modifies the Name Service Switch to use LDAP.
Configures the PAM common session to create users from LDAP.
nmdbase::ssl
Installs a configured client cert.
Installs a configured client key.
nmdbase::yubico
Includes the openssh recipe.
Installs the openssh-client.
Installs the openssh-server.
Enables the ssh service.
Starts the ssh service.
Creates the ssh configuration.
Creates the sshd configuration.
Installs python-software-properties.
Adds the yubico repositories.
Notifies apt-get update when adding yubico repositories.
Installs the libpam-yubico package from the yubico repositories.
Creates a global yubico auth file.
Configures the sshd PAM module.
Prepares for PAM debug logging.
Finished in 4.4 seconds
27 examples, 0 failures
[Coveralls] Outside the Travis environment, not sending data.
Ok … that’s a lot … but what does it mean and what is going on? The first line is the rspec command that is being executed for each of the tests this cookbook provides in it’s /spec directory. Coveralls is a third party service that is used for test coverage metrics by this cookbook. The real value lies in the subsequent output. This cookbook lists four recipes that can be applied to a server and explains, in plain english, exactly what the recipes are doing.
That’s great documentation, but the output is driven by the underlying automated tests. This means that not only can we see what a respective recipe should be doing, but we know that it is backed by tests. This also assures us that later attempts at further integration into a continuous integration platform will be less painful.
But What About Integration Tests?
It’s interesting to point out that if the cookbook is properly crafted there is no reason to run integration tests at this point. An integration test uses kitchen to test that your data is properly defined in the context of the environment. Until we define custom data that varies from the recipes implementation we should have no reason to perform an integration test.