Building Base Boxes with Packer

Packer is a powerful tool that enables the creation of identical multi-platform virtual machine images from a definition file. In this article we will discuss how to create base virtual machines for testing and deployment with nmdpacker.

One of the bigger challenges we face when working with team based infrastructure management is how to provide a library of multiple platforms from which a developer can work against. At NEWMEDIA we use nmdpacker to address this issue. By creating system definitions we are able to quickly update, create, and maintain our library of base boxes.

Since we are creating definitions and not manually creating every platform, this becomes an easily repeatable process that we can use across platforms. Doing so helps eliminate many errors that are associated with maintaining this type of library. The nmdpacker project leverages custom code and tools such as Packer, Bento and Rake to create a fast, manageable, and repeatable process. It creates virtual machine images (VirtualBox and/or VMware) based on our definition of the operating systems configuration and which chef cookbooks to apply, it then applies the latest platform patches to the virtual machines and uploads them to Amazon’s simple storage service (S3) for retrieval.

Platforms are defined in JSON formatted files located in the servers/ directory. The provisioners array is used to customize the installation by defining the scripts that run and cookbooks that are applied.

In the following example we will go over the process of setting up nmdpacker, creating an CentOS 6.5 basebox, and uploading it to s3.

Setup

Install virtualization.
Install Packer.
Install the ImageMagick package on your system.
- apt-get install imagemagick (debian)
- yum install imagemagick (centos)
- brew install imagemagick (osx)
- source
Install the nmd-packer repository.$ git clone [email protected]:newmediadenver/nmdpacker.git ~/nmdpacker $ cd ~/nmdpacker/ $ bundle install

Define a Base Box

In this example we will define a base box with 2GB of RAM, 2 vCPUs, and the mysql::server recipe applied. We want this VM to be a little beefy because it’s a database server.

Make the necessary changes to centos-6.5 json definition file.

diff --git a/servers/centos-6.5-x86_64-base.json b/servers/centos-6.5-x86_64-base.json
index 9a84ff6..f75975b 100644
--- a/servers/centos-6.5-x86_64-base.json
+++ b/servers/centos-6.5-x86_64-base.json
@@ -60,8 +60,8 @@
       "vm_name": "packer-centos-6.5-x86_64",
       "vmx_data": {
         "cpuid.coresPerSocket": "1",
-        "memsize": "1024",
-        "numvcpus": "1"
+        "memsize": "2048",
+        "numvcpus": "2"
       }
     }
   ],
@@ -92,7 +92,7 @@
     {
       "type": "chef-solo",
       "cookbook_paths": ["vendor/cookbooks"],
-      "run_list": [],
+      "run_list": ["mysql::server"],
       "json": {

       }

Find the run_list key and add “mysql::server” as the value. This tells packer (via chef-solo) to apply the mysql::server recipe. To make additional recipes available, or to see what recipes are available, edit the Berksfile in the root of the project. Any recipes that you add to this file will be accessible to the base box during the build process. The json section within chef-solo is the where you would provide specific attributes for the recipe. In this case we are taking the defaults.

Build the base box

Rake is the primary tool used execute commands and tasks within nmdpacker. $ bundle exec rake -D rake build Build a base vagrant box from chef cookbooks - Requires environment variables be set - Settings are read from the following shell environment variables. All required variables can be set to * to build all defined servers. "NMDPACKER_OS: ex: OS=centos" - Required "NMDPACKER_VER: VER=5.10" - Required "NMDPACKER_BITS: ex: BITS=64" - Required "NMDPACKER_VAR: default: base ex: base,lamp, etc" - optional "NMDPACKER_ONLY: Typically virtualbox-iso or vmware-iso" - optional "NMDPACKER_BOX: Adds the new box to your local vagrant" - optional "NMDPACKER_UPLOAD: Uploads the box to s3." - optional rake clean[action] "clean[iso|box|all]" - downloaded iso files, built virtual boxes, all. rake delete[bucket_name,object_name] "delete[BUCKET_NAME, OBJECT_NAME]" s3: Delete an object or a bucket (and its contents). Requires AWS_SECRET_ACCESS_KEY,AWS_ACCESS_KEY_ID, & AWS_REGION environment variables be set. rake upload[vmware] "upload[vmware]" Upload boxes to the designated s3 bucket. Defaults to virtualbox if vmware is not specified. Requires AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID, & AWS_REGION environment variables be set. rake validate Validate all the packer templates in servers directory.
Validate the json templates by executing the following command:$ bundle exec rake validate Template validated successfully. servers/centos-6.5-x86_64-base.json is valid. Template validated successfully. servers/ubuntu-12.04-i386-base.json is valid. Template validated successfully. servers/ubuntu-14.04-amd64-base.json is valid.
Set the required build environment variables. In this case nmdpacker_os, nmdpacker_ver, nmdpacker_bits.$ export NMDPACKER_OS="centos" export NMDPACKER_VER="6.5" export NMDPACKER_BITS="64"
Execute the build command:$ bundle exec rake build ... ... Build 'vmware-iso' finished. ==> Builds finished. The artifacts of successful builds are: --> vmware-iso: 'vmware' provider box: builds/vmware/nmd-centos-6.5-x86_64-base.box
Once complete, you will get a message telling you where the base box file is located.

Test the Base Box

To test the newly created base box you can add it to vagrant by running vagrant box add boxname path/to/box $ vagrant box add testcentos65 builds/vmware/nmd-centos-6.5-x86_64-base.box ==> box: Adding box 'testcentos65' (v0) for provider: box: Downloading: file://builds/vmware/nmd-centos-6.5-x86_64-base.box ==> box: Successfully added box 'testcentos65' (v0) for 'vmware_desktop'!
Then run the following commands to create an instance of the box and login to it to confirm that everything is as you have defined.$ vagrant init testcentos65 $ vagrant up testcentos65 $ vagrant ssh
Once satisfied you can remove the box from vagrant by running the command:$ vagrant box remove testcentos65 Removing box 'testcentos65' with provider 'vmware_desktop'...

Upload the Base Box to S3

Naturally, all of this awesomesauce is far from helpful if we can’t distribute the boxes for developers to use. We use s3 storage for this and have built it into the rake tasks.

Set the required upload environment variables AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID, AWS_REGION, & AWS_BUCKET_NAME.$ export AWS_SECRET_ACCESS_KEY="yourkey" export AWS_ACCESS_KEY_ID="aws_id" export AWS_REGION="aws_region" export AWS_BUCKET_NAME="bucket_name"
Execute the upload command to upload the VirtualBox files:$ bundle exec rake upload The virtualbox is uploaded:Action upload: Uploading ./builds/virtualbox/nmd-centos-6.5-x86_64-base.box to the AWS_BUCKET_NAME nmd-centos-6.5-x86_64-base-latest.box (this could take some time) ... Uploaded to: https://amazonaws.com/AWS_BUCKET_NAME/nmd-centos-6.5-x86_64-base-latest.box
Execute the upload command to upload the VMware files:$ bundle exec rake "upload[vmware]"Then Volia! The vmware box is uploaded:Action upload: Uploading ./builds/vmware/nmd-centos-6.5-x86_64-base.box to the AWS_BUCKET_NAME nmd-centos-6.5-x86_64-base-latest.box (this could take some time) ... Uploaded to: https://amazonaws.com/AWS_BUCKET_NAME/nmd-centos-6.5-x86_64-base-latest.box

Building Base Boxes with Packer

Web Components? Are They Forrealz?

Best Practices For Website Design In 2022

Marketing During the Coronavirus (COVID-19)