Building vSphere Templates From Scratch Using Packer and the vSphere API

Packer (https://www.packer.io/) is amazing tool that enables templates and machine images to be created in an automated fashion with support for various infrastructure providers such as AWS, GCE, Azure, VMware and many more.

Unfortunately there is a key missing feature in the existing VMware builders that hinders the ability to create a CI/CD pipeline for dynamically generating vSphere templates.

Generating a template from scratch using an ISO file provides the cleanest method for creating a new template. This was the driving factor behind the creation of a Packer builder that leverages the vSphere API.

Existing Packer VMware builders

Let’s walkthrough the existing Packer builder that support VMware and look at their limitations in enabling us to create templates from an ISO file in a way that easily supports a CI/CD process.

Packer vmware-iso: This allows the template to be built from an ISO file locally using a VMware Desktop hypervisor like Workstation and Fusion. There is also the ability to build it remotely on an ESXi host but requires a modification to the ESXi host and utilizing a local ESXi account to authenticate.

Limitations

  • Building the template in VMware Workstation or Fusion requires the build agent to support those desktop hypervisors.
  • Requires modifying the ESXi host to support building the template on an ESXi host
  • Requires using a local ESXi account instead of a vCenter account

Packer vmware-vmx: This builder is similar to the vmware-iso build but starts from an existing VM.

Limitations

  • The desire is to create the template from an ISO file and not from an existing VM.

Jetbrains vSphere (https://github.com/jetbrains-infra/packer-builder-vsphere ): This builder leverages the vSphere native API to avoid the need to modify ESXi hosts or use a local ESXi account.

Limitations

  • The desire is to create the template from an ISO file and not from an existing VM.

Now that we’ve looked at the existing builders that support VMware we’ll look at the new builder that’s been created and see how to use it to build a new template.

Packer vSphere ISO Builder

The code for the builder created is based on the vSphere provider created by jetbrains. The existing builder builds the template using an existing template but this didn’t align with the template management paradigm that utilizes a base template created from ISO every time to ensure a pristine starting point.

https://github.com/martezr/packer-builder-vsphere

Requirements
The following is required to utilize the vsphere-iso builder.

  • HashiCorp Packer
  • vsphere-iso Packer plugin
  • Method to automatically install an operating system

Install Packer

Packer is a compiled package that supports Windows, Linux and Mac OS. The package just needs to be downloaded to the system and run.

https://www.packer.io/downloads.html

Install the vsphere-iso plugin

Plugins can be installed in three ways but simply downloading the plugin to the current working directory and triggering a Packer build works well.

vSphere ISO plugin
https://github.com/martezr/packer-builder-vsphere/releases/tag/v1.5

Packer installing plugins
https://www.packer.io/docs/extending/plugins.html#installing-plugins

Automated OS install

One of the key components to the automated template creation process using the vsphere-iso Packer plugin is an automated method for installing the operating system and install VMware tools. For this example an automated CentOS 7 ISO file has been created that contains a kickstart file for automating the installation.

Kickstart File https://gist.github.com/martezr/64ee6bed763ab731a490ee14e0f911d4

Packer JSON Configuration File

An example Packer JSON configuration file is shown below that will create a vSphere template in an automated fashion.

{
  "builders": [
    {
      "type": "vsphere-iso",

      "vcenter_server": "vcenter.domain.com",
      "insecure_connection": "true",
      "username": "[email protected]",
      "password": "secret",
      "cluster": "cluster01",
      "host": "esxi-1.domain.com",

      "vm_name":  "centos7base",
      "convert_to_template": "true",
      "folder": "templates",
      "cpu": "1",
      "ram": "2048",
      "network": "VM Network",
      "network_adapter": "e1000",
      "guest_os_type": "otherGuest",
      "datastore": "ProdDatastore03",
      "disk_size": "5GB",
      "iso": "ISOS/CentOS7.iso",
      "iso_datastore": "NFSDatastore01"

    }
  ]
}

With our Packer JSON file in place we just now need to build our template by running the command displayed below.

packer build centos7.json

vsphere-iso output will be in this color.

==> vsphere-iso: Creating VM...
==> vsphere-iso: Customizing hardware parameters...
==> vsphere-iso: Power on VM...
==> vsphere-iso: Waiting for IP...
==> vsphere-iso: IP address: 192.168.1.110
==> vsphere-iso: Shut down VM...
==> vsphere-iso: VM stopped
==> vsphere-iso: Convert VM into template...
Build 'vsphere-iso' finished.

==> Builds finished. The artifacts of successful builds are:
--> vsphere-iso: centos7base

If all went well then we should be able to go into vSphere and see our newly create vSphere template.

Example Script

The following script provides an example for downloading the vsphere-iso plugin, packer and running the packer build. This script could easily be integrated into a Jenkins job to automate the process.

# Download vsphere-iso plugin
wget https://github.com/martezr/packer-builder-vsphere/releases/download/v1.5/packer-builder-vsphere-iso.linux

# Download Packer
wget https://releases.hashicorp.com/packer/1.1.0/packer_1.1.0_linux_amd64.zip?_ga=2.161158589.1457895794.1505274494-1862920313.1503788741

# Unzip Packer
unzip packer_1.1.0_linux_amd64.zip?_ga=2.161158589.1457895794.1505274494-1862920313.1503788741

# Change permissions on vsphere-iso plugin
chmod 777 packer-builder-vsphere-iso.linux

# Build the vSphere template
./packer build centos7.json

The vsphere-iso plugin greatly simplifies the process for building templates in a vSphere environment using Packer.

Final Thoughts

While the plugin allows us to now easily build templates from scratch using an ISO file there are still some limitations and enhancements that can be made to the plugin.

  • Currently no provisioners are supported
  • Better error handling
  • Additional logic to remove mandatory configuration fields

The current workaround to the lack of provisioner support is to perform all of the template configuration prior to installing vmware-tools at which point Packer will shutdown the VM to convert it to a template.

References

Packer
https://www.packer.io/

Centos 7 Kickstart File https://gist.github.com/martezr/64ee6bed763ab731a490ee14e0f911d4