Puppet Control Repo Metrics

Puppet Control Repo Metrics

A primary responsibility of the Puppet control repo is to provide a mechanism through the use of a Puppetfile and R10k to fetch all defined Puppet module repos from various location. Testing of the control repo to make sure it's working properly is accomplished through commands such as "r10k puppetfile check" to validate the syntax of the Puppetfile. Much like the testing and validation available for Puppet modules it's great to implement to prevent bad code from making it into production but what about historical tracking of that data.

That led to the development of the following ruby code to output key metrics to JSON to allow easy ingestion into a tool such as Splunk for example.
https://github.com/martezr/puppet-control-metrics

The metrics that are currently being gathered are listed below.

Metrics

  • Modules
  • Module count
  • Missing modules (Modules not downloaded due to various problems such as non-existent repository because of a typo)
  • Missing module count
  • R10k puppetfile syntax check
  • Commit hash
  • Repo name
  • Commit author
  • Missing dependencies (Modules that are required by modules in the Puppetfile based upon their metadata.json file)

Jenkins Configuration

We need to create a Jenkins job for testing our control repo and actually running our ruby script.

Create a new freestyle Jenkins job with an appropriate name.

Select "Git" radio button under the "Source Control Management" section and provide the appropriate repository along with the branch.

It is likely that the branch is "production" and not master

Add an "Execute Shell" action under the "Build" section with the following code.

#!/bin/bash

# Install prerequisite software
gem install r10k --no-ri --no-rdoc  
gem install puppet --no-ri --no-rdoc

# Download the Puppet modules into the current directory
echo "r10k puppet module download"  
r10k puppetfile install -v --color

# Generate JSON
puppet-control-metrics > puppet-control-metrics.json  

Now that we've created the Jenkins job we are almost ready to build the job but first we need to add the ruby script to the system that will run the Jenkins job. We'll add the script (https://raw.githubusercontent.com/martezr/puppet-control-metrics/master/puppet-control-metrics.rb) as puppet-control-metrics and place it in the /usr/local/bin/ directory to run the script by it's name. We need to make the script executable (chmod +x puppet-control-metrics) and then we can move on to testing the job.

For testing purposes we can click the "Build Now" to trigger out Jenkins job. After the job has been built we should have a JSON file with the metrics from the control repo in the Jenkins workspace.

{
    "missingdependencies": [],
    "modules": ["activemq", "apache", "apt", "augeas", "bind", "certs", "community_kickstarts", "concat", "createrepo", "datacat", "dhcp", "epel", "erlang", "firewall", "gcc", "git", "gnupg", "hiera", "inifile", "java", "java_ks", "kickstart", "lab_config", "local_user", "lvm", "make", "mcollective", "mysql", "ntp", "pe_gem", "portage", "postgresql", "puppet", "puppetdb", "r10k", "rabbitmq", "ruby", "rvm", "ssh", "staging", "stdlib", "sudo", "tftp", "vcsrepo", "xinetd", "ygrpms"],
    "missingmodules": ["lab_config"],
    "r10k_version": "2.5.1",
    "number_of_modules": "46",
    "number_of_missing_modules": "1",
    "control_repo": "controlrepo",
    "commit_author": "Rob Nelson",
    "commit_hash": "b3e8565",
    "puppetfile_syntax": "pass",
    "puppetfile_debug": "N/A"
}

The sustainable architecture would be to ship that JSON file to a tool such as Splunk for ingestion and create a user-friendly dashboard like the one below.

Subscribe to