With more and more companies utilizing infrastructure as code and configuration management (Puppet, Ansible, Terraform, etc.) so comes the need for rigorously validating and testing that code to ensure that a code change doesn't bring down the entire server farm or corrupt a database.
The intent of this project is to build out a turnkey appliance that can be deployed into an environment and provide continuous integration for Puppet. The appliance will utilize various open source technologies to provide a robust solution that can be easily scaled as needed.
Details about the project can found on the github page. The goal is to develop a number of additional features each week and blog about the progress that is being made with the project.
The solution ties together various technologies as seen in the architecture diagram below.
Docker will provide the base environment for the other services to run on. Docker provides excellent portability and scalability to the solution as it can expand to additional nodes without a major rework of the solution.
Sinatra will present the Web UI and act as the API endpoint for the solution.
RabbitMQ is the glue that connects everything. In the interest of creating an easily scalable distributed system all task requests are sent through the RabbitMQ server and picked up by the appropriate executor.
Jenkins Job Builder
Jenkins Job Builder is utilized to dynamically create the Jenkins jobs on the Jenkins master.
Jenkins is the core component that provides the continuous integration framework for validating and testing our code.
ELK will provide visualization of data from the Puppet CI jobs with metrics such as rspec results, serverspec results, provisioning time, etc. ELK also provides a mechanism for collecting and visualizing data about the environment for things such as health metrics and container logging.
Test kitchen will be used to provide a robust harness for acceptance and integration testing on configured systems.
Vagrant provides the provisioner utilized by Test Kitchen for spinning up machines locally in VirtualBox.
VirtualBox VirtualBox provides the virtualization layer for Vagrant to provision test machines.
Consideration has been given to adding the following technologies to the solution to provide additional functionality.
- Nexus: Artifact Management
- Geminabox: Gem repository
Following the successful update of the configuration settings we can log into Jenkins and see that the Jenkins job for the control repository has been created.
There is still quite a bit of work that needs to be done before the solution can be truly useful but I feel that this is great start to a pretty exciting project. Any feedback or input is greatly appreciated.
Deploying and Testing
Final changes and updates are being to the 0.0.1 release which will showcase initial functionality of the solution and the intention is to release it next week.