cloud-test: Automated Compute Engine and Docker Image Builds with Jenkins, Packer, and Kubernetes

Google Cloud Platform Blog

Automated Compute Engine and Docker Image Builds with Jenkins, Packer, and Kubernetes

Wednesday, May 27, 2015

Everyone loves portability and speed, especially when it comes to cloud. Today we’re excited to introduce a new technical paper and open source reference implementation that will help your Google Compute Engine virtual machines boot even faster, and allow you to build portable images for Docker along with your Compute Engine images.

When you run an application on a Compute Engine instance, you first have to deploy one or more Virtual Machines and configure them so your application works. This configuration usually involves installing the application and its dependencies, then setting up any other configuration your application requires, such as database connection strings or API keys.

You could do this manually by connecting to each instance after it boots and configuring each element, but that is a slow and error-prone process that creates unique, inconsistent “snowflake” instances. Running configuration scripts automatically at startup is a better solution as it’s repeatable and scalable - but it requires a lot of up-front work and is still subject to human error. Additionally, if a package you’re installing is temporarily unavailable, your instances just won’t boot and if the packages you’re installing are large or need to be compiled, boot times gets slower and your ability to auto scale is affected.

Building custom images before you launch instances is a great way to reduce boot times and increase reliability. Today we’re introducing a solution paper and open source reference implementation that describes in detail how to Automate Image Builds with Jenkins, Packer, and Kubernetes. You’ll learn how to use popular open source technologies to continuously build images for your Compute Engine or Docker-based applications. You’ll build the images in a central project, share them with other projects in your organization, and integrate the image build as a step in your continuous integration (CI) pipeline.

The diagram below shows the Jenkins image builder serving as a hub that builds Compute Engine and Docker images for other projects in your organization:

Figure 1: Jenkins building images for other projects in your account.

In addition to creating a secure and scalable image building pipeline, you’ll learn how to run a reliable Jenkins installation on Kubernetes, including how to backup/restore Jenkins and scale your worker nodes.

Head on over to the Automated Image Builds solution page for all the details on using Jenkins, Packer, and Kubernetes with Google Cloud Platform to increase the speed and reliability of your instance and container launches. After that, go deploy the infrastructure yourself by following the tutorial in the reference implementation. We love feedback: GitHub pull requests or issues for suggestions on the tutorial, comment here, or @evandbrown on Twitter to let me know how you’re using Google Cloud Platform!

-Posted by Evan Brown, Solutions Architect