CONCOURSE-CI

concourse_01

Introduction

There are countless CI options out there. Transitioning from one CI system to another can be a huge investment depending on the size of your project.

Concourse holds the following principles to heart. Collectively they reduce the risk of switching to and from Concourse, by encouraging practices that decouple your project from your CI infrastructure’s little details.

Concourse is a response to the complexity introduced by other systems. It is built on the idea that the best tools can be learned in one sitting.

Concepts

Like files and pipes in Unix, the goal is to build an expressive system with as few distinct moving parts as possible.

Concourse limits itself to three core concepts: tasks, resources, and the jobs that compose them. Interesting features like timed triggers and synchronizing usage of external environments are modeled in terms of these, rather than as layers on top.

With these primitives you can model any pipeline, from simple (unit → integration → deploy → ship) to complex (testing on multiple infrastructures, fanning out and in, etc.).

Usable

Concourse is optimized for quickly navigating to the pages you most care about. From the main page, a single click takes you from a pipeline view to the log of a job’s latest failing build.

From there, the job’s entire build history is displayed, and every input for the job is listed with any new inputs highlighted.

The build log is colorized and supports unicode. It emulates your terminal and gets out of your way.

Isolated Builds

Managing the state of the worker VMs in other CI systems is a nightmare. Build pollution is a constant threat, and the workers have to constantly be tweaked to make sure they’re providing the right things for every build.

In Concourse, the workers are stateless. Every task executes in a container defined by its own configuration. Multiple teams can use the same Concourse deployment without worrying about the state of the worker VMs.

Scalable, reproducible deployment

No Concourse deployment is a snowflake. There are no boxes to check; no configuration happens at runtime.

Concourse is statically configured, and as a result can always be recreated from scratch with a single BOSH deploy. If your deployment’s infrastructure burns down, just deploy it somewhere else.

As heavier workloads come in, scaling up the workers is as easy as bumping a number in your BOSH deployment manifest. Scaling back down is the same routine.

Flexible

Features that other systems implement in the core of the product, Concourse implements in “userland”, as resources. This keeps the core of Concourse small and simple, and proves out the extensibility introduced by this simple interface.

The following are features implemented entirely as resources, in the same way that any user can extend Concourse’s functionality :-

  • Timed build triggers are implemented as a Time resource.
  • Integrating with Git repositories is done via the Git resource.
  • Auto-delivering stories in Pivotal Tracker is implemented as a Tracker resource.
  • Integrating with and delivering objects to S3 buckets is provided by the S3 .
  • Building a pipeline for Docker images is provided by the Docker Image resource. Interestingly, this is what is used for delivering the resource images themselves.
  • Concourse vs Jenkins

  • Jenkins servers become snowflakes. More and more important shell script gets pasted into those little textboxes, plugins get installed, and packages are installed on the box. It’s possible to try and store this state but Jenkins servers are extremely resistant to being declaratively configured.

    Concourse does not allow any configuration to enter the server that cannot be easily saved away in version control. Deleting your Concourse server and bringing it back somewhere else is not and should not be a big deal. Builds run inside their own containers so that installing packages on the build machine doesn’t pollute other builds.

  • Jenkins has no first class support for pipelines. There are plugins that add some of this functionality but they never seemed to work perfectly. You can set jobs to run before or after others but this becomes error prone and confusing to perform more complex sequence changes.

    Concourse is built on the assumption that you will be building pipelines. However, instead of making you explicitly say the order in which jobs should be run, Concourse just asks that you say which are valid inputs for each of the jobs in your pipeline and uses this to calculate the pipeline itself.

  • Trying to find the build output log for a failed build in Jenkins can be up to 3 clicks from the home page.

    The Concourse UI is build and designed around making that sort of information as quick to get to as possible. Getting to the most recent build log is a single click.