Building a Deployment Pipeline

Posted By: Nick Floyd on October 5, 2010

In Development at Fellowship Tech, one thing we’ve been working on is an automated build pipeline.  This allows us to build our software and deploy it to our environments with no human intervention.  The complete automation of our builds and deployments is called a deployment pipeline or build pipeline.  It’s called a pipeline because once the build is inserted into the pipeline, a set of mostly automated process act upon it and pending the passing/approval of the results of that process it moves on to the next stage.  Let’s examine the stages of the build process.




http://www.flickr.com/photos/rickz/2113212191/
CC BY-NC-ND 2.0

Stage 1: The build

First and foremost, the software needs to be built.  This is the only time that the software should ever have to be built and packaged.  During the other stages in the pipeline you might make configuration changes, but never, ever, ever will you rebuild the deployment package.  You should never invoke a compiler or pull the latest code from source control after this stage. During this stage you will simply be pulling the latest code (compiling if necessary) and packaging it for deployment in to environments.  Typically if there are any debug options, you want to make sure that they are off so that this build can eventually end up in production.

Stage 2: The unit tests

At this step you will be running the code through a barrage of automated tests.  At this point it should just be tests on the code, this means strictly unit tests.  Tests should run quickly as to not hold up the build process.

Stage 3a*: The acceptance tests

At the acceptance test stage, you should have the software in an environment where it is working and accessible by developers, QA, and any stakeholders that need to approve the build to move forward.  At this stage there can be a hybrid of automated tests (using frameworks like Selenium or Cucumber) and manual tests.  One person should be responsible for getting feedback from all stakeholders and approving the build to move to the next stage.

Stage 3b*: Load testing

At the load testing stage the software is placed under load and average response times are monitored.  A threshold should be defined for what would determine failure of a load test.  A system’s performance should degrade in a logarithmic fashion at best and a linear fashion at worst as the load is increased.  Acceptable upper limits should be determined by manually increasing load on an application until it breaks and then encapsulate those values into an automated process.

Stage 4: Release

Finally the stage that we’ve been waiting for.  The stage where we release the software to production and get it in front of the users of your software.

Gates

Each stage has a series of “gates” which allow or disallow the process to continue to the next stage.  Anything that fails (a unit test fails, the automated acceptance tests fail, a tester finds a manual test fails, etc.) the software is rolled back and notifications are sent out to developers to fix the failing issue.

Final Notes

A build pipeline is enforcing processes with automated solutions so decisions need to be made by teams on how to deliver software.  How is the software installed?  What are your standards for how your source control repositories are structured?  How do you install your software?  Can that be automated?  The road to getting a build pipeline built out is a difficult one but lets you reap amazing benefits through increased time to delivery and reinforcement of quality though extensive testing.

*: Stage 3a and 3b can be executed simultaneously.

Jesse Dearing is a Developer for Fellowship Technologies.  Currently he is focused on DevOps which is where the needs of the infrastructure meet the development of the software.  He likes learning about new technologies and languages (especially ones that are used on the web) as well as learning about new trends and practices in software development.  He is passionate about software development and helping other developers.

Posted In: Tips,

Comments:
Greg said: on October 20, 2010 at 07:04 PM

This is a nice summary of deployment pipelining. Thanks for putting it so succinctly. I’m in the process of building a deployment pipeline for a couple projects that are already using Hudson for continuous integration.

Can anybody recommend open source tools to use in implementing a pipeline?

Benjamin Mauch said: on December 6, 2010 at 09:33 AM

Have you guys given up on blogging? No blogs in 2 months.

Commenting is not available in this channel entry.