Jenkins is a powerful tool in your Continuous Integration and Continuous Delivery toolbox. For my last clients over the past several years, we have utilized Jenkins to deploy software and preform automated testing. One of the most powerful tools I have used to orchestrate deployments and testing has been the Build Flow Plugin. It has allowed me to create simple, extensible and highly-configurable workflows within Jenkins. For more on installing and the build flow basics check out my previous post on the subject.
I will walk through a series of deployment and testing build flows from simple to most complex in order to provide you with a whole host of concepts and templates you may find useful in your Continuous Integration frameworks.
Getting Started
Let’s assume, for the sake of this exercise, that you know how to deploy and run automated tests against your application. Let’s also assume that you have chained all your processes into one monolithic, behemoth task. You should start with creating a job like “DeployMyApplication” and “TestMyApplication”. It’s recommended that you break down your build, deploy and test jobs into small self-contained jobs so that as your process becomes more and more mature or your application undergoes major changes you can easily move the order of jobs or modify fewer jobs around with little effort.
The build flow for your application would look something like this:
build (“DeployMyApplication”) build (“TestMyApplication”)
While this simple example may not provide much value over making the test job run downstream of the deploy, very few organizations have jobs that are this simple, elegant or consistently reliable.
Graceful Recovery
I have never encountered an organization that hasn’t encountered a “flakey” job (typically that job, you say “let’s just re-run it”). Due to technical debt or environmental issues we often require a mechanism to run a job more than once so that a flakey test doesn’t fail an otherwise successful test run.
In this example, I’m assuming the deployment is our inconsistent test. We would modify our build flow like so:
//Try to deploy the application up to three times
retry (3) {
build (“DeployMyApplication”)
}
build (“TestMyApplication”)
This is not the only method of recovery I utilize often. The guard-rescue is the build flow’s equivalent of a try-catch block. I’ve often used a guard-rescue in the following way:
//Try to deploy the application up to three times
retry (3) {
build (“DeployMyApplication”)
}
guard {
//Try the following: Setup the DB à Test the application à Scorch the DB
build (“SetupTestData”)
build (“TestMyApplication”)
build (“ScorchMyApplicationDB”)
} rescue {
//On failure, pull the application log, snapshot the DB and Scorch the DB
build (“PullApplicationLogs”)
build (“SnapshotTheDB”)
build (“ScorchMyApplicationDB”)
build (“FailTheFlow”)
}
Any good test suite typically requires three things: setting up test data, testing the app, and cleaning up the test data. If these were all their own jobs, a test failure would prohibit the DB cleanup activity and potentially leave your database in an undesirable state for the next run. Hence, we put the “ScorchMyApplicationDB” in both the guard and rescue to ensure it runs no matter the state of the tests. Since the failure is also caught by the rescue, we create a job to always fail the flow if the tests fail in the guard.
And Now with Parallelization
Any mature project, should have a lengthy automated test suite. Often when those suites become too big we end up splitting up the suite into smaller chunks. Using test parallelization, we can get a some relatively decent cost savings. In our example we split our tests into: Front-End Tests, Back-End Tests and Integration Tests.
//Try to deploy the application up to three times
retry (3) {
build (“DeployMyApplication”)
}
guard {
//Try the following: Setup the DB à Test the application à Scorch the DB
build (“SetupTestData”)
//Run the tests in parallel
parallel (
{ build(“RunFrontEndTests”) },
{ build (“RunBackEndTests”) },
{ build (“RunIntegrationTests”) }
)
build (“ScorchMyApplicationDB”)
} rescue {
//On failure, pull the application log, snapshot the DB and Scorch the DB
build (“PullApplicationLogs”)
build (“SnapshotTheDB”)
build (“ScorchMyApplicationDB”)
build (“FailTheFlow”)
}
Next, check out parameter passing.
