Building software in the cloud makes scalability an easy and achievable goal for your organization. As the demand on your Jenkins executor capacity increases, you need to be able to scale your Jenkins footprint to keep up with the demand. This is where the Amazon EC2 Plugin can make a huge impact to your team’s productivity while keeping compute costs in line with the demand of the team.
The EC2 Plugin works by monitoring the queues based on labels. In the configuration, specify an AMI ID that will be tied to a specific label. You can also get to specify instance type, subnet, etc. For example, you’d create an AMI with Java and Maven and label it ‘compile’. When a job is queued with label and no executors can serve the request, the plugin will launch a new instance from the AMI, add it to the list of available nodes and start building.
The plugin also helps your organization save money! You only have instances running in your VPC or classic EC2 account when they are needed. The plugin allows you to shut down or terminate the instance after it is idle for a configurable length of time. The plugin is also smart enough to stop or terminate the instance if it is idle before the next billing hour. That way you aren’t stopping an instance moments after you paid for another hour of compute time.
Dynamically creating build nodes really does work! After implementing this plugin on a large development project we saw a 80% increase in the number of deployable releases and 40% cost reduction in instances related to the build pipeline. Spot instances are also an option; however, early termination is something to keep in mind when going this route. Having an instance terminated in the middle of job is only going to frustrate your team.
There are few things to keep in mind when implementing this plugin:
- Review your account regularly to make sure instances are properly being terminated.
- Create an IAM policy to give the instance or access key the least privileges needed to launch and terminate instances.
- Consider using an instance roles instead of access keys.
- If you need instances to come up quickly, have them stopped when idle instead of terminated.
The plugin really is a must have if running a Jenkins environment in AWS.
