Continuous integration (CI) is a key component of agile software development that all organizations should strive to include in their development process. However, for small organizations with little or no infrastructure, purchasing and maintaining a server to use for continuous integration is extremely impractical and often infeasible. In these situations it is more cost-effective to use a service such as Amazon EC2 in order to buy time on a virtualized server. For example, the cost of running a Windows server with a 2.66 GHz Intel Xeon processor and 613 MB of memory is 2 cents per hour or $14.40 for one month. Although this server may be sufficient for testing small applications, it may not be powerful enough to run the tests in a short amount of time when either the application or the tests become sufficiently resource-demanding. In order to address this issue a development team may wish to upgrade its virtual server. However, doing so will increase monthly costs and may waste resources unless the team is building and testing the application multiple times per hour.
For teams that are using Jenkins to perform continuous integration, one way to mitigate this issue is to create a separate EC2 instance with the computational resources needed to run the tests. A Jenkins job can start this “slave” instance, run the tests, and then shut down the slave. By doing this the team can keep its small and inexpensive CI server running all of the time and only start the slave server to run the tests. Jenkins comes with a plugin for starting and stopping EC2 instances in this way; however, it is designed to connect to slaves using ssh, which is not immediately useful for Windows developers. In this post I will describe how I was able to start and stop an Amazon EC2 Windows instance and run jobs on that instance using Jenkins. I will assume that you have experience using Amazon EC2, that you already have Jenkins installed on an Amazon EC2 instance, and that you have some experience with creating and running jobs.
Creating and Configuring the Slave Server
The easiest way to create the slave is to clone the original server instance. To clone the server, go to the EC2 Management Console, right click the instance that you wish to clone, and select “Create Image (EBS AMI)”. Give the image a name and make changes to the other options if needed. After entering this information click “Yes, create.” You will need to wait 10-30 minutes for the image to be created, and during this time you can view the progress of the image creation by clicking on “AMIs” in the Navigation menu. After the image has been created you can right click it and select “Launch Instance”. You will be given a list of options for creating the new instance. Select the correct ones for your situation, but remember that under “Instance Type” you will need to select an instance with the appropriate computational resources for your needs (see: http://aws.amazon.com/ec2/instance-types/ for the resources offered for each instance type). After launching the instance you can connect to it with the credentials that you use to connect to the master instance. This is an advantage of cloning the master server – there is no need to create new credentials on the slave machine.
After you have created a slave instance, go to Jenkins on the master server, click on “Manage Jenkins” on the home page, and then click on “Manage Nodes”. On this page you will have the option to create slave nodes for performing jobs. Create a new node by clicking “New Node”. Give the node a meaningful name and select the “Dumb Slave” option. Jenkins simply sends these types of slaves work and receives results – there isn’t any type of high-level interaction (such as load balancing) between the two nodes. After clicking “OK” you will be taken to a page where you can further configure the slave node. The required fields are the number of executors and the remote filesystem root. The other important field to set is “Launch method” – you should select “Launch slave agents via Java Web Start”. Depending on the job you may want the slave to inherit environment variables and tool locations, in which case you should select these under “Node Properties”. After you are finished configuring the node click “Save”.
The slave should now appear under the list of nodes. After clicking on its name you will be taken to a page describing three options for connecting to the slave. I used the second option – running the Java Web Start command from the slave instance. Copy this command and run it from the command line on the slave instance. A window will pop up and show the slave attempting to connect to the master on a certain port. You should add this port to the list of ports from which to accept incoming traffic. After doing this return to the slave instance and you should see in the pop-up box that the slave has connected. If the connection attempt has timed out you should simply re-run the Java Web Start command. In the pop-up box there should now be a File menu with one option, “Install as Windows Service”. Select that option, confirm that you wish to install the service, and when you are asked if you would like to start the service now, select “OK”. If you go to the list of Windows services you should now see one called “Jenkins Slave”.
After doing this you should be able to go back to the Jenkins page for that slave instance on the main build server and, after refreshing it, see that the slave is connected. When I first tried this, I instead saw a message saying that the connection had timed out. Upon inspecting the jenkins-slave.err.log file (located in the directory in which you ran the Java Web Start command), I discovered that the Jenkins Slave service was unable to validate the self-signed certificate that was installed on the main server. If you experience this problem, the solution is to open the file jenkins-slave.xml (located in the same directory as jenkins-slave.err.log) and append the option ‘-noCertificateCheck’ to <arguments>. After doing this you should be able to start the Jenkins Slave service and connect to the master. Now that you have created and configured the slave node you are ready to start it, stop it, and run jobs on it using Jenkins.
Installing the EC2 API Tools
Before you can start and stop the slave node through Jenkins you need a set of tools for managing EC2 instances. These tools can be downloaded from: http://aws.amazon.com/developertools/351. After downloading the tools on the CI server you should perform the following steps to install them:
- Extract the files from the archive.
- Create an environment variable called “EC2_HOME” that points to the directory containing the extracted files.
- Add “%EC2_HOME%\bin” to the PATH.
- Download and install the JDK.
- Create the “JAVA_HOME” environment variable and set its value to the JDK installation directory.
- Add “%JAVA_HOME%\bin” to the PATH.
- Restart the Jenkins Windows service.
After installing the EC2 API tools you are ready to create jobs that will start and stop EC2 instances.
Starting and Stopping the Slave Server Using Jenkins
The folder %EC2_HOME%\bin contains a number of Windows batch scripts that, when run, perform some action on Amazon EC2 instance(s). The two commands that we are interested in are “ec2-start-instances” and “ec2-stop-instances”. Using these commands we will create two Jenkins jobs – one to start the slave server and another to stop it. The “Start Slave Server” job simply runs one Windows batch command to start the instance:
ec2-start-instances –O [accessKey] –W [secretKey] INSTANCE(S)
In order to find your access key and secret key, log in to your Amazon Web Services account. After logging in, select “My Account / Console” at the top of the page and click on “Security Credentials”. This will take you to a page with a list of access key / secret key pairs for your account. Amazon automatically assigns you one such pair when you create your account.
After identifying the access key and secret key pair for your account, you will also need the name of the slave instance that you wish to start. You can find this by going to the EC2 Management Console and viewing the list of instances for your account. Find the instance that you wish to be the slave in the list and look under the “Instance” column for its name.
After creating the job to start the slave server, you should also create a job to stop the server. Like the job for starting the server, it will simply run one Windows batch command:
ec2-stop-instances –O [accessKey] –W [secretKey] INSTANCE(S)
You should test these jobs by running them and using the EC2 Management Console to determine if the slave instance actually starts up or shuts down. Now that you have jobs to start and stop the slave server you are ready to run a job on the slave server.
Running a Job on the Slave Server
Before creating a job that chains the start, stop, and run tests jobs together, you will need to take two additional steps. First, you will need to install the “Parameterized Trigger Plugin” for Jenkins. This will allow you to create a job that executes Jenkins jobs in sequential order. Second, you should modify the job that you wish to run on the slave server such that it can only run when the slave is available. To do this go to the configuration page for the project, select “Restrict where this project can be run”, and enter the name of the slave node in the “Label expression” field. You can specify multiple nodes here or conditions under which the node will or will not be used (e.g., only use the node if the operating system is 64-bit).
After making these changes you are ready to chain the jobs together. First create a Jenkins job. Under “Build” click on “Add build step” and select “Trigger / call builds on other projects”. Under “Projects to build” type the name of the job that starts the slave server and select “Block until the triggered projects finish their builds” (this ensures that Jenkins will wait until one job finishes before running the next one). Follow these steps to add a second build step that will run the project for executing tests and a third build step to stop the slave. Save this project and then start a build. Jenkins should start the slave server, wait for the Jenkins Slave service to start, run the tests on the slave, and then stop the slave server.
In this post I described a method for configuring a slave EC2 instance to automatically connect to the master node. However, since this requires making changes to the slave before it can be used, it may not be useful in situations where slaves need to be created and then used for running jobs immediately. The Jenkins plugin for provisioning and connecting to EC2 instances does not require this step, so it might be valuable to learn how this plugin works and modify it such that it will work for a Windows machine without ssh.
