Intro
On the current engagement we are using gradle to automate the build process. Gradle can lend itself to scripting through groovy. Anything groovy can do gradle can do. I’m interested in more than just building the application automatically. When it comes to Continuous Integration (CI) we often need to do more than just build the application. Gradle has built in facilities to handle many of the things required for CI but not all. With gradle I can upload artifacts, manipulate large groups of files, work with various different artifact repositories, automatically create ear/war files https://docs.gradle.org/current/dsl/ for a complete listing of capabilities. As useful as gradle is, it can’t do everything, and CI requires a little of everything.
Plugins
Gradle lends itself to being extended through a plugin infrastructure. This is how we get all of the functionality we need when gradle doesn’t have it built in. The downside is that the plugin eco system is the wild west. You need to be careful about the pedigree of what you are using, just because a plugin claims to do or work doesn’t mean it is well tested, it worked for someone once.
A little bit of everything
CI is more than about building your application automatically. Maven and gradle both include automated tests as part of the build framework. Frequent and early testing is part of how we like to create CI pipelines for our customers. The difficulty is that the nature of meaningful test require substantial setup and tooling. database setup can be required, applications state needs to be staged, containers may need to be set up etc. In order to accomplish everything I often have to reach out to the unix command line or another scripting language. This is where technology fragmentation can occur.
When your builds require 17 different scripts which are written in 5 different languages you run into several problems. First, constructing the build machine becomes a work of art. Making sure all the various interpreters and their dependencies are in place (CPAN, eggs, rvm, etc) becomes a daunting task that spirals out of control quickly. If you can limit what you need to the standard set of utilities on your LINUX distribution and gradle, reproducibility becomes less of an issue. Additionally, you have the problem of specialized skill. When a build system grows that large, there are very few practitioners that have the knowledge needed to touch all parts of the system. This is how black boxes form, and leads to the phrase “It just always works but no one knows why”. Gradle can give you the ability to break out of black boxes and skill specialization with automated dependency management and a unified language.
Things I Have Used in Practice
I have not authored any gradle plugins, and I have to thank stackoverflow for much of what comes next, but that is kind of the point of gradle, maven, ant+ivy, etc. Let us start with accessing a mysql database directly from gradle. This is useful for staging some integration test when in memory databases won’t cut it. Then I’ll have an example of making rest call directly from a gradle task. This can be useful for manipulating Jenkins from within a build. Lastly, a trick to allow on demand command line execution.
Mysql from a gradle task
import groovy.sql.Sql import java.sql.DriverManager configurations { mysql } dependencies { mysql "mysql:mysql-connector-java:5.1.36" } task initdb ()<< { configurations.mysql.files.each { Sql.classLoader.addURL(it.toURI().toURL()) } DriverManager.registerDriver(Sql.classLoader.loadClass("com.mysql.jdbc.Driver").newInstance()) def sql = Sql.newInstance("jdbc:mysql://${databasehost}/mysql", "${databaserootuser}", "${databaserootpass}") sql.execute("CREATE USER '" + databaseuser + "'@'" + databasehost +"' IDENTIFIED BY 'password'") sql.close() }
Above is a code snippet that allows me to create a user in a database. This could be useful for initializing a database, or getting a database in the proper state for testing. Thanks to this post I was able to extrapolate the mysql analog and off I went.
Rest Calls
Being able to do this inside a gradle task can be useful for two very good reasons. First you can manipulate jenkins to kick off jobs internally.
dependencies { classpath "org._10ne.gradle:rest-gradle-plugin:0.3.2" } } apply plugin: "org.10ne.rest"
task myRestCall (type: org._10ne.gradle.rest.RestTask) { httpMethod = 'post' uri = myURL username = UserName password = Password contentType = groovyx.net.http.ContentType.JSON requestBody = [ “stuff: injson”] doLast { println serverResponse.getData() } }
In addition to Jenkins control you could test your applications rest interface and drive the tests from gradle. Also, you can now use a jenkins job to drive restful activities. This can be useful with interacting with amazon’s API, or various other services you may need during the course of a build.
Delayed Execution
When you have to use the command line to execute jobs for your build, gradle has a task type for that Exec() . However gradle ALWAYS wants to execute the your command. The doLast and Exec do not mix well. If you want to conditionally execute something at the command line because it takes a long time or if you want it to exec after the build, by default you are out of luck. In practice you need some really fancy shell scripting OR this gem I stumbled upon on a project we inherited.
build.gradle
task mytask (type: GradleBuild) { buildFile = "./mycommand.gradle" tasks = ["mylongtexectask"] startParameter.projectProperties =[myparam1: "${rootProject.myparam1}" ] }
mycommand.gradle
task mytask(type: Exec) { //Do stuff }
The downside is that the passing of data is one way for all practical purposes. Parameters can go into your command, but the results are not easily returned to other gradle tasks. I find this not much of a limitation in general, because when I reach the command line with a really long running item I’m usually at the end of what I want to do in a build. That said, I have been bitten by not being able to move data back.
Conclusion
Unifying your build structure around one language can have several benefits but also comes at a cost. The benefits include a standard language in which to read the build script, narrower skill set for script maintenance, and less overall infrastructure to run the build. The downside is that on more than one occasion you will have to fit the square peg into the round hole. Above are three examples of going just a little outside the box. For the mysql example we had to do some crazy class loading before we could get down to using the mysql classes. The rest plugin mostly works, but is half baked and mostly undocumented. Lastly, in order to make use of command line you sometimes need to separate it out into another build file and external task so it doesn’t always execute. Usually the little bit of persistence and ingenuity required to stick with one language are outweighed by the benefits in my experience. It is usually worth it to try and fit the square peg in the round hole.
