Jenkins Scripted Pipeline: A Groovy Primer

The Jenkins pipeline documentation does a serviceable job describing a Jenkinsfile. There are a number of examples for common tasks, giving code snippets that can be copied and pasted for your own use. When needing to do simple or commonly performed tasks, this method can be sufficient. There is also a helpful pipeline syntax generator built into Jenkins. Less common tasks possibly have a Stack Overflow question or blog post describing how to accomplish them.

However, it is insufficient to merely copy examples or how-to guides from the internet. Doing so prevents one from fully comprehending the syntax and mechanisms by which a language is constructed. It is more difficult to apply learning to new situations or requirements without grasping the core concepts and organization. You should strive to be able to be read a Jenkinsfile and understand whether it is correctly formed, and what tasks will be executed. Not just through pattern recognition, but by having a deep understanding of the language.

There is only so far one get by merely knowing that when a task calls for displaying the text `Hello world`, you need to type echo 'Hello World' in your Jenkinsfile. More is needed. What is echo, what is 'Hello World', and why are they in that order, structured as they are? How can you take that basic example and expand upon it and modify it to suit your specific use case?

You could stumble along and guess at how things should be configured, you could use trial and error to attempt to get things to work, or you can step back, truly understand the underlying Groovy syntax, and how Jenkins leverages it.

The intention of this blog post is to go over a few of the Jenkins scripted pipeline building blocks, and to explain how they relate to the underlying Groovy language, and what exactly is occurring.

Closures

Below is an example of a basic scripted pipeline. It has a single step, in a single stage, that is executed on the node labeled ‘master’. It is easy to remember this structure, because almost all well-formed scripted pipeline examples will be similarly setup.

node('master') {
  stage('Say Hello') {
    echo 'Hello World'
  }
}

But how does Jenkins take this code and parse it into a running pipeline? How does this relate to Groovy?

Because both the node and stage statements are similarly structured, we will only need to discuss one of them in order to understand both. Statements such as withCredentials and withEnv also follow the same structure.

All of these are examples of steps that accept, encapsulate and execute a block of code. The Groovy documentation explains that, “A closure in Groovy is an open, anonymous, block of code that can take arguments, return a value and be assigned to a variable. A closure may reference variables declared in its surrounding scope.”

The code between the curly braces is executed by the given method, which may provide its own code surrounding the closure execution. The following is an example of a simple Groovy method that accepts a label and closure, printing out the label, and executing the closure.

def stage(String label, Closure cl) {
  println "The stage is ${label}"
  cl()
  println "Exiting the stage"
}

stage('Say Hello World') {
  println 'Hello World'
}

The execution of this code results in the following output:

The stage is Say Hello World
Hello World
Exiting the stage

The specific method could include any manner of code before and after the closure execution. This example is not meant to be an accurate representation of the Jenkins stage implementation. However, it is designed to give a more detailed view into how the Jenkins pipeline syntax is translated into a running pipeline.

Steps

When I first started using Jenkins scripted syntax, I was somewhat confused about why and when parentheses or brackets should be present. I would see examples of different steps, and sometimes they would be present, and other times they would not. Sometimes there would be a single value, and other times there would be key and value pairs, singularly present or delimited by commas. I did not have previous experience with Groovy, so it was unclear how everything fit together. I was able to perform the required tasks, but this uncertainty tore at me.

For example, to print Hello world, I would use:

echo 'Hello World'

But if I wanted to check code out from a Git repository, I would need to use a statement such as:

checkout([
  $class: 'GitSCM',
  branches: [[name: '*/master']],
  userRemoteConfigs: [[ url: 'http://git-server/user/repository.git' ]]
])

And if I wanted to use the Artifactory plugin to deploy maven artifacts, I would do something like:

def rtMaven = Artifactory.newMavenBuild()
rtMaven.run pom: 'pom.xml', goals: 'clean install'

Were these things inherently different? Syntactically, they appeared different, though I knew all were examples of steps. But a step was a Jenkins pipeline concept, how did it relate to the underlying Groovy language?

Optional Parentheses

The Groovy style guide helped answer the first of my questions; when and why some steps included parentheses, while others did not. One of the key features of Groovy is the ability to omit parentheses. According to the style guide, this “useless syntactical noise” should only be included if required. Personally, I like parentheses and feel that it is less about cluttering the code with unnecessary syntax, and more about a consistent design that — subjectively — reads much clearer. Be that as it may, I also feel strongly about following best practices and common style guides.

With this new understanding, I realized that all three of the above examples were calling Groovy methods with parameters, with only the checkout method including the parentheses. There are a couple of specific instances that require the inclusion of parentheses, one being when there are no parameters being passed into the method, and another — as shown with checkout — when the initial parameter uses square brackets.

def aMethod(def aParam) { }

// empty parameter
aMethod // THIS IS AN ERROR
aMethod()

// list parameter
aMethod ['1'] // THIS IS AN ERROR
aMethod(['1'])

The below commands are equivalent, and demonstrate the syntax with and without parentheses.

echo 'Hello World'
echo('Hello World')

rtMaven.run pom: 'pom.xml', goals: 'clean install'
rtMaven.run(pom: 'pom.xml', goals: 'clean install')

Maps

The input into the echo step is straight-forward. It is a single string value, the message to output. The checkout and rtMaven.run inputs are more complicated. The initial confusion over the rtMaven.run input was cleared up after reading the Groovy documentation. The input, pom: 'pom.xml', goals: 'clean install' is a single map parameter. When a map is passed as a method parameter, the square brackets may be omitted. They must be omitted when the method parentheses are omitted.

rtMaven.run pom: 'pom.xml', goals: 'clean install' //Valid
rtMaven.run(pom: 'pom.xml', goals: 'clean install') //Valid
rtMaven.run([pom: 'pom.xml', goals: 'clean install']) //Valid
rtMaven.run [pom: 'pom.xml', goals: 'clean install'] //Error: Invalid

checkout([
  $class: 'GitSCM',
  branches: [[name: '*/master']],
  userRemoteConfigs: [[ url: 'http://git-server/user/repository.git' ]]
]) //Valid

checkout $class: 'GitSCM',
  branches: [[name: '*/master']],
  userRemoteConfigs: [[ url: 'http://git-server/user/repository.git' ]] //Valid

checkout [
  $class: 'GitSCM',
  branches: [[name: '*/master']],
  userRemoteConfigs: [[ url: 'http://git-server/user/repository.git' ]]
] //Error: Invalid

Groovy maps — like objects or dictionaries or any other similar concept in other languages — map keys to values. In the previous examples, both rtMaven.run and checkout are being passed a single map parameter. It is because of Groovy’s optional syntax — that at first glance — they seem so disimilar.

Wrap up

Closures, methods and maps are just a few examples of Groovy concepts that Jenkins leverages for its scripted pipeline syntax. While each is only briefly touched upon here, more in-depth detail can be found in the Groovy documentation. The Jenkins site may be sparse in its detailed explanation of the underlying syntax, but the information on the Groovy site and related blogs and articles helps fill in the gaps. It is so much easier to create a scripted pipeline with a firm understanding of the core fundamentals.