IVY, Putting Dependencies in Ant

Ivy – Ant Dependency Bolt On

My last assignment introduced me to Gradle. My experiences have let me run the trifecta for build tools: Ant, Maven, and Gradle. However last time I used Ant, Ivy was not managing the dependencies. Versus Maven and Gradle that have automatic dependency resolution built in. What is automatic dependency resolution? In days of yore, to compile an application, you would put the necessary files in the system libraries, OR in a local reference in LD_LIBRARY_PATH, OR to a local reference in CLASSPATH. Typically there would be some hodgepodge of JAR files in a lib directory. This directory may or may not be committed to the source repository with the code. Lets just say committing JAR files to source control, while better than not storing the JAR files, is the least desirable outcome with the tools available today.

The Basics

First off, in Gradle and Maven dependency management is build it. These tools natively understand what to do with a pom file and how to download a dependency tree. Ant existed before all these high minded ideals about automated management of binary dependencies. As a result you need to bolt Ivy on to Ant. There are two methods I use to do this. The first is the old fashion way:

1. Download the Ivy JAR from here.
2. Put it in the lib directory of your Ant installation OR in your local ant home ~/.ant/lib
3. Create your ivy.xml and ivysetting.xml

The second way is more Ivy like. The point of binary dependencies is that when the latest version changes you can get it by easily changing a config or specifying that you always want the latest version.

<target name="download-ivy" unless="skip.download">
<mkdir dir="${ivy.jar.dir}" />
<echo message="installing ivy...">
<get src="https://repo1.maven.org/maven2/org/apache/ivy/ivy/${ivy.install.version}/ivy-${ivy.install.version}.jar" dest"${ivy.jar.file}" usetimestamp="true" >
";/target>
<target name="install-ivy> depends="download-ivy" description="--> install ivy">

  <path id="ivy.lib.path">
    <fileset dir="${ivy.jar.dir}" includes="*.jar"/>
  </path>
  <taskdef resource="org/apache/ivy/ant/antlib.xml" uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path">
</target> 
Once all of this is in place you can take advantage of the:
<ivy:resolve />
ant statement. This is the statement that does the magic. However you need to keep two things in mind. First off, you no longer need to maintain a lib directory. Anything that you would put in lib and commit to source control can now be handled by Ivy. Ivy will go out and get what you need, but physically where is it? Typically in your home directory:
~/.ivy2/cache/ 
If I’ve specified my ivy.xml file right my JARs are in my cache directory, but how do I use them? You need to use Ivy to reference all of your JARs
<ivy:cachepath organisation="org.myorg" module="jar name" revision="jar version number" pathid="lib.path.id"  inline="true"> 
Using the cachepath ivy command you can build your classpath from the results of your ivy resolve and pass it to the java command. While all this is a little roundabout, it now gives Ant the same ability to handle binary dependencies as Gradle or Maven. One additional thing to keep in mind is that Ivy will not only download the dependencies you ask for but their dependencies as well and so on until the entire dependency tree is satisfied.
Tools
If you are building only one thing and your project does not rely on anything else you can stop here. Maven central or some other  repository will handle all your needs. If you are working on a multi-module project with internal dependencies you may want an artifact repository you control. This keeps your “works in progress away” from the prying eyes of the world and allows you the flexibly to control what third-party libraries people use to develop with on your project. There are several good candidates for this, Nexus, Archiva, Artifactory, etc. Whichever you choose you will be setting up an ibiblio type repository. In addition to an artifact repository that holds the JARs, you will probably want IDE integration as well. For this I recommend using Eclipse as your IDE and use the IvyDE plugin. Now you can add and remove dependencies using a GUI.
Binary Dependencies and CI
CI makes the need for binary dependencies become apparent. When you are compiling on a minutely or hourly basis (this is where you get value from your CI stack) you want to avoid rework. I have helped countless customers with their build infrastructure and I see the same mistake over and over, recompilation. They will blindly compile a module every time something that depends on it needs recompilation. This poses two issues: first an inefficiency in time and second you are never quite sure if one compilation is the same as another. I’ve seen the time inefficiency bring large projects to a grinding halt. Also, no matter how good your CM is, mistakes happen. I’ve witnessed minor compilation issues result in week long bug hunts. If you properly architect your binary dependencies, your developers can have the latest version of code at their fingertips and find integration issues earlier. Investing in using artifacts instead of commit hashes or code versions will result in finding integration errors earlier and better CM. If you have a project of significant size you should invest engineering effort here.
The Twist
If you take a little extra care in your binary repository setup, you can push the recompilation savings all the way down to the developers. Ivy allows you to apply the idea of a branch to a repository. I recommend using Archiva since others repositories do not support the following Ivy constructed hierarchy:
local filesystem branch -> binary repository integration branch -> release artifacts (branchless) 
Ivy allows you to define a local file system repository that only you can access and see. This is not exactly like, but close to the the maven snapshotting system. Ivy does not allow the timestamping of artifacts the same way maven does.
Local Repository
The local filesystem repository allows developers to modify, compile, and test only their changes. This can be helpful if several team members are working on the same branch. When the developer executes a build, they should only have to compile the code they are working on. Their end product will be assembled in the following way, newer component versions in their local snapshot repository will be pulled in with the highest precedence. Next, integration releases that have made it through the CI process but have not been through the final release testing will be pulled down from the remote artifact repository, and lastly release artifacts will be pulled down. This allows a developer to always have the most up to date versions by simply rebuilding the application.
Integration Branch
There will be a set of integration artifacts that sits in the global integration repository for each branch. I typically let the SCM tool decide the branch name, git and svn have similar limitations as Archiva concerning naming and have easy commands for accessing the branch name at the command line. The integration branch allows a team working on the same SCM branch to effortlessly share changes. In order to get into the integration branch in the binary artifact repository, I typically force the build to go through the CI engine (typically Jenkins) and survive some manner of unit and integration tests. This automatically assesses that the code in the integration environment is of a certain quality and is probably fit for sharing.
Release Repository
The release repository is branchless, because releases should be branchless. Items classified as a release should be in production or be production ready. It takes a bit of Ant trickery to allow dependencies from a branched build to resolve to a branchless repository, so this does not work out of the box. I had to customize the Ant release task to allow the hierarchy to be evaluated correctly (javascript was involved). The artifacts in this repository should have gone through significant integration testing, end to end testing, and even complete all manual testing.
Conclusions
By building once and using everywhere you can significantly decrease compilation time and be more assured of which version of the code you are using. Additionally by dropping the build time, you can allow you CI infrastructure to build more frequently. More builds give you more opportunity to execute your automated tests. The sooner you find flaws, the less expensive they are to fix. By using Ant combined with Ivy and adding a hierarchy to you binary repository structure you can realize significant CM and compilation efficiency.