Upstream-friendly JVM development and testing with Docker

I recently experienced some bizarre failures running Akka actors and sbt tests in forked mode on my Fedora laptop. As far as I can tell, the root of both problems was that my network configuration changed when switching wireless networks, which caused java.net.InetAddress.getLocalHost() to return my WAN IP address (which processes on my machine couldn’t bind to or connect to) instead of my LAN IP address.¹ This was especially frustrating since the relevant changes to my network configuration happened not when I switched networks, but after waking my laptop from sleep on the new network – so tests succeeded before sleep but failed after it!

My patience for troubleshooting networking configurations has diminished substantially since the first time I set up pppd on a Linux 1.1 system and I was unwilling to disconnect from the network just to run tests. Fortunately, running my tests in a Docker container proved to be a fairly straightforward workaround – Java was able to correctly identify the right address to bind to when running on a bridged network and the tests ran at native speed. Maintaining a Docker container for Java development has other benefits, as well: while I usually want to work within the Fedora Java ecosystem (to ensure that I can package my work for Fedora), if I’m working on patches to go to upstream projects, I want an environment that more closely resembles the one that upstream CI and conventional deployments will run under.

I’ve created and published a minimal Docker image for a conventional JVM testing environment. This image is based on the Centos 7 image; while I love Fedora for day-to-day and development use, Centos is a much slower-moving target and is thus far more common as a deployment target. My image also installs java, java-devel, maven, ivy, git, and emacs via yum and pulls down Paul Phillips’ excellent sbt-extras script to provide ready access to multiple versions of sbt.

Here’s how to get started using it on Fedora: first, you’ll need to install Docker and start up the Docker daemon (if you haven’t already). The gpasswd line is optional but will enable you to run Docker commands as yourself without su or sudo:

sudo yum install -y docker-io
sudo gpasswd -a ${USER} docker
sudo systemctl start docker

Then you’ll want to pull my image from the Docker Hub (run this and all Docker commands under sudo if you didn’t add yourself to the Docker group above):

docker pull willb/java-dev

You should then be able to see that the image file is installed:

docker images willb/java-dev

Now you’re ready to start a container from the image. I keep most of my ongoing development work in subdirectories of ${HOME}/devel and I’d like to mount that tree from within my container; replace ${HOME}/devel below with whatever path from the host you’d like to expose to the container:

docker run -i -t -v ${HOME}/devel:/devel willb/java-dev:centos7

This will start up the container interactively (-i), allocating a TTY (-t), mounting ${HOME}/devel from the host under /devel in the container, and giving you a zsh prompt. You can now run sbt and Maven builds and tests in a vanilla, upstream-friendly CentOS environment while still using Fedora as your desktop OS.

Regular readers of this site know that I’ve been interested in using and contributing to Apache Spark for quite some time now, and I was running Spark’s test suite when I ran into the network trouble that motivated this post. If you’re also working on Spark, you might want to try an alternate image that has a prepopulated Ivy cache for Spark’s dependencies:²

docker run -i -t -v ${HOME}/devel:/devel willb/java-dev:centos7-spark

Running your initial Spark build and tests in a container with the centos7-spark image will be faster since the necessary artifacts are baked into the image. (If you don’t have the Ivy cache, you’ll need to download dependencies every time you run your first build in a container based on this image.) The easiest way to build such a cache for sbt projects is to have the Dockerfile clone the relevant source repository into a temporary directory and then run sbt update from that clone. Let me know if you build other images with prepopulated dependency caches!

Footnotes

1. Java networking on Linux seems to have a lot of corner cases; StackOverflow and other question/answer sites are full of people who are frustrated that InetAddress.getLocalHost() returns the loopback address instead of a LAN IP, but I had the opposite problem.

2. This image should include all of the necessary build and test dependencies for the Spark master branch as of when I last generated it; I will update it periodically but it may not necessarily contain everything necessary to build and test the current Spark master branch in the future.