Simple and Powerful Integration Tests with Gradle and Docker-Compose

TL;DR: The code example for this guide can be found here: https://github.com/codeal-io/tutorials/tree/main/gradle-docker-compose

One common use case for backend systems is to rely on an external system such as databases or message streams. To properly test if everything is working as expected we must have integration tests where we spin up a local instance of the external service and connect to our tests. One way of achieving it is using gradle.

By the end of this guide we will manage to:

• Spin up a postgres instance before our tests run
• Connect our service with it using random docker ports
• Put the postgres instance down once our tests finish

Step 1: Defining the docker-compose file

The simplest way to run a postgres instance locally is by using docker-compose. If you never used it before, I recommended this excelent guide

Create these two files in the same directory:

Running in the terminal docker-compose up will bring up the postgres instance and initialise the database with a table customer

Step 2: Attach the docker-compose to our test lifecycle

To manage this, we will use a gradle plugin called gradle-docker-compose-plugin (shocker)

Note that for this example we are using gradle with the kotlin DSL. The groovy DSL is very similar, so if you need it you can check the examples in the plugin documentation in the link above.

The first thing we have to do is to install the plugin to our gradle build. Add the following code to your build.gradle.kts file:

This plugin will install some tasks to our build. Running gradle tasks will show them:

Docker tasks
------------
composeBuild - Builds images for services of docker-compose project
composeDown - Stops and removes containers of docker-compose project (only if stopContainers is set to true)
composeDownForced - Stops and removes containers of docker-compose project
composeLogs - Stores log output from services in containers of docker-compose project
composePull - Builds and pulls images of docker-compose project
composePush - Pushes images for services of docker-compose project
composeUp - Builds and starts containers of docker-compose project

Now we need to customize the plugin so it starts before our tests. There is plenty of customisations available, but for now we will keep it simple. Add the following to build.gradle.kts file:

I prefer to add the docker-compose file inside the test resources folder, but that’s just personal preference. If you add it to the same folder as build.gradle.kts of the project, the useComposeFiles property is optional.

Step 3: Expose the database port to be used in the application

If you run gradle test now you will notice the logs:

> Task :composeUp
+------------+----------------+-----------------+
| Name       | Container Port | Mapping         |
+------------+----------------+-----------------+
| postgres_1 | 5432           | localhost:55009 |
+------------+----------------+-----------------+

That means that the docker-compose is up and exposing the postgres port as 55009 (or whatever port it is allocated randomly). You can’t hardcode this value in the application because every time the database starts another random port is allocated.

If you don’t want the port to be randomly allocated, change the port in docker-compose.yml from 5432 to 5432:5432 . The first number is the port that will be exposed in your local network, while the second is the port internal to docker.

What is the advantage of having the port randomly allocated? Why don’t we just define a static port every time?

This matters when you’re running large numbers of containers that use the same port by default, and you don’t want to manually assign or track alternative port numbers. For example, some continuous integration tools don’t implement proper isolation between builds, so two builds running postgres at the same time will create a conflict. You could also have multiple modules that both depend on postgres, so running tests in parallel would only be possible by changing the default port to something unique.

Luckily the plugin makes it very easy to expose random ports. Just add this to the build file:

Now in the code you can connect to the database using the jbdc url:

"jdbc:postgresql://localhost:${System.getenv("POSTGRES_TCP_5432")}/codeal"

or

"jdbc:postgresql://localhost:${System.getProperty("postgres.tcp.5432")}/codeal"

By now you have a fully working setup. The database starts and initialises when running gradle test and fully stops in the end!

Bonus step: Improving the workflow

When developing new features it’s common to run the tests multiple times. It can get time consuming having the database start and stop on every run. An easy fix for this is adding this configuration to the docker compose plugin:

This will prevent the container to stop in the end, so the plugin automatically tries to reconnect to the containers from the previous run.

This works, but the problem with this approach is that it will prevent the containers to stop in any environment, like the continuous integration. Since this behaviour is only useful in our development environment, alternatively, we can create the file ~/.gradle/init.gradle and paste with the content:

Conclusion

Running integration tests that depend on external systems is very easy to achieve using gradle and docker compose. It makes the test suite clean with separation of concerns and the build reproducible and fast.