Running Testcontainers Tests on GitLab CI

Testcontainers is a testing library that enables you to run your tests with dependencies like databases, message queues, search engines, etc., using ephemeral Docker containers. Testcontainers manage the lifecycle of the Docker containers using programmable API, which gives finer control over the required application dependencies setup.

GitLab is a popular Git-based source code management (SCM) platform. In addition to being an SCM platform, GitLab provides CI/CD, issue management, and code review capabilities.

This article will look into how to run Testcontainers-based tests on the GitLab CI Platform. We’ll resolve the problem of lacking a Docker environment in the default containerized runners by using the Docker-in-Docker pattern, and then both simplify the setup and speed up the execution by configuring tests to run with Testcontainers Cloud.

We’re going to use an example Java/SpringBoot application, which you can find on GitHub if you want to follow along.

GitLab CI Pipeline Setup

• Create a new repository in GitLab or import an existing project into GitLab using the Import Project feature.
• Create a .gitlab-ci.yml file at the root of the repository in which we are going to define a pipeline including a job `test` as follows:

variables:
MAVEN_OPTS: >-
  -Dhttps.protocols=TLSv1.2
  -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository
  -Dorg.slf4j.simpleLogger.showDateTime=true
  -Djava.awt.headless=true
MAVEN_CLI_OPTS: >-
  --batch-mode
  --errors
  --fail-at-end
  --show-version
  --no-transfer-progress
image: maven:3-eclipse-temurin-19
cache:
paths:
  - .m2/repository
test:
stage: test
script:
  - 'mvn $MAVEN_CLI_OPTS verify'

We have used a Docker container-based executor and defined a test job within the test stage in which we run tests using Maven. Now, commit the changes and push them to the repository.

The pipeline will automatically be triggered and will fail with the following error: Could not find a valid Docker environment.

06:26:06.194 [testcontainers-lifecycle-0] INFO  org.testcontainers.dockerclient.DockerMachineClientProviderStrategy - docker-machine executable was not found on PATH ([/opt/java/openjdk/bin, /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin])
06:26:06.198 [testcontainers-lifecycle-0] ERROR org.testcontainers.dockerclient.DockerClientProviderStrategy - Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
   UnixSocketClientProviderStrategy: failed with exception InvalidConfigurationException (Could not find unix domain socket). Root cause NoSuchFileException (/var/run/docker.sock)As no valid configuration was found, execution cannot continue.
See https://www.testcontainers.org/on_failure.html for more details.

Testcontainers-based tests failed because the Docker environment is not available in our executor. To fix the issue, we can use the Docker-in-Docker (DinD) approach to provide a Docker environment inside our executor.

Using Docker-in-Docker

Edit the .gitlab-ci.yml file to include the Docker-in-Docker service (docker:dind) and set the DOCKER_HOST variable to tcp://docker:2375 and DOCKER_TLS_CERTDIR to an empty string.

services:
- name: docker:dind
  command: ["--tls=false"]
variables:
DOCKER_HOST: "tcp://docker:2375"
DOCKER_TLS_CERTDIR: ""
DOCKER_DRIVER: overlay2
MAVEN_OPTS: >-
  -Dhttps.protocols=TLSv1.2
  -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository
  -Dorg.slf4j.simpleLogger.showDateTime=true
  -Djava.awt.headless=true
MAVEN_CLI_OPTS: >-
  --batch-mode
  --errors
  --fail-at-end
  --show-version
  --no-transfer-progress
image: maven:3-eclipse-temurin-19
cache:
paths:
  - .m2/repository
test:
stage: test
script:
  - 'mvn $MAVEN_CLI_OPTS verify'

The pipeline will run, and the tests will now run successfully (Figure 1).

However, running DinD is not always a good practice: the setup is complex and easy to get wrong, which often leads to performance problems especially if running the containers is a significant part of the load.

This is where Testcontainers Cloud comes into the picture to make it easy to run Testcontainers-based tests simpler and more reliably. 

By using Testcontainers Cloud, you don’t have to install Docker daemon on the runner. And, containers will be running in the on-demand cloud environments so that you don’t need to use powerful CI workers with high CPU/memory for your builds.

Let’s see how to use Testcontainers Cloud with minimal setup and run Testcontainers-based tests.

Testcontainers Cloud-based setup

Here’s how you can set up Testcontainers Cloud for your GitLab pipeline. In general, the CI configuration for Testcontainers Cloud requires two things. 

The first is an agent app, which runs in user space, doesn’t require special privileges, and is small enough to download as a part of the actual “running tests” step. 

The second requirement is to configure the access token so the agent uses your Testcontainers Cloud environment.

1. Sign up for a Testcontainers Cloud account at https://app.testcontainers.cloud/signup.
2. Once you’re logged in, create an organization.
3. Navigate to the Testcontainers Cloud dashboard and generate a Service Account (Figure 2):

Next, we need to set the TC_CLOUD_TOKEN as an environment variable.

• Go to project’s Settings > CI/CD and expand the Variables section.
• Select Add variable and fill in the details:
  • Key: TC_CLOUD_TOKEN.
  • Value: Enter the Service Account Access Token.
  • Type: Variable.
  • Select Mask Variable checkbox.
  • Select Add Variable.

Now, update .gitlab-ci.yml to remove the DinD configuration and install the Testcontainers Cloud agent as follows:

variables:
MAVEN_OPTS: >-
  -Dhttps.protocols=TLSv1.2
  -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository
  -Dorg.slf4j.simpleLogger.showDateTime=true
  -Djava.awt.headless=true
MAVEN_CLI_OPTS: >-
  --batch-mode
  --errors
  --fail-at-end
  --show-version
  --no-transfer-progress
image: maven:3-eclipse-temurin-19
cache:
paths:
  - .m2/repository
test:
stage: test
script:
  - curl -fsSL https://app.testcontainers.cloud/bash | bash
  - 'mvn $MAVEN_CLI_OPTS verify'

With Testcontainers Cloud tests should run quicker as compared to the  Docker-in-Docker setup (Figure 3).

Just by enabling Testcontainers Cloud, we managed to speed up the build by almost 40%, 8 minutes compared to the initial 13!

We can also leverage Testcontainers Cloud’s Turbo mode in conjunction with build tools that feature parallel run capabilities to run our tests even faster.

In the case of Maven, we can use the -DforkCount=N system property to specify the degree of parallelization. For Gradle, we can specify the degree of parallelization using the maxParallelForks property.

We tried running the tests in parallel with the DinD setup but couldn’t get it to work reliably, because the tests were failing due to the limited CPU/memory resources. This is not surprising, given that without Testcontainers Cloud we are forced to run all the containers for all the forks and the tests themselves on the same worker machine, exhausting its capacity.

Although Testcontainers Cloud allows moving containers off the runner node, the tests themself will consume significant resources while running in parallel. To maximize the effect of the Turbo mode, let’s use a machine type with sufficient CPU/memory specs.

GitLab.com provides different machine types that you can choose based on your needs. Figure 4 shows available machine types for Linux SaaS Runners.

If you are running on a Linux runner, the default machine type is small. To use a different runner, you can specify tags as follows:

test:
tags: [ saas-linux-medium-amd64 ]
stage: test
script:
  - curl -fsSL https://app.testcontainers.cloud/bash | bash
  - 'mvn $MAVEN_CLI_OPTS verify -DforkCount=4'

By using “saas-linux-medium-amd64” runner with Testcontainers Cloud Turbo mode, the tests will run faster (Figure 5).

We can see that the tests ran much quicker by combining the build tool’s parallelization capabilities with Testcontainers Cloud Turbo Mode on a machine type with decent CPU/memory specs. All in all, parallelizing the tests allowed us to get from 13:03 minutes to run the build to a mere 4:24, which is three times faster!

Conclusion

In this article, we have explored how to run Testcontainers-based tests on GitLab.com using the Docker-in-Docker service. Then we learned how to simplify the setup using Testcontainers Cloud and run tests faster. We also explored leveraging Testcontainers Cloud Turbo mode combined with your build tool’s parallel execution capabilities to run tests faster. This made the build three times faster than the original solution and allowed us to avoid the complexity of the Docker-in-Docker configuration.

Although we have demonstrated this setup using a Java project as an example, Testcontainers libraries exist for other popular languages, too, and you can follow the same pattern of configuration to run your Testcontainers-based tests on GitLab CI in Golang, .NET, Python, and Node.js.

Learn more

• Subscribe to the Docker Newsletter.
• Get started with Testcontainers Cloud by creating a free account.
• Connect on the Testcontainers Slack.
• Learn about Testcontainers best practices.
• Get started with the Testcontainers guide.