Running Testcontainers Tests on Buildkite

Buildkite is a powerful CI/CD platform addressing the modern complex software development needs with its rich ecosystem of plugins. Buildkite provides a highly scalable CI/CD platform supporting customizable runners, dynamic pipelines, real-time test analysis, and many more features.

Testcontainers is an open source framework for provisioning throwaway, on-demand containers for development and testing use cases. Testcontainers make it easy to work with databases, message brokers, web browsers, or just about anything that can run in a Docker container.

This article will explore how to run Testcontainers-based tests on the Buildkite platform. We’re going to use an example Java/Spring Boot application, which you can find on GitHub.

Getting started with Buildkite

The Buildkite Pipelines feature uses a hybrid architecture consisting of the following:

• Buildkite dashboard: A software-as-a-service (SaaS) control panel for managing CI/CD pipelines. This coordinates work with Agents and displays results.
• Agents: Small, reliable, and cross-platform build runners. These are hosted by you, either on-premises or in the cloud. They execute the work they receive from the Buildkite dashboard.

Figure 1 shows the split in Buildkite between the SaaS platform and the agents running on your infrastructure.

To use Buildkite, you first need to sign up for an account. You can sign up using your email and password or sign up with GitHub. Once your account is created, you can create an organization, which is a central place for your pipelines, test suites, teams, and dashboards. The next step is to set up an agent.

Agent setup

The Buildkite agent is a small, reliable and cross-platform build runner that makes it easy to run automated builds on your own infrastructure. The agent’s main responsibilities are polling buildkite.com for work, running build jobs, reporting back the status code and output log of the job, and uploading the job’s artifacts.

A Buildkite agent can be installed on a wide range of operating systems and cloud platforms. Once you go to your organization’s Dashboard (​​https://buildkite.com/<organization>), click on Agents in the top navigation bar. Refer to Agent Quick Start Guides for different platforms.

In this tutorial, we will use a Ubuntu VM as an agent. You can create a Ubuntu VM locally or create a VM based on Ubuntu OS on any cloud platform like AWS, Azure, GCP, etc.

Note: The actual builds will be run on the agents. So, make sure you have allocated enough RAM and storage space to be able to run your workloads.

In the Agent Quick Start Guides, click on Ubuntu and follow the instructions to install the Buildkite agent. If you successfully installed the agent, then it will be listed in the Agents page similar to the example in Figure 2:

The actual builds will be executed on the agents, so we should have the necessary software installed to run the build pipelines. As our application is a Java/Spring Boot 3 application that uses Testcontainers for testing, we should have Java 17+ and Docker installed on the agent.

On our Ubuntu VM, let’s install Java 17 using the following command:

$ sudo apt install openjdk-17-jdk
$ java -version
openjdk version "17.0.8.1" 2023-08-24
OpenJDK Runtime Environment (build 17.0.8.1+1-Ubuntu-0ubuntu123.04)
OpenJDK 64-Bit Server VM (build 17.0.8.1+1-Ubuntu-0ubuntu123.04, mixed mode, sharing)

Next, we’ll install Docker on our Ubuntu VM by following the official documentation. The Buildkite agent will be running the agent with the user buildkite-agent. So, let’s add the user “buildkite-agent” to the docker group using the following command:

sudo usermod -aG docker buildkite-agent

For the group changes to take effect, restart the Ubuntu VM, and we should be good to run our build pipelines on our agent.

Project setup

Next, we are going to create a project on GitHub and configure it to use Buildkite Pipelines as our CI platform. If you already have an application, you can use it or create a new project.

We are going to use the testcontainers-showcase repository, which is a Spring Boot application using Testcontainers for testing.

Create .buildkite/pipelines.yml file with the following content in the root directory of the project, then commit and push the changes.

steps:
 - label: "Test"
   commands:
     - ./mvnw test
     # if you are using Gradle
     # - ./gradlew test

Buildkite Pipeline setup

Now go to Buildkite Dashboard, select the New pipeline button, and configure the pipeline for your project by following the steps below:

1. Enter the Git Repository URL as: https://github.com/testcontainers/testcontainers-showcase.git
2. Enter Name as “testcontainers-showcase”.
3. Under the Steps section, select the Add button and Read steps from the repository option. This will read the pipeline definition from the .buildkite/pipeline.yml file in the repository.
4. Select the Create Pipeline button.
5. Optionally, you can configure Webhooks as mentioned in the next step so that whenever you push any changes into the repository, the build will be triggered.

Next, select the New Build button to manually trigger the pipeline, provide a Message, and select the Create Build button. 

The Buildkite Pipeline should be executed successfully as shown in Figure 3.

We have created a Buildkite agent using a Ubuntu VM, but there are other options like using Docker containers as agents as well. Setting up Docker inside Docker containers is tricky in certain environments.

This is where Testcontainers Cloud comes into the picture to make it easy to run Testcontainers-based tests simpler and more reliably.

By using Testcontainers Cloud, you don’t even need to have Docker daemon running on the agent, and containers will be running in the on-demand cloud environments so that you don’t need to use powerful CI agents with high CPU/memory for your builds.

Let’s see how to use Testcontainers Cloud with minimal setup and run Testcontainers-based tests.

Testcontainers Cloud setup

Testcontainers Cloud helps you to run Testcontainers-based tests at scale by spinning up the dependent services as Docker containers on cloud and running your tests connecting to those services.

If you don’t have a Testcontainers Cloud account already, create an account and get a Service Account Token as follows:

1. Sign up for a Testcontainers Cloud account at https://app.testcontainers.cloud/signup.
2. Once logged in, create an organization.
3. Navigate to the Testcontainers Cloud dashboard and generate a Service account (Figure 4).

Next, we need to set the TC_CLOUD_TOKEN as an environment variable. As mentioned in the Managing pipeline secrets, we can use various secret storage services like AWS Secrets Manager, Hashicorp Vault, etc. to store your sensitive information. We can also use Buildkite agent’s environment hook to export secrets to a job.

To keep it simple for this tutorial, let’s configure TC_CLOUD_TOKEN as an environment variable using environment hooks. Create a file with name environment in /etc/buildkite-agent/hooks/ directory with the following content:

#!/bin/bash
export TC_CLOUD_TOKEN=<YOUR_TOKEN_VALUE>
set -e

Next, update the .buildkite/pipelines.yml file as follows:

steps:
 - label: "Test"
   commands:
     - sh -c "$(curl -fsSL https://get.testcontainers.cloud/bash)"
     - ./mvnw test

We have included a command before executing our tests to start the Testcontainers Cloud agent, which takes the TC_CLOUD_TOKEN as an environment variable using environment hooks.

Now if you commit and push the updated .buildkite/pipelines.yml file, then the pipeline will run the tests using Testcontainers Cloud. You should see the following logs statements indicating that the Testcontainers-based tests are using Testcontainers Cloud instead of the default Docker daemon.

Running global environment hook
$ /etc/buildkite-agent/hooks/environment
# TC_CLOUD_TOKEN added
...
...
...
04:19:19.600 [Test worker] INFO  org.testcontainers.DockerClientFactory - Connected to docker:
 Server Version: 78+testcontainerscloud (via Testcontainers Desktop 1.5.3)
 API Version: 1.43
 Operating System: Ubuntu 20.04 LTS
 Total Memory: 7407 MB

If you are a paid customer of Testcontainers Cloud, you can also leverage Testcontainers Cloud’s Turbo mode in conjunction with build tools that feature parallel run capabilities to run our tests even faster. You can check Testcontainers Cloud pricing information on the website.

In the case of Maven, we can use the -DforkCount=N system property to specify the degree of parallelization. For Gradle, we can specify the degree of parallelization using the maxParallelForks property.

We can enable parallel execution of our tests using four forks in .buildkite/pipelines.yml as follows:

steps:
 - label: "Test"
   commands:
     - sh -c "$(curl -fsSL https://get.testcontainers.cloud/bash)"
     - ./mvnw test -DforkCount=4

For more information on using Turbo Mode, please refer to the documentation.

Conclusion

In this article, we explored how to run Testcontainers-based tests on Buildkite Pipelines using the default Docker daemon. Then, we learned how to create a Testcontainers Cloud account and configure the pipeline to run tests using Testcontainers Cloud. We also explored leveraging Testcontainers Cloud TurboMode combined with your build tool’s parallel execution capabilities. 

Although we have demonstrated this setup using a Java project as an example, Testcontainers libraries exist for other popular languages, too, and you can follow the same pattern of configuration to run your Testcontainers-based tests on Azure Pipelines in Golang, .NET, Python, Node.js, etc.

Learn more

• Subscribe to the Docker Newsletter.
• Get started with Testcontainers Cloud by creating a free account.
• Have questions about Testcontainers? Connect on the Testcontainers Slack.
• Learn about Testcontainers best practices.
• Get started with the Testcontainers guide.