“Build once, deploy anywhere” is really nice on the paper but if you want to use ARM targets to reduce your bill, such as Raspberry Pis and AWS A1 instances, or even keep using your old i386 servers, deploying everywhere can become a tricky problem as you need to build your software for these platforms. To fix this problem, Docker introduced the principle of multi-arch builds and we’ll see how to use this and put it into production.
Quick setup
To be able to use the docker manifest command, you’ll have to enable the experimental features.
On macOS and Windows, it’s really simple. Open the Preferences > Command Line panel and just enable the experimental features.
On Linux, you’ll have to edit ~/.docker/config.json and restart the engine.
Under the hood
OK, now we understand why multi-arch images are interesting, but how do we produce them? How do they work?
Each Docker image is represented by a manifest. A manifest is a JSON file containing all the information about a Docker image. This includes references to each of its layers, their corresponding sizes, the hash of the image, its size and also the platform it’s supposed to work on. This manifest can then be referenced by a tag so that it’s easy to find.
For example, if you run the following command, you’ll get the manifest of a non-multi-arch image in the rustlang/rust repository with the nightly-slim tag:
$ docker manifest inspect --verbose rustlang/rust:nightly-slim
{
"Ref": "docker.io/amd64/rust:1.42-slim-buster",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:1bf29985958d1436197c3b507e697fbf1ae99489ea69e59972a30654cdce70cb",
"size": 742,
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
"SchemaV2Manifest": {
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 4830,
"digest": "sha256:dbeae51214f7ff96fb23481776002739cf29b47bce62ca8ebc5191d9ddcd85ae"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 27091862,
"digest": "sha256:c499e6d256d6d4a546f1c141e04b5b4951983ba7581e39deaf5cc595289ee70f"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 175987238,
"digest": "sha256:e2f298701fbeb02568c3dcb9822f8488e24ef12f5430bc2e8562016ba8670f0d"
}
]
}
}
The question is now, how can we put multiple Docker images, each supporting a different architecture, behind the sametag?
What if this manifest file contained a list of manifests, so that the Docker Engine could pick the one that it matches at runtime? That’s exactly how the manifest is built for a multi-arch image. This type of manifest is called a manifest list.
Let’s take a look at a multi-arch image:
$ docker manifest inspect ‐‐verbose rust:1.42-slim-buster
[
{
"Ref": "docker.io/library/rust:1.42-slim-buster@sha256:1bf29985958d1436197c3b507e697fbf1ae99489ea69e59972a30654cdce70cb",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:1bf29985958d1436197c3b507e697fbf1ae99489ea69e59972a30654cdce70cb",
"size": 742,
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
"SchemaV2Manifest": { ... }
},
{
"Ref": "docker.io/library/rust:1.42-slim-buster@sha256:116d243c6346c44f3d458e650e8cc4e0b66ae0bcd37897e77f06054a5691c570",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:116d243c6346c44f3d458e650e8cc4e0b66ae0bcd37897e77f06054a5691c570",
"size": 742,
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v7"
}
},
"SchemaV2Manifest": { ... }
...
]
We can see that it’s a simple list of the manifests of all the different images, each with a platform section that can be used by the Docker Engine to match itself to.
How they’re made
There are two ways to use Docker to build a multiarch image: using docker manifest or using docker buildx.
To demonstrate this, we will need a project to play. We’ll use the following Dockerfile which just results in a Debian based image that includes the curl binary.
ARG ARCH=
FROM ${ARCH}debian:buster-slim
RUN apt-get update \
&& apt-get install -y curl \
&& rm -rf /var/lib/apt/lists/*
ENTRYPOINT [ "curl" ]
Now we are ready to start building our multi-arch image.
The hard way with docker manifest
We’ll start by doing it the hard way with `docker manifest` because it’s the oldest tool made by Docker to build multiarch images.
To begin our journey, we’ll first need to build and push the images for each architecture to the Docker Hub. We will then combine all these images in a manifest list referenced by a tag.
# AMD64
$ docker build -t your-username/multiarch-example:manifest-amd64 --build-arg ARCH=amd64/ .
$ docker push your-username/multiarch-example:manifest-amd64
# ARM32V7
$ docker build -t your-username/multiarch-example:manifest-arm32v7 --build-arg ARCH=arm32v7/ .
$ docker push your-username/multiarch-example:manifest-arm32v7
# ARM64V8
$ docker build -t your-username/multiarch-example:manifest-arm64v8 --build-arg ARCH=arm64v8/ .
$ docker push your-username/multiarch-example:manifest-arm64v8
Now that we have built our images and pushed them, we are able to reference them all in a manifest list using the docker manifest command.
$ docker manifest create \
your-username/multiarch-example:manifest-latest \
--amend your-username/multiarch-example:manifest-amd64 \
--amend your-username/multiarch-example:manifest-arm32v7 \
--amend your-username/multiarch-example:manifest-arm64v8
Once the manifest list has been created, we can push it to Docker Hub.
$ docker manifest push your-username/multiarch-example:manifest-latest
If you now go to Docker Hub, you’ll be able to see the new tag referencing the images:
The simple way with docker buildx
You should be aware that buildx is still experimental.
If you are on Mac or Windows, you have nothing to worry about, buildx is shipped with Docker Desktop. If you are on linux, you might need to install it by following the documentation here https://github.com/docker/buildx
The magic of buildx is that the whole above process can be done with a single command.
$ docker buildx build \
--push \
--platform linux/arm/v7,linux/arm64/v8,linux/amd64 \ --tag your-username/multiarch-example:buildx-latest .
And that’s it, one command, one tag and multiple images.
Let’s go to production
We’ll now try to target the CI and use GitHub Actions to build a multiarch image and push it to the Hub.
To do so, we’ll write a configuration file that we’ll put in .github/workflows/image.yml of our git repository.
name: build our image
on:
push:
branches: master
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: checkout code
uses: actions/checkout@v2
- name: install buildx
id: buildx
uses: crazy-max/ghaction-docker-buildx@v1
with:
version: latest
- name: build the image
run: |
docker buildx build \
--tag your-username/multiarch-example:latest \
--platform linux/amd64,linux/arm/v7,linux/arm64 .
Thanks to the GitHub Action crazy-max/docker-buildx we can install and configure buildx with only one step.
To be able to push, we now have to get an access token on Docker Hub in the security settings.
Once you created it, you’ll have to set it in your repository settings in the Secrets section. We’ll create DOCKER_USERNAME and DOCKER_PASSWORD variables to login afterward.
Now, we can update the GitHub Action configuration file and add the login step before the build. And then, we can add the --push to the buildx command.
...
- name: login to docker hub
run: echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_USERNAME }}" --password-stdin
- name: build the image
run: |
docker buildx build --push \
--tag your-username/multiarch-example:latest \
--platform linux/amd64,linux/arm/v7,linux/arm64 .
We now have our image being built and pushed each time something is pushed on master.
Conclusion
This post gives an example of how to build a multiarch Docker image and push it to the Docker Hub. It also showed how to automate this process for git repositories using GitHub Actions; but this can be done from any other CI system too.
An example of building multiarch image on Circle CI, Gitlab CI and Travis can be found here.