Part 2 in the series on Using Docker Desktop and Docker Hub Together
Introduction
In part 1 of this series, we took a look at installing Docker Desktop, building images, configuring our builds to use build arguments, running our application in containers, and finally, we took a look at how Docker Compose helps in this process.
In this article, we’ll walk through deploying our code to the cloud, how to use Docker Hub to build our images when we push to GitHub and how to use Docker Hub to automate running tests.
Docker Hub
Docker Hub is the easiest way to create, manage, and ship your team’s images to your cloud environments whether on-premises or into a public cloud.
This first thing you will want to do is create a Docker ID, if you do not already have one, and log in to Hub.
Creating Repositories
Once you’re logged in, let’s create a couple of repos where we will push our images to.
Click on “Repositories” in the main navigation bar and then click the “Create Repository” button at the top of the screen.
You should now see the “Create Repository” screen.
You can create repositories for your account or for an organization. Choose your Docker ID from the dropdown. This will create the repository for your Docker ID.
Now let’s give our repository a name and description. Type projectz-ui in the name field and a short description such as: This is our super awesome UI for the Projectz application.
We also have the ability to make the repository Public or Private. Let’s keep the repository Public for now.
We can also connect your repository to a source control system. You have the option to choose GitHub or Bitbucket but we’ll be doing this later in the article. So, for now, do not connect to a source control system.
Go ahead and click the “Create” button to create a new repository.
Your repository will be created and you will be taken to the General tab of your new repository.
This is the repository screen where we can manage tags, builds, collaborators, webhooks, and visibility settings.
Click on the Tags tab. As expected, we do not have any tags at this time because we have not pushed an image to our repository yet.
We also need a repository for your services application. Follow the previous steps and create a new repository for the projectz-services application. Use the following settings to do so:
Repository name: projectz-services
Description: This is our super awesome services for the Projectz application
Visibility: Public
Build Settings: None
Excellent. We now have two Docker Hub Repositories setup.
Structure Project
For simplicity in part 1 of this series, we only had one git repository. For this article, I refactored our project and broke them into two different git repositories to more align with today’s microservices world.
Pushing Images
Now let’s build our images and push them to the repos we created above.
Fork Repos
Open your favorite browser and navigate to the pmckeetx/projectz-ui repository.
Create a copy of the repo in your GitHub account by clicking the “Fork” button in the top right corner.
Repeat the processes for the pmckeetx/projectz-svc repository.
Clone the repositories
Open a terminal on your local development machine and navigate to wherever you work on your source code. Let’s create a directory where we will clone our repos and do all our work in.
$ cd ~/projects
$ mkdir projectzNow let’s clone the two repositories you just forked above. Back in your browser click the green “Clone or download” button and copy the URL. Use these URLs to clone the repo to your local machine.
$ git clone https://github.com/[github-id]/projectz-ui.git ui
$ git clone https://github.com/[github-id]/projectz-svc.git services(Remember to substitute your GitHub ID for [github-id] in the above commands)
If you have SSH keys set up for your github account, you can use the SSH URLs instead.
List local images
Let’s take a look at the list of Docker images we have locally on our machine. Run the following command to see a list of images.
$ docker images
You can see that I have the nginx, projectz-svc, projectz-ui, and node images on my machine. If you do not see the above images, that’s okay, we are going to recreate them now.
Remove local images
Let’s first remove projectz-svc and projectz-ui images. We’ll use the remove image (rmi) command. You can skip this step if you do not have the projectz-svc and projectz-ui on your local machine.
$ docker rmi projectz-svc projectz-ui
If you get the following or similar error: Error response from daemon: conflict: unable to remove repository reference "projectz-svc" (must force) - container 6b1b99cc899c is using its referenced image 6b9eadff19ae
This means that the image you are trying to remove is being used by a container and can not be removed. You need to stop and rm (remove) the container before you can remove the image. To do so, run the following commands.
First, find the running container:
$ docker ps -a
Here we can see that the container named services is using the image projectz-svc which we are trying to remove.
Let’s stop and remove this container. We can do this at the same time by using the --force option to the rm command.
If we tried to remove the container by using docker rm services without first stopping it, we would get the following error: Error response from daemon: You cannot remove a running container 6b1b99cc899c. Stop the container before attempting removal or force remove
So we’ll use the --force option to tell Docker to send a SIGKILL to the container and then remove it.
$ docker rm --force services
Do the same for the UI container, if it is still running.
Now that we stopped and removed the containers, we can now remove the images.
$ docker rmi projectz-svc projectz-ui
Let’s list our images again.
$ docker images
Now you should see that the projectz-ui and projectz-services images are gone.
Building images
Let’s build our images for the UI and Services projects now. Run the following commands:
$ cd [working dir]/projectz/services
$ docker build --tag projectz-svc .
$ cd ../ui
$ docker build --tag projectz-ui .If you would like a more in-depth discussion around building images and Dockerfiles, refer back to part 1 of this series.
Pushing images
Okay, now that we have our images built, let’s take a look at pushing them to Docker Hub.
Tagging images
If you look back at the beginning of the post where we set up our Docker Hub repositories, you’ll see that we created the repositories in our Docker ID namespace. Before we can push our images to Hub, we’ll need to tag them using this namespace.
Open your favorite browser and navigate to Docker Hub and let’s review real quick.
Login to Hub, if you’ve not already done so, and take a look at the dashboard. You should see a list of images. Choose your Docker ID from the dropdown to only show images associated with your Docker ID.
Click on the row for the projectz-ui repository.
Towards the top right of the window, you should see a docker command highlighting in grey.
This is the Docker Push command followed by the image name. You’ll see that this command uses your Docker ID followed by a slash followed by the image name and tag, separated by a colon. You can read more about pushing to repositories and tagging images in our documentation.
Let’s tag our local images to match the Docker Hub Repository. Run the following commands anywhere in your terminal.
$ docker tag projectz-ui [dockerid]/projectz-ui:latest
$ docker tag projectz-svc [dockerid]/projectz-svc:latest(Remember to substitute your Docker ID for [dockerid] in the above commands)
Now list your local images and see the newly tagged images.
$ docker images
Pushing
Okay, now that we have our images tagged correctly, let’s push our images to Hub.
The first thing we need to do is make sure we logged into Docker Hub on the terminal. Although the repositories we created earlier are “public”, only the owner of the repository can push by default. If you would like to allow folks on your team to be able to push images and manage repositories. Take a look at Organizations and Teams in Hub.
$ docker login
Login with your Docker ID to push and pull images from Docker Hub...
Username:
Enter your username (Docker ID) and password.
Now we can push our images.
$ docker push [dockerid]/projectz-ui:latest
$ docker push [dockerid]/projectz-svc:latest
Open your favorite browser and navigate to Docker Hub, select one of the repositories we created earlier and then click the “Tags” tab. You will now see the images and tag we just pushed.
Automatically Build and Test Images
That was pretty straightforward but we had to run a lot of manual commands. What if we wanted to build an image, run tests and publish to a repository so we could deploy our latest changes?
We might be tempted to write a shell script and have everybody on the team run it after they completed a feature. But this wouldn’t be very efficient.
What we want is a continuous integration (CI) pipeline. Docker Hub provides these features using AutoBuilds and AutoTests
Connecting Source Control
Docker Hub can be connected to GitHub and Bitbucket to listen to push notifications so it can trigger AutoBuilds.
I’ve already connected my Hub account to my GitHub account. To connect your own Hub account to your version control system follow these simple steps in our documentation.
Setup AutoBuilds
Let’s set up AutoBuilds for our two repositories. The steps are the same for both repositories so I’ll only walk you through one of them.
Open Hub in your browser, and navigate to the detail page for the projectz-ui repository.
Click on the “Builds” tab and then click the “Link to GitHub” button in the middle of the page.
Now in the Build Configuration screen. Select your organization and repository from the dropdowns. Once you select a repository, the screen will expand with more options.
Leave the AUTOTEST setting to Off and the REPOSITORY LINKS to Off also.
The next thing we can configure is Build Rules. Docker Hub automatically configures the first BUILD RULE using the master branch of our repo. But we can configure more.
We have a couple of options we can set for build rules.
The first is Source Type which can either be a Branch or a Tag.
Then we can set the Source, this is referring to either the Branch you want to watch or the Tag name you would like to watch. You can enter a string literal or a RegExp that will be used for matching.
Next, we’ll set the Docker Tag that we want to use when the image is built and tagged.
We can also tell Hub what Dockerfile to use and where the Build Context is located.
The next option turns off or on the Build Rule.
We also have the option to use the Build Cache.
Save and Build
We’ll leave the default Build Rule that Hub added for us. Click the “Save and Build” button.
Our Build options will be saved and an AutoBuild will be kicked off. You can watch this build run on the “Builds” tab of your image page.
To view the build logs, click on the build that is in progress and you will be taken to the build details page where you can view the logs.
Once the build is complete, you can view the newly created image by clicking on the “Tags” tab. There you will see that our image was built and tagged with “latest”.
Follow the same steps to set up the projectz-svc repository.
Trigger a build from Git Push
Now that we see that our image is being built, let’s make a change to our project and trigger it from git push command.
Open the projectz-svc/src/routes.js file in your favorite editor and add the following code snippet anywhere before the module.exports = appRouter line at the bottom of the file.
...
appRouter.get( '/services/hello', function( req, res ) {
res.json({ code: 'success', payload: 'World' })
})
...
module.exports = appRouterSave the file and commit the changes locally.
$ git commit -am "add hello - world route"
Now, if we push the changes to GitHub, GitHub will trigger a webhook to Docker Hub which will in turn trigger a new build of our image. Let’s do that now.
$ git push
Navigate over to Hub in your browser and scroll down. You should see that a build was just triggered.
After the build finishes, navigate to the “Tags” tab and see that the image was updated.
Setup AutoTests
Excellent! We now have both our images building when we push to our source control repo. But this is just step one in our CI process. We should only push new images to the repository if all tests pass.
Docker Hub will automatically run tests if you have a docker-compose.test.yml file that defines a sut service. Let’s create this now and run our tests.
Open the projectz-svc project in your editor and create a new file name: docker-compose.test.yml and add the following yaml.
version: "3.6"
services:
sut:
build:
context: .
args:
NODE_ENV: test
ports:
- "8080:80"
command: npm run test
Commit the changes and push to GitHub.
$ git add docker-compose.test.yml
$ git commit -m “add docker-compose.test.yml for hub autotests”
$ git push origin master
Now navigate back to Hub and the projectz-svc repo. Once the build finishes, click on the build link and scroll to the bottom of the build logs. There you can see that the tests were run and the image was pushed to the repo.
If the build fails, you will see that the status turns to FAILURE and you will be able to see the error in the build logs.
Conclusion
In part 2 of this series, we showed you how Docker Hub is one of the easiest ways to automatically build your images and run tests without having to use a separate CI system. If you’d like to go further you can take a look at: