Before and After Docker Init

We are going to talk about Docker init, this new feature of the Docker client. And we are going to see what we were doing before that, what we are doing with Docker init, and what we can do in the future with this, with this comment. So my name is Angel Borroy, and I’m a Developer Advocate at Hyland. I’m also teaching in the university. I’m a Docker community leader. So I’m really involved with open source containers and so on, mainly because of the products we are working with. So they are open source products related to document management work.

Introduction

The main problem we are trying to address today, it’s a matter of assets. So we need to create some assets for Docker to be run. And these assets are not only for developers, but also for the pipelines, for the DevOps engineers, and also for the users. So we are trying to solve this requirement today. This is like the agenda. So before Docker init, we were just trying some experiments that we are going to present. Then we got this Docker init command with some default templates that we are going to explore. And finally, we are going to present a sample approach for community templates. Not only templates provided by Docker, but also creating the templates for yourself.

The first one is before we have this Docker init command. I’m going to explain our use case. You can have your own case, but this is what we need to solve. So we are building a project that is composed by, we say, well-sized services. So they are not microservices at all, but they are well-sized services. With this product, we are releasing the product all the time. So we have many different releases of the product. And also the services are not required on every deployment. So we need also to combine different services depending on the deployment. And also the product can be extended.

Services

You can see all the different services, the UI, that is mainly developed in Angular. We have our repository, that is developed with Java. We have transformation service, search service that are developed by the first engineers. But we have also some other services that we are using to support the deployment, like the database. This is our starting scenario. Right? So we have a lot of services, some services that we are developing, and some other services that we are using.

In fact, what we have is a lot of different choices. They are not countless because they are limited in number. But we have different releases — 6.0, 6.1, 7.1, 7.4. We have also some configuration because we can use different protocols, FTP, or SMPP, or many different protocols. We can also flag different services for transfer, for search, for some other features. We are also supporting many different databases — MariaDB, Oracle, many of them.

Then we need also to accept extensions for the product. These extensions are not only provided by the product itself, but we have also a community that is providing extensions for the product. And we need also to support these custom extensions. And finally, we need also to support the development of the other platform. So if you are developing on a service, then you need to deploy all the other parts, not all the other parts, but some of all the other services of the platform.

Tools

This is like our initial situation. For that, we were creating some tools during the years before Docker init was available. So the first one was developing Python. And we were just addressing the generation of these Docker assets on the pipeline, on the testing pipeline. So we are going to see that. After that, we found that in some use cases, we need also user interaction. So not only something that can be created in an automated way, but also the user just saying, okay, one version of this thing, or two, or I want to use this database or not. So this is why we created the Yeoman generator. That was our second move forward. And finally, we found that that includes a lot of dependencies. Because you need Yeoman, you need NPM, you need a lot of different things. So we wrap everything inside our Docker image. So the only dependency you have is Docker. And you are going to produce Docker. You are using Docker. It makes sense.

Let’s move on to this. So the first one is a Python generator. Remember that you have always the source code available for these projects. It’s on the link up on the right. And in this case, we wanted to create something that is designed to be part of the testing pipeline of these specific service for this search service. So this service required some other services. And we were creating this one in order to produce different Docker configuration for different use cases.

So we were inventing some templates. You can see this one with some expressions, some replacement expressions, like this Solr host. And we were also using our imagination. When we don’t have a variable, we were adding something that populated — I will populate that later. So that was started by my colleague, Tom Page. But we were also evolving that with time. So it was fine. So we were able to create with a single command, Python generator with a different parameter.

I want to deploy Alfresco with the transformation and the AIO transformers. And that was producing the Docker assets for that. That was automatic. And it only required Python and this model. So it was fast. But when we want to move to the user interaction, yes, continuing that development in Python, it was not the best choice. Because you know that we have this Yeoman generator that is providing all the user interaction. So it’s providing by default the user interaction. It has also a template language, so you can add your expressions to the templates. And you can also get the user interaction. That’s fine. And that was designed to deploy the full stack, not only the server services and the related services, but the full stack, given the user interaction. So the templates are based in Yeoman. So you can see more or less the same thing.

Conditions

Now we have also conditions. So we can use conditions on the information that was typed by the user. We have also some replacement. But this gives us the ability to create more complex deployments or just to create something that is considered in more use cases. In this case, there are many different contributors today to the project. So this is a project that is very used in the Alfresco community.

So it’s more or less the same thing. We can use a command-line invocation with parameters. So we are also supporting the previous use case. You can just create everything only using parameters, or you can reply to interactive problems. And this interactive prompts; we are going to see that later. So we can select different versions. We can type in a value. We can also reply to a Boolean question, yes or no. Or we can even have a multiple choice. If we want to deploy more than one module, then we have also the multiple choice in that. So it looks pretty good because we are able also to create different things with both approaches.

Include directive

On the version two of this one, because in the end, these templates were like two complex. So we have many conditionals. We have many different sections of the code. And in the end, we have like, I don’t know, 800 lines of template. That was a mess. So this is why on the second iteration of the same project, we invented the include directive.

You know that Docker Compose is supporting include on the latest release. So we were doing more or less the same thing, but have many. We had some comments on the original file. And then we have the partial declaration of the service in another file. If you want to use the service, I’m replacing this one by the actual configuration of the service.

So with that, the templates were easier to read. So this was our second version. And, as we found that some people found problems to install Yeoman, NPM, and all this stuff, what we did was to wrap everything inside our Docker image. That made sense. So you don’t need to take care of that request. So it’s exactly the same thing, but we have all the Yeoman, NPM, Node, and so on inside Docker. So you got all the benefits for the previous approaches. And you have also no requirements. We think that we achieved this frictionless goal. So with that solution, we have something that is able to be attached to a pipeline that allows also using interaction when you want to allow that. And that is frictionless. So it was nice.

But you can, again, you can play with that by using the product. So this is the Alfresco enterprise. Docker is one of the projects. I need to install the NPM, the Yeoman. OK. So this is the requirement. And if I type yo alfresco-enterprise-docker with no arguments, then I have the prompts. So I can choose a version of the first deployment 7.03. I can choose between different transform services. For instance, this one, I can choose between different search services. I can also choose to deploy or not an application. And the same thing for this application, another application, another service that I can use or not. And with that, we are creating the assets.

So we have the Docker Compose created with all the services I need and with the right configuration for each one. As you can see, we have also these comments that are part of the mechanisms. OK. That was fine. We were happy with it, more or less. But at that time, Docker init came. At that time, well, after five years of using that, then we have this Docker init.

What is Docker init?

What is Docker init? Docker init is available from Docker Desktop for 18. Currently, it’s still in beta status. And it’s more or less the same thing we were seeing before. So you type docker init, and this will be producing the Docker assets for you. But in this case, it’s oriented to applications, to a single application that needs to create this Docker image file and the Compose for deployment. Initially, we have this Go, Node, Python, Rust, and ASP.net. So you can create the Docker assets for projects using this kind of technologies.

It’s mainly oriented to services. So they are not like command-line applications. They are expecting to have a service there. And if you want something new, you have this “let us know” link. So you can ask for new technology support or product support to Docker. You have the official documentation. By now, this is closed source code. Despite this is part of a Docker key project, by now it is closed source code. I don’t know if that will be open in the future to be honest, but I hope so.

So on the Docker roadmap, if you know this page, they are accepting your recommendation, your requests. And for the Docker init currently, we have more support for different stacks, like Java or PHP. And we have also the Docker init — that is the one we have. And there is also that custom-template support. So this is exactly what we are going to see later, to create custom templates to that initial catalog that is quite small. It’s only creating applications for that technology. So we want to go farther with that.

Using Docker init

I don’t know who has tried Docker init before this. No one? Okay, so let’s make, again, a test. I’m going to use the Node.js deployment. So, I have a small Node.js application. I have a server, the pretty “hello world” — that is a server that is listening on port 8080 and just returning this hello world. If I type docker init, that is available by default, we can see that it’s detecting the language we have.

So we are doing that from the project itself. So it’s detecting that we have Node. So we can choose that. And after that, I’m going to get some questions, like the version of Node I want to use, the backup manager I want to use. It is detecting NPM. And what command line I want to use to start. So in my case, Node server.js, but you can use also NPM start if you want, but I prefer to use this one. And the port was 8080.

With that, we have these three Docker assets created for us. We have the first one, the .dockerignore. So it’s the local folders, the local files that I don’t want to add to the Docker image for the building process. We have the Dockerfile — that is the description for the build of the Docker image. And, finally, I have also the compose.yaml. You know that last year, I guess, they changed this convention. So docker/compose.yml has been changed to compose.yaml. So you are creating something new. You can use this new convention name.

And inside that, we have all the ignoring information. So, for Go, mainly we are not going to use the Node models folder. And we have also the Dockerfile that includes best practices. So we have the building. We have no building in this case. Okay, if you were doing something that is not compiled, as in this case, then you have the building layer and then the running layer. And you can copy the resources from the building to the running layer. But you have also some basic stuff, like not running the Docker image with the root user. So we are creating a new user. And we are using that user in order to start the service. And finally, you have to start in the service. And we have also this compose.yaml file that is including the server.

So we have everything ready. And with that, okay, let’s say, when you are ready, you have type, Docker compose, up, build. And then your service will be available. So they are giving also instructions on how to use that. So docker compose up –-build , we are expecting to have a running web server on localhost 8080. So just coming up with back, you have all these source codes on the data. If that worked, what we have on the localhost 8080 is the “Hello World” message. So it’s working fine. I have an application. I can use the Docker init to create all these Docker assets. They are applying best practices. And it’s working. So that’s fine. But we want more than that. And we need more than that. So we are moving to the final iteration. Let me stop the container because it’s fine. Okay.

Let’s move to the next one. Now that we are able to code like crazy big fools, it’s because we can create Docker with templates. We can create our own templates in order to use this Docker init. Obviously, we want that to be at us to a pipeline. We want to use an interaction. And we want that to be fixed on this.

For that, I create a sample project. You also have the project in GitHub. All the source code is a project developing goal. It’s using exactly this. I didn’t create a Docker key plugin, because this source code is just for testing purposes. It’s not oriented to production usage. But it’s using exactly the same technology that you were using a Docker key plugin. So we are using the same key interaction packets, that is Cobra for Docker. We are using a prompt definition with YAML. We want to define all these prompts we have. We are using default Go templates in order to build it in place in all these questions. And we are also using an external expression evaluator for this kind of conditions we need to add.

The client interaction is just the same, so Docker init. But in this case, in addition to that Docker init, we have some parameters. Because we want that to be also used to a pipeline. So we want to get all the parameters. On the parameters, we have the template. So the name of the template to be used, because we can have a catalog of templates. We go to the previous use case to the Docker init default. We can say use the node.js template. But we can add also the value of the prompts. And we are supporting also external templates. Not only templates that are offered by default. When we type Docker init, there is no local file on my machine. It’s inside the Docker key program. In this case, we have also the default templates. But you can just create your local templates and use a local directory in order to provide this template to the program. And, finally, we have an output directory to write all the Docker assets. We have also some other common assets, that is the catalog.

Imagine that you want to use that. But you don’t know what templates are available, or you don’t know what prompts the template is expecting. This is why you have this Docker init catalog command. You can say, okay, give me all your templates available, Alfresco, and so on. And I want to use the Alfresco template, so give me the prompts I need to use. On the right, you can see a prompts.yaml file, that is the one defining the prompts. And you can use also this with the templates that are included with the Docker init program or with external templates.

With that, we have a way to create Docker assets based on templates. If you want to try that, we are going to see today two different templates. But, obviously, you can try your own. So you just need to define a folder. The name doesn’t need to be templates. And using samples later, but you need a name. Then you need the subfolder with a template name, like Alfresco. And then inside that, you need to define one required file that is the prompts.yaml file is more or less this file, that is, including your prompts. Then you can add everything you want — always the suffix of the file is .tpl to declare that it is a template file. With that, in this case, we have, for instance, templates. Then we have the template name that is the root folder. Then I recommend you create instructions in order to run the project. So you can create a customized README, because you have conditions. If you’re including one service or the other, you can also add that to the README.

You have Docker images, like Dockerfiles, you can include source code, Docker Compose files. And the only thing that is required is the prompts definition. Great, that’s nice. And the prompt definition, this is the syntax that I invented. Obviously, we can evolve that. And this is based on Alec Aivazis’ module for Go that is allowing us to create these different prompts. So we have again this select. We declared that the identifier is volumes, the label we are going to show to the user, is volume method. And we have some options: none, native, or bind. We can also use a value prompt. What is the name of the server? We can provide also a default value, just to help the user. We have also a password, just to include as there is, instead of the actual typing. We have multiple choice. It’s the same thing, with this multiple true, you can choose more than one from the list. And, finally, we have Boolean, yes or no, prompts. Just remember to, if you have some Go knowledge, and if you want to use this module, just help this guy, because he is a bit busy, and he’s not maintaining the project anymore. So the project needs some love. If you have that, yes, please help this guy, because the project is really nice. You can see that we are building many different prompts. So just try to help him.

Finally, we need also to define the syntax for the templates. We need to use that TPL extension. We can use expressions with the values. If we go back, for instance, if we have volumes and the selection is native, we can use this native volumes equals native syntax inside the template. That is handy. There is one rule that was fine for me. I don’t know if that is really something too kinetic. But if the template is evaluated to 0 bytes, then I’m removing this stuff. Okay, we have conditional blocks. This one is just using this legacy UI from ID with the value yes. If that happens, then I’m adding this block. Also I can add values from the prompts with that syntax. But this is a standard syntax. This is from the text template packets of Go. So you have many different options to compose your templates. So you can check all the information there. And I know that this is very, very hard to understand without touching it. So we are going to touch that.

Demo

Again, remember that you have the project available, Docker init with templates is a goal project. You have also the instructions and so on. And we are going to play with two different templates. The first one is an external template folder. This sample folder is not included in the goal program, but I include that in the repository for you to use. This is like the classic Nginx golang DB sample that is just creating a Go program with Postgres or MariaDB.

For that, as I said, we have the prompts. So we can define the prompts. In this case, I have only one prompt, that is the database. Choose the database, Postgres or MariaDB. And then I have the Compose file. In this Compose file, I have a condition in order to add the MariaDB configuration or to add the Postgres configuration, the Postgres service to the cloud and Postgres service to Compose. And we have also some other finds, but we can even have conditions on the source code. Why not? So we can also use that as a template, and we can add conditions.

So, if the database is MariaDB, I will run this SQL command. If that is PostgreSQL, I’m running this other SQL command. You know, it can be easier than that. But you can add every file with a template’s expression on it. And, obviously, the Dockerfile, if you want, and again, and I think that that is pretty interesting. We can generate a README that is providing the right instructions. So in this case, yeah, we have, for instance, this. We can say if the database is MariaDB, then this line, or if not, that. So you get your instructions also for what you need.

Let’s move to this one. You can download the program. So the program is available. The compiler to the binary is available on releases. This is my latest release. I’m not a golang developer, skill developer. So just bear with my source code. And use that as reference for nothing. But you can use this compiled program, the binary. So I have this binary on my machine. I rename it as fake Docker. Just not to have any legal implication. And I have also a local folder that is the sample folder. That is exactly the templates and the prompt definition. So with that, we can just say fake Docker init. If we say catalog, then that is only providing the templates that are on the binary. That is a first go by default. But if I add the samples, the local samples folder, then I have Alfresco nginx-golang-db. And if I want to get into details of the template, then I can also ask for details. So I know that I need a database parameter with the low-weight values of Postgres or MariaDB. And if we want to create the assets for that, we can say fake Docker. We can say fake Docker init. We have all the parameters. We need to add the samples directory, because I’m going to use a template that is on a local folder. And I can choose the one I want to use. That is the Nginx Golang DB. So just with that, I need to choose Postgres or MariaDB. I’m choosing that. And we have the result.

We have the result with only the configuration that’s too Postgres. And if we run that, believe me, it works. And it’s also showing the Postgres configuration. You can see that this, for instance, this template of the README is only containing Postgres information. Okay? So that’s fine. That’s easy. We have a very easy sample for that. So you can create your own, because you just need to define your prompts to apply these expressions to your templates. And it’s done. But if we move to something a bit more complex, that was the thing I was trying to solve for myself in a selfish way, we can go to the template for Alfresco. That has many different services and many different files involved.

For that, what I applied is the Docker include directive, the Docker Compose include directive. That allows me to have a list of services that I can switch or I can remove when I need that because of the configuration. So, do you know that Docker Compose include directive, that is also new? Who has played with that? No one? So it’s simple to define the compose.yaml individually. When you can use more than one service, but I’m using that for an individual service. I have this proxy compose.yaml that includes only the information of the proxy. Or I have this legacy UI that includes only this information. And when I create a list with all of them, that is like creating a single compose.yaml file with all the services. But this allows me some nice features like this one. I’m only going to include the messaging service if you choose to deploy the messaging service. And if not, I can ignore that. So I’m combining new technologies.

Also, in this case, the prompts are more complex. We are going to see that. We have conditions also — I guess I didn’t explain conditions. If you choose to deploy the messaging service, then I’m going to ask you if you want credentials for the service. If you are not deploying messaging services, I’m not going to use this prompt. And the same thing for the next one. So you can define conditions on the prompt. And you can also have this latest one, where you can choose more than one option. Okay, but again, we can go back to this. And we can do fake-docker-init. In this case, as this template is inside the binary, I just don’t need the external folder. So I say Docker in the first code.

The first thing, as I said, we have many different versions. I’m going to trade with version 7.3. Then the database, and including only two, but we are supporting five or six different database engines. We are going to deploy Postgres. And what do you want? Use it or not the messaging service, yes? OK, let’s do that. But without credentials. OK, you see that without credentials, yes, the next prompt that was just give me the credentials. Then do you want to use the legacy UI? That’s fine. Which container volume method do you want to use? Native. Then enter the server with a default value, if I hit enter, then the value will be localhost. I can choose more than one item. With that, I have all the Docker assets created, and I have also customized README.

So, as you were using the messaging service, I can give you the endpoint for the messaging service. If you are not selecting that, I can exclude that.

And just a final test, let’s try to save. That works. I’m not only producing Docker assets, but I’m producing the expected Docker assets. This is what I am testing. Okay, so this is starting all the many different services that we use in Alfresco. And after a time, I can get Alfresco up and running. And I can see that this configuration, it’s fine. So here we go. Okay, so we have Alfresco running with all the options we choose. Great.

That was everything for today. We have the things we were doing before, the things we can do with the Docker init now, and the things we can do with an extension, with more features of the Docker init command. So I recommend you to make some noise, because this was my use case. And I was solving my use case in a selfish way. But if you have some other requirements, you have a write-in issue on the Docker key project, and they will consider your request or try to create something that helps you with it.

Q&A

Do you have some questions? Thanks.

I was wondering how you deal with ongoing maintenance after you’ve set it up. So this looks like a really good tool for the initial creation of the project with the various Docker Compose files and so on. But what if you want to, in the future, make a modification? You want to add a new service. You want to change a version of something. Or, let’s say, you found a problem in your initial templates, and you want to fix that in projects you’ve already used Docker init to create.

Yeah. The first thing is this is not for production, yes, please don’t use that in a production environment. That I guess that is why we have versions. So I’m creating versions of the default templates, and you can create also versions of your templates. So a nice addition would be to add the version to the template.

The project is open. Pull requests are welcome. So if you want to do that, I would be more than happy to get that contribution. I don’t know if I’m going to maintain this project in the future, because it was created for DockerCon as a sample. But if we find it useful, I will be joining to that to grow this until Docker says, no more. I’m going to add that to the product. This is why I say that we need to make noise. I hope that was all useful for you. Thanks.

Learn more

• Docker Init: Initialize Dockerfiles and Compose files with a single CLI command
• Docker init docs
• Get the latest release of Docker Desktop.
• Vote on what’s next! Check out our public roadmap.
• Have questions? The Docker community is here to help.
• New to Docker? Get started.
• Subscribe to the Docker Newsletter.