On-Demand Training

Container Orchestration 101

 

Transcript

Hello, welcome to this tech talk session about container orchestration. In previous sessions, we’ve seen that images and containers are a standard way to easily run and distribute applications across computers and servers. However, a production machine often needs to operate multiple containers. We have already glimpsed this with Docker Compose, but typically such a machine will run in groups of containers for different teams or complex applications that include the databases, microservices, monitoring, observability, and so on. At this point, we will need tools to automate the deployment management, scaling, and networking of these containers. It is from this point that we start talking about container orchestration.

 

Table of Contents

 

What is Container Orchestration? (0:45)

Container orchestration is the process of automating the deployment, management, and scaling of containerized applications across multiple host environments. It helps organizations efficiently run and manage their containerized workload at scale.

So we need orchestration because of the scalability, orchestration enables the automatic scaling of containers based on demand and showing optimal resource utilization. The reliability orchestration provides self-healing capabilities, automatically detecting and replacing failed containers. Under consistency, orchestration ensures standardized and predictable environment for deploying and running containerized applications.

The main key features of container orchestration are the service discovery. You can automatically identify and connect containers within a distributed system. Load balancing: the system distributes incoming traffic across multiple containers for improved performance and availability. Scaling: the system can automatically scale container up or database on the resource utilization and demand.

There are several orchestrators in the industry now, each with their own features and trade-offs. We have, for example, Swarm, initially developed by Docker, now Swarm is a Mirantis product. We have Elastic container service by AWS. It is deeply integrated with the other services. We have Nomad from HashiCorp, it’s another popular orchestration service and it’s flexible enough to orchestrate several types of workloads, including container and virtual machines. And of course, we have Kubernetes, which is the leading open source container orchestration platform providing a robust and feature-rich solution. Kubernetes is probably the most commonly used system, perhaps the most complicated tool. Managing Kubernetes cluster is a real job.

To help illustrate the idea of orchestration, let’s go through a little story. Imagine we wanted to run three containerized applications. First, we need a machine to run on. That machine, will need a container engine. To run our containers, we can simply SSH and run Docker Run to launch three containers. And now we have our containers up and running. But if we want to run highly available, we should have multiple copies. So let’s spin up a second node, and a third node. Now, we have several copies to ensure we can still respond if we have a machine failure. But now if we need to deploy and update to one of our containers, we have to touch three machines.

Instead, imagine if we had a controller, much like an air traffic controller that can monitor the fleet of machines. What if we could simply tell it, I want three copies of each of these applications. We could then run smaller agent on each of the nodes that communicate back and forth with the controller to pass along current stages, new events and receive jobs to run. That agent can then interact with the host container engine to spin up, tear down and monitor the running of the containerized workloads. At the end of the day, this is orchestration in a nutshell. As far as how it works, each system uses different communication protocols, has different ways to tell the controller what you want to do, and so on. But fundamentally, this is how they all work.

Container orchestration is the umbrella term used for systems that observe and manage containers at scale. That observing can be on one node or thousands of nodes. All of them use some form of declarative statements where a user, either a person or machine, declares a desired state: “I want three copies of this container only.” The system then works to make the actual state reflect that desired state. This feedback loop is called the reconciliation loop. Basically, the system is constantly observing the current state through event-driven mechanism and then performing actions to best line the actual state with the desired state.

To do a quick summary about the container orchestration, I would say that thanks to the orchestration, you will get several benefits. The scalability, the system will automatically scale, or then scale the containers based on demand for improved resource utilization. The automation, the deployment, management and scaling of containerized applications become simpler thanks to the automation. The reliability, thanks to an orchestrator, you will guarantee the high availability and fault tolerance, through self-healing mechanisms. And finally, the efficiency. You will gain all the tools to optimize the resource utilization and to reduce the operational override.

But that means you will face some challenges like the network complexity, managing and securing network communication between containers can be challenging, the security, ensuring the security of containers and the overall orchestration platform is critical, and the monitoring and observability, providing visibility into the performance and health of containerized application is essential.

 

Kubernetes: first contact (6:22)

Now we are ready to talk about Kubernetes, not because it’s a solution we endorse and promote, but because we need to pick one to demonstrate. And right now, it’s probably the most used solution. The big advantage of containers is that you can use the same container workload in whatever orchestration platform you would like to use. The Kubernetes cluster is a powerful tool for automating the deployment, scaling and operations of application containers across a group of hosts. It is especially useful in managing complex applications with many components across multiple machines.

But first, a quick background. Kubernetes is the baby of an internal project at Google. This project is the Borg platform. Google created Kubernetes and donated it to the CNCF. Kubernetes stands for Helmsman in Greek. You can use the abbreviation k8s, but you will read Kubernetes. Again, Kubernetes is designed to be extremely flexible and extensible. So it can be complicated to spin up. By the way, in Docker Desktop, there is a feature to launch a single node cluster, but there are a variety of other ways to run a cluster as well.

Now, let’s talk about the main components of Kubernetes cluster. First, the control plane. The control plane is the brain of the cluster. It makes global decisions about the cluster, like scheduling applications, detecting and responding to cluster events, like starting a new container when one fails. The control plane includes several key components, like the API server, it serves at the front-end for Kubernetes. The user’s management tools and components of the cluster all communicate through the API server. Etcd: It’s a reliable distributed data store that constantly stores the cluster configuration and state. Scheduler, it decides which nodes should run the newly created pods based on resource availability. And the controller manager, it runs controller processes which control tasks in the cluster, such as maintaining the desired states, managing different kinds of resources, and so on.

The second main component of a cluster is the node. Nodes are the workers that run new applications. The Kubernetes cluster typically has several nodes, which help Kubernetes span multiple containers and services. Each node includes a kubelet. It’s an agent running on each node that ensures containers are running in the pod. I will speak about the pod in a moment. The container runtime, the software that is responsible for running containers, and the Kube proxy. It maintains network calls on nodes. This network calls a low network communication to your pods from network sessions inside or outside of your cluster.

When a deployment is created in Kubernetes, the control plane takes the configuration of the desired state for your application and makes it happen. The control plane schedules the pods to run on various nodes in the cluster. The control plane scheduler selects the most suitable node for each pod, balancing the workload effectively across the cluster. The kubelet on each node communicates with the control plane to receive instructions and ensure that the containers specified in the pod definition are up and running. Meanwhile, kube proxy manages the networking roles to allow traffic to flow to and from these containers.

Kubernetes provides a lot of different ways to deploy containerized applications, which are called pods in Kubernetes. A ReplicaSet, it lets me run multiple instances of a pod, which is useful to ensure high availability of an application. But most of the time, you will use a deployment. A deployment manages the roll-out of updated applications and actually it uses ReplicaSets another way. A DaemonSet, it lets you run a pod on every node in a cluster, which is useful for storage controllers, network tooling, log or metrics collectors and more. A job lets you run a pod to completion, such as a database migration or backup task and so on. A CronJob then lets you run that job on a schedule.

The pod is the smallest unit deployable in a cluster and the pod can have several states in its life. Unknown: container images are not yet been created. Running: bound to a node, all containers are created at least one container is running. Succeed: all containers are terminated with success and will not be restarted. Failed: at least one container is in failure. And Unknown, the status of pod could not be obtained. Sometimes it’s a temporary problem of communication with the host of the pod for example.

We have the Namespaces. Namespaces is a way of isolation. So the resources names must be unique inside the same namespace. Each resource belongs to only one namespace. And if you delete a namespace, you delete all the related resources of the namespace.

 

Deploy on Kubernetes with Docker Desktop (12:23)

It’s almost the demo time, but I need to talk about the Kubernetes server into Docker Desktop first. Docker Desktop includes a standard Kubernetes server. The Kubernetes server runs locally within your Docker instance and it’s not configurable. And it is a single node cluster. It runs within a Docker container on your local system and it is only for local testing and for development. First you need to activate in Docker Desktop the Kubernetes server. Then you need to install some tools like Kubectl to send command to the Kubernetes cluster.

To connect to the Docker Kubernetes cluster, first you have to check the current context. Using the kubectl command config get-contexts. On this slide, you can see that I’m connected to the Wookiee context, so I need to change the context by using kubectl config use-context and the name of the context. In this case, it’s the docker-desktop context. And now I can send command to the cluster to get some information. For example, with the kubectl command cluster-info.

 

Pod Demo (13:52

Demo time! So let’s play with Redis and Kubernetes. I use a very useful tool named K9S to manage and monitor my Kubernetes cluster. It’s an open source tool. So I will start it like this. I am connected to the Docker desktop context. So now, I want to create a Redis pod with the Redis Docker image. So I will use kubectl for that. On the left, you can see that the Redis pod is created. So Redis Dashboard is the name I give to the pod, but you can name it as you want. I can enter in the pod with the S shortcut. So I can start the Redis CLI, like this. I can set the message like this. And thanks to kubectl, I can enter in the pod in an iterative mode with the exec command and the -IT option. And I will start automatically the Redis CLI. So that, and you can see that if I type GET message, the message was persisted. So it’s very simple to use. I can type other commands like kubectl get pods like this. So this is the pods inside the default namespace. This is the one. In fact, I can generate a Yaml manifest from a running pod. And in this Yaml manifest, you will find all the properties of this running pod. I will use the kubectl get pods command. Like this one. So you can see that the Yaml file is generated with a lot of things. But this is very useful because we can use it even if the pod is dropped to recreate the pod. Let me show you that. I will delete the Redis pod with the kubectl delete command, delete pod command. So the Redis pod is deleted. And if I use the kubectl apply -f command with the path to the manifest, I can recreate the Redis pod. And if I want to delete it, I can delete it from the manifest with the kubectl -f command. So you see it’s pretty simple. Let’s go back to the presentation.

There are several other types of objects that can be defined as well. Like the services which provide the ability to connect pods together and make them discoverable using in-cluster DNS. They can also be used to expose pods directly on a node or through external load balancers. The ingress that provides the ability to define routing rules, the config maps and secrets that provide the ability to configure applications in a variety of ways either using environment variables or files.

 

Best dev practices for prod deploys (18:38)

With a basic understanding of orchestration, let’s talk about a few of the best practices a developer can follow to help production environments be as successful as possible. All orchestration frameworks have the ability to define healthchecks. Kubernetes calls them probes. You will use this healthcheck to validate if a workload is healthy. In general, if the healthchecks are failing, the workload will be automatically restarted. Other events are also generated which can be used for overall monitoring. As a developer, this means you need to think about how do I know if my application is healthy. In this example, a request will be sent to the healthz on port 8080. This request needs to be quick and should have minimal impact on resources like databases, caches, etc. And it should also not cause any side effects.

This next best practice is around the resources your containerized workloads need to work. In most systems, there is a difference between resource limits and requests. The request is how much the application requires to run. This is used to help schedule the workload. What’s not available memory and CPU resources? Limits are the maximum amount of resources the workload can leverage. Note that for CPU, the workload is throttled when trying to use more than this example. If the nginx server is trying to use more than one CPU, it’s limited. For memory, it’s a little more complicated. If the memory request goes over the limit, the kernel will kill the process with an out of memory error. If this happens, an event is captured and reported back to Kubernetes, which is accessible in the pod event logs. As a developer, it is important to note this. Often, it’s best to set high levels, so struggling out of memory errors, etc. won’t impact your users and scale them down as you gain a better understanding of how your application works.

Promote, don’t rebuild. The final practice mentioned here, focus on reusing the same container image for each of your environment. Build once, run everywhere. If you don’t know the 12 factors, I engage you to go to the website 12factor.net. It’s a guide on application scale and portability, written by the founders of Heroku. The guide explains what to do to create cloud-native applications. One of the most important factors is the configuration factor. It’s about extracting all the elements that change from one environment to another environment, such as the database connection, the API’s keys, and so on. Containerized workloads provide two main possibilities here. Environment variables and file usage. With orchestration, you can define and easily share config and add it injected into the container at startup. As a developer, you need to think about the various ways, your application might need to be configured. But then also make sure your support, either file or environment variable based configuration.

 

Kubernetes: Deploying Your First Application (22:25)

So, demo time again. I will try to deploy a web application on Kubernetes. So, the workflow to deploy a microservice or web application to Kubernetes is pretty simple. First, you will Dockerize your application, then you will build the image and push it to Docker Hub or any other registry. After that, you will define a manifest. This is a Yaml file to describe what to do to deploy your image into the Kubernetes cluster. By using the command kubectl you can apply -f, followed by the path name of the manifest, like with the Redis pod demonstration. But let’s see how to do this.

So, I again will use k9s to manage and monitor my Kubernetes cluster. To be able to reach my application in my browser, I will need to define an ingress. I will make a quick shortcut. We need a URL to access the web app with the browser. And the Traefik component on the traffic software will help in this. Traefik is a modern open source, reverse proxy and load balancer designed to manage and route traffic to your microservices or web application. It simplifies the deployment of application by automatically detecting the services and routing traffic to them. And then with that, we are ready to play with Kubernetes.

 

Deployment Demo (24:13)

So, again, I will switch the view. Again, everything is explained in the repository, so it will be easy for you to reproduce the demonstration. I already installed traefik. You can see it here. So, this time, I will create a demo namespace. To keep a specific space for my pods. I will use the kubectl create namespace command with the demo name. Like this, –dry-run option is to explain to Kubernetes that if the namespace already exists, I won’t have an error message. I can do it again. You see, everything is okay. Now, I will deploy a small web application. It’s a golang application. It’s a simple HTTP server that displays a message, the content of the message, the name and the message. The content of the message comes from the message on the environment variable. And the name is on the name. I use string arrays to generate different names. So, if I have three pods of the same application, and start the application, I will see three different names, in fact. I, of course, use a Dockerfile. I already built it and published it on the Docker Hub. With this kind of source code, and now to deploy it on Kubernetes, I will use a Yaml manifest, like this one. So, you have the service to define the service name, the export port. You have the deployment part to explain that I want. three replicas. So, I will deploy the same application and I will get three pods. For load balancing, For example, it could be very interesting. I explained that I will use this image. I can set the environment variable with a specific message. This is the listening port. And then I will define an ingress to get a URL for my new application. So, let’s do it. I will use kubectl apply -f and the name of the yaml file -n, and the name of the namespace. So, you can see I have three pods and they are all running. You can see the log using the L shortcut. And now I will try to access to my application. Let’s check if the ingress is okay, with the kubectl described ingress command, with the name of the service. So, this is the external URL to access to the service. So, let’s see if you have something. So, I copy the URL. I will switch to a browser. So, you can see this very fancy application. I said that I have three pods for the same application with three different names. So, the name is this one. So, if I refresh the page, you can see that I have three different names. But, let’s go back to the demonstration.

If I want to deploy a different instance of my application, I mean with a different message, for example, I am changing the environment variable value here. So, it will be another service. This time, I think I have only one replica. So, I can start it with the apply command. But, with another Yaml manifest, like this. So, you can see that we have a new service, the demo-tiny-two service. And I can do the same to get a third service. This is the same application. But, this time again, I changed the environment variable message. So, let’s wait – easy, deployed. Yes, we can check, of course, the ingress for each service. So, this time is demo-tiny-two. Let’s check with the third one. It’s demo-tiny-three. And now, we can just check if the service is okay. And we switch to the browser. So, I have the tiny-two service. And I will just check the third one. And this is the other message. So, you see that it’s pretty easy to deploy and scale an application with Kubernetes. If you want to un-deploy or to delete a service, you will use the kubectl command with the delete command with the -f option and the manifest, like this. And you see, I dropped the three pods at the same time. And I can do this for every service, of course. And again, I can remove everything with the delete namespace demo command. Let’s return to the presentation.

 

Recap (32:23)

So, as we wrap up, we have just a few things to take away. Having an understanding of how orchestration works even at the high will help you write better applications. You can create better health checks, better utilize your resources and make your applications easier to configure. And so on. At the end of the day, the container’s standards are what make all of this possible. By having a standardized “box” and container runtime, the orchestration systems can serve as an air traffic controller over your fleet of machines. And finally, orchestration makes it possible for you to simply declare your desired state and the system will do it best to make it a reality. So, thank you for your attention.

 

Learn more

Discover the basics and value of systems such as Kubernetes, Swarm, ECS, and Nomad in running containerized workloads in production.

Our speakers

Philippe Charrière

Senior Solutions Architect
Docker