Highlights
- Cloudflare pushes an average of 1,500 distinct Docker images daily
- 6,000+ multi-architecture images managed with Docker’s manifest lists and ARM support
- Optimized containerized infrastructure while serving 320 cities in 120+ countries
Introduction
Cloudflare, the leading connectivity cloud company, delivers fast and secure internet experiences to users worldwide. With significant global internet traffic flowing through its network, Cloudflare continually seeks innovative solutions to enhance its infrastructure’s efficiency and reliability.
Cloudflare has rapidly grown into a $23 billion tech giant traded on the NYSE, building one of the world’s largest and most powerful cloud networks. Its global footprint spans more than 320 cities across more than 120 countries. At the core are data centers in 320 cities interconnected with more than 13,000 networks globally, and Docker is the foundation for applications deployed across them.
Underpinning Cloudflare’s services like DDoS protection and content delivery is staggering network capacity exceeding 100Tbps of throughput. About 70% of that flows over private interconnects with more than 8,800 networks, while the company’s Anycast network taps 600+ Internet exchanges to absorb volumetric attacks over 37 Tbps. This combination of a massively distributed presence and bandwidth scalability allows Cloudflare to provide low-latency performance and security for its customers worldwide.
At the heart of Cloudflare’s success is its dedicated team of engineers. Sanchita Agarwal is a Senior Software Engineer in the developer productivity team, also known as the DevTools team at Cloudflare. Agarwal and her team are crucial in enabling their productivity through self-service development tools and environments. Their mission is to empower engineers across the company through the entire software development lifecycle (SDLC), ensuring that all critical services, self-hosted CI/CD platforms, and infrastructure are running smoothly.
The DevTools team is the backbone of Cloudflare’s engineering ecosystem, providing the tools, frameworks, and support needed to optimize developer productivity. They work closely with various engineering teams to understand their unique requirements and deliver tailored solutions that enhance efficiency and streamline workflows. By abstracting away the complexities of infrastructure management and providing intuitive tools, the DevTools team enables engineers to focus on what they do best — writing code and delivering value to Cloudflare’s customers.
Cloudflare’s relationship with Docker dates back to the company’s early days, joining Docker Hub in June of 2014. Early on, they used Docker before many of the new Docker products and services existed. As early adopters of containerization technology, Cloudflare recognized Docker’s potential to revolutionize how applications are developed, deployed, and managed. By embracing Docker as a Docker Business customer, Cloudflare embarked on a transformative journey that would reshape its entire engineering ecosystem.
Problem
Providing self-service to more than 1,000 developers across different architectures
As Cloudflare expanded its global presence and customer base, it faced the challenge of efficiently managing and deploying its vast network infrastructure. With more than 1,000 engineers working on critical services and a rapidly increasing number of nodes at the edge, Cloudflare needed a solution to streamline its development and deployment processes. The company sought to reduce redundancy, optimize resource utilization, and ensure consistency across its diverse ecosystem.
Cloudflare’s engineering teams grappled with the complexities of managing multiple environments and ensuring compatibility across different systems. As Cloudflare was growing and expanding, its tech stack needed more support to handle tagging multi-architecture images. They were looking for a single solution to manage multi-architecture images, which they were adding to their infrastructure, specifically ARM.
Cloudflare realized that to stay agile and efficient, they required new tooling or to double down on existing tools that could grow with their expanding needs. They believed Docker could help them maintain their competitive edge and deliver exceptional customer performance.
Solution
Harnessing the power of Docker for seamless application delivery
Cloudflare turned to Docker to scale its application development and deployment approach. In 2017, Docker introduced manifest lists, which Cloudflare quickly leveraged in 2018 to manage multi-architecture images. This capability enabled the generation of manifests for each architecture, tagging images with the same name and version, and allowing the Docker daemon to pull them based on architecture.
Docker’s portable containers enabled engineers to package applications and their dependencies into consistent, standardized units across all environments. This approach promotes uniformity across different environments and significantly streamlines development workflows. By adopting Docker’s containerization technology, Cloudflare unlocked a new level of efficiency and agility.
Key benefits
Support for multi-architecture images
Docker's support for multi-architecture images using manifest lists has been a game-changer for Cloudflare. Before the introduction of manifest lists, Cloudflare had to manage multiple images with different tags for each architecture, even if the only difference was the base image. This approach led to a proliferation of images, with different versions and tags for each architecture, resulting in increased complexity and redundancy in image management.
Harnessing Docker’s continuous innovation
Cloudflare was particularly impressed by Docker’s commitment to continuous innovation, which was evident in their ongoing enhancements to tools like manifest lists and BuildX. Introduced in 2017, manifest lists significantly simplified Cloudflare’s image management across different architectures. Also, Cloudflare leveraged another improvement brought to market by Docker, IPv6 networking capabilities, to establish new networks within their CI agents. This capability was crucial for deploying and testing applications on IPv6 networks, enhancing compatibility, and extending Cloudflare's service reach to IPv6-dependent environments.
Streamlined development and deployment processes
Docker's containerization technology enables Cloudflare's engineers to package applications and their dependencies into standardized units, ensuring consistency across different environments. This approach streamlines the development and deployment processes, reducing the time and effort required to set up and manage complex application stacks.
Enhanced scalability and flexibility
Docker's portability and flexibility enable Cloudflare to easily scale its applications and services to meet its customers' growing demands. Containerization allows for the rapid deployment and scaling of applications across different environments and architectures, including AMD and ARM64, providing Cloudflare with the agility needed to respond to changing business requirements.
Seamless integration with Kubernetes for orchestration
Docker integrates with Kubernetes, the leading container orchestration platform. Cloudflare leverages this integration to automate the deployment, scaling, and management of its containerized applications. Kubernetes provides a robust and scalable framework for orchestrating Docker containers, enabling Cloudflare to manage its infrastructure at scale efficiently.
Docker usage across Cloudflare
Cloudflare heavily leverages Docker Compose to define and run multi-container applications. The company has built wrapper tools around Docker Compose that allow engineers to reuse everyday tasks and configurations easily. This approach significantly reduces the time and effort required to set up and manage complex application stacks. By abstracting away the intricacies of infrastructure management, Docker empowers Cloudflare's engineers to focus on writing code and delivering value to customers.
The DevTools team is vital in creating a self-service development platform that empowers engineers to be more productive and autonomous. They have built innovative applications that serve as wrappers around Docker containers, abstracting away the complexities of infrastructure management. These applications provide a user-friendly interface for engineers to spin up development environments, run tests, and quickly deploy their code.
The impact of this self-service platform on developer happiness cannot be overstated. Engineers don’t have to wait for infrastructure provisioning or spend time troubleshooting environment issues. They can quickly iterate on their code, test their changes, and confidently deploy to production. This level of autonomy and efficiency has led to developer satisfaction and productivity, leading to faster innovation and better outcomes for Cloudflare and its customers.
Cloudflare and open source transparencyOne of Docker's critical advantages for Cloudflare is its vibrant open source community and transparent development process. The Docker community is known for its active engagement, with developers worldwide contributing to the project and sharing their expertise.
Cloudflare actively participates in the Docker community, contributing base images on Docker Hub and engaging with other users to share knowledge and best practices. The company has been a vocal advocate for new features and improvements, such as introducing multi-architecture support through manifest lists. By leveraging the collective intelligence of the Docker community, Cloudflare has been able to stay at the forefront of containerization technology and deliver cutting-edge solutions to its customers.
The transparency of the Docker project has been a significant benefit for Cloudflare. With an open development process and a public issue tracker, Cloudflare can actively engage with the Docker team and provide feedback on critical features and bug fixes. This level of transparency fosters a sense of trust and collaboration between Cloudflare and the Docker community, enabling the company to influence the project's direction and ensure that its needs are met.
Outcomes
Unparalleled velocity and efficiency gains
By embracing Docker, Cloudflare has achieved remarkable outcomes in terms of velocity and efficiency. The company now pushes an average of 1,500 distinct images daily, translating to more than 6,000 multi-architecture images when considering short and long tags. This level of productivity would have been unthinkable without Docker.
As Agarwal explains, “Without manifest lists, we would have single images of different versions, and each image would basically act as a single image with a different architecture within its tag. The engineer would be responsible for pulling those images.”
Docker has empowered Cloudflare’s engineers to focus on innovation and delivering value to customers. The ability to quickly spin up containers, test applications locally, and deploy them seamlessly has dramatically reduced development cycles. Engineers can now iterate faster, experiment with new ideas, and swiftly respond to customer needs. This increased velocity has allowed Cloudflare to stay ahead of the competition and continuously deliver new features and improvements to its platform.
Looking ahead, Cloudflare continues to explore new ways to leverage Docker’s capabilities. The company is actively investigating Docker-in-Docker solutions to optimize its CI/CD pipelines further. By running Docker inside Docker containers, Cloudflare aims to streamline its build and deployment processes, enabling even faster iterations and more efficient resource utilization. This ongoing exploration demonstrates Cloudflare’s commitment to pushing the boundaries of what’s possible with containerization technology.
Cloudflare’s journey with Docker is a testament to the transformative power of commercial open source and community-driven innovation. By embracing Docker and actively engaging with its vibrant ecosystem, Cloudflare has unlocked unprecedented velocity, efficiency, and scalability levels. The company’s success story inspires other organizations seeking to modernize their infrastructure and stay ahead in an ever-evolving technological landscape.
With Docker as their partner for improving developer experience, Cloudflare is well-positioned to tackle future challenges and remain a leader in the CDN space. The company’s commitment to transparency, collaboration, and continuous improvement aligns perfectly with the values of the Docker community. As Cloudflare continues to innovate and push the boundaries of what’s possible, its partnership with Docker will undoubtedly play a crucial role in shaping the future of the internet.