One of the most productive meetings I had KubeCon in San Diego last November was a meeting with Docker, Amazon and Microsoft to plan a collaboration around a new version of the CNCF project Notary. We held the Notary v2 kickoff meeting a few weeks later in Seattle in the Amazon offices.
Emphasising that this is a cross-industry collaboration, we had eighteen people in the room (with more dialed in) from Amazon, Microsoft, Docker, IBM, Google, Red Hat, Sylabs and JFrog. This represented all the container registry providers and developers, other than the VMware Harbor developers who could unfortunately not make it in person. Unfortunately, we forgot to take a picture of everyone!
The consensus and community are important because of the aims of Notary v2. But let’s go back a bit as some of you may not know what Notary is and what it is for.
The Notary project was originally started at Docker back in 2015 to provide a general signing infrastructure for containers based on The Update Framework (TUF), a model for package management security developed by Justin Cappos and his team at New York University. This is what supports the “docker trust” set of commands that allow signing containers, and the DOCKER_CONTENT_TRUST settings for validating signatures.
In 2017, Notary was donated to the CNCF, along with the TUF specification, to make it a cross-industry standard. It began to be shipped in other places as well as Docker Hub, including the Docker Trusted Registry (now a Mirantis product), IBM’s container registry, the Azure Container Registry, and with the Harbor project, another CNCF project. TUF also expanded its use cases, in the package management community, and in projects such as Uptane, a framework for updating firmware on automobiles.
So why a version 2 now? Part of the answer is that we learnt a lot of things about the usage of containers since 2015. Are container years like dog years? I am not sure, but a lot has happened since then, and the usage of containers has expanded enormously. I covered a lot of the reasons in-depth in my KubeCon talk:
Supply chain security – making sure that you ship what you intended to ship into production – has become increasingly important, as attacks on software supply chains have increased in recent years. Signatures are an important part of the validation needed in container supply chains.
Integrating Signatures in the Registry
The first big change that we want to make is because at present not every registry supports Notary. This means that if you use a mixture of registries, some may support signatures while others do not. In addition, you cannot move signatures between registries. Both of these are related to the design of Notary as in effect a registry sidecar. While Notary shares the same authentication as a registry, it is built as a separate service, with its own database and API.
Back when Notary was designed this did not seem so important. But now many people use, or want to use, complex registry configurations with local registries close to a production cluster, or at the cloud provider code is running on, or in an edge location which may be disconnected. The solution that we are working on is that rather than being a standalone sidecar service, signatures will be integrated into the OCI image specification and supported by all registries. The details of this are still being worked out, but this will make portability much easier, as signatures will be able to be pushed and pulled with images.
Improving Usability
The second big set of changes is around usability. The current way of signing containers and checking signatures is complex, as is the key management. One of the aims of Notary v2 is to have signatures and checking on by default where possible. There have been many issues stopping this with the current Notary, many of which are detailed in the KubeCon talk, including the large number of keys involved due to lack of hierarchy and delegation, and lack of standard interfaces for signature checking on different platforms such as Kubernetes.
If you want to learn more, there are weekly meetings on Mondays at 10 a.m. Pacific Time – see the CNCF community calendar for details. The Slack channel is #notary-v2 in the CNCF Slack. There will be two sessions at KubeCon Amsterdam, one introductory overview and state of where we are, and another deep dive working session on current issues. Hope to see you there!