# CRI: the Container Runtime Interface ## What is CRI? CRI (_Container Runtime Interface_) consists of a [protobuf API](../../pkg/kubelet/api/v1alpha1/runtime/api.proto), specifications/requirements (to-be-added), and [libraries] (https://github.com/kubernetes/kubernetes/tree/master/pkg/kubelet/server/streaming) for container runtimes to integrate with kubelet on a node. CRI is currently in Alpha. In the future, we plan to add more developer tools such as the CRI validation tests. ## Why develop CRI? Prior to the existence of CRI, container runtimes (e.g., `docker`, `rkt`) were integrated with kubelet through implementing an internal, high-level interface in kubelet. The entrance barrier for runtimes was high because the integration required understanding the internals of kubelet and contributing to the main Kubernetes repository. More importantly, this would not scale because every new addition incurs a significant maintenance overhead in the main kubernetes repository. Kubernetes aims to be extensible. CRI is one small, yet important step to enable pluggable container runtimes and build a healthier ecosystem. ## How to use CRI? 1. Start the image and runtime services on your node. You can have a single service acting as both image and runtime services. 2. Set the kubelet flags - Pass the unix socket(s) to which your services listen to kubelet: `--container-runtime-endpoint` and `--image-service-endpoint`. - Enable CRI in kubelet by`--experimental-cri=true`. - Use the "remote" runtime by `--container-runtime=remote`. Please see the [Status Update](#status-update) section for known issues for each release. Note that CRI is still in its early stages. We are actively incorporating feedback from early developers to improve the API. Developers should expect occasional API breaking changes. ## Does Kubelet use CRI today? No, but we are working on it. The first step is to switch kubelet to integrate with Docker via CRI by default. The current [Docker CRI implementation](https://github.com/kubernetes/kubernetes/blob/release-1.5/pkg/kubelet/dockershim) already passes most end-to-end tests, and has mandatory PR builders to prevent regressions. While we are expanding the test coverage gradually, it is difficult to test on all combinations of OS distributions, platforms, and plugins. There are also many experimental or even undocumented features relied upon by some users. We would like to **encourage the community to help test this Docker-CRI integration and report bugs and/or missing features** to smooth the transition in the near future. Please file a Github issue and include @kubernetes/sig-node for any CRI problem. ### How to test the new Docker CRI integration? Start kubelet with the following flags: - Use the Docker container runtime by `--container-runtime=docker`(the default). - Enable CRI in kubelet by`--experimental-cri=true`. Please also see the [known issues](#docker-cri-1.5-known-issues) before trying out. ## Design docs and proposals We plan to add CRI specifications/requirements in the near future. For now, these proposals and design docs are the best sources to understand CRI besides discussions on Github issues. - [Original proposal](https://github.com/kubernetes/kubernetes/blob/release-1.5/docs/proposals/container-runtime-interface-v1.md) - [Exec/attach/port-forward streaming requests](https://docs.google.com/document/d/1OE_QoInPlVCK9rMAx9aybRmgFiVjHpJCHI9LrfdNM_s/edit?usp=sharing) - [Container stdout/stderr logs](https://github.com/kubernetes/kubernetes/blob/release-1.5/docs/proposals/kubelet-cri-logging.md) - Networking: The CRI runtime handles network plugins and the setup/teardown of the pod sandbox. ## Work-In-Progress CRI runtimes - [cri-o](https://github.com/kubernetes-incubator/cri-o) - [rktlet](https://github.com/kubernetes-incubator/rktlet) - [frakti](https://github.com/kubernetes/frakti) ## [Status update](#status-update) ### Kubernetes v1.5 release (CRI v1alpha1) - [v1alpha1 version](https://github.com/kubernetes/kubernetes/blob/release-1.5/pkg/kubelet/api/v1alpha1/runtime/api.proto) of CRI is released. #### [CRI known issues](#cri-1.5-known-issues): - [#27097](https://github.com/kubernetes/kubernetes/issues/27097): Container metrics are not yet defined in CRI. - [#36401](https://github.com/kubernetes/kubernetes/issues/36401): The new container log path/format is not yet supported by the logging pipeline (e.g., fluentd, GCL). - CRI may not be compatible with other experimental features (e.g., Seccomp). - Streaming server needs to be hardened. - [#36666](https://github.com/kubernetes/kubernetes/issues/36666): Authentication. - [#36187](https://github.com/kubernetes/kubernetes/issues/36187): Avoid including user data in the redirect URL. #### [Docker CRI integration known issues](#docker-cri-1.5-known-issues) - Docker compatibility: Support only Docker v1.11 and v1.12. - Network: - [#35457](https://github.com/kubernetes/kubernetes/issues/35457): Does not support host ports. - [#37315](https://github.com/kubernetes/kubernetes/issues/37315): Does not support bandwidth shaping. - Exec/attach/port-forward (streaming requests): - [#35747](https://github.com/kubernetes/kubernetes/issues/35747): Does not support `nsenter` as the exec handler (`--exec-handler=nsenter`). - Also see (#cri-1.5-known-issues) for limitations on CRI streaming. ## Contacts - Email: sig-node (kubernetes-sig-node@googlegroups.com) - Slack: https://kubernetes.slack.com/messages/sig-node [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/container-runtime-interface.md?pixel)]()