github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Proposal for client-server container runtime

pull/6/head
Pengfei Ni 2016-06-10 14:34:19 +08:00
parent 61a9358dbd
commit e12044518e
7 changed files with 3902 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

235
docs/proposals/runtime-client-server.md Normal file
View File

@ -0,0 +1,235 @@
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- BEGIN STRIP_FOR_RELEASE -->
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
width="25" height="25">
<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>
If you are using a released version of Kubernetes, you should
refer to the docs that go with that version.
Documentation for other releases can be found at
[releases.k8s.io](http://releases.k8s.io).
</strong>
--
<!-- END STRIP_FOR_RELEASE -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Client/Server container runtime
## Abstract
A proposal of client/server implementation of kubelet container runtime interface.
## Motivation
Currently, any container runtime has to be linked into the kubelet. This makes
experimentation difficult, and prevents users from landing an alternate
container runtime without landing code in core kubernetes.
To facilitate experimentation and to enable user choice, this proposal adds a
client/server implementation of the [new container runtime interface](https://github.com/kubernetes/kubernetes/pull/25899). The main goal
of this proposal is:
- make it easy to integrate new container runtimes
- improve code maintainability
## Proposed design
**Design of client/server container runtime**
The main idea of client/server container runtime is to keep main control logic in kubelet while letting remote runtime only do dedicated actions. An alpha [container runtime API](../../pkg/kubelet/api/v1alpha1/runtime/api.proto) is introduced for integrating new container runtimes. The API is based on [protobuf](https://developers.google.com/protocol-buffers/) and [gRPC](http://www.grpc.io) for a number of benefits:
- Perform faster than json
- Get client bindings for free: gRPC supports ten languages
- No encoding/decoding codes needed
- Manage api interfaces easily: server and client interfaces are generated automatically
A new container runtime manager `KubeletGenericRuntimeManager` will be introduced to kubelet, which will
- conforms to kubelet's [Runtime](../../pkg/kubelet/container/runtime.go#L58) interface
- manage Pods and Containers lifecycle according to kubelet policies
- call remote runtime's API to perform specific pod, container or image operations
A simple workflow of invoking remote runtime API on starting a Pod with two containers can be shown:
```
Kubelet KubeletGenericRuntimeManager RemoteRuntime
+ + +
| | |
+---------SyncPod------------->+ |
| | |
| +---- Create PodSandbox ------->+
| +<------------------------------+
| | |
| XXXXXXXXXXXX |
| | X |
| | NetworkPlugin. |
| | SetupPod |
| | X |
| XXXXXXXXXXXX |
| | |
| +<------------------------------+
| +---- Pull image1 -------->+
| +<------------------------------+
| +---- Create container1 ------->+
| +<------------------------------+
| +---- Start container1 -------->+
| +<------------------------------+
| | |
| +<------------------------------+
| +---- Pull image2 -------->+
| +<------------------------------+
| +---- Create container2 ------->+
| +<------------------------------+
| +---- Start container2 -------->+
| +<------------------------------+
| | |
| <-------Success--------------+ |
| | |
+ + +
```
And deleting a pod can be shown:
```
Kubelet KubeletGenericRuntimeManager RemoteRuntime
+ + +
| | |
+---------SyncPod------------->+ |
| | |
| +---- Stop container1 ----->+
| +<------------------------------+
| +---- Delete container1 ----->+
| +<------------------------------+
| | |
| +---- Stop container2 ------>+
| +<------------------------------+
| +---- Delete container2 ------>+
| +<------------------------------+
| | |
| XXXXXXXXXXXX |
| | X |
| | NetworkPlugin. |
| | TeardownPod |
| | X |
| XXXXXXXXXXXX |
| | |
| | |
| +---- Delete PodSandbox ------>+
| +<------------------------------+
| | |
| <-------Success--------------+ |
| | |
+ + +
```
**API definition**
Since we are going to introduce more image formats and want to separate image management from containers and pods, this proposal introduces two services `RuntimeService` and `ImageService`. Both services are defined at [pkg/kubelet/api/v1alpha1/runtime/api.proto](../../pkg/kubelet/api/v1alpha1/runtime/api.proto):
```proto
// Runtime service defines the public APIs for remote container runtimes
service RuntimeService {
// Version returns the runtime name, runtime version and runtime API version
rpc Version(VersionRequest) returns (VersionResponse) {}
// CreatePodSandbox creates a pod-level sandbox.
// The definition of PodSandbox is at https://github.com/kubernetes/kubernetes/pull/25899
rpc CreatePodSandbox(CreatePodSandboxRequest) returns (CreatePodSandboxResponse) {}
// StopPodSandbox stops the sandbox. If there are any running containers in the
// sandbox, they should be force terminated.
rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {}
// DeletePodSandbox deletes the sandbox. If there are any running containers in the
// sandbox, they should be force deleted.
rpc DeletePodSandbox(DeletePodSandboxRequest) returns (DeletePodSandboxResponse) {}
// PodSandboxStatus returns the Status of the PodSandbox.
rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {}
// ListPodSandbox returns a list of SandBox.
rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {}
// CreateContainer creates a new container in specified PodSandbox
rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {}
// StartContainer starts the container.
rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {}
// StopContainer stops a running container with a grace period (i.e., timeout).
rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {}
// RemoveContainer removes the container. If the container is running, the container
// should be force removed.
rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {}
// ListContainers lists all containers by filters.
rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {}
// ContainerStatus returns status of the container.
rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {}
// Exec executes the command in the container.
rpc Exec(stream ExecRequest) returns (stream ExecResponse) {}
}
// Image service defines the public APIs for managing images
service ImageService {
// ListImages lists existing images.
rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {}
// ImageStatus returns the status of the image.
rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {}
// PullImage pulls a image with authentication config.
rpc PullImage(PullImageRequest) returns (PullImageResponse) {}
// RemoveImage removes the image.
rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {}
}
```
Note that some types in [pkg/kubelet/api/v1alpha1/runtime/api.proto](../../pkg/kubelet/api/v1alpha1/runtime/api.proto) are already defined at [Container runtime interface/integration](https://github.com/kubernetes/kubernetes/pull/25899).
We should decide how to integrate the types in [#25899](https://github.com/kubernetes/kubernetes/pull/25899) with gRPC services:
* Auto-generate those types into protobuf by [go2idl](../../cmd/libs/go2idl/)
- Pros:
- trace type changes automatically, all type changes in Go will be automatically generated into proto files
- Cons:
- type change may break existing API implementations, e.g. new fields added automatically may not noticed by remote runtime
- needs to convert Go types to gRPC generated types, and vise versa
- needs processing attributes order carefully so as not to break generated protobufs (this could be done by using [protobuf tag](https://developers.google.com/protocol-buffers/docs/gotutorial))
- go2idl doesn't support gRPC, [protoc-gen-gogo](https://github.com/gogo/protobuf) is still required for generating gRPC client
* Embed those types as raw protobuf definitions and generate Go files by [protoc-gen-gogo](https://github.com/gogo/protobuf)
- Pros:
- decouple type definitions, all type changes in Go will be added to proto manually, so it's easier to track gRPC API version changes
- Kubelet could reuse Go types generated by `protoc-gen-gogo` to avoid type conversions
- Cons:
- duplicate definition of same types
- hard to track type changes automatically
- need to manage proto files manually
For better version controlling and fast iterations, this proposal embeds all those types in `api.proto` directly.
## Implementation
Each new runtime should implement the [gRPC](http://www.grpc.io) server based on [pkg/kubelet/api/v1alpha1/runtime/api.proto](../../pkg/kubelet/api/v1alpha1/runtime/api.proto). For version controlling, `KubeletGenericRuntimeManager` will request `RemoteRuntime`'s `Version()` interface with the runtime api version. To keep backward compatibility, the API follows standard [protobuf guide](https://developers.google.com/protocol-buffers/docs/proto) to deprecate or add new interfaces.
A new flag `--container-runtime-endpoint` (overrides `--container-runtime`) will be introduced to kubelet which identifies the unix socket file of the remote runtime service. And new flag `--image-service-endpoint` will be introduced to kubelet which identifies the unix socket file of the image service.
To facilitate switching current container runtime (e.g. `docker` or `rkt`) to new runtime API, `KubeletGenericRuntimeManager` will provide a plugin mechanism allowing to specify local implementation or gRPC implementation.
## Community Discussion
This proposal is first filed by [@brendandburns](https://github.com/brendandburns) at [kubernetes/13768](https://github.com/kubernetes/kubernetes/issues/13768):
* [kubernetes/13768](https://github.com/kubernetes/kubernetes/issues/13768)
* [kubernetes/13709](https://github.com/kubernetes/kubernetes/pull/13079)
* [New container runtime interface](https://github.com/kubernetes/kubernetes/pull/25899)
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/proposals/runtime-client-server.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->

50
hack/update-generated-runtime-dockerized.sh Executable file
View File

@ -0,0 +1,50 @@
#!/bin/bash
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -o errexit
set -o nounset
set -o pipefail
KUBE_ROOT=$(dirname "${BASH_SOURCE}")/..
KUBE_REMOTE_RUNTIME_ROOT="${KUBE_ROOT}/pkg/kubelet/api/v1alpha1/runtime"
source "${KUBE_ROOT}/hack/lib/init.sh"
kube::golang::setup_env
hack/build-go.sh cmd/libs/go2idl/go-to-protobuf/protoc-gen-gogo
if [[ -z "$(which protoc)" || "$(protoc --version)" != "libprotoc 3.0."* ]]; then
echo "Generating protobuf requires protoc 3.0.0-beta1 or newer. Please download and"
echo "install the platform appropriate Protobuf package for your OS: "
echo
echo " https://github.com/google/protobuf/releases"
echo
echo "WARNING: Protobuf changes are not being validated"
exit 1
fi
function cleanup {
rm -f ${KUBE_REMOTE_RUNTIME_ROOT}/api.pb.go.bak
}
trap cleanup EXIT
gogopath=$(dirname $(kube::util::find-binary "protoc-gen-gogo"))
export PATH=$gogopath:$PATH
protoc -I${KUBE_REMOTE_RUNTIME_ROOT} --gogo_out=plugins=grpc:${KUBE_REMOTE_RUNTIME_ROOT} ${KUBE_REMOTE_RUNTIME_ROOT}/api.proto
echo "$(cat hack/boilerplate/boilerplate.go.txt ${KUBE_REMOTE_RUNTIME_ROOT}/api.pb.go)" > ${KUBE_REMOTE_RUNTIME_ROOT}/api.pb.go
sed -i".bak" "s/Copyright YEAR/Copyright $(date '+%Y')/g" ${KUBE_REMOTE_RUNTIME_ROOT}/api.pb.go

59
hack/update-generated-runtime.sh Executable file
View File

@ -0,0 +1,59 @@
#!/bin/bash
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -o errexit
set -o nounset
set -o pipefail
KUBE_ROOT=$(dirname "${BASH_SOURCE}")/..
source "${KUBE_ROOT}/build/common.sh"
kube::golang::setup_env
function prereqs() {
kube::log::status "Verifying Prerequisites...."
kube::build::ensure_docker_in_path || return 1
if kube::build::is_osx; then
kube::build::docker_available_on_osx || return 1
fi
kube::build::ensure_docker_daemon_connectivity || return 1
KUBE_ROOT_HASH=$(kube::build::short_hash "${HOSTNAME:-}:${REPO_DIR:-${KUBE_ROOT}}/go-to-protobuf")
KUBE_BUILD_IMAGE_TAG="build-${KUBE_ROOT_HASH}"
KUBE_BUILD_IMAGE="${KUBE_BUILD_IMAGE_REPO}:${KUBE_BUILD_IMAGE_TAG}"
KUBE_BUILD_CONTAINER_NAME="kube-build-${KUBE_ROOT_HASH}"
KUBE_BUILD_DATA_CONTAINER_NAME="kube-build-data-${KUBE_ROOT_HASH}"
DOCKER_MOUNT_ARGS=(
--volume "${REPO_DIR:-${KUBE_ROOT}}/cluster:/go/src/${KUBE_GO_PACKAGE}/cluster"
--volume "${REPO_DIR:-${KUBE_ROOT}}/cmd:/go/src/${KUBE_GO_PACKAGE}/cmd"
--volume "${REPO_DIR:-${KUBE_ROOT}}/vendor:/go/src/${KUBE_GO_PACKAGE}/vendor"
--volume "${REPO_DIR:-${KUBE_ROOT}}/hack:/go/src/${KUBE_GO_PACKAGE}/hack"
--volume "${REPO_DIR:-${KUBE_ROOT}}/pkg:/go/src/${KUBE_GO_PACKAGE}/pkg"
--volume "${REPO_DIR:-${KUBE_ROOT}}/federation:/go/src/${KUBE_GO_PACKAGE}/federation"
--volume "${REPO_DIR:-${KUBE_ROOT}}/third_party:/go/src/${KUBE_GO_PACKAGE}/third_party"
--volume /etc/localtime:/etc/localtime:ro
--volumes-from "${KUBE_BUILD_DATA_CONTAINER_NAME}"
)
LOCAL_OUTPUT_BUILD_CONTEXT="${LOCAL_OUTPUT_IMAGE_STAGING}/${KUBE_BUILD_IMAGE}"
}
prereqs
mkdir -p "${LOCAL_OUTPUT_BUILD_CONTEXT}"
cp "${KUBE_ROOT}/cmd/libs/go2idl/go-to-protobuf/build-image/Dockerfile" "${LOCAL_OUTPUT_BUILD_CONTEXT}/Dockerfile"
kube::build::update_dockerfile
kube::build::docker_build "${KUBE_BUILD_IMAGE}" "${LOCAL_OUTPUT_BUILD_CONTEXT}" 'false'
kube::build::run_build_command hack/update-generated-runtime-dockerized.sh "$@"

4
hack/verify-flags/exceptions.txt
View File

@ -79,6 +79,10 @@ hack/test-update-storage-objects.sh: local storage_media_type=${2:-""}
hack/test-update-storage-objects.sh: local storage_versions=${1:-""}
hack/test-update-storage-objects.sh: source_file=${test_data[0]}
hack/test-update-storage-objects.sh:# source_file,resource,namespace,name,old_version,new_version
pkg/kubelet/api/v1alpha1/runtime/api.pb.go: ContainerPort *int32 `protobuf:"varint,3,opt,name=container_port" json:"container_port,omitempty"`
pkg/kubelet/api/v1alpha1/runtime/api.pb.go: OomScoreAdj *int64 `protobuf:"varint,5,opt,name=oom_score_adj" json:"oom_score_adj,omitempty"`
pkg/kubelet/api/v1alpha1/runtime/api.proto: optional int32 container_port = 3;
pkg/kubelet/api/v1alpha1/runtime/api.proto: optional int64 oom_score_adj = 5;
pkg/kubelet/network/hairpin/hairpin.go: hairpinModeRelativePath = "hairpin_mode"
pkg/kubelet/qos/policy_test.go: t.Errorf("oom_score_adj should be between %d and %d, but was %d", test.lowOOMScoreAdj, test.highOOMScoreAdj, oomScoreAdj)
pkg/kubelet/qos/policy_test.go: highOOMScoreAdj int // The min oom_score_adj score the container should be assigned.

44
hack/verify-generated-runtime.sh Executable file
View File

@ -0,0 +1,44 @@
#!/bin/bash
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -o errexit
set -o nounset
set -o pipefail
KUBE_ROOT=$(dirname "${BASH_SOURCE}")/..
KUBE_REMOTE_RUNTIME_ROOT="${KUBE_ROOT}/pkg/kubelet/api/v1alpha1/runtime"
source "${KUBE_ROOT}/hack/lib/init.sh"
kube::golang::setup_env
function cleanup {
rm -rf ${KUBE_REMOTE_RUNTIME_ROOT}/_tmp/
}
trap cleanup EXIT
mkdir -p ${KUBE_REMOTE_RUNTIME_ROOT}/_tmp
cp ${KUBE_REMOTE_RUNTIME_ROOT}/api.pb.go ${KUBE_REMOTE_RUNTIME_ROOT}/_tmp/
ret=0
hack/update-generated-runtime.sh
diff -I "gzipped FileDescriptorProto" -I "0x" -Naupr ${KUBE_REMOTE_RUNTIME_ROOT}/_tmp/api.pb.go ${KUBE_REMOTE_RUNTIME_ROOT}/api.pb.go || ret=$?
if [[ $ret -eq 0 ]]; then
echo "Generated container runtime api is up to date."
else
echo "Generated container runtime api is out of date. Please run hack/update-generated-runtime.sh"
exit 1
fi

2867
pkg/kubelet/api/v1alpha1/runtime/api.pb.go Normal file
View File

File diff suppressed because it is too large Load Diff

643
pkg/kubelet/api/v1alpha1/runtime/api.proto Normal file
View File

@ -0,0 +1,643 @@
// api.pb.go could be generate by hack/update-generate-runtime.sh
syntax = 'proto2';
package runtime;
// Runtime service defines the public APIs for remote container runtimes
service RuntimeService {
// Version returns the runtime name, runtime version and runtime API version
rpc Version(VersionRequest) returns (VersionResponse) {}
// CreatePodSandbox creates a pod-level sandbox.
// The definition of PodSandbox is at https://github.com/kubernetes/kubernetes/pull/25899
rpc CreatePodSandbox(CreatePodSandboxRequest) returns (CreatePodSandboxResponse) {}
// StopPodSandbox stops the sandbox. If there are any running containers in the
// sandbox, they should be force terminated.
rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse) {}
// DeletePodSandbox deletes the sandbox. If there are any running containers in the
// sandbox, they should be force deleted.
rpc DeletePodSandbox(DeletePodSandboxRequest) returns (DeletePodSandboxResponse) {}
// PodSandboxStatus returns the Status of the PodSandbox.
rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse) {}
// ListPodSandbox returns a list of SandBox.
rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse) {}
// CreateContainer creates a new container in specified PodSandbox
rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse) {}
// StartContainer starts the container.
rpc StartContainer(StartContainerRequest) returns (StartContainerResponse) {}
// StopContainer stops a running container with a grace period (i.e., timeout).
rpc StopContainer(StopContainerRequest) returns (StopContainerResponse) {}
// RemoveContainer removes the container. If the container is running, the container
// should be force removed.
rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse) {}
// ListContainers lists all containers by filters.
rpc ListContainers(ListContainersRequest) returns (ListContainersResponse) {}
// ContainerStatus returns status of the container.
rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse) {}
// Exec executes the command in the container.
rpc Exec(stream ExecRequest) returns (stream ExecResponse) {}
}
// Image service defines the public APIs for managing images
service ImageService {
// ListImages lists existing images.
rpc ListImages(ListImagesRequest) returns (ListImagesResponse) {}
// ImageStatus returns the status of the image.
rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse) {}
// PullImage pulls a image with authentication config.
rpc PullImage(PullImageRequest) returns (PullImageResponse) {}
// RemoveImage removes the image.
rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse) {}
}
message VersionRequest {
// The version of kubelet runtime API.
optional string version = 1;
}
message VersionResponse {
// The version of the kubelet runtime API.
optional string version = 1;
// The name of the container runtime.
optional string runtime_name = 2;
// The version of the container runtime.
optional string runtime_version = 3;
// The API version of the container runtime.
optional string runtime_api_version = 4;
}
// DNSOption specifies the DNS servers and search domains.
message DNSOption {
// List of DNS servers of the cluster.
repeated string servers = 1;
// List of DNS search domains of the cluster.
repeated string searches = 2;
}
enum Protocol {
TCP = 0;
UDP = 1;
}
// PortMapping specifies the port mapping configurations of sandbox
message PortMapping {
// The name of the port mapping
optional string name = 1;
// The protocol of the port mapping.
optional Protocol protocol = 2;
// The port number within the container.
optional int32 container_port = 3;
// The port number on the host.
optional int32 host_port = 4;
// The host IP.
optional string host_ip = 5;
}
// Mount specifies the volume mount for the sandbox
message Mount {
// The name of the volume mount.
optional string name = 1;
// The path of the mount within the container.
optional string container_path = 2;
// The path of the mount on the host.
optional string host_path = 3;
// If set, the mount is read-only.
optional bool readonly = 4;
// If set, the mount needs SELinux relabeling
optional bool selinux_relabel = 5;
}
// ResourceRequirements contains a set of resources
// Valid resources are:
// - cpu, in cores. (500m = .5 cores)
// - memory, in bytes. (500Gi = 500GiB = 500 * 1024 * 1024 * 1024)
message ResourceRequirements {
// The maximum amount of compute resources allowed.
optional double limits = 1;
// The minimum amount of compute resources required.
// If Request is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value
optional double requests = 2;
}
// PodSandboxResources contains the CPU/memory resource requirements.
message PodSandboxResources {
// CPU resource requirement.
optional ResourceRequirements cpu = 1;
// Memory resource requirement.
optional ResourceRequirements memory = 2;
}
// NamespaceOption provides options for Linux namespaces.
message NamespaceOption {
// If set, use the host's network namespace.
optional bool host_network = 1;
// If set, use the host's pid namesapce.
optional bool host_pid = 2;
// If set, use the host's ipc namespace.
optional bool host_ipc = 3;
}
// LinuxPodSandboxConfig holds platform-specific configuraions for Linux
// host platforms and Linux-based containers.
message LinuxPodSandboxConfig {
// The parent cgroup of the pod sandbox.
// The cgroupfs style syntax will be used, but the container runtime can
// convert it to systemd semantices if needed.
optional string cgroup_parent = 1;
// The configurations for the sandbox's namespaces.
// This will be used only if the PodSandbox uses namespace for isolation.
optional NamespaceOption namespace_options = 2;
}
// PodSandboxConfig holds all the required and optional fields for creating a
// sandbox.
message PodSandboxConfig {
// The name of the sandbox.
optional string name = 1;
// The hostname of the sandbox.
optional string hostname = 2;
// Path to the directory on the host in which container log files are
// stored.
// By default the log of a container going into the LogDirectory will be
// hooked up to STDOUT and STDERR. However, the LogDirectory may contain
// binary log files with structured logging data from the individual
// containers. For example the files might be newline seperated JSON
// structured logs, systemd-journald journal files, gRPC trace files, etc.
// E.g.,
// PodSandboxConfig.LogDirectory = `/var/log/pods/<podUID>/`
// ContainerConfig.LogPath = `containerName_Instance#.log`
//
// WARNING: Log managment and how kubelet should interface with the
// container logs are under active discussion in
// https://issues.k8s.io/24677. There *may* be future change of direction
// for logging as the discussion carries on.
optional string log_directory = 3;
// The DNS options for the sandbox.
optional DNSOption dns_options = 4;
// The port mappings for the sandbox.
repeated PortMapping port_mappings = 5;
// Resources specifies the resource limits for the sandbox (i.e., the
// aggregate cpu/memory resources limits of all containers).
// Note: On a Linux host, kubelet will create a pod-level cgroup and pass
// it as the cgroup parent for the PodSandbox. For some runtimes, this is
// sufficent. For others, e.g., hypervisor-based runtimes, explicit
// resource limits for the sandbox are needed at creation time.
optional PodSandboxResources resources = 6;
// Labels are key value pairs that may be used to scope and select individual resources.
map<string, string> labels = 7;
// Annotations is an unstructured key value map that may be set by external
// tools to store and retrieve arbitrary metadata.
map<string, string> annotations = 8;
// Optional configurations specific to Linux hosts.
optional LinuxPodSandboxConfig linux = 9;
}
message CreatePodSandboxRequest {
// The configuration for creating a PodSandBox.
optional PodSandboxConfig config = 1;
}
message CreatePodSandboxResponse {
// The id of the PodSandBox
optional string pod_sandbox_id = 1;
}
message StopPodSandboxRequest {
// The id of the PodSandBox
optional string pod_sandbox_id = 1;
}
message StopPodSandboxResponse {}
message DeletePodSandboxRequest {
// The id of the PodSandBox
optional string pod_sandbox_id = 1;
}
message DeletePodSandboxResponse {}
message PodSandboxStatusRequest {
// The id of the PodSandBox
optional string pod_sandbox_id = 1;
}
// PodSandboxNetworkStatus is the status of the network for a PodSandbox.
message PodSandboxNetworkStatus {
// The IP address of the PodSandbox
optional string ip = 1;
}
// Namespace contains paths to the namespaces.
message Namespace {
// Network is the path to the network namespace.
optional string network = 1;
// Options is the namespace options for linux namespaces
optional NamespaceOption options = 2;
}
// LinuxSandBoxStatus contains status specific to Linux sandboxes.
message LinuxPodSandboxStatus {
// Namespaces contains paths to the sandbox's namespaces.
optional Namespace namespaces = 1;
}
enum PodSandBoxState {
READY = 0;
NOTREADY = 1;
}
// PodSandboxStatus contains the status of the PodSandbox.
message PodSandboxStatus {
// ID of the sandbox.
optional string id = 1;
// Name of the sandbox
optional string name = 2;
// State of the sandbox.
optional PodSandBoxState state = 3;
// Creation timestamp of the sandbox
optional int64 created_at = 4;
// Network contains network status if network is handled by the runtime.
optional PodSandboxNetworkStatus network = 5;
// Linux specific status to a pod sandbox.
optional LinuxPodSandboxStatus linux = 6;
// Labels are key value pairs that may be used to scope and select individual resources.
map<string, string> labels = 7;
// Annotations is an unstructured key value map that may be set by external
// tools to store and retrieve arbitrary metadata.
map<string, string> annotations = 8;
}
message PodSandboxStatusResponse {
// The status of the PodSandbox
optional PodSandboxStatus status = 1;
}
// PodSandboxFilter is used to filter a list of PodSandboxes.
// All those fields are combined with 'AND'
message PodSandboxFilter {
// Name of the sandbox.
optional string name = 1;
// ID of the sandbox.
optional string id = 2;
// State of the sandbox.
optional PodSandBoxState state = 3;
// LabelSelector to select matches.
// Only api.MatchLabels is supported for now and the requirements
// are ANDed. MatchExpressions is not supported yet.
map<string, string> label_selector = 4;
}
message ListPodSandboxRequest {
// PodSandboxFilter to filter a list of PodSandboxes.
optional PodSandboxFilter filter = 1;
}
// PodSandbox contains minimal information about a sandbox.
message PodSandbox {
// The id of the PodSandbox
optional string id = 1;
// The name of the PodSandbox
optional string name = 2;
// The state of the PodSandbox
optional PodSandBoxState state = 3;
// Creation timestamps of the sandbox
optional int64 created_at = 4;
// The labels of the PodSandbox
map<string, string> labels = 5;
}
message ListPodSandboxResponse {
// List of PodSandbox
repeated PodSandbox items = 1;
}
// ImageSpec is an internal representation of an image. Currently, it wraps the
// value of a Container's Image field (e.g. imageName, imageName:tag, or
// imageName:digest), but in the future it will include more detailed
// information about the different image types.
message ImageSpec {
optional string image = 1;
}
message KeyValue {
optional string key = 1;
optional string value = 2;
}
// LinuxContainerResources specifies Linux specific configuration for
// resources.
// TODO: Consider using Resources from opencontainers/runtime-spec/specs-go
// directly.
message LinuxContainerResources {
// CPU CFS (Completely Fair Scheduler) period
optional int64 cpu_period = 1;
// CPU CFS (Completely Fair Scheduler) quota
optional int64 cpu_quota = 2;
// CPU shares (relative weight vs. other containers)
optional int64 cpu_shares = 3;
// Memory limit in bytes
optional int64 memory_limit_in_bytes = 4;
// OOMScoreAdj adjusts the oom-killer score.
optional int64 oom_score_adj = 5;
}
// SELinuxOption are the labels to be applied to the container.
message SELinuxOption {
optional string user = 1;
optional string role = 2;
optional string type = 3;
optional string level = 4;
}
// Capability contains the container capabilities to add or drop
message Capability {
// List of capabilities to add.
repeated string add_capabilities = 1;
// List of capabilities to drop.
repeated string drop_capabilities = 2;
}
// LinuxContainerConfig contains platform-specific configuration for
// Linux-based containers.
message LinuxContainerConfig {
// Resources specification for the container.
optional LinuxContainerResources resources = 1;
// Capabilities to add or drop.
optional Capability capabilities = 2;
// Optional SELinux context to be applied.
optional SELinuxOption selinux_options = 3;
}
message ContainerConfig {
// Name of the container.
optional string name = 1;
// Image to use.
optional ImageSpec image = 2;
// Command to execute (i.e., entrypoint for docker)
repeated string command = 3;
// Args for the Command (i.e., command for docker)
repeated string args = 4;
// Current working directory of the command.
optional string working_dir = 5;
// List of environment variable to set in the container
repeated KeyValue envs = 6;
// Mounts specifies mounts for the container
repeated Mount mounts = 7;
// Labels are key value pairs that may be used to scope and select individual resources.
// Label keys are of the form:
// label-key ::= prefixed-name | name
// prefixed-name ::= prefix '/' name
// prefix ::= DNS_SUBDOMAIN
// name ::= DNS_LABEL
map<string, string> labels = 8;
// Annotations is an unstructured key value map that may be set by external
// tools to store and retrieve arbitrary metadata.
map<string, string> annotations = 9;
// If set, run container in privileged mode.
// Processes in privileged containers are essentially equivalent to root on the host.
optional bool privileged = 10;
// If set, the root filesystem of the container is read-only.
optional bool readonly_rootfs = 11;
// Path relative to PodSandboxConfig.LogDirectory for container to store
// the log (STDOUT and STDERR) on the host.
// E.g.,
// PodSandboxConfig.LogDirectory = `/var/log/pods/<podUID>/`
// ContainerConfig.LogPath = `containerName_Instance#.log`
//
// WARNING: Log managment and how kubelet should interface with the
// container logs are under active discussion in
// https://issues.k8s.io/24677. There *may* be future change of direction
// for logging as the discussion carries on.
optional string log_path = 12;
// The hash of container config
// Variables for interactive containers, these have very specialized
// use-cases (e.g. debugging).
// TODO: Determine if we need to continue supporting these fields that are
// part of Kubernetes's Container Spec.
optional bool stdin = 13;
optional bool stdin_once = 14;
optional bool tty = 15;
// Linux contains configuration specific to Linux containers.
optional LinuxContainerConfig linux = 16;
}
message CreateContainerRequest {
// The id of the PodSandbox
optional string pod_sandbox_id = 1;
// The config of the container
optional ContainerConfig config = 2;
// The config of the PodSandbox
optional PodSandboxConfig sandbox_config = 3;
}
message CreateContainerResponse {
// The id of the created container
optional string container_id = 1;
}
message StartContainerRequest {
// The id of the container
optional string container_id = 1;
}
message StartContainerResponse {}
message StopContainerRequest {
// The id of the container
optional string container_id = 1;
// Timeout in seconds to stop the container
optional int64 timeout = 2;
}
message StopContainerResponse {}
message RemoveContainerRequest {
// The id of the container
optional string container_id = 1;
}
message RemoveContainerResponse {}
enum ContainerState {
CREATED = 0;
RUNNING = 1;
EXITED = 2;
UNKNOWN = 3;
}
// ContainerFilter is used to filter containers.
// All those fields are combined with 'AND'
message ContainerFilter {
// Name of the container.
optional string name = 1;
// ID of the container.
optional string id = 2;
// State of the contianer.
optional ContainerState state = 3;
// The id of the pod sandbox
optional string pod_sandbox_id = 4;
// LabelSelector to select matches.
// Only api.MatchLabels is supported for now and the requirements
// are ANDed. MatchExpressions is not supported yet.
map<string, string> label_selector = 5;
}
message ListContainersRequest {
optional ContainerFilter filter = 1;
}
// Container provides the runtime information for a container, such as ID, hash,
// state of the container.
message Container {
// The ID of the container, used by the container runtime to identify
// a container.
optional string id = 1;
// The name of the container
optional string name = 2;
// The spec of the image
optional ImageSpec image = 3;
// Reference to the image in use. For most runtimes, this should be an
// image ID.
optional string image_ref = 4;
// State is the state of the container.
optional ContainerState state = 5;
// Labels are key value pairs that may be used to scope and select individual resources.
map<string, string> labels = 6;
}
message ListContainersResponse {
// List of containers
repeated Container containers = 1;
}
message ContainerStatusRequest {
// The id of the container
optional string container_id = 1;
}
// ContainerStatus represents the status of a container.
message ContainerStatus {
// ID of the container.
optional string id = 1;
// Name of the container.
optional string name = 2;
// Status of the container.
optional ContainerState state = 3;
// Creation time of the container.
optional int64 created_at = 4;
// Start time of the container.
optional int64 started_at = 5;
// Finish time of the container.
optional int64 finished_at = 6;
// Exit code of the container.
optional int32 exit_code = 7;
// The spec of the image
optional ImageSpec image = 8;
// Reference to the image in use. For most runtimes, this should be an
// image ID
optional string image_ref = 9;
// A string explains why container is in such a status.
optional string reason = 10;
// Labels are key value pairs that may be used to scope and select individual resources.
map<string,string> labels = 11;
// Annotations is an unstructured key value map.
map<string,string> annotations = 12;
// Mounts specifies mounts for the container
repeated Mount mounts = 13;
}
message ContainerStatusResponse {
// The status of the container
optional ContainerStatus status = 1;
}
message ExecRequest {
// The id of the container
optional string container_id = 1;
// The cmd to execute
repeated string cmd = 2;
// Whether use tty
optional bool tty = 3;
// Streaming stdin
optional bytes stdin = 4;
}
message ExecResponse {
// Streaming stdout
optional bytes stdout = 1;
// Streaming stderr
optional bytes stderr = 2;
}
message ImageFilter {
// The spec of the image
optional ImageSpec image = 1;
}
message ListImagesRequest {
// The filter to list images
optional ImageFilter filter = 1;
}
// Basic information about a container image.
message Image {
// ID of the image.
optional string id = 1;
// Other names by which this image is known.
repeated string repo_tags = 2;
// Digests by which this image is known.
repeated string repo_digests = 3;
// The size of the image in bytes.
optional uint64 size = 4;
}
message ListImagesResponse {
// List of images
repeated Image images = 1;
}
message ImageStatusRequest {
// The spec of the image
optional ImageSpec image = 1;
}
message ImageStatusResponse {
// The status of the image
optional Image image = 1;
}
// AuthConfig contains authorization information for connecting to a registry.
message AuthConfig {
optional string username = 1;
optional string password = 2;
optional string auth = 3;
optional string server_address = 4;
// IdentityToken is used to authenticate the user and get
// an access token for the registry.
optional string identity_token = 5;
// RegistryToken is a bearer token to be sent to a registry
optional string registry_token = 6;
}
message PullImageRequest {
// The spec of the image
optional ImageSpec image = 1;
// The auth config for pulling image
optional AuthConfig auth = 2;
// The config of the PodSandbox, which is used to pull image in PodSandbox context
optional PodSandboxConfig sandbox_config = 3;
}
message PullImageResponse {}
message RemoveImageRequest {
// The spec of the image
optional ImageSpec image = 1;
}
message RemoveImageResponse {}