mirror of https://github.com/k3s-io/k3s
Merge pull request #33951 from pmorie/selinux-overhaul
Automatic merge from submit-queue Proposal: SELinux enhancements TLDR: Try to make SELinux support better by not requiring Kubelet directory to be labeled with an SELinux type usable from the container. cc @kubernetes/sig-node @yifan-gupull/6/head
Kubernetes Submit Queue
GitHub
commit
6f78c0d912
|
|
@ -0,0 +1,247 @@
|
|||
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
|
||||
|
||||
<!-- BEGIN STRIP_FOR_RELEASE -->
|
||||
|
||||
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
|
||||
<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>
|
||||
|
||||
If you are using a released version of Kubernetes, you should
|
||||
refer to the docs that go with that version.
|
||||
|
||||
Documentation for other releases can be found at
|
||||
[releases.k8s.io](http://releases.k8s.io).
|
||||
</strong>
|
||||
--
|
||||
|
||||
<!-- END STRIP_FOR_RELEASE -->
|
||||
|
||||
<!-- END MUNGE: UNVERSIONED_WARNING -->
|
||||
|
||||
## Abstract
|
||||
|
||||
Presents a proposal for enhancing the security of Kubernetes clusters using
|
||||
SELinux and simplifying the implementation of SELinux support within the
|
||||
Kubelet by removing the need to label the Kubelet directory with an SELinux
|
||||
context usable from a container.
|
||||
|
||||
## Motivation
|
||||
|
||||
The current Kubernetes codebase relies upon the Kubelet directory being
|
||||
labeled with an SELinux context usable from a container. This means that a
|
||||
container escaping namespace isolation will be able to use any file within the
|
||||
Kubelet directory without defeating kernel
|
||||
[MAC (mandatory access control)](https://en.wikipedia.org/wiki/Mandatory_access_control).
|
||||
In order to limit the attack surface, we should enhance the Kubelet to relabel
|
||||
any bind-mounts into containers into a usable SELinux context without depending
|
||||
on the Kubelet directory's SELinux context.
|
||||
|
||||
## Constraints and Assumptions
|
||||
|
||||
1. No API changes allowed
|
||||
2. Behavior must be fully backward compatible
|
||||
3. No new admission controllers - make incremental improvements without huge
|
||||
refactorings
|
||||
|
||||
## Use Cases
|
||||
|
||||
1. As a cluster operator, I want to avoid having to label the Kubelet
|
||||
directory with a label usable from a container, so that I can limit the
|
||||
attack surface available to a container escaping its namespace isolation
|
||||
2. As a user, I want to run a pod without an SELinux context explicitly
|
||||
specified and be isolated using MCS (multi-category security) on systems
|
||||
where SELinux is enabled, so that the pods on each host are isolated from
|
||||
one another
|
||||
3. As a user, I want to run a pod that uses the host IPC or PID namespace and
|
||||
want the system to do the right thing with regard to SELinux, so that no
|
||||
unnecessary relabel actions are performed
|
||||
|
||||
### Labeling the Kubelet directory
|
||||
|
||||
As previously stated, the current codebase relies on the Kubelet directory
|
||||
being labeled with an SELinux context usable from a container. The Kubelet
|
||||
uses the SELinux context of this directory to determine what SELinux context
|
||||
`tmpfs` mounts (provided by the EmptyDir memory-medium option) should receive.
|
||||
The problem with this is that it opens an attack surface to a container that
|
||||
escapes its namespace isolation; such a container would be able to use any
|
||||
file in the Kubelet directory without defeating kernel MAC.
|
||||
|
||||
### SELinux when no context is specified
|
||||
|
||||
When no SELinux context is specified, Kubernetes should just do the right
|
||||
thing, where doing the right thing is defined as isolating pods with a node-
|
||||
unique set of categories. Node-uniqueness means unique among the pods
|
||||
scheduled onto the node. Long-term, we want to have a cluster-wide allocator
|
||||
for MCS labels. Node-unique MCS labels are a good middle ground that is
|
||||
possible without a new, large, feature.
|
||||
|
||||
### SELinux and host IPC and PID namespaces
|
||||
|
||||
Containers in pods that use the host IPC or PID namespaces need access to
|
||||
other processes and IPC mechanisms on the host. Therefore, these containers
|
||||
should be run with the `spc_t` SELinux type by the container runtime. The
|
||||
`spc_t` type is an unconfined type that other SELinux domains are allowed to
|
||||
connect to. In the case where a pod uses one of these host namespaces, it
|
||||
should be unnecessary to relabel the pod's volumes.
|
||||
|
||||
## Analysis
|
||||
|
||||
### Libcontainer SELinux library
|
||||
|
||||
Docker and rkt both use the libcontainer SELinux library. This library
|
||||
provides a method, `GetLxcContexts`, that returns the a unique SELinux
|
||||
contexts for container processes and files used by them. `GetLxcContexts`
|
||||
reads the base SELinux context information from a file at `/etc/selinux/<policy-
|
||||
name>/contexts/lxc_contexts` and then adds a process-unique MCS label.
|
||||
|
||||
Docker and rkt both leverage this call to determine the 'starting' SELinux
|
||||
contexts for containers.
|
||||
|
||||
### Docker
|
||||
|
||||
Docker's behavior when no SELinux context is defined for a container is to
|
||||
give the container a node-unique MCS label.
|
||||
|
||||
#### Sharing IPC namespaces
|
||||
|
||||
On the Docker runtime, the containers in a Kubernetes pod share the IPC and
|
||||
PID namespaces of the pod's infra container.
|
||||
|
||||
Docker's behavior for containers sharing these namespaces is as follows: if a
|
||||
container B shares the IPC namespace of another container A, container B is
|
||||
given the SELinux context of container A. Therefore, for Kubernetes pods
|
||||
running on docker, in a vacuum the containers in a pod should have the same
|
||||
SELinux context.
|
||||
|
||||
[**Known issue**](https://bugzilla.redhat.com/show_bug.cgi?id=1377869): When
|
||||
the seccomp profile is set on a docker container that shares the IPC namespace
|
||||
of another container, that container will not receive the other container's
|
||||
SELinux context.
|
||||
|
||||
#### Host IPC and PID namespaces
|
||||
|
||||
In the case of a pod that shares the host IPC or PID namespace, this flag is
|
||||
simply ignored and the container receives the `spc_t` SELinux type. The
|
||||
`spc_t` type is unconfined, and so no relabeling needs to be done for volumes
|
||||
for these pods. Currently, however, there is code which relabels volumes into
|
||||
explicitly specified SELinux contexts for these pods. This code is unnecessary
|
||||
and should be removed.
|
||||
|
||||
#### Relabeling bind-mounts
|
||||
|
||||
Docker is capable of relabeling bind-mounts into containers using the `:Z`
|
||||
bind-mount flag. However, in the current implementation of the docker runtime
|
||||
in Kubernetes, the `:Z` option is only applied when the pod's SecurityContext
|
||||
contains an SELinux context. We could easily implement the correct behaviors
|
||||
by always setting `:Z` on systems where SELinux is enabled.
|
||||
|
||||
### rkt
|
||||
|
||||
rkt's behavior when no SELinux context is defined for a pod is similar to
|
||||
Docker's -- an SELinux context with a node-unique MCS label is given to the
|
||||
containers of a pod.
|
||||
|
||||
#### Sharing IPC namespaces
|
||||
|
||||
Containers (apps, in rkt terminology) in rkt pods share an IPC and PID
|
||||
namespace by default.
|
||||
|
||||
#### Relabeling bind-mounts
|
||||
|
||||
Bind-mounts into rkt pods are automatically relabeled into the pod's SELinux
|
||||
context.
|
||||
|
||||
#### Host IPC and PID namespaces
|
||||
|
||||
Using the host IPC and PID namespaces is not currently supported by rkt.
|
||||
|
||||
## Proposed Changes
|
||||
|
||||
### Refactor `pkg/util/selinux`
|
||||
|
||||
1. The `selinux` package should provide a method `SELinuxEnabled` that returns
|
||||
whether SELinux is enabled, and is built for all platforms (the
|
||||
libcontainer SELinux is only built on linux)
|
||||
2. The `SelinuxContextRunner` interface should be renamed to `SELinuxRunner`
|
||||
and be changed to have the same method names and signatures as the
|
||||
libcontainer methods its implementations wrap
|
||||
3. The `SELinuxRunner` interface should have a new method added called
|
||||
`GetLxcContexts`; this should return a **shared** (ie, without MCS labels)
|
||||
SELinux context usable by a container
|
||||
|
||||
```go
|
||||
package selinux
|
||||
|
||||
// Note: the libcontainer SELinux package is only built for Linux, so it is
|
||||
// necessary to have a NOP wrapper which is built for non-Linux platforms to
|
||||
// allow code that links to this package not to differentiate its own methods
|
||||
// for Linux and non-Linux platforms.
|
||||
//
|
||||
// SELinuxRunner wraps certain libcontainer SELinux calls. For more
|
||||
// information, see:
|
||||
//
|
||||
// https://github.com/opencontainers/runc/blob/master/libcontainer/selinux/selinux.go
|
||||
type SELinuxRunner interface {
|
||||
// Setfilecon sets the SELinux context for the given path or returns an
|
||||
// error.
|
||||
Setfilecon(path, context string) error
|
||||
|
||||
// Getfilecon returns the SELinux context for the given path or returns an
|
||||
// error.
|
||||
Getfilecon(path string) (string, error)
|
||||
|
||||
// GetLxcContexts returns the process and file SELinux contexts to use for
|
||||
// containers.
|
||||
GetLxcContexts() (string, string)
|
||||
}
|
||||
```
|
||||
|
||||
### Kubelet Changes
|
||||
|
||||
1. The `relabelVolumes` method in `kubelet_volumes.go` is not needed and can
|
||||
be removed
|
||||
2. The `GenerateRunContainerOptions` method in `kubelet_pods.go` should no
|
||||
longer call `relabelVolumes`
|
||||
3. The `makeHostsMount` method in `kubelet_pods.go` should set the
|
||||
`SELinuxRelabel` attribute of the mount for the pod's hosts file to `true`
|
||||
|
||||
### Changes to `pkg/kubelet/dockertools/`
|
||||
|
||||
1. The `makeMountBindings` should be changed to:
|
||||
1. No longer accept the `podHasSELinuxLabel` parameter
|
||||
2. Always use the `:Z` bind-mount flag when SELinux is enabled and the mount
|
||||
has the `SELinuxRelabel` attribute set to `true`
|
||||
2. The `runContainer` method should be changed to always use the `:Z`
|
||||
bind-mount flag on the termination message mount when SELinux is enabled
|
||||
|
||||
### Changes to `pkg/kubelet/rkt`
|
||||
|
||||
The should not be any required changes for the rkt runtime; we should test to
|
||||
ensure things work as expected under rkt.
|
||||
|
||||
### Changes to volume plugins and infrastructure
|
||||
|
||||
1. The `VolumeHost` interface contains a method called `GetRootContext`; this
|
||||
is an artifact of the old assumptions about the Kubelet directory's SELinux
|
||||
context and can be removed
|
||||
2. The `empty_dir.go` file should be changed to create an `SELinuxRunner` and
|
||||
call its `GetLxcContexts` method to determine the right SELinux context to
|
||||
give `tmpfs` mounts
|
||||
|
||||
### Changes to `pkg/controller/...`
|
||||
|
||||
The `VolumeHost` abstraction is used in a couple of PV controllers as NOP
|
||||
implementations. These should be altered to no longer include `GetRootContext`.
|
||||
|
||||
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
||||
[]()
|
||||
<!-- END MUNGE: GENERATED_ANALYTICS -->
|
||||
Loading…
Reference in New Issue