github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Replace limits with request where appropriate

pull/6/head
dinghaiyang 2015-09-04 18:44:56 +08:00
parent 0f4fa4ed25
commit d57263b94b
4 changed files with 51 additions and 35 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
docs/design/resources.md
View File

@ -33,8 +33,8 @@ Documentation for other releases can be found at
**Note: this is a design doc, which describes features that have not been completely implemented. **Note: this is a design doc, which describes features that have not been completely implemented.
User documentation of the current state is [here](../user-guide/compute-resources.md). The tracking issue for User documentation of the current state is [here](../user-guide/compute-resources.md). The tracking issue for
implementation of this model is implementation of this model is
[#168](http://issue.k8s.io/168). Currently, only memory and [#168](http://issue.k8s.io/168). Currently, both limits and requests of memory and
cpu limits on containers (not pods) are supported. "memory" is in bytes and "cpu" is in cpu on containers (not pods) are supported. "memory" is in bytes and "cpu" is in
milli-cores.** milli-cores.**
# The Kubernetes resource model # The Kubernetes resource model
@ -123,7 +123,6 @@ Where:
* Internally, the Kubernetes master can decide the defaulting behavior and the kubelet implementation may expected an absolute specification. For example, if the master decided that "the default is unbounded" it would pass 2^64 to the kubelet. * Internally, the Kubernetes master can decide the defaulting behavior and the kubelet implementation may expected an absolute specification. For example, if the master decided that "the default is unbounded" it would pass 2^64 to the kubelet.
## Kubernetes-defined resource types ## Kubernetes-defined resource types
The following resource types are predefined ("reserved") by Kubernetes in the `kubernetes.io` namespace, and so cannot be used for user-defined resources. Note that the syntax of all resource types in the resource spec is deliberately similar, but some resource types (e.g., CPU) may receive significantly more support than simply tracking quantities in the schedulers and/or the Kubelet. The following resource types are predefined ("reserved") by Kubernetes in the `kubernetes.io` namespace, and so cannot be used for user-defined resources. Note that the syntax of all resource types in the resource spec is deliberately similar, but some resource types (e.g., CPU) may receive significantly more support than simply tracking quantities in the schedulers and/or the Kubelet.

3
docs/proposals/resource-qos.md
View File

@ -101,7 +101,8 @@ API changes for request
- Add validation code that checks request <= limit, and validation test cases (api/validation/validation.go) - Add validation code that checks request <= limit, and validation test cases (api/validation/validation.go)
Scheduler Changes Scheduler Changes
- Use requests instead of limits in CheckPodsExceedingCapacity and PodFitsResources (scheduler/algorithm/predicates.go) - Predicates: Use requests instead of limits in CheckPodsExceedingCapacity and PodFitsResources (scheduler/algorithm/predicates/predicates.go)
- Priorities: Use requests instead of limits in LeastRequestedPriority and BalancedResourceAllocation(scheduler/algorithm/priorities/priorities.go)(PR #12718)
Container Manager Changes Container Manager Changes
- Use requests to assign CPU shares for Docker (kubelet/dockertools/container_manager.go) - Use requests to assign CPU shares for Docker (kubelet/dockertools/container_manager.go)

71
docs/user-guide/compute-resources.md
View File

@ -37,8 +37,8 @@ Documentation for other releases can be found at
<!-- BEGIN MUNGE: GENERATED_TOC --> <!-- BEGIN MUNGE: GENERATED_TOC -->
- [Compute Resources](#compute-resources) - [Compute Resources](#compute-resources)
- [Container and Pod Resource Limits](#container-and-pod-resource-limits) - [Resource Requests and Limits of Pod and Container](#resource-requests-and-limits-of-pod-and-container)
- [How Pods with Resource Limits are Scheduled](#how-pods-with-resource-limits-are-scheduled) - [How Pods with Resource Requests are Scheduled](#how-pods-with-resource-requests-are-scheduled)
- [How Pods with Resource Limits are Run](#how-pods-with-resource-limits-are-run) - [How Pods with Resource Limits are Run](#how-pods-with-resource-limits-are-run)
- [Monitoring Compute Resource Usage](#monitoring-compute-resource-usage) - [Monitoring Compute Resource Usage](#monitoring-compute-resource-usage)
- [Troubleshooting](#troubleshooting) - [Troubleshooting](#troubleshooting)
@ -49,9 +49,11 @@ Documentation for other releases can be found at
<!-- END MUNGE: GENERATED_TOC --> <!-- END MUNGE: GENERATED_TOC -->
When specifying a [pod](pods.md), you can optionally specify how much CPU and memory (RAM) each When specifying a [pod](pods.md), you can optionally specify how much CPU and memory (RAM) each
container needs. When containers have resource limits, the scheduler is able to make better container needs. When containers have their resource requests specified, the scheduler is
decisions about which nodes to place pods on, and contention for resources can be handled in a able to make better decisions about which nodes to place pods on; and when containers have their
consistent manner. limits specified, contention for resources on a node can be handled in a specified manner. For
more details about the difference between requests and limits, please refer to
[Resource QoS](../proposals/resource-qos.md).
*CPU* and *memory* are each a *resource type*. A resource type has a base unit. CPU is specified *CPU* and *memory* are each a *resource type*. A resource type has a base unit. CPU is specified
in units of cores. Memory is specified in units of bytes. in units of cores. Memory is specified in units of bytes.
@ -62,22 +64,26 @@ distinct from [API resources](working-with-resources.md). API resources, such a
[services](services.md) are objects that can be written to and retrieved from the Kubernetes API [services](services.md) are objects that can be written to and retrieved from the Kubernetes API
server. server.
## Container and Pod Resource Limits ## Resource Requests and Limits of Pod and Container
Each container of a Pod can optionally specify `spec.container[].resources.limits.cpu` and/or Each container of a Pod can optionally specify `spec.container[].resources.limits.cpu` and/or
`spec.container[].resources.limits.memory`. The `spec.container[].resources.requests` field is not `spec.container[].resources.limits.memory` and/or `spec.container[].resources.requests.cpu`
currently used and need not be set. and/or `spec.container[].resources.requests.memory`.
Specifying resource limits is optional. In some clusters, an unset value may be replaced with a Specifying resource requests and/or limits is optional. In some clusters, unset limits or requests
default value when a pod is created or updated. The default value depends on how the cluster is may be replaced with default values when a pod is created or updated. The default value depends on
configured. how the cluster is configured. If value of requests is not specified, they are set to be equal
to limits by default. Please note that resource limits must be greater than or equal to resource
requests.
Although limits can only be specified on individual containers, it is convenient to talk about pod Although requests/limits can only be specified on individual containers, it is convenient to talk
resource limits. A *pod resource limit* for a particular resource type is the sum of the resource about pod resource requests/limits. A *pod resource request/limit* for a particular resource
limits of that type for each container in the pod, with unset values treated as zero. type is the sum of the resource requests/limits of that type for each container in the pod, with
unset values treated as zero (or equal to default values in some cluster configurations).
The following pod has two containers. Each has a limit of 0.5 core of cpu and 128MiB The following pod has two containers. Each has a request of 0.25 core of cpu and 64MiB
(2<sup>20</sup> bytes) of memory. The pod can be said to have a limit of 1 core and 256MiB of (2<sup>20</sup> bytes) of memory and a limit of 0.5 core of cpu and 128MiB of memory. The pod can
be said to have a request of 0.5 core and 128 MiB of memory and a limit of 1 core and 256MiB of
memory. memory.
```yaml ```yaml
@ -90,32 +96,35 @@ spec:
- name: db - name: db
image: mysql image: mysql
resources: resources:
requests:
memory: "64Mi"
cpu: "250m"
limits: limits:
memory: "128Mi" memory: "128Mi"
cpu: "500m" cpu: "500m"
- name: wp - name: wp
image: wordpress image: wordpress
resources: resources:
limits: requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi" memory: "128Mi"
cpu: "500m" cpu: "500m"
``` ```
## How Pods with Resource Limits are Scheduled ## How Pods with Resource Requests are Scheduled
When a pod is created, the Kubernetes scheduler selects a node for the pod to When a pod is created, the Kubernetes scheduler selects a node for the pod to
run on. Each node has a maximum capacity for each of the resource types: the run on. Each node has a maximum capacity for each of the resource types: the
amount of CPU and memory it can provide for pods. The scheduler ensures that, amount of CPU and memory it can provide for pods. The scheduler ensures that,
for each resource type (CPU and memory), the sum of the resource limits of the for each resource type (CPU and memory), the sum of the resource requests of the
containers scheduled to the node is less than the capacity of the node. Note containers scheduled to the node is less than the capacity of the node. Note
that although actual memory or CPU resource usage on nodes is very low, the that although actual memory or CPU resource usage on nodes is very low, the
scheduler will still refuse to place pods onto nodes if the capacity check scheduler will still refuse to place pods onto nodes if the capacity check
fails. This protects against a resource shortage on a node when resource usage fails. This protects against a resource shortage on a node when resource usage
later increases, such as due to a daily peak in request rate. later increases, such as due to a daily peak in request rate.
Note: Although the scheduler normally spreads pods out across nodes, there are currently some cases
where pods with no limits (unset values) might all land on the same node.
## How Pods with Resource Limits are Run ## How Pods with Resource Limits are Run
When kubelet starts a container of a pod, it passes the CPU and memory limits to the container When kubelet starts a container of a pod, it passes the CPU and memory limits to the container
@ -157,13 +166,17 @@ until a place can be found. An event will be produced each time the scheduler
place for the pod, like this: place for the pod, like this:
```console ```console
$ kubectl describe pods/frontend | grep -A 3 Events $ kubectl describe pod frontend | grep -A 3 Events
Events: Events:
FirstSeen LastSeen Count From SubobjectPath Reason Message FirstSeen LastSeen Count From Subobject PathReason Message
Tue, 30 Jun 2015 09:01:41 -0700 Tue, 30 Jun 2015 09:39:27 -0700 128 {scheduler } failedScheduling Error scheduling: For each of these fitness predicates, pod frontend failed on at least one node: PodFitsResources. 36s 5s 6 {scheduler } FailedScheduling Failed for reason PodExceedsFreeCPU and possibly others
``` ```
If a pod or pods are pending with this message, then there are several things to try: In the case shown above, the pod "frontend" fails to be scheduled due to insufficient
CPU resource on the node. Similar error messages can also suggest failure due to insufficient
memory (PodExceedsFreeMemory). In general, if a pod or pods are pending with this message and
alike, then there are several things to try:
- Add more nodes to the cluster. - Add more nodes to the cluster.
- Terminate unneeded pods to make room for pending pods. - Terminate unneeded pods to make room for pending pods.
- Check that the pod is not larger than all the nodes. For example, if all the nodes - Check that the pod is not larger than all the nodes. For example, if all the nodes
@ -266,13 +279,11 @@ The current system only allows resource quantities to be specified on a containe
It is planned to improve accounting for resources which are shared by all containers in a pod, It is planned to improve accounting for resources which are shared by all containers in a pod,
such as [EmptyDir volumes](volumes.md#emptydir). such as [EmptyDir volumes](volumes.md#emptydir).
The current system only supports container limits for CPU and Memory. The current system only supports container requests and limits for CPU and Memory.
It is planned to add new resource types, including a node disk space It is planned to add new resource types, including a node disk space
resource, and a framework for adding custom [resource types](../design/resources.md#resource-types). resource, and a framework for adding custom [resource types](../design/resources.md#resource-types).
The current system does not facilitate overcommitment of resources because resources reserved Kubernetes supports overcommitment of resources by supporting multiple levels of [Quality of Service](http://issue.k8s.io/168).
with container limits are assured. It is planned to support multiple levels of [Quality of
Service](http://issue.k8s.io/168).
Currently, one unit of CPU means different things on different cloud providers, and on different Currently, one unit of CPU means different things on different cloud providers, and on different
machine types within the same cloud providers. For example, on AWS, the capacity of a node machine types within the same cloud providers. For example, on AWS, the capacity of a node

7
docs/user-guide/production-pods.md
View File

@ -272,9 +272,14 @@ spec:
cpu: 500m cpu: 500m
# memory units are bytes # memory units are bytes
memory: 64Mi memory: 64Mi
requests:
# cpu units are cores
cpu: 500m
# memory units are bytes
memory: 64Mi
``` ```
The container will die due to OOM (out of memory) if it exceeds its specified limit, so specifying a value a little higher than expected generally improves reliability. The container will die due to OOM (out of memory) if it exceeds its specified limit, so specifying a value a little higher than expected generally improves reliability. By specifying request, pod is guaranteed to be able to use that much of resource when needed. See [Resource QoS](../proposals/resource-qos.md) for the difference between resource limits and requests.
If you’re not sure how much resources to request, you can first launch the application without specifying resources, and use [resource usage monitoring](monitoring.md) to determine appropriate values. If you’re not sure how much resources to request, you can first launch the application without specifying resources, and use [resource usage monitoring](monitoring.md) to determine appropriate values.