mirror of https://github.com/k3s-io/k3s
274 lines
12 KiB
Markdown
274 lines
12 KiB
Markdown
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
|
|
|
|
<!-- BEGIN STRIP_FOR_RELEASE -->
|
|
|
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
|
width="25" height="25">
|
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
|
width="25" height="25">
|
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
|
width="25" height="25">
|
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
|
width="25" height="25">
|
|
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
|
width="25" height="25">
|
|
|
|
<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>
|
|
|
|
If you are using a released version of Kubernetes, you should
|
|
refer to the docs that go with that version.
|
|
|
|
<!-- TAG RELEASE_LINK, added by the munger automatically -->
|
|
<strong>
|
|
The latest release of this document can be found
|
|
[here](http://releases.k8s.io/release-1.1/docs/design/horizontal-pod-autoscaler.md).
|
|
|
|
Documentation for other releases can be found at
|
|
[releases.k8s.io](http://releases.k8s.io).
|
|
</strong>
|
|
--
|
|
|
|
<!-- END STRIP_FOR_RELEASE -->
|
|
|
|
<!-- END MUNGE: UNVERSIONED_WARNING -->
|
|
|
|
# Horizontal Pod Autoscaling
|
|
|
|
## Preface
|
|
|
|
This document briefly describes the design of the horizontal autoscaler for pods.
|
|
The autoscaler (implemented as a Kubernetes API resource and controller) is responsible for dynamically controlling
|
|
the number of replicas of some collection (e.g. the pods of a ReplicationController) to meet some objective(s),
|
|
for example a target per-pod CPU utilization.
|
|
|
|
This design supersedes [autoscaling.md](http://releases.k8s.io/release-1.0/docs/proposals/autoscaling.md).
|
|
|
|
## Overview
|
|
|
|
The resource usage of a serving application usually varies over time: sometimes the demand for the application rises,
|
|
and sometimes it drops.
|
|
In Kubernetes version 1.0, a user can only manually set the number of serving pods.
|
|
Our aim is to provide a mechanism for the automatic adjustment of the number of pods based on CPU utilization statistics
|
|
(a future version will allow autoscaling based on other resources/metrics).
|
|
|
|
## Scale Subresource
|
|
|
|
In Kubernetes version 1.1, we are introducing Scale subresource and implementing horizontal autoscaling of pods based on it.
|
|
Scale subresource is supported for replication controllers and deployments.
|
|
Scale subresource is a Virtual Resource (does not correspond to an object stored in etcd).
|
|
It is only present in the API as an interface that a controller (in this case the HorizontalPodAutoscaler) can use to dynamically scale
|
|
the number of replicas controlled by some other API object (currently ReplicationController and Deployment) and to learn the current number of replicas.
|
|
Scale is a subresource of the API object that it serves as the interface for.
|
|
The Scale subresource is useful because whenever we introduce another type we want to autoscale, we just need to implement the Scale subresource for it.
|
|
The wider discussion regarding Scale took place in [#1629](https://github.com/kubernetes/kubernetes/issues/1629).
|
|
|
|
Scale subresource is in API for replication controller or deployment under the following paths:
|
|
|
|
`apis/extensions/v1beta1/replicationcontrollers/myrc/scale`
|
|
|
|
`apis/extensions/v1beta1/deployments/mydeployment/scale`
|
|
|
|
It has the following structure:
|
|
|
|
```go
|
|
// represents a scaling request for a resource.
|
|
type Scale struct {
|
|
unversioned.TypeMeta
|
|
api.ObjectMeta
|
|
|
|
// defines the behavior of the scale.
|
|
Spec ScaleSpec
|
|
|
|
// current status of the scale.
|
|
Status ScaleStatus
|
|
}
|
|
|
|
// describes the attributes of a scale subresource
|
|
type ScaleSpec struct {
|
|
// desired number of instances for the scaled object.
|
|
Replicas int `json:"replicas,omitempty"`
|
|
}
|
|
|
|
// represents the current status of a scale subresource.
|
|
type ScaleStatus struct {
|
|
// actual number of observed instances of the scaled object.
|
|
Replicas int `json:"replicas"`
|
|
|
|
// label query over pods that should match the replicas count.
|
|
Selector map[string]string `json:"selector,omitempty"`
|
|
}
|
|
```
|
|
|
|
Writing to `ScaleSpec.Replicas` resizes the replication controller/deployment associated with
|
|
the given Scale subresource.
|
|
`ScaleStatus.Replicas` reports how many pods are currently running in the replication controller/deployment,
|
|
and `ScaleStatus.Selector` returns selector for the pods.
|
|
|
|
## HorizontalPodAutoscaler Object
|
|
|
|
In Kubernetes version 1.1, we are introducing HorizontalPodAutoscaler object. It is accessible under:
|
|
|
|
`apis/extensions/v1beta1/horizontalpodautoscalers/myautoscaler`
|
|
|
|
It has the following structure:
|
|
|
|
```go
|
|
// configuration of a horizontal pod autoscaler.
|
|
type HorizontalPodAutoscaler struct {
|
|
unversioned.TypeMeta
|
|
api.ObjectMeta
|
|
|
|
// behavior of autoscaler.
|
|
Spec HorizontalPodAutoscalerSpec
|
|
|
|
// current information about the autoscaler.
|
|
Status HorizontalPodAutoscalerStatus
|
|
}
|
|
|
|
// specification of a horizontal pod autoscaler.
|
|
type HorizontalPodAutoscalerSpec struct {
|
|
// reference to Scale subresource; horizontal pod autoscaler will learn the current resource
|
|
// consumption from its status,and will set the desired number of pods by modifying its spec.
|
|
ScaleRef SubresourceReference
|
|
// lower limit for the number of pods that can be set by the autoscaler, default 1.
|
|
MinReplicas *int
|
|
// upper limit for the number of pods that can be set by the autoscaler.
|
|
// It cannot be smaller than MinReplicas.
|
|
MaxReplicas int
|
|
// target average CPU utilization (represented as a percentage of requested CPU) over all the pods;
|
|
// if not specified it defaults to the target CPU utilization at 80% of the requested resources.
|
|
CPUUtilization *CPUTargetUtilization
|
|
}
|
|
|
|
type CPUTargetUtilization struct {
|
|
// fraction of the requested CPU that should be utilized/used,
|
|
// e.g. 70 means that 70% of the requested CPU should be in use.
|
|
TargetPercentage int
|
|
}
|
|
|
|
// current status of a horizontal pod autoscaler
|
|
type HorizontalPodAutoscalerStatus struct {
|
|
// most recent generation observed by this autoscaler.
|
|
ObservedGeneration *int64
|
|
|
|
// last time the HorizontalPodAutoscaler scaled the number of pods;
|
|
// used by the autoscaler to control how often the number of pods is changed.
|
|
LastScaleTime *unversioned.Time
|
|
|
|
// current number of replicas of pods managed by this autoscaler.
|
|
CurrentReplicas int
|
|
|
|
// desired number of replicas of pods managed by this autoscaler.
|
|
DesiredReplicas int
|
|
|
|
// current average CPU utilization over all pods, represented as a percentage of requested CPU,
|
|
// e.g. 70 means that an average pod is using now 70% of its requested CPU.
|
|
CurrentCPUUtilizationPercentage *int
|
|
}
|
|
```
|
|
|
|
`ScaleRef` is a reference to the Scale subresource.
|
|
`MinReplicas`, `MaxReplicas` and `CPUUtilization` define autoscaler configuration.
|
|
We are also introducing HorizontalPodAutoscalerList object to enable listing all autoscalers in a namespace:
|
|
|
|
```go
|
|
// list of horizontal pod autoscaler objects.
|
|
type HorizontalPodAutoscalerList struct {
|
|
unversioned.TypeMeta
|
|
unversioned.ListMeta
|
|
|
|
// list of horizontal pod autoscaler objects.
|
|
Items []HorizontalPodAutoscaler
|
|
}
|
|
```
|
|
|
|
## Autoscaling Algorithm
|
|
|
|
The autoscaler is implemented as a control loop. It periodically queries pods described by `Status.PodSelector` of Scale subresource, and collects their CPU utilization.
|
|
Then, it compares the arithmetic mean of the pods' CPU utilization with the target defined in `Spec.CPUUtilization`,
|
|
and adjust the replicas of the Scale if needed to match the target
|
|
(preserving condition: MinReplicas <= Replicas <= MaxReplicas).
|
|
|
|
The period of the autoscaler is controlled by `--horizontal-pod-autoscaler-sync-period` flag of controller manager.
|
|
The default value is 30 seconds.
|
|
|
|
|
|
CPU utilization is the recent CPU usage of a pod (average across the last 1 minute) divided by the CPU requested by the pod.
|
|
In Kubernetes version 1.1, CPU usage is taken directly from Heapster.
|
|
In future, there will be API on master for this purpose
|
|
(see [#11951](https://github.com/kubernetes/kubernetes/issues/11951)).
|
|
|
|
The target number of pods is calculated from the following formula:
|
|
|
|
```
|
|
TargetNumOfPods = ceil(sum(CurrentPodsCPUUtilization) / Target)
|
|
```
|
|
|
|
Starting and stopping pods may introduce noise to the metric (for instance, starting may temporarily increase CPU).
|
|
So, after each action, the autoscaler should wait some time for reliable data.
|
|
Scale-up can only happen if there was no rescaling within the last 3 minutes.
|
|
Scale-down will wait for 5 minutes from the last rescaling.
|
|
Moreover any scaling will only be made if: `avg(CurrentPodsConsumption) / Target` drops below 0.9 or increases above 1.1 (10% tolerance).
|
|
Such approach has two benefits:
|
|
|
|
* Autoscaler works in a conservative way.
|
|
If new user load appears, it is important for us to rapidly increase the number of pods,
|
|
so that user requests will not be rejected.
|
|
Lowering the number of pods is not that urgent.
|
|
|
|
* Autoscaler avoids thrashing, i.e.: prevents rapid execution of conflicting decision if the load is not stable.
|
|
|
|
## Relative vs. absolute metrics
|
|
|
|
We chose values of the target metric to be relative (e.g. 90% of requested CPU resource) rather than absolute (e.g. 0.6 core) for the following reason.
|
|
If we choose absolute metric, user will need to guarantee that the target is lower than the request.
|
|
Otherwise, overloaded pods may not be able to consume more than the autoscaler's absolute target utilization,
|
|
thereby preventing the autoscaler from seeing high enough utilization to trigger it to scale up.
|
|
This may be especially troublesome when user changes requested resources for a pod
|
|
because they would need to also change the autoscaler utilization threshold.
|
|
Therefore, we decided to choose relative metric.
|
|
For user, it is enough to set it to a value smaller than 100%, and further changes of requested resources will not invalidate it.
|
|
|
|
## Support in kubectl
|
|
|
|
To make manipulation of HorizontalPodAutoscaler object simpler, we added support for
|
|
creating/updating/deleting/listing of HorizontalPodAutoscaler to kubectl.
|
|
In addition, in future, we are planning to add kubectl support for the following use-cases:
|
|
* When creating a replication controller or deployment with `kubectl create [-f]`, there should be
|
|
a possibility to specify an additional autoscaler object.
|
|
(This should work out-of-the-box when creation of autoscaler is supported by kubectl as we may include
|
|
multiple objects in the same config file).
|
|
* *[future]* When running an image with `kubectl run`, there should be an additional option to create
|
|
an autoscaler for it.
|
|
* *[future]* We will add a new command `kubectl autoscale` that will allow for easy creation of an autoscaler object
|
|
for already existing replication controller/deployment.
|
|
|
|
## Next steps
|
|
|
|
We list here some features that are not supported in Kubernetes version 1.1.
|
|
However, we want to keep them in mind, as they will most probably be needed in future.
|
|
Our design is in general compatible with them.
|
|
* *[future]* **Autoscale pods based on metrics different than CPU** (e.g. memory, network traffic, qps).
|
|
This includes scaling based on a custom/application metric.
|
|
* *[future]* **Autoscale pods base on an aggregate metric.**
|
|
Autoscaler, instead of computing average for a target metric across pods, will use a single, external, metric (e.g. qps metric from load balancer).
|
|
The metric will be aggregated while the target will remain per-pod
|
|
(e.g. when observing 100 qps on load balancer while the target is 20 qps per pod, autoscaler will set the number of replicas to 5).
|
|
* *[future]* **Autoscale pods based on multiple metrics.**
|
|
If the target numbers of pods for different metrics are different, choose the largest target number of pods.
|
|
* *[future]* **Scale the number of pods starting from 0.**
|
|
All pods can be turned-off, and then turned-on when there is a demand for them.
|
|
When a request to service with no pods arrives, kube-proxy will generate an event for autoscaler
|
|
to create a new pod.
|
|
Discussed in [#3247](https://github.com/kubernetes/kubernetes/issues/3247).
|
|
* *[future]* **When scaling down, make more educated decision which pods to kill.**
|
|
E.g.: if two or more pods from the same replication controller are on the same node, kill one of them.
|
|
Discussed in [#4301](https://github.com/kubernetes/kubernetes/issues/4301).
|
|
|
|
|
|
|
|
|
|
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
|
[]()
|
|
<!-- END MUNGE: GENERATED_ANALYTICS -->
|