github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
k3s/CHANGELOG.md

898 KiB
Raw Blame History

v1.6.7

Documentation & Examples

Downloads for v1.6.7

filename sha256 hash
kubernetes.tar.gz 6522086d9666543ed4e88a791626953acd1ea843eb024f16f4a4a2390dcbb2b2
kubernetes-src.tar.gz b2a73f140966ba0080ce16e3b9a67d5fd9849b36942f3490e9f8daa0fe4511c4

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz ffa06a16a3091b2697ef14f8e28bb08000455bd9b719cf0f510f011b864cd1e0
kubernetes-client-darwin-amd64.tar.gz 32de3e38f7a60c9171a63f43a2c7f0b2d8f8ba55d51468d8dbf7847dbd943b45
kubernetes-client-linux-386.tar.gz d9c27321007607cc5afb2ff5b3cac210471d55dd1c3a478c6703ab72d187211e
kubernetes-client-linux-amd64.tar.gz 54947ef84181e89f9dbacedd54717cbed5cc7f9c36cb37bc8afc9097648e2c91
kubernetes-client-linux-arm64.tar.gz e96d300eb6526705b1c1bedaaf3f4746f3e5d6b49ccc7e60650eb9ee022fba0e
kubernetes-client-linux-arm.tar.gz e4605dca3948264fba603dc8f95b202528eb8ad4ca99c7f3a61f77031e7ba756
kubernetes-client-linux-ppc64le.tar.gz 8b77793aea5abf1c17b73f7e11476b9d387f3dc89e5d8405ffadd1a395258483
kubernetes-client-linux-s390x.tar.gz ff3ddec930a0ffdc83fe324d544d4657d57a64a3973fb9df4ddaa7a98228d7fb
kubernetes-client-windows-386.tar.gz ce09e4b071bb06039ad9bdf6a1059d59cf129dce942600fcdc9d320ff0c07a7a
kubernetes-client-windows-amd64.tar.gz e985644f582945274e82764742f02bd175f05128c1945e987d06973dd5f5a56d

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 1287bb85f1057eae53f8bb4e4475c990783e43d2f57ea1c551fdf2da7ca5345d
kubernetes-server-linux-arm64.tar.gz 51623850475669be59f6428922ba316d4dd60d977f892adfaf0ca0845c38506c
kubernetes-server-linux-arm.tar.gz a5331022d29f085e6b7fc4ae064af64024eba6a02ae54e78c2e84b40d0aec598
kubernetes-server-linux-ppc64le.tar.gz 93d52e84d0fea5bdf3ede6784b8da6c501e0430c74430da3a125bd45c557e10a
kubernetes-server-linux-s390x.tar.gz baccbb6fc497f433c2bd93146c31fbca1da427e0d6ac8483df26dd42ccb79c6e

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz 0cfdd51de879869e7ef40a17dfa1a303a596833fb567c3b7e4f82ba0cf863839
kubernetes-node-linux-arm64.tar.gz d07ef669d94ea20a4a9e3a38868ac389dab4d3f2bdf8b27280724fe63f4de3c3
kubernetes-node-linux-arm.tar.gz 1cc9b6a8aee4e59967421cbded21c0a20f02c39288781f504e55ad6ca71d1037
kubernetes-node-linux-ppc64le.tar.gz 3f412096d8b249d671f924c3ee4aecf3656186fde4509ce9f560f67a9a166b6d
kubernetes-node-linux-s390x.tar.gz 2cca7629c1236b3435e6e31498c1f8216d7cca4236d8ad0ae10c83a422519a34
kubernetes-node-windows-amd64.tar.gz 4f859fba52c044a9ce703528760967e1efa47a359603b5466c0dc0748eb25e36

Changelog since v1.6.6

Other notable changes

  • kubeadm: Expose only the cluster-info ConfigMap in the kube-public ns (#48050, @luxas)
  • Fix kubelet request timeout when stopping a container. (#46267, @Random-Liu)
  • Add generic NoSchedule toleration to fluentd in gcp config. (#48182, @gmarek)
  • Update cluster-proportional-autoscaler, fluentd-gcp, and kube-addon-manager, and kube-dns addons with refreshed base images containing fixes for CVE-2016-9841, CVE-2016-9843, CVE-2017-2616, and CVE-2017-6512. (#47454, @ixdy)
  • Fix fluentd-gcp configuration to facilitate JSON parsing (#48139, @crassirostris)
  • Bump runc to v1.0.0-rc2-49-gd223e2a - fixes failed to initialise top level QOS containers kubelet error. (#48117, @sjenning)
  • kubefed init correctly checks for RBAC API enablement. (#48077, @liggitt)
  • kubectl api-versions now always fetches information about enabled API groups and versions instead of using the local cache. (#48016, @liggitt)
  • Fix kubelet event recording for selected events. (#46246, @derekwaynecarr)
  • Fix Invalid value: "foregroundDeletion" error when attempting to delete a resource. (#46500, @tnozicka)

v1.7.0

Documentation & Examples

Downloads for v1.7.0

filename sha256 hash
kubernetes.tar.gz 947f1dd9a9b6b427faac84067a30c86e83e6391eb42f09ddcc50a8694765c31a
kubernetes-src.tar.gz d3d8b0bfc31164dd703b38d8484cfed7981cacd1e496731880afa87f8bf39aac

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz da298e24318e57ac8a558c390117bd7e9e596b3bdf1c5960979898fefe6c5c88
kubernetes-client-darwin-amd64.tar.gz c22f72e1592731155db5b05d0d660f1d7314288cb020f7980e2a109d9e7ba0e5
kubernetes-client-linux-386.tar.gz fc8e90e96360c3a2c8ec56903ab5acde1dffa4d641e1ee27b804ee6d8e824cf6
kubernetes-client-linux-amd64.tar.gz 8b3ed03f8a4b3a1ec124abde01632ee6dcec9daf9376f0288fd7500b5173981c
kubernetes-client-linux-arm64.tar.gz 8930c74dab9ada31e6994f0dc3fb22d41a602a2880b6b17112718ce73eac0574
kubernetes-client-linux-arm.tar.gz 20a6f4645cab3c0aef72f849ae90b2691605fd3f670ce36cc8aa11aef31c6edb
kubernetes-client-linux-ppc64le.tar.gz 509e214d55e8df1906894cbdc166e791761a3b82a52bcea0de65ceca3143c8b5
kubernetes-client-linux-s390x.tar.gz fd39f47b691fc608f2ea3fed35408dd4c0b1d198605ec17363b0987b123a4702
kubernetes-client-windows-386.tar.gz d9b72cfeefee0cd2db5f6a388bdb9da1e33514498f4d88be1b04282db5bfbd3d
kubernetes-client-windows-amd64.tar.gz c536952bd29a7ae12c8fa148d592cc3c353dea4d0079e8497edaf8a759a16006

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 175fc9360d4f26b5f60b467798d851061f01d0ca555c254ef44a8a9822cf7560
kubernetes-server-linux-arm64.tar.gz f1e039e0e2923d1ea02fd76453aa51715ca83c5c26ca1a761ace2c717b79154f
kubernetes-server-linux-arm.tar.gz 48dc95e5230d7a44b64b379f9cf2e1ec72b7c4c7c62f4f3e92a73076ad6376db
kubernetes-server-linux-ppc64le.tar.gz dc079cd18333c201cfd0f5b0e93e602d020a9e665d8c13968170a2cd89eebeb4
kubernetes-server-linux-s390x.tar.gz fe6674e7d69aeffd522e543e957897e2cb943e82d5ccd368ccb9009e1128273f

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz 6c6cece62bad5bfeaf4a4b14e93c9ba99c96dc82b7855a2214cdf37a65251de8
kubernetes-node-linux-arm64.tar.gz dd75dc044fb1f337b60cb4b27c9bbdca4742d8bc0a1d03d13553a1b8fc593e98
kubernetes-node-linux-arm.tar.gz c5d832c93c24d77414a880d8b7c4fac9a7443305e8e5c704f637ff023ff56f94
kubernetes-node-linux-ppc64le.tar.gz 649813a257353c5b85605869e33aeeb0c070e64e6fee18bc9c6e70472aa05677
kubernetes-node-linux-s390x.tar.gz 5ca0a7e9e90b2de7aff7bbdc84f662140ce847ea46cdb78802ce75459e0cc043
kubernetes-node-windows-amd64.tar.gz 4b84b0025aff1d4406f3e5cd5fa86940f594e3ec6e1d12d3ce1eea5f5b3fc55d

Major Themes

Kubernetes 1.7 is a milestone release that adds security, stateful application, and extensibility features motivated by widespread production use of Kubernetes.

Security enhancements in this release include encrypted secrets (alpha), network policy for pod-to-pod communication, the node authorizer to limit Kubelet access to API resources, and Kubelet client / server TLS certificate rotation (alpha).

Major features for stateful applications include automated updates to StatefulSets, enhanced updates for DaemonSets, a burst mode for faster StatefulSets scaling, and (alpha) support for local storage.

Extensibility features include API aggregation (beta), CustomResourceDefinitions (beta) in favor of ThirdPartyResources, support for extensible admission controllers (alpha), pluggable cloud providers (alpha), and container runtime interface (CRI) enhancements.

Action Required Before Upgrading

Network

  • NetworkPolicy has been promoted from extensions/v1beta1 to the new networking.k8s.io/v1 API group. The structure remains unchanged from the v1beta1 API. The net.beta.kubernetes.io/network-policy annotation on Namespaces (used to opt in to isolation) has been removed. Instead, isolation is now determined on a per-pod basis. A NetworkPolicy may target a pod for isolation by including the pod in its spec.podSelector. Targeted Pods accept the traffic specified in the respective NetworkPolicy (and nothing else). Pods not targeted by any NetworkPolicy accept all traffic by default. (#39164, @danwinship)

    Action Required: When upgrading to Kubernetes 1.7 (and a network plugin that supports the new NetworkPolicy v1 semantics), you should consider the following.

    The v1beta1 API used an annotation on Namespaces to activate the DefaultDeny policy for an entire Namespace. To activate default deny in the v1 API, you can create a NetworkPolicy that matches all Pods but does not allow any traffic:

    kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: default-deny
spec:
  podSelector:

    This will ensure that Pods that aren't matched by any other NetworkPolicy will continue to be fully-isolated, as they were in v1beta1.

    In Namespaces that previously did not have the "DefaultDeny" annotation, you should delete any existing NetworkPolicy objects. These had no effect in the v1beta1 API, but with v1 semantics they might cause some traffic to be unintentionally blocked.

Storage

  • Alpha volume provisioning is removed and default storage class should be used instead. (#44090, @NickrenREN)

  • Portworx volume driver no longer has to run on the master. (#45518, @harsh-px)

  • Default behavior in Cinder storageclass is changed. If availability is not specified, the zone is chosen by algorithm. It makes possible to spread stateful pods across many zones. (#44798, @zetaab)

  • PodSpecs containing parent directory references such as .. (for example, ../bar) in hostPath volume path or in volumeMount subpaths must be changed to the simple absolute path. Backsteps .. are no longer allowed.(#47290, @jhorwit2).

API Machinery

  • The Namespace API object no longer supports the deletecollection operation. (#46407, @liggitt)

  • The following alpha API groups were unintentionally enabled by default in previous releases, and will no longer be enabled by default in v1.8: (#47690, @caesarxuchao)

    • rbac.authorization.k8s.io/v1alpha1

    • settings.k8s.io/v1alpha1

    • If you wish to continue using them in v1.8, please enable them explicitly using the --runtime-config flag on the apiserver (for example, --runtime-config="rbac.authorization.k8s.io/v1alpha1,settings.k8s.io/v1alpha1")

  • cluster/update-storage-objects.sh now supports updating StorageClasses in etcd to storage.k8s.io/v1. You must do this prior to upgrading to 1.8. (#46116, @ncdc)

Controller Manager

  • kube-controller-manager has dropped support for the --insecure-experimental-approve-all-kubelet-csrs-for-group flag. It is accepted in 1.7, but ignored. Instead, the csrapproving controller uses authorization checks to determine whether to approve certificate signing requests: (#45619, @mikedanese)

    • Before upgrading, users must ensure their controller manager will enable the csrapproving controller, create an RBAC ClusterRole and ClusterRoleBinding to approve CSRs for the same group, then upgrade. Example roles to enable the equivalent behavior can be found in the TLS bootstrapping documentation.

kubectl (CLI)

  • kubectl create role and kubectl create clusterrole invocations must be updated to specify multiple resource names as repeated --resource-name arguments instead of comma-separated arguments to a single --resource-name argument. E.g. --resource-name=x,y must become --resource-name x --resource-name y (#44950, @xilabao)

  • kubectl create rolebinding and kubectl create clusterrolebinding invocations must be updated to specify multiple subjects as repeated --user, --group, or --serviceaccount arguments instead of comma-separated arguments to a single --user, --group, or --serviceaccount. E.g. --user=x,y must become --user x --user y (#43903, @xilabao)

kubeadm

  • kubeadm: Modifications to cluster-internal resources installed by kubeadm will be overwritten when upgrading from v1.6 to v1.7. (#47081, @luxas)

  • kubeadm deb/rpm packages: cAdvisor doesn't listen on 0.0.0.0:4194 without authentication/authorization because of the possible information leakage. The cAdvisor API can still be accessed via https://{node-ip}:10250/stats/, though. (kubernetes/release#356, @luxas)

Cloud Providers

  • Azure: Container permissions for provisioned volumes have changed to private. If you have existing Azure volumes that were created by Kubernetes v1.6.0-v1.6.5, you should change the permissions on them manually. (#47605, @brendandburns)

  • GKE/GCE: New and upgraded 1.7 GCE/GKE clusters no longer have an RBAC ClusterRoleBinding that grants the cluster-admin ClusterRole to the default service account in the kube-system Namespace. (#46750, @cjcullen). If this permission is still desired, run the following command to explicitly grant it, either before or after upgrading to 1.7:

    kubectl create clusterrolebinding kube-system-default --serviceaccount=kube-system:default --clusterrole=cluster-admin

Known Issues

Populated via v1.7.x known issues / FAQ accumulator

  • The kube-apiserver discovery APIs (for example, /apis) return information about the API groups being served, and can change dynamically. During server startup, prior to the server reporting healthy (via /healthz), not all API groups may be reported. Wait for the server to report healthy (via /healthz) before depending on the information provided by the discovery APIs. Additionally, since the information returned from the discovery APIs may change dynamically, a cache of the results should not be considered authoritative. ETag support is planned in a future version to facilitate client caching. (#47977, #44957)

  • The DaemonSet controller will evict running Pods that do not tolerate the NoSchedule taint if the taint is added to a Node. There is an open PR (#48189) to resolve this issue, but as this issue also exists in 1.6, and as we do not wish to risk release stability by merging it directly prior to a release without sufficient testing, we have decided to defer merging the PR until the next point release for each minor version (#48190).

  • Protobuf serialization does not distinguish between [] and null. API fields previously capable of storing and returning either [] and null via JSON API requests (for example, the Endpoints subsets field) can now store only null when created using the protobuf content-type or stored in etcd using protobuf serialization (the default in 1.6). JSON API clients should tolerate null values for such fields, and treat null and [] as equivalent in meaning unless specifically documented otherwise for a particular field. (#44593)

  • Local volume source paths that are directories and not mount points fail to unmount. A fix is in process (#48331).

Deprecations

Cluster provisioning scripts

Client libraries

  • Swagger 1.2 spec (/swaggerapi/*) is deprecated. Please use OpenAPI instead.

DaemonSet

  • DaemonSet’s spec.templateGeneration has been deprecated. (#45924, @janetkuo)

kube-proxy

  • In 1.7, the kube-proxy component has been converted to use a configuration file. The old flags still work in 1.7, but they are being deprecated and will be removed in a future release. Cluster administrators are advised to switch to using the configuration file, but no action is strictly necessary in 1.7. (#34727, @ncdc)

Namespace

  • The Namespace API object no longer supports the deletecollection operation. (#46407, @liggitt)

Scheduling

  • If you are using AffinityInAnnotations=true in --feature-gates, then the 1.7 release is your last opportunity to convert from specifying affinity/anti-affinity using the scheduler.alpha.kubernetes.io/affinity annotation on Pods, to using the Affinity field of PodSpec. Support for the alpha version of node and pod affinity (which uses the scheduler.alpha.kubernetes.io/affinity annotations on Pods) is going away in Kubernetes 1.8 (not this release, but the next release). If you have not enabled AffinityInAnnotations=true in --feature-gates, then this change does not affect you.

Notable Features

Features for this release were tracked via the use of the kubernetes/features issues repo. Each Feature issue is owned by a Special Interest Group from kubernetes/community

Kubefed

  • Deprecate the --secret-name flag from kubefed join, instead generating the secret name arbitrarily. (#42513, @perotinus)

Kubernetes API

User Provided Extensions

Application Deployment

StatefulSet

  • [beta] StatefulSet supports RollingUpdate and OnDelete update strategies.

  • [alpha] StatefulSet authors should be able to relax the ordering and parallelism policies for software that can safely support rapid, out-of-order changes.

DaemonSet

Deployments

  • [beta] Deployments uses a hashing collision avoidance mechanism that ensures new rollouts will not block on hashing collisions anymore. (kubernetes/features#287)

PodDisruptionBudget

Security

Admission Control

TLS Bootstrapping

Audit Logging

  • [alpha] Advanced Auditing enhances the Kubernetes API audit logging capabilities through a customizable policy, pluggable audit backends, and richer audit data.

Encryption at Rest

Node Authorization

  • [beta] A new Node authorization mode and NodeRestriction admission plugin, when used in combination, limit nodes' access to specific APIs, so that they may only modify their own Node API object, only modify Pod objects bound to themselves, and only retrieve secrets and configmaps referenced by pods bound to themselves. See Using Node Authorization for more information.

Application Autoscaling

Horizontal Pod Autoscaler

Cluster Lifecycle

kubeadm

Cloud Provider Support

Cluster Federation

Placement Policy

  • [alpha] The federation-apiserver now supports a SchedulingPolicy admission controller that enables policy-based control over placement of federated resources. For more information, see Set up placement policies in Federation.

Cluster Selection

Instrumentation

Core Metrics API

  • [alpha] Introduces a lightweight monitoring component for serving the core resource metrics API used by the Horizontal Pod Autoscaler and other components (kubernetes/features#271)

Internationalization

kubectl (CLI)

  • Features

    • kubectl logs supports specifying a container name when using label selectors (#44282, @derekwaynecarr)

    • kubectl rollout supports undo and history for DaemonSet (#46144, @janetkuo)

    • kubectl rollout supports status and history for StatefulSet (#46669, @kow3ns).

    • Implement kubectl get controllerrevisions (#46655, @janetkuo)

    • kubectl create clusterrole supports --non-resource-url (#45809, @CaoShuFeng)

    • kubectl logs and kubectl attach support specifying a wait timeout with --pod-running-timeout

    • (#41813, @shiywang)

    • New commands

    • Strategic Merge Patch

      • Reference docs now display the patch type and patch merge key used by kubectl apply to merge and identify unique elements in arrays.

        • kubectl edit and kubectl apply will keep the ordering of elements in merged lists (#45980, @mengqiy)

        • New patch directive (retainKeys) to specifying clearing fields missing from the request (#44597, @mengqiy)

        • Open API now includes strategic merge patch tags (previously only in go struct tags) (#44121, @mbohlool)

    • Plugins

      • Introduces the ability to extend kubectl by adding third-party plugins. Developer preview, please refer to the documentation for instructions about how to use it. (#37499, @fabianofranz)

      • Added support for a hierarchy of kubectl plugins (a tree of plugins as children of other plugins). (#45981, @fabianofranz)

      • Added exported env vars to kubectl plugins so that plugin developers have access to global flags, namespace, the plugin descriptor and the full path to the caller binary.

    • Enhancement

      • kubectl auth can-i now supports non-resource URLs (#46432, @CaoShuFeng)

      • kubectl set selector and kubectl set subject no longer print "running in local/dry-run mode..." at the top. The output can now be piped and interpretted as yaml or json (#46507, @bboreham)

      • When using an in-cluster client with an empty configuration, the --namespace flag is now honored (#46299, @ncdc)

      • The help message for missingResourceError is now generic (#45582, @CaoShuFeng)

      • kubectl taint node now supports label selectors (#44740, @ravisantoshgudimetla)

      • kubectl proxy --www now logs a warning when the dir is invalid (#44952, @CaoShuFeng)

      • kubectl taint output has been enhanced with the operation (#43171, @ravisantoshgudimetla)

      • kubectl --user and --cluster now support completion (#44251, @superbrothers)

      • kubectl config use-context now supports completion (#42336, @superbrothers)

      • kubectl version now supports --output (#39858, @alejandroEsc)

        • kubectl create configmap has a new option --from-env-file that populates a configmap from file which follows a key=val format for each line. (#38882, @fraenkel)

        • kubectl create secret has a new option --from-env-file that populates a configmap from file which follows a key=val format for each line.

    • Printing/describe

      • Print conditions of RC/RS in kubectl describe command. (#44710, @xiangpengzhao)

      • Improved output on kubectl get and kubectl describe for generic objects. (#44222, @fabianofranz)

      • In kubectl describe, find controllers with ControllerRef, instead of showing the original creator. (#42849, @janetkuo)

        • kubectl version has new flag --output (=json or yaml) allowing result of the command to be parsed in either json format or yaml. (#39858, @alejandroEsc)

    • Bug fixes

      • Fix some false negatives in detection of meaningful conflicts during strategic merge patch with maps and lists. (#43469, @enisoc)

        • Fix false positive "meaningful conflict" detection for strategic merge patch with integer values. (#44788, @enisoc)

        • Restored the ability of kubectl running inside a pod to consume resource files specifying a different namespace than the one the pod is running in. (#44862, @liggitt)

      • Kubectl commands run inside a pod using a kubeconfig file now use the namespace specified in the kubeconfig file, instead of using the pod namespace. If no kubeconfig file is used, or the kubeconfig does not specify a namespace, the pod namespace is still used as a fallback. (#44570, @liggitt)

      • Fixed kubectl cluster-info dump to support multi-container pod. (#44088, @xingzhou)

      • Kubectl will print a warning when deleting the current context (#42538, @adohe)

      • Fix VolumeClaims/capacity in kubectl describe statefulsets output. (#47573, @k82cn)

Networking

Network Policy

Load Balancing

  • [stable] Source IP Preservation - change Cloud load-balancer strategy to health-checks and respond to health check only on nodes that host pods for the service. See Create an External Load Balancer - Preserving the client source IP. Two annotations have been promoted to API fields:

    • Service.Spec.ExternalTrafficPolicy was 'service.beta.kubernetes.io/external-traffic' annotation.

    • Service.Spec.HealthCheckNodePort was 'service.beta.kubernetes.io/healthcheck-nodeport' annotation.

Node Components

Container Runtime Interface

Scheduling

Scheduler Extender

Storage

Local Storage

  • [alpha] This feature adds capacity isolation support for local storage at node, container, and volume levels. See updated Reserve Compute Resources for System Daemons documentation.

  • [alpha] Make locally attached (non-network attached) storage available as a persistent volume source. For more information, see Storage Volumes - local.

Volume Plugins

Metrics

Other notable changes

Admission plugin

  • OwnerReferencesPermissionEnforcement admission plugin ignores pods/status. (#45747, @derekwaynecarr)

  • Ignored mirror pods in PodPreset admission plugin. (#45958, @k82cn)

API Machinery

  • The protobuf serialization of API objects has been updated to store maps in a predictable order to ensure that the representation of that object does not change when saved into etcd. This prevents the same object from being seen as being modified, even when no values have changed. (#47701, @smarterclayton)

  • API resource discovery now includes the singularName used to refer to the resource. (#43312, @deads2k)

  • Enhance the garbage collection admission plugin so that a user who doesn't have delete permission of the owning object cannot modify the blockOwnerDeletion field of existing ownerReferences, or add new ownerReferences with blockOwnerDeletion=true (#43876, @caesarxuchao)

  • Exec and portforward actions over SPDY now properly handle redirects sent by the Kubelet (#44451, @ncdc)

  • The proxy subresource APIs for nodes, services, and pods now support the HTTP PATCH method. (#44929, @liggitt)

  • The Categories []string field on discovered API resources represents the list of group aliases (e.g. "all") that each resource belongs to. (#43338, @fabianofranz)

  • [alpha] The Kubernetes API supports retrieving tabular output for API resources via a new mime-type application/json;as=Table;v=v1alpha1;g=meta.k8s.io. The returned object (if the server supports it) will be of type meta.k8s.io/v1alpha1 with Table, and contain column and row information related to the resource. Each row will contain information about the resource - by default it will be the object metadata, but callers can add the ?includeObject=Object query parameter and receive the full object. In the future kubectl will use this to retrieve the results of kubectl get. (#40848, @smarterclayton)

  • The behavior of some watch calls to the server when filtering on fields was incorrect. If watching objects with a filter, when an update was made that no longer matched the filter a DELETE event was correctly sent. However, the object that was returned by that delete was not the (correct) version before the update, but instead, the newer version. That meant the new object was not matched by the filter. This was a regression from behavior between cached watches on the server side and uncached watches, and thus broke downstream API clients. (#46223, @smarterclayton)

  • OpenAPI spec is now available in protobuf binary and gzip format (with ETag support) (#45836, @mbohlool)

  • Updating apiserver to return UID of the deleted resource. Clients can use this UID to verify that the resource was deleted or waiting for finalizers. (#45600, @nikhiljindal)

  • Fix incorrect conflict errors applying strategic merge patches to resources. (#43871, @liggitt)

  • Fix init container status reporting when active deadline is exceeded. (#46305, @sjenning)

  • Moved qos to api.helpers. (#44906, @k82cn)

  • Fix issue with the resource quota controller causing add quota to be resynced at the wrong (#45685, @derekwaynecarr)

  • Added Group/Version/Kind and Action extension to OpenAPI Operations (#44787, @mbohlool)

  • Make clear that meta.KindToResource is only a guess (#45272, @sttts)

  • Add APIService conditions (#43301, @deads2k)

  • Create and push a docker image for the cloud-controller-manager (#45154, @luxas)

  • Deprecated Binding objects in 1.7. (#47041, @k82cn)

  • Adds the Categories []string field to API resources, which represents the list of group aliases (e.g. "all") that every resource belongs to. (#43338, @fabianofranz)

  • --service-account-lookup now defaults to true, requiring the Secret API object containing the token to exist in order for a service account token to be valid. This enables service account tokens to be revoked by deleting the Secret object containing the token. (#44071, @liggitt)

  • API Registration is now in beta. (#45247, @mbohlool)

  • The Kubernetes API server now exits if it encounters a networking failure (e.g. the networking interface hosting its address goes away) to allow a process manager (systemd/kubelet/etc) to react to the problem. Previously the server would log the failure and try again to bind to its configured address:port. (#42272, @marun)

  • The Prometheus metrics for the kube-apiserver for tracking incoming API requests and latencies now return the subresource label for correctly attributing the type of API call. (#46354, @smarterclayton)

  • kube-apiserver now drops unneeded path information if an older version of Windows kubectl sends it. (#44421, @mml)

Application autoscaling

  • Make "upscale forbidden window" and "downscale forbidden window" duration configurable in arguments of kube-controller-manager. (#42101, @Dmitry1987)

Application Deployment

  • StatefulSetStatus now tracks replicas, readyReplicas, currentReplicas, and updatedReplicas. The semantics of replicas is now consistent with DaemonSet and ReplicaSet, and readyReplicas has the semantics that replicas did prior to 1.7 (#46669, @kow3ns).

  • ControllerRevision type has been added for StatefulSet and DaemonSet history. Clients should not depend on the stability of this type as it may change, as necessary, in future releases to support StatefulSet and DaemonSet update and rollback. We enable this type as we do with beta features, because StatefulSet update and DaemonSet update are enabled. (#45867, @kow3ns)

  • PodDisruptionBudget now uses ControllerRef to decide which controller owns a given Pod, so it doesn't get confused by controllers with overlapping selectors. (#45003, @krmayankk)

  • Deployments are updated to use (1) a more stable hashing algorithm (fnv) than the previous one (adler) and (2) a hashing collision avoidance mechanism that will ensure new rollouts will not block on hashing collisions anymore. (#44774, @kargakis)(kubernetes/features#287)

  • Deployments and DaemonSets rollouts are considered complete when all of the desired replicas are updated and available. This change affects kubectl rollout status and Deployment condition. (#44672, @kargakis)

  • Job controller now respects ControllerRef to avoid fighting over Pods. (#42176, @enisoc)

  • CronJob controller now respects ControllerRef to avoid fighting with other controllers. (#42177, @enisoc)

Cluster Autoscaling

  • Cluster Autoscaler 0.6. More information available here.

  • cluster-autoscaler: Fix duplicate writing of logs. (#45017, @MaciekPytel)

Cloud Provider Enhancement

  • AWS:

    • New 'service.beta.kubernetes.io/aws-load-balancer-extra-security-groups' Service annotation to specify extra Security Groups to be added to ELB created by AWS cloudprovider (#45268, @redbaron)

    • Clean up blackhole routes when using kubenet (#47572, @justinsb)

    • Maintain a cache of all instances, to fix problem with > 200 nodes with ELBs (#47410, @justinsb)

    • Avoid spurious ELB listener recreation - ignore case when matching protocol (#47391, @justinsb)

    • Allow configuration of a single security group for ELBs (#45500, @nbutton23)

    • Remove check that forces loadBalancerSourceRanges to be 0.0.0.0/0. (#38636, @dhawal55)

      • Allow setting KubernetesClusterID or KubernetesClusterTag in combination with VPC. (#42512, @scheeles)

      • Start recording cloud provider metrics for AWS (#43477, @gnufied)

      • AWS: Batch DescribeInstance calls with nodeNames to 150 limit, to stay within AWS filter limits. (#47516, @gnufied)

      • AWS: Process disk attachments even with duplicate NodeNames (#47406, @justinsb)

    • Allow configuration of a single security group for ELBs (#45500, @nbutton23)

    • Fix support running the master with a different AWS account or even on a different cloud provider than the nodes. (#44235, @mrIncompetent)

    • Support node port health check (#43585, @foolusion)

    • Support for ELB tagging by users (#45932, @lpabon)

  • Azure:

  • GCP:

    • Bump GLBC version to 0.9.5 - fixes loss of manually modified GCLB health check settings upon upgrade from pre-1.6.4 to either 1.6.4 or 1.6.5. (#47567, @nicksardo)

    • [beta] Support creation of GCP Internal Load Balancers from Service objects (#46663, @nicksardo)

    • GCE installs will now avoid IP masquerade for all RFC-1918 IP blocks, rather than just 10.0.0.0/8. This means that clusters can be created in 192.168.0.0./16 and 172.16.0.0/12 while preserving the container IPs (which would be lost before). (#46473, @thockin)

    • The Calico version included in kube-up for GCE has been updated to v2.2. (#38169, @caseydavenport)

    • Add ip-masq-agent addon to the addons folder which is used in GCE if --non-masquerade-cidr is set to 0/0 (#46038, @dnardo)

    • Enable kubelet csr bootstrap in GCE/GKE (#40760, @mikedanese)

    • Adds support for allocation of pod IPs via IP aliases. (#42147, @bowei)

      • gce kube-up: The Node authorization mode and NodeRestriction admission controller are now enabled (#46796, @mikedanese)

      • Tokens retrieved from Google Cloud with application default credentials will not be cached if the client fails authorization (#46694, @matt-tyler)

      • Add metrics to all major gce operations {latency, errors} (#44510, @bowei)

        • The new metrics are:

        • cloudprovider_gce_api_request_duration_seconds{request, region, zone}

        • cloudprovider_gce_api_request_errors{request, region, zone}

        • request is the specific function that is used.

        • region is the target region (Will be "<n/a>" if not applicable)

        • zone is the target zone (Will be "<n/a>" if not applicable)

        • Note: this fixes some issues with the previous implementation of metrics for disks:

          • Time duration tracked was of the initial API call, not the entire operation.

          • Metrics label tuple would have resulted in many independent histograms stored, one for each disk. (Did not aggregate well).

    • Fluentd now tolerates all NoExecute Taints when run in gcp configuration. (#45715, @gmarek)

      • Taints support in gce/salt startup scripts. (#47632, @mwielgus)

      • GCE installs will now avoid IP masquerade for all RFC-1918 IP blocks, rather than just 10.0.0.0/8. This means that clusters can (#46473, @thockin) be created in 192.168.0.0./16 and 172.16.0.0/12 while preserving the container IPs (which would be lost before).

      • Support running Ubuntu image on GCE node (#44744, @yguo0905)

    • The gce metadata server can now be hidden behind a proxy, hiding the kubelet's token. (#45565, @Q-Lee)

  • OpenStack:

  • vSphere:

Cluster Provisioning

  • Juju:

    • Add Kubernetes 1.6 support to Juju charms (#44500, @Cynerva)

      • Add metric collection to charms for autoscaling

      • Update kubernetes-e2e charm to fail when test suite fails

      • Update Juju charms to use snaps

      • Add registry action to the kubernetes-worker charm

      • Add support for kube-proxy cluster-cidr option to kubernetes-worker charm

      • Fix kubernetes-master charm starting services before TLS certs are saved

      • Fix kubernetes-worker charm failures in LXD

      • Fix stop hook failure on kubernetes-worker charm

      • Fix handling of juju kubernetes-worker.restart-needed state

      • Fix nagios checks in charms

    • Enable GPU mode if GPU hardware detected (#43467, @tvansteenburgh)

    • Fix ceph-secret type to kubernetes.io/rbd in kubernetes-master charm (#44635, @Cynerva)

    • Disallows installation of upstream docker from PPA in the Juju kubernetes-worker charm. (#44681, @wwwtyro)

    • Resolves juju vsphere hostname bug showing only a single node in a scaled node-pool. (#44780, @chuckbutler)

    • Fixes a bug in the kubernetes-worker Juju charm code that attempted to give kube-proxy more than one api endpoint. (#44677, @wwwtyro)

    • Added CIFS PV support for Juju Charms (#45117, @chuckbutler)

    • Fixes juju kubernetes master: 1. Get certs from a dead leader. 2. Append tokens. (#43620, @ktsakalozos)

    • kubernetes-master juju charm properly detects etcd-scale events and reconfigures appropriately. (#44967, @chuckbutler)

      • Use correct option name in the kubernetes-worker layer registry action (#44921, @jacekn)

      • Send dns details only after cdk-addons are configured (#44945, @ktsakalozos)

      • Added support to the pause action in the kubernetes-worker charm for new flag --delete-local-data (#44931, @chuckbutler)

      • Add namespace-{list, create, delete} actions to the kubernetes-master layer (#44277, @jacekn)

      • Using http2 in kubeapi-load-balancer to fix kubectl exec uses (#43625, @mbruzek)

    • Don't append :443 to registry domain in the kubernetes-worker layer registry action (#45550, @jacekn)

  • kubeadm

    • Enable the Node Authorizer/Admission plugin in v1.7 (#46879, @luxas)

    • Users can now pass extra parameters to etcd in a kubeadm cluster (#42246, @jamiehannaford)

    • Make kubeadm use the new CSR approver in v1.7 (#46864, @luxas)

    • Allow enabling multiple authorization modes at the same time (#42557, @xilabao)

    • add proxy client-certs to kube-apiserver to allow it to proxy aggregated api servers (#43715, @deads2k)* CentOS provider

  • hyperkube

    • The hyperkube image has been slimmed down and no longer includes addon manifests and other various scripts. These were introduced for the now removed docker-multinode setup system. (#44555, @luxas)

  • Support secure etcd cluster for centos provider. (#42994, @Shawyeok)

  • Update to kube-addon-manager:v6.4-beta.2: kubectl v1.6.4 and refreshed base images (#47389, @ixdy)

  • Remove Initializers from admission-control in kubernetes-master charm for pre-1.7 (#46987, @Cynerva)

  • Added state guards to the idle_status messaging in the kubernetes-master charm to make deployment faster on initial deployment. (#47183, @chuckbutler)

Cluster federation

  • Features:

    • Adds annotations to all Federation objects created by kubefed. (#42683, @perotinus)

      • Mechanism of adding federation domain maps to kube-dns deployment via --federations flag is superseded by adding/updating federations key in kube-system/kube-dns configmap. If user is using kubefed tool to join cluster federation, adding federation domain maps to kube-dns is already taken care by kubefed join and does not need further action.

      • Prints out status updates when running kubefed init (#41849, @perotinus)

      • kubefed init now supports overriding the default etcd image name with the --etcd-image parameter. (#46247, @marun)

      • kubefed will now configure NodeInternalIP as the federation API server endpoint when NodeExternalIP is unavailable for federation API servers exposed as NodePort services (#46960, @lukaszo)

      • Automate configuring nameserver in cluster-dns for CoreDNS provider (#42895, @shashidharatd)

      • A new controller for managing DNS records is introduced which can be optionally disabled to enable third party components to manage DNS records for federated services. (#45034, @shashidharatd)

    • Remove the --secret-name flag from kubefed join, instead generating the secret name arbitrarily. (#42513, @perotinus)

    • Use StorageClassName for etcd pvc (#46323, @marun)

  • Bug fixes:

    • Allow disabling federation controllers through override args (#44209, @irfanurrehman)

    • Kubefed: Use service accounts instead of the user's credentials when accessing joined clusters' API servers. (#42042, @perotinus)

    • Avoid panic if route53 fields are nil (#44380, @justinsb)

Credential provider

Information for Kubernetes clients (openapi, swagger, client-go)

  • Features:

    • Add Host field to TCPSocketAction (#42902, @louyihua)

      • Add the ability to lock on ConfigMaps to support HA for self hosted components (#42666, @timothysc)

      • validateClusterInfo: use clientcmdapi.NewCluster() (#44221, @ncdc)

      • OpenAPI spec is now available in protobuf binary and gzip format (with ETag support) (#45836, @mbohlool)

      • HostAliases is now parsed with hostAliases json keys to be in line with the feature's name. (#47512, @rickypai)

      • Add redirect support to SpdyRoundTripper (#44451, @ncdc)

      • Duplicate recurring Events now include the latest event's Message string (#46034, @kensimon)

  • Bug fixes:

    • Fix serialization of EnforceNodeAllocatable (#44606, @ivan4th)

      • Use OS-specific libs when computing client User-Agent in kubectl, etc. (#44423, @monopole)

Instrumentation

Internal storage layer

  • prevent pods/status from touching ownerreferences (#45826, @deads2k)

  • Ensure that autoscaling/v1 is the preferred version for API discovery when autoscaling/v2alpha1 is enabled. (#45741, @DirectXMan12)

  • The proxy subresource APIs for nodes, services, and pods now support the HTTP PATCH method. (#44929, @liggitt)

  • Fluentd now tolerates all NoExecute Taints when run in gcp configuration. (#45715, @gmarek)

Kubernetes Dashboard

kube-dns

  • Updates kube-dns to 1.14.2 (#45684, @bowei)

    • Support kube-master-url flag without kubeconfig

    • Fix concurrent R/Ws in dns.go

    • Fix confusing logging when initialize server

    • Fix printf in cmd/kube-dns/app/server.go

    • Fix version on startup and --version flag

    • Support specifying port number for nameserver in stubDomains

kube-proxy

  • Features:

    • ratelimit runs of iptables by sync-period flags (#46266, @thockin)

    • Log warning when invalid dir passed to kubectl proxy --www (#44952, @CaoShuFeng)

    • Add --write-config-to flag to kube-proxy to allow users to write the default configuration settings to a file. (#45908, @ncdc)

      • When switching from the service.beta.kubernetes.io/external-traffic annotation to the new (#46716, @thockin) externalTrafficPolicy field, the values chnag as follows: * "OnlyLocal" becomes "Local" * "Global" becomes "Cluster".

  • Bug fixes:

kube-scheduler

  • Scheduler can receive its policy configuration from a ConfigMap (#43892, @bsalamat)

  • Aggregated used ports at the NodeInfo level for PodFitsHostPorts predicate. (#42524, @k82cn)

  • leader election lock based on scheduler name (#42961, @wanghaoran1988)

Storage

  • Features

    • The options passed to a Flexvolume plugin's mount command now contains the pod name (kubernetes.io/pod.name), namespace (kubernetes.io/pod.namespace), uid (kubernetes.io/pod.uid), and service account name (kubernetes.io/serviceAccount.name). (#39488, @liggitt)

    • GCE and AWS dynamic provisioners extension: admins can configure zone(s) in which a persistent volume shall be created. (#38505, @pospispa)

    • Implement API usage metrics for GCE storage. (#40338, @gnufied)

    • Add support for emitting metrics from openstack cloudprovider about storage operations. (#46008, @NickrenREN)

    • vSphere cloud provider: vSphere storage policy support for dynamic volume provisioning. (#46176, @BaluDontu)

    • Support StorageClass in Azure file volume (#42170, @rootfs)

    • Start recording cloud provider metrics for AWS (#43477, @gnufied)

    • Support iSCSI CHAP authentication (#43396, @rootfs)

    • Openstack cinder v1/v2/auto API support (#40423, @mkutsevol](https://github.com/kubernetes/kubernetes/pull/41498), @mikebryant)

    • Alpha feature: allows users to set storage limit to isolate EmptyDir volumes. It enforces the limit by evicting pods that exceed their storage limits (#45686, @jingxu97)

  • Bug fixes

    • Fixes issue with Flexvolume, introduced in 1.6.0, where drivers without an attacher would fail (node indefinitely waiting for attach). A driver API addition is introduced: drivers that don't implement attach should return attach: false on init. (#47503, @chakri-nelluri)

    • Fix dynamic provisioning of PVs with inaccurate AccessModes by refusing to provision when PVCs ask for AccessModes that can't be satisfied by the PVs' underlying volume plugin. (#47274, @wongma7)

    • Fix pods failing to start if they specify a file as a volume subPath to mount. (#45623, @wongma7)

    • Fix erroneous FailedSync and FailedMount events being periodically and indefinitely posted on Pods after kubelet is restarted. (#44781, @wongma7)

    • Fix AWS EBS volumes not getting detached from node if routine to verify volumes are attached runs while the node is down (#46463, @wongma7)

    • Improves performance of Cinder volume attach/detach operations. (#41785, @jamiehannaford)

    • Fix iSCSI iSER mounting. (#47281, @mtanino)

    • iscsi storage plugin: Fix dangling session when using multiple target portal addresses. (#46239, @mtanino)

    • Fix log spam due to unnecessary status update when node is deleted. (#45923, @verult)

    • Don't try to attach volume to new node if it is already attached to another node and the volume does not support multi-attach. (#45346, @codablock)

    • detach the volume when pod is terminated (#45286, @gnufied)

    • Roll up volume error messages in the kubelet sync loop. (#44938, @jayunit100)

    • Catch error when failed to make directory in NFS volume plugin (#38801, @nak3)

Networking

  • DNS and name resolution

    • Updates kube-dns to 1.14.2 (#45684, @bowei)

      • Support kube-master-url flag without kubeconfig

      • Fix concurrent R/Ws in dns.go

      • Fix confusing logging when initializing server

      • Support specifying port number for nameserver in stubDomains

    • A new field hostAliases has been added to pod.spec to support adding entries to a Pod's /etc/hosts file. (#44641, @rickypai)

    • Fix DNS suffix search list support in Windows kube-proxy. (#45642, @JiangtianLi)

  • Kube-proxy

    • ratelimit runs of iptables by sync-period flags (#46266, @thockin)

    • Fix corner-case with OnlyLocal Service healthchecks. (#44313, @thockin)

  • Exclude nodes labeled as master from LoadBalancer / NodePort; restores documented behaviour. (#44745, @justinsb)

  • Adds support for CNI ConfigLists, which permit plugin chaining. (#42202, @squeed)

  • Fix node selection logic on initial LB creation (#45773, @justinsb)

  • When switching from the service.beta.kubernetes.io/external-traffic annotation to the new externalTrafficPolicy field, the values change as follows: * "OnlyLocal" becomes "Local" * "Global" becomes "Cluster". (#46716, @thockin)

  • servicecontroller: Fix node selection logic on initial LB creation (#45773, @justinsb)

  • fixed HostAlias in PodSpec to allow foo.bar hostnames instead of just foo DNS labels. (#46809, @rickypai)

Node controller

Node Components

  • Features

    • Removes the deprecated kubelet flag --babysit-daemons (#44230, @mtaufen)

    • make dockershim.sock configurable (#43914, @ncdc)

    • Support running Ubuntu image on GCE node (#44744, @yguo0905)

    • Kubernetes now shares a single PID namespace among all containers in a pod when running with docker >= 1.13.1. This means processes can now signal processes in other containers in a pod, but it also means that the kubectl exec {pod} kill 1 pattern will cause the Pod to be restarted rather than a single container. (#45236, @verb)

    • A new field hostAliases has been added to the pod spec to support adding entries to a Pod's /etc/hosts file. (#44641, @rickypai)

    • With --feature-gates=RotateKubeletClientCertificate=true set, the Kubelet will (#41912, @jcbsmpsn)

      • request a client certificate from the API server during the boot cycle and pause

      • waiting for the request to be satisfied. It will continually refresh the certificate

    • Create clusters with GPUs in GCE by specifying type=<gpu-type>,count=<gpu-count> to NODE_ACCELERATORS environment variable. (#45130, @vishh)

    • Disk Pressure triggers the deletion of terminated containers on the node. (#45896, @dashpole)

    • Support status.hostIP in downward API (#42717, @andrewsykim)

    • Upgrade Node Problem Detector to v0.4.1. New features added:

      • Add /dev/kmsg support for kernel log parsing. (#112, @euank)

      • Add ABRT support. (#105, @juliusmilan)

      • Add a docker image corruption problem detection in the default docker monitor config. (#117, @ajitak)

    • Upgrade CAdvisor to v0.26.1. New features added:

      • Add Docker overlay2 storage driver support.

      • Add ZFS support.

      • Add UDP metrics (collection disabled by default).

    • Roll up volume error messages in the kubelet sync loop. (#44938, @jayunit100)

    • Allow pods to opt out of PodPreset mutation via an annotation on the pod. (#44965, @jpeeler)

    • Add generic Toleration for NoExecute Taints to NodeProblemDetector, so that NPD can be scheduled to nodes with NoExecute taints by default. (#45883, @gmarek)

    • Prevent kubelet from setting allocatable < 0 for a resource upon initial creation. (#46516, @derekwaynecarr)

  • Bug fixes

    • Changed Kubelet default image-gc-high-threshold to 85% to resolve a conflict with default settings in docker that prevented image garbage collection from resolving low disk space situations when using devicemapper storage. (#40432, @sjenning)

    • Mark all static pods on the Master node as critical to prevent preemption (#47356, @dashpole)

    • Restrict active deadline seconds max allowed value to be maximum uint32 to avoid overflow (#46640, @derekwaynecarr)

    • Fix a bug with cAdvisorPort in the KubeletConfiguration that prevented setting it to 0, which is in fact a valid option, as noted in issue #11710. (#46876, @mtaufen)

    • Fix a bug where container cannot run as root when SecurityContext.RunAsNonRoot is false. (#47009, @yujuhong)

    • Fix the Kubelet PLEG update timestamp to better reflect the health of the component when the container runtime request hangs. (#45496, @andyxning)

    • Avoid failing sync loop health check on container runtime errors (#47124, @andyxning)

    • Fix a bug where Kubelet does not ignore pod manifest files starting with dots (#45111, @dwradcliffe)

    • Fix kubelet reset liveness probe failure count across pod restart boundaries (#46371, @sjenning)

    • Fix log spam due to unnecessary status update when node is deleted. (#45923, @verult)

    • Fix kubelet event recording for selected events. (#46246, @derekwaynecarr)

    • Fix image garbage collector attempting to remove in-use images. (#46121, @Random-Liu)

    • Detach the volume when pod is terminated (#45286, @gnufied)

    • CRI: Fix StopContainer timeout (#44970, @Random-Liu)

    • CRI: Fix kubelet failing to start when using rkt. (#44569, @yujuhong)

    • CRI: kubectl logs -f now stops following when container stops, as it did pre-CRI. (#44406, @Random-Liu)

    • Fixes a bug where pods were evicted even after images are successfully deleted. (#44986, @dashpole)

    • When creating a container using envFrom, (#42083, @fraenkel

      • validate the name of the ConfigMap in a ConfigMapRef
      • validate the name of the Secret in a SecretRef

    • Fix the bug where StartedAt time is not reported for exited containers. (#45977, @yujuhong)

  • Changes/deprecations

    • Marks the Kubelet's --master-service-namespace flag deprecated (#44250, @mtaufen)

    • Remove PodSandboxStatus.Linux.Namespaces.Network from CRI since it is not used/needed. (#45166, @feiskyer)

    • Remove the --enable-cri flag. CRI is now the default, and the only way to integrate with Kubelet for the container runtimes.(#45194, @yujuhong)

    • CRI has been moved to package pkg/kubelet/apis/cri/v1alpha1/runtime as part of Kubelet API path cleanup. (#47113, @feiskyer)

Scheduling

Security

  • Features:

    • Permission to use a PodSecurityPolicy can now be granted within a single namespace by allowing the use verb on the podsecuritypolicies resource within the namespace. (#42360, @liggitt)

    • Break the 'certificatesigningrequests' controller into a 'csrapprover' controller and 'csrsigner' controller. (#45514, @mikedanese)

    • kubectl auth can-i now supports non-resource URLs (#46432, @CaoShuFeng)

    • Promote kubelet tls bootstrap to beta. Add a non-experimental flag to use it and deprecate the old flag. (#46799, @mikedanese)

    • Add the alpha.image-policy.k8s.io/failed-open=true annotation when the image policy webhook encounters an error and fails open. (#46264, @Q-Lee)

    • Add an AEAD encrypting transformer for storing secrets encrypted at rest (#41939, @smarterclayton)

    • Add secretbox and AES-CBC encryption modes to at rest encryption. AES-CBC is considered superior to AES-GCM because it is resistant to nonce-reuse attacks, and secretbox uses Poly1305 and XSalsa20. (#46916, @smarterclayton)

  • Bug fixes:

    • Make gcp auth provider not to override the Auth header if it's already exits (#45575, @wanghaoran1988)

    • The oidc client plugin has reduce round trips and fix scopes requested (#45317, @ericchiang)

    • API requests using impersonation now include the system:authenticated group in the impersonated user automatically. (#44076, @liggitt)

    • RBAC role and rolebinding auto-reconciliation is now performed only when the RBAC authorization mode is enabled. (#43813, @liggitt)

    • PodSecurityPolicy now recognizes pods that specify runAsNonRoot: false in their security context and does not overwrite the specified value (#47073, @Q-Lee)

    • Tokens retrieved from Google Cloud with application default credentials will not be cached if the client fails authorization (#46694, @matt-tyler)

    • Update kube-dns, metadata-proxy, and fluentd-gcp, event-exporter, prometheus-to-sd, and ip-masq-agent addons with new base images containing fixes for CVE-2016-4448, CVE-2016-9841, CVE-2016-9843, CVE-2017-1000366, CVE-2017-2616, and CVE-2017-9526. (#47877, @ixdy)

    • Fixed an issue mounting the wrong secret into pods as a service account token. (#44102, @ncdc)

Scalability

  • The HorizontalPodAutoscaler controller will now only send updates when it has new status information, reducing the number of writes caused by the controller. (#47078, @DirectXMan12)

External Dependency Version Information

Continuous integration builds have used the following versions of external dependencies, however, this is not a strong recommendation and users should consult an appropriate installation or upgrade guide before deciding what versions of etcd, docker or rkt to use.

  • Docker versions 1.10.3, 1.11.2, 1.12.6 have been validated

    • Docker version 1.12.6 known issues

      • overlay2 driver not fully supported

      • live-restore not fully supported

      • no shared pid namespace support

    • Docker version 1.11.2 known issues

      • Kernel crash with Aufs storage driver on Debian Jessie (#27885) which can be identified by the node problem detector

      • Leaked File descriptors (#275)

      • Additional memory overhead per container (#21737)

    • Docker 1.10.3 contains backports provided by RedHat for known issues

  • For issues with Docker 1.13.X please see the 1.13.X tracking issue

  • rkt version 1.23.0+

  • etcd version 3.0.17

  • Go version: 1.8.3. Link to announcement

    • Kubernetes can only be compiled with Go 1.8. Support for all other versions is dropped.

Previous Releases Included in v1.7.0

v1.7.0-rc.1

Documentation & Examples

Downloads for v1.7.0-rc.1

filename sha256 hash
kubernetes.tar.gz 9da0e04de83e14f87540b5b58f415b5cdb78e552e07dc35985ddb1b7f618a2f2
kubernetes-src.tar.gz f4e6cfd0d859d7880d14d1052919a9eb79c26e1cd4105330dda8b05f073cab40

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 5f161559ce91321577c09f03edf6d3416f1964056644c8725394d9c23089b052
kubernetes-client-darwin-amd64.tar.gz c54b07d2b0240e2be57ff6bf95794bf826a082a7b4e8316c9ec45e92539d6252
kubernetes-client-linux-386.tar.gz d61874a51678dee6cb1e5514e703b7070c27fb728e8b18533a5233fcca2e30fd
kubernetes-client-linux-amd64.tar.gz 4004cec39c637fa7a2e3d309d941f3e73e0a16a3511c5e46cbb2fa6bb27d89e5
kubernetes-client-linux-arm64.tar.gz 88c37ea21d7a2c464be6fee29db4f295d738028871127197253923cec00cf179
kubernetes-client-linux-arm.tar.gz 0e5e5f52fe93a78003c6cac171a6aae8cb1f2f761e325d509558df84aba57b32
kubernetes-client-linux-ppc64le.tar.gz d4586a64f239654a53faf1a6c18fc5d5c99bb95df593bf92b5e9fac0daba71e2
kubernetes-client-linux-s390x.tar.gz 728097218b051df26b90863779588517183fa4e1f55dee414aff188e4a50e7df
kubernetes-client-windows-386.tar.gz d949bd6977a707b46609ee740f3a16592e7676a6dc81ad495d9f511cb4d2cb98
kubernetes-client-windows-amd64.tar.gz b787198e3320ef4094112f44e0442f062c04ce2137c14bbec10f5df9fbb3f404

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz e5eaa8951d021621b160d41bc1350dcf64178c46a0e6e656be78a5e5b267dc5d
kubernetes-server-linux-arm64.tar.gz 08b694b46bf7b5906408a331a9ccfb9143114d414d64fcca8a6daf6ec79c282b
kubernetes-server-linux-arm.tar.gz ca980d1669e22cc3846fc2bdf77e6bdc1c49820327128db0d0388c4def77bc16
kubernetes-server-linux-ppc64le.tar.gz c656106048696bd2c4b66a3f8e348b37634abf48a9dc1f4eb941e01da9597b26
kubernetes-server-linux-s390x.tar.gz 7888ed82b33b0002a488224ffa7a93e865e1d2b01e4ccc44b8d04ff4be5fef71

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz 26c74018b048e2ec0d2df61216bda77bdf29c23f34dac6d7b8a55a56f0f95927
kubernetes-node-linux-arm64.tar.gz e5c6d38556f840067b0eea4ca862c5c79a89ff47063dccecf1c0fdc2c25a9a9b
kubernetes-node-linux-arm.tar.gz 4cf1d7843ede557bd629970d1bc21a936b76bf9138fc96224e538c5a61f6e203
kubernetes-node-linux-ppc64le.tar.gz e7a870c53af210cc00f0854e2ffad8ee06b20c4028f256d60d04f31a630291d1
kubernetes-node-linux-s390x.tar.gz 78865fe4029a39744865e0acb4dd15f6f22de8264f7c65a65df52891c3b91967
kubernetes-node-windows-amd64.tar.gz 8b632e7c79e750e7102d02120508f0394d3f11a2c36b42d2c5f96ec4f0f1f1ed

Changelog since v1.7.0-beta.2

Action Required

  • The following alpha API groups were unintentionally enabled by default in previous releases, and will no longer be enabled by default in v1.8: (#47690, @caesarxuchao)
    • rbac.authorization.k8s.io/v1alpha1
    • settings.k8s.io/v1alpha1
    • If you wish to continue using them in v1.8, please enable them explicitly using the --runtime-config flag of the apiserver (for example, --runtime-config="rbac.authorization.k8s.io/v1alpha1,settings.k8s.io/v1alpha1")
  • Paths containing backsteps (for example, "../bar") are no longer allowed in hostPath volume paths, or in volumeMount subpaths (#47290, @jhorwit2)
  • Azure: Change container permissions to private for provisioned volumes. If you have existing Azure volumes that were created by Kubernetes v1.6.0-v1.6.5, you should change the permissions on them manually. (#47605, @brendandburns)

Other notable changes

  • Update kube-dns, metadata-proxy, and fluentd-gcp, event-exporter, prometheus-to-sd, and ip-masq-agent addons with new base images containing fixes for CVE-2016-4448, CVE-2016-9841, CVE-2016-9843, CVE-2017-1000366, CVE-2017-2616, and CVE-2017-9526. (#47877, @ixdy)
  • Bump the memory request/limit for ip-masq-daemon. (#47887, @dnardo)
  • HostAliases is now parsed with hostAliases json keys to be in line with the feature's name. (#47512, @rickypai)
  • Fixes issue w/Flex volume, introduced in 1.6.0, where drivers without an attacher would fail (node indefinitely waiting for attach). Drivers that don't implement attach should return attach: false on init. (#47503, @chakri-nelluri)
  • Tokens retrieved from Google Cloud with application default credentials will not be cached if the client fails authorization (#46694, @matt-tyler)
  • ip-masq-agent is now the default for GCE (#47794, @dnardo)
  • Taints support in gce/salt startup scripts. (#47632, @mwielgus)
  • Fix VolumeClaims/capacity in "kubectl describe statefulsets" output. (#47573, @k82cn)
  • New 'service.beta.kubernetes.io/aws-load-balancer-extra-security-groups' Service annotation to specify extra Security Groups to be added to ELB created by AWS cloudprovider (#45268, @redbaron)
  • AWS: clean up blackhole routes when using kubenet (#47572, @justinsb)
  • The protobuf serialization of API objects has been updated to store maps in a predictable order to ensure that the representation of that object does not change when saved into etcd. This prevents the same object from being seen as being modified, even when no values have changed. (#47701, @smarterclayton)
  • Mark Static pods on the Master as critical (#47356, @dashpole)
  • kubectl logs with label selector supports specifying a container name (#44282, @derekwaynecarr)
  • Adds an approval work flow to the the certificate approver that will approve certificate signing requests from kubelets that meet all the criteria of kubelet server certificates. (#46884, @jcbsmpsn)
  • AWS: Maintain a cache of all instances, to fix problem with > 200 nodes with ELBs (#47410, @justinsb)
  • Bump GLBC version to 0.9.5 - fixes loss of manually modified GCLB health check settings upon upgrade from pre-1.6.4 to either 1.6.4 or 1.6.5. (#47567, @nicksardo)
  • Update cluster-proportional-autoscaler, metadata-proxy, and fluentd-gcp addons with fixes for CVE-2016-4448, CVE-2016-8859, CVE-2016-9841, CVE-2016-9843, and CVE-2017-9526. (#47545, @ixdy)
  • AWS: Batch DescribeInstance calls with nodeNames to 150 limit, to stay within AWS filter limits. (#47516, @gnufied)

v1.8.0-alpha.1

Documentation & Examples

Downloads for v1.8.0-alpha.1

filename sha256 hash
kubernetes.tar.gz 47088d4a0b79ce75a90e73b1dd7f864fc17fe5ff5cea553a072c7a277a70a104
kubernetes-src.tar.gz ec2cb19b55e24c7b9728437fb9e39a442c07b68eaea636b2f6bb340e4b9696dc

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz c2fb538ce73f0ed74bd343485cd8873efcff580e4d948ea4bf2732f1b059e463
kubernetes-client-darwin-amd64.tar.gz 01a1cb673fbb764e47edaea07c1d3fdddd99bbd7b025f9b2498f38c99d5be4b2
kubernetes-client-linux-386.tar.gz 5bebebf12fb39db8be10f9758a92ce385013d07e629741421b09da88bd9fc0f1
kubernetes-client-linux-amd64.tar.gz b02ae110b3694562b195189c3cb8eca21095153d0cb5552360053304dee425f1
kubernetes-client-linux-arm64.tar.gz e6220b9e62856ad8345cb845c1365b3f177ee22d6f9718f11a1f373d7a70fd21
kubernetes-client-linux-arm.tar.gz e35c62a3781841898c91724af136fbb35fd99cf15ca5ec947c1a4bc2f6e4a73d
kubernetes-client-linux-ppc64le.tar.gz 7b02c25a764bd367e9931006def88d3fc03cf9e846cce2e77cfbc95f0e206433
kubernetes-client-linux-s390x.tar.gz ab6ba1bf43dd28c776a8cc5cae44413c45a7405f2996c277aba5ee3f6f73e305
kubernetes-client-windows-386.tar.gz eb1516db15807111ef03547b0104dcb89a310481ef8f867a65f3c57f20f56e30
kubernetes-client-windows-amd64.tar.gz 525e599a2846fe166a5f1eb14483edee9d6b866aa096e16896f6544afad31768

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz bb0a37bb1fefa735ec1eb651fec60c22b180c9bca1bd5e0317e1bcdbf4aa0819
kubernetes-server-linux-arm64.tar.gz 68fd804bd1f4d944a25112a67ef8b1cbae55051b110134850715b6f51f93f40c
kubernetes-server-linux-arm.tar.gz 822161bee3e8b3b64bb7cea297264729b3cc6d6a008c86f16b4aef16cde5b0de
kubernetes-server-linux-ppc64le.tar.gz 9354336df2694427e3d6bc9b0b1fe286f3f9a7f6ef8f239bd6319b4af1c02162
kubernetes-server-linux-s390x.tar.gz d4a87e3713f190a4cc7db1f43a6105c3c95e1eb8de45ae269b9bd1ecd52296ce

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz dc7c5865041008fcfdad050380fb33c23a361f7a1f4fbce78b164e2906a1b7f9
kubernetes-node-linux-arm64.tar.gz d572cec5ec679e5543e9ee5e2529a51bb8d5ca5f3773e4218c5491a0bd77b7a4
kubernetes-node-linux-arm.tar.gz 4b0fae35ed01ca66fb0f82ea2ea7f804378f592d0c15425dc3934f4b7b6f19a8
kubernetes-node-linux-ppc64le.tar.gz d5684a2d1a640e7b0fdf82a3faa0edef2b20e50a83ff6baea461699b0d74b583
kubernetes-node-linux-s390x.tar.gz bb444cc79035044cfb58cbe3d7bccd7998522dcf6d993441cf29fd03c249897c
kubernetes-node-windows-amd64.tar.gz 9b54e823c504601193b5ae2d37cb1d297ae9b5acfa1497b6f530a835071a7b6d

Changelog since v1.7.0-alpha.4

Action Required

  • The following alpha API groups were unintentionally enabled by default in previous releases, and will no longer be enabled by default in v1.8: (#47690, @caesarxuchao)
    • rbac.authorization.k8s.io/v1alpha1
    • settings.k8s.io/v1alpha1
    • If you wish to continue using them in v1.8, please enable them explicitly using the --runtime-config flag of the apiserver (for example, --runtime-config="rbac.authorization.k8s.io/v1alpha1,settings.k8s.io/v1alpha1")
  • Paths containing backsteps (for example, "../bar") are no longer allowed in hostPath volume paths, or in volumeMount subpaths (#47290, @jhorwit2)
  • Azure: Change container permissions to private for provisioned volumes. If you have existing Azure volumes that were created by Kubernetes v1.6.0-v1.6.5, you should change the permissions on them manually. (#47605, @brendandburns)
  • New and upgraded 1.7 GCE/GKE clusters no longer have an RBAC ClusterRoleBinding that grants the cluster-admin ClusterRole to the default service account in the kube-system namespace. (#46750, @cjcullen)
    • If this permission is still desired, run the following command to explicitly grant it, either before or after upgrading to 1.7:
    • kubectl create clusterrolebinding kube-system-default --serviceaccount=kube-system:default --clusterrole=cluster-admin
  • kube-apiserver: a new authorization mode (--authorization-mode=Node) authorizes nodes to access secrets, configmaps, persistent volume claims and persistent volumes related to their pods. (#46076, @liggitt) * Nodes must use client credentials that place them in the system:nodes group with a username of system:node:<nodeName> in order to be authorized by the node authorizer (the credentials obtained by the kubelet via TLS bootstrapping satisfy these requirements) * When used in combination with the RBAC authorization mode (--authorization-mode=Node,RBAC), the system:node role is no longer automatically granted to the system:nodes group.
  • kube-controller-manager has dropped support for the --insecure-experimental-approve-all-kubelet-csrs-for-group flag. Instead, the csrapproving controller uses authorization checks to determine whether to approve certificate signing requests: (#45619, @mikedanese) * requests for a TLS client certificate for any node are approved if the CSR creator has create permission on the certificatesigningrequests resource and nodeclient subresource in the certificates.k8s.io API group * requests from a node for a TLS client certificate for itself are approved if the CSR creator has create permission on the certificatesigningrequests resource and the selfnodeclient subresource in the certificates.k8s.io API group * requests from a node for a TLS serving certificate for itself are approved if the CSR creator has create permission on the certificatesigningrequests resource and the selfnodeserver subresource in the certificates.k8s.io API group
  • Support updating storageclasses in etcd to storage.k8s.io/v1. You must do this prior to upgrading to 1.8. (#46116, @ncdc)
  • The namespace API object no longer supports the deletecollection operation. (#46407, @liggitt)
  • NetworkPolicy has been moved from extensions/v1beta1 to the new (#39164, @danwinship) networking.k8s.io/v1 API group. The structure remains unchanged from the beta1 API. The net.beta.kubernetes.io/network-policy annotation on Namespaces to opt in to isolation has been removed. Instead, isolation is now determined at a per-pod level, with pods being isolated if there is any NetworkPolicy whose spec.podSelector targets them. Pods that are targeted by NetworkPolicies accept traffic that is accepted by any of the NetworkPolicies (and nothing else), and pods that are not targeted by any NetworkPolicy accept all traffic by default. Action Required: When upgrading to Kubernetes 1.7 (and a network plugin that supports the new NetworkPolicy v1 semantics), to ensure full behavioral compatibility with v1beta1:

    1. In Namespaces that previously had the "DefaultDeny" annotation, you can create equivalent v1 semantics by creating a NetworkPolicy that matches all pods but does not allow any traffic:

          kind: NetworkPolicy
    apiVersion: networking.k8s.io/v1
    metadata:
      name: default-deny
    spec:
      podSelector:

      This will ensure that pods that aren't matched by any other NetworkPolicy will continue to be fully-isolated, as they were before.

    2. In Namespaces that previously did not have the "DefaultDeny" annotation, you should delete any existing NetworkPolicy objects. These would have had no effect before, but with v1 semantics they might cause some traffic to be blocked that you didn't intend to be blocked.

Other notable changes

  • kubectl logs with label selector supports specifying a container name (#44282, @derekwaynecarr)
  • Adds an approval work flow to the the certificate approver that will approve certificate signing requests from kubelets that meet all the criteria of kubelet server certificates. (#46884, @jcbsmpsn)
  • AWS: Maintain a cache of all instances, to fix problem with > 200 nodes with ELBs (#47410, @justinsb)
  • Bump GLBC version to 0.9.5 - fixes loss of manually modified GCLB health check settings upon upgrade from pre-1.6.4 to either 1.6.4 or 1.6.5. (#47567, @nicksardo)
  • Update cluster-proportional-autoscaler, metadata-proxy, and fluentd-gcp addons with fixes for CVE-2016-4448, CVE-2016-8859, CVE-2016-9841, CVE-2016-9843, and CVE-2017-9526. (#47545, @ixdy)
  • AWS: Batch DescribeInstance calls with nodeNames to 150 limit, to stay within AWS filter limits. (#47516, @gnufied)
  • AWS: Process disk attachments even with duplicate NodeNames (#47406, @justinsb)
  • kubefed will now configure NodeInternalIP as the federation API server endpoint when NodeExternalIP is unavailable for federation API servers exposed as NodePort services (#46960, @lukaszo)
  • PodSecurityPolicy now recognizes pods that specify runAsNonRoot: false in their security context and does not overwrite the specified value (#47073, @Q-Lee)
  • Bump GLBC version to 0.9.4 (#47468, @nicksardo)
  • Stackdriver Logging deployment exposes metrics on node port 31337 when enabled. (#47402, @crassirostris)
  • Update to kube-addon-manager:v6.4-beta.2: kubectl v1.6.4 and refreshed base images (#47389, @ixdy)
  • Enable iptables -w in kubeadm selfhosted (#46372, @cmluciano)
  • Azure plugin for client auth (#43987, @cosmincojocar)
  • Fix dynamic provisioning of PVs with inaccurate AccessModes by refusing to provision when PVCs ask for AccessModes that can't be satisfied by the PVs' underlying volume plugin (#47274, @wongma7)
  • AWS: Avoid spurious ELB listener recreation - ignore case when matching protocol (#47391, @justinsb)
  • gce kube-up: The Node authorization mode and NodeRestriction admission controller are now enabled (#46796, @mikedanese)
  • update gophercloud/gophercloud dependency for reauthentication fixes (#45545, @stuart-warren)
  • fix sync loop health check with seperating runtime errors (#47124, @andyxning)
  • servicecontroller: Fix node selection logic on initial LB creation (#45773, @justinsb)
  • Fix iSCSI iSER mounting. (#47281, @mtanino)
  • StorageOS Volume Driver (#42156, @croomes)
    • StorageOS can be used as a storage provider for Kubernetes. With StorageOS, capacity from local or attached storage is pooled across the cluster, providing converged infrastructure for cloud-native applications.
  • CRI has been moved to package pkg/kubelet/apis/cri/v1alpha1/runtime. (#47113, @feiskyer)
  • Make gcp auth provider not to override the Auth header if it's already exits (#45575, @wanghaoran1988)
  • Allow pods to opt out of PodPreset mutation via an annotation on the pod. (#44965, @jpeeler)
  • Add Traditional Chinese translation for kubectl (#46559, @warmchang)
  • Remove Initializers from admission-control in kubernetes-master charm for pre-1.7 (#46987, @Cynerva)
  • Added state guards to the idle_status messaging in the kubernetes-master charm to make deployment faster on initial deployment. (#47183, @chuckbutler)
  • Bump up Node Problem Detector version to v0.4.0, which added support of parsing log from /dev/kmsg and ABRT. (#46743, @Random-Liu)
  • kubeadm: Enable the Node Authorizer/Admission plugin in v1.7 (#46879, @luxas)
  • Deprecated Binding objects in 1.7. (#47041, @k82cn)
  • Add secretbox and AES-CBC encryption modes to at rest encryption. AES-CBC is considered superior to AES-GCM because it is resistant to nonce-reuse attacks, and secretbox uses Poly1305 and XSalsa20. (#46916, @smarterclayton)
  • The HorizontalPodAutoscaler controller will now only send updates when it has new status information, reducing the number of writes caused by the controller. (#47078, @DirectXMan12)
  • gpusInUse info error when kubelet restarts (#46087, @tianshapjq)
  • kubeadm: Modifications to cluster-internal resources installed by kubeadm will be overwritten when upgrading from v1.6 to v1.7. (#47081, @luxas)
  • Added exponential backoff to Azure cloudprovider (#46660, @jackfrancis)
  • fixed HostAlias in PodSpec to allow foo.bar hostnames instead of just foo DNS labels. (#46809, @rickypai)
  • Implements rolling update for StatefulSets. Updates can be performed using the RollingUpdate, Paritioned, or OnDelete strategies. OnDelete implements the manual behavior from 1.6. status now tracks (#46669, @kow3ns)
    • replicas, readyReplicas, currentReplicas, and updatedReplicas. The semantics of replicas is now consistent with DaemonSet and ReplicaSet, and readyReplicas has the semantics that replicas did prior to this release.
  • Add Japanese translation for kubectl (#46756, @girikuncoro)
  • federation: Add admission controller for policy-based placement (#44786, @tsandall)
  • Get command uses OpenAPI schema to enhance display for a resource if run with flag 'use-openapi-print-columns'. (#46235, @droot)
    • An example command:
    • kubectl get pods --use-openapi-print-columns
  • add gzip compression to GET and LIST requests (#45666, @ilackarms)
  • Fix the bug where container cannot run as root when SecurityContext.RunAsNonRoot is false. (#47009, @yujuhong)
  • Fixes a bug with cAdvisorPort in the KubeletConfiguration that prevented setting it to 0, which is in fact a valid option, as noted in issue #11710. (#46876, @mtaufen)
  • Stackdriver cluster logging now deploys a new component to export Kubernetes events. (#46700, @crassirostris)
  • Alpha feature: allows users to set storage limit to isolate EmptyDir volumes. It enforces the limit by evicting pods that exceed their storage limits (#45686, @jingxu97)
  • Adds the Categories []string field to API resources, which represents the list of group aliases (e.g. "all") that every resource belongs to. (#43338, @fabianofranz)
  • Promote kubelet tls bootstrap to beta. Add a non-experimental flag to use it and deprecate the old flag. (#46799, @mikedanese)
  • Fix disk partition discovery for brtfs (#46816, @dashpole)
    • Add ZFS support
    • Add overlay2 storage driver support
  • Support creation of GCP Internal Load Balancers from Service objects (#46663, @nicksardo)
  • Introduces status conditions to the HorizontalPodAutoscaler in autoscaling/v2alpha1, indicating the current status of a given HorizontalPodAutoscaler, and why it is or is not scaling. (#46550, @DirectXMan12)
  • Support OpenAPI spec aggregation for kube-aggregator (#46734, @mbohlool)
  • Implement kubectl rollout undo and history for DaemonSet (#46144, @janetkuo)
  • Respect PDBs during node upgrades and add test coverage to the ServiceTest upgrade test. (#45748, @mml)
  • Disk Pressure triggers the deletion of terminated containers on the node. (#45896, @dashpole)
  • Add the alpha.image-policy.k8s.io/failed-open=true annotation when the image policy webhook encounters an error and fails open. (#46264, @Q-Lee)
  • Enable kubelet csr bootstrap in GCE/GKE (#40760, @mikedanese)
  • Implement Daemonset history (#45924, @janetkuo)
  • When switching from the service.beta.kubernetes.io/external-traffic annotation to the new (#46716, @thockin)
    • externalTrafficPolicy field, the values chnag as follows: * "OnlyLocal" becomes "Local" * "Global" becomes "Cluster".
  • Fix kubelet reset liveness probe failure count across pod restart boundaries (#46371, @sjenning)
  • The gce metadata server can be hidden behind a proxy, hiding the kubelet's token. (#45565, @Q-Lee)
  • AWS: Allow configuration of a single security group for ELBs (#45500, @nbutton23)
  • Allow remote admission controllers to be dynamically added and removed by administrators. External admission controllers make an HTTP POST containing details of the requested action which the service can approve or reject. (#46388, @lavalamp)
  • iscsi storage plugin: Fix dangling session when using multiple target portal addresses. (#46239, @mtanino)
  • Duplicate recurring Events now include the latest event's Message string (#46034, @kensimon)
  • With --feature-gates=RotateKubeletClientCertificate=true set, the kubelet will (#41912, @jcbsmpsn)
    • request a client certificate from the API server during the boot cycle and pause
    • waiting for the request to be satisfied. It will continually refresh the certificate
    • as the certificates expiration approaches.
  • The Kubernetes API supports retrieving tabular output for API resources via a new mime-type application/json;as=Table;v=v1alpha1;g=meta.k8s.io. The returned object (if the server supports it) will be of type meta.k8s.io/v1alpha1 with Table, and contain column and row information related to the resource. Each row will contain information about the resource - by default it will be the object metadata, but callers can add the ?includeObject=Object query parameter and receive the full object. In the future kubectl will use this to retrieve the results of kubectl get. (#40848, @smarterclayton)
  • This change add nonResourceURL to kubectl auth cani (#46432, @CaoShuFeng)
  • Webhook added to the API server which omits structured audit log events. (#45919, @ericchiang)
  • By default, --low-diskspace-threshold-mb is not set, and --eviction-hard includes "nodefs.available<10%,nodefs.inodesFree<5%" (#46448, @dashpole)
  • kubectl edit and kubectl apply will keep the ordering of elements in merged lists (#45980, @mengqiy)
  • [Federation][kubefed]: Use StorageClassName for etcd pvc (#46323, @marun)
  • Restrict active deadline seconds max allowed value to be maximum uint32 (#46640, @derekwaynecarr)
  • Implement kubectl get controllerrevisions (#46655, @janetkuo)
  • Local storage plugin (#44897, @msau42)
  • With --feature-gates=RotateKubeletServerCertificate=true set, the kubelet will (#45059, @jcbsmpsn)
    • request a server certificate from the API server during the boot cycle and pause
    • waiting for the request to be satisfied. It will continually refresh the certificate as
    • the certificates expiration approaches.
  • Allow PSP's to specify a whitelist of allowed paths for host volume based on path prefixes (#43946, @jhorwit2)
  • Add kubectl config rename-context (#46114, @arthur0)
  • Fix AWS EBS volumes not getting detached from node if routine to verify volumes are attached runs while the node is down (#46463, @wongma7)
  • Move hardPodAffinitySymmetricWeight to scheduler policy config (#44159, @wanghaoran1988)
  • AWS: support node port health check (#43585, @foolusion)
  • Add generic Toleration for NoExecute Taints to NodeProblemDetector (#45883, @gmarek)
  • support replaceKeys patch strategy and directive for strategic merge patch (#44597, @mengqiy)
  • Augment CRI to support retrieving container stats from the runtime. (#45614, @yujuhong)
  • Prevent kubelet from setting allocatable < 0 for a resource upon initial creation. (#46516, @derekwaynecarr)
  • add --non-resource-url to kubectl create clusterrole (#45809, @CaoShuFeng)
  • Add kubectl apply edit-last-applied subcommand (#42256, @shiywang)
  • Adding admissionregistration API group which enables dynamic registration of initializers and external admission webhooks. It is an alpha feature. (#46294, @caesarxuchao)
  • Fix log spam due to unnecessary status update when node is deleted. (#45923, @verult)
  • GCE installs will now avoid IP masquerade for all RFC-1918 IP blocks, rather than just 10.0.0.0/8. This means that clusters can (#46473, @thockin)
    • be created in 192.168.0.0./16 and 172.16.0.0/12 while preserving the container IPs (which would be lost before).
  • set selector and set subject no longer print "running in local/dry-run mode..." at the top, so their output can be piped as valid yaml or json (#46507, @bboreham)
  • ControllerRevision type added for StatefulSet and DaemonSet history. (#45867, @kow3ns)
  • Bump Go version to 1.8.3 (#46429, @wojtek-t)
  • Upgrade Elasticsearch Addon to v5.4.0 (#45589, @it-svit)
  • PodDisruptionBudget now uses ControllerRef to decide which controller owns a given Pod, so it doesn't get confused by controllers with overlapping selectors. (#45003, @krmayankk)
  • aws: Support for ELB tagging by users (#45932, @lpabon)
  • Portworx volume driver no longer has to run on the master. (#45518, @harsh-px)
  • kube-proxy: ratelimit runs of iptables by sync-period flags (#46266, @thockin)
  • Deployments are updated to use (1) a more stable hashing algorithm (fnv) than the previous one (adler) and (2) a hashing collision avoidance mechanism that will ensure new rollouts will not block on hashing collisions anymore. (#44774, @kargakis)
  • The Prometheus metrics for the kube-apiserver for tracking incoming API requests and latencies now return the subresource label for correctly attributing the type of API call. (#46354, @smarterclayton)
  • Add Simplified Chinese translation for kubectl (#45573, @shiywang)
  • The --namespace flag is now honored for in-cluster clients that have an empty configuration. (#46299, @ncdc)
  • Fix init container status reporting when active deadline is exceeded. (#46305, @sjenning)
  • Improves performance of Cinder volume attach/detach operations (#41785, @jamiehannaford)
  • GCE and AWS dynamic provisioners extension: admins can configure zone(s) in which a persistent volume shall be created. (#38505, @pospispa)
  • Break the 'certificatesigningrequests' controller into a 'csrapprover' controller and 'csrsigner' controller. (#45514, @mikedanese)
  • Modifies kubefed to create and the federation controller manager to use credentials associated with a service account rather than the user's credentials. (#42042, @perotinus)
  • Adds a MaxUnavailable field to PodDisruptionBudget (#45587, @foxish)
  • The behavior of some watch calls to the server when filtering on fields was incorrect. If watching objects with a filter, when an update was made that no longer matched the filter a DELETE event was correctly sent. However, the object that was returned by that delete was not the (correct) version before the update, but instead, the newer version. That meant the new object was not matched by the filter. This was a regression from behavior between cached watches on the server side and uncached watches, and thus broke downstream API clients. (#46223, @smarterclayton)
  • vSphere cloud provider: vSphere Storage policy Support for dynamic volume provisioning (#46176, @BaluDontu)
  • Add support for emitting metrics from openstack cloudprovider about storage operations. (#46008, @NickrenREN)
  • 'kubefed init' now supports overriding the default etcd image name with the --etcd-image parameter. (#46247, @marun)
  • remove the elasticsearch template (#45952, @harryge00)
  • Adds the CustomResourceDefinition (crd) types to the kube-apiserver. These are the successors to ThirdPartyResource. See https://github.com/kubernetes/community/blob/master/contributors/design-proposals/thirdpartyresources.md for more details. (#46055, @deads2k)
  • StatefulSets now include an alpha scaling feature accessible by setting the spec.podManagementPolicy field to Parallel. The controller will not wait for pods to be ready before adding the other pods, and will replace deleted pods as needed. Since parallel scaling creates pods out of order, you cannot depend on predictable membership changes within your set. (#44899, @smarterclayton)
  • fix kubelet event recording for selected events. (#46246, @derekwaynecarr)
  • Moved qos to api.helpers. (#44906, @k82cn)
  • Kubelet PLEG updates the relist timestamp only after successfully relisting. (#45496, @andyxning)
  • OpenAPI spec is now available in protobuf binary and gzip format (with ETag support) (#45836, @mbohlool)
  • Added support to a hierarchy of kubectl plugins (a tree of plugins as children of other plugins). (#45981, @fabianofranz)
    • Added exported env vars to kubectl plugins so that plugin developers have access to global flags, namespace, the plugin descriptor and the full path to the caller binary.
  • Ignored mirror pods in PodPreset admission plugin. (#45958, @k82cn)
  • Don't try to attach volume to new node if it is already attached to another node and the volume does not support multi-attach. (#45346, @codablock)
  • The Calico version included in kube-up for GCE has been updated to v2.2. (#38169, @caseydavenport)
  • Kubelet: Fix image garbage collector attempting to remove in-use images. (#46121, @Random-Liu)
  • Add ip-masq-agent addon to the addons folder which is used in GCE if --non-masquerade-cidr is set to 0/0 (#46038, @dnardo)
  • Fix serialization of EnforceNodeAllocatable (#44606, @ivan4th)
  • Add --write-config-to flag to kube-proxy to allow users to write the default configuration settings to a file. (#45908, @ncdc)
  • The NodeRestriction admission plugin limits the Node and Pod objects a kubelet can modify. In order to be limited by this admission plugin, kubelets must use credentials in the system:nodes group, with a username in the form system:node:<nodeName>. Such kubelets will only be allowed to modify their own Node API object, and only modify Pod API objects that are bound to their node. (#45929, @liggitt)
  • vSphere cloud provider: Report same Node IP as both internal and external. (#45201, @abrarshivani)
  • The options passed to a flexvolume plugin's mount command now contains the pod name (kubernetes.io/pod.name), namespace (kubernetes.io/pod.namespace), uid (kubernetes.io/pod.uid), and service account name (kubernetes.io/serviceAccount.name). (#39488, @liggitt)

v1.6.6

Documentation & Examples

Downloads for v1.6.6

filename sha256 hash
kubernetes.tar.gz 1574d868d43f5d88cfd1af255226e8cd6da72cd65fb9e1285557279c34f8a645
kubernetes-src.tar.gz 305f372320f78855e298b6caea062c8d1f7db117c7b44943ff5ddd0169424033

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz d93ca6f95cd80856b04d9c76a98ca986d6ba183d9fa100619fcda9f157bfd7f6
kubernetes-client-darwin-amd64.tar.gz facda65133f2893296f64c1067807dd7b354e2a4440afdd1ee62c05c07bcb91a
kubernetes-client-linux-386.tar.gz 5d1bd3ecc96f9e1cb9f20cef88c5aa2ec9c09370e8892557fc8a7cfe3cba595b
kubernetes-client-linux-amd64.tar.gz 94b2c9cd29981a8e150c187193bab0d8c0b6e906260f837367feff99860a6376
kubernetes-client-linux-arm64.tar.gz a7554e496403b50c12f5dbfaa00f2b887773905894ae5c330e2accd7b7e137c9
kubernetes-client-linux-arm.tar.gz 5a3414f4b47c84b173c879379d90b447af0540730bb86f10baf6d6933b09d41d
kubernetes-client-linux-ppc64le.tar.gz 904bab541dd8f1236d5e47f97cd2509c649f629fdc3af88a3968ca3c5575886d
kubernetes-client-linux-s390x.tar.gz d4a85694f796e4e1c14e6bddc295d9f776001fd8ac92ed32565606b964a843b0
kubernetes-client-windows-386.tar.gz 47258f6fc7fbd17ac1ddb38387adc5a2ddc2e39c5792cf3d354f9b19d373a6b2
kubernetes-client-windows-amd64.tar.gz e8c2957688acf19463e28d39cc1b4b1e9a12b3f10101dff179d1d63754b34236

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 7bbf43f81f5dbc3729c1956705d95c218b848591d03789d48f10e58fa865a0ba
kubernetes-server-linux-arm64.tar.gz f725f491a8998bdf164470029441135336ec0414360d6b57a5d8daf73d09334f
kubernetes-server-linux-arm.tar.gz 9026ca6fdbffef1d02409a86649e4dd0a7667ff6c27df318a3d851c271fb38d0
kubernetes-server-linux-ppc64le.tar.gz cd3b1b693b4e8225f78751bff533024785b0e20c2c51228956692db2a21d9f60
kubernetes-server-linux-s390x.tar.gz 9cd42506f4891be4691b7f4a8be7b894109dca54d0e9130651bc869101d7ed1f

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz 962f327f9a7b038c3b125a6cd06b690012fa38c827de0dae75955bb2be717126
kubernetes-node-linux-arm64.tar.gz e09af9f1130f8e1d4f6b45ec79eedb94e98354edb813b635c0dc097437834a1b
kubernetes-node-linux-arm.tar.gz 7647ec0a51308ca73e4c4eb4cbe09f2f9609c809e530d129a718d960a25d339d
kubernetes-node-linux-ppc64le.tar.gz 439b2135b179b699ca951a2d619629583d92dbdfb60b3920a1fa872b4cd65b6d
kubernetes-node-linux-s390x.tar.gz eed95c80bddad4a67d81c036b0d047dfb7bef8fb13a4e1b4817c1f2a595b993e
kubernetes-node-windows-amd64.tar.gz 611a92f9ab4f6dd48349843eec844b6abf861d76a151cce91b216a56bb6c821f

Changelog since v1.6.5

Action Required

  • Azure: Change container permissions to private for provisioned volumes. If you have existing Azure volumes that were created by Kubernetes v1.6.0-v1.6.5, you should change the permissions on them manually. (#47605, @brendandburns)

Other notable changes

v1.7.0-beta.2

Documentation & Examples

Downloads for v1.7.0-beta.2

filename sha256 hash
kubernetes.tar.gz 40814fcc343ee49df6a999165486714b5e970d90a368332c8e233a5741306a4c
kubernetes-src.tar.gz 864561a13af5869722276eb0f2d7c0c3bb8946c4ea23551b6a8a68027737cf1b

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz f4802f28767b55b0b29251485482e4db06dc15b257d9e9c8917d47a8531ebc20
kubernetes-client-darwin-amd64.tar.gz 0a9bb88dec66390e428f499046b35a9e3fbb253d1357006821240f3854fd391e
kubernetes-client-linux-386.tar.gz fbf5c1c9b0d9bfa987936539c8635d809becf2ab447187f6e908ad3d5acebdc5
kubernetes-client-linux-amd64.tar.gz 6b56b70519093c87a6a86543bcd137d8bea7b8ae172fdaa2914793baf47883eb
kubernetes-client-linux-arm64.tar.gz ff075b68d0dbbfd04788772d39299f16ee4c1a0f8ff175ed697afca206574707
kubernetes-client-linux-arm.tar.gz 81fec317664151ae318eca49436c9273e106ec869267b453c377544446d865e8
kubernetes-client-linux-ppc64le.tar.gz 91ee08c0209b767a576164eb6b44450f12ef29dedbca78b3daa447c6516b42fb
kubernetes-client-linux-s390x.tar.gz 28868e4bdd72861c87dd6bce4218fe56e578dd5998cab2da56bde0335904a26b
kubernetes-client-windows-386.tar.gz 779e7d864d762af4b039e511e14362426d8e60491a02f5ef571092aac9bc2b22
kubernetes-client-windows-amd64.tar.gz d35a306cb041026625335a330b4edffa8babec8e0b2d90b170ab8f318af87ff6

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 27f71259e3a7e819a6f5ffcf8ad63827f09e928173402e85690ec6943ef3a2fe
kubernetes-server-linux-arm64.tar.gz c9e331c452902293ea00e89ea1944d144c9200b97f033b56f469636c8c7b718d
kubernetes-server-linux-arm.tar.gz bf3e1b45982ef0a25483bd212553570fa3a1cda49f9a097a9796400fbb70e810
kubernetes-server-linux-ppc64le.tar.gz 90da52c556b0634241d2da84347537c49b16bfcb0d226afb4213f4ea5a9b80ec
kubernetes-server-linux-s390x.tar.gz 0c4243bae5310764508dba649d8440afbbd11fde2cac3ce651872a9f22694d45

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz d6c9d9642c31150b68b8da5143384bd4eee0617e16833d9bbafff94f25a76161
kubernetes-node-linux-arm64.tar.gz b91b52b5708539710817a9378295ca4c19afbb75016aa2908c00678709d641ec
kubernetes-node-linux-arm.tar.gz 3b3421abb90985773745a68159df338eb12c47645434a56c3806dd48e92cb023
kubernetes-node-linux-ppc64le.tar.gz a6b843af1284252636cf31a9523ff825c23dee5d57da24bf970031c846242ce5
kubernetes-node-linux-s390x.tar.gz 43830c0509e9477534661292fc3f4a100250adbee316028c5e869644d75aa478
kubernetes-node-windows-amd64.tar.gz 0ea1ee0dfc483248b3d20177bf023375289214ba153a6466a68764cf02931b52

Changelog since v1.7.0-beta.1

Action Required

  • New and upgraded 1.7 GCE/GKE clusters no longer have an RBAC ClusterRoleBinding that grants the cluster-admin ClusterRole to the default service account in the kube-system namespace. (#46750, @cjcullen)
    • If this permission is still desired, run the following command to explicitly grant it, either before or after upgrading to 1.7:
    • kubectl create clusterrolebinding kube-system-default --serviceaccount=kube-system:default --clusterrole=cluster-admin

Other notable changes

  • AWS: Process disk attachments even with duplicate NodeNames (#47406, @justinsb)
  • kubefed will now configure NodeInternalIP as the federation API server endpoint when NodeExternalIP is unavailable for federation API servers exposed as NodePort services (#46960, @lukaszo)
  • PodSecurityPolicy now recognizes pods that specify runAsNonRoot: false in their security context and does not overwrite the specified value (#47073, @Q-Lee)
  • Bump GLBC version to 0.9.4 (#47468, @nicksardo)
  • Stackdriver Logging deployment exposes metrics on node port 31337 when enabled. (#47402, @crassirostris)
  • Update to kube-addon-manager:v6.4-beta.2: kubectl v1.6.4 and refreshed base images (#47389, @ixdy)
  • Enable iptables -w in kubeadm selfhosted (#46372, @cmluciano)
  • Azure plugin for client auth (#43987, @cosmincojocar)
  • Fix dynamic provisioning of PVs with inaccurate AccessModes by refusing to provision when PVCs ask for AccessModes that can't be satisfied by the PVs' underlying volume plugin (#47274, @wongma7)
  • AWS: Avoid spurious ELB listener recreation - ignore case when matching protocol (#47391, @justinsb)
  • gce kube-up: The Node authorization mode and NodeRestriction admission controller are now enabled (#46796, @mikedanese)
  • update gophercloud/gophercloud dependency for reauthentication fixes (#45545, @stuart-warren)
  • fix sync loop health check with seperating runtime errors (#47124, @andyxning)
  • servicecontroller: Fix node selection logic on initial LB creation (#45773, @justinsb)
  • Fix iSCSI iSER mounting. (#47281, @mtanino)
  • StorageOS Volume Driver (#42156, @croomes)
    • StorageOS can be used as a storage provider for Kubernetes. With StorageOS, capacity from local or attached storage is pooled across the cluster, providing converged infrastructure for cloud-native applications.
  • CRI has been moved to package pkg/kubelet/apis/cri/v1alpha1/runtime. (#47113, @feiskyer)
  • Make gcp auth provider not to override the Auth header if it's already exits (#45575, @wanghaoran1988)
  • Allow pods to opt out of PodPreset mutation via an annotation on the pod. (#44965, @jpeeler)
  • Add Traditional Chinese translation for kubectl (#46559, @warmchang)
  • Remove Initializers from admission-control in kubernetes-master charm for pre-1.7 (#46987, @Cynerva)
  • Added state guards to the idle_status messaging in the kubernetes-master charm to make deployment faster on initial deployment. (#47183, @chuckbutler)
  • Bump up Node Problem Detector version to v0.4.0, which added support of parsing log from /dev/kmsg and ABRT. (#46743, @Random-Liu)
  • kubeadm: Enable the Node Authorizer/Admission plugin in v1.7 (#46879, @luxas)
  • Deprecated Binding objects in 1.7. (#47041, @k82cn)
  • Add secretbox and AES-CBC encryption modes to at rest encryption. AES-CBC is considered superior to AES-GCM because it is resistant to nonce-reuse attacks, and secretbox uses Poly1305 and XSalsa20. (#46916, @smarterclayton)
  • The HorizontalPodAutoscaler controller will now only send updates when it has new status information, reducing the number of writes caused by the controller. (#47078, @DirectXMan12)
  • gpusInUse info error when kubelet restarts (#46087, @tianshapjq)
  • kubeadm: Modifications to cluster-internal resources installed by kubeadm will be overwritten when upgrading from v1.6 to v1.7. (#47081, @luxas)

v1.6.5

Documentation & Examples

Known Issues for v1.6.5

Downloads for v1.6.5

filename sha256 hash
kubernetes.tar.gz e497ddd9d0fb03a71babd6c7f879951ba8e2d791c9884613d60794f7df9a5a51
kubernetes-src.tar.gz 3971e41435f6e22914b603ea56bec72f78673fca9f5a5436a4beda7957dc24e1

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz af2290d1c3c923a6f873736739e0fc381328d19eb726419a17f2ae51e3ac2b45
kubernetes-client-darwin-amd64.tar.gz 8a138c2487871807bc8461a5bb0867d75cf9da228ecb6acdc5d22c08b2ed107d
kubernetes-client-linux-386.tar.gz 3f6867dd51e7838f58cf7e95174ef914d8d280ff275ac56cc8b285566ce30996
kubernetes-client-linux-amd64.tar.gz 8f38e71b1c68d69299af86ab4432297ae0d72cdee1d1e1c9975d77e448096c9c
kubernetes-client-linux-arm64.tar.gz 1d01ae4423eb9794d09202ff4f24207c9d62b32a1b8f4906801a66fcd70b8fa5
kubernetes-client-linux-arm.tar.gz 438a8b4388960fe48e87aa7e97954f1cf9f9cc5c957eee108c3cc8040786fdce
kubernetes-client-linux-ppc64le.tar.gz 3b64d6ce09ffb65d3fe8c4101309c3b39fdab3643d89f91e89445c8e3279884e
kubernetes-client-linux-s390x.tar.gz f5fb3ddc6a6203ae68b0213624bbfa12b114a70a38e85151272273cbd8fd4fbd
kubernetes-client-windows-386.tar.gz 6ddc9b300746eefdc5d71aae193dea171115dab9db00010d416728d3f2035f19
kubernetes-client-windows-amd64.tar.gz 95854266bf64b84b59d1e6a3ca0501cf3c85fbd0258cb540893d5771aca74ace

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 68809d34d3429685eaafedf398f690bba1bcc1f793cd2d8193559b90492c68b1
kubernetes-server-linux-arm64.tar.gz 96bf4da5cbaa8f7b0f8909276005e0766ca4fa30431396ba30b9e77be1abb7f0
kubernetes-server-linux-arm.tar.gz 2f9c5275a6a2b5b10416a3e76f64e996851f486eb6b2dbe0f106b81bb63e63a9
kubernetes-server-linux-ppc64le.tar.gz f47bc83530dc7448879e7d11c2ba1613adc318fd6c1cbba76e5d7181d850667c
kubernetes-server-linux-s390x.tar.gz ebbd82df12da7470299e9877185797def86b859a44e72486e7be995c01eae56c

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz f842694701f8966cdfceca5f8f9d2b49bc84baf7446b67f7bf23b7581c245cc3
kubernetes-node-linux-arm64.tar.gz 4efc4c8d9df77ad66763ce5cc39650ea1ebd43dd4f789622c93b0aa872f7a186
kubernetes-node-linux-arm.tar.gz 6f4e2b764aec5c458e8bf28159190c0c725e20ab94cf9ced3279dc75caa3fe21
kubernetes-node-linux-ppc64le.tar.gz ba0e8ea11050273dbdf7b6d179251a95021b85000523e539c4940312586fd716
kubernetes-node-linux-s390x.tar.gz b6555e9a94a38e8bda502ec3eb951d4d854ebe8c44ba31320882a497703ab2bf
kubernetes-node-windows-amd64.tar.gz 0eaf3789c93996a45c74b30cc8a4d7169a639b7cf3fcf589ec387a0cfd0782d8

Changelog since v1.6.4

Other notable changes

  • Fix iSCSI iSER mounting. (#47281, @mtanino)
  • Added exponential backoff to Azure cloudprovider (#46660, @jackfrancis)
  • Update kube-dns to 1.14.2. (#45684, @bowei)
  • Fix log spam due to unnecessary status update when node is deleted. (#45923, @verult)
  • Poll all active pods periodically for volumes to fix out of order pod & node addition events. Fixes volumes not getting detached after controller restart. (#42033, @NickrenREN)
  • iscsi storage plugin: Fix dangling session when using multiple target portal addresses. (#46239, @mtanino)
  • The namespace API object no longer supports the deletecollection operation, which was previously allowed by mistake and did not respect expected namespace lifecycle semantics. (#47098, @liggitt)
  • Azure: Fix support for multiple loadBalancerSourceRanges, and add support for UDP ports and the Service spec's sessionAffinity. (#45523, @colemickens)
  • Fix the bug where container cannot run as root when SecurityContext.RunAsNonRoot is false. (#47009, @yujuhong)
  • Remove broken getvolumename and pass PV or volume name to attach call (#46249, @chakri-nelluri)
  • Portworx volume driver no longer has to run on the master. (#45518, @harsh-px)
  • Upgrade golang version to 1.7.6 (#46405, @cblecker)
  • kube-proxy handling of services with no endpoints now applies to both INPUT and OUTPUT chains, preventing socket leak. (#43972, @thockin)
  • Fix kube-apiserver crash when patching TPR data (#44612, @nikhita)
  • vSphere cloud provider: Report same Node IP as both internal and external. (#45201, @abrarshivani)
  • Kubelet: Fix image garbage collector attempting to remove in-use images. (#46121, @Random-Liu)
  • Add metrics exporter to the fluentd-gcp deployment (#45096, @crassirostris)
  • Fix the bug where StartedAt time is not reported for exited containers. (#45977, @yujuhong)
  • Enable basic auth username rotation for GCI (#44590, @ihmccreery)
  • vSphere cloud provider: Filter out IPV6 node addresses. (#45181, @BaluDontu)
  • vSphere cloud provider: Fix volume detach on node failure. (#45569, @divyenpatel)
  • Job controller: Retry with rate-limiting if a Pod create/delete operation fails. (#45870, @tacy)
  • Ensure that autoscaling/v1 is the preferred version for API discovery when autoscaling/v2alpha1 is enabled. (#45741, @DirectXMan12)
  • Add support for Azure internal load balancer. (#45690, @jdumars)
  • Fix kubectl delete --cascade=false for resources that don't have legacy overrides to orphan by default. (#45713, @kargakis)
  • Fix erroneous FailedSync and FailedMount events being periodically and indefinitely posted on Pods after kubelet is restarted (#44781, @wongma7)
  • fluentd will tolerate all NoExecute Taints when run in gcp configuration. (#45715, @gmarek)
  • Fix Deployments with Recreate strategy not waiting for Pod termination. (#45639, @ChipmunkV)
  • vSphere cloud provider: Remove the dependency of login information on worker nodes for vsphere cloud provider. (#43545, @luomiao)
  • vSphere cloud provider: Fix fetching of VM UUID on Ubuntu 16.04 and Fedora. (#45311, @divyenpatel)

v1.7.0-beta.1

Documentation & Examples

Downloads for v1.7.0-beta.1

filename sha256 hash
kubernetes.tar.gz e2fe83b443544dbb17c5ce481b6b3dcc9e62fbc573b5e270939282a31a910543
kubernetes-src.tar.gz 321df2749cf4687ec62549bc532eb9e17f159c26f4748732746bce1a4d41e77f

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 308cc980ee14aca49235569302e188dac08879f9236ed405884dada3b4984f44
kubernetes-client-darwin-amd64.tar.gz 791bc498c2bfd858497d7257500954088bec19dbfeb9809e7c09983fba04f2a6
kubernetes-client-linux-386.tar.gz d9ecac5521cedcc6a94d6b07a57f58f15bb25e43bd766911d2f16cf491a985ac
kubernetes-client-linux-amd64.tar.gz 33e800a541a1ce7a89e26dcfaa3650c06cf7239ae22272da944fb0d1288380e1
kubernetes-client-linux-arm64.tar.gz 8b245f239ebbede700adac1380f63a71025b8e1f7010e97665c77a0af84effaf
kubernetes-client-linux-arm.tar.gz 730aeeda02e500cc9300c7a555d4e0a1221b7cf182e95e6a9fbe16d90bbbc762
kubernetes-client-linux-ppc64le.tar.gz 7c97431547f40e9dece33e602993c19eab53306e64d16bf44c5e881ba52e5ab4
kubernetes-client-linux-s390x.tar.gz 8e95fcc59d9741d67789a8e6370a545c273206f7ff07e19154fe8f0126754571
kubernetes-client-windows-386.tar.gz 8bcd3ed7b6081e2a68e5a68cca71632104fef57e96ec5c16191028d113d7e54b
kubernetes-client-windows-amd64.tar.gz 1b32e418255f0c6b122b7aba5df9798d37c44c594ac36915ef081076d7464d52

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 2df51991734490871a6d6933ad15e785d543ecae2b06563fc92eb97a019f7eea
kubernetes-server-linux-arm64.tar.gz 8c97a97249d644fffbdcd87867e516f1029a3609979379ac4c6ea077f5b5b9b7
kubernetes-server-linux-arm.tar.gz 8e98741d19bd4a51ad275ca6bf793e0c305b75f2ac6569fb553b6cb62daa943e
kubernetes-server-linux-ppc64le.tar.gz 71398347d2aae5345431f4e4c2bedcbdf5c3f406952ce254ef0ae9e4f55355a1
kubernetes-server-linux-s390x.tar.gz 1f4fcbc1a70692a57accdab420ad2411acd4672f546473e977ef1c09357418bb

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz b84d291bc3e35912b4da067b3bf328dded87f875dc479b994408a161867c80e5
kubernetes-node-linux-arm64.tar.gz 2d306f1e757c49f9358791d7b0176e29f1aa32b6e6d70369b0e40c11a18b2df0
kubernetes-node-linux-arm.tar.gz 3957988bd800514a67ee1cf9e21f99f7e0797810ef3c22fd1604f0b6d1d6dad4
kubernetes-node-linux-ppc64le.tar.gz f7b3c9c01a25e6afd31dafaeed1eb926f6aae741c0f0967cca2c12492e509fd0
kubernetes-node-linux-s390x.tar.gz de7db84acd32cd7d5b3ac0957cded289335e187539e5495899e05b4043974892
kubernetes-node-windows-amd64.tar.gz efbafcae12ee121cf3a507bba8e36ac43d23d8262dc1a575b85e546ff81030fb

Changelog since v1.7.0-alpha.4

Action Required

  • kube-apiserver: a new authorization mode (--authorization-mode=Node) authorizes nodes to access secrets, configmaps, persistent volume claims and persistent volumes related to their pods. (#46076, @liggitt) * Nodes must use client credentials that place them in the system:nodes group with a username of system:node:<nodeName> in order to be authorized by the node authorizer (the credentials obtained by the kubelet via TLS bootstrapping satisfy these requirements) * When used in combination with the RBAC authorization mode (--authorization-mode=Node,RBAC), the system:node role is no longer automatically granted to the system:nodes group.
  • kube-controller-manager has dropped support for the --insecure-experimental-approve-all-kubelet-csrs-for-group flag. Instead, the csrapproving controller uses authorization checks to determine whether to approve certificate signing requests: (#45619, @mikedanese) * requests for a TLS client certificate for any node are approved if the CSR creator has create permission on the certificatesigningrequests resource and nodeclient subresource in the certificates.k8s.io API group * requests from a node for a TLS client certificate for itself are approved if the CSR creator has create permission on the certificatesigningrequests resource and the selfnodeclient subresource in the certificates.k8s.io API group * requests from a node for a TLS serving certificate for itself are approved if the CSR creator has create permission on the certificatesigningrequests resource and the selfnodeserver subresource in the certificates.k8s.io API group
  • Support updating storageclasses in etcd to storage.k8s.io/v1. You must do this prior to upgrading to 1.8. (#46116, @ncdc)
  • The namespace API object no longer supports the deletecollection operation. (#46407, @liggitt)
  • NetworkPolicy has been moved from extensions/v1beta1 to the new (#39164, @danwinship) networking.k8s.io/v1 API group. The structure remains unchanged from the beta1 API. The net.beta.kubernetes.io/network-policy annotation on Namespaces to opt in to isolation has been removed. Instead, isolation is now determined at a per-pod level, with pods being isolated if there is any NetworkPolicy whose spec.podSelector targets them. Pods that are targeted by NetworkPolicies accept traffic that is accepted by any of the NetworkPolicies (and nothing else), and pods that are not targeted by any NetworkPolicy accept all traffic by default. Action Required: When upgrading to Kubernetes 1.7 (and a network plugin that supports the new NetworkPolicy v1 semantics), to ensure full behavioral compatibility with v1beta1:

    1. In Namespaces that previously had the "DefaultDeny" annotation, you can create equivalent v1 semantics by creating a NetworkPolicy that matches all pods but does not allow any traffic:

          kind: NetworkPolicy
    apiVersion: networking.k8s.io/v1
    metadata:
      name: default-deny
    spec:
      podSelector:

      This will ensure that pods that aren't matched by any other NetworkPolicy will continue to be fully-isolated, as they were before.

    2. In Namespaces that previously did not have the "DefaultDeny" annotation, you should delete any existing NetworkPolicy objects. These would have had no effect before, but with v1 semantics they might cause some traffic to be blocked that you didn't intend to be blocked.

Other notable changes

  • Added exponential backoff to Azure cloudprovider (#46660, @jackfrancis)
  • fixed HostAlias in PodSpec to allow foo.bar hostnames instead of just foo DNS labels. (#46809, @rickypai)
  • Implements rolling update for StatefulSets. Updates can be performed using the RollingUpdate, Paritioned, or OnDelete strategies. OnDelete implements the manual behavior from 1.6. status now tracks (#46669, @kow3ns)
    • replicas, readyReplicas, currentReplicas, and updatedReplicas. The semantics of replicas is now consistent with DaemonSet and ReplicaSet, and readyReplicas has the semantics that replicas did prior to this release.
  • Add Japanese translation for kubectl (#46756, @girikuncoro)
  • federation: Add admission controller for policy-based placement (#44786, @tsandall)
  • Get command uses OpenAPI schema to enhance display for a resource if run with flag 'use-openapi-print-columns'. (#46235, @droot)
    • An example command:
    • kubectl get pods --use-openapi-print-columns
  • add gzip compression to GET and LIST requests (#45666, @ilackarms)
  • Fix the bug where container cannot run as root when SecurityContext.RunAsNonRoot is false. (#47009, @yujuhong)
  • Fixes a bug with cAdvisorPort in the KubeletConfiguration that prevented setting it to 0, which is in fact a valid option, as noted in issue #11710. (#46876, @mtaufen)
  • Stackdriver cluster logging now deploys a new component to export Kubernetes events. (#46700, @crassirostris)
  • Alpha feature: allows users to set storage limit to isolate EmptyDir volumes. It enforces the limit by evicting pods that exceed their storage limits (#45686, @jingxu97)
  • Adds the Categories []string field to API resources, which represents the list of group aliases (e.g. "all") that every resource belongs to. (#43338, @fabianofranz)
  • Promote kubelet tls bootstrap to beta. Add a non-experimental flag to use it and deprecate the old flag. (#46799, @mikedanese)
  • Fix disk partition discovery for brtfs (#46816, @dashpole)
    • Add ZFS support
    • Add overlay2 storage driver support
  • Support creation of GCP Internal Load Balancers from Service objects (#46663, @nicksardo)
  • Introduces status conditions to the HorizontalPodAutoscaler in autoscaling/v2alpha1, indicating the current status of a given HorizontalPodAutoscaler, and why it is or is not scaling. (#46550, @DirectXMan12)
  • Support OpenAPI spec aggregation for kube-aggregator (#46734, @mbohlool)
  • Implement kubectl rollout undo and history for DaemonSet (#46144, @janetkuo)
  • Respect PDBs during node upgrades and add test coverage to the ServiceTest upgrade test. (#45748, @mml)
  • Disk Pressure triggers the deletion of terminated containers on the node. (#45896, @dashpole)
  • Add the alpha.image-policy.k8s.io/failed-open=true annotation when the image policy webhook encounters an error and fails open. (#46264, @Q-Lee)
  • Enable kubelet csr bootstrap in GCE/GKE (#40760, @mikedanese)
  • Implement Daemonset history (#45924, @janetkuo)
  • When switching from the service.beta.kubernetes.io/external-traffic annotation to the new (#46716, @thockin)
    • externalTrafficPolicy field, the values chnag as follows: * "OnlyLocal" becomes "Local" * "Global" becomes "Cluster".
  • Fix kubelet reset liveness probe failure count across pod restart boundaries (#46371, @sjenning)
  • The gce metadata server can be hidden behind a proxy, hiding the kubelet's token. (#45565, @Q-Lee)
  • AWS: Allow configuration of a single security group for ELBs (#45500, @nbutton23)
  • Allow remote admission controllers to be dynamically added and removed by administrators. External admission controllers make an HTTP POST containing details of the requested action which the service can approve or reject. (#46388, @lavalamp)
  • iscsi storage plugin: Fix dangling session when using multiple target portal addresses. (#46239, @mtanino)
  • Duplicate recurring Events now include the latest event's Message string (#46034, @kensimon)
  • With --feature-gates=RotateKubeletClientCertificate=true set, the kubelet will (#41912, @jcbsmpsn)
    • request a client certificate from the API server during the boot cycle and pause
    • waiting for the request to be satisfied. It will continually refresh the certificate
    • as the certificates expiration approaches.
  • The Kubernetes API supports retrieving tabular output for API resources via a new mime-type application/json;as=Table;v=v1alpha1;g=meta.k8s.io. The returned object (if the server supports it) will be of type meta.k8s.io/v1alpha1 with Table, and contain column and row information related to the resource. Each row will contain information about the resource - by default it will be the object metadata, but callers can add the ?includeObject=Object query parameter and receive the full object. In the future kubectl will use this to retrieve the results of kubectl get. (#40848, @smarterclayton)
  • This change add nonResourceURL to kubectl auth cani (#46432, @CaoShuFeng)
  • Webhook added to the API server which omits structured audit log events. (#45919, @ericchiang)
  • By default, --low-diskspace-threshold-mb is not set, and --eviction-hard includes "nodefs.available<10%,nodefs.inodesFree<5%" (#46448, @dashpole)
  • kubectl edit and kubectl apply will keep the ordering of elements in merged lists (#45980, @mengqiy)
  • [Federation][kubefed]: Use StorageClassName for etcd pvc (#46323, @marun)
  • Restrict active deadline seconds max allowed value to be maximum uint32 (#46640, @derekwaynecarr)
  • Implement kubectl get controllerrevisions (#46655, @janetkuo)
  • Local storage plugin (#44897, @msau42)
  • With --feature-gates=RotateKubeletServerCertificate=true set, the kubelet will (#45059, @jcbsmpsn)
    • request a server certificate from the API server during the boot cycle and pause
    • waiting for the request to be satisfied. It will continually refresh the certificate as
    • the certificates expiration approaches.
  • Allow PSP's to specify a whitelist of allowed paths for host volume based on path prefixes (#43946, @jhorwit2)
  • Add kubectl config rename-context (#46114, @arthur0)
  • Fix AWS EBS volumes not getting detached from node if routine to verify volumes are attached runs while the node is down (#46463, @wongma7)
  • Move hardPodAffinitySymmetricWeight to scheduler policy config (#44159, @wanghaoran1988)
  • AWS: support node port health check (#43585, @foolusion)
  • Add generic Toleration for NoExecute Taints to NodeProblemDetector (#45883, @gmarek)
  • support replaceKeys patch strategy and directive for strategic merge patch (#44597, @mengqiy)
  • Augment CRI to support retrieving container stats from the runtime. (#45614, @yujuhong)
  • Prevent kubelet from setting allocatable < 0 for a resource upon initial creation. (#46516, @derekwaynecarr)
  • add --non-resource-url to kubectl create clusterrole (#45809, @CaoShuFeng)
  • Add kubectl apply edit-last-applied subcommand (#42256, @shiywang)
  • Adding admissionregistration API group which enables dynamic registration of initializers and external admission webhooks. It is an alpha feature. (#46294, @caesarxuchao)
  • Fix log spam due to unnecessary status update when node is deleted. (#45923, @verult)
  • GCE installs will now avoid IP masquerade for all RFC-1918 IP blocks, rather than just 10.0.0.0/8. This means that clusters can (#46473, @thockin)
    • be created in 192.168.0.0./16 and 172.16.0.0/12 while preserving the container IPs (which would be lost before).
  • set selector and set subject no longer print "running in local/dry-run mode..." at the top, so their output can be piped as valid yaml or json (#46507, @bboreham)
  • ControllerRevision type added for StatefulSet and DaemonSet history. (#45867, @kow3ns)
  • Bump Go version to 1.8.3 (#46429, @wojtek-t)
  • Upgrade Elasticsearch Addon to v5.4.0 (#45589, @it-svit)
  • PodDisruptionBudget now uses ControllerRef to decide which controller owns a given Pod, so it doesn't get confused by controllers with overlapping selectors. (#45003, @krmayankk)
  • aws: Support for ELB tagging by users (#45932, @lpabon)
  • Portworx volume driver no longer has to run on the master. (#45518, @harsh-px)
  • kube-proxy: ratelimit runs of iptables by sync-period flags (#46266, @thockin)
  • Deployments are updated to use (1) a more stable hashing algorithm (fnv) than the previous one (adler) and (2) a hashing collision avoidance mechanism that will ensure new rollouts will not block on hashing collisions anymore. (#44774, @kargakis)
  • The Prometheus metrics for the kube-apiserver for tracking incoming API requests and latencies now return the subresource label for correctly attributing the type of API call. (#46354, @smarterclayton)
  • Add Simplified Chinese translation for kubectl (#45573, @shiywang)
  • The --namespace flag is now honored for in-cluster clients that have an empty configuration. (#46299, @ncdc)
  • Fix init container status reporting when active deadline is exceeded. (#46305, @sjenning)
  • Improves performance of Cinder volume attach/detach operations (#41785, @jamiehannaford)
  • GCE and AWS dynamic provisioners extension: admins can configure zone(s) in which a persistent volume shall be created. (#38505, @pospispa)
  • Break the 'certificatesigningrequests' controller into a 'csrapprover' controller and 'csrsigner' controller. (#45514, @mikedanese)
  • Modifies kubefed to create and the federation controller manager to use credentials associated with a service account rather than the user's credentials. (#42042, @perotinus)
  • Adds a MaxUnavailable field to PodDisruptionBudget (#45587, @foxish)
  • The behavior of some watch calls to the server when filtering on fields was incorrect. If watching objects with a filter, when an update was made that no longer matched the filter a DELETE event was correctly sent. However, the object that was returned by that delete was not the (correct) version before the update, but instead, the newer version. That meant the new object was not matched by the filter. This was a regression from behavior between cached watches on the server side and uncached watches, and thus broke downstream API clients. (#46223, @smarterclayton)
  • vSphere cloud provider: vSphere Storage policy Support for dynamic volume provisioning (#46176, @BaluDontu)
  • Add support for emitting metrics from openstack cloudprovider about storage operations. (#46008, @NickrenREN)
  • 'kubefed init' now supports overriding the default etcd image name with the --etcd-image parameter. (#46247, @marun)
  • remove the elasticsearch template (#45952, @harryge00)
  • Adds the CustomResourceDefinition (crd) types to the kube-apiserver. These are the successors to ThirdPartyResource. See https://github.com/kubernetes/community/blob/master/contributors/design-proposals/thirdpartyresources.md for more details. (#46055, @deads2k)
  • StatefulSets now include an alpha scaling feature accessible by setting the spec.podManagementPolicy field to Parallel. The controller will not wait for pods to be ready before adding the other pods, and will replace deleted pods as needed. Since parallel scaling creates pods out of order, you cannot depend on predictable membership changes within your set. (#44899, @smarterclayton)
  • fix kubelet event recording for selected events. (#46246, @derekwaynecarr)
  • Moved qos to api.helpers. (#44906, @k82cn)
  • Kubelet PLEG updates the relist timestamp only after successfully relisting. (#45496, @andyxning)
  • OpenAPI spec is now available in protobuf binary and gzip format (with ETag support) (#45836, @mbohlool)
  • Added support to a hierarchy of kubectl plugins (a tree of plugins as children of other plugins). (#45981, @fabianofranz)
    • Added exported env vars to kubectl plugins so that plugin developers have access to global flags, namespace, the plugin descriptor and the full path to the caller binary.
  • Ignored mirror pods in PodPreset admission plugin. (#45958, @k82cn)
  • Don't try to attach volume to new node if it is already attached to another node and the volume does not support multi-attach. (#45346, @codablock)
  • The Calico version included in kube-up for GCE has been updated to v2.2. (#38169, @caseydavenport)
  • Kubelet: Fix image garbage collector attempting to remove in-use images. (#46121, @Random-Liu)
  • Add ip-masq-agent addon to the addons folder which is used in GCE if --non-masquerade-cidr is set to 0/0 (#46038, @dnardo)
  • Fix serialization of EnforceNodeAllocatable (#44606, @ivan4th)
  • Add --write-config-to flag to kube-proxy to allow users to write the default configuration settings to a file. (#45908, @ncdc)
  • The NodeRestriction admission plugin limits the Node and Pod objects a kubelet can modify. In order to be limited by this admission plugin, kubelets must use credentials in the system:nodes group, with a username in the form system:node:<nodeName>. Such kubelets will only be allowed to modify their own Node API object, and only modify Pod API objects that are bound to their node. (#45929, @liggitt)
  • vSphere cloud provider: Report same Node IP as both internal and external. (#45201, @abrarshivani)
  • The options passed to a flexvolume plugin's mount command now contains the pod name (kubernetes.io/pod.name), namespace (kubernetes.io/pod.namespace), uid (kubernetes.io/pod.uid), and service account name (kubernetes.io/serviceAccount.name). (#39488, @liggitt)

v1.6.4

Documentation & Examples

Known Issues for v1.6.4

Downloads for v1.6.4

filename sha256 hash
kubernetes.tar.gz fef6a97be8195fee1108b40acbd7bea61ef5244b8be23e799d2d01ee365907dd
kubernetes-src.tar.gz 2915465e9b389c5af0fa660f6e7cbc36a416d1286094201e2a2da5b084a37cb3

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz e2db37a1cf3dff342e9ba25506c96edba0cbc9b65984dfe985a7ab45df64f93e
kubernetes-client-darwin-amd64.tar.gz 0d49df4b06f76b5a6e168f72ac0088194d4267cc888880f7d0f23e558cd0ee32
kubernetes-client-linux-386.tar.gz 5e218cc7f01d6fa71d0a10a30eea2724ee111db3bbae5a03f0c560cd0d24a5df
kubernetes-client-linux-amd64.tar.gz 4c8dbd03a66d871f03592e84ed98d205d2d0e0c0f6d48c7b60f3e9840ad04df8
kubernetes-client-linux-arm64.tar.gz 9bf29b0f8bdde972d62556bdd14622199f979f7dcf56d3948d429b1d73feca08
kubernetes-client-linux-arm.tar.gz bbca1efe8fb95c6df7b330054776ce619652ba9d4c5428eabef9c435c61d1d9a
kubernetes-client-linux-ppc64le.tar.gz 7aa02e261f36a816dc1c7c898e16d732d9199d827ac4828f8e120b4a9ce5aa05
kubernetes-client-linux-s390x.tar.gz 531d00c43a49bb72365f2d6c1f31ad99ff09893e41f6b28d21980dbdd3ab0de4
kubernetes-client-windows-386.tar.gz 256fa2ffa77a1107e87a5a232bf8aa245afbcb5d383eda18f19f3fedbbad9a69
kubernetes-client-windows-amd64.tar.gz c8a97087b81defdc513a9fe4aaf317d10ad6fc3368170465dd4ea64c23655939

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 76a1d6dbce630b50fd3a5f566fcea6ef1a04996cf4f4c568338a3db0d3b6a3d5
kubernetes-server-linux-arm64.tar.gz 8b731307138a71ae90e025cb85ec7b4ac9179ebd8923f7abd1c839a2966ff2b0
kubernetes-server-linux-arm.tar.gz 0d3039f22cdc36526257f211c55099552b8d55cda25a05405f2701c869bb4be2
kubernetes-server-linux-ppc64le.tar.gz 6de3a85392ff65c019fd90173f1219a41f56559aebd07b18ed497e46645fcffc
kubernetes-server-linux-s390x.tar.gz 622a137c06a9fda23ec5941dd41607564804eeede0e6d3976cda6cc136e010c6

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz df40c178ffbd92376e98dd258113e35c0a46a8313f188d34d391a834baeb1da8
kubernetes-node-linux-arm64.tar.gz a27b15a0edcfd78470db54181ea2c2c942b5d4489b6f7a4ba78bb1fac34f8fa8
kubernetes-node-linux-arm.tar.gz 2b4dceee70ba7b508a0acc3cc5ce072d92f9c32c1a6961911b93a5da02ace9f7
kubernetes-node-linux-ppc64le.tar.gz c5e01f9f7de6ae2d73004bbcd288f5c942414b6639099c1bf52a98e592147745
kubernetes-node-linux-s390x.tar.gz eded4d2b94c9c22ae6c865e32a7965c1228d564aebf90c533607c716ed89b676
kubernetes-node-windows-amd64.tar.gz da561264f5665fe1ae9a41999731403b147b91c3a5c47439eb828ed688b0738f

Changelog since v1.6.3

Other notable changes

  • Fix kubelet panic during disk eviction introduced in 1.6.3. (#46049, @enisoc)
  • Fix pods failing to start if they specify a file as a volume subPath to mount. (#46046, @enisoc)

v1.7.0-alpha.4

Documentation & Examples

Downloads for v1.7.0-alpha.4

filename sha256 hash
kubernetes.tar.gz 14ef2ce3c9348dce7e83aeb167be324da93b90dbb8016f2aecb097c982abf790
kubernetes-src.tar.gz faef422988e805a3970985eabff03ed88cfb95ad0d2223abe03011145016e5d0

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 077dc5637f42a35c316a5e1c3a38e09625971894a186dd7b1e60408c9a0ac4b8
kubernetes-client-darwin-amd64.tar.gz 8e43eb7d1969e82eeb17973e4f09e9fe44ff3430cd2c35170d72a631c460deeb
kubernetes-client-linux-386.tar.gz 6ddfdbcb25243901c965b1e009e26a90b1fd08d6483906e1235ef380f6f93c97
kubernetes-client-linux-amd64.tar.gz 3e7cdd8e0e4d67ff2a0ee2548a4c48a433f84a25384ee9d22c06f4eb2e6db6d7
kubernetes-client-linux-arm64.tar.gz 3970c88d2c36fcb43a64d4e889a3eb2cc298e893f6084b9a3c902879d777487d
kubernetes-client-linux-arm.tar.gz 156909c55feb06036afff72aa180bd20c14758690cd04c7d8867f49c968e6372
kubernetes-client-linux-ppc64le.tar.gz 601fe881a131ce7868fdecfb1439da94ab5a1f1d3700efe4b8319617ceb23d4e
kubernetes-client-linux-s390x.tar.gz 2ed3e74e6a972d9ed5b2206fa5e811663497082384f488eada9901e9a99929c7
kubernetes-client-windows-386.tar.gz 1aba520fe0bf620f0e77f697194dfd5e336e4a97e2af01f8b94b0f03dbb6299c
kubernetes-client-windows-amd64.tar.gz aaf4a42549ea1113915649e636612ea738ead383140d92944c80f3c0d5df8161

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 1389c798e7805ec26826c0d3b17ab0d4bd51e0db21cf2f5d4bda5e2b530a6bf1
kubernetes-server-linux-arm64.tar.gz ccb99da4b069e63695b3b1d8add9a173e21a0bcaf03d031014460092ec726fb4
kubernetes-server-linux-arm.tar.gz 6eb3fe27e5017ed834a309cba21342a8c1443486a75ec87611fa66649dd5926a
kubernetes-server-linux-ppc64le.tar.gz 9b5030b0205ccccfd08b832eec917853fee8bcd34b04033ba35f17698be4a32f
kubernetes-server-linux-s390x.tar.gz 36b692c221005b52c2a243ddfc16e41a7b157e10fee8662bcd8270280b3f0927

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz bba76ad441716f938df0fd8c23c48588d1f80603e39dcca1a29c8b3bbe8c1658
kubernetes-node-linux-arm64.tar.gz e3e729847a13fd41ee7f969aabb14d3a0f6f8523f6f079f77a618bf5d781fb9c
kubernetes-node-linux-arm.tar.gz 520f98f244dd35bb0d96072003548f8b3aacc1e7beb31b5bc527416f07af9d32
kubernetes-node-linux-ppc64le.tar.gz 686490ba55ea8c7569b3b506f898315c8b1b243de23733e0cd537e2db8e067cb
kubernetes-node-linux-s390x.tar.gz a36bb76b390007b271868987739c550c8ac4e856f218f67f2fd780309a2a522e
kubernetes-node-windows-amd64.tar.gz e78c5a32584d96ec177e38b445c053e40c358e0549b925981c118f4c23578261

Changelog since v1.7.0-alpha.3

Action Required

  • kubectl create role and kubectl create clusterrole no longer allow specifying multiple resource names as comma-separated arguments. Use repeated --resource-name arguments to specify multiple resource names. (#44950, @xilabao)

Other notable changes

  • avoid concrete examples for missingResourceError (#45582, @CaoShuFeng)
  • Fix DNS suffix search list support in Windows kube-proxy. (#45642, @JiangtianLi)
  • Fix the bug where StartedAt time is not reported for exited containers. (#45977, @yujuhong)
  • Update Dashboard version to 1.6.1 (#45953, @maciaszczykm)
  • Examples: fixed cassandra mirror detection that assumes an FTP site will always be presented (#45965, @pompomJuice)
  • Removes the deprecated kubelet flag --babysit-daemons (#44230, @mtaufen)
  • [Federation] Automate configuring nameserver in cluster-dns for CoreDNS provider (#42895, @shashidharatd)
  • Add an AEAD encrypting transformer for storing secrets encrypted at rest (#41939, @smarterclayton)
  • Update Minio example (#45444, @NitishT)
  • [Federation] Segregate DNS related code to separate controller (#45034, @shashidharatd)
  • API Registration is now in beta. (#45247, @mbohlool)
  • Allow kcm and scheduler to lock on ConfigMaps. (#45739, @timothysc)
  • kubelet config should actually ignore files starting with dots (#45111, @dwradcliffe)
  • Fix lint failures on kubernetes-e2e charm (#45832, @Cynerva)
  • Mirror pods must now indicate the nodeName they are bound to on creation. The mirror pod annotation is now treated as immutable and cannot be added to an existing pod, removed from a pod, or modified. (#45775, @liggitt)
  • Updating apiserver to return UID of the deleted resource. Clients can use this UID to verify that the resource was deleted or waiting for finalizers. (#45600, @nikhiljindal)
  • OwnerReferencesPermissionEnforcement admission plugin ignores pods/status. (#45747, @derekwaynecarr)
  • prevent pods/status from touching ownerreferences (#45826, @deads2k)
  • Fix lint errors in juju kubernetes master and e2e charms (#45494, @ktsakalozos)
  • Ensure that autoscaling/v1 is the preferred version for API discovery when autoscaling/v2alpha1 is enabled. (#45741, @DirectXMan12)
  • Promotes Source IP preservation for Virtual IPs to GA. (#41162, @MrHohn)
    • Two api fields are defined correspondingly:
      • Service.Spec.ExternalTrafficPolicy <- 'service.beta.kubernetes.io/external-traffic' annotation.
      • Service.Spec.HealthCheckNodePort <- 'service.beta.kubernetes.io/healthcheck-nodeport' annotation.
  • Fix pods failing to start if they specify a file as a volume subPath to mount. (#45623, @wongma7)
  • the resource quota controller was not adding quota to be resynced at proper interval (#45685, @derekwaynecarr)
  • Marks the Kubelet's --master-service-namespace flag deprecated (#44250, @mtaufen)
  • fluentd will tolerate all NoExecute Taints when run in gcp configuration. (#45715, @gmarek)
  • Added Group/Version/Kind and Action extension to OpenAPI Operations (#44787, @mbohlool)
  • Updates kube-dns to 1.14.2 (#45684, @bowei)
      • Support kube-master-url flag without kubeconfig
      • Fix concurrent R/Ws in dns.go
      • Fix confusing logging when initialize server
      • Fix printf in cmd/kube-dns/app/server.go
      • Fix version on startup and --version flag
      • Support specifying port number for nameserver in stubDomains
  • detach the volume when pod is terminated (#45286, @gnufied)
  • Don't append :443 to registry domain in the kubernetes-worker layer registry action (#45550, @jacekn)
  • vSphere cloud provider: Fix volume detach on node failure. (#45569, @divyenpatel)
  • Remove the deprecated --enable-cri flag. CRI is now the default, (#45194, @yujuhong)
    • and the only way to integrate with kubelet for the container runtimes.
  • AWS: Remove check that forces loadBalancerSourceRanges to be 0.0.0.0/0. (#38636, @dhawal55)
  • Fix erroneous FailedSync and FailedMount events being periodically and indefinitely posted on Pods after kubelet is restarted (#44781, @wongma7)
  • Kubernetes now shares a single PID namespace among all containers in a pod when running with docker >= 1.13.1. This means processes can now signal processes in other containers in a pod, but it also means that the kubectl exec {pod} kill 1 pattern will cause the pod to be restarted rather than a single container. (#45236, @verb)
  • azure: add support for UDP ports (#45523, @colemickens)
    • azure: fix support for multiple loadBalancerSourceRanges
    • azure: support the Service spec's sessionAffinity
  • The fix makes scheduling go routine waiting for cache (e.g. Pod) to be synced. (#45453, @k82cn)
  • vSphere cloud provider: Filter out IPV6 node addresses. (#45181, @BaluDontu)
  • Default behaviour in cinder storageclass is changed. If availability is not specified, the zone is chosen by algorithm. It makes possible to spread stateful pods across many zones. (#44798, @zetaab)
  • A small clean up to remove unnecessary functions. (#45018, @ravisantoshgudimetla)
  • Removed old scheduler constructor. (#45472, @k82cn)
  • vSphere cloud provider: Fix fetching of VM UUID on Ubuntu 16.04 and Fedora. (#45311, @divyenpatel)
  • This fixes the overflow for priorityconfig- valid range {1, 9223372036854775806}. (#45122, @ravisantoshgudimetla)
  • Bump cluster autoscaler to v0.5.4, which fixes scale down issues with pods ignoring SIGTERM. (#45483, @mwielgus)
  • Create clusters with GPUs in GKE by specifying "type=,count=" to NODE_ACCELERATORS env var. (#45130, @vishh)
  • Remove deprecated node address type NodeLegacyHostIP. (#44830, @NickrenREN)
  • UIDs and GIDs now use apimachinery types (#44714, @jamiehannaford)
  • Enable basic auth username rotation for GCI (#44590, @ihmccreery)
  • Kubectl taint node based on label selector (#44740, @ravisantoshgudimetla)
  • Scheduler perf modular extensions. (#44770, @ravisantoshgudimetla)

v1.6.3

Documentation & Examples

Known Issues for v1.6.3

  • This release introduced a regression when using subPath. If the subPath is a file rather than a directory, Pods may fail to start (#45613).

    Do not upgrade to v1.6.3 if your cluster may run Pods with such subPaths.

Downloads for v1.6.3

filename sha256 hash
kubernetes.tar.gz 0af5914fcea36b3c65c8185f31e94b2839eaed52dfdd666d31dfa14534a7d69c
kubernetes-src.tar.gz 0d3cbc716b0d08bf3be779ddd096df965609b5bcb55c8b9ea084c6bb2d6ef1fd

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 2f2f58e8853eef7df293e579e8c94e1b6e75318b08bd1bf5747685ad8d16ebe2
kubernetes-client-darwin-amd64.tar.gz 122c20e2e92c1ed4a592c8a3851867452acff181ffe5251e8fee0ec8284704ac
kubernetes-client-linux-386.tar.gz 47c970bbe75a41634b9e5d0ae109a01f4823fdb83cf1c6c40a1ad4034b6d2764
kubernetes-client-linux-amd64.tar.gz ae141e0cd011889c4468b5b8b841d8cd62c1802c4ccba4dfd8c42beaccaf7e75
kubernetes-client-linux-arm64.tar.gz 07a83a7f7df555e859f4f8e143274f9ed1f475d597f01d1c79e95cdfda289c94
kubernetes-client-linux-arm.tar.gz 4a0b89b4985e651a1c29590ae2ea16ea00203d7cbad7ffc8a541b8a13569e1be
kubernetes-client-linux-ppc64le.tar.gz 1c0116942a61580da717845c9b7fc69aa58438aaa176888cd3e57267c9c717c0
kubernetes-client-linux-s390x.tar.gz 94307d778e0819dc5a64e12d794e95a028207d06603204d82610f369e040ce67
kubernetes-client-windows-386.tar.gz 322d2db5dadd4b388c785d1caf651bcc76c566afb6d19e84bdf6abcc40fa19d4
kubernetes-client-windows-amd64.tar.gz 9ef35675f7cd6acb81fa69ded37174e9a55cc0f58a2f8159bfc5512230495763

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 22eadeff9c3a45bf4d490ffca50bd257b6c282a3d16b5b8b40b8c31070a94de1
kubernetes-server-linux-arm64.tar.gz 2f9d976dd6d436a8258a5eb0c4a270329746f4331db607acc6b893f81f25e1c9
kubernetes-server-linux-arm.tar.gz 11f6a859438805250b84b415427df5f07d44a2a2ffb37591b6cdc6c8dc317382
kubernetes-server-linux-ppc64le.tar.gz 670fc921b50cca1c4fc20fbe58be640eeae766d38f6b2053b68c1a1293e88ba0
kubernetes-server-linux-s390x.tar.gz c5f2358bf81ea34fc01dbe5b639f03a10b5799ad75f8465863bb5c2b797b4f0b

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz 428332868f42f77e02f337779a18a6332894b27f2432c5b804a8ff753895b883
kubernetes-node-linux-arm64.tar.gz a8bdefd9c0ba9a247536a5a1bb7481b7a937cf39951256be774e45b8e40250cc
kubernetes-node-linux-arm.tar.gz 6b5aa71b27c0524b714489de80b2100e66bef99112f452aeaedcde1f890d2916
kubernetes-node-linux-ppc64le.tar.gz 34afa6e39ff8eb8a6f8f29874b6a3e5426fa6b64cc1b0f4466b17ae0f91f71ad
kubernetes-node-linux-s390x.tar.gz 170953b40e70242249c31e32456de73dacbed54e537efa4275d273441df98f7d
kubernetes-node-windows-amd64.tar.gz 410f175a47335b93f212cff5f3921a062ca6e945fa336d3cf0971f9bebba73e5

Changelog since v1.6.2

Other notable changes

  • Bump GLBC version to 0.9.3 (#45055, @nicksardo)
  • kubeadm: Fix invalid assign statement so it is possible to register the master kubelet with other initial Taints (#45376, @luxas)
  • Fix a bug that caused invalid Tolerations to be applied to Pods created by DaemonSets. (#45349, @gmarek)
  • Bump cluster autoscaler to v0.5.4, which fixes scale down issues with pods ignoring SIGTERM. (#45483, @mwielgus)
  • Fixes and minor cleanups to pod (anti)affinity predicate (#45098, @wojtek-t)
  • StatefulSet: Fix to fully preserve restart and termination order when StatefulSet is rapidly scaled up and down. (#45113, @kow3ns)
  • Fix some false negatives in detection of meaningful conflicts during strategic merge patch with maps and lists. (#43469, @enisoc)
  • cluster-autoscaler: Fix duplicate writing of logs. (#45017, @MaciekPytel)
  • Fixes a bug where pods were evicted even after images are successfully deleted. (#44986, @dashpole)
  • CRI: respect container's stopping timeout. (#44998, @feiskyer)
  • Fix problems with scaling up the cluster when unschedulable pods have some persistent volume claims. (#44860, @mwielgus)
  • Exclude nodes labeled as master from LoadBalancer / NodePort; restores documented behaviour. (#44745, @justinsb)
  • Fix for scaling down remaining good replicas when a failed Deployment is paused. (#44616, @kargakis)
  • kubectl commands run inside a pod using a kubeconfig file now use the namespace specified in the kubeconfig file, instead of using the pod namespace. If no kubeconfig file is used, or the kubeconfig does not specify a namespace, the pod namespace is still used as a fallback. (#44570, @liggitt)
  • Restored the ability of kubectl running inside a pod to consume resource files specifying a different namespace than the one the pod is running in. (#44862, @liggitt)
  • Fix false positive "meaningful conflict" detection for strategic merge patch with integer values. (#44788, @enisoc)
  • Fix insufficient permissions to read/write volumes when mounting them with a subPath. (#43775, @wongma7)
  • vSphere cloud provider: Allow specifying fstype in storage class. (#41929, @abrarshivani)
  • vSphere cloud provider: Allow specifying VSAN Storage Capabilities during dynamic volume provisioning. (#42974, @BaluDontu)

v1.7.0-alpha.3

Documentation & Examples

Downloads for v1.7.0-alpha.3

filename sha256 hash
kubernetes.tar.gz 03437cacddd91bb7dc21960c960d673ceb99b53040860638aa1d1fbde6d59fb5
kubernetes-src.tar.gz 190441318abddb44cfcbaec2f1b91d1a76167b91165ce5ae0d1a99c1130a2a36

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 1c3dcc57e014b15395a140eeeb285e38cf5510939b4113d053006d57d8e13087
kubernetes-client-darwin-amd64.tar.gz c33d893f67d8ac90834c36284ef88c529c43662c7179e2a9e4b17671c057400b
kubernetes-client-linux-386.tar.gz 5f3e44b8450db4f93a7ea1f366259c6333007a4536cb242212837bb241c3bbef
kubernetes-client-linux-amd64.tar.gz 85ac41dd849f3f9e033d4e123f79c4bd5d7b43bdd877d57dfc8fd2cadcef94be
kubernetes-client-linux-arm64.tar.gz f693032dde194de67900fe8cc5252959d70992b89a24ea43e11e9949835df5db
kubernetes-client-linux-arm.tar.gz 22fa2d2a77310acac1b08a7091929b03977afb2e4a246b054d38b3da15b84e33
kubernetes-client-linux-ppc64le.tar.gz 8717e6042a79f6a79f4527370adb1bbc903b0b9930c6aeee0174687b7443f9d4
kubernetes-client-linux-s390x.tar.gz 161c1da92b681decfb9800854bf3b9ff0110ba75c11008a784b891f3a57b032d
kubernetes-client-windows-386.tar.gz 19f5898a1fdef8c4caf27c6c2b79b0e085127b1d209f57361bce52ca8080842d
kubernetes-client-windows-amd64.tar.gz ff79c61efa87af3eeb7357740a495997d223d256b2e54c139572154e113dc247

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 13677b0400758f0d74087768be7abf3fd7bd927f0b874b8d6becc11394cdec2c
kubernetes-server-linux-arm64.tar.gz 0a2df3a6ebe157aa8a7e89bd8805dbad3623e122cc0f3614bfcb4ad528bd6ab1
kubernetes-server-linux-arm.tar.gz 76611e01de80c07ec954c91612a550063b9efc0c223e5dd638d71f4a3f3d9430
kubernetes-server-linux-ppc64le.tar.gz 2fe29a5871afe693f020e9642e6bc664c497e71598b70673d4f2c4523f57e28b
kubernetes-server-linux-s390x.tar.gz 33a1eb93a5d7004987de38ef54e888f0593e31cf9250be3e25118a1d1b474c07

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz de369ca9e5207fb67b26788b41cee1c75935baae348fedc1adf9dbae8c066e7d
kubernetes-node-linux-arm64.tar.gz 21839fe6c2a3fd3c165dea6ddbacdec008cdd154c9704866d13ac4dfb14ad7ae
kubernetes-node-linux-arm.tar.gz 2326a074f7c9ba205d996f4f42b8f511c33d909aefd3ea329cc579c4c14b5300
kubernetes-node-linux-ppc64le.tar.gz 58a3aeb5d55d040fd3133dbaa26eb966057ed2b35a5e0522ce8c1ebf4e9b2364
kubernetes-node-linux-s390x.tar.gz 2c231a8357d891012574b522ee7fa5e25c6b62b6d888d9bbbb195950cbe18536
kubernetes-node-windows-amd64.tar.gz 870bb1ab53a3f2bb5a3c068b425cd6330e71c86dc2ab899c79f733b63ddb51c5

Changelog since v1.7.0-alpha.2

Action Required

Other notable changes

  • kubeadm: Fix invalid assign statement so it is possible to register the master kubelet with other initial Taints (#45376, @luxas)
  • Use Docker API Version instead of docker version (#44068, @mkumatag)
  • bump(golang.org/x/oauth2): a6bd8cefa1811bd24b86f8902872e4e8225f74c4 (#45056, @ericchiang)
  • apimachinery: make explicit that meta.KindToResource is only a guess (#45272, @sttts)
  • Remove PodSandboxStatus.Linux.Namespaces.Network from CRI. (#45166, @feiskyer)
  • Fixed misspelled http URL in the cluster-dns example (#45246, @psiwczak)
  • separate discovery from the apiserver (#43003, @deads2k)
  • Remove the --secret-name flag from kubefed join, instead generating the secret name arbitrarily. (#42513, @perotinus)
  • Added InterPodAffinity unit test case with Namespace. (#45152, @k82cn)
  • Use munged semantic version for side-loaded docker tag (#44981, @ixdy)
  • Increase Dashboard's memory requests and limits (#44712, @maciaszczykm)
  • PodSpec's HostAliases now write entries into the Kubernetes-managed hosts file. (#45148, @rickypai)
  • Create and push a docker image for the cloud-controller-manager (#45154, @luxas)
  • Align Extender's validation with prioritizers. (#45091, @k82cn)
  • Retry calls we report config changes quickly. (#44959, @ktsakalozos)
  • A new field hostAliases has been added to pod.spec to support adding entries to a Pod's /etc/hosts file. (#44641, @rickypai)
  • Added CIFS PV support for Juju Charms (#45117, @chuckbutler)
  • Some container runtimes share a process (PID) namespace for all containers in a pod. This will become the default for Docker in a future release of Kubernetes. You can preview this functionality if running with the CRI and Docker 1.13.1 by enabling the --experimental-docker-enable-shared-pid kubelet flag. (#41583, @verb)
  • add APIService conditions (#43301, @deads2k)
  • Log warning when invalid dir passed to kubectl proxy --www (#44952, @CaoShuFeng)
  • Roll up volume error messages in the kubelet sync loop. (#44938, @jayunit100)
  • Introduces the ability to extend kubectl by adding third-party plugins. Developer preview, please refer to the documentation for instructions about how to use it. (#37499, @fabianofranz)
  • Fixes juju kubernetes master: 1. Get certs from a dead leader. 2. Append tokens. (#43620, @ktsakalozos)
  • Use correct option name in the kubernetes-worker layer registry action (#44921, @jacekn)
  • Start recording cloud provider metrics for AWS (#43477, @gnufied)
  • Bump GLBC version to 0.9.3 (#45055, @nicksardo)
  • Add metrics to all major gce operations {latency, errors} (#44510, @bowei)

    • The new metrics are:

    • cloudprovider_gce_api_request_duration_seconds{request, region, zone}

    • cloudprovider_gce_api_request_errors{request, region, zone}

    • request is the specific function that is used.

    • region is the target region (Will be "<n/a>" if not applicable)

    • zone is the target zone (Will be "<n/a>" if not applicable)

    • Note: this fixes some issues with the previous implementation of

    • metrics for disks:

      • Time duration tracked was of the initial API call, not the entire

    • operation.

      • Metrics label tuple would have resulted in many independent

    • histograms stored, one for each disk. (Did not aggregate well).

  • Update kubernetes-e2e charm to use snaps (#45044, @Cynerva)
  • Log the error (if any) in e2e metrics gathering step (#45039, @shyamjvs)
  • The proxy subresource APIs for nodes, services, and pods now support the HTTP PATCH method. (#44929, @liggitt)
  • cluster-autoscaler: Fix duplicate writing of logs. (#45017, @MaciekPytel)
  • CRI: Fix StopContainer timeout (#44970, @Random-Liu)
  • Fixes a bug where pods were evicted even after images are successfully deleted. (#44986, @dashpole)
  • Fix some false negatives in detection of meaningful conflicts during strategic merge patch with maps and lists. (#43469, @enisoc)
  • kubernetes-master juju charm properly detects etcd-scale events and reconfigures appropriately. (#44967, @chuckbutler)
  • Add redirect support to SpdyRoundTripper (#44451, @ncdc)
  • Support running Ubuntu image on GCE node (#44744, @yguo0905)
  • Send dns details only after cdk-addons are configured (#44945, @ktsakalozos)
  • Added support to the pause action in the kubernetes-worker charm for new flag --delete-local-data (#44931, @chuckbutler)
  • Upgrade go version to v1.8 (#41636, @luxas)
  • Add namespace-{list, create, delete} actions to the kubernetes-master layer (#44277, @jacekn)
  • Fix problems with scaling up the cluster when unschedulable pods have some persistent volume claims. (#44860, @mwielgus)
  • Feature/hpa upscale downscale delay configurable (#42101, @Dmitry1987)
  • Add short name "netpol" for networkpolicies (#42241, @xiangpengzhao)
  • Restored the ability of kubectl running inside a pod to consume resource files specifying a different namespace than the one the pod is running in. (#44862, @liggitt)
  • e2e: handle nil ReplicaSet in checkDeploymentRevision (#44859, @sttts)
  • Fix false positive "meaningful conflict" detection for strategic merge patch with integer values. (#44788, @enisoc)
  • Documented NodePort networking for CDK. (#44863, @chuckbutler)
  • Deployments and DaemonSets are now considered complete once all of the new pods are up and running - affects kubectl rollout status (and ProgressDeadlineSeconds for Deployments) (#44672, @kargakis)
  • Exclude nodes labeled as master from LoadBalancer / NodePort; restores documented behaviour. (#44745, @justinsb)
  • Fixes issue during LB creation where ports where incorrectly assigned to a floating IP (#44387, @jamiehannaford)
  • Remove redis-proxy.yaml sample, as the image is nowhere to be found. (#44801, @klausenbusk)
  • Resolves juju vsphere hostname bug showing only a single node in a scaled node-pool. (#44780, @chuckbutler)
  • kubectl commands run inside a pod using a kubeconfig file now use the namespace specified in the kubeconfig file, instead of using the pod namespace. If no kubeconfig file is used, or the kubeconfig does not specify a namespace, the pod namespace is still used as a fallback. (#44570, @liggitt)
  • This adds support for CNI ConfigLists, which permit plugin chaining. (#42202, @squeed)
  • API requests using impersonation now include the system:authenticated group in the impersonated user automatically. (#44076, @liggitt)
  • Print conditions of RC/RS in 'kubectl describe' command. (#44710, @xiangpengzhao)
  • cinder: Add support for the KVM virtio-scsi driver (#41498, @mikebryant)
  • Disallows installation of upstream docker from PPA in the Juju kubernetes-worker charm. (#44681, @wwwtyro)
  • Fluentd manifest pod is no longer created on non-registered master when creating clusters using kube-up.sh. (#44721, @piosz)
  • Job controller now respects ControllerRef to avoid fighting over Pods. (#42176, @enisoc)
  • CronJob controller now respects ControllerRef to avoid fighting with other controllers. (#42177, @enisoc)
  • The hyperkube image has been slimmed down and no longer includes addon manifests and other various scripts. These were introduced for the now removed docker-multinode setup system. (#44555, @luxas)
  • Refactoring reorganize taints function in kubectl to expose operations (#43171, @ravisantoshgudimetla)
  • The Kubernetes API server now exits if it encounters a networking failure (e.g. the networking interface hosting its address goes away) to allow a process manager (systemd/kubelet/etc) to react to the problem. Previously the server would log the failure and try again to bind to its configured address:port. (#42272, @marun)
  • Fixes a bug in the kubernetes-worker Juju charm code that attempted to give kube-proxy more than one api endpoint. (#44677, @wwwtyro)
  • Fixes a missing comma in a list of strings. (#44678, @wwwtyro)
  • Fix ceph-secret type to kubernetes.io/rbd in kubernetes-master charm (#44635, @Cynerva)

v1.5.7

Documentation & Examples

Downloads for v1.5.7

filename sha256 hash
kubernetes.tar.gz 36bc0bcdce4060546f3fef7186f1207d30d5fd340e72113ff592966bd6684827
kubernetes-src.tar.gz b329b02e9542049b9b85f8083a466e51799691bcf06fdf172b9c0f1cb61bdb6d

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 824ea7e5987e4ac7915b11fcd86658221a5a1e942a3f5210383435953509f96f
kubernetes-client-darwin-amd64.tar.gz 251a91eff457640066dd395393b16aae81850225db29207c07321b62fd9213ab
kubernetes-client-linux-386.tar.gz 84c69d23010304308459ad520375fd017f57562f8a78b6157ef0ea093636a8b6
kubernetes-client-linux-amd64.tar.gz 991e1eab65d1817ca3600e3ba3bc63ed86cf139a4669f84899f593ff684fb36c
kubernetes-client-linux-arm64.tar.gz afe9c001a41b88da351ddf0cb3d506d3d8da7d9a94ae2d4b05062b2927c81fec
kubernetes-client-linux-arm.tar.gz a936578c04887a2e1fe0a25e05f4d9663cd673d3fbac0c522bf75710d7f39f9b
kubernetes-client-windows-386.tar.gz 529ae014f0603868c82ee89601668fac17fa55932535d5925a7b61b1f301e61f
kubernetes-client-windows-amd64.tar.gz f1f7e588dca059a4cbe97b4a28a983d346f93fc2bb0d4a1dbbb7d55a3e33caef

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz ae18d659811da316d4a8bbdce15c4396fdee0068f9d3247a72c3a23433fee44c
kubernetes-server-linux-arm64.tar.gz d56187d19b42848b7ff09e82c0452120c173ae56709cae88f96312ee7c41b0c4
kubernetes-server-linux-arm.tar.gz aaa4d9414620bb1834401a17f2b877fe1347a4f8fc37c940092ac7f112e22934

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz 40c294ef5af4d548d37a599ee7fa07462f116fa5784d2b1501d95eeb04b8d34d
kubernetes-node-linux-arm64.tar.gz 37482d5933c99fca526d0d47f0cfb2b69101f2e60dd5240b490b9768c8e4171e
kubernetes-node-linux-arm.tar.gz 786ddb390a9fac6e41caa4bb8054240ddb5255b9937bb37d01d77e23857bb407
kubernetes-node-windows-amd64.tar.gz c3e89390c8026845fcf72765e84b7e3cd383de705ef46f4e3d422b343d66bd47

Changelog since v1.5.6

Other notable changes

v1.4.12

Documentation & Examples

Downloads for v1.4.12

filename sha256 hash
kubernetes.tar.gz f0d7ca7e1c92174c900d49087347d043b817eb589803eacc7727a84df9280ed2
kubernetes-src.tar.gz 251835f258d79f186d8c715b18f2ccb93312270b35c22434b4ff27bc1de50eda

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz e91c76b6281fe7b488f2f30aeaeecde58a6df1a0e23f6c431b6dc9d1adc1ff1a
kubernetes-client-darwin-amd64.tar.gz 4504bc965bd1b5bcea91d18c3a879252026796fdd251b72e3541499c65ac20e0
kubernetes-client-linux-386.tar.gz adf1f939db2da0b87bca876d9bee69e0d6bf4ca4a78e64195e9a08960e5ef010
kubernetes-client-linux-amd64.tar.gz 5419bdbba8144b55bf7bf2af1aefa531e25279f31a02d692f19b505862d0204f
kubernetes-client-linux-arm64.tar.gz 98ae30ac2e447b9e3c2768cac6861de5368d80cbd2db1983697c5436a2a2fe75
kubernetes-client-linux-arm.tar.gz ed8e9901c130aebfd295a6016cccb123ee42d826619815250a6add2d03942c69
kubernetes-client-windows-386.tar.gz bdca3096bed1a4c485942ab1d3f9351f5de00962058adefbb5297d50071461d4
kubernetes-client-windows-amd64.tar.gz a74934eca20dd2e753d385ddca912e76dafbfff2a65e3e3a1ec3c5c40fd92bc8

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz bf8aa3e2e204c1f782645f7df9338767daab7be3ab47a4670e2df08ee410ee7f
kubernetes-server-linux-arm64.tar.gz 7c5cfe06fe1fcfe11bd754921e88582d16887aacb6cee0eb82573c88debce65e
kubernetes-server-linux-arm.tar.gz 551c2bc2e3d1c0b8fa30cc0b0c8fae1acf561b5e303e9ddaf647e49239a97e6e

Node Binaries

filename sha256 hash
kubernetes-node*.tar.gz ``

Changelog since v1.4.9

Other notable changes

  • kube-apiserver now drops unneeded path information if an older version of Windows kubectl sends it. (#44586, @mml)
  • Bump gcr.io/google_containers/glbc from 0.8.0 to 0.9.2. Release notes: 0.9.0, 0.9.1, 0.9.2 (#43098, @timstclair)
  • Patch CVE-2016-8859 in alpine based images: (#42937, @timstclair)
      • gcr.io/google-containers/etcd-empty-dir-cleanup
      • gcr.io/google-containers/kube-dnsmasq-amd64
  • Check if pathExists before performing Unmount (#39311, @rkouj)
  • Unmount operation should not fail if volume is already unmounted (#38547, @rkouj)
  • Updates base image used for kube-addon-manager to latest python:2.7-slim and embedded kubectl to v1.3.10. No functionality changes expected. (#42842, @ixdy)
  • list-resources: don't fail if the grep fails to match any resources (#41933, @ixdy)
  • Update gcr.io/google-containers/rescheduler to v0.2.2, which uses busybox as a base image instead of ubuntu. (#41911, @ixdy)
  • Backporting TPR fix to 1.4 (#42380, @foxish)
  • Fix AWS device allocator to only use valid device names (#41455, @gnufied)
  • Reverts to looking up the current VM in vSphere using the machine's UUID, either obtained via sysfs or via the vm-uuid parameter in the cloud configuration file. (#40892, @robdaemon)
  • We change the default attach_detach_controller sync period to 1 minute to reduce the query frequency through cloud provider to check whether volumes are attached or not. (#41363, @jingxu97)
  • Bump GCI to gci-stable-56-9000-84-2: Fixed google-accounts-daemon breaks on GCI when network is unavailable. Fixed iptables-restore performance regression. (#41831, @freehan)
  • Update fluentd-gcp addon to 1.25.2 (#41863, @ixdy)
  • Bump GCE ContainerVM to container-vm-v20170214 to address CVE-2016-9962. (#41449, @zmerlynn)

v1.7.0-alpha.2

Documentation & Examples

Downloads for v1.7.0-alpha.2

filename sha256 hash
kubernetes.tar.gz d60465c07b8aa4b5bc8e3de98769d72d22985489e5cdfd1a3165e36c755d6c3b
kubernetes-src.tar.gz b0b388571225e37a5b9bca6624a92e69273af907cdb300a6d0ac6a0d0d364bd4

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 55b04bc43c45bd93cf30174036ad64109ca1070ab3b331882e956f483dac2b6a
kubernetes-client-darwin-amd64.tar.gz d61c055ca90aacb6feb10f45feaaf11f188052598cfef79f4930358bb37e09ad
kubernetes-client-linux-386.tar.gz e10ce9339ee6158759675bfb002409fa7f70c701aa5a8a5ac97abc56742561b7
kubernetes-client-linux-amd64.tar.gz b9cb60ba71dfa144ed1e6f2116afd078782372d427912838c56f3b77a74afda0
kubernetes-client-linux-arm64.tar.gz bc0446c484dba91d8f1e32c0175b81dca5c6ff0ac9f5dd3f69cff529afb83aff
kubernetes-client-linux-arm.tar.gz f794765ca98a2c0611fda32756250eff743c25b66cd4d973fc5720a55771c1c6
kubernetes-client-linux-ppc64le.tar.gz 216cb6e96ba6af5ae259c069576fcd873c48a8a4e8918f5e08ac13427fbefd57
kubernetes-client-linux-s390x.tar.gz fb7903d028744fdfe3119ade6b2ee71532e3d69a82bd5834206fe84e50821253
kubernetes-client-windows-386.tar.gz 6bdfbd12361f814c86f268dcc807314f322efe9390ca2d91087e617814e91684
kubernetes-client-windows-amd64.tar.gz fd26fc5f0e967b9f6ab18bc28893f2037712891179ddb67b035434c94612f7e3

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz e14c0748789f6a1c3840ab05d0ad5b796a0f03722ee923f8208740f702c0bc19
kubernetes-server-linux-arm64.tar.gz 270e0a6fcc0a2f38c8c6e8929a4a593535014bde88f69479a52c5b625bca435c
kubernetes-server-linux-arm.tar.gz 0bd58c2f8d8b6e8110354ccd71eb97eb873aca7b074ce9f83dab4f62a696e964
kubernetes-server-linux-ppc64le.tar.gz 57a4a5dcdb573fb6dc08dbd53d0f196c66d245fa2159a92bf8da0d29128e486d
kubernetes-server-linux-s390x.tar.gz 404c8dcc300281f5588e6f4dd15e3c41f858c6597e37a817913112d545a7f736

Changelog since v1.7.0-alpha.1

Action Required

  • kubectl create rolebinding and kubectl create clusterrolebinding no longer allow specifying multiple subjects as comma-separated arguments. Use repeated --user, --group, or --serviceaccount arguments to specify multiple subjects. (#43903, @xilabao)

Other notable changes

  • Add support for Azure internal load balancer (#43510, @karataliu)
  • Improved output on 'kubectl get' and 'kubectl describe' for generic objects. (#44222, @fabianofranz)
  • Add Kubernetes 1.6 support to Juju charms (#44500, @Cynerva)
    • Add metric collection to charms for autoscaling
    • Update kubernetes-e2e charm to fail when test suite fails
    • Update Juju charms to use snaps
    • Add registry action to the kubernetes-worker charm
    • Add support for kube-proxy cluster-cidr option to kubernetes-worker charm
    • Fix kubernetes-master charm starting services before TLS certs are saved
    • Fix kubernetes-worker charm failures in LXD
    • Fix stop hook failure on kubernetes-worker charm
    • Fix handling of juju kubernetes-worker.restart-needed state
    • Fix nagios checks in charms
  • Users can now specify listen and advertise URLs for etcd in a kubeadm cluster (#42246, @jamiehannaford)
  • Fixed kubectl cluster-info dump to support multi-container pod. (#44088, @xingzhou)
  • Prints out status updates when running kubefed init (#41849, @perotinus)
  • CRI: Fix kubelet failing to start when using rkt. (#44569, @yujuhong)
  • Remove deprecatedPublicIPs field (#44519, @thockin)
  • Remove deprecated ubuntu kube-up deployment. (#44344, @mikedanese)
  • Use OS-specific libs when computing client User-Agent in kubectl, etc. (#44423, @monopole)
  • kube-apiserver now drops unneeded path information if an older version of Windows kubectl sends it. (#44421, @mml)
  • Extending the gc admission plugin so that a user who doesn't have delete permission of the owner cannot modify blockOwnerDeletion field of existing ownerReferences, or add new ownerReference with blockOwnerDeletion=true (#43876, @caesarxuchao)
  • kube-apiserver: --service-account-lookup now defaults to true, requiring the Secret API object containing the token to exist in order for a service account token to be valid. This enables service account tokens to be revoked by deleting the Secret object containing the token. (#44071, @liggitt)
  • CRI: kubectl logs -f now stops following when container stops, as it did pre-CRI. (#44406, @Random-Liu)
  • Add completion support for --namespace and --cluster to kubectl (#44251, @superbrothers)
  • dnsprovider: avoid panic if route53 fields are nil (#44380, @justinsb)
  • In 'kubectl describe', find controllers with ControllerRef, instead of showing the original creator. (#42849, @janetkuo)
  • Heat cluster operations now support environments that have multiple Swift URLs (#41561, @jamiehannaford)
  • Adds support for allocation of pod IPs via IP aliases. (#42147, @bowei)
  • alpha volume provisioning is removed and default storage class should be used instead. (#44090, @NickrenREN)
  • validateClusterInfo: use clientcmdapi.NewCluster() (#44221, @ncdc)
  • Fix corner-case with OnlyLocal Service healthchecks. (#44313, @thockin)
  • Adds annotations to all Federation objects created by kubefed. (#42683, @perotinus)
  • [Federation][Kubefed] Bug fix to enable disabling federation controllers through override args (#44209, @irfanurrehman)
  • [Federation] Remove deprecated federation-apiserver-kubeconfig secret (#44287, @shashidharatd)
  • Scheduler can recieve its policy configuration from a ConfigMap (#43892, @bsalamat)
  • AWS cloud provider: fix support running the master with a different AWS account or even on a different cloud provider than the nodes. (#44235, @mrIncompetent)
  • add rancher credential provider (#40160, @wlan0)
  • Support generating Open API extensions for strategic merge patch tags in go struct tags (#44121, @mbohlool)
  • Use go1.8.1 for arm and ppc64le (#44216, @mkumatag)
  • Aggregated used ports at the NodeInfo level for PodFitsHostPorts predicate. (#42524, @k82cn)
  • Catch error when failed to make directory in NFS volume plugin (#38801, @nak3)
  • Support iSCSI CHAP authentication (#43396, @rootfs)
  • Support context completion for kubectl config use-context (#42336, @superbrothers)
  • print warning when delete current context (#42538, @adohe)
  • Add node e2e tests for hostPid (#44119, @feiskyer)
  • kubeadm: Make kubeadm reset tolerant of a disabled docker service. (#43951, @luxas)
  • kubelet: make dockershim.sock configurable (#43914, @ncdc)
  • Fix broken service accounts when using dedicated service account key. (#44169, @mikedanese)
  • Fix incorrect conflict errors applying strategic merge patches to resources. (#43871, @liggitt)
  • Fix transition between NotReady and Unreachable taints. (#44042, @gmarek)
  • leader election lock based on scheduler name (#42961, @wanghaoran1988)
  • [Federation] Remove FEDERATIONS_DOMAIN_MAP references (#43137, @shashidharatd)
  • Fix for federation failing to propagate cascading deletion. (#44108, @csbell)
  • Fix bug with service nodeports that have no backends not being rejected, when they should be. This is not a regression vs v1.5 - it's a fix that didn't quite fix hard enough. (#43972, @thockin)
  • Fix for failure to delete federation controllers with finalizers. (#44084, @nikhiljindal)
  • Fix container hostPid settings. (#44097, @feiskyer)
  • Fixed an issue mounting the wrong secret into pods as a service account token. (#44102, @ncdc)

v1.6.2

Documentation & Examples

Downloads for v1.6.2

filename sha256 hash
kubernetes.tar.gz 240f66a98bf75246b53ef7c1fa3a2be36a92cbc173bc8626e7bc4427bef9ce6a
kubernetes-src.tar.gz dbf19a8f2e50b3e691eeba0c418fe057f1ea8527b8c0194ba9749c12c801b24b

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 3b1437cbc9d10e5466c83304c54ab06f5a880e0b047e2b0ea775530ee893b6b6
kubernetes-client-darwin-amd64.tar.gz e3dad0848b3d6c1737199d0704c692e74bf979e6a81fabea79c5b2499ca3fa73
kubernetes-client-linux-386.tar.gz 962f576e67f13f8f8afc958f89f0371c7496b2540372ef7f8e1bd0e43a67e829
kubernetes-client-linux-amd64.tar.gz f8ef17b8b4bb8f6974fa2b3faa992af3c39ad318c30bdfe1efab957361d8bdfe
kubernetes-client-linux-arm64.tar.gz 9582e6783e7effa10b0af2f662d1bc4471bbf8205d9df07a543edb099ba7f27c
kubernetes-client-linux-arm.tar.gz 165b642ba6900f7d779bc6a27f89ccdb09eefcbb310a8bc5f6d101db27e2e9cc
kubernetes-client-linux-ppc64le.tar.gz 514a308ccfb978111a74b5bf843cf6862464743f0f4e2aaffada33add4c2bb0f
kubernetes-client-linux-s390x.tar.gz e030593109a369bc3288c9f47703843248dbe4a9ade496f936d8cc355a874ba6
kubernetes-client-windows-386.tar.gz a2b0053de6b62d09123d8fcc1927a8cf9ab2a5a312794a858e7b423f4ffdbe3e
kubernetes-client-windows-amd64.tar.gz eafdaa29a70d1820be0dc181074c5788127996402bbd5af53b0b765ed244e476

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 016bc4db69a8f90495e82fbe6e5ec9a12e56ecab58a8eb2e5471bf9cab827ad2
kubernetes-server-linux-arm64.tar.gz 1985596d769656d68ec692030dd31bbec8081daf52ddaef6a2a7af7d6b1f7876
kubernetes-server-linux-arm.tar.gz e0d4c53c55de5c100973379005aabe1441004ed4213b96a6e492c88d5a9b7f49
kubernetes-server-linux-ppc64le.tar.gz 652a8230c4511bc30d8f3a6ae11ebeee8c6d350648d879f8f2e1aa34777323d0
kubernetes-server-linux-s390x.tar.gz 1eab2d36beecf4f74e3b7b74734a75faf43ed6428d312aebe2e940df4cceb5ed

Changelog since v1.6.1

Other notable changes

v1.7.0-alpha.1

Documentation & Examples

Downloads for v1.7.0-alpha.1

filename sha256 hash
kubernetes.tar.gz a8430f678ae5abb16909183bb6472d49084b26c2990854dac73f55be69941435
kubernetes-src.tar.gz 09792d0b31c3c0f085f54a62c0d151029026cee3c57ac8c3456751ef2243967f

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 115543a5ec55f9039136e0ecfd90d6510b146075d13987fad9c03db3761fbac6
kubernetes-client-darwin-amd64.tar.gz 91b7cc89386041125af2ecafd3c6e73197f0b7af3ec817d9aed4822e1543eee9
kubernetes-client-linux-386.tar.gz 7a77bfec2873907ad1f955e33414a9afa029d37d90849bf652e7bab1f2c668ed
kubernetes-client-linux-amd64.tar.gz 674d1a839869ac308f3a273ab41be42dab8b52e96526effdbd268255ab6ad4c1
kubernetes-client-linux-arm64.tar.gz 4b0164b0474987df5829dcd88c0cdf2d16dbcba30a03cd0ad5ca860d6b4a2f3f
kubernetes-client-linux-arm.tar.gz cb5a941c3e61465eab544c7b23acd4be6969d74ac23bd9370aa3f9dfc24f2b42
kubernetes-client-linux-ppc64le.tar.gz d583aff4c86de142b5e6e23cd5c8eb9617fea6574acede9fa2420169405429c6
kubernetes-client-linux-s390x.tar.gz ab14c4806b4e9c7a41993924467969886e1288216d80d2d077a2c35f26fc8cc5
kubernetes-client-windows-386.tar.gz 0af3f9d1193d9ea49bb4e1cb46142b846b70ceb49ab47ad6fc2497a0dc88395d
kubernetes-client-windows-amd64.tar.gz 12a9dffda6ba8916149b681f49af506790be97275fe6fc16552ac765aef20a99

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz d6b4c285a89172692e4ba82b777cc9df5b2f5061caa0a9cef6add246a848eeb9
kubernetes-server-linux-arm64.tar.gz e73fb04d4ff692f19de09cfc3cfa17014e23df4150b26c20c3329f688c164358
kubernetes-server-linux-arm.tar.gz 98763b72ba6652abfd5b671981506f8c35ab522d34af34636e5095413769eeb5
kubernetes-server-linux-ppc64le.tar.gz b39dbb0dc96dcdf1ec4cbd5788e00e46c0d11efb42c6dbdec64758aa8aa9d8e5
kubernetes-server-linux-s390x.tar.gz c0171e2f22c4e51f25185e71387301ad2c0ade90139fe96dec1c2f999de71716

Changelog since v1.6.0

Other notable changes

  • Juju: Enable GPU mode if GPU hardware detected (#43467, @tvansteenburgh)
  • Check the error before parsing the apiversion (#44047, @yujuhong)
  • get-kube-local.sh checks pods with option "--namespace=kube-system" (#42518, @mtanino)
  • Using http2 in kubeapi-load-balancer to fix kubectl exec uses (#43625, @mbruzek)
  • Support status.hostIP in downward API (#42717, @andrewsykim)
  • AWS cloud provider: allow to set KubernetesClusterID or KubernetesClusterTag in combination with VPC. (#42512, @scheeles)
  • changed kubelet default image-gc-high-threshold to 85% to resolve a conflict with default settings in docker that prevented image garbage collection from resolving low disk space situations when using devicemapper storage. (#40432, @sjenning)
  • When creating a container using envFrom, (#42083, @fraenkel)
      1. validate the name of the ConfigMap in a ConfigMapRef
      1. validate the name of the Secret in a SecretRef
  • RBAC role and rolebinding auto-reconciliation is now performed only when the RBAC authorization mode is enabled. (#43813, @liggitt)
  • Permission to use a PodSecurityPolicy can now be granted within a single namespace by allowing the use verb on the podsecuritypolicies resource within the namespace. (#42360, @liggitt)
  • Enable audit log in local cluster (#42379, @xilabao)
  • Fix a deadlock in kubeadm master initialization. (#43835, @mikedanese)
  • Implement API usage metrics for gce storage (#40338, @gnufied)
  • kubeadm: clean up exited containers and network checkpoints (#43836, @yujuhong)
  • ActiveDeadlineSeconds is validated in workload controllers now, make sure it's not set anywhere (it shouldn't be set by default and having it set means your controller will restart the Pods at some point) (#38741, @sandflee)
  • azure: all clients poll duration is now 5 seconds (#43699, @colemickens)
  • addressing issue #39427 adding a flag --output to 'kubectl version' (#39858, @alejandroEsc)
  • Support secure etcd cluster for centos provider. (#42994, @Shawyeok)
  • Use Cluster Autoscaler 0.5.1, which fixes an issue in Cluster Autoscaler 0.5 where the cluster may be scaled up unnecessarily. Also the status of Cluster Autoscaler is now exposed in kube-system/cluster-autoscaler-status config map. (#43745, @mwielgus)
  • Use ProviderID to address nodes in the cloudprovider (#42604, @wlan0)
  • Openstack cinder v1/v2/auto API support (#40423, @mkutsevol)
  • API resource discovery now includes the singularName used to refer to the resource. (#43312, @deads2k)
  • Add the ability to lock on ConfigMaps to support HA for self hosted components (#42666, @timothysc)
  • OpenStack clusters can now specify whether worker nodes are assigned a floating IP (#42638, @jamiehannaford)
  • Add Host field to TCPSocketAction (#42902, @louyihua)
  • Support StorageClass in Azure file volume (#42170, @rootfs)
  • Be able to specify the timeout to wait for pod for kubectl logs/attach (#41813, @shiywang)
  • Add support for bring-your-own ip address for Services on Azure (#42034, @brendandburns)
    1. create configmap has a new option --from-env-file that populates a configmap from file which follows a key=val format for each line. (#38882, @fraenkel)
      1. create secret has a new option --from-env-file that populates a configmap from file which follows a key=val format for each line.
  • update the signing key for percona debian and ubuntu packages (#41186, @dixudx)
  • release-note-none (#41139, @juanvallejo)
  • fc: Drop multipath.conf snippet (#36698, @fabiand)

v1.6.1

Documentation & Examples

Downloads for v1.6.1

filename sha256 hash
kubernetes.tar.gz f1634e22ee3fe8af5c355c3f53d1b93f946490addfab029e19acf5317c51cd38
kubernetes-src.tar.gz b818f29661dd34db77d1e46c6ba98df6d35906dbc36ac1fdfe45f770b0f695c1

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 6eb7a0749de4c66d66630ac831f9a0aa73af9be856776c428d6adb3e07479d4a
kubernetes-client-darwin-amd64.tar.gz 05715224efdda0da3241960ec6cc71c2b008d3a53d217e5f90b159b5274db240
kubernetes-client-linux-386.tar.gz 7608a4754e48297dac8be9e863c429676f35afb97a9a10897e15038df6499a2e
kubernetes-client-linux-amd64.tar.gz 21e85cd3388b131fd1b63b06ea7ace8eef9555b7c558900b0cf1f9a3f2733e9a
kubernetes-client-linux-arm64.tar.gz b798e4b440df52e35809310147f8678a1d9b822dce85212fcf382d19ec2bd8c3
kubernetes-client-linux-arm.tar.gz 3227e745db3986a6be9c16c8ffb4f40ce604a400c680e2e6ff92368e72a597c3
kubernetes-client-linux-ppc64le.tar.gz ab7c9b2516d3cda8b4c236e00a179448e0787670cfd20c66dfb1b0c6c73ef0db
kubernetes-client-linux-s390x.tar.gz 9fbcb5f1b092573e5db5188689d7709a03b2bfdae635f61b5dbf72ae9dde2aaf
kubernetes-client-windows-386.tar.gz 306566c6903111769f01b0d05ba66aed63c133501adf49ef8daa38cc6a78097d
kubernetes-client-windows-amd64.tar.gz 5ca89e1672fd29a13a7cb997480216643e98afeba4d19ab081877281d0db8060

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 3e5c7103f44f20a95db29243a43f04aca731c8a4d411c80592ea49f7550d875c
kubernetes-server-linux-arm64.tar.gz 3fad77963f993396786e1777aecb770572b8b06cc3fe985c688916a70ffee2fd
kubernetes-server-linux-arm.tar.gz 4b981751da6a0bf471e52e55b548d62c038f7e6d8ed628b8177389f24cfd0434
kubernetes-server-linux-ppc64le.tar.gz 7b4bdf49cc2510af81205f2a65653a577fc79623c76c7ed3c29c2fbe1d835773
kubernetes-server-linux-s390x.tar.gz 3c55f1322ca39b7acb4914dd174978b015c1124e1ddd5431ec14c97b1b45f69f

Changelog since v1.6.0

Other notable changes

  • Fix a deadlock in kubeadm master initialization. (#43835, @mikedanese)
  • Use Cluster Autoscaler 0.5.1, which fixes an issue in Cluster Autoscaler 0.5 where the cluster may be scaled up unnecessarily. Also the status of Cluster Autoscaler is now exposed in kube-system/cluster-autoscaler-status config map. (#43745, @mwielgus)

v1.6.0

Documentation & Examples

Downloads for v1.6.0

filename sha256 hash
kubernetes.tar.gz e89318b88ea340e68c427d0aad701e544ce2291195dc1d5901222e7bae48f03b
kubernetes-src.tar.gz 0b03d27e1c7af3be5d94ecd5f679e5f55588d801376cf1ae170d9ec0a229b1e2

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 274277a67a85e9081d9fee5e763ed7c3dd096acf641c31a9ccc916a3981fead2
kubernetes-client-darwin-amd64.tar.gz af8d1aa034721b31fd14f7b93f5ef16f8dbac4fdcd9e312c3c61e6cf03295057
kubernetes-client-linux-386.tar.gz 4c6a3c12f131c3c88612f888257d00a3cc7fef67947757b897b0d8be9fab547c
kubernetes-client-linux-amd64.tar.gz 4a5daf0459dffc24bf0ccbb2fc881f688008e91ae41fde961b81d09b0801004c
kubernetes-client-linux-arm64.tar.gz 91d5e31407140a55cf00c0dc6e20aa8433cc918615dedd842637585e81694ebd
kubernetes-client-linux-arm.tar.gz 985fecd7fb94b42c25b8a56efde1831b2616a6792d3d5a02549248e01ce524ed
kubernetes-client-linux-ppc64le.tar.gz 303279f935289cadfb97a6a5d3f58b0da67d1b88b5ed049e6a98fc203b7b9d52
kubernetes-client-linux-s390x.tar.gz 2bd216c6b7d4f09de02c3b5d20021124f7d04f483ab600b291c515386003ca74
kubernetes-client-windows-386.tar.gz 11d5459b0d413a25045c55ce3548d30616ddc2d62451280d3299baa9f3e3e014
kubernetes-client-windows-amd64.tar.gz 84eba6f2b2b82a95397688b1e2ca4deb8d7daf1f8a40919fa0312741ca253799

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 3625b63d573aa4d28eaa30b291017f775f2ddc0523f40d25023ad1da9c30390e
kubernetes-server-linux-arm64.tar.gz 906496c985d4d836466b73e1c9e618ea8ce07f396aba3a96edcdc6045e0ab4e3
kubernetes-server-linux-arm.tar.gz 3b63f481156f57729bc8100566d8b3d7856791e5b36bb70042e17d21f11f8d5d
kubernetes-server-linux-ppc64le.tar.gz 382666b3892fd4d89be5e4bb052dd0ef0d1c1d213c1ae7a92435083a105064fd
kubernetes-server-linux-s390x.tar.gz e15de8896bd84a9b74756adc1a2e20c6912a65f6ff0a3f3dfabc8b463e31d19b

WARNING: etcd backup strongly recommended

Before updating to 1.6, you are strongly recommended to back up your etcd data. Please consult the installation procedure you are using (kargo, kops, kube-up, kube-aws, kubeadm etc) for specific advice.

1.6 encourages etcd3, and switching from etcd2 to etcd3 involves a full migration of data between different storage engines. You must stop the API from writing to etcd during an etcd2 -> etcd3 migration. HA installations cannot be migrated at the current time using the official Kubernetes procedure.

1.6 will also default to protobuf encoding if using etcd3. This change is irreversible. To rollback, you must restore from a backup made before the protobuf/etcd3 switch, and any changes since the backup will be lost. As 1.5 does not support protobuf encoding, if you roll back to 1.5 after upgrading to protobuf you will be forced to restore from backup, and you will lose any changes since you converted to protobuf. After conversion to protobuf, you should validate the correct operation of your cluster thoroughly before returning it to normal operation.

Backups are always a good precaution, particularly around upgrades, but this upgrade has multiple known issues where the only remedy is to restore from backup.

Also, please note:

  • using application/vnd.kubernetes.protobuf as the media storage type for 1.6 is default but not required

  • the ability to rollback to etcd2 can be preserved by setting the storage media type flag on kube-apiserver

    --storage-media-type application/json

    to continue to use application/json as the storage media type which can be changed to application/vnd.kubernetes.protobuf at a later time.

Major updates and release themes

  • Kubernetes now supports up to 5,000 nodes via etcd v3, which is enabled by default.
  • Role-based access control (RBAC) has graduated to beta, and defines secure default roles for control plane, node, and controller components.
  • The kubeadm cluster bootstrap tool has graduated to beta. Some highlights:
    • WARNING: A known issue in v1.6.0 causes kubeadm init to hang. Please use v1.6.1, which fixes the issue.
    • All communication is now over TLS
    • Authorization plugins can be installed by kubeadm, including the new default of RBAC
    • The bootstrap token system now allows token management and expiration
  • The kubefed federation bootstrap tool has also graduated to beta.
  • Interaction with container runtimes is now through the CRI interface, enabling easier integration of runtimes with the kubelet. Docker remains the default runtime via Docker-CRI (which moves to beta).
    • WARNING: A known issue in v1.6.0 causes Pod.Spec.HostPid (using the host PID namespace for the pod) to always be false. Please wait for v1.6.2, which will include a fix for this issue.
  • Various scheduling features have graduated to beta:
  • You can now specify (per pod) how long a pod should stay bound to a node, when there is a node problem.
  • Various storage features have graduated to GA:
  • DaemonSets can now be updated by a rolling update.

Action Required

Certificates API

  • Users of the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading. (#39772, @mikedanese)

Cluster Autoscaler

  • If you are using (or planning to use) Cluster Autoscaler please wait for Kubernetes 1.6.1. In 1.6.0 Cluster Autoscaler may occasionally increase the size of the cluster a bit more than it is actually needed, when there are unschedulable pods, scale up is required and cloud provider is slow to set up networking for new nodes. Anyway, the cluster should get back to the proper size after 10 min.

Deployment

Federation

  • The --dns-provider argument of 'kubefed init' is now mandatory and does not default to google-clouddns. To initialize a Federation control plane with Google Cloud DNS, use the following invocation: 'kubefed init --dns-provider=google-clouddns' (#42092, @marun)
  • Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data. (#37770, @enj)

Internal Storage Layer

  • upgrade to etcd3 prior to upgrading to 1.6 OR explicitly specify --storage-backend=etcd2 --storage-media-type=application/json when starting the apiserver

Node Components

  • Kubelet with the Docker-CRI implementation
    • The Docker-CRI implementation is enabled by default.
    • It is not compatible with containers created by older Kubelets. It is recommended to drain your node before upgrade. If you choose to perform an in-place upgrade, the Kubelet will automatically restart all Kubernetes-managed containers on the node.
    • It is not compatible with CNI plugins that do not conform to the error handling behavior in the spec. The plugins are being updated to resolve this issue (#43488). You can disable CRI explicitly (--enable-cri=false) as a temporary workaround.
      • The standard bridge plugin have been validated to interoperate with the new CRI + CNI code path.
      • If you are using plugins other than bridge, make sure you have updated custom plugins to the latest version that is compatible.
  • CNI plugins now affect node readiness
    • Kubelet will now block node readiness until a CNI configuration file is present in /etc/cni/net.d or a given custom CNI configuration path. This change ensures kubelet does not start pods that require networking before networking is ready. This change may require changes to clients that gate pod creation and/or scheduling on the node condition type Ready status being True for pods that need to run prior to the network being ready.
  • Enhance Kubelet QoS:
    • Pods are placed under a new cgroup hierarchy by default. This feature requires draining and restarting the node as part of upgrades. To opt-out set --cgroups-per-qos=false.
    • If kube-reserved and/or system-reserved are specified, node allocatable will be enforced on all pods by default. To opt-out set --enforce-node-allocatable=””
    • Hard Eviction Thresholds will be subtracted from Capacity while calculating Node Allocatable. This will result in a reduction of schedulable capacity in clusters post upgrade where kubelet hard eviction has been turned on for memory. To opt-out set --experimental-allocatable-ignore-eviction=true.
    • More details on these feature here: https://kubernetes.io/docs/concepts/cluster-administration/node-allocatable/
  • Drop the support for docker 1.9.x. Docker versions 1.10.3, 1.11.2, 1.12.6 have been validated.
  • The following deprecated kubelet flags are removed: --config, --auth-path, --resource-container, --system-container, --reconcile-cidr
  • Remove the temporary fix for pre-1.0 mirror pods. Upgrade directly from pre-1.0 to 1.6 kubelet is not supported.
  • Fluentd was migrated to Daemon Set, which targets nodes with beta.kubernetes.io/fluentd-ds-ready=true label. If you use fluentd in your cluster please make sure that the nodes with version 1.6+ contains this label.

kubectl

  • Running kubectl taint (alpha in 1.5) against a 1.6 server requires upgrading kubectl to version 1.6
  • Running kubectl taint (alpha in 1.5) against a 1.5 server requires a kubectl version of 1.5
  • Running kubectl create secret no longer accepts passing multiple values to a single --from-literal flag using comma separation
    • Update command invocations to pass separate --from-literal flags for each value

RBAC

  • Default ClusterRole and ClusterRoleBinding objects are automatically updated at server start to add missing permissions and subjects (extra permissions and subjects are left in place). To prevent autoupdating a particular role or rolebinding, annotate it with rbac.authorization.kubernetes.io/autoupdate=false. (#41155, @liggitt)
  • v1beta1 RoleBinding/ClusterRoleBinding subjects changed apiVersion to apiGroup to fully-qualify a subject. ServiceAccount subjects default to an apiGroup of "", User and Group subjects default to an apiGroup of "rbac.authorization.k8s.io". (#41184, @liggitt)
  • To create or update an RBAC RoleBinding or ClusterRoleBinding object, a user must: (#39383, @liggitt)
      1. Be authorized to make the create or update API request
      1. Be allowed to bind the referenced role, either by already having all of the permissions contained in the referenced role, or by having the "bind" permission on the referenced role.
  • The --authorization-rbac-super-user flag (alpha in 1.5) is deprecated; the system:masters group has privileged access (#38121, @deads2k)
  • special handling of the user * in RoleBinding and ClusterRoleBinding objects is removed in v1beta1. To match all users, explicitly bind to the group system:authenticated and/or system:unauthenticated. Existing v1alpha1 bindings to the user * are automatically converted to the group system:authenticated. (#38981, @liggitt)

Scheduling

  • Multiple schedulers

    • Modify your PodSpecs that currently use the scheduler.alpha.kubernetes.io/name annotation on Pod, to instead use the schedulerName field in the PodSpec.
    • Modify any custom scheduler(s) you have written so that they read the schedulerName field on Pod instead of the scheduler.alpha.kubernetes.io/name annotation.
    • Note that you can only start using the schedulerName field after you upgrade to 1.6; it is not recognized in 1.5.

  • Node affinity/anti-affinity and pod affinity/anti-affinity

    • You can continue to use the alpha version of this feature (with one caveat -- see below), in which you specify affinity requests using Pod annotations, in 1.6 by including AffinityInAnnotations=true in --feature-gates, such as --feature-gates=FooBar=true,AffinityInAnnotations=true. Otherwise, you must modify your PodSpecs that currently use the scheduler.alpha.kubernetes.io/affinity annotation on Pod, to instead use the affinity field in the PodSpec. Support for the annotation will be removed in a future release, so we encourage you to switch to using the field as soon as possible.
    • Caveat: The alpha version no longer supports, and the beta version does not support, the "empty podAffinityTerm.namespaces list means all namespaces" behavior. In both alpha and beta it now means "same namespace as the pod specifying this affinity rule."
    • Note that you can only start using the affinity field after you upgrade to 1.6; it is not recognized in 1.5.
    • The --failure-domains scheduler command line-argument is not supported in the beta version of the feature.

  • Taints

    • You will need to use kubectl taint to re-create all of your taints after kubectl and the master are upgraded to 1.6. Between the time the master is upgraded to 1.6 and when you do this, your existing taints will have no effect.
    • You can find out what taints you have in place on a node while you are still running Kubernetes 1.5 by doing kubectl describe node <node name>; the Taints section will show the taints you have in place. To see the taints that were created under 1.5 when you are running 1.6, do kubectl get node <node name> -o yaml and look for the "Annotation" section with the annotation key scheduler.alpha.kubernetes.io/taints
    • You can remove the "old" taints (stored internally as annotations) at any time after the upgrade by doing kubectl annotate nodes <node name> scheduler.alpha.kubernetes.io/taints- (note the minus at the end, which means "delete the taint with this key")
    • Note that because any taints you might have created with Kubernetes 1.5 can only affect the scheduling of new pods (the NoExecute taint effect is introduced in 1.6), neither the master upgrade nor your running kubectl taint to re-create the taints will affect pods that are already running.
    • Rescheduler relies on taints, which were changed in a backward incompatible way. Rescheduler 0.3 shipped together with Kubernetes 1.6 understands the new taints and will clean up the old annotations, but Rescheduler 0.2 shipped together with Kubernetes 1.5 doesn't understand the new taints. In order to avoid eviction loop during 1.5->1.6 upgrade you need to either upgrade the master node atomically (for example by using upgrade.sh script for GCE) or ensure that the rescheduler is upgraded first, then the scheduler and then all the other master components.

  • Tolerations

    • After your master is upgraded to 1.6, you will need to update your PodSpecs to set the tolerations field of the PodSpec and remove the scheduler.alpha.kubernetes.io/tolerations annotation on the Pod. (It's not strictly necessary to remove the annotation, as it will have no effect once you upgrade to 1.6.) Between the time the master is upgraded to 1.6 and when you do this, tolerations attached to Pods that are created will have no effect. Pods that are already running will not be affected by the upgrade.
    • You can find the PodSpec tolerations that were created as annotations (if any) in a controller definition by doing kubectl get <controller kind> <controller name> -o yaml and looking for the "Annotation" section with the annotation key scheduler.alpha.kubernetes.io/tolerations (This will work whether you are running Kubernetes 1.5 or 1.6).
    • To update a controller's PodSpec to use the field instead of the annotation, use one of the kubectl commands that do update ("kubectl replace" or "kubectl apply" or "kubectl patch") or just delete the controller entirely and re-create it with a new pod template. Note that you will be able to do these things only after the upgrade.

Service

  • The 'endpoints.beta.kubernetes.io/hostnames-map' annotation is no longer supported. Users can use the 'Endpoints.subsets[].addresses[].hostname' field instead. (#39284, @bowei)

StatefulSet

  • StatefulSet now respects ControllerRef to avoid fighting over Pods. At the time of upgrade, you must not have StatefulSets with selectors that overlap with any other controllers (such as ReplicaSets), or else ownership of Pods may change. (#42080, @enisoc)

Volumes

  • StorageClass pre-installed and set as default on Azure, AWS, GCE, OpenStack, and vSphere.
    • This is something to pay close attention to if you’ve been using Kubernetes for a while, because it changes the default behavior of PersistentVolumeClaim objects on these clouds.
    • Marking a StorageClass as default makes it so that even a PersistentVolumeClaim without a StorageClass specified will trigger dynamic provisioning (instead of binding to an existing pool of PVs).
    • If you depend on the old behavior of volumes binding to existing pool of PersistentVolume objects then modify the StorageClass object and set storageclass.beta.kubernetes.io/is-default-class to false.
  • Flex volume plugin is updated to support attach/detach interfaces. It broke backward compatibility. Please update your drivers and implement the new callouts. (#41804, @chakri-nelluri)

Notable Features

Features for this release were tracked via the use of the kubernetes/features issues repo. Each Feature issue is owned by a Special Interest Group from the kubernetes/community.

Autoscaling

  • [alpha] The Horizontal Pod Autoscaler now supports drawing metrics through the API server aggregator.
  • [alpha] The Horizontal Pod Autoscaler now supports scaling on multiple, custom metrics.
  • Cluster Autoscaler publishes its status to kube-system/cluster-autoscaler-status ConfigMap.
  • Cluster Autoscaler can continue operations while some nodes are broken or unready.
  • Cluster Autoscaler respects Pod Disruption Budgets when removing a node.

DaemonSet

Deployment

  • [beta] Deployments that cannot make progress in rolling out the newest version will now indicate via the API they are blocked (docs)

Federation

  • [beta] kubefed has graduated to beta: supports hosting federation on on-prem clusters, automatically configures kube-dns in joining clusters and allows passing arguments to federation components.

Internal Storage Layer

  • [stable] The internal storage layer for kubernetes cluster state has been updated to use etcd v3 by default. Existing clusters will have to plan for a data migration window. (docs)(kubernetes/features#44)

kubeadm

  • [beta] Introduces an API for clients to request TLS certificates from the API server. See the tutorial.
  • [beta] kubeadm is enhanced and improved with a baseline feature set and command line flags that are now marked as beta. Other parts of kubeadm, including subcommands under kubeadm alpha, are still in alpha. Using it is considered safe, although note that upgrades and HA are not yet supported. Please try it out and give us feedback!
  • [alpha] New Bootstrap Token authentication and management method. Works well with kubeadm. kubeadm now supports managing tokens, including time based expiration, after the cluster is launched. See kubeadm reference docs for details.
  • [alpha] Adds a new cloud-controller-manager binary that may be used for testing the new out-of-core cloudprovider flow.

Node Components

  • [stable] Init containers have graduated to GA and now appear as a field. The beta annotation value will still be respected and overrides the field value.
  • Kubelet Container Runtime Interface (CRI) support
    • [beta] The Docker-CRI implementation is enabled by default in kubelet. You can disable it by --enable-cri=false. See notes on the new implementation for more details.
    • [alpha] Alpha support for other runtimes: cri-o, frakti, rkt.
  • [beta] Node Problem Detector is beta (v0.3.0) now. New features added into node-problem-detector:v0.3.0:
    • Add support for journald.
    • The ability to monitor any system daemon logs. Previously only kernel logs were supported.
    • The ability to be deployed and run as a native daemon.
  • [alpha] Critical Pods: When feature gate "ExperimentalCriticalPodAnnotation" is set to true, the system will guarantee scheduling and admission of pods with the following annotation - scheduler.alpha.kubernetes.io/critical-pod
    • This feature should be used in conjunction with the rescheduler to guarantee resource availability for critical system pods.
  • [alpha] --experimental-nvidia-gpus flag is replaced by Accelerators alpha feature gate along with addition of support for multiple Nvidia GPUs.
    • To use GPUs, pass Accelerators=true as part of --feature-gates flag.
    • More information here.
  • A pod’s Quality of Service Class is now available in its Status.
  • Upgrade cAdvisor library to v0.25.0. Notable changes include,
    • Container filesystem usage tracking disabled for device mapper due to excessive IOPS.
    • Ignore .mount cgroups, fixing disappearing stats.
  • A new field terminationMessagePolicy has been added to containers that allows a user to request FallbackToLogsOnError, which will read from the container's logs to populate the termination message if the user does not write to the termination message log file. The termination message file is now properly readable for end users and has a maximum size (4k bytes) to prevent abuse. Each pod may have up to 12k bytes of termination messages before the contents of each will be truncated.
  • Do not delete pod objects until all its compute resource footprint has been reclaimed.
    • This feature prevents the node and scheduler from oversubscribing resources that are still being used by a pod in the process of being cleaned up.
    • This feature can result in increase of Pod Deletion latency especially if the pod has a large filesystem footprint.

RBAC

  • [beta] RBAC API is promoted to v1beta1 (rbac.authorization.k8s.io/v1beta1), and defines default roles for control plane, node, and controller components.
  • [beta] The Docker-CRI implementation is Beta and is enabled by default in kubelet. You can disable it by --enable-cri=false. See notes on the new implementation for more details.

Scheduling

  • [beta] The multiple schedulers. This feature allows you to run multiple schedulers in parallel, each responsible for different sets of pods. When using multiple schedulers, the scheduler name is now specified in a new-in-1.6 schedulerName field of the PodSpec rather than using the scheduler.alpha.kubernetes.io/name annotation on the Pod. When you upgrade to 1.6, the Kubernetes default scheduler will start using the schedulerName field of the PodSpec and will ignore the annotation.
  • [beta] Node affinity/anti-affinity and [beta] pod affinity/anti-affinity. Node affinity/anti-affinity allow you to specify rules for restricting which node(s) a pod can schedule onto, based on the labels on the node. Pod affinity/anti-affinity allow you to specify rules for spreading and packing pods relative to one another, across arbitrary topologies (node, zone, etc.) These affinity rules are now be specified in a new-in-1.6 affinity field of the PodSpec. Kubernetes 1.6 continues to support the alpha scheduler.alpha.kubernetes.io/affinity annotation on the Pod if you explicitly enable the alpha feature "AffinityInAnnotations", but it will be removed in a future release. When you upgrade to 1.6, if you have not enabled the alpha feature, then the scheduler will use the affinity field in PodSpec and will ignore the annotation. If you have enabled the alpha feature, then the scheduler will use the affinity field in PodSpec if it is present, and otherwise will use the annotation.
  • [beta] Taints and tolerations. This feature allows you to specify rules for "repelling" pods from nodes by default, which enables use cases like dedicated nodes and reserving nodes with special features for pods that need those features. We've also added a NoExecute taint type that evicts already-running pods, and an associated tolerationSeconds field to tolerations to delay the eviction for a specified amount of time. As before, taints are created using kubectl taint (but internally they are now represented as a field taints in the NodeSpec rather than using the scheduler.alpha.kubernetes.io/taints annotation on Node). Tolerations are now specified in a new-in-1.6 tolerations field of the PodSpec rather than using the scheduler.alpha.kubernetes.io/tolerations annotation on the Pod. When you upgrade to 1.6, the scheduler will start using the fields and will ignore the annotations.
  • [alpha] Represent node problems "not ready" and "unreachable" using NoExecute taints. In combination with tolerationSeconds described below, this allows per-pod specification of how long to remain bound to a node that becomes unreachable or not ready, rather than using the default of 5 minutes. You can enable this alpha feature by including TaintBasedEvictions=true in --feature-gates, such as --feature-gates=FooBar=true,TaintBasedEvictions=true. Documentation is here.

Service Catalog

  • [alpha] Adds a new API resource PodPreset and admission controller to enable defining cross-cutting injection of Volumes and Environment into Pods.

Volumes

  • [stable] StorageClass API is promoted to v1 (storage.k8s.io/v1).
  • [stable] Default storage classes are deployed during installation on Azure, AWS, GCE, OpenStack and vSphere.
  • [stable] Added ability to populate environment variables from a configmap or secret.
  • [stable] Support for user-written/run dynamic PV provisioners. The Kubernetes Incubator contains a golang library and examples.
  • [stable] Volume plugin for ScaleIO enabling pods to seamlessly access and use data stored on Dell EMC ScaleIO volumes.
  • [stable] Volume plugin for Portworx added capability to use Portworx as a storage provider for Kubernetes clusters. Portworx pools server capacity and turns servers or cloud instances into converged, highly available compute and storage nodes.
  • [stable] Add support to use NFSv3, NFSv4, and GlusterFS on GCE/GKE GCI image based clusters.
  • [beta] Added support for mount options in persistent volumes.
  • [alpha] All in one volume proposal - a new volume driver capable of projecting secrets, configmaps, and downward API items into the same directory.
  • [alpha] Flex volume API and Improved lifecycle (flexvolume).

Deprecations

  • Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator. (#38614, @soltysh)
  • federation/deploy/deploy.sh was an interim solution introduced in Kubernetes v1.4 to simplify the federation control plane deployment experience. Now that we have kubefed, we are deprecating deploy.sh scripts. (#38902, @madhusudancs)

Cluster Provisioning Scripts

kubeadm

  • Quite a few flags been renamed or removed. Those options that are removed as flags can still be accessed via the config file. Most noteably this includes external etcd settings and the option for setting the cloud provider on the API server. The kubeadm reference documentation is up to date with the new flags.

Other Deprecations

  • Remove cmd/kube-discovery from the tree since it's not necessary anymore (#42070, @luxas)

Changes to API Resources

ABAC

  • ABAC policies using "user":"*" or "group":"*" to match all users or groups will only match authenticated requests. To match unauthenticated requests, ABAC policies must explicitly specify "group":"system:unauthenticated" (#38968, @liggitt)

Admission Control

  • Adds a new API resource PodPreset and admission controller to enable defining cross-cutting injection of Volumes and Environment into Pods. (#41931, @jessfraz)
  • Enable DefaultTolerationSeconds admission controller by default (#41815, @kevin-wangzefeng)
  • ResourceQuota ability to support default limited resources (#36765, @derekwaynecarr)
  • Add defaultTolerationSeconds admission controller (#41414, @kevin-wangzefeng)
  • An automountServiceAccountToken field was added to ServiceAccount and PodSpec objects. If set to false on a pod spec, no service account token is automounted in the pod. If set to false on a service account, no service account token is automounted for that service account unless explicitly overridden in the pod spec. (#37953, @liggitt)
  • Admission control support for versioned configuration files (#39109, @derekwaynecarr)
  • Ability to quota storage by storage class (#34554, @derekwaynecarr)

Authentication

  • The authentication.k8s.io API group was promoted to v1(#41058, @liggitt)

Authorization

  • The authorization.k8s.io API group was promoted to v1 (#40709, @liggitt)
  • The SubjectAccessReview API now passes subresource and resource name information to the authorizer to answer authorization queries. (#40935, @liggitt)

Autoscaling

  • Introduces an new alpha version of the Horizontal Pod Autoscaler including expanded support for specifying metrics. (#36033, @DirectXMan12
  • HorizontalPodAutoscaler is no longer supported in extensions/v1beta1 version. Use autoscaling/v1 instead. (#35782, @piosz)
  • Fixes an HPA-related panic due to division-by-zero. (#39694, @DirectXMan12)

Certificates

  • The CertificateSigningRequest API added the extra field to persist all information about the requesting user. This mirrors the fields in the SubjectAccessReview API used to check authorization. (#41755, @liggitt)
  • Native support for token based bootstrap flow. This includes signing a well known ConfigMap in the kube-public namespace and cleaning out expired tokens. (#36101, @jbeda)

ConfigMap

  • Volumes and environment variables populated from ConfigMap and Secret objects can now tolerate the named source object or specific keys being missing, by adding optional: true to the volume or environment variable source specifications. (#39981, @fraenkel)
  • Allow pods to define multiple environment variables from a whole ConfigMap (#36245, @fraenkel)

CronJob

  • Add configurable limits to CronJob resource to specify how many successful and failed jobs are preserved. (#40932, @peay)

DaemonSet

  • DaemonSet now respects ControllerRef to avoid fighting over Pods. (#42173, @enisoc)
  • Make DaemonSet respect critical pods annotation when scheduling. (#42028, @janetkuo)
  • Implement the update feature for DaemonSet. (#41116, @lukaszo)
  • Make DaemonSets survive taint-based evictions when nodes turn unreachable/notReady. (#41896, @kevin-wangzefeng)
  • Make DaemonSet controller respect node taints and pod tolerations. (#41172, @janetkuo)
  • DaemonSet controller actively kills failed pods (to recreate them) (#40330, @janetkuo)
  • DaemonSet ObservedGeneration (#39157, @lukaszo)

Deployment

  • Add ready replicas in Deployments (#37959, @kargakis)
  • Deployments that cannot make progress in rolling out the newest version will now indicate via the API they are blocked
  • Introduce apps/v1beta1.Deployments resource with modified defaults compared to extensions/v1beta1.Deployments. (#39683, @soltysh)
  • Introduce new generator for apps/v1beta1 deployments (#42362, @soltysh)

Node

Pod

  • Init containers have graduated to GA and now appear as a field. The beta annotation value will still be respected and overrides the field value. (#38382, @hodovska)
  • A new field terminationMessagePolicy has been added to containers that allows a user to request FallbackToLogsOnError, which will read from the container's logs to populate the termination message if the user does not write to the termination message log file. The termination message file is now properly readable for end users and has a maximum size (4k bytes) to prevent abuse. Each pod may have up to 12k bytes of termination messages before the contents of each will be truncated. (#39341, @smarterclayton)
  • Fix issue with PodDisruptionBudgets in which minAvailable specified as a percentage did not work with StatefulSet Pods. (#39454, @foxish)
  • Node affinity has moved from annotations to api fields in the pod spec. Node affinity that is defined in the annotations will be ignored. (#37299, @rrati)
  • Moved pod affinity and anti-affinity from annotations to api fields #25319 (#39478, @rrati)
  • Add QoS pod status field (#37968, @sjenning)

Pod Security Policy

  • PodSecurityPolicy resource is now enabled by default in the extensions API group. (#39743, @pweil-)

RBAC

  • The attributeRestrictions field has been removed from the PolicyRule type in the rbac.authorization.k8s.io/v1alpha1 API. The field was not used by the RBAC authorizer. (#39625, @deads2k)
  • A user can now be authorized to bind a particular role by having permission to perform the bind verb on the referenced role (#39383, @liggitt)

ReplicaSet

  • ReplicaSet has onwer ref of the Deployment that created it (#35676, @krmayankk)

Secrets

  • Populate environment variables from a secrets. (#39446, @fraenkel)
  • Added a new secret type "bootstrap.kubernetes.io/token" for dynamically creating TLS bootstrapping bearer tokens. (#41281, @ericchiang)

Service

  • Endpoints, that tolerate unready Pods, are now listing Pods in state Terminating as well (#37093, @simonswine)
  • Fix Service Update on LoadBalancerSourceRanges Field (#37720, @freehan)
  • Bug fix. Incoming UDP packets not reach newly deployed services (#32561, @zreigz)
  • Services of type loadbalancer consume both loadbalancer and nodeport quota. (#39364, @zhouhaibing089)

StatefulSet

  • Fix zone placement heuristics so that multiple mounts in a StatefulSet pod are created in the same zone (#40910, @justinsb)
  • Fixes issue #38418 which, under circumstance, could cause StatefulSet to deadlock. (#40838, @kow3ns)
    • Mediates issue #36859. StatefulSet only acts on Pods whose identity matches the StatefulSet, providing a partial mediation for overlapping controllers.

Taints and Tolerations

  • Add a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag. (#40355, @gmarek)
  • Add an alpha feature that makes NodeController set Taints instead of deleting Pods from not Ready Nodes. (#41133, @gmarek)
  • Make tolerations respect wildcard key (#39914, @kevin-wangzefeng)
  • Forgiveness alpha version api definition (#39469, @kevin-wangzefeng)
  • Rescheduler uses taints in v1beta1 and will remove old ones (in version v1alpha1) right after its start. (#43106, @piosz)

Volumes

  • StorageClassName attribute has been added to PersistentVolume and PersistentVolumeClaim objects and should be used instead of annotation volume.beta.kubernetes.io/storage-class. The beta annotation is still working in this release, however it will be removed in a future release. (#42128, @jsafrane)
  • Parameter keys in a StorageClass parameters map may not use the kubernetes.io or k8s.io namespaces. (#41837, @liggitt)
  • Add storage.k8s.io/v1 API (#40088, @jsafrane)
  • Alpha version of dynamic volume provisioning is removed in this release. Annotation (#40000, @jsafrane)
  • Reduce verbosity of volume reconciler when attaching volumes (#36900, @codablock)
  • We change the default attach_detach_controller sync period to 1 minute to reduce the query frequency through cloud provider to check whether volumes are attached or not. (#41363, @jingxu97)

Changes to Major Components

API Server

  • --anonymous-auth is enabled by default, unless the API server is started with the AlwaysAllow authorizer. (#38706, @deads2k)
  • When using OIDC authentication and specifying --oidc-username-claim=email, an "email_verified":true claim must be returned from the identity provider. (#36087, @ericchiang)
    • --basic-auth-file supports optionally specifying groups in the fourth column of the file (#39651, @liggitt)
  • API server now has two separate limits for read-only and mutating inflight requests. (#36064, @gmarek)
  • Restored normalization of custom --etcd-prefix when --storage-backend is set to etcd3 (#42506, @liggitt)
  • Updating apiserver to return http status code 202 for a delete request when the resource is not immediately deleted because of user requesting cascading deletion using DeleteOptions.OrphanDependents=false. (#41165, @nikhiljindal)
  • Use full package path for definition name in OpenAPI spec (#40124, @mbohlool)
  • Follow redirects for streaming requests (exec/attach/port-forward) in the apiserver by default (alpha -> beta). (#40039, @timstclair)
  • Add 'X-Content-Type-Options: nosniff" to some error messages (#37190, @brendandburns)
  • Fixes bug in resolving client-requested API versions (#38533, @DirectXMan12)
  • Replace glog.Fatals with fmt.Errorfs (#38175, @sttts)
  • Pipe get options to storage (#37693, @wojtek-t)
  • The --long-running-request-regexp flag to kube-apiserver is deprecated and will be removed in a future release. Long-running requests are now detected based on specific verbs (watch, proxy) or subresources (proxy, portforward, log, exec, attach). (#38119, @liggitt)
  • if kube-apiserver is started with --storage-backend=etcd2, the media type application/json is used. (#43122, @liggitt)
  • API fields that previously serialized null arrays as null and empty arrays as [] no longer distinguish between those values and always output [] when serializing to JSON. (#43422, @liggitt)
  • Generate OpenAPI definition for inlined types (#39466, @mbohlool)

API Server Aggregator

Generic API Server

  • Move pkg/api/rest into genericapiserver (#39948, @sttts)
  • Move non-generic apiserver code out of the generic packages (#38191, @sttts)
  • Fixes API compatibility issue with empty lists incorrectly returning a null items field instead of an empty array. (#39834, @liggitt)
  • Re-add /healthz/ping handler in genericapiserver (#38603, @sttts)
  • Remove genericapiserver.Options.MasterServiceNamespace (#38186, @sttts)
  • genericapiserver: cut off more dependencies – episode 3 (#40426, @sttts)
  • genericapiserver: more dependency cutoffs (#40216, @sttts)
  • genericapiserver: cut off kube pkg/version dependency (#39943, @sttts)
  • genericapiserver: cut off pkg/serviceaccount dependency (#39945, @sttts)
  • genericapiserver: cut off pkg/apis/extensions and pkg/storage dependencies (#39946, @sttts)
  • genericapiserver: cut off certificates api dependency (#39947, @sttts)
  • genericapiserver: extract CA cert from server cert and SNI cert chains (#39022, @sttts)
  • genericapiserver: turn APIContainer.SecretRoutes into a real ServeMux (#38826, @sttts)
  • genericapiserver: unify swagger and openapi in config (#38690, @sttts)

Client

  • Use Prometheus instrumentation conventions (#36704, @fabxc)
  • Clients now use the ?watch=true parameter to make watch API calls, instead of the /watch/ path prefix (#41722, @liggitt)
  • Move private key parsing from serviceaccount/jwt.go to client-go/util/cert (#40907, @cblecker)
  • Caching added to the OIDC client auth plugin to fix races and reduce the time kubectl commands using this plugin take by several seconds. (#38167, @ericchiang)
  • Fix resync goroutine leak in ListAndWatch (#35672, @tatsuhiro-t)

client-go

Cloud Provider

AWS

  • Allow to running the master with a different AWS account or even on a different cloud provider than the nodes. (#39996, @scheeles)
  • Support shared tag kubernetes.io/cluster/<clusterid> (#41695, @justinsb)
  • Do not consider master instance zones for dynamic volume creation (#41702, @justinsb)
  • Fix AWS device allocator to only use valid device names (#41455, @gnufied)
  • Trust region if found from AWS metadata (#38880, @justinsb)
  • Remove duplicate calls to DescribeInstance during volume operations (#39842, @gnufied)
  • Recognize eu-west-2 region (#38746, @justinsb)
  • Recognize ca-central-1 region (#38410, @justinsb)
  • Add sequential allocator for device names. (#38818, @jsafrane)

Azure

GCE

  • On GCI by default logrotate is disabled for application containers in favor of rotation mechanism provided by docker logging driver. (#40634, @crassirostris)

GKE

vSphere

  • Reverts to looking up the current VM in vSphere using the machine's UUID, either obtained via sysfs or via the vm-uuid parameter in the cloud configuration file. (#40892, @robdaemon)
  • Fix for detach volume when node is not present/ powered off (#40118, @BaluDontu)
  • Adding vmdk file extension for vmDiskPath in vsphere DeleteVolume (#40538, @divyenpatel)
  • Changed default scsi controller type in vSphere Cloud Provider (#38426, @abrarshivani)
  • Fixes NotAuthenticated errors that appear in the kubelet and kube-controller-manager due to never logging in to vSphere (#36169, @robdaemon)
  • Fix panic in vSphere cloud provider (#38423, @BaluDontu)
  • Fix space issue in volumePath with vSphere Cloud Provider (#38338, @BaluDontu)

Federation

kubefed

Other Notable Changes

  • Federated Ingress over GCE no longer requires separate firewall rules to be created for each cluster to circumvent flapping firewall health checks. (#41942, @csbell)
  • Add support for finalizers in federated configmaps (deletes configmaps from underlying clusters). (#40464, @csbell)
  • Add support for DeleteOptions.OrphanDependents for federated services. Setting it to false while deleting a federated service also deletes the corresponding services from all registered clusters. (#36390, @nikhiljindal)
  • Add --controllers flag to federation controller manager for enable/disable federation ingress controller (#36643, @kzwang)
  • Stop deleting services from underlying clusters when federated service is deleted. (#37353, @nikhiljindal)
  • Expose autoscaling apis through federation api server (#38976, @irfanurrehman)
  • Federation: Add batch/jobs API objects to federation-apiserver (#35943, @jianhuiz)
  • Add logging of route53 calls (#39964, @justinsb)

Garbage Collector

  • Added foreground garbage collection: the owner object will not be deleted until all its dependents are deleted by the garbage collector. Please checkout the user doc for details. (#38676, @caesarxuchao)
    • deleteOptions.orphanDependents is going to be deprecated in 1.7. Please use deleteOptions.propagationPolicy instead.

kubeadm

  • A new label and taint is used for marking the master. The label is node-role.kubernetes.io/master="" and the taint has the effect NoSchedule. Tolerate the node-role.kubernetes.io/master="":NoSchedule taint to schedule a workload on the master (a networking DaemonSet for example).
  • The kubelet API is now secured, only cluster admins are allowed to access it.
  • Insecure access to the API Server over localhost:8080 is now disabled.
  • The control plane components now talk securely to each other. The API Server talks securely to the kubelets in the cluster.
  • kubeadm creates RBAC-enabled clusters. This means that some add-ons which worked previously won't work without having their YAML configuration updated. Please see each vendor's own documentation to check that the add-ons you are using will work with Kubernetes 1.6.
  • The kube-discovery Deployment isn't used anymore when creating a kubeadm cluster and shouldn't be used anywhere else either due to its insecure nature.
  • The Certificates API has graduated from alpha to beta. This is a backwards-incompatible change since the alpha support is dropped, and therefore kubeadm v1.5 and v1.6 don't work together (for example kubeadm v1.5 can't join a kubeadm v1.6 cluster).
  • Supporting only etcd3, with 3.0.14 as the minimum version.
  • kubeadm reset will no longer drain nodes automatically. This is because the credentials on nodes do not have permission to perform this operation. We have documented an alternate procedure, driven from the API server/master.
  • Hook up kubeadm against the BootstrapSigner (#41417, @luxas)
  • Rename some flags for beta UI and fixup some logic (#42064, @luxas)
  • Insecure access to the API Server at localhost:8080 will be turned off in v1.6 when using kubeadm (#42066, @luxas)
  • Flag --use-kubernetes-version for kubeadm init renamed to --kubernetes-version (#41820, @kad)
  • Remove the --cloud-provider flag for beta init UX (#41710, @luxas)
  • Fixed an SELinux issue in kubeadm on Docker 1.12+ by moving etcd SELinux options from container to pod. (#40682, @dgoodwin)
  • Add authorization mode to kubeadm (#39846, @andrewrynhard)
  • Refactor the certificate and kubeconfig code in the kubeadm binary into two phases (#39280, @luxas)
  • Added kubeadm commands to manage bootstrap tokens and the duration they are valid for. (#35805, @dgoodwin)

kubectl

New Commands

  • apply set-last-applied updates the applied-applied-configuration annotation (#41694, @shiywang)
  • kubectl apply view-last-applied viewing the last configuration file applied (#41146, @shiywang)

Create subcommands

Updates to existing commands

  • Printing and output
    • Import a numeric ordering sorting library and use it in the sorting printer. (#40746, @matthyx)
    • DaemonSet get and describe show status fields. (#42843, @janetkuo)
    • Pods describe shows tolerationSeconds (#42162, @kevin-wangzefeng)
    • Node describe contains closing paren (#39217, @luksa)
    • Display pod node selectors with kubectl describe. (#36396, @aveshagarwal)
    • Add Version to the resource printer for 'get nodes' (#37943, @ailusazh)
    • Added support for printing in all supported --output formats to kubectl create ... and kubectl apply ... (#38112, @juanvallejo)
    • Add three more columns to kubectl get deploy -o wide output. (#39240, @xingzhou)
    • Fix kubectl get -f -o so it prints all items in the file (#39038, @ncdc)
    • kubectl describe no longer prints the last-applied-configuration annotation for secrets. (#34664, @ymqytw)
    • Completed pods should not be hidden when requested by name via kubectl get. (#42216, @smarterclayton)
    • Improve formatting of EventSource in kubectl get and kubectl describe (#40073, @matthyx)
  • Attach now supports multiple types (#40365, @shiywang)
  • Create now accepts the label selector flag for filtering objects to create (#40057, @MrHohn)
  • Top now accepts short forms for "node" and "pod" ("no", "po") (#39218, @luksa)
  • Begin paths for internationalization in kubectl (#36802, @brendandburns)

Updates to apply

  • New command apply set-last-applied updates the applied-applied-configuration annotation (#41694, @shiywang)
  • New command apply view-last-applied command for viewing the last configuration file applied (#41146, @shiywang)
  • apply now supports explicitly clearing values by setting them to null (#40630, @liggitt)
  • Warn user when using apply on a resource lacking the LastAppliedConfig annotation (#36672, @ymqytw)
  • PATCH (i.e. apply and edit) now supports merging lists of primitives (#38665, @ymqytw)
  • apply falls back to generic 3-way JSON merge patch for Third Party Resource or unregistered types (#40666, @ymqytw)

Updates to edit

  • edit now supports Third party resources and extension API servers. (#41304, @liggitt)
    • Now to edit a particular API version, provide the fully-qualify the resource, version, and group used to fetch the object (for example, job.v1.batch/myjob)
    • Client-side conversion is no longer done, and the --output-version option is deprecated for kubectl edit.
  • edit works as intended with apply and will not change the annotation
    • No longer updates the last-applied-configuration annotation when --save-config is unspecified or false. (#41924, @ymqytw)
    • Fixes issue that caused apply to revert changes made by edit

Bug fixes

  • Fixed --save-config in create subcommand to save the annotation (#40289, @xilabao)
  • Fixed an issue where 'kubectl get --sort-by=' would return an error if the specified fields in sort were not specified in one or more of the returned objects. (#40541, @fabianofranz)
    • Previously this would cause the command to fail regardless of whether or not the field was present in the object model
    • Now the command will succeed even if the sort-by field is missing from one or more of the objects
  • Fixed issue with kubectl proxy so it will now proxy an empty path - e.g. http://localhost:8001 (#39226, @luksa)
  • Fixed an issue where commas were not accepted in --from-literal flags for the creation of secrets. (#35191, @SamiHiltunen)
    • Passing multiple values separated by a comma in a single --from-literal flag is no longer supported. Please use multiple --from-literal flags to provide multiple values.
  • Fixed a bug where the --server, --token, and --certificate-authority flags were not overriding the related in-cluster configs when provided in a kubectl call inside a cluster. (#39006, @fabianofranz)

Other Notable Changes

  • The api server will publish the extensions/Deployments API as preferred over the apps/Deployment (introduced in 1.6). (#43553, @liggitt)
    • This will ensure certain commands in 1.5 versions of kubectl continue function when run against a 1.6 server. (e.g. kubectl edit deployment)
  • Taint
    • The taint command will not function in a skewed 1.5 / 1.6 environment - client and server versions must match (See Action required section above)
    • Change taints/tolerations to api fields (#38957, @aveshagarwal)
    • Make kubectl taint command respect effect NoExecute (#42120, @kevin-wangzefeng)
  • Allow drain --force to remove pods whose managing resource is deleted. (#41864, @marun)
  • --output-version is ignored for all commands except kubectl convert. This is consistent with the generic nature of kubectl CRUD commands and the previous removal of --api-version. Specific versions can be specified in the resource field: resource.version.group, jobs.v1.batch. (#41576, @deads2k)
  • Allow missing keys in templates by default (#39486, @ncdc)
  • Add error message when trying to use clusterrole with namespace in kubectl (#36424, @xilabao)
  • When deleting an object with --grace-period=0, the client will begin a graceful deletion and wait until the resource is fully deleted. To force deletion, use the --force flag. (#37263, @smarterclayton)

Node Components

  • Kubelet config should ignore file start with dots. (#39196, @resouer)
  • Bump GCI to gci-stable-56-9000-84-2. (#41819, @dchen1107)
  • Bump GCE ContainerVM to container-vm-v20170214 to address CVE-2016-9962. (#41449, @zmerlynn)
  • Kubelet: Remove the PLEG health check from /healthz, Kubelet will now report
  • NodeNotReady on failed PLEG health check. (#41569, @yujuhong)
  • CRI: upgrade protobuf to v3. Protobuf v2 and v3 are not compatible. (#39158, @feiskyer)
  • kubelet exports metrics for cgroup management (#41988, @sjenning)
  • Experimental support to reserve a pod's memory request from being utilized by pods in lower QoS tiers. (#41149, @sjenning)
  • Port forwarding can forward over websockets or SPDY. (#33684, @fraenkel)
  • Flag gate faster evictions based on node memory pressure using kernel memcg notifications - --experimental-kernel-memcg-notification. (#38258, @derekwaynecarr)
  • Nodes can now report two additional address types in their status: InternalDNS and ExternalDNS. The apiserver can use --kubelet-preferred-address-types to give priority to the type of address it uses to reach nodes. (#34259, @liggitt)

Bug fixes

kube-controller-manager

kube-dns

  • Adds support for configurable DNS stub domains and upstream nameservers. The following configuration options have been added to the kube-system:kube-dns ConfigMap:

    "stubDomains": {
  "acme.local": ["1.2.3.4"]
},

    is a map of domain to list of nameservers for the domain. This is used to inject private DNS domains into the kube-dns namespace. In the above example, any DNS requests for *.acme.local will be served by the nameserver 1.2.3.4.

    "upstreamNameservers": ["8.8.8.8", "8.8.4.4"]

    is a list of upstreamNameservers to use, overriding the configuration specified in /etc/resolv.conf.

  • An empty kube-system:kube-dns ConfigMap will be created for the cluster if one did not already exist.

kube-proxy

  • - Add tcp/udp userspace proxy support for Windows. (#41487, @anhowe)
  • Add DNS suffix search list support in Windows kube-proxy. (#41618, @JiangtianLi)
  • Add a KUBERNETES_NODE_* section to build kubelet/kube-proxy for windows (#38919, @brendandburns)
  • Remove outdated net.experimental.kubernetes.io/proxy-mode and net.beta.kubernetes.io/proxy-mode annotations from kube-proxy. (#40585, @cblecker)
  • proxy/iptables: don't sync proxy rules if services map didn't change (#38996, @dcbw)
  • Update kube-proxy image to be based off of Debian 8.6 base image. (#39695, @ixdy)
  • Update amd64 kube-proxy base image to debian-iptables-amd64:v5 (#39725, @ixdy)
  • Clean up the kube-proxy container image by removing unnecessary packages and files. (#42090, @timstclair)
  • Better compat with very old iptables (e.g. CentOS 6) (#37594, @thockin)

Scheduler

  • Add the support to the scheduler for spreading pods of StatefulSets. (#41708, @bsalamat)
  • Added support to minimize sending verbose node information to scheduler extender by sending only node names and expecting extenders to cache the rest of the node information (#41119, @sarat-k)
  • Support KUBE_MAX_PD_VOLS on Azure (#41398, @codablock)
  • Mark multi-scheduler graduation to beta and then v1. (#38871, @k82cn)
  • Scheduler treats StatefulSet pods as belonging to a single equivalence class. (#39718, @foxish)
  • Update FitError as a message component into the PodConditionUpdater. (#39491, @jayunit100)
  • Fix comment and optimize code (#38084, @tanshanshan)
  • Add flag to enable contention profiling in scheduler. (#37357, @gmarek)
  • Try self-repair scheduler cache or panic (#37379, @wojtek-t)

Volume Plugins

Azure Disk

GlusterFS

  • The glusterfs dynamic volume provisioner will now choose a unique GID for new persistent volumes from a range that can be configured in the storage class with the "gidMin" and "gidMax" parameters. The default range is 2000 - 2147483647 (max int32). (#37886, @obnoxxx)

Photon

  • Fix photon controller plugin to construct with correct PdID (#37167, @luomiao)

rbd

  • force unlock rbd image if the image is not used (#41597, @rootfs)

vSphere

Other Notable Changes

  • Implement bulk polling of volumes (#41306, @gnufied)
  • Check if pathExists before performing Unmount (#39311, @rkouj)
  • Unmount operation should not fail if volume is already unmounted (#38547, @rkouj)
  • Provide kubernetes-controller-manager flags to control volume attach/detach reconciler sync. The duration of the syncs can be controlled, and the syncs can be shut off as well. (#39551, @chrislovecnm)
  • Fix unmountDevice issue caused by shared mount in GCI (#38411, @jingxu97)
  • Fix permissions when using fsGroup (#37009, @sjenning)
  • Fixed issues (#39202), (#41041) and (#40941) that caused the iSCSI connections to be prematurely closed when deleting a pod with an iSCSI persistent volume attached and that prevented the use of newly created LUNs on targets with preestablished connections. (#41196), @CristianPop)

Changes to Cluster Provisioning Scripts

AWS

Juju

  • The kubernetes-master, kubernetes-worker and kubeapi-load-balancer charms have gained an nrpe-external-master relation, allowing the integration of their monitoring in an external Nagios server. (#41923, @Cynerva)
  • Disable anonymous auth on kubelet (#41919, @Cynerva)
  • Fix shebangs in charm actions to use python3 (#42058, @Cynerva)
  • K8s master charm now properly keeps distributed master files in sync for an HA control plane. (#41351, @chuckbutler)
  • Improve status messages (#40691, @Cynerva)
  • Splits Juju Charm layers into master/worker roles (#40324, @chuckbutler)
    • Adds support for 1.5.x series of Kubernetes
    • Introduces a tactic for keeping templates in sync with upstream eliminating template drift
    • Adds CNI support to the Juju Charms
    • Adds durable storage support to the Juju Charms
    • Introduces an e2e Charm layer for repeatable testing efforts and validation of clusters

libvirt CoreOS

GCE

  • the gce provider enables both RBAC authorization and the permissive legacy ABAC policy that makes all service accounts superusers. To opt out of the permissive ABAC policy, export the environment variable ENABLE_LEGACY_ABAC=false before running cluster/kube-up.sh. (#43544, @liggitt)
  • the gce provider ensures the bootstrap admin token user is included in the super-user group (#39537, @liggitt)
  • Remove support for debian masters in GCE kube-up. (#41666, @mikedanese)
  • Remove support for trusty in GCE kube-up. (#41670, @mikedanese)
  • Don't fail if the grep fails to match any resources (#41933, @ixdy)
  • Fix the output of health-mointor.sh (#41525, @yujuhong)
  • Added configurable etcd initial-cluster-state to kube-up script. (#41332, @jszczepkowski)
  • The kube-apiserver basic audit log can be enabled in GCE by exporting the environment variable ENABLE_APISERVER_BASIC_AUDIT=true before running cluster/kube-up.sh. This will log to /var/log/kube-apiserver-audit.log and use the same logrotate settings as /var/log/kube-apiserver.log. (#41211, @enisoc)
  • On kube-up.sh clusters on GCE, kube-scheduler now contacts the API on the secured port. (#41285, @liggitt)
  • Use existing ABAC policy file when upgrading GCE cluster (#40172, @liggitt)
  • Ensure the GCI metadata files do not have newline at the end (#38727, @Amey-D)
  • Fixed detection of master during creation of multizone nodes cluster by kube-up. (#38617, @jszczepkowski)
  • Fixed validation of multizone cluster for GCE (#38695, @jszczepkowski)
  • Fix GCI mounter issue (#38124, @jingxu97)
  • Exit with error if is not the final parameter. (#37723, @mtaufen)
  • GCI: Remove /var/lib/docker/network (#37593, @yujuhong)
  • Fix the equality checks for numeric values in cluster/gce/util.sh. (#37638, @roberthbailey)
  • Modify GCI mounter to enable NFSv3 (#37582, @jingxu97)
  • Use gsed on the Mac (#37562, @roberthbailey)
  • Bump GCI
  • to gci-beta-56-9000-80-0 (#41027, @dchen1107)
  • to gci-stable-56-9000-84-2 (#41819, @dchen1107)
  • Bump GCE ContainerVM

OpenStack

  • Do not daemonize salt-minion for the openstack-heat provider. (#40722, @micmro)
  • OpenStack-Heat will now look for an image named "CentOS-7-x86_64-GenericCloud-1604". To restore the previous behavior set OPENSTACK_IMAGE_NAME="CentOS7" (#40368, @sc68cal)
  • Fixes a bug in the OpenStack-Heat kubernetes provider, in the handling of differences between the Identity v2 and Identity v3 APIs (#40105, @sc68cal)

Container Images

  • Update gcr.io/google-containers/rescheduler to v0.2.2, which uses busybox as a base image instead of ubuntu. (#41911, @ixdy)
  • Remove unnecessary metrics (http/process/go) from being exposed by etcd-version-monitor (#41807, @shyamjvs)
  • Align the hyperkube image to support running binaries at /usr/local/bin/ like the other server images (#41017, @luxas)
  • Bump up GLBC version from 0.9.0-beta to 0.9.1 (#41037, @bprashanth)

Other Notable Changes

  • The default client certificate generated by kube-up now contains the superuser system:masters group (#39966, @liggitt)
  • Added support for creating HA clusters for centos using kube-up.sh. (#39462, @Shawyeok)
  • Enable lazy inode table and journal initialization for ext3 and ext4 (#38865, @codablock)
  • Since kubernetes.tar.gz no longer includes client or server binaries, cluster/kube-{up,down,push}.sh now automatically download released binaries if they are missing. (#38730, @ixdy)
  • Fix broken cluster/centos and enhance the style (#34002, @xiaoping378)
  • Set kernel.softlockup_panic =1 based on the flag. (#38001, @dchen1107)
  • Configure local-up-cluster.sh to handle auth proxies (#36838, @deads2k)
  • kube-up.sh/kube-down.sh no longer force update gcloud for provider=gce|gke. (#36292, @jlowdermilk)
  • Collect logs for dead kubelets too (#37671, @mtaufen)

Changes to Addons

Dashboard

DNS

  • Updates the dnsmasq cache/mux layer to be managed by dnsmasq-nanny. (#41826, @bowei) dnsmasq-nanny manages dnsmasq based on values from the kube-system:kube-dns configmap:

    "stubDomains": {
"acme.local": ["1.2.3.4"]
 },

    is a map of domain to list of nameservers for the domain. This is used to inject private DNS domains into the kube-dns namespace. In the above example, any DNS requests for *.acme.local will be served by the

    nameserver 1.2.3.4.

    upstreamNameservers": ["8.8.8.8", "8.8.4.4"]

    is a list of upstreamNameservers to use, overriding the configuration specified in /etc/resolv.conf.

  • kube-dns now runs using a separate system:serviceaccount:kube-system:kube-dns service account which is automatically bound to the correct RBAC permissions. (#38816, @deads2k)

  • Use kube-dns:1.11.0 (#39925, @sadlil)

DNS Autoscaler

  • Patch CVE-2016-8859 in gcr.io/google-containers/cluster-proportional-autoscaler-amd64 (#42933, @timstclair)

Cluster Autoscaler

  • Allow the Horizontal Pod Autoscaler controller to talk to the metrics API and custom metrics API as standard APIs. (#41824, @DirectXMan12)

Cluster Load Balancing

etcd Empty Dir Cleanup

  • Base etcd-empty-dir-cleanup on busybox, run as nobody, and update to etcdctl 3.0.14 (#41674, @ixdy)

Fluentd

  • Migrated fluentd addon to daemon set (#32088, @piosz)
  • Fluentd was migrated to Daemon Set, which targets nodes with beta.kubernetes.io/fluentd-ds-ready=true label. If you use fluentd in your cluster please make sure that the nodes with version 1.6+ contains this label. (#42931, @piosz)
  • Fluentd-gcp containers spawned by DaemonSet are now configured using ConfigMap (#42126, @crassirostris)
  • Cleanup fluentd-gcp image: rebase on debian-base, switch to upstream packages, remove fluent-ui & rails (#41998, @timstclair)
  • On GCE, the apiserver audit log (/var/log/kube-apiserver-audit.log) will be sent through fluentd if enabled. It will go to the same place as kube-apiserver.log, but tagged as its own stream. (#41360, @enisoc)
  • If experimentalCriticalPodAnnotation feature gate is set to true, fluentd pods will not be evicted by the kubelet. (#41035, @vishh)
  • fluentd config for GKE clusters updated: detect exceptions in container log streams and forward them as one log entry. (#39656, @thomasschickinger)
  • Make fluentd pods critical (#39146, @crassirostris)
  • Fluentd/Elastisearch add-on: correctly parse and index kubernetes labels (#36857, @Shrugs)

Heapster

Registry

External Dependency Version Information

Continuous integration builds have used the following versions of external dependencies, however, this is not a strong recommendation and users should consult an appropriate installation or upgrade guide before deciding what versions of etcd, docker or rkt to use.

  • Docker versions 1.10.3, 1.11.2, 1.12.6 have been validated
    • Docker version 1.12.6 known issues
      • overlay2 driver not fully supported
      • live-restore not fully supported
      • no shared pid namespace support
    • Docker version 1.11.2 known issues
      • Kernel crash with Aufs storage driver on Debian Jessie (#27885) which can be identified by the node problem detector
      • Leaked File descriptors (#275)
      • Additional memory overhead per container (#21737)
    • Docker 1.10.3 contains backports provided by RedHat for known issues
    • Support for Docker version 1.9.x has been removed
  • rkt version 1.23.0+
  • etcd version 3.0.17

Changelog since v1.6.0-rc.1

Previous Releases Included in v1.6.0

v1.5.6

Documentation & Examples

Downloads for v1.5.6

filename sha256 hash
kubernetes.tar.gz 14a514bb9ed331eb1854a1d66cfaa53290c382641e154c901bcb14eb2cd683b4
kubernetes-src.tar.gz cf3d85bcfd148ed6a54c64b4102a10cc4e54332907fb3d9a6c6e2658a31ca2e9

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 30b819eb1427317be38a4f534fc2c369d43e67499e5df79cdd5d4cfac14f8d36
kubernetes-client-darwin-amd64.tar.gz 3eedf919b2feff4c21edcadb493247013274a3672f6a3d46f19e13af211cea4e
kubernetes-client-linux-386.tar.gz 351bb189f6be835baadda3b87909472c4a9f522ece6e6425250ef227937f2d58
kubernetes-client-linux-amd64.tar.gz d7c3508dc5029c6fefb1bf6f381af92d8626ac5a4b7246009832c03768ae670f
kubernetes-client-linux-arm64.tar.gz 2eaf838ab853c94f05c362a8ce089f32acdb6062356399a6f5fe7cdb13a6fa0c
kubernetes-client-linux-arm.tar.gz e5212f6d9577bd090c88a7124edba86f925e08c710865623d9fb914a5b72e67f
kubernetes-client-windows-386.tar.gz 5a4fdbf0cb88f0e889d8dca1e6c073c167a8c3c7d7b1caad10dbe0dc2eb46677
kubernetes-client-windows-amd64.tar.gz b1170a33c5c6fe2c3f71e820f11cf877f0ee72b60a6546aaf989267c89598656

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 995959c43661c22b0f2ede45b62061f37c25a53388bcdd8988f928574070390a
kubernetes-server-linux-arm64.tar.gz c6b20af2f0c5e3abe20c18aac734846923c8ff3afda637ef1fbd6d3b3820e3b7
kubernetes-server-linux-arm.tar.gz 7d439b2012a0280d40441a5871b25a07b540f5e84c561d2bf8c67725ebbf115d

Changelog since v1.5.5

Other notable changes

  • kube-up (with gce/gci and gce/coreos providers) now ensures the authentication token file contains correct tokens for the control plane components, even if the file already exists (ensures upgrades and downgrades work successfully) (#43676, @liggitt)
  • Patch CVE-2016-8859 in alpine based images: (#42936, @timstclair)
      • gcr.io/google-containers/cluster-proportional-autoscaler-amd64
      • gcr.io/google-containers/dnsmasq-metrics-amd64
      • gcr.io/google-containers/etcd-empty-dir-cleanup
      • gcr.io/google-containers/kube-addon-manager
      • gcr.io/google-containers/kube-dnsmasq-amd64
    • Disable thin_ls due to excessive iops (#43113, @dashpole)
        • Ignore .mount cgroups, fixing dissappearing stats
        • Fix wc goroutine leak
        • Update aws-sdk-go dependency to 1.6.10
  • PodSecurityPolicy authorization is correctly enforced by the PodSecurityPolicy admission plugin. (#43489, @liggitt)
  • Bump gcr.io/google_containers/glbc from 0.9.1 to 0.9.2. Release notes: 0.9.2 (#43097, @timstclair)
  • Update gcr.io/google-containers/rescheduler to v0.2.2, which uses busybox as a base image instead of ubuntu. (#41911, @ixdy)
  • restored normalization of custom --etcd-prefix when --storage-backend is set to etcd3 (#42506, @liggitt)

v1.6.0-rc.1

Documentation & Examples

Downloads for v1.6.0-rc.1

filename sha256 hash
kubernetes.tar.gz b92be4b71184888ba4a2781d4068ea369442b6030dfb38e23029192a554651e5
kubernetes-src.tar.gz e45e993edfdba176a6750c6d3c2207d54d60b5b1fc80a0fe47274d4d9b233d66

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 1d56088fb85fba02362f7f87a5d5f899c05caa605f4d11b8616749cb0d970384
kubernetes-client-darwin-amd64.tar.gz f3df7b558c2ecf6ed8344668515f436b7211a2f840d982f81c55586e1ec84a7b
kubernetes-client-linux-386.tar.gz c5ee1787d69d508d8448675428936d70e21f17b21ff44e22db4462483adcebe2
kubernetes-client-linux-amd64.tar.gz 0960505da11330c8cc66b7df4e4413680afd2a62afc2341bad6bbd88c73e3a56
kubernetes-client-linux-arm64.tar.gz dc113881b9cd09ef8cecbdf8f4ff41eddeba7df3ad7af70461e513eb79757e54
kubernetes-client-linux-arm.tar.gz 5f6bd182852ffe3776b520fbf2db3546c8246133df166dcf6c81ece4b0974227
kubernetes-client-linux-ppc64le.tar.gz ea91e79a779eac8c00a5eb80be1fd5b227b9f5ae767e30a12354bfa691f198d5
kubernetes-client-linux-s390x.tar.gz 874d7410078d39b80fe07a44018f6e95655cb9e05c99ec66dea012d06633fbbb
kubernetes-client-windows-386.tar.gz b02d6d8e436322294a65def8c9c576f232df9387093c4ca61e57dd3bdf184b87
kubernetes-client-windows-amd64.tar.gz 68dbf06824b3785027a85d448f9f2b9928a4092912b382e67c1579e30bb58bbd

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz aa9e5d1cb60c1d33d048a75003fdce9ffa0985ede3748b38b4357d961943d603
kubernetes-server-linux-arm64.tar.gz 0343a1dead1efb8b829ab485028d2ec58ffc4aa7845b3415da5a5bb6fd8bcbfd
kubernetes-server-linux-arm.tar.gz 77a724b28e071e92113759440fdca7e12ea00c5b41a5334ce7581a0139d8f264
kubernetes-server-linux-ppc64le.tar.gz ae21bc7cced29a3c20c76dbf57262a8ea276fe120e411d950ff5267fe4b6cd50
kubernetes-server-linux-s390x.tar.gz 424f5cd9f4aee3e53a8a760ccdc16bd7e2683913e57d56519b536bf4a98f56e5

Changelog since v1.6.0-beta.4

Other notable changes

  • kube-up.sh using the gce provider enables both RBAC authorization and the permissive legacy ABAC policy that makes all service accounts superusers. To opt out of the permissive ABAC policy, export the environment variable ENABLE_LEGACY_ABAC=false before running cluster/kube-up.sh. (#43544, @liggitt)
  • Bump CNI consumers to v0.5.1 (#43546, @calebamiles)
  • The API server discovery document now prioritizes the extensions API group over the apps API group. This ensures certain commands in 1.5 versions of kubectl (such as kubectl edit deployment) continue to function against a 1.6 API. (#43553, @liggitt)
  • Fix adding disks to more than one scsi adapter. (#42422, @kerneltime)
  • PodSecurityPolicy authorization is correctly enforced by the PodSecurityPolicy admission plugin. (#43489, @liggitt)
  • API fields that previously serialized null arrays as null and empty arrays as [] no longer distinguish between those values and always output [] when serializing to JSON. (#43422, @liggitt)
  • Apply taint tolerations for NoExecute for all static pods. (#43116, @dchen1107)
  • Bumped Heapster to v1.3.0. (#43298, @piosz)

v1.5.5

This release contains a fix for a PodSecurityPolicy vulnerability which allows users to make use of any existing PodSecurityPolicy object, even ones they are not authorized to use.

Other then that, this release contains no other changes from 1.5.4.

The vulnerability is tracked in http://issue.k8s.io/43459.

Who is affected?

Only Kubernetes 1.5.0-1.5.4 installations that do all of the following:

  • Enable the PodSecurityPolicy API (which is not enabled by default):
    • --runtime-config=extensions/v1beta1/podsecuritypolicy=true
  • Enable the PodSecurityPolicy admission plugin (which is not enabled by default):
    • --admission-control=...,PodSecurityPolicy,...
  • Use authorization to limit users' ability to use specific PodSecurityPolicy objects

What is the impact?

A user that is authorized to create pods can make use of any existing PodSecurityPolicy, even ones they are not authorized to use.

How can I mitigate this prior to installing 1.5.5?

  1. Export existing PodSecurityPolicy objects:
  • kubectl get podsecuritypolicies -o yaml > psp.yaml
  1. Review and delete any PodSecurityPolicy objects you do not want all pod-creating users to be able to use (NOTE: Privileged users that were making use of those policies will also lose access to those policies). For example:
  • kubectl delete podsecuritypolicies/my-privileged-policy
  1. After upgrading to 1.5.5, re-create the exported PodSecurityPolicy objects:
  • kubectl create -f psp.yaml

Downloads for v1.5.5

filename sha256 hash
kubernetes.tar.gz ff171d53b6dba2aace899dbfa06044d3a54d798896f7b6dd483f20d2c05374ed
kubernetes-src.tar.gz 25207344982bcf76172c7d156106357a7113b3909ac851e19b437dbba9402af6

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 92eb19b1464674078927263642205498a9b4e496909138626de721f8ff2eb3f1
kubernetes-client-darwin-amd64.tar.gz dd2076d8a3062459b82481bf064d80a198df580f2c34efe7132a091c19d8084c
kubernetes-client-linux-386.tar.gz 8366a72910c987e4140db42244741752efac8e06f0e13f5d0cbc1cc9bec9733c
kubernetes-client-linux-amd64.tar.gz 73536e200fee9f4de19ebfd7d2e063a04f5ccb93073982032e79dc47ae92e89a
kubernetes-client-linux-arm64.tar.gz 8f679bd012ecbc58f0a916f393d3fc79de6dc2624320b04edc1b9249213a49f8
kubernetes-client-linux-arm.tar.gz 1998d6398aef02898babc5ff20484fe7c538f75f78c650631afea1a555aee8d1
kubernetes-client-windows-386.tar.gz dff6fe02a6090feb949acc5753633891bcbdb7ecfb2bff3fa132d025713cbd55
kubernetes-client-windows-amd64.tar.gz bd7c7c39122135b58da89a700580475a3cadbb31aa1b35175ff2f80067bedc0d

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 578977b62af58639548d743991cd2f71b0fd58f9caa729131824f8dde85b5c6e
kubernetes-server-linux-arm64.tar.gz 01a1104d8c5a22c26b8b0a402bf0362d749b7d13a636b31c64fb51bb61ea3a01
kubernetes-server-linux-arm.tar.gz 06c5ca1f962f368219835ed6d075ef6e3a72685f2f0988823f44dd2e602e1980

Changelog since v1.5.4

No notable changes for this release

v1.6.0-beta.4

Documentation & Examples

Downloads for v1.6.0-beta.4

filename sha256 hash
kubernetes.tar.gz 8f308a87bcc367656c777f74270a82ad6986517c28a08c7158c77b1d7954e243
kubernetes-src.tar.gz 3ba73cf27a05f78026d1cfb3a0e47c6e5e33932aefc630a0a5aa3619561bc4dc

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 7007e8024257fd2436c9f68ddb25383e889d58379e30a60c9bb6bffb1a6809df
kubernetes-client-darwin-amd64.tar.gz f1bc3f0c8e4c8c9e0aa2f66fffea163a5bf139d528160eb4266cd5322cf112e1
kubernetes-client-linux-386.tar.gz 7ceb47d4b282b31d300aa7a81bf00eef744fb58df58d613e1ea01930287c85d9
kubernetes-client-linux-amd64.tar.gz d25c73f0ebb3338fc3e674d4a667d3023d073b8bc4942eb98f1a3fc9001675ef
kubernetes-client-linux-arm64.tar.gz 3d5a1188f638cddad7cd5eca0668d25f46a6a6f80641a8e9e6f3777a23af0f7c
kubernetes-client-linux-arm.tar.gz e9d40ad06385266cd993adf436270972412dd5407d436e682bddf30706fddbda
kubernetes-client-linux-ppc64le.tar.gz 17f9a60eb6175e28aa0a9ba534cc0ceda24ff8ab81eaf06d04c362146f707e81
kubernetes-client-linux-s390x.tar.gz 30017ac4603bda496a125162cd956e9e874a4d04eff170972c72c8095a9f9121
kubernetes-client-windows-386.tar.gz 6165b8d0781894b36b2f2cd72d79063ce95621012cd1ca38bd7d936feeea8416
kubernetes-client-windows-amd64.tar.gz ac45e5ddf44dd0a680abc5641ae1cb59ad9c7ab8d4132e3b110ebca7ed2119ac

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz a37f0b431aea2cc7e259ddf118fd42315589236106b279de5803e2d50af08531
kubernetes-server-linux-arm64.tar.gz e6a5c8a9e59a12df5294766a4e31e08603a041dd75bcc23f19fb7d20d8a30b9a
kubernetes-server-linux-arm.tar.gz ebe1ccf95a80a829c294fe8bb216a10a096bc7f311fb0f74b7a121772c4d238b
kubernetes-server-linux-ppc64le.tar.gz 8a09baa5c2ddfbc579a1601f76b7079dab695c1423d22a04acd039256e26355c
kubernetes-server-linux-s390x.tar.gz 621feb08ac3bee0b9f5b31c648b3011f91883c44954db04268c0da4ef59f16f1

Changelog since v1.6.0-beta.3

Other notable changes

  • Update dashboard version to v1.6.0 (#43210, @floreks)
  • Update photon controller go SDK in vendor code. (#43108, @luomiao)
  • Fluentd was migrated to Daemon Set, which targets nodes with beta.kubernetes.io/fluentd-ds-ready=true label. If you use fluentd in your cluster please make sure that the nodes with version 1.6+ contains this label. (#42931, @piosz)
  • if kube-apiserver is started with --storage-backend=etcd2, the media type application/json is used. (#43122, @liggitt)
  • Add -p to mkdirs in gci-mounter function of gce configure.sh script (#43134, @shyamjvs)
  • Rescheduler uses taints in v1beta1 and will remove old ones (in version v1alpha1) right after its start. (#43106, @piosz)
  • kubeadm: kubeadm reset won't drain and remove the current node anymore (#42713, @luxas)
  • hack/godep-restore.sh: use godep v79 which works (#42965, @sttts)
  • Patch CVE-2016-8859 in gcr.io/google-containers/cluster-proportional-autoscaler-amd64 (#42933, @timstclair)
  • Disable devicemapper thin_ls due to excessive iops (#42899, @dashpole)

v1.6.0-beta.3

Documentation & Examples

Downloads for v1.6.0-beta.3

filename sha256 hash
kubernetes.tar.gz 3903f0a49945abe26f5775c20deb25ba65a8607a2944da8802255bd50d20aca7
kubernetes-src.tar.gz 62f5f9459c14163e319f878494d10296052d75345da7906e8b38a2d6d0d2a25c

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 1294256f09d3a78a05cf2c85466495b08f87830911e68fd0206964f0466682e3
kubernetes-client-darwin-amd64.tar.gz ff6d8561163d9039c807f4cf05144dd3837104b111fac1ae4b667e2b8548d135
kubernetes-client-linux-386.tar.gz d32a07b4a24a88cfee589cff91336e411a89ed470557b8f74f34bb6636adc800
kubernetes-client-linux-amd64.tar.gz e3663e134cd42bbf71f4f6f0395e6c3ea2080d8621bdab9cc668c77f5478196a
kubernetes-client-linux-arm64.tar.gz 39465c409396a4cc0ae672f0f0c0db971e482de52e9dff835eb43a8f7e3412e9
kubernetes-client-linux-arm.tar.gz 8897b38e59cee396213f50453bdcb88808cd56d63be762362d79454ce526b1ea
kubernetes-client-linux-ppc64le.tar.gz a32b85c5e495dd3645845f2e8ff0eb366fb4ae4795e2bdafceae97cfe71e34b5
kubernetes-client-linux-s390x.tar.gz 0af4f0d7778cb67c1acc3b2f3870283e3008c6e1ea8d532c6b90b5a7f1475db8
kubernetes-client-windows-386.tar.gz b298561b924c8c88b062759cc69b733187310a7e1af74b1b3987ed256f422b05
kubernetes-client-windows-amd64.tar.gz 42ec39178885bb06cba4002844e80948e0c9c3618bfb0a009618a3fab1797a69

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 333ea0cf5c25f65dbb5d353cac002af3fa5e6f8431e81eaba2534005164c9ce9
kubernetes-server-linux-arm64.tar.gz 2979a04409863f6e4dbc745eebfd57ee90e0b38ed4449dcb15cfd87d8f80dadc
kubernetes-server-linux-arm.tar.gz 2ed1e98b2566b4f552951d9496537b18b28ae53eb9e36c6fd17202e9e498eae5
kubernetes-server-linux-ppc64le.tar.gz f4989351a6a98746c1d269d72d2fa87dba8ce782bdfc088d9f7f8d10029aa3fe
kubernetes-server-linux-s390x.tar.gz 31fb764136e97e851d1640464154f3ce4fc696e3286314f538da7b19eed3e2fe

Changelog since v1.6.0-beta.2

Other notable changes

  • Introduce new generator for apps/v1beta1 deployments (#42362, @soltysh)
  • Use Prometheus instrumentation conventions (#36704, @fabxc)
  • Add new DaemonSet status fields to kubectl printer and describer. (#42843, @janetkuo)
  • Dropped the support for docker 1.9.x and the belows. (#42694, @dchen1107)

v1.6.0-beta.2

Documentation & Examples

Downloads for v1.6.0-beta.2

filename sha256 hash
kubernetes.tar.gz 8199c781f8c98ed7048e939a590ea10a6c39f6a74bd35ed526b459fa18e20f50
kubernetes-src.tar.gz f8de26ab6493d4547f9068f5ef396650c169702863535ba63feaf815464a6702

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 0b2350c5ffec582f86d2bd95fa9ecd1b4213fbcd3af79f2a7f67d071c3a0373f
kubernetes-client-darwin-amd64.tar.gz a1c81457a34258f2622f841b9971ba490c66f6a0f5725c089430d0f0fb09dc8c
kubernetes-client-linux-386.tar.gz 40d45492f6741980afde0c83bb752382b699b3d62ac36203faca16fcd9fadd21
kubernetes-client-linux-amd64.tar.gz a06baf31249b06375fde1f608ffea041bdbad0f4814ba8ea69839a7778fa4646
kubernetes-client-linux-arm64.tar.gz d9e18ceb7efacee5cc2a579e204919bb4c272c586bc15750963946e7fe5dc741
kubernetes-client-linux-arm.tar.gz 7abc1a2e5c0e40b46b9839b9c9ca065fceec486413ee3c0687e832dc668560ca
kubernetes-client-linux-ppc64le.tar.gz 84d82f1c2a2b07bea7c827c20cc208f0741b72cf732319673edbf73b42f1b687
kubernetes-client-linux-s390x.tar.gz 605852ee99117abb5bf62f4239c7e2c7e3976f1f497e24ffed50ba4817c301dc
kubernetes-client-windows-386.tar.gz 2c442dbfaa393f67f2fe2a1fd2c10267092e99385ca40f7bed732d48bb36ae62
kubernetes-client-windows-amd64.tar.gz 6b64521cde0b239d9e23e0896919653dfe30ad064d363a9931305feefe04b359

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 0a49f719cd295be9a4947b7b8b0fe68c29d8168d7e03a9e37173de340490b578
kubernetes-server-linux-arm64.tar.gz fb1464794b9e6375cc7f5b8b72125d81921416b6825fe2c37073aef006e846d1
kubernetes-server-linux-arm.tar.gz 27acc02302c6d45ef6314a2bceca6012e35c88d0678b219528d21d8ee4c6b424
kubernetes-server-linux-ppc64le.tar.gz 8eef21ab0700ba2802ef70d2c0b84ee0b27ae0833d2259d425429984f972690e
kubernetes-server-linux-s390x.tar.gz c6e383f897ceb8143a7b1f023e155c9c39e9e7c220e989cc6c1bfcffdb886dd5

Changelog since v1.6.0-beta.1

Action Required

  • Deployment now fully respects ControllerRef to avoid fighting over Pods and ReplicaSets. At the time of upgrade, you must not have Deployments with selectors that overlap, or else ownership of ReplicaSets may change. (#42175, @enisoc)
  • StatefulSet now respects ControllerRef to avoid fighting over Pods. At the time of upgrade, you must not have StatefulSets with selectors that overlap with any other controllers (such as ReplicaSets), or else ownership of Pods may change. (#42080, @enisoc)

Other notable changes

  • DaemonSet now respects ControllerRef to avoid fighting over Pods. (#42173, @enisoc)
  • restored normalization of custom --etcd-prefix when --storage-backend is set to etcd3 (#42506, @liggitt)
  • kubelet created cgroups follow lowercase naming conventions (#42497, @derekwaynecarr)
  • Support whitespace in command path for gcp auth plugin (#41653, @jlowdermilk)
  • Updates the dnsmasq cache/mux layer to be managed by dnsmasq-nanny. (#41826, @bowei)
    • dnsmasq-nanny manages dnsmasq based on values from the
    • kube-system:kube-dns configmap:
    • "stubDomains": {
    • "acme.local": ["1.2.3.4"]
    • },
    • is a map of domain to list of nameservers for the domain. This is used
    • to inject private DNS domains into the kube-dns namespace. In the above
    • example, any DNS requests for *.acme.local will be served by the
    • nameserver 1.2.3.4.
    • "upstreamNameservers": ["8.8.8.8", "8.8.4.4"]
    • is a list of upstreamNameservers to use, overriding the configuration
    • specified in /etc/resolv.conf.
  • kubelet exports metrics for cgroup management (#41988, @sjenning)
  • kubectl: respect deployment strategy parameters for rollout status (#41809, @kargakis)
  • Remove cmd/kube-discovery from the tree since it's not necessary anymore (#42070, @luxas)
  • kubeadm: Hook up kubeadm against the BootstrapSigner (#41417, @luxas)
  • Federated Ingress over GCE no longer requires separate firewall rules to be created for each cluster to circumvent flapping firewall health checks. (#41942, @csbell)
  • ScaleIO Kubernetes Volume Plugin added enabling pods to seamlessly access and use data stored on ScaleIO volumes. (#38924, @vladimirvivien)
  • Pods are launched in a separate cgroup hierarchy than system services. (#42350, @vishh)
  • Experimental support to reserve a pod's memory request from being utilized by pods in lower QoS tiers. (#41149, @sjenning)
  • Juju: Disable anonymous auth on kubelet (#41919, @Cynerva)
  • Remove support for debian masters in GCE kube-up. (#41666, @mikedanese)
  • Implement bulk polling of volumes (#41306, @gnufied)
  • stop kubectl edit from updating the last-applied-configuration annotation when --save-config is unspecified or false. (#41924, @ymqytw)

v1.5.4

Documentation & Examples

Downloads for v1.5.4

filename sha256 hash
kubernetes.tar.gz 2ff668c687c1bdf8351dcae102901b1d46cc50e446bde08a244c2e65739de4c3
kubernetes-src.tar.gz 172d33787ec2d11345d152becdc96982d3057ed16426910302c1b103980b634b

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 53e7c4839025ad04c1104b99e1f8b45f4fe639397c623e2e050acb53cb0a8cbd
kubernetes-client-darwin-amd64.tar.gz 6fac39282c9599566874d63c57b305798e4096a42ef83a8965f615c1d709559c
kubernetes-client-linux-386.tar.gz 80719626f7e6db6d2d04e57bb7edad3077b774a11ebccea3fcddadaa48cbf0a6
kubernetes-client-linux-amd64.tar.gz 24001bc0c7ddb32cd72ac9bed55543830424fba734587ac23b812d8d047a9091
kubernetes-client-linux-arm64.tar.gz 094ff4fe7a10e23a397803869a11a3cc508f3990d9e3b4fbccaefe44be2ad81a
kubernetes-client-linux-arm.tar.gz b12b823d12942d7fccaf791343e9c9854073de3e03cc57a7e4bd7b03fec9806b
kubernetes-client-windows-386.tar.gz e5ae9775cfe695d2d855b29c01f19b0fd0fad008071d8e95f47f70beb16291a8
kubernetes-client-windows-amd64.tar.gz 40cc26a8216e703217264194b68d6b5af28ffa1b9b48b23232027c5d63d8b28c

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz a61cb36d64c8a4111cf04f9d1aac5d8418d07a7c8a682522203b0dfa76f9c806
kubernetes-server-linux-arm64.tar.gz abaa5052f9d0daaebf6b7375c9667c9160355b8ea074daac76ba8a79a24cab37
kubernetes-server-linux-arm.tar.gz ffff55a0f5f5848fdde32a2766dc63cdf26629ca4f91db458381ffb55cf49535

Changelog since v1.5.3

Other notable changes

  • Fix AWS device allocator to only use valid device names (#41455, @gnufied)
  • The kube-apiserver basic audit log can be enabled in GCE by exporting the environment variable ENABLE_APISERVER_BASIC_AUDIT=true before running cluster/kube-up.sh. This will log to /var/log/kube-apiserver-audit.log and use the same logrotate settings as /var/log/kube-apiserver.log. (#41211, @enisoc)
  • list-resources: don't fail if the grep fails to match any resources (#41933, @ixdy)
  • Bump GCE ContainerVM to container-vm-v20170214 to address CVE-2016-9962. (#41449, @zmerlynn)

v1.6.0-beta.1

Documentation & Examples

Downloads for v1.6.0-beta.1

filename sha256 hash
kubernetes.tar.gz ca17c4f1ebdd4bbbd0e570bf3a29d52be1e962742155bc5e20765434f3141f2d
kubernetes-src.tar.gz 4aefc25b42594f0aab48e43608c8ef6eca8c115022fcc76a9a0d34430e33be0f

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 7629da89467e758e6e70c513d5332e6231941de60e99b6621376bc72f9ede314
kubernetes-client-darwin-amd64.tar.gz 3a6d6f78ca307486189c7a92e874508233d6b9b5697a0c42cb2803f4b17ccfb2
kubernetes-client-linux-386.tar.gz 544b944fdcbebb0dbf0e1acedf7e1deb40fd795c46b8f5afe5d622d2091f0ac9
kubernetes-client-linux-amd64.tar.gz d13f3bede2beb1d7fbca7f01a2c0775938d9127073b0fa1cecba4fd152947eae
kubernetes-client-linux-arm64.tar.gz 8820b18ae1c3bdcb8c93b5641e9322aa8dba25ec42362aa86ecbe6ae690a9809
kubernetes-client-linux-arm.tar.gz d928f7e772a74cf715cf382d66ba757394afcf02a03727edfe43305f279fdb87
kubernetes-client-linux-ppc64le.tar.gz 56ccc3a9def527278cd41ba1ce5b0528238ef7b7b5886d6ebc944b11e2f5228c
kubernetes-client-linux-s390x.tar.gz 4923b9617b5306b321e47450bbfe701242b46b2d27800d82a7289fbabe7a107d
kubernetes-client-windows-386.tar.gz 598703591fa2be13cc47930088117fc12731431b679f8ca2a5717430bb45fb93
kubernetes-client-windows-amd64.tar.gz 9e839346effcbe9c469519002967069c8d109282aaceb72e02f25cf606a691b2

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 726b9e4ead829ebd293fe6674ab334f72aa163b1544963febb9bc35d1fb26e6f
kubernetes-server-linux-arm64.tar.gz 975d02629619d441f60442ca07c42721e103e9e5bbcc2eea302b7c936303d26b
kubernetes-server-linux-arm.tar.gz 98223dd80f34eed3cdb30fb57df1da96630db9c0f04aae6a685e22a29c16398d
kubernetes-server-linux-ppc64le.tar.gz a73715b7db73d6d0ad0b78b01829fe9f22566b557eebe2c1a960a81693b0c8b5
kubernetes-server-linux-s390x.tar.gz d3cb54e9193c773ea9998106c75bb3f0af705477fb844bcc8f82c845c44bb00d

Changelog since v1.6.0-alpha.3

Action Required

  • The --dns-provider argument of 'kubefed init' is now mandatory and does not default to google-clouddns. To initialize a Federation control plane with Google Cloud DNS, use the following invocation: 'kubefed init --dns-provider=google-clouddns' (#42092, @marun)
  • Change taints/tolerations to api fields (#38957, @aveshagarwal)

Other notable changes

  • kubeadm: Rename some flags for beta UI and fixup some logic (#42064, @luxas)
  • StorageClassName attribute has been added to PersistentVolume and PersistentVolumeClaim objects and should be used instead of annotation volume.beta.kubernetes.io/storage-class. The beta annotation is still working in this release, however it will be removed in a future release. (#42128, @jsafrane)
  • Remove Azure kube-up as the Azure community has focused efforts elsewhere. (#41672, @mikedanese)
  • Fluentd-gcp containers spawned by DaemonSet are now configured using ConfigMap (#42126, @Crassirostris)
  • Modified kubemark startup scripts to restore master on reboot (#41980, @shyamjvs)
  • Added new Api PodPreset to enable defining cross-cutting injection of Volumes and Environment into Pods. (#41931, @jessfraz)
  • AWS cloud provider: allow to run the master with a different AWS account or even on a different cloud provider than the nodes. (#39996, @scheeles)
  • Update defaultbackend image to 1.3 (#42212, @timstclair)
  • Allow the Horizontal Pod Autoscaler controller to talk to the metrics API and custom metrics API as standard APIs. (#41824, @DirectXMan12)
  • Implement support for mount options in PVs (#41906, @gnufied)
  • Introduce apps/v1beta1.Deployments resource with modified defaults compared to extensions/v1beta1.Deployments. (#39683, @soltysh)
  • Add DNS suffix search list support in Windows kube-proxy. (#41618, @JiangtianLi)
  • --experimental-nvidia-gpus flag is replaced by Accelerators alpha feature gate along with support for multiple Nvidia GPUs. (#42116, @vishh)
    • To use GPUs, pass Accelerators=true as part of --feature-gates flag.
    • Works only with Docker runtime.
  • Clean up the kube-proxy container image by removing unnecessary packages and files. (#42090, @timstclair)
  • AWS: Support shared tag kubernetes.io/cluster/<clusterid> (#41695, @justinsb)
  • Insecure access to the API Server at localhost:8080 will be turned off in v1.6 when using kubeadm (#42066, @luxas)
  • AWS: Do not consider master instance zones for dynamic volume creation (#41702, @justinsb)
  • Added foreground garbage collection: the owner object will not be deleted until all its dependents are deleted by the garbage collector. Please checkout the user doc for details. (#38676, @caesarxuchao)
    • deleteOptions.orphanDependents is going to be deprecated in 1.7. Please use deleteOptions.propagationPolicy instead.
  • force unlock rbd image if the image is not used (#41597, @rootfs)
  • The kubernetes-master, kubernetes-worker and kubeapi-load-balancer charms have gained an nrpe-external-master relation, allowing the integration of their monitoring in an external Nagios server. (#41923, @Cynerva)
  • make kubectl describe pod show tolerationSeconds (#42162, @kevin-wangzefeng)
  • Completed pods should not be hidden when requested by name via kubectl get. (#42216, @smarterclayton)
  • [Federation][Kubefed] Flag cleanup (#41335, @irfanurrehman)
  • Add the support to the scheduler for spreading pods of StatefulSets. (#41708, @bsalamat)
  • Portworx Volume Plugin added enabling Portworx to be used as a storage provider for Kubernetes clusters. Portworx pools your servers capacity and turns your servers or cloud instances into converged, highly available compute and storage nodes. (#39535, @adityadani)
  • Remove support for trusty in GCE kube-up. (#41670, @mikedanese)
  • Import a natural sorting library and use it in the sorting printer. (#40746, @matthyx)
  • Parameter keys in a StorageClass parameters map may not use the kubernetes.io or k8s.io namespaces. (#41837, @liggitt)
  • Make DaemonSet respect critical pods annotation when scheduling. (#42028, @janetkuo)
  • New Kubelet flag --enforce-node-allocatable with a default value of pods is added which will make kubelet create a top level cgroup for all pods to enforce Node Allocatable. Optionally, system-reserved & kube-reserved values can also be specified separated by comma to enforce node allocatable on cgroups specified via --system-reserved-cgroup & --kube-reserved-cgroup respectively. Note the default value of the latter flags are "". (#41234, @vishh)
    • This feature requires a Node Drain prior to upgrade failing which pods will be restarted if possible or terminated if they have a RestartNever policy.
  • Deployment of AWS Kubernetes clusters using the in-tree bash deployment (i.e. cluster/kube-up.sh or get-kube.sh) is obsolete. v1.5.x will be the last release to support cluster/kube-up.sh with AWS. For a list of viable alternatives, see: http://kubernetes.io/docs/getting-started-guides/aws/ (#42196, @zmerlynn)
  • kubectl logs allows getting logs directly from deployment, job and statefulset (#40927, @soltysh)
  • make kubectl taint command respect effect NoExecute (#42120, @kevin-wangzefeng)
  • Flex volume plugin is updated to support attach/detach interfaces. It broke backward compatibility. Please update your drivers and implement the new callouts. (#41804, @chakri-nelluri)
  • Implement the update feature for DaemonSet. (#41116, @lukaszo)
  • [Federation] Create configmap for the cluster kube-dns when cluster joins and remove when it unjoins (#39338, @irfanurrehman)
  • New GKE certificates controller. (#41160, @pipejakob)
  • Juju: Fix shebangs in charm actions to use python3 (#42058, @Cynerva)
  • Support kubectl apply set-last-applied command to update the applied-applied-configuration annotation (#41694, @shiywang)
  • On GCI by default logrotate is disabled for application containers in favor of rotation mechanism provided by docker logging driver. (#40634, @Crassirostris)
  • Cleanup fluentd-gcp image: rebase on debian-base, switch to upstream packages, remove fluent-ui & rails (#41998, @timstclair)
  • Updating apiserver to return http status code 202 for a delete request when the resource is not immediately deleted because of user requesting cascading deletion using DeleteOptions.OrphanDependents=false. (#41165, @nikhiljindal)
  • [Federation][kubefed] Support configuring dns-provider (#40528, @shashidharatd)
  • Added support to minimize sending verbose node information to scheduler extender by sending only node names and expecting extenders to cache the rest of the node information (#41119, @sarat-k)
  • Guaranteed admission for Critical Pods (#40952, @dashpole)
  • Switch to the node-role.kubernetes.io/master label for marking and tainting the master node in kubeadm (#41835, @luxas)
  • Allow drain --force to remove pods whose managing resource is deleted. (#41864, @marun)
  • add kubectl can-i to see if you can perform an action (#41077, @deads2k)
  • enable DefaultTolerationSeconds admission controller by default (#41815, @kevin-wangzefeng)
  • Make DaemonSets survive taint-based evictions when nodes turn unreachable/notReady. (#41896, @kevin-wangzefeng)
  • Add configurable limits to CronJob resource to specify how many successful and failed jobs are preserved. (#40932, @peay)
  • Deprecate outofdisk-transition-frequency and low-diskspace-threshold-mb flags (#41941, @dashpole)
  • Add OWNERS for sample-apiserver in staging (#42094, @sttts)
  • Update gcr.io/google-containers/rescheduler to v0.2.2, which uses busybox as a base image instead of ubuntu. (#41911, @ixdy)
  • Add storage.k8s.io/v1 API (#40088, @jsafrane)
  • Juju - K8s master charm now properly keeps distributed master files in sync for an HA control plane. (#41351, @chuckbutler)
  • Fix zsh completion: unknown file attribute error (#38104, @elipapa)
  • kubelet config should ignore file start with dots (#39196, @resouer)
  • Add an alpha feature that makes NodeController set Taints instead of deleting Pods from not Ready Nodes. (#41133, @gmarek)
  • Base etcd-empty-dir-cleanup on busybox, run as nobody, and update to etcdctl 3.0.14 (#41674, @ixdy)
  • Fix zone placement heuristics so that multiple mounts in a StatefulSet pod are created in the same zone (#40910, @justinsb)
  • Flag --use-kubernetes-version for kubeadm init renamed to --kubernetes-version (#41820, @kad)
  • kube-dns now runs using a separate system:serviceaccount:kube-system:kube-dns service account which is automatically bound to the correct RBAC permissions. (#38816, @deads2k)
  • [Kubemark] Fixed hollow-npd container command to log to file (#41858, @shyamjvs)
  • kubeadm: Remove the --cloud-provider flag for beta init UX (#41710, @luxas)
  • The CertificateSigningRequest API added the extra field to persist all information about the requesting user. This mirrors the fields in the SubjectAccessReview API used to check authorization. (#41755, @liggitt)
  • Upgrade golang versions to 1.7.5 (#41771, @cblecker)
  • Added a new secret type "bootstrap.kubernetes.io/token" for dynamically creating TLS bootstrapping bearer tokens. (#41281, @ericchiang)
  • Remove unnecessary metrics (http/process/go) from being exposed by etcd-version-monitor (#41807, @shyamjvs)
  • Added kubectl create clusterrole command. (#41538, @xingzhou)
  • Support new kubectl apply view-last-applied command for viewing the last configuration file applied (#41146, @shiywang)
  • Bump GCI to gci-stable-56-9000-84-2 (#41819, @dchen1107)
  • list-resources: don't fail if the grep fails to match any resources (#41933, @ixdy)
  • client-go no longer imports GCP OAuth2 and OpenID Connect packages by default. (#41532, @ericchiang)
  • Each pod has its own associated cgroup by default. (#41349, @derekwaynecarr)
  • Whitelist kubemark in node_ssh_supported_providers for log dump (#41800, @shyamjvs)
  • Support KUBE_MAX_PD_VOLS on Azure (#41398, @codablock)
  • Projected volume plugin (#37237, @jpeeler)
  • --output-version is ignored for all commands except kubectl convert. This is consistent with the generic nature of kubectl CRUD commands and the previous removal of --api-version. Specific versions can be specified in the resource field: resource.version.group, jobs.v1.batch. (#41576, @deads2k)
  • Added bool type support for jsonpath. (#39063, @xingzhou)
  • Nodes can now report two additional address types in their status: InternalDNS and ExternalDNS. The apiserver can use --kubelet-preferred-address-types to give priority to the type of address it uses to reach nodes. (#34259, @liggitt)
  • Clients now use the ?watch=true parameter to make watch API calls, instead of the /watch/ path prefix (#41722, @liggitt)
  • ResourceQuota ability to support default limited resources (#36765, @derekwaynecarr)
  • Fix kubemark default e2e test suite's name (#41751, @shyamjvs)
  • federation aws: add logging of route53 calls (#39964, @justinsb)
  • Fix ConfigMap for Windows Containers. (#39373, @jbhurat)
  • add defaultTolerationSeconds admission controller (#41414, @kevin-wangzefeng)
  • Node Problem Detector is beta now. New features added: journald support, standalone mode and arbitrary system log monitoring. (#40206, @Random-Liu)
  • Fix the output of health-mointor.sh (#41525, @yujuhong)
  • kubectl describe no longer prints the last-applied-configuration annotation for secrets. (#34664, @ymqytw)
  • Report node not ready on failed PLEG health check (#41569, @yujuhong)
  • Delay Deletion of a Pod until volumes are cleaned up (#41456, @dashpole)
  • Alpha version of dynamic volume provisioning is removed in this release. Annotation (#40000, @jsafrane)
  • An automountServiceAccountToken *bool field was added to ServiceAccount and PodSpec objects. If set to false on a pod spec, no service account token is automounted in the pod. If set to false on a service account, no service account token is automounted for that service account unless explicitly overridden in the pod spec. (#37953, @liggitt)
  • Bump addon-manager version to v6.4-alpha.1 in kubemark (#41506, @shyamjvs)
  • Do not daemonize salt-minion for the openstack-heat provider. (#40722, @micmro)
  • Move private key parsing from serviceaccount/jwt.go to client-go/util/cert (#40907, @cblecker)
  • Added configurable etcd initial-cluster-state to kube-up script. (#41332, @jszczepkowski)

v1.6.0-alpha.3

Documentation & Examples

Downloads for v1.6.0-alpha.3

filename sha256 hash
kubernetes.tar.gz 41b5e9edd973cbb8e68eb5d8d758c4f0afa11dfbd65df49e1c361206706a974c
kubernetes-src.tar.gz ec13e22322c85752918c23b0b498ba02087a1227b8fdc169f19acdf128f907c4

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 5a631b7604a69ef13c27b43e6df10f8bf14ff9170440fb07d0c46bc88a5a1eac
kubernetes-client-darwin-amd64.tar.gz cfba71e38a924b783fcdbc0b1a342671d52af3588a8211e35048e9c071ed03b2
kubernetes-client-linux-386.tar.gz ceeee264b12959cb2b314efa9df4c165ea1598b8824ec652eb3994096f4ec07f
kubernetes-client-linux-amd64.tar.gz 1bd3a4b64ab1535780f18b3e7a56dd1301a8ea8d66869ee704f66985c1fca9b4
kubernetes-client-linux-arm64.tar.gz d1615b3223c6e83422ed8409fc8d0a7a6069982d3413a482e12966b953520fe0
kubernetes-client-linux-arm.tar.gz 19133867e2d104db3e01212dbc4a702a315310a10e86076b6b80a16b94cf7954
kubernetes-client-linux-ppc64le.tar.gz 0f89e17eb881c7db39195bc94874e3ec54866d2f57eef1540b5d843bedbe4326
kubernetes-client-linux-s390x.tar.gz 3ed06cb89ffec011e4248c14d9e1c88c815b7363d1fdba217ed17e900f29960b
kubernetes-client-windows-386.tar.gz 87927cbe26cefa296e2752075d018a58826bc7fa141c4cbe56116a254a3470cc
kubernetes-client-windows-amd64.tar.gz e97e7dafbf670140d3c4879a6738b970ac77d917861df3eea0c502238dd297b0

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 3aa82b838be450ce8dedbebfda45c453864c15aae6363ae5f1c0b0d285ffad2a
kubernetes-server-linux-arm64.tar.gz 0bdeac3524ab7ef366f3bb75e2fbff3db156dcba2b862e8b2de393e4ec4377c9
kubernetes-server-linux-arm.tar.gz 1f37886aba4027ec682afe5f02a4d66a6645af2476f2954933c1b437ec66dafa
kubernetes-server-linux-ppc64le.tar.gz eb81d3cdd703790d5c96e24917183dc123aeabbe9a291c2dd86c68d21d9fd213
kubernetes-server-linux-s390x.tar.gz a50a57c689583f97fd4ff7af766bf7ae79c9fd97e46720bc41f385f2c51e1f99

Changelog since v1.6.0-alpha.2

Other notable changes

  • Fix AWS device allocator to only use valid device names (#41455, @gnufied)
  • [Federation][Kubefed] Bug fix relating kubeconfig path in kubefed init (#41410, @irfanurrehman)
  • The apiserver audit log (/var/log/kube-apiserver-audit.log) will be sent through fluentd if enabled. (#41360, @enisoc)
  • Bump GCE ContainerVM to container-vm-v20170214 to address CVE-2016-9962. (#41449, @zmerlynn)
  • Fixed issues #39202, #41041 and #40941 that caused the iSCSI connections to be prematurely closed when deleting a pod with an iSCSI persistent volume attached and that prevented the use of newly created LUNs on targets with preestablished connections. (#41196, @CristianPop)
  • The kube-apiserver basic audit log can be enabled in GCE by exporting the environment variable ENABLE_APISERVER_BASIC_AUDIT=true before running cluster/kube-up.sh. This will log to /var/log/kube-apiserver-audit.log and use the same logrotate settings as /var/log/kube-apiserver.log. (#41211, @enisoc)
  • On kube-up.sh clusters on GCE, kube-scheduler now contacts the API on the secured port. (#41285, @liggitt)
  • Default RBAC ClusterRole and ClusterRoleBinding objects are automatically updated at server start to add missing permissions and subjects (extra permissions and subjects are left in place). To prevent autoupdating a particular role or rolebinding, annotate it with rbac.authorization.kubernetes.io/autoupdate=false. (#41155, @liggitt)
  • Make EnableCRI default to true (#41378, @yujuhong)
  • kubectl edit now edits objects exactly as they were retrieved from the API. This allows using kubectl edit with third-party resources and extension API servers. Because client-side conversion is no longer done, the --output-version option is deprecated for kubectl edit. To edit using a particular API version, fully-qualify the resource, version, and group used to fetch the object (for example, job.v1.batch/myjob) (#41304, @liggitt)
  • We change the default attach_detach_controller sync period to 1 minute to reduce the query frequency through cloud provider to check whether volumes are attached or not. (#41363, @jingxu97)
  • RBAC v1beta1 RoleBinding/ClusterRoleBinding subjects changed apiVersion to apiGroup to fully-qualify a subject. ServiceAccount subjects default to an apiGroup of "", User and Group subjects default to an apiGroup of "rbac.authorization.k8s.io". (#41184, @liggitt)
  • Add support for finalizers in federated configmaps (deletes configmaps from underlying clusters). (#40464, @csbell)
  • Make DaemonSet controller respect node taints and pod tolerations. (#41172, @janetkuo)
  • Added kubectl create role command (#39852, @xingzhou)
  • If experimentalCriticalPodAnnotation feature gate is set to true, fluentd pods will not be evicted by the kubelet. (#41035, @vishh)

v1.4.9

Documentation & Examples

Downloads for v1.4.9

filename sha256 hash
kubernetes.tar.gz 9d385d555073c7cf509a92ce3aa96d0414a93c21c51bcf020744c70b4b290aa2
kubernetes-src.tar.gz 6fd7d33775356f0245d06b401ac74d8227a92abd07cc5a0ef362bac16e01f011

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 16b362f3cf56dee7b0c291188767222fd65176ed9573a8b87e8acf7eb6b22ed9
kubernetes-client-darwin-amd64.tar.gz 537e5c5d8a9148cd464f5d6d0a796e214add04c185b859ea9e39a4cc7264394c
kubernetes-client-linux-386.tar.gz e9d2e55b42e002771c32d9f26e8eb0b65c257ea257e8ab19f7fd928f21caace8
kubernetes-client-linux-amd64.tar.gz 1ba81d64d1ae165b73375d61d364c642068385d6a1d68196d90e42a8d0fd6c7d
kubernetes-client-linux-arm64.tar.gz d0398d2b11ed591575adde3ce9e1ad877fe37b8b56bd2be5b2aee344a35db330
kubernetes-client-linux-arm.tar.gz 714b06319bf047084514803531edab6a0a262c5f38a0d0bfda0a8e59672595b6
kubernetes-client-windows-386.tar.gz 16a7224313889d2f98a7d072f328198790531fd0e724eaeeccffe82521ae63b8
kubernetes-client-windows-amd64.tar.gz dc19651287701ea6dcbd7b4949db2331468f730e8ebe951de1216f1105761d97

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 6a104d143f8568a8ce16c979d1cb2eb357263d96ab43bd399b05d28f8da2b961
kubernetes-server-linux-arm64.tar.gz 8137ecde19574e6aba0cd9efe127f3b3eb02c312d7691745df3a23e40b7a5d72
kubernetes-server-linux-arm.tar.gz 085195abeb9133cb43f0e6198e638ded7f15beca44d19503c2836339a7e604aa

Changelog since v1.4.8

Other notable changes

  • Bump GCE ContainerVM to container-vm-v20170201 to address CVE-2016-9962. (#40828, @zmerlynn)
  • Bump GCI to gci-beta-56-9000-80-0 (#41027, @dchen1107)
  • Fix for detach volume when node is not present/ powered off (#40118, @BaluDontu)
  • Bump GCI to gci-beta-56-9000-80-0 (#41027, @dchen1107)
  • Move b.gcr.io/k8s_authenticated_test to gcr.io/k8s-authenticated-test (#40335, @zmerlynn)
  • Prep node_e2e for GCI to COS name change (#41088, @jessfraz)
  • If ExperimentalCriticalPodAnnotation=True flag gate is set, kubelet will ensure that pods with scheduler.alpha.kubernetes.io/critical-pod annotation will be admitted even under resource pressure, will not be evicted, and are reasonably protected from system OOMs. (#41052, @vishh)
  • Fix resync goroutine leak in ListAndWatch (#35672, @tatsuhiro-t)
  • Kubelet will no longer set hairpin mode on every interface on the machine when an error occurs in setting up hairpin for a specific interface. (#36990, @bboreham)
  • Bump GCE ContainerVM to container-vm-v20170201 to address CVE-2016-9962. (#40828, @zmerlynn)
  • Adding vmdk file extension for vmDiskPath in vsphere DeleteVolume (#40538, @divyenpatel)
  • Prevent hotloops on error conditions, which could fill up the disk faster than log rotation can free space. (#40497, @lavalamp)
  • Update GCE ContainerVM deployment to container-vm-v20170117 to pick up CVE fixes in base image. (#40094, @zmerlynn)
  • Update kube-proxy image to be based off of Debian 8.6 base image. (#39695, @ixdy)
  • Update amd64 kube-proxy base image to debian-iptables-amd64:v5 (#39725, @ixdy)

v1.5.3

Documentation & Examples

Downloads for v1.5.3

filename sha256 hash
kubernetes.tar.gz a4d997be9e3ac0f9838a58fb80d08c2ab02e00afb9d16d3db18d99c85b88b316
kubernetes-src.tar.gz a23636ee40a60c1bb3255a03177f522c28133f74c6d09a5437f6b56b7e1d5296

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 2f8eeb772c22c7dad5a32d6ee17e8b309503b56fbcb0abdc74e1f94e86b33520
kubernetes-client-darwin-amd64.tar.gz b044240271223aa93f8bdb8054824a48ba5571460d2e6c90688dccd0892e5c7e
kubernetes-client-linux-386.tar.gz d2649a41e4a64c2027e321254e4ef3e690371bd0c7eece12d3395e49d8171617
kubernetes-client-linux-amd64.tar.gz eaf386a46eeee324bb71349bba7d5d3f41d7d19af75537cf9e4e7045d7068f68
kubernetes-client-linux-arm64.tar.gz 2f2d45296651e5696f373838ba019e8b8bb11b2a2772a55f0a6e367ec6c18e2d
kubernetes-client-linux-arm.tar.gz 56b8b207fd914dc7c16fdb675a3917ab9bff0efbe745ee1675abbff2b5854d32
kubernetes-client-windows-386.tar.gz fe3136e3c6bd983e55396341c451f896e478e8c9d0b3d1418e1d1fccee3d7b75
kubernetes-client-windows-amd64.tar.gz 8e315cb48135a4ed26585e9d8cf88f550ac51e3658b981bb53cb0952e9b3393a

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz ad4d101bec0ef981a7e1efbe11223e502ff644368d70ad54915e15fcb3ad6735
kubernetes-server-linux-arm64.tar.gz bfd66c57d1071bdd213d4c6d124d491959ae3509994e5a23cc2720a8ad18526d
kubernetes-server-linux-arm.tar.gz 12b335637b7a4aa019cee600b0161d51e6317a87bec0500e1f9d85990f6352d5

Node Binaries

filename sha256 hash
kubernetes-node-linux-amd64.tar.gz 3f54e2d101b6351513ce9425a23f9a196e965326c3a7f78a98ef1dad452e5830
kubernetes-node-linux-arm.tar.gz 6508b64755dc0ff90f23921d2b8bb6c0c321c38edeaf24fd4c22282880a87a11
kubernetes-node-linux-arm64.tar.gz 578ef8a6958fb4bf2e0438cdef7707d12456186a1b8c4b18aa66f47b9221a713
kubernetes-node-windows-amd64.tar.gz aa166b275b3d0f80cbf23fbee7f42358b6176f37fd9ef66837f38910d4626079

Changelog since v1.5.2

Other notable changes

  • We change the default attach_detach_controller sync period to 1 minute to reduce the query frequency through cloud provider to check whether volumes are attached or not. (#41363, @jingxu97)
  • Added configurable etcd initial-cluster-state to kube-up script (#41320, @jszczepkowski)
  • If ExperimentalCriticalPodAnnotation=True flag gate is set, kubelet will ensure that pods with scheduler.alpha.kubernetes.io/critical-pod annotation will be admitted even under resource pressure, will not be evicted, and are reasonably protected from system OOMs. (#41052, @vishh)
  • Reverts to looking up the current VM in vSphere using the machine's UUID, either obtained via sysfs or via the vm-uuid parameter in the cloud configuration file. (#40892, @robdaemon)
  • Fix for detach volume when node is not present/ powered off (#40118, @BaluDontu)
  • Move b.gcr.io/k8s_authenticated_test to gcr.io/k8s-authenticated-test (#40335, @zmerlynn)
  • Bump up GLBC version from 0.9.0-beta to 0.9.1 (#41037, @bprashanth)
  • azure: fix Azure Container Registry integration (#40142, @colemickens)
  • azure disk: restrict name length for Azure specifications (#40030, @colemickens)
  • Bump GCI to gci-beta-56-9000-80-0 (#41027, @dchen1107)
  • Bump up glbc version to 0.9.0-beta.1 (#40565, @bprashanth)
  • Enable lazy inode table and journal initialization for ext3 and ext4 (#38865, @codablock)
  • Kubelet will no longer set hairpin mode on every interface on the machine when an error occurs in setting up hairpin for a specific interface. (#36990, @bboreham)
  • The SubjectAccessReview API passes subresource and resource name information to the authorizer to answer authorization queries. (#40935, @liggitt)
  • Bump GCE ContainerVM to container-vm-v20170201 to address CVE-2016-9962. (#40828, @zmerlynn)
  • Reduce time needed to attach Azure disks (#40066, @codablock)
  • Fixes request header authenticator by presenting the request header client CA so that the front proxy will authenticate using its client certificate. (#40301, @deads2k)
  • Fix failing load balancers in Azure (#40405, @codablock)
  • Add a KUBERNETES_NODE_* section to build kubelet/kube-proxy for windows (#38919, @brendandburns)
  • Update GCE ContainerVM deployment to container-vm-v20170117 to pick up CVE fixes in base image. (#40094, @zmerlynn)
  • Adding vmdk file extension for vmDiskPath in vsphere DeleteVolume (#40538, @divyenpatel)
  • AWS: Remove duplicate calls to DescribeInstance during volume operations (#39842, @gnufied)
  • Caching added to the OIDC client auth plugin to fix races and reduce the time kubectl commands using this plugin take by several seconds. (#38167, @ericchiang)
  • Actually fix local-cluster-up on 1.5 branch (#40501, @lavalamp)
  • Prevent hotloops on error conditions, which could fill up the disk faster than log rotation can free space. (#40497, @lavalamp)
  • Fix issue with PodDisruptionBudgets in which minAvailable specified as a percentage did not work with StatefulSet Pods. (#39454, @foxish)
  • Fix panic in vSphere cloud provider (#38423, @BaluDontu)
  • Allow missing keys in templates by default (#39486, @ncdc)
  • Fix kubectl get -f -o so it prints all items in the file (#39038, @ncdc)
  • Endpoints, that tolerate unready Pods, are now listing Pods in state Terminating as well (#37093, @simonswine)
  • Add path exist check in getPodVolumePathListFromDisk (#38909, @jingxu97)
  • Ensure the GCI metadata files do not have newline at the end (#38727, @Amey-D)
  • AWS: recognize eu-west-2 region (#38746, @justinsb)
  • Fix space issue in volumePath with vSphere Cloud Provider (#38338, @BaluDontu)
  • Fix issue when attempting to unmount a wrong vSphere volume (#37413, @BaluDontu)
  • Changed default scsi controller type in vSphere Cloud Provider (#38426, @abrarshivani)
  • Fixes API compatibility issue with empty lists incorrectly returning a null items field instead of an empty array. (#39834, @liggitt)
  • AWS: Add sequential allocator for device names. (#38818, @jsafrane)
  • Fix fsGroup to vSphere (#38655, @abrarshivani)

v1.6.0-alpha.2

Documentation & Examples

Downloads for v1.6.0-alpha.2

filename sha256 hash
kubernetes.tar.gz d1a5c7bc435c0f58dca9eab54e8155b6e4797d4b5d6b0cb8feab968dd3132165
kubernetes-src.tar.gz 8d09b973f3debfe3d10b0ad392e56141446bc0d04aac60df6d761189997d97ed

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz b1a7d002768fff8282f8873fde6a0ed215d20fd72197d8e583c024b7cbbe3eb8
kubernetes-client-darwin-amd64.tar.gz b0f872bbefdc1ecc9585dfdeb9c7e094f7a16ebbe4db11c64c70d1ef7f93e361
kubernetes-client-linux-386.tar.gz 53be6adde2d13058d03d0f283ca166bb495cc49cd2e36339696dc46f85f78c8f
kubernetes-client-linux-amd64.tar.gz 6436463d51ed54b50023cd725b054fd2b039e391095d8a618e91979fc55d4ee0
kubernetes-client-linux-arm64.tar.gz d6638c8950a9e03ed64c3f3e1ad82166642c02aeb8f7fb2079db1f137170a145
kubernetes-client-linux-arm.tar.gz 05d0e466a27fc9a5b6535cbd7683e08739cd37aa2c6213c000fdaa305e4eb888
kubernetes-client-linux-ppc64le.tar.gz 38971be682cbf1f194eeb9ad683272a58042f4f91992db6fc34720de28d88dd6
kubernetes-client-linux-s390x.tar.gz 774a002482d6b62338550b20d90b914a08481965ed1b78cd348535d90f88f344
kubernetes-client-windows-386.tar.gz 690ab995ac27c90c811578677fb8688d43198499bfc451a2f908ad7a76474ee8
kubernetes-client-windows-amd64.tar.gz 757fe9e2083e2da706b52c96da34548aa72bbbbff50bb261c1198c80b36189c3

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 5aa3161a1030ddb02985171d4343a99d14ad6e09e130549b20fc96797588ff1e
kubernetes-server-linux-arm64.tar.gz babce8c402c8e88310ba82315f758e981d4cfe20dcbf3047e55b279d5d4f3a33
kubernetes-server-linux-arm.tar.gz 4b1d4c4ba95d86481d32daf09e769f7f9e4e0d71e11d39a5a4e205804b023172
kubernetes-server-linux-ppc64le.tar.gz 070423e62a0dff7ef17e29a63ac2eda5a46b6e1badf67214c07970b83610bf6c
kubernetes-server-linux-s390x.tar.gz 3a826fef775819a4f03e7db91684db6edfdaaad3e3eb366c4321bdc5ec0b0f25

Changelog since v1.6.0-alpha.1

Other notable changes

  • Align the hyperkube image to support running binaries at /usr/local/bin/ like the other server images (#41017, @luxas)
  • Native support for token based bootstrap flow. This includes signing a well known ConfigMap in the kube-public namespace and cleaning out expired tokens. (#36101, @jbeda)
  • Reverts to looking up the current VM in vSphere using the machine's UUID, either obtained via sysfs or via the vm-uuid parameter in the cloud configuration file. (#40892, @robdaemon)
  • This PR adds a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag. (#40355, @gmarek)
  • The authentication.k8s.io API group was promoted to v1 (#41058, @liggitt)
  • Fixes issue #38418 which, under circumstance, could cause StatefulSet to deadlock. (#40838, @kow3ns)
    • Mediates issue #36859. StatefulSet only acts on Pods whose identity matches the StatefulSet, providing a partial mediation for overlapping controllers.
  • Introduces an new alpha version of the Horizontal Pod Autoscaler including expanded support for specifying metrics. (#36033, @DirectXMan12)
  • Set all node conditions to Unknown when node is unreachable (#36592, @andrewsykim)
  • [Federation] Add override flags options to kubefed init (#40917, @irfanurrehman)
  • Fix for detach volume when node is not present/ powered off (#40118, @BaluDontu)
  • Bump up GLBC version from 0.9.0-beta to 0.9.1 (#41037, @bprashanth)
  • The deprecated flags --config, --auth-path, --resource-container, and --system-container were removed. (#40048, @mtaufen)
  • Add kubectl attach support for multiple types (#40365, @shiywang)
  • [Kubelet] Delay deletion of pod from the API server until volumes are deleted (#41095, @dashpole)
  • remove the create-external-load-balancer flag in cmd/expose.go (#38183, @tianshapjq)
  • The authorization.k8s.io API group was promoted to v1 (#40709, @liggitt)
  • Bump GCI to gci-beta-56-9000-80-0 (#41027, @dchen1107)
  • [Federation][kubefed] Add option to expose federation apiserver on nodeport service (#40516, @shashidharatd)
  • Rename --experiemental-cgroups-per-qos to --cgroups-per-qos (#39972, @derekwaynecarr)
  • PV E2E: provide each spec with a fresh nfs host (#40879, @copejon)
  • Remove the temporary fix for pre-1.0 mirror pods (#40877, @yujuhong)
  • fix --save-config in create subcommand (#40289, @xilabao)
  • We should mention the caveats of in-place upgrade in release note. (#40727, @Random-Liu)
  • The SubjectAccessReview API passes subresource and resource name information to the authorizer to answer authorization queries. (#40935, @liggitt)
  • When feature gate "ExperimentalCriticalPodAnnotation" is set, Kubelet will avoid evicting pods in "kube-system" namespace that contains a special annotation - scheduler.alpha.kubernetes.io/critical-pod (#40655, @vishh)
  • make tolerations respect wildcard key (#39914, @kevin-wangzefeng)
  • [Federation][kubefed] Add option to disable persistence storage for etcd (#40862, @shashidharatd)
  • Init containers have graduated to GA and now appear as a field. The beta annotation value will still be respected and overrides the field value. (#38382, @hodovska)
  • apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK (#40666, @ymqytw)
  • HorizontalPodAutoscaler is no longer supported in extensions/v1beta1 version. Use autoscaling/v1 instead. (#35782, @piosz)
  • Port forwarding can forward over websockets or SPDY. (#33684, @fraenkel)
  • Improve kubectl describe node output by adding closing paren (#39217, @luksa)
  • Bump GCE ContainerVM to container-vm-v20170201 to address CVE-2016-9962. (#40828, @zmerlynn)
  • Use full package path for definition name in OpenAPI spec (#40124, @mbohlool)
  • kubectl apply now supports explicitly clearing values not present in the config by setting them to null (#40630, @liggitt)
  • Fixed an issue where 'kubectl get --sort-by=' would return an error when the specified field were not present in at least one of the returned objects, even that being a valid field in the object model. (#40541, @fabianofranz)
  • Add initial french translations for kubectl (#40645, @brendandburns)
  • OpenStack-Heat will now look for an image named "CentOS-7-x86_64-GenericCloud-1604". To restore the previous behavior set OPENSTACK_IMAGE_NAME="CentOS7" (#40368, @sc68cal)
  • Preventing nil pointer reference in client_config (#40508, @vjsamuel)
  • The bash AWS deployment via kube-up.sh has been deprecated. See http://kubernetes.io/docs/getting-started-guides/aws/ for alternatives. (#38772, @zmerlynn)
  • Fix failing load balancers in Azure (#40405, @codablock)
  • kubefed init creates a service account for federation controller manager in the federation-system namespace and binds that service account to the federation-system:federation-controller-manager role that has read and list access on secrets in the federation-system namespace. (#40392, @madhusudancs)
  • Fixed an SELinux issue in kubeadm on Docker 1.12+ by moving etcd SELinux options from container to pod. (#40682, @dgoodwin)
  • Juju kubernetes-master charm: improve status messages (#40691, @Cynerva)

v1.6.0-alpha.1

Documentation & Examples

Downloads for v1.6.0-alpha.1

filename sha256 hash
kubernetes.tar.gz abda73bc2a27ae16c66a5aea9e96cd59486ed8cf994afc55da35a3cea2edc1db
kubernetes-src.tar.gz b429579ba83f9a3fa80e72ceb65b046659b17f16a0b7f70105e7096a441f32b9

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz d5b8adee6169515324f4f420e65e9d4e14cca3402f58ec99660a1947fd79d3ea
kubernetes-client-darwin-amd64.tar.gz 6232a7e46b2cbd1e30a1f44c9b424448c5def11f5132c742bf62bac8a4f26fa2
kubernetes-client-linux-386.tar.gz 2974ed14b76885947c95b3c86fb8585aa08ccefe8cc11cd202dcc3cb8fcc0d1a
kubernetes-client-linux-amd64.tar.gz 3bcb40f4aa3a295ec23fe42a5e17b081ef8de174b7dfa03cb89c27a00ac16f5a
kubernetes-client-linux-arm64.tar.gz f9a55bcb6af2a415d24d69ae919c56968f6b02369675fd7def63acbde6534430
kubernetes-client-linux-arm.tar.gz b0bd7070eab2f19b9bc1840fb4da5307c7ce274f2e28f76f94c714aa08f087bf
kubernetes-client-linux-ppc64le.tar.gz 39f38972f93f64542ae325d9c937c9d527968bc0830fdd38dd38dc246b6f0c56
kubernetes-client-linux-s390x.tar.gz 032e21fe0000333f36e29ddf24e207cfd6c92cb9a6d69cc0123c31aacd12338c
kubernetes-client-windows-386.tar.gz 84b7d58012760111067ec0907070cc2f6d4c95893e771533a9b335cc8d8c72b7
kubernetes-client-windows-amd64.tar.gz e9ce76f48a4cf58b261aec38f076ed3b61c444c55c123f30099c1d1763d6191f

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz fafa4b93a522b9e73b6c31e42d32dc5eb3fccb5ca1548425da03726b48a19175
kubernetes-server-linux-arm64.tar.gz f00f4874785d1c9cba4fd262e8086aa693027649da44b0eec69e96951e013913
kubernetes-server-linux-arm.tar.gz e843a6357e0a2e441277e6b32d8d64a6b6fe367c3d223edec781c21d8b379ac6
kubernetes-server-linux-ppc64le.tar.gz 12b73f7cc4928543eee09af91d632bcf5c4ed56c44d48d62d0087c7fcbaa3e02
kubernetes-server-linux-s390x.tar.gz aad15f260d166c2b295921468837efe4a51cb7a04bcaccb3faecc9b5979861e5

Changelog since v1.5.0

Action Required

  • Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading. (#39772, @mikedanese)
  • Switch default etcd version to 3.0.14. (#36229, @wojtek-t)
    • Switch default storage backend flag in apiserver to etcd3 mode.
  • RBAC's special handling of the user * in RoleBinding and ClusterRoleBinding objects is deprecated and will be removed in v1beta1. To match all users, explicitly bind to the group system:authenticated and/or system:unauthenticated. Existing v1alpha1 bindings to the user * will be automatically converted to the group system:authenticated. (#38981, @liggitt)
  • The 'endpoints.beta.kubernetes.io/hostnames-map' annotation is no longer supported. Users can use the 'Endpoints.subsets[].addresses[].hostname' field instead. (#39284, @bowei)
  • federation/deploy/deploy.sh was an interim solution introduced in Kubernetes v1.4 to simplify the federation control plane deployment experience. Now that we have kubefed, we are deprecating deploy.sh scripts. (#38902, @madhusudancs)
  • Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data. (#37770, @enj)
  • etcd2: watching from 0 returns all initial states as ADDED events (#38079, @hongchaodeng)

Other notable changes

  • kube-up.sh on GCE now includes the bootstrap admin in the super-user group, and ensures the auth token file is correct on upgrades (#39537, @liggitt)
  • genericapiserver: cut off more dependencies – episode 3 (#40426, @sttts)
  • Adding vmdk file extension for vmDiskPath in vsphere DeleteVolume (#40538, @divyenpatel)
  • Remove outdated net.experimental.kubernetes.io/proxy-mode and net.beta.kubernetes.io/proxy-mode annotations from kube-proxy. (#40585, @cblecker)
  • Improve the ARM builds and make hyperkube on ARM working again by upgrading the Go version for ARM to go1.8beta2 (#38926, @luxas)
  • Prevent hotloops on error conditions, which could fill up the disk faster than log rotation can free space. (#40497, @lavalamp)
  • DaemonSet controller actively kills failed pods (to recreate them) (#40330, @janetkuo)
  • forgiveness alpha version api definition (#39469, @kevin-wangzefeng)
  • Bump up glbc version to 0.9.0-beta.1 (#40565, @bprashanth)
  • Improve formatting of EventSource in kubectl get and kubectl describe (#40073, @matthyx)
  • CRI: use more gogoprotobuf plugins (#40397, @yujuhong)
  • Adds shortNames to the APIResource from discovery which is a list of recommended shortNames for clients like kubectl. (#40312, @p0lyn0mial)
  • Use existing ABAC policy file when upgrading GCE cluster (#40172, @liggitt)
  • Added support for creating HA clusters for centos using kube-up.sh. (#39462, @Shawyeok)
  • azure: fix Azure Container Registry integration (#40142, @colemickens)
    • Splits Juju Charm layers into master/worker roles (#40324, @chuckbutler)
        • Adds support for 1.5.x series of Kubernetes
        • Introduces a tactic for keeping templates in sync with upstream eliminating template drift
        • Adds CNI support to the Juju Charms
        • Adds durable storage support to the Juju Charms
        • Introduces an e2e Charm layer for repeatable testing efforts and validation of clusters
  • genericapiserver: more dependency cutoffs (#40216, @sttts)
  • AWS: trust region if found from AWS metadata (#38880, @justinsb)
  • Volumes and environment variables populated from ConfigMap and Secret objects can now tolerate the named source object or specific keys being missing, by adding optional: true to the volume or environment variable source specifications. (#39981, @fraenkel)
  • kubectl create now accepts the label selector flag for filtering objects to create (#40057, @MrHohn)
  • A new field terminationMessagePolicy has been added to containers that allows a user to request FallbackToLogsOnError, which will read from the container's logs to populate the termination message if the user does not write to the termination message log file. The termination message file is now properly readable for end users and has a maximum size (4k bytes) to prevent abuse. Each pod may have up to 12k bytes of termination messages before the contents of each will be truncated. (#39341, @smarterclayton)
  • Add a special purpose tool for editing individual fields in a ConfigMap with kubectl (#38445, @brendandburns)
  • [Federation] Expose autoscaling apis through federation api server (#38976, @irfanurrehman)
  • Powershell script to start kubelet and kube-proxy (#36250, @jbhurat)
  • Reduce time needed to attach Azure disks (#40066, @codablock)
  • fixing Cassandra shutdown example to avoid data corruption (#39199, @deimosfr)
  • kubeadm: add optional self-hosted deployment for apiserver, controller-manager and scheduler. (#40075, @pires)
  • kubelet tears down pod volumes on pod termination rather than pod deletion (#37228, @sjenning)
  • The default client certificate generated by kube-up now contains the superuser system:masters group (#39966, @liggitt)
  • CRI: upgrade protobuf to v3 (#39158, @feiskyer)
  • Add SIGCHLD handler to pause container (#36853, @verb)
  • Populate environment variables from a secrets. (#39446, @fraenkel)
  • fluentd config for GKE clusters updated: detect exceptions in container log streams and forward them as one log entry. (#39656, @thomasschickinger)
  • Made multi-scheduler graduated to Beta and then v1. (#38871, @k82cn)
  • Fixed forming resolver search line for pods: exclude duplicates, obey libc limitations, logging and eventing appropriately. (#29666, @vefimova)
  • Add authorization mode to kubeadm (#39846, @andrewrynhard)
  • Update dependencies: aws-sdk-go to 1.6.10; also cadvisor (#40095, @dashpole)
  • Fixes a bug in the OpenStack-Heat kubernetes provider, in the handling of differences between the Identity v2 and Identity v3 APIs (#40105, @sc68cal)
  • Update GCE ContainerVM deployment to container-vm-v20170117 to pick up CVE fixes in base image. (#40094, @zmerlynn)
  • AWS: Remove duplicate calls to DescribeInstance during volume operations (#39842, @gnufied)
  • The attributeRestrictions field has been removed from the PolicyRule type in the rbac.authorization.k8s.io/v1alpha1 API. The field was not used by the RBAC authorizer. (#39625, @deads2k)
  • Enable lazy inode table and journal initialization for ext3 and ext4 (#38865, @codablock)
  • azure disk: restrict name length for Azure specifications (#40030, @colemickens)
  • Follow redirects for streaming requests (exec/attach/port-forward) in the apiserver by default (alpha -> beta). (#40039, @timstclair)
  • Use kube-dns:1.11.0 (#39925, @sadlil)
  • Anonymous authentication is now automatically disabled if the API server is started with the AlwaysAllow authorizer. (#38706, @deads2k)
  • genericapiserver: cut off kube pkg/version dependency (#39943, @sttts)
  • genericapiserver: cut off pkg/serviceaccount dependency (#39945, @sttts)
  • Move pkg/api/rest into genericapiserver (#39948, @sttts)
  • genericapiserver: cut off pkg/apis/extensions and pkg/storage dependencies (#39946, @sttts)
  • genericapiserver: cut off certificates api dependency (#39947, @sttts)
  • Admission control support for versioned configuration files (#39109, @derekwaynecarr)
  • Fix issue around merging lists of primitives when using PATCH or kubectl apply. (#38665, @ymqytw)
  • Fixes API compatibility issue with empty lists incorrectly returning a null items field instead of an empty array. (#39834, @liggitt)
  • [scheduling] Moved pod affinity and anti-affinity from annotations to api fields #25319 (#39478, @rrati)
  • PodSecurityPolicy resource is now enabled by default in the extensions API group. (#39743, @pweil-)
  • add --controllers to controller manager (#39740, @deads2k)
  • proxy/iptables: don't sync proxy rules if services map didn't change (#38996, @dcbw)
  • Update amd64 kube-proxy base image to debian-iptables-amd64:v5 (#39725, @ixdy)
  • Update dashboard version to v1.5.1 (#39662, @rf232)
  • Fix kubectl get -f -o so it prints all items in the file (#39038, @ncdc)
  • Scheduler treats StatefulSet pods as belonging to a single equivalence class. (#39718, @foxish)
  • --basic-auth-file supports optionally specifying groups in the fourth column of the file (#39651, @liggitt)
  • To create or update an RBAC RoleBinding or ClusterRoleBinding object, a user must: (#39383, @liggitt)
    • Be authorized to make the create or update API request
    • Be allowed to bind the referenced role, either by already having all of the permissions contained in the referenced role, or by having the bind permission on the referenced role.
  • Fixes an HPA-related panic due to division-by-zero. (#39694, @DirectXMan12)
  • federation: Adding support for DeleteOptions.OrphanDependents for federated services. Setting it to false while deleting a federated service also deletes the corresponding services from all registered clusters. (#36390, @nikhiljindal)
  • Update kube-proxy image to be based off of Debian 8.6 base image. (#39695, @ixdy)
  • Update FitError as a message component into the PodConditionUpdater. (#39491, @jayunit100)
  • rename kubernetes-discovery to kube-aggregator (#39619, @deads2k)
  • Allow missing keys in templates by default (#39486, @ncdc)
  • Caching added to the OIDC client auth plugin to fix races and reduce the time kubectl commands using this plugin take by several seconds. (#38167, @ericchiang)
  • AWS: recognize eu-west-2 region (#38746, @justinsb)
  • Provide kubernetes-controller-manager flags to control volume attach/detach reconciler sync. The duration of the syncs can be controlled, and the syncs can be shut off as well. (#39551, @chrislovecnm)
  • Generate OpenAPI definition for inlined types (#39466, @mbohlool)
  • ShortcutExpander has been extended in a way that it will examine a ha… (#38835, @p0lyn0mial)
  • fixes nil dereference when doing a volume type check on persistent volumes (#39493, @sjenning)
  • Fix issue with PodDisruptionBudgets in which minAvailable specified as a percentage did not work with StatefulSet Pods. (#39454, @foxish)
  • fix issue with kubectl proxy so that it will proxy an empty path - e.g. http://localhost:8001 (#39226, @luksa)
  • Check if pathExists before performing Unmount (#39311, @rkouj)
  • Adding kubectl tests for federation (#38844, @nikhiljindal)
  • Fix comment and optimize code (#38084, @tanshanshan)
  • When using OIDC authentication and specifying --oidc-username-claim=email, an "email_verified":true claim must be returned from the identity provider. (#36087, @ericchiang)
  • Allow pods to define multiple environment variables from a whole ConfigMap (#36245, @fraenkel)
  • Refactor the certificate and kubeconfig code in the kubeadm binary into two phases (#39280, @luxas)
  • Added support for printing in all supported --output formats to kubectl create ... and kubectl apply ... (#38112, @juanvallejo)
  • genericapiserver: extract CA cert from server cert and SNI cert chains (#39022, @sttts)
  • Endpoints, that tolerate unready Pods, are now listing Pods in state Terminating as well (#37093, @simonswine)
  • DaemonSet ObservedGeneration (#39157, @lukaszo)
  • The --reconcile-cidr kubelet flag was removed since it had been deprecated since v1.5 (#39322, @luxas)
  • Add ready replicas in Deployments (#37959, @kargakis)
  • Remove all MAINTAINER statements in Dockerfiles in the codebase as they are deprecated by docker (#38927, @luxas)
  • Remove the deprecated vsphere kube-up. (#39140, @kerneltime)
  • Kubectl top now also accepts short forms for "node" and "pod" ("no", "po") (#39218, @luksa)
  • Remove 'exec' network plugin - use CNI instead (#39254, @freehan)
  • Add three more columns to kubectl get deploy -o wide output. (#39240, @xingzhou)
  • Add path exist check in getPodVolumePathListFromDisk (#38909, @jingxu97)
  • Begin paths for internationalization in kubectl (#36802, @brendandburns)
  • Fixes an issue where commas were not accepted in --from-literal flags when creating secrets. Passing multiple values separated by a comma in a single --from-literal flag is no longer supported. Please use multiple --from-literal flags to provide multiple values. (#35191, @SamiHiltunen)
  • Support loading UTF16 files if a byte-order-mark is present (#39008, @brendandburns)
  • Fix fsGroup to vSphere (#38655, @abrarshivani)
  • ReplicaSet has onwer ref of the Deployment that created it (#35676, @krmayankk)
  • Don't evict static pods (#39059, @bprashanth)
  • Fixed a bug where the --server, --token, and --certificate-authority flags were not overriding the related in-cluster configs when provided in a kubectl call inside a cluster. (#39006, @fabianofranz)
  • Make fluentd pods critical (#39146, @Crassirostris)
  • assign -998 as the oom_score_adj for critical pods (e.g. kube-proxy) (#39114, @dchen1107)
  • delete continue in monitorNodeStatus (#38798, @NickrenREN)
  • add create rolebinding (#38991, @deads2k)
  • Add new command "kubectl set selector" (#38966, @kargakis)
  • Federation: Add batch/jobs API objects to federation-apiserver (#35943, @jianhuiz)
  • ABAC policies using "user":"*" or "group":"*" to match all users or groups will only match authenticated requests. To match unauthenticated requests, ABAC policies must explicitly specify "group":"system:unauthenticated" (#38968, @liggitt)
  • To add local registry to libvirt_coreos (#36751, @sdminonne)
  • Add a KUBERNETES_NODE_* section to build kubelet/kube-proxy for windows (#38919, @brendandburns)
  • Added kubeadm commands to manage bootstrap tokens and the duration they are valid for. (#35805, @dgoodwin)
  • AWS: Recognize ca-central-1 region (#38410, @justinsb)
  • Unmount operation should not fail if volume is already unmounted (#38547, @rkouj)
  • Changed default scsi controller type in vSphere Cloud Provider (#38426, @abrarshivani)
  • Move non-generic apiserver code out of the generic packages (#38191, @sttts)
  • Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator. (#38614, @soltysh)
  • Admit critical pods in the kubelet (#38836, @bprashanth)
  • Use daemonset in docker registry add on (#35582, @surajssd)
  • Node affinity has moved from annotations to api fields in the pod spec. Node affinity that is defined in the annotations will be ignored. (#37299, @rrati)
  • Since kubernetes.tar.gz no longer includes client or server binaries, cluster/kube-{up,down,push}.sh now automatically download released binaries if they are missing. (#38730, @ixdy)
  • genericapiserver: turn APIContainer.SecretRoutes into a real ServeMux (#38826, @sttts)
  • Migrated fluentd addon to daemon set (#32088, @piosz)
  • AWS: Add sequential allocator for device names. (#38818, @jsafrane)
  • Add 'X-Content-Type-Options: nosniff" to some error messages (#37190, @brendandburns)
  • Display pod node selectors with kubectl describe. (#36396, @aveshagarwal)
  • The main repository does not keep multiple releases of clientsets anymore. Please find previous releases at https://github.com/kubernetes/client-go (#38154, @caesarxuchao)
  • Remove a release-note on multiple OpenAPI specs (#38732, @mbohlool)
  • genericapiserver: unify swagger and openapi in config (#38690, @sttts)
  • fix connection upgrades through kuberentes-discovery (#38724, @deads2k)
  • Fixes bug in resolving client-requested API versions (#38533, @DirectXMan12)
  • apiserver(s): Replace glog.Fatals with fmt.Errorfs (#38175, @sttts)
  • Remove Azure Subnet RouteTable check (#38334, @mogthesprog)
  • Ensure the GCI metadata files do not have newline at the end (#38727, @Amey-D)
  • Significantly speed-up make (#38700, @sttts)
  • add QoS pod status field (#37968, @sjenning)
  • Fixed validation of multizone cluster for GCE (#38695, @jszczepkowski)
  • Fixed detection of master during creation of multizone nodes cluster by kube-up. (#38617, @jszczepkowski)
  • Update CHANGELOG.md to warn about anon auth flag (#38675, @erictune)
  • Fixes NotAuthenticated errors that appear in the kubelet and kube-controller-manager due to never logging in to vSphere (#36169, @robdaemon)
  • Fix an issue where AWS tear-down leaks an DHCP Option Set. (#38645, @zmerlynn)
  • Issue a warning when using kubectl apply on a resource lacking the LastAppliedConfig annotation (#36672, @ymqytw)
  • Re-add /healthz/ping handler in genericapiserver (#38603, @sttts)
  • Fail kubelet if runtime is unresponsive for 30 seconds (#38527, @derekwaynecarr)
  • Add support for Azure Container Registry, update Azure dependencies (#37783, @brendandburns)
  • Fix panic in vSphere cloud provider (#38423, @BaluDontu)
  • fix broken cluster/centos and enhance the style (#34002, @xiaoping378)
  • Ability to quota storage by storage class (#34554, @derekwaynecarr)
  • [Part 2] Adding s390x cross-compilation support for gcr.io images in this repo (#36050, @gajju26)
  • kubectl run --rm no longer prints "pod xxx deleted" (#38429, @duglin)
  • Bump GCE debian image to container-vm-v20161208 (release notes) (#38432, @timstclair)
  • Kubelet: Add image cache. (#38375, @Random-Liu)
  • Allow a selector when retrieving logs (#32752, @fraenkel)
  • Fix unmountDevice issue caused by shared mount in GCI (#38411, @jingxu97)
  • [Federation] Implement dry run support in kubefed init (#36447, @irfanurrehman)
  • Fix space issue in volumePath with vSphere Cloud Provider (#38338, @BaluDontu)
  • [Federation] Make federation etcd PVC size configurable (#36310, @irfanurrehman)
  • Allow no ports when exposing headless service (#32811, @fraenkel)
  • Wait for the port to be ready before starting (#38260, @fraenkel)
  • Add Version to the resource printer for 'get nodes' (#37943, @ailusazh)
  • contribute deis/registry-proxy as a replacement for kube-registry-proxy (#35797, @bacongobbler)
  • [Part 1] Add support for cross-compiling s390x binaries (#37092, @gajju26)
  • kernel memcg notification enabled via experimental flag (#38258, @derekwaynecarr)
  • fix permissions when using fsGroup (#37009, @sjenning)
  • Pipe get options to storage (#37693, @wojtek-t)
  • add a configuration for kubelet to register as a node with taints (#31647, @mikedanese)
  • Remove genericapiserver.Options.MasterServiceNamespace (#38186, @sttts)
  • The --long-running-request-regexp flag to kube-apiserver is deprecated and will be removed in a future release. Long-running requests are now detected based on specific verbs (watch, proxy) or subresources (proxy, portforward, log, exec, attach). (#38119, @liggitt)
  • Better compat with very old iptables (e.g. CentOS 6) (#37594, @thockin)
  • Fix GCI mounter issue (#38124, @jingxu97)
  • Kubelet will no longer set hairpin mode on every interface on the machine when an error occurs in setting up hairpin for a specific interface. (#36990, @bboreham)
  • fix mesos unit tests (#38196, @deads2k)
  • Add --controllers flag to federation controller manager for enable/disable federation ingress controller (#36643, @kzwang)
  • Allow backendpools in Azure Load Balancers which are not owned by cloud provider (#36882, @codablock)
  • remove rbac super user (#38121, @deads2k)
  • API server have two separate limits for read-only and mutating inflight requests. (#36064, @gmarek)
  • check the value of min and max in kubectl (#37789, @yarntime)
  • The glusterfs dynamic volume provisioner will now choose a unique GID for new persistent volumes from a range that can be configured in the storage class with the "gidMin" and "gidMax" parameters. The default range is 2000 - 2147483647 (max int32). (#37886, @obnoxxx)
  • Add kubectl create poddisruptionbudget command (#36646, @kargakis)
  • Set kernel.softlockup_panic =1 based on the flag. (#38001, @dchen1107)
  • portfordwardtester: avoid data loss during send+close+exit (#37103, @sttts)
  • Enable containerized mounter only for nfs and glusterfs types (#37990, @jingxu97)
  • Add flag to enable contention profiling in scheduler. (#37357, @gmarek)
  • Add kubernetes-anywhere as a new e2e deployment option (#37019, @pipejakob)
  • add create clusterrolebinding command (#37098, @deads2k)
  • kubectl create service externalname (#34789, @AdoHe)
  • Fix logic error in graceful deletion (#37721, @derekwaynecarr)
  • Exit with error if is not the final parameter. (#37723, @mtaufen)
  • Fix Service Update on LoadBalancerSourceRanges Field (#37720, @freehan)
  • Bug fix. Incoming UDP packets not reach newly deployed services (#32561, @zreigz)
  • GCI: Remove /var/lib/docker/network (#37593, @yujuhong)
  • configure local-up-cluster.sh to handle auth proxies (#36838, @deads2k)
  • Add clusterid, an optional parameter to storageclass. (#36437, @humblec)
  • local-up-cluster: avoid sudo for control plane (#37443, @sttts)
  • Add error message when trying to use clusterrole with namespace in kubectl (#36424, @xilabao)
  • kube-up.sh/kube-down.sh no longer force update gcloud for provider=gce|gke. (#36292, @jlowdermilk)
  • Fix issue when attempting to unmount a wrong vSphere volume (#37413, @BaluDontu)
  • Fix the equality checks for numeric values in cluster/gce/util.sh. (#37638, @roberthbailey)
  • kubelet: don't reject pods without adding them to the pod manager (#37661, @yujuhong)
  • Modify GCI mounter to enable NFSv3 (#37582, @jingxu97)
  • Fix photon controller plugin to construct with correct PdID (#37167, @luomiao)
  • Collect logs for dead kubelets too (#37671, @mtaufen)
  • Set Dashboard UI version to v1.5.0 (#37684, @rf232)
  • When deleting an object with --grace-period=0, the client will begin a graceful deletion and wait until the resource is fully deleted. To force deletion, use the --force flag. (#37263, @smarterclayton)
  • federation service controller: stop deleting services from underlying clusters when federated service is deleted. (#37353, @nikhiljindal)
  • Fix nil pointer dereference in test framework (#37583, @mtaufen)
  • Kubernetes now automatically installs a StorageClass object when deployed on (#31617, @jsafrane)
    • AWS, Google Compute Engine, Google Container Engine, and OpenStack using
    • the default kube-up.sh scripts. This StorageClass is marked as default so that
    • a PersistentVolumeClaim without a StorageClass annotation now results in
    • automatic provisioning of storage (GCE PersistentDisk on Google Cloud, AWS
    • EBS on AWS, and Cinder on OpenStack). In previous versions of Kubernetes
    • a PersistentVolumeClaim without a StorageClass annotation on these cloud
    • platforms would be satisfied by manually-created PersistentVolume objects.
    • Administrators can choose to disable this behavior by deleting the automatically
    • installed StorageClass API object. Alternatively, administrators may choose to
    • keep the automatically installed StorageClass and only disable the defaulting
    • behavior by removing the "is-default-class" annotation from the StorageClass
    • API object.
  • Fix TestServiceAlloc flakes (#37487, @wojtek-t)
  • Use gsed on the Mac (#37562, @roberthbailey)
  • Try self-repair scheduler cache or panic (#37379, @wojtek-t)
  • Fluentd/Elastisearch add-on: correctly parse and index kubernetes labels (#36857, @Shrugs)
  • Mention overflows when mistakenly call function FromInt (#36487, @xialonglee)
  • Update doc for kubectl apply (#37397, @ymqytw)
  • Removes shorthand flag -w from kubectl apply (#37345, @MrHohn)
  • Curating Owners: pkg/api (#36525, @apelisse)
  • Reduce verbosity of volume reconciler when attaching volumes (#36900, @codablock)
  • Federated Ingress Proposal (#29793, @quinton-hoole)

v1.5.2

Documentation & Examples

Downloads for v1.5.2

filename sha256 hash
kubernetes.tar.gz 67344958325a70348db5c4e35e59f9c3552232cdc34defb8a0a799ed91c671a3
kubernetes-src.tar.gz 93241d0f7b69de71d68384699b225ed8a5439bde03dc154827a2b7a6a343791e

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 1e8a3186907fe5e00f8afcd2ca7a207703d5c499d86c80839333cd7cc4eee9ad
kubernetes-client-darwin-amd64.tar.gz 64ebd769d96aa5a12f13c4d8c4f6ddce58eae90765c55b7942872dc91447e4d7
kubernetes-client-linux-386.tar.gz a8ecb343a7baf9e01459cd903c09291dbbe72e12431e259e60e11b243b2740f7
kubernetes-client-linux-amd64.tar.gz 9d5b6edebb5ee09b20f35d821d3d233ff4d5935880fc8ea8f1fa654d5fd23e51
kubernetes-client-linux-arm64.tar.gz 03fd45f96e5d2b66c568b213d0ab6a216aad8c383d5ea4654f7ba8ef5c4d6747
kubernetes-client-linux-arm.tar.gz 527fbf42e2e4a2785ad367484a4db619b04484621006fa098cde0ffc3ad3496f
kubernetes-client-windows-386.tar.gz 3afe8d3ef470e81a4d793539c2a05fbbca9f0710ced1c132b1105469924e3cea
kubernetes-client-windows-amd64.tar.gz dbb63c5211d62512b412efcb52d0a394f19a8417f3e5cd153a7f04c619eb5b41

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 8c4be20caa87530fdd17e539abe6f2d3cfccaef9156d262d4d9859ca8b6e3a38
kubernetes-server-linux-arm64.tar.gz e0251c3209acebf55e98db521cf29aaa74076a4119b1b19780620faf81d18f44
kubernetes-server-linux-arm.tar.gz 548ad7e061263ff53b80f3ab10a3c7f9289e89a4c56b5a8f49ae513ba88ea93a

Changelog since v1.5.1

Other notable changes

  • Fixes NotAuthenticated errors that appear in the kubelet and kube-controller-manager due to never logging in to vSphere (#36169, @robdaemon)
  • Update amd64 kube-proxy base image to debian-iptables-amd64:v5 (#39725, @ixdy)
  • Update kube-proxy image to be based off of Debian 8.6 base image. (#39695, @ixdy)
  • Fixes an HPA-related panic due to division-by-zero. (#39694, @DirectXMan12)
  • Update fluentd-gcp addon to 1.28.1 (#39706, @ixdy)
  • Provide kubernetes-controller-manager flags to control volume attach/detach reconciler sync. The duration of the syncs can be controlled, and the syncs can be shut off as well. (#39551, @chrislovecnm)
  • AWS: Recognize ca-central-1 region (#38410, @justinsb)
  • fix nil dereference when doing a volume type check on persistent volumes (#39529, @sjenning)
  • Generate OpenAPI definition for inlined types (#39466, @mbohlool)
  • Admit critical pods in the kubelet (#38836, @bprashanth)
  • assign -998 as the oom_score_adj for critical pods (e.g. kube-proxy) (#39114, @dchen1107)
  • Don't evict static pods (#39059, @bprashanth)
  • Fix an issue where AWS tear-down leaks an DHCP Option Set. (#38645, @zmerlynn)
  • Give apply the versioned struct that generated from the type defined in the restmapping. (#38982, @ymqytw)
  • Add support for Azure Container Registry, update Azure dependencies (#37783, @brendandburns)
  • Fixes an issue where hack/local-up-cluster.sh would fail on the API server start with (#38898, @deads2k)
  • Since kubernetes.tar.gz no longer includes client or server binaries, cluster/kube-{up,down,push}.sh now automatically download released binaries if they are missing. (#38730, @ixdy)
  • Fixed validation of multizone cluster for GCE (#38695, @jszczepkowski)
  • Fix nil pointer dereference in test framework (#37583, @mtaufen)
  • Fixed detection of master during creation of multizone nodes cluster by kube-up. (#38617, @jszczepkowski)
  • Kubelet: Add image cache. (#38375, @Random-Liu)

v1.4.8

Documentation & Examples

Downloads for v1.4.8

filename sha256 hash
kubernetes.tar.gz 888d2e6c5136e8805805498729a1da55cf89addfd28f098e0d2cf3f28697ab5c
kubernetes-src.tar.gz 0992c3f4f4cb21011fea32187c909babc1a3806f35cec86aacfe9c3d8bef2485

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 8b1c9931544b7b42df64ea98e0d8e1430d09eea3c9f78309834e4e18b091dc18
kubernetes-client-darwin-amd64.tar.gz a306a687979013b8a27acae244d000de9a77f73714ccf96510ecf0398d677051
kubernetes-client-linux-386.tar.gz 81fc5e1b5aba4e0aead37c82c7e45891c4493c7df51da5200f83462b6f7ad98f
kubernetes-client-linux-amd64.tar.gz 704a5f8424190406821b69283f802ade95e39944efcce10bcaf4bd7b3183abc4
kubernetes-client-linux-arm64.tar.gz 7f3e5e8dadb51257afa8650bcd3db3e8f3bc60e767c1a13d946b88fa8625a326
kubernetes-client-linux-arm.tar.gz 461d359067cd90542ce2ceb46a4b2ec9d92dd8fd1e7d21a9d9f469c98f446e56
kubernetes-client-windows-386.tar.gz 894a9c8667e4c4942cb25ac32d10c4f6de8477c6bbbad94e9e6f47121151f5df
kubernetes-client-windows-amd64.tar.gz b2bd4afdd3eaea305c03b94b0864c5622abf19113c6794dedff4ad85327fda01

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz c3dc0e26c00bbe40bd19f61d2d7faeaa56384355c58a0efc4227a360b3eb2da2
kubernetes-server-linux-arm64.tar.gz 745d7ba03bb9c6b57a5a36b389f6467a0707f0a1476d7536ad47417c853eeffd
kubernetes-server-linux-arm.tar.gz dc21f9c659f1d762cad9d0cce0a32146c11cd0d41c58eb2dcbfb0c9f9707349f

Changelog since v1.4.7

Other notable changes

v1.5.1

Documentation & Examples

Downloads for v1.5.1

filename sha256 hash
kubernetes.tar.gz adc4f6ec1fc8f97ed19f474ffcc0af2d050f92dc20ecec2799741802019205ec
kubernetes-src.tar.gz 27e5009b906b9f233a7be1efcf51140be945446d828c006c171d03fe07e43565

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 06f8155f0df381bca3b4e27bbd28834f7601e32cbe3d0c1f24be90516c5b8a3b
kubernetes-client-darwin-amd64.tar.gz 3ede7d74c5f2f918547bca4d813901e33580c8b8f19828da21a5c2296ff4b8be
kubernetes-client-linux-386.tar.gz b96c3c359146e4fc4d8ff4cf09216bbbb9dbaf3f405488d4aaa45ac741c98f99
kubernetes-client-linux-amd64.tar.gz 662fc57057290deb38ec49dd7daf4a4a5b91def2dbdb7ee7a4494dec611379a5
kubernetes-client-linux-arm64.tar.gz c33936b7a27f296c7b85bbfac1fe303573580a948dd1f3174916da9a5a954d49
kubernetes-client-linux-arm.tar.gz 31ea3e4cbcc9574a37566a2cc3c809105d56a739e9cbd387bf878acacedf9ec8
kubernetes-client-windows-386.tar.gz 95420d0d49e2875703ac09a1b6021252644ba162349c6c506b06f2677852de5d
kubernetes-client-windows-amd64.tar.gz 534a3c5bdde989c7339df05c4e7793c6c50e5ebc0a663b1a9cdd25bce43a5a74

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 871a9f35e1c73f571b7113e01a91d7bfc5bfe3501e910c921a18313774b25fd1
kubernetes-server-linux-arm64.tar.gz e13b070ef70d2cea512a839095dbf95249d2f7b5dcbfb378539548c888efe196
kubernetes-server-linux-arm.tar.gz c54cf106e919149731a23da60ad354eadc53b3bf544ab91d4d48ff0c87fdaa7e

Changelog since v1.5.0

Other notable changes

Known Issues for v1.5.1

  • hack/local-up-cluster.sh script times out waiting for apiserver to answer, see #38847. To workaround this, modify the script to pass --anonymous-auth=true to sudo -E "${GO_OUT}/hyperkube" apiserver ... when starting kube-apiserver.

v1.5.0

Documentation & Examples

Downloads for v1.5.0

filename sha256 hash
kubernetes.tar.gz 52b7df98ea05fb3ebbababf1ccb7f6d4e6f4cad00b8d09350f270aa7e3ad7e85
kubernetes-src.tar.gz fbefb2544667f96045c346cee595b0f315282dfdbd41a8f2d5ccc74054a4078e

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 27d71bb6b16a26387ee30272bd4ee5758deccafafdc91b38f3d0dc19a34e129e
kubernetes-client-darwin-amd64.tar.gz 5fa8550235919568d7d839b19de00e9bdd72a97cfde21dbdbe07fefd6d6290dc
kubernetes-client-linux-386.tar.gz 032a17701c014b8bbbb83c7da1046d8992a41031628cf7e1959a94378f5f195b
kubernetes-client-linux-amd64.tar.gz afae4fadb7bbb1532967f88fef1de6458abda17219f634cc2c41608fd83ae7f6
kubernetes-client-linux-arm64.tar.gz acca7607dae678a0165b7e10685e0eff0d418beebe7c25eaffe18c85717b5cc4
kubernetes-client-linux-arm.tar.gz fbc182b6d9ae476c7c509486d773074fd1007032886a8177735e08010c43f89d
kubernetes-client-windows-386.tar.gz a8ddea329bc8d57267294464c163d8c2f7837f6353f8c685271864ed8b8bc54d
kubernetes-client-windows-amd64.tar.gz bc3a76f1414fa1f4b2fb92732de2100d346edb7b870ed5414ea062bb401a8ebd

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz b9c122d709c0556c1e19d31d98bf26ee530f91c0119f4454fb930cef5a0c1aa7
kubernetes-server-linux-arm64.tar.gz 3bbba5c8dedc47db8f9ebdfac5468398cce2470617de9d550affef9702b724c9
kubernetes-server-linux-arm.tar.gz 3ff9ccdd641690fd1c8878408cd369beca1f9f8b212198e251862d40cf2dadc0

Major Themes

  • StatefulSets (ex-PetSets)
    • StatefulSets are beta now (fixes and stabilization)
  • Improved Federation Support
    • New command: kubefed
    • DaemonSets
    • Deployments
    • ConfigMaps
  • Simplified Cluster Deployment
    • Improvements to kubeadm
    • HA Setup for Master
  • Node Robustness and Extensibility
    • Windows Server Container support
    • CRI for pluggable container runtimes
    • kubelet API supports authentication and authorization

Features

Features for this release were tracked via the use of the kubernetes/features issues repo. Each Feature issue is owned by a Special Interest Group from kubernetes/community

  • API Machinery
  • Apps
    • [stable] When replica sets cannot create pods, they will now report detail via the API about the underlying reason (kubernetes/features#120)
    • [stable] kubectl apply is now able to delete resources you no longer need with --prune (kubernetes/features#128)
    • [beta] Deployments that cannot make progress in rolling out the newest version will now indicate via the API they are blocked (docs) (kubernetes/features#122)
    • [beta] StatefulSets allow workloads that require persistent identity or per-instance storage to be created and managed on Kubernetes. (docs) (kubernetes/features#137)
    • [beta] In order to preserve safety guarantees the cluster no longer force deletes pods on un-responsive nodes and users are now warned if they try to force delete pods via the CLI. (docs) (kubernetes/features#119)
  • Auth
  • AWS
  • Cluster Lifecycle
  • Cluster Ops
    • [alpha] Added ability to create/remove clusters w/highly available (replicated) masters on GCE using kube-up/kube-down scripts. (docs) (kubernetes/features#48)
  • Federation
  • Network
    • [stable] Services can reference another service by DNS name, rather than being hosted in pods (kubernetes/features#33)
    • [beta] Opt in source ip preservation for Services with Type NodePort or LoadBalancer (docs) (kubernetes/features#27)
    • [stable] Enable DNS Horizontal Autoscaling with beta ConfigMap parameters support (docs)
  • Node
    • [alpha] Added ability to preserve access to host userns when userns remapping is enabled in container runtime (kubernetes/features#127)
    • [alpha] Introducing the v1alpha1 CRI API to allow pluggable container runtimes; an experimental docker-CRI integration is ready for testing and feedback. (docs) (kubernetes/features#54)
    • [alpha] Kubelet launches container in a per pod cgroup hierarchy based on quality of service tier (kubernetes/features#126)
    • [beta] Kubelet integrates with memcg notification API to detect when a hard eviction threshold is crossed (kubernetes/features#125)
    • [beta] Introducing the beta version containerized node conformance test gcr.io/google_containers/node-test:0.2 for users to verify node setup. (docs) (kubernetes/features#84)
  • Scheduling
  • UI
  • Windows

Known Issues

Populated via v1.5.0 known issues / FAQ accumulator

  • CRI known issues and limitations
  • getDeviceNameFromMount() function doesn't return the volume path correctly when the volume path contains spaces #37712
  • Federation alpha features do not have feature gates defined and are hence enabled by default. This will be fixed in a future release. #38593
  • Federation control plane can be upgraded by updating the image fields in the Deployment specs of the control plane components. However, federation control plane upgrades were not tested in this release 38537

Notable Changes to Existing Behavior

  • Node controller no longer force-deletes pods from the api-server. (#35235, @foxish)

    • For StatefulSet (previously PetSet), this change means creation of replacement pods is blocked until old pods are definitely not running (indicated either by the kubelet returning from partitioned state, deletion of the Node object, deletion of the instance in the cloud provider, or force deletion of the pod from the api-server). This helps prevent "split brain" scenarios in clustered applications by ensuring that unreachable pods will not be presumed dead unless some "fencing" operation has provided one of the above indications.
    • For all other existing controllers except StatefulSet, this has no effect on the ability of the controller to replace pods because the controllers do not reuse pod names (they use generate-name).
    • User-written controllers that reuse names of pod objects should evaluate this change.
    • When deleting an object with kubectl delete ... --grace-period=0, the client will begin a graceful deletion and wait until the resource is fully deleted. To force deletion immediately, use the --force flag. This prevents users from accidentally allowing two Stateful Set pods to share the same persistent volume which could lead to data corruption #37263

  • Allow anonymous API server access, decorate authenticated users with system:authenticated group (#32386, @liggitt)

    • kube-apiserver learned the '--anonymous-auth' flag, which defaults to true. When enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of 'system:anonymous' and a group of 'system:unauthenticated'.
    • Authenticated users are decorated with a 'system:authenticated' group.
    • IMPORTANT: See Action Required for important actions related to this change.

  • kubectl get -o jsonpath=... will now throw an error if the path is to a field not present in the json, even if the path is for a field valid for the type. This is a change from the pre-1.5 behavior, which would return the default value for some fields even if they were not present in the json. (#37991, @pwittrock)

  • The strategicmerge patchMergeKey for VolumeMounts was changed from "name" to "mountPath". This was necessary because the name field refers to the name of the Volume, and is not a unique key for the VolumeMount. Multiple VolumeMounts will have the same Volume name if mounting the same volume more than once. The "mountPath" is verified to be unique and can act as the mergekey. (#35071, @pwittrock)

Deprecations

  • extensions/v1beta1.Jobs is deprecated, use batch/v1.Job instead (#36355, @soltysh)
  • The kubelet --reconcile-cdir flag is deprecated because it has no function anymore. (#35523, @luxas)
  • Notice of deprecation for recycler #36760
  • The init-container (pod.beta.kubernetes.io/init-containers) annotations used to accept capitalized field names that could be accidently generated by the k8s.io/kubernetes/pkg/api package. Using an upper case field name will now return an error and all users should use the versioned API types from pkg/api/v1 when serializing from Golang.

Action Required Before Upgrading

  • **Important Security-related changes before upgrading
    • You MUST set --anonymous-auth=false flag on your kube-apiserver unless you are a developer testing this feature and understand it. If you do not, you risk allowing unauthorized users to access your apiserver.
    • You MUST set --anonymous-auth=false flag on your federation apiserver unless you are a developer testing this feature and understand it. If you do not, you risk allowing unauthorized users to access your federation apiserver.
    • You do not need to adjust this flag on Kubelet: there was no authorization for the Kubelet APIs in 1.4.
  • batch/v2alpha1.ScheduledJob has been renamed, use batch/v2alpha1.CronJob instead (#36021, @soltysh)
  • PetSet has been renamed to StatefulSet. If you have existing PetSets, you must perform extra migration steps both before and after upgrading to convert them to StatefulSets. (docs) (#35663, @janetkuo)
  • If you are upgrading your Cluster Federation components from v1.4.x, please update your federation-apiserver and federation-controller-manager manifests to the new version (#30601, @madhusudancs)
  • The deprecated kubelet --configure-cbr0 flag has been removed, and with that the "classic" networking mode as well. If you depend on this mode, please investigate whether the other network plugins kubenet or cni meet your needs. (#34906, @luxas)
  • New client-go structure, refer to kubernetes/client-go for versioning policy (#34989, @caesarxuchao)
  • The deprecated kube-scheduler --bind-pods-qps and --bind-pods burst flags have been removed, use --kube-api-qps and --kube-api-burst instead (#34471, @timothysc)
  • If you used the PodDisruptionBudget feature in 1.4 (i.e. created PodDisruptionBudget objects), then BEFORE upgrading from 1.4 to 1.5, you must delete all PodDisruptionBudget objects (policy/v1alpha1/PodDisruptionBudget) that you have created. It is not possible to delete these objects after you upgrade, and their presence will prevent you from using the beta PodDisruptionBudget feature in 1.5 (which uses policy/v1beta1/PodDisruptionBudget). If you have already upgraded, you will need to downgrade the master to 1.4 to delete the policy/v1alpha1/PodDisruptionBudget objects.

External Dependency Version Information

Continuous integration builds have used the following versions of external dependencies, however, this is not a strong recommendation and users should consult an appropriate installation or upgrade guide before deciding what versions of etcd, docker or rkt to use.

  • Docker versions 1.10.3 - 1.12.3
    • Docker version 1.11.2 known issues
      • Kernel crash with Aufs storage driver on Debian Jessie (#27885 which can be identified by the node problem detector
      • Leaked File descriptors (#275)
      • Additional memory overhead per container (#21737
    • Docker version 1.12.1 has been validated through the Kubernetes docker automated validation framework as has Docker version 1.12.3
  • Docker 1.10.3 contains backports provided by RedHat for known issues
  • Docker versions as old as may 1.9.1 work with known issues but this is not guaranteed
  • rkt version 1.21.0
  • etcd version 2.2.1

Changelog since v1.5.0-beta.3

Other notable changes

Previous Releases Included in v1.5.0

v1.4.7

Documentation & Examples

Downloads for v1.4.7

filename sha256 hash
kubernetes.tar.gz d193f76e70322010b3e86ac61c7a893175f9e62d37bece87cfd14ea068c8d187
kubernetes-src.tar.gz 7c7ef45e903ed2691c73bb2752805f190b4042ba233a6260f2cdeab7d0ac9bd3

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz a5a3ec9f5270156cf507b4c6bf2d08da67062a2ed9cb5f21e8891f2fd83f438a
kubernetes-client-darwin-amd64.tar.gz e5328781640b19e86b59aa8afd665dd21999c6740acbee8332cfa20745d6a5ce
kubernetes-client-linux-386.tar.gz 61082afc6aee2dc5bbd35bfda2e5991bd9f9730192f1c9396b6db500fc64e121
kubernetes-client-linux-amd64.tar.gz 36232c9e21298f5f53dbf4851520a8cc53a2d6b6d2be8810cf5258a067570314
kubernetes-client-linux-arm64.tar.gz 802d0c5e7bb55dacdd19afe73ed71d0726960ec9933c49e77051df7e2594790b
kubernetes-client-linux-arm.tar.gz f42d8d2d918b31564d12d742bce2263df0c93807619bd03194028ff2714f1a17
kubernetes-client-windows-386.tar.gz b45dcdfe0ba0177fad5419b4fd6b5b80bf9bca0e56e7fe19d2bc217c9aae1f9d
kubernetes-client-windows-amd64.tar.gz ae4666aea8fa74ef1cce746d1d90cbadc972850560b65a8eeff4417fdede6b4e

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 56e01e9788d1ef0499b1783768022cb188b5bb840d1499a62e9f0a18c2bd2bd5
kubernetes-server-linux-arm64.tar.gz 6654ef3c142694a79ec2596929ceec36a399407e1fb74b09be1a67c59b30ca42
kubernetes-server-linux-arm.tar.gz b10e78286dea804d69311e3805c35f5414b0669094edec7a2e0ba99170a5d04a

Changelog since v1.4.6

Other notable changes

v1.5.0-beta.3

Documentation & Examples

Downloads for v1.5.0-beta.3

filename sha256 hash
kubernetes.tar.gz c2b29b38d29829b7b2591559d0d36495d463de0e18a2611bd1d66f2baea6352c
kubernetes-src.tar.gz 0b3327b6f0b024c989aba1e546d50d56fc89ed6df74c09fc55b9f9c4a667b771

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 82a7144ae1371c3320019c8e6a76e95242d85aae9dedccc4884b677cda544c0e
kubernetes-client-darwin-amd64.tar.gz 3aeea90acfbaf776e2c812e34df4c11a44720e4c5b86c4c0e9a8aaf221149335
kubernetes-client-linux-386.tar.gz d55fb1dfe64e62bffbf03f1a7c8bd666562014ad0d438049f0f801f5fa583914
kubernetes-client-linux-amd64.tar.gz 779b2f1c0eb3eca7dd60332972ccfc79e557e34f080c210dfb6aa6e18e71bbf4
kubernetes-client-linux-arm64.tar.gz b5f0a3b23d7082eaefe7090d7a8f9952fd8b00d44a90137200bc5a91001b6e95
kubernetes-client-linux-arm.tar.gz ccadbef7ce7c89fc48988c57585c0ccb7488d2dcc7e96f4e43c5bb64e44b9e29
kubernetes-client-windows-386.tar.gz da1428b6ed138134358c72af570a65565c5188a1c6e50cee42becb1a48441d91
kubernetes-client-windows-amd64.tar.gz 7b74aeb215b0f0ff86bae262af5bafe7083a44293e1ab2545f5de3ac42deda0b

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz c56aa39fd4e732c86a2729aa427ca2fc95130bd788053aa8e8f6a8efd9e1310e
kubernetes-server-linux-arm64.tar.gz 9f55082ca5face2db2d6d54bed2a831622e747e1aa527ee8adc61d0ed3fcfab8
kubernetes-server-linux-arm.tar.gz 4a7c037ac221531eee4e47b66a2aa12fce4044d2d4acbef0e48b09e0a8fe950b

Changelog since v1.5.0-beta.2

Other notable changes

v1.5.0-beta.2

Documentation & Examples

Downloads for v1.5.0-beta.2

filename sha256 hash
kubernetes.tar.gz 4a6cb512dee2312ffe291f4209759309576ca477cf51fb8447b30a7cb2a887ed
kubernetes-src.tar.gz fe71f19b607183da4abf5f537e7ccbe72ac3306b0933ee1f519253c78bf9252f

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 37bcd12754a28ba6b4d030c68526bc6369f1fa3b7b0e405277bb13989ed0f9da
kubernetes-client-darwin-amd64.tar.gz 760817040ca040dd4ba8929cfb714b8bf6704c6ac2ec9985b56fa77b4da03d2c
kubernetes-client-linux-386.tar.gz 87d694445a3e532748d07e0d0da05c1ae8b84b46c54ec1415c9603533747a465
kubernetes-client-linux-amd64.tar.gz b2bcd07a525428fe24da628afca22b019b8f2847d1999da8fce72b7342cf64ed
kubernetes-client-linux-arm64.tar.gz 262c4fa70039389aa5d5b73a0def325471bd24b858157d60c0389fbee5ca671e
kubernetes-client-linux-arm.tar.gz 52c9341c1e6aa923aed4497c061121c192f209c90fcf31135edc45241a684bfa
kubernetes-client-windows-386.tar.gz 7d8e3bcdfa9dc3d5fde70c60a37e543cc59d23b25e2b0a2274e672d0bae013c2
kubernetes-client-windows-amd64.tar.gz 75143c176bc817fc49a79229dfae8c7429d0a3deeaba54a397dddce3e37e8550

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 61c209048da1612796a30b880076b7f9b59038821da63bbecac4c56f24216312
kubernetes-server-linux-arm64.tar.gz 2c6952e16c0b0c153ca3d424b3deca9b43a8e421b1a59359bc10260309bf470c
kubernetes-server-linux-arm.tar.gz cf3e37a89358cae1d2d36aaad10f3e906269bc3df611279dbed9f50e81449fad

Changelog since v1.5.0-beta.1

Other notable changes

  • Modify GCI mounter to enable NFSv3 (#36610, @jingxu97)
  • Third party resources are now deleted when a namespace is deleted. (#35947, @brendandburns)
  • kube-dns (#36775, @bowei)
    • Added --config-map and --config-map-namespace command line options.
    • If --config-map is set, kube-dns will load dynamic configuration from the config map
    • referenced by --config-map-namespace, --config-map. The config-map supports
    • the following properties: "federations".
    • --federations flag is now deprecated. Prefer to set federations via the config-map.
    • Federations can be configured by settings the "federations" field to the value currently
    • set in the command line.
    • Example:
    • kind: ConfigMap
    • apiVersion: v1
    • metadata:
    • name: kube-dns

    • namespace: kube-system
    • data:
    • federations: abc=def
  • azure: support multiple ipconfigs on a NIC (#36841, @colemickens)
  • Fix issue in converting AWS volume ID from mount paths (#36840, @jingxu97)
  • fix leaking memory backed volumes of terminated pods (#36779, @sjenning)
  • Default logging subsystem's resiliency was greatly improved, fluentd memory consumption and OOM error probability was reduced. (#37021, @Crassirostris)

v1.5.0-beta.1

Documentation & Examples

Downloads for v1.5.0-beta.1

filename sha256 hash
kubernetes.tar.gz 62c51bcee460794cda30e720c65509b679b51015c62c075e6e735fe29d089e2b
kubernetes-src.tar.gz 8c950c7377eb40670d0438ccb68bbeaf1100ed2e919e012bc98479ff07ddd393

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz e71af85542837842ff3b0fb8137332f4e1ce4c453d225da292e1fa781f1c74d7
kubernetes-client-darwin-amd64.tar.gz 033d02c1382553f977057827b6a5b82f1b69aecd44b649c937781d1cccb763d1
kubernetes-client-linux-386.tar.gz 1e7a435f2f7d06e3de9bd8c8d0457b6548aa15ad5cdab4241391f290a28b804f
kubernetes-client-linux-amd64.tar.gz 3c07a89e8eb785a7b37842d4b0bc0471fcc7b4e3a4bd973e6f8936cbc6030d76
kubernetes-client-linux-arm64.tar.gz 680a2786d9782395b613e27509df2d0f671a2471a43533ccdbc6b71cfb332072
kubernetes-client-linux-arm.tar.gz 2a5b10fbd69ce9b1da0403a80d71684ee2cf4d75298a5ec19e069ae826da81ed
kubernetes-client-windows-386.tar.gz 10acbf09ffbc04f549d1cffff98a533b456562d5c09a2d0f315523b70072c35d
kubernetes-client-windows-amd64.tar.gz 3317f90da242b0fb95a3cbc669fc4941d7b56b5ff90ac528c166e915bee31fdf

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz fdb257c0bbf64304441fd377a5ee330de10696aa0b5c1b6c27fa73a6c00121ae
kubernetes-server-linux-arm64.tar.gz a174cf6c9351da786b8780f5edca158a4e021d4af597bcc66f238601fb37c2b1
kubernetes-server-linux-arm.tar.gz 1dc520b9a4428321225ba6cfa0f79b702965d7f6994357c15e0195c5af1528ff

Changelog since v1.5.0-alpha.2

Action Required

Other notable changes

  • Federation: allow specification of dns zone by ID (#36336, @justinsb)
  • K8s 1.5 keeps container-vm as the default node image on GCE for backwards compatibility reasons. Please beware that container-vm is officially deprecated (supported with security patches only) and you should replace it with GCI if at all possible. You can review the migration guide here for more detail: https://cloud.google.com/container-engine/docs/node-image-migration (#36822, @mtaufen)
  • Add a flag allowing contention profiling of the API server (#36756, @gmarek)
  • Rename --cgroups-per-qos to --experimental-cgroups-per-qos in Kubelet (#36767, @vishh)
  • Implement CanMount() for gfsMounter for linux (#36686, @rkouj)
  • Default host user namespace via experimental flag (#31169, @pweil-)
  • Use generous limits in the resource usage tracking tests (#36623, @yujuhong)
  • Update Dashboard UI version to 1.4.2 (#35895, @rf232)
  • Add support for service load balancer source ranges to Azure load balancers. (#36696, @brendandburns)
  • gci-dev-56-8977-0-0: (#36681, @mtaufen)
  • Fix strategic patch for list of primitive type with merge sementic (#35647, @ymqytw)
  • Fix issue in reconstruct volume data when kubelet restarts (#36616, @jingxu97)
  • Ensure proper serialization of updates and creates in federation test watcher (#36613, @mwielgus)
  • Add support for SourceIP preservation in Azure LBs (#36557, @brendandburns)
  • Fix fetching pids running in a cgroup, which caused problems with OOM score adjustments & setting the /system cgroup ("misc" in the summary API). (#36551, @timstclair)
  • federation: Adding support for DeleteOptions.OrphanDependents for federated replicasets and deployments. Setting it to false while deleting a federated replicaset or deployment also deletes the corresponding resource from all registered clusters. (#36476, @nikhiljindal)
  • kubectl: show node label if defined (#35901, @justinsb)
  • Migrates addons from RCs to Deployments (#36008, @MrHohn)
  • Avoid setting S_ISGID on files in volumes (#36386, @sjenning)
  • federation: Adding support for DeleteOptions.OrphanDependents for federated daemonsets and ingresses. Setting it to false while deleting a federated daemonset or ingress also deletes the corresponding resource from all registered clusters. (#36330, @nikhiljindal)
  • Add authz to psp admission (#33080, @pweil-)
  • Better messaging for missing volume binaries on host (#36280, @rkouj)
  • Add Windows support to kube-proxy (#36079, @jbhurat)
  • Support persistent volume usage for kubernetes running on Photon Controller platform (#36133, @luomiao)
  • GCI nodes use an external mounter script to mount NFS & GlusterFS storage volumes (#36267, @vishh)
  • Add retry to node scheduability marking. (#36211, @brendandburns)
  • specify custom ca file to verify the keystone server (#35488, @dixudx)
  • AWS: Support default value for ExternalHost (#33568, @justinsb)
  • HPA: Consider unready pods separately (#33593, @DirectXMan12)
  • Node Conformance Test: Containerize the node e2e test (#31093, @Random-Liu)
  • federation: Adding support for DeleteOptions.OrphanDependents for federated secrets. Setting it to false while deleting a federated secret also deletes the corresponding secrets from all registered clusters. (#36296, @nikhiljindal)
  • Deploy kube-dns with cluster-proportional-autoscaler (#33239, @MrHohn)
  • Adds support for StatefulSets in kubectl drain. (#35483, @ymqytw)
    • Switches to use the eviction sub-resource instead of deletion in kubectl drain, if server supports.
  • azure: load balancer preserves destination ip address (#36256, @colemickens)
  • LegacyHostIP will be deprecated in 1.7. (#36095, @caesarxuchao)
  • Fix LBaaS version detection in openstack cloudprovider (#36249, @sjenning)
  • Node Conformance Test: Add system verification (#32427, @Random-Liu)
  • kubelet bootstrap: start hostNetwork pods before we have PodCIDR (#35526, @justinsb)
  • Enable HPA controller based on autoscaling/v1 api group (#36215, @piosz)
  • Remove unused WaitForDetach from Detacher interface and plugins (#35629, @kiall)
  • Initial work on running windows containers on Kubernetes (#31707, @alexbrand)
  • Per Volume Inode Accounting (#35132, @dashpole)
  • [AppArmor] Hold bad AppArmor pods in pending rather than rejecting (#35342, @timstclair)
  • Federation: separate notion of zone-name & dns-suffix (#35372, @justinsb)
  • In order to bypass graceful deletion of pods (to immediately remove the pod from the API) the user must now provide the --force flag in addition to --grace-period=0. This prevents users from accidentally force deleting pods without being aware of the consequences of force deletion. Force deleting pods for resources like StatefulSets can result in multiple pods with the same name having running processes in the cluster, which may lead to data corruption or data inconsistency when using shared storage or common API endpoints. (#35484, @smarterclayton)
  • NPD: Add e2e test for NPD v0.2. (#35740, @Random-Liu)
  • DELETE requests can now pass in their DeleteOptions as a query parameter or a body parameter, rather than just as a body parameter. (#35806, @bdbauer)
  • make using service account credentials from controllers optional (#35970, @deads2k)
  • AWS: strong-typing for k8s vs aws volume ids (#35883, @justinsb)
  • Controller changes for perma failed deployments (#35691, @kargakis)
  • Proxy min sync period (#35334, @timothysc)
  • Federated ConfigMap controller (#35635, @mwielgus)
  • have basic kubectl crud agnostic of registered types (#36085, @deads2k)
  • Fix how we iterate over active jobs when removing them for Replace policy (#36161, @soltysh)
  • Adds TCPCloseWaitTimeout option to kube-proxy for sysctl nf_conntrack_tcp_timeout_time_wait (#35919, @bowei)
  • Pods that are terminating due to eviction by the nodecontroller (typically due to unresponsive kubelet, or network partition) now surface in kubectl get output (#36017, @foxish)
    • as being in state "Unknown", along with a longer description in kubectl describe output.
  • The hostname of the node (as autodetected by the kubelet, specified via --hostname-override, or determined by the cloudprovider) is now recorded as an address of type "Hostname" in the status of the Node API object. The hostname is expected to be resolveable from the apiserver. (#25532, @mkulke)
  • [Kubelet] Add alpha support for --cgroups-per-qos using the configured --cgroup-driver. Disabled by default. (#31546, @derekwaynecarr)
  • Move Statefulset (previously PetSet) to v1beta1 (#35731, @janetkuo)
  • The error handling behavior of pkg/client/restclient.Result has changed. Calls to Result.Raw() will no longer parse the body, although they will still return errors that react to pkg/api/errors.Is*() as in previous releases. Callers of Get() and Into() will continue to receive errors that are parsed from the body if the kind and apiVersion of the body match the Status object. (#36001, @smarterclayton)
    • This more closely aligns rest client as a generic RESTful client, while preserving the special Kube API extended error handling for the Get and Into methods (which most Kube clients use).
  • Making the pod.alpha.kubernetes.io/initialized annotation optional in PetSet pods (#35739, @foxish)
  • AWS: recognize us-east-2 region (#35013, @justinsb)
  • Eviction manager evicts based on inode consumption (#35137, @dashpole)
  • SELinux Overhaul (#33663, @pmorie)
  • Add SNI support to the apiserver (#35109, @sttts)
  • The main kubernetes repository stops hosting archived version of released clients. Please use client-go. (#35928, @caesarxuchao)
  • Correct the article in generated documents (#32557, @asalkeld)
  • Update PodAntiAffinity to ignore calls to subresources (#35608, @soltysh)
  • The apiserver can now select which type of kubelet-reported address to use for apiserver->node communications, using the --kubelet-preferred-address-types flag. (#35497, @liggitt)
  • update list of vailable resources (#32687, @jouve)
  • Remove stale volumes if endpoint/svc creation fails. (#35285, @humblec)
  • add kubectl cp (#34914, @brendandburns)
  • Remove Job also from .status.active for Replace strategy (#35420, @soltysh)
  • Let release_1_5 clientset include multiple versions of a group (#35471, @caesarxuchao)
  • support editing before creating resource (#33250, @ymqytw)
  • allow authentication through a front-proxy (#35452, @deads2k)
  • On GCI, cleanup kubelet startup (#35319, @vishh)
  • Add a retry when reading a file content from a container (#35560, @jingxu97)
  • Fix cadvisor_unsupported and the crossbuild (#35817, @luxas)
  • [PHASE 1] Opaque integer resource accounting. (#31652, @ConnorDoyle)
  • Add sync state loop in master's volume reconciler (#34859, @jingxu97)
  • Bump GCE debian image to container-vm-v20161025 (CVE-2016-5195 Dirty… (#35825, @dchen1107)
  • GC pod ips (#35572, @bprashanth)
  • Stop including arch-specific binaries in kubernetes.tar.gz (#35737, @ixdy)
  • Rename PetSet to StatefulSet (#35663, @janetkuo)
  • Enable containerized storage plugins mounter on GCI (#35350, @vishh)
  • Bump container-vm version in config-test.sh (#35705, @mtaufen)
  • Cadvisor root path configuration (#35136, @dashpole)
  • ssh pubkey parsing: prevent segfault (#35323, @mikkeloscar)

v1.4.6

Documentation & Examples

Downloads for v1.4.6

filename sha256 hash
kubernetes.tar.gz 6f8242aa29493e1f824997748419e4a287c28b06ed13f17b1ba94bf07fdfa3be
kubernetes-src.tar.gz a2a2d885d246300b52adb5d7e1471b382c77d90a816618518c2a6e9941208e40

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 4db6349c976f893d0000dcb5b2ab09327824d0c38b3beab961711a0951cdfc82
kubernetes-client-darwin-amd64.tar.gz 2d31dea858569f518410effb20d3c3b9a6798d706dacbafd85f1f67f9ccbe288
kubernetes-client-linux-386.tar.gz 7980cf6132a7a6bf3816b8fd60d7bc1c9cb447d45196c31312b9d73567010909
kubernetes-client-linux-amd64.tar.gz 95b3cbd339f7d104d5b69b08d53060bfc78bd4ee7a94ede7ba4c0a76b615f8b1
kubernetes-client-linux-arm64.tar.gz 0f03cff262b0f4cc218b0f79294b4cbd8f92146c31137c75a27012d956864c79
kubernetes-client-linux-arm.tar.gz f8c76fe8c41a5084cc1a1ab3e08d7e2d815f7baedfadac0dc6f9157ed2c607c9
kubernetes-client-windows-386.tar.gz c29b3c8c8a72246852db048e922ad2221f35e1c309571f73fd9f3d9b01be5f79
kubernetes-client-windows-amd64.tar.gz 95bf20bdbe354476bbd3647adf72985698ded53a59819baa8268b5811e19f952

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz f0a60c45f3360696431288826e56df3b8c18c1dc6fc3f0ea83409f970395e38f
kubernetes-server-linux-arm64.tar.gz 8c667d4792fcfee821a2041e5d0356e1abc2b3fa6fe7b69c5479e48c858ba29c
kubernetes-server-linux-arm.tar.gz c57246d484b5f98d6aa16591f2b4c4c1a01ebbc7be05bce8690a4f3b88582844

Changelog since v1.4.5

Other notable changes

  gci-beta-55-8872-47-0:
  Date:           Nov 11, 2016
  Kernel:         ChromiumOS-4.4
  Kubernetes:     v1.4.5
  Docker:         v1.11.2
  Changelog (vs 55-8872-18-0)
    * Cherry-pick runc PR#608: Eliminate redundant parsing of mountinfo
    * Updated kubernetes to v1.4.5
    * Fixed a bug in e2fsprogs that caused mke2fs to take a very long time. Upstream fix: http://git.kernel.org/cgit/fs/ext2/e2fsprogs.git/commit/?h=next&id=d33e690fe7a6cbeb51349d9f2c7fb16a6ebec9c2
  • Fix fetching pids running in a cgroup, which caused problems with OOM score adjustments & setting the /system cgroup ("misc" in the summary API). (#36614, @timstclair)
  • DELETE requests can now pass in their DeleteOptions as a query parameter or a body parameter, rather than just as a body parameter. (#35806, @bdbauer)
  • rkt: Convert image name to be a valid acidentifier (#34375, @euank)
  • Remove stale volumes if endpoint/svc creation fails. (#35285, @humblec)
  • Remove Job also from .status.active for Replace strategy (#35420, @soltysh)
  • Update PodAntiAffinity to ignore calls to subresources (#35608, @soltysh)
  • Adds TCPCloseWaitTimeout option to kube-proxy for sysctl nf_conntrack_tcp_timeout_time_wait (#35919, @bowei)
  • Fix how we iterate over active jobs when removing them for Replace policy (#36161, @soltysh)
  • Bump GCI version to latest m55 version in GCE for K8s 1.4 (#36302, @mtaufen)
  • Add a check for file size if the reading content returns empty (#33976, @jingxu97)
  • Add a retry when reading a file content from a container (#35560, @jingxu97)
  • Skip CLOSE_WAIT e2e test if server is 1.4.5 (#36404, @bowei)
  • Adds etcd3 changes (#36232, @wojtek-t)
  • Adds TCPCloseWaitTimeout option to kube-proxy for sysctl nf_conntrack_tcp_timeout_time_wait (#36099, @bowei)

v1.3.10

Documentation & Examples

Downloads for v1.3.10

filename sha256 hash
kubernetes.tar.gz 0f61517fbab1feafbe1024da0b88bfe16e61fed7e612285d70e3ecb53ce518cf
kubernetes-src.tar.gz 7b1be0dcc12ae1b0cb1928b770c1025755fd0858ce7520907bacda19e5bfa53f

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 64a7012411a506ff7825e7b9c64b50197917d6f4e1128ea0e7b30a121059da47
kubernetes-client-darwin-amd64.tar.gz 5d85843e643eaebe3e34e48810f4786430b5ecce915144e01ba2d8539aa77364
kubernetes-client-linux-386.tar.gz 06d478c601b1d4aa1fc539e9120adbcbbd2fb370d062516f84a064e465d8eadc
kubernetes-client-linux-amd64.tar.gz fe571542482b8ba3ff94b9e5e9657f6ab4fc0feb8971930dc80b7ae2548d669b
kubernetes-client-linux-arm64.tar.gz 176b52d35150ca9f08a7e90e33e2839b7574afe350edf4fafa46745d77bb5aa4
kubernetes-client-linux-arm.tar.gz 1c3bf4ac1e4eb0e02f785db725efd490beaf06c8acd26d694971ba510b60a94d
kubernetes-client-linux-ppc64le.tar.gz 172cd0af71fcba7c51e9476732dbe86ba251c03b1d74f912111e4e755be540ce
kubernetes-client-windows-386.tar.gz f2d2f82d7e285c98d8cc58a8a6e13a1122c9f60bb2c73e4cefe3555f963e56cd
kubernetes-client-windows-amd64.tar.gz ac0aa2b09dfeb8001e76f3aefe82c7bd2fda5bd0ef744ac3aed966b99c8dc8e5

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz bf0d3924ff84c95c316fcb4b21876cc019bd648ca8ab87fd6b2712ccda30992b
kubernetes-server-linux-arm64.tar.gz 45e88d1c8edc17d7f1deab8d040a769d8647203c465d76763abb1ce445a98773
kubernetes-server-linux-arm.tar.gz 40ac46a265021615637f07d532cd563b4256dcf340a27c594bfd3501fe66b84c
kubernetes-server-linux-ppc64le.tar.gz faa5075ab3e6688666bbbb274fa55a825513ee082a3b17bcddb5b8f4fd6f9aa0

Changelog since v1.3.9

Other notable changes

  • gci: decouple from the built-in kubelet version (#31367, @Amey-D)
  • Bump GCE debian image to container-vm-v20161025 (CVE-2016-5195 Dirty… (#35825, @dchen1107)
  • Add RELEASE_INFRA_PUSH related code to support pushes from kubernetes/release. (#28922, @david-mcmahon)

v1.4.5

Documentation & Examples

Downloads for v1.4.5

filename sha256 hash
kubernetes.tar.gz 339f4d1c7a374ddb32334268c4af8dae0b86d1567a9c812087d672a7defe233c
kubernetes-src.tar.gz 69b1b022400794d491200a9365ea9bf735567348d0299920462cf7167c76ba61

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 6012dab54687f7eb41ce9cd6b4676e15b774fbfbeadb7e00c806ba3f63fe10ce
kubernetes-client-darwin-amd64.tar.gz 981b321f4393fc9892c6558321e1d8ee6d8256b85f09266c8794fdcee9cb1c07
kubernetes-client-linux-386.tar.gz 75ce408ef9f4b277718701c025955cd628eeee4180d8e9e7fd8ecf008878429f
kubernetes-client-linux-amd64.tar.gz 0c0768d7646cec490ca1e47a4e2f519724fc75d984d411aa92fe17a82356532b
kubernetes-client-linux-arm64.tar.gz 910a6465b1ecbf1aae8f6cd16e35ac7ad7b0e598557941937d02d16520e2e37c
kubernetes-client-linux-arm.tar.gz 29644cca627cdce6c7aad057d9680eee87d21b1bbd6af02f7277f24eccbc95f7
kubernetes-client-windows-386.tar.gz dc249cc0f6cbb0e0705f7b43929461b6702ae91148218da070bb99e8a8f6f108
kubernetes-client-windows-amd64.tar.gz d60d275ad5f45ebe83a458912de96fd8381540d4bcf91023fe2173af6acd535b

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 25e12aaf3f93c320f6aa640bb1430d4c0e99e3b0e83bcef660d2a513bdef2c20
kubernetes-server-linux-arm64.tar.gz e768146c9476b96f092409030349b4c5bb9682287567fe2732888ad5ed1d3ede
kubernetes-server-linux-arm.tar.gz 26581dc0fc31542c831a588baad9ad391598e5b2ff299a0fc92a2c04990b3edd

Changelog since v1.4.4

Other notable changes

v1.5.0-alpha.2

Documentation & Examples

Downloads for v1.5.0-alpha.2

filename sha256 hash
kubernetes.tar.gz 77f04c646657b683210a17aeca62e56bf985702b267942b41729406970c40cee
kubernetes-src.tar.gz f6090cc853e56159099bf12169f0d84e29fd2c055b0c7dbdac755ee94439a6a6

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 917adbc70156d55371c1aea62279a521e930e7ff130728aa176505f0268182e3
kubernetes-client-darwin-amd64.tar.gz 9c8084eeab05b6db0508f789cb8a05b4f864ee23ea37b43e17af0026fb67defa
kubernetes-client-linux-386.tar.gz 3498f9cd73bb947b7cd8c4e5fb3ebe0676fbc98cf346a807f1b7c252aa068d68
kubernetes-client-linux-amd64.tar.gz e9bf2e48212bb275b113d0a1f6091c4692126c8af3c4e0a986e483ec27190e82
kubernetes-client-linux-arm64.tar.gz 9c514a482d4dd44d64f3d47eb3d64b434343f10abdecf1b5176ff0078d3b7008
kubernetes-client-linux-arm.tar.gz c51a8ebc2c3ca2f914042a6017852feb315fd3ceba8b0d5186349b553da11fdb
kubernetes-client-windows-386.tar.gz 32b006e1f9e6c14fe399806bb82ec4bf8658ab9828753d1b14732bb8dbb72062
kubernetes-client-windows-amd64.tar.gz 1e142f1fe76bdd660b4f1be51eef4e51705585fccb94e674a7d891ffe8c3b4e3

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 4a3b550a1ede8bebd14413a37e3fc10c8403a3e3fbbce096de443351d076817a
kubernetes-server-linux-arm64.tar.gz 00e58bb04bf150c554f28d8fd2f72fbdd1e7918999aaea9c88c91c8f71946ffe
kubernetes-server-linux-arm.tar.gz 6837ff73249c0f3e7ba2d7c00321274db0f97b5cd0b4dc58d5cc3a2119e1c820

Changelog since v1.5.0-alpha.1

Action Required

  • Deprecate the --reconcile-cidr kubelet flag because it has no function anymore (#35523, @luxas)
  • Removed the deprecated kubelet --configure-cbr0 flag, and with that the "classic" networking mode as well (#34906, @luxas)
  • New client-go structure (#34989, @caesarxuchao)
  • Remove scheduler flags that were marked as deprecated 2+ releases ago. (#34471, @timothysc)

Other notable changes

  • Make the fake RESTClient usable by all the API groups, not just core. (#35492, @madhusudancs)
  • Adding support for DeleteOptions.OrphanDependents for federated namespaces. Setting it to false while deleting a federated namespace also deletes the corresponding namespace from all registered clusters. (#34648, @nikhiljindal)
  • Kubelet flag '--mounter-path' renamed to '--experimental-mounter-path' (#35646, @vishh)
  • Node status updater should SetNodeStatusUpdateNeeded if it fails to update status (#34368, @jingxu97)
  • Deprecate OpenAPI spec for GroupVersion endpoints in favor of single spec /swagger.json (#35388, @mbohlool)
  • kubelet authn/authz (#34381, @liggitt)
  • Fix volume states out of sync problem after kubelet restarts (#33616, @jingxu97)
  • Added rkt binary to GCI (#35321, @vishh)
  • Fixed mutation warning in Attach/Detach controller (#35273, @jsafrane)
  • Don't count failed pods as "not-ready" (#35404, @brendandburns)
  • fixed typo in script which made setting custom cidr in gce using kube-up impossible (#35267, @tommywo)
  • The podGC controller will now always run, irrespective of the value supplied to the "terminated-pod-gc-threshold" flag supplied to the controller manager. (#35476, @foxish)
    • The specific behavior of the podGC controller to clean up terminated pods is still governed by the flag, but the podGC's responsibilities have evolved beyond just cleaning up terminated pods.
  • Update grafana version used by default in kubernetes to 3.1.1 (#35435, @Crassirostris)
  • vSphere Kube-up: resolve vm-names on all nodes (#35365, @kerneltime)
  • bootstrap: Start hostNetwork pods even if network plugin not ready (#33347, @justinsb)
  • Factor out post-init swagger and OpenAPI routes (#32590, @sttts)
  • Substitute gcloud regex with regexp (#35346, @bprashanth)
  • Remove support for multi-architecture code in kubeadm, which was released untested. (#35124, @errordeveloper)
  • vSphere kube-up: Wait for cbr0 configuration to complete before setting up routes. (#35232, @kerneltime)
  • Remove last probe time from replica sets (#35199, @kargakis)
  • Update the GCI image to gci-dev-55-8872-18-0 (#35243, @maisem)
  • Add --mounter-path flag to kubelet that will allow overriding the mount command used by kubelet (#34994, @jingxu97)
  • Fix a bug under the rkt runtime whereby image-registries with ports would not be fetched from (#34375, @euank)
  • Updated default Elasticsearch and Kibana used for elasticsearch logging destination to versions 2.4.1 and 4.6.1 respectively. (#34969, @Crassirostris)
  • Loadbalanced client src ip preservation enters beta (#33957, @bprashanth)
  • Add NodePort value in kubectl output (#34922, @zreigz)
  • kubectl drain now waits until pods have been delete from the Node before exiting (#34778, @ymqytw)
  • Don't report FS stats for system containers in the Kubelet Summary API (#34998, @timstclair)
  • Fixed flakes caused by petset tests. (#35158, @foxish)
  • Add validation that detects repeated keys in the labels and annotations maps (#34407, @brendandburns)
  • Change merge key for VolumeMount to mountPath (#35071, @thockin)
  • kubelet: storage: don't hang kubelet on unresponsive nfs (#35038, @sjenning)
  • Fix kube vsphere.kerneltime (#34997, @kerneltime)
  • Add PSP support for seccomp profiles (#28300, @pweil-)
  • Updated Go to 1.7 (#28742, @jessfraz)
  • HPA: fixed wrong count for target replicas calculations (#34821). (#34955, @jszczepkowski)
  • Improves how 'kubectl' uses the terminal size when printing help and usage. (#34502, @fabianofranz)
  • Updated Elasticsearch image from version 1.5.1 to version 2.4.1. Updated Kibana image from version 4.0.2 to version 4.6.1. (#34562, @Crassirostris)
  • libvirt-coreos: Download the coreos_production_qemu_image over SSL. (#34646, @roberthbailey)
  • Add a new global option "--request-timeout" to the kubectl client (#33958, @juanvallejo)
  • Add support for admission controller based on namespace node selectors. (#24980, @aveshagarwal)
  • Add 'kubectl set resources' (#27206, @JacobTanenbaum)
  • Support trust id as a scope in the OpenStack authentication logic (#32111, @MatMaul)
  • Only wait for cache syncs once in NodeController (#34851, @ncdc)
  • NodeController waits for informer sync before doing anything (#34809, @gmarek)
  • azure: lower log priority for skipped nic update message (#34730, @colemickens)
  • Security Group support for OpenStack Load Balancers (#31921, @grahamhayes)
  • Make NodeController recognize deletion tombstones (#34786, @davidopp)
  • Delete all firewall rules (and optionally network) on GCE/GKE cluster teardown (#34577, @ixdy)
  • Fix panic in NodeController caused by receiving DeletedFinalStateUnknown object from the cache. (#34694, @gmarek)
  • azure: add PrimaryAvailabilitySet to config, only use nodes in that set in the loadbalancer pool (#34526, @colemickens)
  • Fix leaking ingress resources in federated ingress e2e test. (#34652, @quinton-hoole)
  • pvc.Spec.Resources.Requests min and max can be enforced with a LimitRange of type "PersistentVolumeClaim" in the namespace (#30145, @markturansky)
  • Federated DaemonSet controller. Supports all the API that regular DaemonSet has. (#34319, @mwielgus)
  • New federation deployment mechanism now allows non-GCP clusters. (#34620, @madhusudancs) * Writes the federation kubeconfig to the local kubeconfig file.
  • Update the series and the README to reflect the change. (#30374, @mbruzek)
  • Replica set conditions API (#33905, @kargakis)
  • etcd3: avoid unnecessary decoding in etcd3 client (#34435, @wojtek-t)
  • Test x509 intermediates correctly (#34524, @liggitt)
  • Add cifs-utils to the hyperkube image. (#34416, @colemickens)
  • etcd3: use PrevKV to remove additional get (#34246, @hongchaodeng)
  • Fix upgrade.sh image setup (#34468, @mtaufen)

v1.2.7

Documentation & Examples

Downloads for v1.2.7

filename sha256 hash
kubernetes.tar.gz 53db157923c17fa7a0addb3e4dfe7d1b9194b9266a87d371a251d5bb790a1832
kubernetes-src.tar.gz e6e46831706743d8263581d0575507cf5ffc265096d22e5e84cf1c3ae925db5e

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz 8418767e45c62c2ef5f9b4479ed02af64e190ce07dcbafa1920e93e71f419c55
kubernetes-client-darwin-amd64.tar.gz 41d742c2c55e7686311978eaaddee3844b990a0fe49fa8597158bcb0ee4c05c9
kubernetes-client-linux-386.tar.gz 619e0a450cddf10ed1d42ed1d6330d41a75b9c1e00eb654cbe4b0422cd6099c5
kubernetes-client-linux-amd64.tar.gz 9a5fcd87514b88eb25173e574aef5b5343816c07ab5947d06787c9f12c40f54a
kubernetes-client-linux-arm.tar.gz fd6e39b4a56e03448382825f27f4f30a2e981a8d20f4a8cedbd084bbb4577d42
kubernetes-client-windows-386.tar.gz 862625cb3d9445cff1b09e4ebcdb60dd93b5b2dc34bb6022d2eeed7c8d8bc5d8
kubernetes-client-windows-amd64.tar.gz 054337e41187e39950de93e4670bc78a95b6901cc2f95c50ff437d9825ae94c5

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz fef041e9cbe5bcf8fd708f81ee2e2783429af1ab9cfb151d645ef9be96e19b73
kubernetes-server-linux-arm.tar.gz ce02d7bcd75c31db4f7b9922c19ea2a3312b0ba579b0dcd96b279b661eca18a8

Changelog since v1.2.6

Other notable changes

v1.4.4

Documentation & Examples

Downloads for v1.4.4

filename sha256 hash
kubernetes.tar.gz 2732bfc56ceabc872b6af3f460cbda68c2384c95a1c0c72eb33e5ff0e03dc9da
kubernetes-src.tar.gz 29c6cf1567e6b7f6c3ecb71acead083b7535b22ac20bd8166b29074e8a0f6441

Client Binaries

filename sha256 hash
kubernetes-client-darwin-386.tar.gz e983b1837e4165e4bc8e361000468421f16dbd5ae90b0c49af6280dbcecf57b1
kubernetes-client-darwin-amd64.tar.gz 8c58231c8340e546336b70d86b6a76285b9f7a0c13b802b350b68610dfaedb35
kubernetes-client-linux-386.tar.gz 33e5d2da52325367db08bcc80791cef2e21fdae176b496b063b3a37115f3f075
kubernetes-client-linux-amd64.tar.gz 5fd6215ef0673f5a8e385660cf233d67d26dd79568c69e2328b103fbf1bd752a
kubernetes-client-linux-arm64.tar.gz 2d6d0400cd59b042e2da074cbd3b13b9dc61da1dbba04468d67119294cf72435
kubernetes-client-linux-arm.tar.gz ff99f26082a77e37caa66aa07ec56bfc7963e6ac782550be5090a8b158f7e89a
kubernetes-client-windows-386.tar.gz 82e762727a8f607180a1e339e058cc9739ad55960d3517c5170bcd5b64179f13
kubernetes-client-windows-amd64.tar.gz 4de735ba72c729589efbcd2b8fc4920786fffd96850173c13cbf469819d00808

Server Binaries

filename sha256 hash
kubernetes-server-linux-amd64.tar.gz 6d5ff37941328df33c0efc5876bb7b82722bc584f1976fe632915db7bf3f316a
kubernetes-server-linux-arm64.tar.gz 6ec40848ea29c0982b89c746d716b0958438a6eb774aea20a5ef7885a7060aed
kubernetes-server-linux-arm.tar.gz 43d6a3260d73cfe652af2ffa7b7092444fe57429cb45e90eb99f0a70012ee033

Changelog since v1.4.3

Other notable changes

v1.3.9

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz a994c732d2b852bbee55a78601d50d046323021a99b0801aea07dacf64c2c59a

Changelog since v1.3.8

Other notable changes

v1.4.3

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz c3dccccc005bc22eaf814ccb8e72b4f876167ab38ac594bb7e44c98f162a0f1c

Changelog since v1.4.2-beta.1

Other notable changes

  • Fix non-starting node controller in 1.4 branch (#34895, @wojtek-t)
  • Cherrypick #34851 "Only wait for cache syncs once in NodeController" (#34861, @jessfraz)
  • NodeController waits for informer sync before doing anything (#34809, @gmarek)
  • Make NodeController recognize deletion tombstones (#34786, @davidopp)
  • Fix panic in NodeController caused by receiving DeletedFinalStateUnknown object from the cache. (#34694, @gmarek)
  • Update GlusterFS provisioning readme with endpoint/service details (#31854, @humblec)
  • Add logging for enabled/disabled API Groups (#32198, @deads2k)
  • New federation deployment mechanism now allows non-GCP clusters. (#34620, @madhusudancs) * Writes the federation kubeconfig to the local kubeconfig file.

v1.4.2

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 0730e207944ca96c9d9588a571a5eff0f8fdbb0e1287423513a2b2a4baca9f77

Changelog since v1.4.2-beta.1

Other notable changes

  • Cherrypick #34851 "Only wait for cache syncs once in NodeController" (#34861, @jessfraz)
  • NodeController waits for informer sync before doing anything (#34809, @gmarek)
  • Make NodeController recognize deletion tombstones (#34786, @davidopp)
  • Fix panic in NodeController caused by receiving DeletedFinalStateUnknown object from the cache. (#34694, @gmarek)
  • Update GlusterFS provisioning readme with endpoint/service details (#31854, @humblec)
  • Add logging for enabled/disabled API Groups (#32198, @deads2k)
  • New federation deployment mechanism now allows non-GCP clusters. (#34620, @madhusudancs) * Writes the federation kubeconfig to the local kubeconfig file.

v1.5.0-alpha.1

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 86bfcfffaa210ddf18983ff066470ef9c06ee00449b2238043e2777aac2c906d

Changelog since v1.4.0-alpha.3

Experimental Features

Action Required

  • kube-apiserver learned the '--anonymous-auth' flag, which defaults to true. When enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of 'system:anonymous' and a group of 'system:unauthenticated'. (#32386, @liggitt)
    • Authenticated users are decorated with a 'system:authenticated' group.
    • NOTE: anonymous access is enabled by default. If you rely on authentication alone to authorize access, change to use an authorization mode other than AlwaysAllow, or or set '--anonymous-auth=false'.
  • The NamespaceExists and NamespaceAutoProvision admission controllers have been removed. (#31250, @derekwaynecarr)
    • All cluster operators should use NamespaceLifecycle.
  • Federation binaries and their corresponding docker images - federation-apiserver and federation-controller-manager are now folded in to the hyperkube binary. If you were using one of these binaries or docker images, please switch to using the hyperkube version. Please refer to the federation manifests - federation/manifests/federation-apiserver.yaml and federation/manifests/federation-controller-manager-deployment.yaml for examples. (#29929, @madhusudancs)

Other notable changes

  • The kube-apiserver --service-account-key-file option can be specified multiple times, or can point to a file containing multiple keys, to enable rotation of signing keys. (#34029, @liggitt)
  • The apiserver now uses addresses reported by the kubelet in the Node object's status for apiserver->kubelet communications, rather than the name of the Node object. The address type used defaults to InternalIP, ExternalIP, and LegacyHostIP address types, in that order. (#33718, @justinsb)
  • Federated deployment controller that supports the same api as the regular kubernetes deployment controller. (#34109, @mwielgus)
  • Match GroupVersionKind against specific version (#34010, @soltysh)
  • fix yaml decode issue (#34297, @AdoHe)
  • kubectl annotate now supports --dry-run (#34199, @asalkeld)
  • kubectl: Add external ip information to node when '-o wide' is used (#33552, @floreks)
  • Update GCI base image: (#34156, @adityakali) * Enabled VXLAN and IP_SET config options in kernel to support some networking tools (ebtools) * OpenSSL CVE fixes
  • ContainerVm/GCI image: try to use ifdown/ifup if available (#33595, @freehan)
  • Use manifest digest (as docker-pullable://) as ImageID when available (exposes a canonical, pullable image ID for containers). (#33014, @DirectXMan12)
  • Add kubelet awareness to taint tolerant match caculator. (#26501, @resouer)
  • Fix nil pointer issue when getting metrics from volume mounter (#34251, @jingxu97)
  • Enforce Disk based pod eviction with GCI base image in Kubelet (#33520, @vishh)
  • Remove headers that are unnecessary for proxy target (#34076, @mbohlool)
  • Add missing argument to log message in federated ingress controller. (#34158, @quinton-hoole)
  • The kubelet --eviction-minimum-reclaim option can now take precentages as well as absolute values for resources quantities (#33392, @sjenning)
  • The implicit registration of Prometheus metrics for workqueue has been removed, and a plug-able interface was added. If you were using workqueue in your own binaries and want these metrics, add the following to your imports in the main package: "k8s.io/pkg/util/workqueue/prometheus". (#33792, @caesarxuchao)
  • Add kubectl --node-port option for specifying the service nodeport (#33319, @juanvallejo)
  • To reduce memory usage to reasonable levels in smaller clusters, kube-apiserver now sets the deserialization cache size based on the target memory usage. (#34000, @wojtek-t)
  • use service accounts as clients for controllers (#33310, @deads2k)
  • Add a new option "--local" to the kubectl annotate (#34074, @asalkeld)
  • Add a new option "--local" to the kubectl label (#33990, @asalkeld)
  • Initialize podsWithAffinity to avoid scheduler panic (#33967, @xiang90)
  • Fix base image pinning during upgrades via cluster/gce/upgrade.sh (#33147, @vishh)
  • Remove the flannel experimental overlay (#33862, @luxas)
  • CRI: Remove the mount name and port name. (#33970, @yifan-gu)
  • Enable kubectl describe rs to work when apiserver does not support pods (#33794, @nikhiljindal)
  • Heal the namespaceless ingresses in federation e2e. (#33977, @quinton-hoole)
  • Fix issue in updating device path when volume is attached multiple times (#33796, @jingxu97)
  • ECDSA keys can now be used for signing and verifying service account tokens. (#33565, @liggitt)
  • OnlyLocal nodeports (#33587, @bprashanth)
  • Remove flannel because now everything here is upstreamed (#33860, @luxas)
  • Use patched golang1.7.1 for cross-builds targeting darwin (#33803, @ixdy)
  • Bump up addon kube-dns to v20 for graceful termination (#33774, @MrHohn)
  • Creating LoadBalancer Service with "None" ClusterIP is no longer possible (#33274, @nebril)
  • Increase timeout for federated ingress test. (#33610, @quinton-hoole)
  • Use UpdateStatus, not Update, to add LoadBalancerStatus to Federated Ingress. (#33605, @quinton-hoole)
  • add anytoken authenticator (#33378, @deads2k)
  • Fixes in HPA: consider only running pods; proper denominator in avg request calculations. (#33735, @jszczepkowski)
  • When CORS Handler is enabled, we now add a new HTTP header named "Access-Control-Expose-Headers" with a value of "Date". This allows the "Date" HTTP header to be accessed from XHR/JavaScript. (#33242, @dims)
  • promote contrib/mesos to incubator (#33658, @deads2k)
  • MinReadySeconds / AvailableReplicas for ReplicaSets (#32771, @kargakis)
  • Kubectl drain will now drain finished Pods (#31763, @fraenkel)
  • Adds the -deployment option to e2e.go, adds the ability to run e2e.go using a kops deployment. (#33518, @zmerlynn)
  • Tune down initialDelaySeconds for readinessProbe. (#33146, @MrHohn)
  • kube-proxy: Add a lower-bound for conntrack (128k default) (#33051, @thockin)
  • local-up-cluster.sh: add SERVICE_CLUSTER_IP_RANGE as option (#32921, @aanm)
  • Default HTTP2 on, post fixes from #29001 (#32231, @timothysc)
  • Split dns healthcheck into two different urls (#32406, @MrHohn)
  • Remove kubectl namespace command (#33275, @maciaszczykm)
  • Automatic generation of man pages (#33277, @mkumatag)
  • Fixes memory/goroutine leak in Federation Service controller. (#33359, @shashidharatd)
  • Switch k8s on GCE to use GCI by default (#33353, @vishh)
  • Move HighWaterMark to the top of the struct in order to fix arm, second time (#33376, @luxas)
  • Fix race condition in setting node statusUpdateNeeded flag (#32807, @jingxu97)
  • Fix the DOCKER_OPTS appending bug. (#33163, @DjangoPeng)
  • Send recycle events from pod to pv. (#27714, @jsafrane)
  • Add port forwarding for rkt with kvm stage1 (#32126, @jjlakis)
  • The value of the versioned.Event object (returned by watch APIs) in the Swagger 1.2 schemas has been updated from *versioned.Event which was not expected by many client tools. The new value is consistent with other structs returned by the API. (#33007, @smarterclayton)
  • Remove cpu limits for dns pod to avoid CPU starvation (#33227, @vishh)
  • Allow secure access to apiserver from Admission Controllers (#31491, @dims)
  • Resolves x509 verification issue with masters dialing nodes when started with --kubelet-certificate-authority (#33141, @liggitt)
  • Fix possible panic in PodAffinityChecker (#33086, @ivan4th)
  • Upgrading Container-VM base image for k8s on GCE. Brief changelog as follows: (#32738, @Amey-D) 
    • - Fixed performance regression in veth device driver

    • - Docker and related binaries are statically linked

    • - Fixed the issue of systemd being oom-killable
  • Move HighWaterMark to the top of the struct in order to fix arm (#33117, @luxas)
  • kubenet: SyncHostports for both running and ready to run pods. (#31388, @yifan-gu)
  • Limit the number of names per image reported in the node status (#32914, @yujuhong)
  • Support Quobyte as StorageClass (#31434, @johscheuer)
  • Use a patched go1.7.1 for building linux/arm (#32517, @luxas)
  • Add line break after events in kubectl describe (#31463, @fabianofranz)
  • Specific error message on failed rolling update issued by older kubectl against 1.4 master (#32751, @caesarxuchao)
  • Make the informer library available for the go client library. (#32718, @mikedanese)
  • Added --log-facility flag to enhance dnsmasq logging (#32422, @MrHohn)
  • Set Dashboard UI to final 1.4 version (#32666, @bryk)
  • Fix audit_test regex for iso8601 timestamps (#32593, @johnbieren)
  • Docker digest validation is too strict (#32627, @smarterclayton)
  • Bumped Heapster to v1.2.0. (#32649, @piosz)
  • add local subject access review API (#32407, @deads2k)
  • make --runtime-config=api/all=true|false work (#32582, @jlowdermilk)
  • Added new kubelet flags --cni-bin-dir and --cni-conf-dir to specify where CNI files are located. (#32151, @bboreham)
    • Fixed CNI configuration on GCI platform when using CNI.
  • Move push-ci-build.sh to kubernetes/release repo (#32444, @david-mcmahon)
  • vendor: update github.com/coreos/go-oidc client package (#31564, @ericchiang)
  • Fixed an issue that caused a credential error when deploying federation control plane onto a GKE cluster. (#31747, @madhusudancs)
  • Error if a contextName is provided but not found in the kubeconfig. (#31767, @asalkeld)
  • Use a Deployment for kube-dns (#32018, @MrHohn)
  • Support graceful termination in kube-dns (#31894, @MrHohn)
  • When prompting for passwords, don't echo to the terminal (#31586, @brendandburns)
  • add group prefix matching for kubectl usage (#32140, @deads2k)
  • Stick to 2.2.1 etcd (#32404, @caesarxuchao)
  • Fix a bug in kubelet hostport logic which flushes KUBE-MARK-MASQ iptables chain (#32413, @freehan)
  • Make sure finalizers prevent deletion on storage that supports graceful deletion (#32351, @caesarxuchao)
  • AWS: Change default networking for kube-up to kubenet (#32239, @zmerlynn)
  • Use etcd 2.3.7 (#32359, @wojtek-t)
  • Allow missing keys in jsonpath (#31714, @smarterclayton)
  • Changes 'kubectl rollout status' to wait until all updated replicas are available before finishing. (#31499, @areed)
  • add selfsubjectaccessreview API (#31271, @deads2k)
  • Add kubectl describe cmd support for vSphere volume (#31045, @abrarshivani)
  • Enable kubelet eviction whenever inodes free is < 5% on GCE (#31545, @vishh)
  • Use federated namespace instead of the bootstrap cluster's namespace in Ingress e2e tests. (#32105, @madhusudancs)
  • Move StorageClass to a storage group (#31886, @deads2k)
  • Some components like kube-dns and kube-proxy could fail to load the service account token when started within a pod. Properly handle empty configurations to try loading the service account config. (#31947, @smarterclayton)
  • Removed comments in json config when using kubectl edit with -o json (#31685, @jellonek)
  • fixes invalid null selector issue in sysdig example yaml (#31393, @baldwinSPC)
  • Rescheduler which ensures that critical pods are always scheduled enabled by default in GCE. (#31974, @piosz)
  • retry oauth token fetch in gce cloudprovider (#32021, @mikedanese)
  • Deprecate the old cbr0 and flannel networking modes (#31197, @freehan)
  • AWS: fix volume device assignment race condition (#31090, @justinsb)
  • The certificates API group has been renamed to certificates.k8s.io (#31887, @liggitt)
  • Increase Dashboard UI version to v1.4.0-beta2 (#31518, @bryk)
  • Fixed incomplete kubectl bash completion. (#31333, @xingzhou)
  • Added liveness probe to Heapster service. (#31878, @mksalawa)
  • Adding clusters to the list of valid resources printed by kubectl help (#31719, @nikhiljindal)
  • Kubernetes server components using kubeconfig files no longer default to http://localhost:8080. Administrators must specify a server value in their kubeconfig files. (#30808, @smarterclayton)
  • Update influxdb to 0.12 (#31519, @piosz)
  • Include security options in the container created event (#31557, @timstclair)
  • Federation can now be deployed using the federation/deploy/deploy.sh script. This script does not depend on any of the development environment shell library/scripts. This is an alternative to the current federation-up.sh/federation-down.sh scripts. Both the scripts are going to co-exist in this release, but the federation-up.sh/federation-down.sh scripts might be removed in a future release in favor of federation/deploy/deploy.sh script. (#30744, @madhusudancs)
  • Add get/delete cluster, delete context to kubectl config (#29821, @alexbrand)
  • rkt: Force rkt fetch to fetch from remote to conform the image pull policy. (#31378, @yifan-gu)
  • Allow services which use same port, different protocol to use the same nodePort for both (#30253, @AdoHe)
  • Handle overlapping deployments gracefully (#30730, @janetkuo)
  • Remove environment variables and internal Kubernetes Docker labels from cAdvisor Prometheus metric labels. (#31064, @grobie)
    • Old behavior:
      • environment variables explicitly whitelisted via --docker-env-metadata-whitelist were exported as container_env_*=*. Default is zero so by default non were exported
      • all docker labels were exported as container_label_*=*
    • New behavior:
      • Only container_name, pod_name, namespace, id, image, and name labels are exposed
      • no environment variables will be exposed ever via /metrics, even if whitelisted
  • Filter duplicate network packets in promiscuous bridge mode (with ebtables) (#28717, @freehan)
  • Refactor to simplify the hard-traveled path of the KubeletConfiguration object (#29216, @mtaufen)
  • Fix overflow issue in controller-manager rate limiter (#31396, @foxish)

v1.4.2-beta.1

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz b72986a0adcb7e08feb580c5d72de129ac2ecc128c154fd79785bac2d2e760f7

Changelog since v1.4.1

Other notable changes

v1.4.1

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz b51971d872426ba71bb09b9a9191bb95fc0e48390dc287a9080e3876c8e19a95

Changelog since v1.4.1-beta.2

No notable changes for this release

v1.4.1-beta.2

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 708fbaabf17a69c69c2c9a715e152a29d47334b8c98d217ba17e9b42d6770f25

Changelog since v1.4.0

Other notable changes

  • Update GCI base image: (#34156, @adityakali) * Enabled VXLAN and IP_SET config options in kernel to support some networking tools (ebtools) * OpenSSL CVE fixes
  • ContainerVm/GCI image: try to use ifdown/ifup if available (#33595, @freehan)
  • Make the informer library available for the go client library. (#32718, @mikedanese)
  • Enforce Disk based pod eviction with GCI base image in Kubelet (#33520, @vishh)
  • Fix nil pointer issue when getting metrics from volume mounter (#34251, @jingxu97)
  • Enable kubectl describe rs to work when apiserver does not support pods (#33794, @nikhiljindal)
  • Increase timeout for federated ingress test. (#33610, @quinton-hoole)
  • Remove headers that are unnecessary for proxy target (#34076, @mbohlool)
  • Support graceful termination in kube-dns (#31894, @MrHohn)
  • Added --log-facility flag to enhance dnsmasq logging (#32422, @MrHohn)
  • Split dns healthcheck into two different urls (#32406, @MrHohn)
  • Tune down initialDelaySeconds for readinessProbe. (#33146, @MrHohn)
  • Bump up addon kube-dns to v20 for graceful termination (#33774, @MrHohn)
  • Send recycle events from pod to pv. (#27714, @jsafrane)
  • Limit the number of names per image reported in the node status (#32914, @yujuhong)
  • Fixes in HPA: consider only running pods; proper denominator in avg request calculations. (#33735, @jszczepkowski)
  • Fix audit_test regex for iso8601 timestamps (#32593, @johnbieren)
  • Limit the number of names per image reported in the node status (#32914, @yujuhong)
  • Fix the DOCKER_OPTS appending bug. (#33163, @DjangoPeng)
  • Remove cpu limits for dns pod to avoid CPU starvation (#33227, @vishh)
  • Fixes memory/goroutine leak in Federation Service controller. (#33359, @shashidharatd)
  • Use UpdateStatus, not Update, to add LoadBalancerStatus to Federated Ingress. (#33605, @quinton-hoole)
  • Initialize podsWithAffinity to avoid scheduler panic (#33967, @xiang90)
  • Heal the namespaceless ingresses in federation e2e. (#33977, @quinton-hoole)
  • Add missing argument to log message in federated ingress controller. (#34158, @quinton-hoole)
  • Fix issue in updating device path when volume is attached multiple times (#33796, @jingxu97)
  • To reduce memory usage to reasonable levels in smaller clusters, kube-apiserver now sets the deserialization cache size based on the target memory usage. (#34000, @wojtek-t)
  • Fix possible panic in PodAffinityChecker (#33086, @ivan4th)
  • Fix race condition in setting node statusUpdateNeeded flag (#32807, @jingxu97)
  • kube-proxy: Add a lower-bound for conntrack (128k default) (#33051, @thockin)
  • Use patched golang1.7.1 for cross-builds targeting darwin (#33803, @ixdy)
  • Move HighWaterMark to the top of the struct in order to fix arm (#33117, @luxas)
  • Move HighWaterMark to the top of the struct in order to fix arm, second time (#33376, @luxas)

v1.3.8

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 66cf72d8f07e2f700acfcb11536694e0d904483611ff154f34a8380c63720a8d

Changelog since v1.3.7

Other notable changes

v1.4.0

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 6cf3d78230f7659b87fa399a56a7aaed1fde6a73be9d05e25feedacfbd8d5a16

Major Themes

  • Simplified User Experience
    • Easier to get a cluster up and running (eg: kubeadm, intra-cluster bootstrapping)
    • Easier to understand a cluster (eg: API audit logs, server-based API defaults)
  • Stateful Appplication Support
    • Enhanced persistence capabilities (eg: StorageClasses, new volume plugins)
    • New resources and scheduler features (eg: ScheduledJob resource, pod/node affinity/anti-affinity)
  • Cluster Federation
    • Global Multi-cluster HTTP(S) Ingress across GCE and GKE clusters.
    • Expanded support for federated hybrid-cloud resources including ReplicaSets, Secrets, Namespaces and Events.
  • Security
    • Increased pod-level security granularity (eg: Container Image Policies, AppArmor and sysctl support)
    • Increased cluster-level security granularity (eg: Access Review API)

Features

This is the first release tracked via the use of the kubernetes/features issues repo. Each Feature issue is owned by a Special Interest Group from kubernetes/community

  • API Machinery
    • [alpha] Generate audit logs for every request user performs against secured API server endpoint. (docs) (kubernetes/features#22)
    • [beta] kube-apiserver now publishes a swagger 2.0 spec in addition to a swagger 1.2 spec (kubernetes/features#53)
    • [beta] Server-side garbage collection is enabled by default. See user-guide
  • Apps
    • [alpha] Introducing 'ScheduledJobs', which allow running time based Jobs, namely once at a specified time or repeatedly at specified point in time. (docs) (kubernetes/features#19)
  • Auth
    • [alpha] Container Image Policy allows an access controller to determine whether a pod may be scheduled based on a policy (docs) (kubernetes/features#59)
    • [alpha] Access Review APIs expose authorization engine to external inquiries for delegation, inspection, and debugging (docs) (kubernetes/features#37)
  • Cluster Lifecycle
    • [alpha] Ensure critical cluster infrastructure pods (Heapster, DNS, etc.) can schedule by evicting regular pods when necessary to make the critical pods schedule. (docs) (kubernetes/features#62)
    • [alpha] Simplifies bootstrapping of TLS secured communication between the API server and kubelet. (docs) (kubernetes/features#43)
    • [alpha] The kubeadm tool makes it much easier to bootstrap Kubernetes. (docs) (kubernetes/features#11)
  • Federation
    • [alpha] Creating a Federated Ingress is as simple as submitting an Ingress creation request to the Federation API Server. The Federation control system then creates and maintains a single global virtual IP to load balance incoming HTTP(S) traffic across some or all the registered clusters, across all regions. Google's GCE L7 LoadBalancer is the first supported implementation, and is available in this release. (docs) (kubernetes/features#82)
    • [beta] Federated Replica Sets create and maintain matching Replica Sets in some or all clusters in a federation, with the desired replica count distributed equally or according to specified per-cluster weights. (docs) (kubernetes/features#46)
    • [beta] Federated Secrets are created and kept consistent across all clusters in a federation. (docs) (kubernetes/features#68)
    • [beta] Federation API server gained support for events and many federation controllers now report important events. (docs) (kubernetes/features#70)
    • [alpha] Creating a Federated Namespace causes matching Namespaces to be created and maintained in all the clusters registered with that federation. (docs) (kubernetes/features#69)
    • [alpha] ingress has alpha support for a single master multi zone cluster (docs) (kubernetes/features#52)
  • Network
  • Node
  • Scheduling
    • [alpha] Allows pods to require or prohibit (or prefer or prefer not) co-scheduling on the same node (or zone or other topology domain) as another set of pods. (docs (kubernetes/features#51)
  • Storage
  • UI
    • [stable] Kubernetes Dashboard UI - a great looking Kubernetes Dashboard UI with 90% CLI parity for at-a-glance management. docs
    • [stable] kubectl no longer applies defaults before sending objects to the server in create and update requests, allowing the server to apply the defaults. (kubernetes/features#55)

Known Issues

  • Completed pods lose logs across node upgrade (#32324)
  • Pods are deleted across node upgrade (#32323)
  • Secure master -> node communication (#11816)
  • upgrading master doesn't upgrade kubectl (#32538)
  • Specific error message on failed rolling update issued by older kubectl against 1.4 master (#32751)
  • bump master cidr range from /30 to /29 (#32886)
  • non-hostNetwork daemonsets will almost always have a pod that fails to schedule (#32900)
  • Service loadBalancerSourceRanges doesn't respect updates (#33033)
  • disallow user to update loadbalancerSourceRanges (#33346)

Notable Changes to Existing Behavior

Deployments

  • ReplicaSets of paused Deployments are now scaled while the Deployment is paused. This is retroactive to existing Deployments.
  • When scaling a Deployment during a rollout, the ReplicaSets of all Deployments are now scaled proportionally based on the number of replicas they each have instead of only scaling the newest ReplicaSet.

kubectl rolling-update: < v1.4.0 client vs >=v1.4.0 cluster

Old version kubectl's rolling-update command is compatible with Kubernetes 1.4 and higher only if you specify a new replication controller name. You will need to update to kubectl 1.4 or higher to use the rolling update command against a 1.4 cluster if you want to keep the original name, or you'll have to do two rolling updates.

If you do happen to use old version kubectl's rolling update against a 1.4 cluster, it will fail, usually with an error message that will direct you here. If you saw that error, then don't worry, the operation succeeded except for the part where the new replication controller is renamed back to the old name. You can just do another rolling update using kubectl 1.4 or higher to change the name back: look for a replication controller that has the original name plus a random suffix.

Unfortunately, there is a much rarer second possible failure mode: the replication controller gets renamed to the old name, but there is a duplicated set of pods in the cluster. kubectl will not report an error since it thinks its job is done.

If this happens to you, you can wait at most 10 minutes for the replication controller to start a resync, the extra pods will then be deleted. Or, you can manually trigger a resync by change the replicas in the spec of the replication controller.

kubectl delete: < v1.4.0 client vs >=v1.4.0 cluster

If you use an old version kubectl to delete a replication controller or replicaset, then after the delete command has returned, the replication controller or the replicaset will continue to exist in the key-value store for a short period of time (<1s). You probably will not notice any difference if you use kubectl manually, but you might notice it if you are using kubectl in a script.

DELETE operation in REST API

  • Replication controller & Replicaset: the DELETE request of a replication controller or a replicaset becomes asynchronous by default. The object will continue to exist in the key-value store for some time. The API server will set its metadata.deletionTimestamp, add the "orphan" finalizer to its metadata.finalizers. The object will be deleted from the key-value store after the garbage collector orphans its dependents. Please refer to this user-guide for more information regarding the garbage collection.

  • Other objects: no changes unless you explicitly request orphaning.

Action Required Before Upgrading

  • If you are using Kubernetes to manage docker containers, please be aware Kubernetes has been validated to work with docker 1.9.1, docker 1.11.2 (#23397), and docker 1.12.0 (#28698)
  • If you upgrade your apiserver to 1.4.x but leave your kubelets at 1.3.x, they will not report init container status, but init containers will work properly. Upgrading kubelets to 1.4.x fixes this.
  • The NamespaceExists and NamespaceAutoProvision admission controllers have been removed, use the NamespaceLifecycle admission controller instead (#31250, @derekwaynecarr)
  • If upgrading Cluster Federation components from 1.3.x, the federation-apiserver and federation-controller-manager binaries have been folded into hyperkube. Please switch to using that instead. (#29929, @madhusudancs)
  • If you are using the PodSecurityPolicy feature (eg: kubectl get podsecuritypolicy does not error, and returns one or more objects), be aware that init containers have moved from alpha to beta. If there are any pods with the key pods.beta.kubernetes.io/init-containers, then that pod may not have been filtered by the PodSecurityPolicy. You should find such pods and either delete them or audit them to ensure they do not use features that you intend to be blocked by PodSecurityPolicy. (#31026, @erictune)
  • If upgrading Cluster Federation components from 1.3.x, please ensure your cluster name is a valid DNS label (#30956, @nikhiljindal)
  • kubelet's --config flag has been deprecated, use --pod-manifest-path instead (#29999, @mtaufen)
  • If upgrading Cluster Federation components from 1.3.x, be aware the federation-controller-manager now looks for a different secret name. Run the following to migrate (#28938, @madhusudancs)
kubectl --namespace=federation get secret federation-apiserver-secret -o json | sed 's/federation-apiserver-secret/federation-apiserver-kubeconfig/g' | kubectl create -f -
# optionally, remove the old secret
kubectl delete secret --namespace=federation federation-apiserver-secret
  • Kubernetes components no longer handle panics, and instead actively crash. All Kubernetes components should be run by something that actively restarts them. This is true of the default setups, but those with custom environments may need to double-check (#28800, @lavalamp)
  • kubelet now defaults to --cloud-provider=auto-detect, use --cloud-provider='' to preserve previous default of no cloud provider (#28258, @vishh)

Previous Releases Included in v1.4.0

For a detailed list of all changes that were included in this release, please refer to the following CHANGELOG entries:

v1.4.0-beta.11

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 993e785f501d2fa86c9035b55a875c420059b3541a32b5822acf5fefb9a61916

Changelog since v1.4.0-beta.10

No notable changes for this release

v1.4.0-beta.10

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz f3f1f0e5cf8234d640c8e9444c73343f04be8685f92b6a1ad66190f84de2e3a7

Changelog since v1.4.0-beta.8

Other notable changes

  • Remove cpu limits for dns pod to avoid CPU starvation (#33227, @vishh)
  • Resolves x509 verification issue with masters dialing nodes when started with --kubelet-certificate-authority (#33141, @liggitt)
  • Upgrading Container-VM base image for k8s on GCE. Brief changelog as follows: (#32738, @Amey-D) 
    • - Fixed performance regression in veth device driver

    • - Docker and related binaries are statically linked

    • - Fixed the issue of systemd being oom-killable
  • Update cAdvisor to v0.24.0 - see the cAdvisor changelog for the full list of changes. (#33052, @timstclair)

v1.4.0-beta.8

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 31701c5c675c137887b58d7914e39b4c8a9c03767c0c3d89198a52f4476278ca

Changelog since v1.4.0-beta.7

No notable changes for this release

v1.4.0-beta.7

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 51e8f3ebe55cfcfbe582dd6e5ea60ae125d89373477571c0faee70eff51bab31

Changelog since v1.4.0-beta.6

Other notable changes

  • Use a patched go1.7.1 for building linux/arm (#32517, @luxas)
  • Specific error message on failed rolling update issued by older kubectl against 1.4 master (#32751, @caesarxuchao)

v1.4.0-beta.6

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 0b0158e4745663b48c55527247d3e64cc3649f875fa7611fc7b38fa5c3b736bd

Changelog since v1.4.0-beta.5

Other notable changes

  • Set Dashboard UI to final 1.4 version (#32666, @bryk)

v1.4.0-beta.5

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz ec6b233b0448472e05e6820b8ea1644119ae4f9fe3a1516cf978117c19bad0a9

Changelog since v1.4.0-beta.3

Other notable changes

v1.3.7

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz ad18566a09ff87b36107c2ea238fa5e20988d7a62c85df9c8598920679fec4a1

Changelog since v1.3.6

Other notable changes

  • AWS: Add ap-south-1 to list of known AWS regions (#28428, @justinsb)
  • Back porting critical vSphere bug fixes to release 1.3 (#31993, @dagnello)
  • Back port - Openstack provider allowing more than one service port for lbaas v2 (#32001, @dagnello)
  • Fix a bug in kubelet hostport logic which flushes KUBE-MARK-MASQ iptables chain (#32413, @freehan)
  • Fixes the panic that occurs in the federation controller manager when registering a GKE cluster to the federation. Fixes issue #30790. (#30940, @madhusudancs)

v1.4.0-beta.3

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 5a6802703c6b0b652e72166a4347fee7899c46205463f6797dc78f8086876465

Changelog since v1.4.0-beta.2

No notable changes for this release

Behavior changes caused by enabling the garbage collector

kubectl rolling-update

Old version kubectl's rolling-update command is compatible with Kubernetes 1.4 and higher only if you specify a new replication controller name. You will need to update to kubectl 1.4 or higher to use the rolling update command against a 1.4 cluster if you want to keep the original name, or you'll have to do two rolling updates.

If you do happen to use old version kubectl's rolling update against a 1.4 cluster, it will fail, usually with an error message that will direct you here. If you saw that error, then don't worry, the operation succeeded except for the part where the new replication controller is renamed back to the old name. You can just do another rolling update using kubectl 1.4 or higher to change the name back: look for a replication controller that has the original name plus a random suffix.

Unfortunately, there is a much rarer second possible failure mode: the replication controller gets renamed to the old name, but there is a duplicate set of pods in the cluster. kubectl will not report an error since it thinks its job is done.

If this happens to you, you can wait at most 10 minutes for the replication controller to start a resync, the extra pods will then be deleted. Or, you can manually trigger a resync by change the replicas in the spec of the replication controller.

kubectl delete

If you use an old version kubectl to delete a replication controller or a replicaset, then after the delete command has returned, the replication controller or the replicaset will continue to exist in the key-value store for a short period of time (<1s). You probably will not notice any difference if you use kubectl manually, but you might notice it if you are using kubectl in a script. To fix it, you can poll the API server to confirm the object is deleted.

DELETE operation in REST API

  • Replication controller & Replicaset: the DELETE request of a replication controller or a replicaset becomes asynchronous by default. The object will continue to exist in the key-value store for some time. The API server will set its metadata.deletionTimestamp, add the "orphan" finalizer to its metadata.finalizers. The object will be deleted from the key-value store after the garbage collector orphans its dependents. Please refer to this user-guide for more information regarding the garbage collection.

  • Other objects: no changes unless you explicitly request orphaning.

v1.4.0-beta.2

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 0c6f54eb9059090c88f10a448ed5bcb6ef663abbd76c79281fd8dcb72faa6315

Changelog since v1.4.0-beta.1

Other notable changes

  • Fix a bug in kubelet hostport logic which flushes KUBE-MARK-MASQ iptables chain (#32413, @freehan)
  • Stick to 2.2.1 etcd (#32404, @caesarxuchao)
  • Use etcd 2.3.7 (#32359, @wojtek-t)
  • AWS: Change default networking for kube-up to kubenet (#32239, @zmerlynn)
  • Make sure finalizers prevent deletion on storage that supports graceful deletion (#32351, @caesarxuchao)
  • Some components like kube-dns and kube-proxy could fail to load the service account token when started within a pod. Properly handle empty configurations to try loading the service account config. (#31947, @smarterclayton)
  • Use federated namespace instead of the bootstrap cluster's namespace in Ingress e2e tests. (#32105, @madhusudancs)

v1.4.0-beta.1

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 837296455933629b6792a8954f2c5b17d55c1149c12b644101f2f02549d06d25

Changelog since v1.4.0-alpha.3

Action Required

  • The NamespaceExists and NamespaceAutoProvision admission controllers have been removed. (#31250, @derekwaynecarr)
    • All cluster operators should use NamespaceLifecycle.
  • Federation binaries and their corresponding docker images - federation-apiserver and federation-controller-manager are now folded in to the hyperkube binary. If you were using one of these binaries or docker images, please switch to using the hyperkube version. Please refer to the federation manifests - federation/manifests/federation-apiserver.yaml and federation/manifests/federation-controller-manager-deployment.yaml for examples. (#29929, @madhusudancs)
  • Use upgraded container-vm by default on worker nodes for GCE k8s clusters (#31023, @vishh)

Other notable changes

  • Enable kubelet eviction whenever inodes free is < 5% on GCE (#31545, @vishh)
  • Move StorageClass to a storage group (#31886, @deads2k)
  • Some components like kube-dns and kube-proxy could fail to load the service account token when started within a pod. Properly handle empty configurations to try loading the service account config. (#31947, @smarterclayton)
  • Removed comments in json config when using kubectl edit with -o json (#31685, @jellonek)
  • fixes invalid null selector issue in sysdig example yaml (#31393, @baldwinSPC)
  • Rescheduler which ensures that critical pods are always scheduled enabled by default in GCE. (#31974, @piosz)
  • retry oauth token fetch in gce cloudprovider (#32021, @mikedanese)
  • Deprecate the old cbr0 and flannel networking modes (#31197, @freehan)
  • AWS: fix volume device assignment race condition (#31090, @justinsb)
  • The certificates API group has been renamed to certificates.k8s.io (#31887, @liggitt)
  • Increase Dashboard UI version to v1.4.0-beta2 (#31518, @bryk)
  • Fixed incomplete kubectl bash completion. (#31333, @xingzhou)
  • Added liveness probe to Heapster service. (#31878, @mksalawa)
  • Adding clusters to the list of valid resources printed by kubectl help (#31719, @nikhiljindal)
  • Kubernetes server components using kubeconfig files no longer default to http://localhost:8080. Administrators must specify a server value in their kubeconfig files. (#30808, @smarterclayton)
  • Update influxdb to 0.12 (#31519, @piosz)
  • Include security options in the container created event (#31557, @timstclair)
  • Federation can now be deployed using the federation/deploy/deploy.sh script. This script does not depend on any of the development environment shell library/scripts. This is an alternative to the current federation-up.sh/federation-down.sh scripts. Both the scripts are going to co-exist in this release, but the federation-up.sh/federation-down.sh scripts might be removed in a future release in favor of federation/deploy/deploy.sh script. (#30744, @madhusudancs)
  • Add get/delete cluster, delete context to kubectl config (#29821, @alexbrand)
  • rkt: Force rkt fetch to fetch from remote to conform the image pull policy. (#31378, @yifan-gu)
  • Allow services which use same port, different protocol to use the same nodePort for both (#30253, @AdoHe)
  • Handle overlapping deployments gracefully (#30730, @janetkuo)
  • Remove environment variables and internal Kubernetes Docker labels from cAdvisor Prometheus metric labels. (#31064, @grobie)
    • Old behavior:
      • environment variables explicitly whitelisted via --docker-env-metadata-whitelist were exported as container_env_*=*. Default is zero so by default non were exported
      • all docker labels were exported as container_label_*=*
    • New behavior:
      • Only container_name, pod_name, namespace, id, image, and name labels are exposed
      • no environment variables will be exposed ever via /metrics, even if whitelisted
  • Filter duplicate network packets in promiscuous bridge mode (with ebtables) (#28717, @freehan)
  • Refactor to simplify the hard-traveled path of the KubeletConfiguration object (#29216, @mtaufen)
  • Fix overflow issue in controller-manager rate limiter (#31396, @foxish)

v1.3.6

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 2db7ace2f72a2e162329a6dc969a5a158bb8c5d0f8054c5b1b2b1063aa22020d

Changelog since v1.3.5

Other notable changes

  • Addresses vSphere Volume Attach limits (#29881, @dagnello)
  • Increase request timeout based on termination grace period (#31275, @dims)
  • Skip safe to detach check if node API object no longer exists (#30737, @saad-ali)
  • Nodecontroller doesn't flip readiness on pods if kubeletVersion < 1.2.0 (#30828, @bprashanth)
  • Update cadvisor to v0.23.9 to fix a problem where attempting to gather container filesystem usage statistics could result in corrupted devicemapper thin pool storage for Docker. (#30307, @sjenning)

v1.4.0-alpha.3

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 8055f0373e3b6bdee865749ef9bcfc765396a40f39ec2fa3cd31b675d1bbf5d9

Changelog since v1.4.0-alpha.2

Action Required

  • Moved init-container feature from alpha to beta. (#31026, @erictune)
    • Security Action Required:
    • This only applies to you if you use the PodSecurityPolicy feature. You are using that feature if kubectl get podsecuritypolicy returns one or more objects. If it returns an error, you are not using it.
    • If there are any pods with the key pods.beta.kubernetes.io/init-containers, then that pod may not have been filtered by the PodSecurityPolicy. You should find such pods and either delete them or audit them to ensure they do not use features that you intend to be blocked by PodSecurityPolicy.
    • Explanation of Feature
    • In 1.3, an init container is specified with this annotation key
    • on the pod or pod template: pods.alpha.kubernetes.io/init-containers.
    • In 1.4, either that key or this key: pods.beta.kubernetes.io/init-containers,
    • can be used.
    • When you GET an object, you will see both annotation keys with the same values.
    • You can safely roll back from 1.4 to 1.3, and things with init-containers
    • will still work (pods, deployments, etc).
    • If you are running 1.3, only use the alpha annotation, or it may be lost when
    • rolling forward.
    • The status has moved from annotation key
    • pods.beta.kubernetes.io/init-container-statuses to
    • pods.beta.kubernetes.io/init-container-statuses.
    • Any code that inspects this annotation should be changed to use the new key.
    • State of Initialization will continue to be reported in both pods.alpha.kubernetes.io/initialized
    • and in podStatus.conditions.{status: "True", type: Initialized}
  • Action required: federation-only: Please update your cluster name to be a valid DNS label. (#30956, @nikhiljindal)
    • Updating federation.v1beta1.Cluster API to disallow subdomains as valid cluster names. Only DNS labels are allowed as valid cluster names now.
  • [Kubelet] Rename --config to --pod-manifest-path. --config is deprecated. (#29999, @mtaufen)

Other notable changes

  • rkt: Improve support for privileged pod (pod whose all containers are privileged) (#31286, @yifan-gu)
  • The pod annotation security.alpha.kubernetes.io/sysctls now allows customization of namespaced and well isolated kernel parameters (sysctls), starting with kernel.shm_rmid_forced, net.ipv4.ip_local_port_range and net.ipv4.tcp_syncookies for Kubernetes 1.4. (#27180, @sttts)
    • The pod annotation security.alpha.kubernetes.io/unsafe-sysctls allows customization of namespaced sysctls where isolation is unclear. Unsafe sysctls must be enabled at-your-own-risk on the kubelet with the --experimental-allowed-unsafe-sysctls flag. Future versions will improve on resource isolation and more sysctls will be considered safe.
  • Increase request timeout based on termination grace period (#31275, @dims)
  • Fixed two issues of kubectl bash completion. (#31135, @xingzhou)
  • Reduced size of fluentd images. (#31239, @aledbf)
  • support Azure data disk volume (#29836, @rootfs)
  • fix Openstack provider to allow more than one service port for lbaas v2 (#30649, @dagnello)
  • Add kubelet --network-plugin-mtu flag for MTU selection (#30376, @justinsb)
  • Let Services preserve client IPs and not double-hop from external LBs (alpha) (#29409, @girishkalele)
  • [Kubelet] Optionally consume configuration from named config maps (#30090, @mtaufen)
  • [GarbageCollector] Allow per-resource default garbage collection behavior (#30838, @caesarxuchao)
  • Action required: If you have a running federation control plane, you will have to ensure that for all federation resources, the corresponding namespace exists in federation control plane. (#31139, @nikhiljindal)
    • federation-apiserver now supports NamespaceLifecycle admission control, which is enabled by default. Set the --admission-control flag on the server to change that.
  • Configure webhook (#30923, @Q-Lee)
  • Federated Ingress Controller (#30419, @quinton-hoole)
  • Federation replicaset controller (#29741, @jianhuiz)
  • AWS: More ELB attributes via service annotations (#30695, @krancour)
  • Impersonate user extra (#30881, @deads2k)
  • DNS, Heapster and UI are critical addons (#30995, @piosz)
  • AWS: Support HTTP->HTTP mode for ELB (#30563, @knarz)
  • kube-up: Allow IP restrictions for SSH and HTTPS API access on AWS. (#27061, @Naddiseo)
  • Add readyReplicas to replica sets (#29481, @kargakis)
  • The implicit registration of Prometheus metrics for request count and latency have been removed, and a plug-able interface was added. If you were using our client libraries in your own binaries and want these metrics, add the following to your imports in the main package: "k8s.io/pkg/client/metrics/prometheus". (#30638, @krousey)
  • Add support for --image-pull-policy to 'kubectl run' (#30614, @AdoHe)
  • x509 authenticator: get groups from subject's organization field (#30392, @ericchiang)
  • Add initial support for TokenFile to to the client config file. (#29696, @brendandburns)
  • update kubectl help output for better organization (#25524, @AdoHe)
  • daemonset controller should respect taints (#31020, @mikedanese)
  • Implement TLS bootstrap for kubelet using --experimental-bootstrap-kubeconfig (2nd take) (#30922, @yifan-gu)
  • rkt: Support subPath volume mounts feature (#30934, @yifan-gu)
  • Return container command exit codes in kubectl run/exec (#26541, @sttts)
  • Fix kubectl describe to display a container's resource limit env vars as node allocatable when the limits are not set (#29849, @aveshagarwal)
  • The valueFrom.fieldRef.name field on environment variables in pods and objects with pod templates now allows two additional fields to be used: (#27880, @smarterclayton) * spec.nodeName will return the name of the node this pod is running on * spec.serviceAccountName will return the name of the service account this pod is running under
  • Adding ImagePolicyWebhook admission controller. (#30631, @ecordell)
  • Validate involvedObject.Namespace matches event.Namespace (#30533, @liggitt)
  • allow group impersonation (#30803, @deads2k)
  • Always return command output for exec probes and kubelet RunInContainer (#30731, @ncdc)
  • Enable the garbage collector by default (#30480, @caesarxuchao)
  • use valid_resources to replace kubectl.PossibleResourceTypes (#30955, @lojies)
  • oidc auth provider: don't trim issuer URL (#30944, @ericchiang)
  • Add a short -n for kubectl --namespace (#30630, @silasbw)
  • Federated secret controller (#30669, @kshafiee)
  • Add Events for operation_executor to show status of mounts, failed/successful to show in describe events (#27778, @screeley44)
  • Alpha support for OpenAPI (aka. Swagger 2.0) specification served on /swagger.json (enabled by default) (#30233, @mbohlool)
  • Disable linux/ppc64le compilation by default (#30659, @ixdy)
  • Implement dynamic provisioning (beta) of PersistentVolumes via StorageClass (#29006, @jsafrane)
  • Allow setting permission mode bits on secrets, configmaps and downwardAPI files (#28936, @rata)
  • Skip safe to detach check if node API object no longer exists (#30737, @saad-ali)
  • The Kubelet now supports the --require-kubeconfig option which reads all client config from the provided --kubeconfig file and will cause the Kubelet to exit with error code 1 on error. It also forces the Kubelet to use the server URL from the kubeconfig file rather than the --api-servers flag. Without this flag set, a failure to read the kubeconfig file would only result in a warning message. (#30798, @smarterclayton)
    • In a future release, the value of this flag will be defaulted to true.
  • Adding container image verification webhook API. (#30241, @Q-Lee)
  • Nodecontroller doesn't flip readiness on pods if kubeletVersion < 1.2.0 (#30828, @bprashanth)
  • AWS: Handle kube-down case where the LaunchConfig is dangling (#30816, @zmerlynn)
  • kubectl will no longer do client-side defaulting on create and replace. (#30250, @krousey)
  • Added warning msg for kubectl get (#28352, @vefimova)
  • Removed support for HPA in extensions client. (#30504, @piosz)
  • Implement DisruptionController. (#25921, @mml)
  • [Kubelet] Check if kubelet is running as uid 0 (#30466, @vishh)
  • Fix third party APIResource reporting (#29724, @brendandburns)
  • speed up RC scaler (#30383, @deads2k)
  • Set pod state as "unknown" when CNI plugin fails (#30137, @nhlfr)
  • Cluster Federation components can now be built and deployed using the make command. Please see federation/README.md for details. (#29515, @madhusudancs)
  • Adding events to federation control plane (#30421, @nikhiljindal)
  • [kubelet] Introduce --protect-kernel-defaults flag to make the tunable behaviour configurable (#27874, @ingvagabund)
  • Add support for kube-up.sh to deploy Calico network policy to GCI masters (#29037, @matthewdupre)
  • Added 'kubectl top' command showing the resource usage metrics. (#28844, @mksalawa)
  • Add basic audit logging (#27087, @soltysh)
  • Marked NodePhase deprecated. (#30005, @dchen1107)
  • Name the job created by scheduledjob (sj) deterministically with sj's name and a hash of job's scheduled time. (#30420, @janetkuo)
  • add metrics for workqueues (#30296, @deads2k)
  • Adding ingress resource to federation apiserver (#30112, @nikhiljindal)
  • Update Dashboard UI to version v1.1.1 (#30273, @bryk)
  • Update etcd 2.2 references to use 3.0.x (#29399, @timothysc)
  • HPA: ignore scale targets whose replica count is 0 (#29212, @sjenning)
  • Add total inodes to kubelet summary api (#30231, @derekwaynecarr)
  • Updates required for juju kubernetes to use the tls-terminated etcd charm. (#30104, @mbruzek)
  • Fix PVC.Status.Capacity and AccessModes after binding (#29982, @jsafrane)
  • allow a read-only rbd image mounted by multiple pods (#29622, @rootfs)
  • [kubelet] Auto-discover node IP if neither cloud provider exists and IP is not explicitly specified (#29907, @luxas)
  • kubectl config set-crentials: add arguments for auth providers (#30007, @ericchiang)
  • Scheduledjob controller (#29137, @janetkuo)
  • add subjectaccessreviews resource (#20573, @deads2k)
  • AWS/GCE: Rework use of master name (#30047, @zmerlynn)
  • Add density (batch pods creation latency and resource) and resource performance tests to `test-e2e-node' built for Linux only (#30026, @coufon)
  • Clean up items from moving local cluster setup guides (#30035, @pwittrock)
  • federation: Adding secret API (#29138, @kshafiee)
  • Introducing ScheduledJobs as described in the proposal as part of batch/v2alpha1 version (experimental feature). (#25816, @soltysh)
  • Node disk pressure should induce image gc (#29880, @derekwaynecarr)
  • oidc authentication plugin: don't trim issuer URLs with trailing slashes (#29860, @ericchiang)
  • Allow leading * in ingress hostname (#29204, @aledbf)
  • Rewrite service controller to apply best controller pattern (#25189, @mfanjie)
  • Fix issue with kubectl annotate when --resource-version is provided. (#29319, @juanvallejo)
  • Reverted conversion of influx-db to Pet Set, it is now a Replication Controller. (#30080, @jszczepkowski)
  • rbac validation: rules can't combine non-resource URLs and regular resources (#29930, @ericchiang)
  • VSAN support for VSphere Volume Plugin (#29172, @abrarshivani)
  • Addresses vSphere Volume Attach limits (#29881, @dagnello)
  • allow restricting subresource access (#29988, @deads2k)
  • Add density (batch pods creation latency and resource) and resource performance tests to `test-e2e-node' (#29764, @coufon)
  • Allow Secret & ConfigMap keys to contain caps, dots, and underscores (#25458, @errm)
  • allow watching old resources with kubectl (#27392, @sjenning)
  • azure: kube-up respects AZURE_RESOURCE_GROUP (#28700, @colemickens)
  • Modified influxdb petset to provision persistent volume. (#28840, @jszczepkowski)
  • Allow service names up to 63 characters (RFC 1035) (#29523, @fraenkel)
  • Change eviction policies in NodeController: (#28897, @gmarek)
      • add a "partialDisruption" mode, when more than 33% of Nodes in the zone are not Ready
      • add "fullDisruption" mode, when all Nodes in the zone are not Ready
    • Eviction behavior depends on the mode in which NodeController is operating:
      • if the new state is "partialDisruption" or "fullDisruption" we call a user defined function that returns a new QPS to use (default 1/10 of the default rate, and the default rate respectively),
      • if the new state is "normal" we resume normal operation (go back to default limiter settings),
      • if all zones in the cluster are in "fullDisruption" state we stop all evictions.
  • Add a flag for kubectl exposeto set ClusterIP and allow headless services (#28239, @ApsOps)
  • Add support to quota pvc storage requests (#28636, @derekwaynecarr)

v1.3.5

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 46be88ce927124f7cef7e280720b42c63051086880b7ebdba298b561dbe19f82

Changelog since v1.3.4

Other notable changes

v1.3.4

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 818acc1a8ba61cff434d4c0c5aa3d342d06e6907b565cfd8651b8cfcf3f0a1e6

Changelog since v1.3.3

Other notable changes

v1.4.0-alpha.2

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz 787ce63a5149a1cb47d14c55450172e3a045d85349682d2e17ff492de9e415b9

Changelog since v1.4.0-alpha.1

Action Required

  • Federation API server kubeconfig secret consumed by federation-controller-manager has a new name. (#28938, @madhusudancs)
    • If you are upgrading your Cluster Federation components from v1.3.x, please run this command to migrate the federation-apiserver-secret to federation-apiserver-kubeconfig serect;
    • $ kubectl --namespace=federation get secret federation-apiserver-secret -o json | sed 's/federation-apiserver-secret/federation-apiserver-kubeconfig/g' | kubectl create -f -
    • You might also want to delete the old secret using this command:
    • $ kubectl delete secret --namespace=federation federation-apiserver-secret
  • Stop eating panics (#28800, @lavalamp)

Other notable changes

  • Add API for StorageClasses (#29694, @childsb)
  • Fix kubectl help command (#29737, @andreykurilin)
  • add shorthand cm for configmaps (#29652, @lojies)
  • Bump cadvisor dependencies to latest head. (#29492, @Random-Liu)
  • If a service of type node port declares multiple ports, quota on "services.nodeports" will charge for each port in the service. (#29457, @derekwaynecarr)
  • Add an Azure CloudProvider Implementation (#28821, @colemickens)
  • Add support for kubectl create quota command (#28351, @sttts)
  • Assume volume is detached if node doesn't exist (#29485, @saad-ali)
  • kube-up: increase download timeout for kubernetes.tar.gz (#29426, @justinsb)
  • Allow multiple APIs to register for the same API Group (#28414, @brendandburns)
  • Fix a problem with multiple APIs clobbering each other in registration. (#28431, @brendandburns)
  • Removing images with multiple tags (#29316, @ronnielai)
  • add enhanced volume and mount logging for block devices (#24797, @screeley44)
  • append an abac rule for $KUBE_USER. (#29164, @cjcullen)
  • add tokenreviews endpoint to implement webhook (#28788, @deads2k)
  • Fix "PVC Volume not detached if pod deleted via namespace deletion" issue (#29077, @saad-ali)
  • Allow mounts to run in parallel for non-attachable volumes (#28939, @saad-ali)
  • Fix working_set calculation in kubelet (#29153, @vishh)
  • Fix RBAC authorizer of ServiceAccount (#29071, @albatross0)
  • kubectl proxy changed to now allow urls to pods with "attach" or "exec" in the pod name (#28765, @nhlfr)
  • AWS: Added experimental option to skip zone check (#28417, @kevensen)
  • Ubuntu: Enable ssh compression when downloading binaries during cluster creation (#26746, @MHBauer)
  • Add extensions/replicaset to federation-apiserver (#24764, @jianhuiz)
  • federation: Adding namespaces API (#26298, @nikhiljindal)
  • Improve quota controller performance by eliminating unneeded list calls (#29134, @derekwaynecarr)
  • Make Daemonset use GeneralPredicates (#28803, @lukaszo)
  • Update docker engine-api to dea108d3aa (#29144, @ronnielai)
  • Fixing kube-up for CVM masters. (#29140, @maisem)
  • Fix logrotate config on GCI (#29139, @adityakali)
  • GCE bring-up: Differentiate NODE_TAGS from NODE_INSTANCE_PREFIX (#29141, @zmerlynn)
  • hyperkube: fix build for 3rd party registry (again) (#28489, @liyimeng)
  • Detect flakes in PR builder e2e runs (#27898, @lavalamp)
  • Remove examples moved to docs site (#23513, @erictune)
  • Do not query the metadata server to find out if running on GCE. Retry metadata server query for gcr if running on gce. (#28871, @vishh)
  • Change maxsize to size in logrotate. (#29128, @bprashanth)
  • Change setting "kubectl --record=false" to stop updating the change-cause when a previous change-cause is found. (#28234, @damemi)
  • Add "kubectl --overwrite" flag to automatically resolve conflicts between the modified and live configuration using values from the modified configuration. (#26136, @AdoHe)
  • Make discovery summarizer call servers in parallel (#26705, @nebril)
  • Don't recreate lb cloud resources on kcm restart (#29082, @bprashanth)
  • List all nodes and occupy cidr map before starting allocations (#29062, @bprashanth)
  • Fix GPU resource validation (#28743, @therc)
  • Make PD E2E Tests Wait for Detach to Prevent Kernel Errors (#29031, @saad-ali)
  • Scale kube-proxy conntrack limits by cores (new default behavior) (#28876, @thockin)
  • [Kubelet] Improving QOS in kubelet by introducing QoS level Cgroups - --cgroups-per-qos (#27853, @dubstack)
  • AWS: Add ap-south-1 to list of known AWS regions (#28428, @justinsb)
  • Add RELEASE_INFRA_PUSH related code to support pushes from kubernetes/release. (#28922, @david-mcmahon)
  • Fix watch cache filtering (#28966, @liggitt)
  • Deprecate deleting-pods-burst ControllerManager flag (#28882, @gmarek)
  • Add support for terminal resizing for exec, attach, and run. Note that for Docker, exec sessions (#25273, @ncdc)
    • inherit the environment from the primary process, so if the container was created with tty=false,
    • that means the exec session's TERM variable will default to "dumb". Users can override this by
    • setting TERM=xterm (or whatever is appropriate) to get the correct "smart" terminal behavior.
  • Implement alpha version of PreferAvoidPods (#20699, @jiangyaoguo)
  • Retry when apiserver fails to listen on insecure port (#28797, @aaronlevy)
  • Add SSH_OPTS to config ssh and scp port (#28872, @lojies)
  • kube-up: install new Docker pre-requisite (libltdl7) when not in image (#28745, @justinsb)
  • Separate rate limiters for Pod evictions for different zones in NodeController (#28843, @gmarek)
  • Add --quiet to hide the 'waiting for pods to be running' message in kubectl run (#28801, @janetkuo)
  • Controllers doesn't take any actions when being deleted. (#27438, @gmarek)
  • Add "deploy" abbrev for deployments to kubectl (#24087, @Frostman)
  • --no-header available now for custom-column (#26696, @gitfred)

v1.3.3

Documentation & Examples

Downloads

binary sha256 hash
kubernetes.tar.gz a92a74a0d3f7d02d01ac2c8dfb5ee2e97b0485819e77b2110eb7c6b7c782478c

Changelog since v1.3.2

Other notable changes

  • Removing images with multiple tags (#29316, @ronnielai)
  • kubectl: don't display an empty list when trying to get a single resource that isn't found (#28294, @ncdc)
  • Fix working_set calculation in kubelet (#29154, @vishh)
  • Don't delete affinity when endpoints are empty (#28655, @freehan)
  • GCE bring-up: Differentiate NODE_TAGS from NODE_INSTANCE_PREFIX (#29141, @zmerlynn)
  • Fix logrotate config on GCI (#29139, @adityakali)
  • Do not query the metadata server to find out if running on GCE. Retry metadata server query for gcr if running on gce. (#28871, @vishh)
  • Fix GPU resource validation (#28743, @therc)
  • Scale kube-proxy conntrack limits by cores (new default behavior) (#28876, @thockin)
  • Don't recreate lb cloud resources on kcm restart (#29082, @bprashanth)

Known Issues

There are a number of known issues that have been found and are being worked on. Please be aware of them as you test your workloads.

  • PVC Volume not detached if pod deleted via namespace deletion (29051)
  • Google Compute Engine PD Detach fails if node no longer exists (29358)
  • Mounting (only 'default-token') volume takes a long time when creating a batch of pods (parallelization issue) (28616)
  • Error while tearing down pod, "device or resource busy" on service account secret (28750)

v1.3.2

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz f46664d04dc2966c77d8727bba57f57b5f917572 1a5b0639941054585d0432dd5ce3abc7

Changelog since v1.3.1

Other notable changes

v1.3.1

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 5645b12beda22137204439de8260c62c9925f89b ae6e9902ec70c1322d9a0a29ef385190

Changelog since v1.3.0

Other notable changes

v1.2.6

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 50023455d00af52c41a7158b4bd117b2dfd4a100 cf0411bcb620eb13b08b93578efffc43

Changelog since v1.2.5

Other notable changes

v1.4.0-alpha.1

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 11a199208c5164a291c1767a1b9e64e45fdea747 334f349daf9268d8ac091d7fcc8e4626

Changelog since v1.3.0

Experimental Features

  • An alpha implementation of the TLS bootstrap API described in docs/proposals/kubelet-tls-bootstrap.md. (#25562, @gtank)

Action Required

  • [kubelet] Allow opting out of automatic cloud provider detection in kubelet. By default kubelet will auto-detect cloud providers (#28258, @vishh)
  • If you use one of the kube-dns replication controller manifest in cluster/saltbase/salt/kube-dns, i.e. cluster/saltbase/salt/kube-dns/{skydns-rc.yaml.base,skydns-rc.yaml.in}, either substitute one of __PILLAR__FEDERATIONS__DOMAIN__MAP__ or {{ pillar['federations_domain_map'] }} with the corresponding federation name to domain name value or remove them if you do not support cluster federation at this time. If you plan to substitute the parameter with its value, here is an example for {{ pillar['federations_domain_map'] } (#28132, @madhusudancs)
    • pillar['federations_domain_map'] = "- --federations=myfederation=federation.test"
    • where myfederation is the name of the federation and federation.test is the domain name registered for the federation.
  • Proportionally scale paused and rolling deployments (#20273, @kargakis)

Other notable changes

v1.3.0

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 88249c443d438666928379aa7fe865b389ed72ea 9270f001aef8c03ff5db63456ca9eecc

Highlights

  • Authorization:
    • Alpha RBAC authorization API group
  • Federation
    • federation api group is now beta
    • Services from all federated clusters are now registered in Cloud DNS (AWS and GCP).
  • Stateful Apps:
    • alpha PetSets manage stateful apps
    • alpha Init containers provide one-time setup for stateful containers
  • Updating:
    • Retry Pod/RC updates in kubectl rolling-update.
    • Stop 'kubectl drain' deleting pods with local storage.
    • Add kubectl rollout status
  • Security/Auth
    • L7 LB controller and disk attach controllers run on master, so nodes do not need those privileges.
    • Setting TLS1.2 minimum
    • kubectl create secret tls command
    • Webhook Token Authenticator
    • beta PodSecurityPolicy objects limits use of security-sensitive features by pods.
  • Kubectl
    • Display line number on JSON errors
    • Add flag -t as shorthand for --tty
  • Resources
    • Improved node stability by optionally evicting pods upon memory pressure - Design Doc
    • alpha: NVIDIA GPU support (#24836, @therc)
    • Adding loadBalancer services and nodeports services to quota system

Known Issues and Important Steps before Upgrading

The following versions of Docker Engine are supported - v1.10, v1.11 Although v1.9 is still compatible, we recommend upgrading to one of the supported versions. All prior versions of docker will not be supported.

ThirdPartyResource

If you use ThirdPartyResource objects, they have moved from being namespaced-scoped to be cluster-scoped. Before upgrading to 1.3.0, export and delete any existing ThirdPartyResource objects using a 1.2.x client:

kubectl get thirdpartyresource --all-namespaces -o yaml > tprs.yaml kubectl delete -f tprs.yaml

After upgrading to 1.3.0, re-register the third party resource objects at the root scope (using a 1.3 server and client):

kubectl create -f tprs.yaml

kubectl

Kubectl flag --container-port flag is deprecated: it will be removed in the future, please use --target-port instead.

kubernetes Core Known Issues

  • Kube Proxy crashes infrequently due to a docker bug (#24000)
    • This issue can be resolved by restarting docker daemon
  • CORS works only in insecure mode (#24086)
  • Persistent volume claims gets added incorrectly after being deleted under stress. Happens very infrequently. (#26082)

Docker runtime Known Issues

  • Kernel crash with Aufs storage driver on Debian Jessie (#27885)

  • File descriptors are leaked in docker v1.11 (#275)

  • Additional memory overhead per container in docker v1.11 (#21737)

  • List of upstream fixes for docker v1.10 identified by RedHat

Rkt runtime Known Issues

  • A detailed list of known issues can be found here

More Instructions coming soon

Provider-specific Notes

  • AWS
    • Support for ap-northeast-2 region (Seoul)
    • Allow cross-region image pulling with ECR
    • More reliable kube-up/kube-down
    • Enable ICMP Type 3 Code 4 for ELBs
    • ARP caching fix
    • Use /dev/xvdXX names
    • ELB:
      • ELB proxy protocol support
      • mixed plaintext/encrypted ports support in ELBs
      • SSL support for ELB listeners
    • Allow VPC CIDR to be specified (experimental)
    • Fix problems with >2 security groups
  • GCP:
    • Enable using gcr.io as a Docker registry mirror.
    • Make bigger master root disks in GCE for large clusters.
    • Change default clusterCIDRs from /16 to /14 allowing 1000 Node clusters by default.
    • Allow Debian Jessie on GCE.
    • Node problem detector addon pod detects and reports kernel deadlocks.
  • OpenStack
    • Provider added.
  • VSphere:
    • Provider updated.

Previous Releases Included in v1.3.0

v1.3.0-beta.3

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 9d18964a294f356bfdc841957dcad8ff35ed909c ee5fcdf86645135ed132663967876dd6

Changelog since v1.3.0-beta.2

Action Required

  • [kubelet] Allow opting out of automatic cloud provider detection in kubelet. By default kubelet will auto-detect cloud providers (#28258, @vishh)
  • If you use one of the kube-dns replication controller manifest in cluster/saltbase/salt/kube-dns, i.e. cluster/saltbase/salt/kube-dns/{skydns-rc.yaml.base,skydns-rc.yaml.in}, either substitute one of __PILLAR__FEDERATIONS__DOMAIN__MAP__ or {{ pillar['federations_domain_map'] }} with the corresponding federation name to domain name value or remove them if you do not support cluster federation at this time. If you plan to substitute the parameter with its value, here is an example for {{ pillar['federations_domain_map'] } (#28132, @madhusudancs)
    • pillar['federations_domain_map'] = "- --federations=myfederation=federation.test"
    • where myfederation is the name of the federation and federation.test is the domain name registered for the federation.
  • federation: Upgrading the groupversion to v1beta1 (#28186, @nikhiljindal)
  • Set Dashboard UI version to v1.1.0 (#27869, @bryk)

Other notable changes

  • Build: Add KUBE_GCS_RELEASE_BUCKET_MIRROR option to push-ci-build.sh (#28172, @zmerlynn)
  • Image GC logic should compensate for reserved blocks (#27996, @ronnielai)
  • Bump minimum API version for docker to 1.21 (#27208, @yujuhong)
  • Adding lock files for kubeconfig updating (#28034, @krousey)
  • federation service controller: fixing the logic to update DNS records (#27999, @quinton-hoole)
  • federation: Updating KubeDNS to try finding a local service first for federation query (#27708, @nikhiljindal)
  • Support journal logs in fluentd-gcp on GCI (#27981, @a-robinson)
  • Copy and display source location prominently on Kubernetes instances (#27985, @maisem)
  • Federation e2e support for AWS (#27791, @colhom)
  • Copy and display source location prominently on Kubernetes instances (#27840, @zmerlynn)
  • AWS/GCE: Spread PetSet volume creation across zones, create GCE volumes in non-master zones (#27553, @justinsb)
  • GCE provider: Create TargetPool with 200 instances, then update with rest (#27829, @zmerlynn)
  • Add sources to server tarballs. (#27830, @david-mcmahon)
  • Retry Pod/RC updates in kubectl rolling-update (#27509, @janetkuo)
  • AWS kube-up: Authorize route53 in the IAM policy (#27794, @justinsb)
  • Allow conformance tests to run on non-GCE providers (#26932, @aaronlevy)
  • AWS kube-up: move to Docker 1.11.2 (#27676, @justinsb)
  • Fixed an issue that Deployment may be scaled down further than allowed by maxUnavailable when minReadySeconds is set. (#27728, @janetkuo)

v1.2.5

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz ddf12d7f37dfef25308798d71ad547761d0785ac 69d770df8fa4eceb57167e34df3962ca

Changes since v1.2.4

Other notable changes

  • Retry Pod/RC updates in kubectl rolling-update (#27509, @janetkuo)
  • GCE provider: Create TargetPool with 200 instances, then update with rest (#27865, @zmerlynn)
  • GCE provider: Limit Filter calls to regexps rather than large blobs (#27741, @zmerlynn)
  • Fix strategic merge diff list diff bug (#26418, @AdoHe)
  • AWS: Fix long-standing bug in stringSetToPointers (#26331, @therc)
  • AWS kube-up: Increase timeout waiting for docker start (#25405, @justinsb)
  • Fix hyperkube flag parsing (#25512, @colhom)
  • kubectl rolling-update support for same image (#24645, @jlowdermilk)
  • Return "410 Gone" errors via watch stream when using watch cache (#25369, @liggitt)

v1.3.0-beta.2

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 9c95762970b943d6c6547f0841c1e5471148b0e3 dc9e8560f24459b2313317b15910bee7

Changes since v1.3.0-beta.1

Experimental Features

  • Init containers enable pod authors to perform tasks before their normal containers start. Each init container is started in order, and failing containers will prevent the application from starting. (#23666, @smarterclayton)

Other notable changes

  • GCE provider: Limit Filter calls to regexps rather than large blobs (#27741, @zmerlynn)
  • Show LASTSEEN, the sorting key, as the first column in kubectl get event output (#27549, @therc)
  • GCI: fix kubectl permission issue #27643 (#27740, @andyzheng0831)
  • Add federation api and cm servers to hyperkube (#27586, @colhom)
  • federation: Creating kubeconfig files to be used for creating secrets for clusters on aws and gke (#27332, @nikhiljindal)
  • AWS: Enable ICMP Type 3 Code 4 for ELBs (#27677, @justinsb)
  • Bumped Heapster to v1.1.0. (#27542, @piosz)
  • Deleting federation-push.sh (#27400, @nikhiljindal)
  • Validate-cluster finishes shortly after at most ALLOWED_NOTREADY_NODE… (#26778, @gmarek)
  • AWS kube-down: Issue warning if VPC not found (#27518, @justinsb)
  • gce/kube-down: Parallelize IGM deletion, batch more (#27302, @zmerlynn)
  • Enable dynamic allocation of heapster/eventer cpu request/limit (#27185, @gmarek)
  • 'kubectl describe pv' now shows events (#27431, @jsafrane)
  • AWS kube-up: set net.ipv4.neigh.default.gc_thresh1=0 to avoid ARP over-caching (#27682, @justinsb)
  • AWS volumes: Use /dev/xvdXX names with EC2 (#27628, @justinsb)
  • Add a test config variable to specify desired Docker version to run on GCI. (#26813, @wonderfly)
  • Check for thin_is binary in path for devicemapper when using ThinPoolWatcher and fix uint64 overflow issue for CPU stats (#27591, @dchen1107)
  • Change default value of deleting-pods-burst to 1 (#27606, @gmarek)
  • MESOS: fix race condition in contrib/mesos/pkg/queue/delay (#24916, @jdef)
  • including federation binaries in the list of images we push during release (#27396, @nikhiljindal)
  • fix updatePod() of RS and RC controllers (#27415, @caesarxuchao)
  • Change default value of deleting-pods-burst to 1 (#27422, @gmarek)
  • A new volume manager was introduced in kubelet that synchronizes volume mount/unmount (and attach/detach, if attach/detach controller is not enabled). (#26801, @saad-ali)
    • This eliminates the race conditions between the pod creation loop and the orphaned volumes loops. It also removes the unmount/detach from the syncPod() path so volume clean up never blocks the syncPod loop.

v1.3.0-beta.1

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 2b54995ee8f52d78dc31c3d7291e8dfa5c809fe7 f1022a84c3441cae4ebe1d295470be8f

Changes since v1.3.0-alpha.5

Action Required

  • Fixing logic to generate ExternalHost in genericapiserver (#26796, @nikhiljindal)
  • federation: Updating federation-controller-manager to use secret to get federation-apiserver's kubeconfig (#26819, @nikhiljindal)

Other notable changes

  • federation: fix dns provider initialization issues (#27252, @mfanjie)
  • Updating federation up scripts to work in non e2e setup (#27260, @nikhiljindal)
  • version bump for gci to milestone 53 (#27210, @adityakali)
  • kubectl apply: retry applying a patch if a version conflict error is encountered (#26557, @AdoHe)
  • Revert "Wait for arc.getArchive() to complete before running tests" (#27130, @pwittrock)
  • ResourceQuota BestEffort scope aligned with Pod level QoS (#26969, @derekwaynecarr)
  • The AWS cloudprovider will cache results from DescribeInstances() if the set of nodes hasn't changed (#26900, @therc)
  • GCE provider: Log full contents of long operations (#26962, @zmerlynn)
  • Fix system container detection in kubelet on systemd. (#26586, @derekwaynecarr)
    • This fixed environments where CPU and Memory Accounting were not enabled on the unit that launched the kubelet or docker from reporting the root cgroup when monitoring usage stats for those components.
  • New default horizontalpodautoscaler/v1 generator for kubectl autoscale. (#26775, @piosz)
    • Use autoscaling/v1 in kubectl by default.
  • federation: Adding dnsprovider flags to federation-controller-manager (#27158, @nikhiljindal)
  • federation service controller: fixing a bug so that existing services are created in newly registered clusters (#27028, @mfanjie)
  • Rename environment variables (KUBE_)ENABLE_NODE_AUTOSCALER to (KUBE_)ENABLE_CLUSTER_AUTOSCALER. (#27117, @mwielgus)
  • support for mounting local-ssds on GCI (#27143, @adityakali)
  • AWS: support mixed plaintext/encrypted ports in ELBs via service.beta.kubernetes.io/aws-load-balancer-ssl-ports annotation (#26976, @therc)
  • Updating e2e docs with instructions on running federation tests (#27072, @colhom)
  • LBaaS v2 Support for Openstack Cloud Provider Plugin (#25987, @dagnello)
  • GCI: add support for network plugin (#27027, @andyzheng0831)
  • Bump cAdvisor to v0.23.3 (#27065, @timstclair)
  • Stop 'kubectl drain' deleting pods with local storage. (#26667, @mml)
  • Networking e2es: Wait for all nodes to be schedulable in kubeproxy and networking tests (#27008, @zmerlynn)
  • change clientset of service controller to versioned (#26694, @mfanjie)
  • Use gcr.io as a Docker registry mirror when setting up a cluster in GCE. (#25841, @ojarjur)
  • correction on rbd volume object and defaults (#25490, @rootfs)
  • Bump GCE debian image to container-v1-3-v20160604 (#26851, @zmerlynn)
  • Option to enable http2 on client connections. (#25280, @timothysc)
  • kubectl get ingress output remove rules (#26684, @AdoHe)
  • AWS kube-up: Remove SecurityContextDeny admission controller (to mirror GCE) (#25381, @zquestz)
  • Fix third party (#25894, @brendandburns)
  • AWS Route53 dnsprovider (#26049, @quinton-hoole)
  • GCI/Trusty: support the Docker registry mirror (#26745, @andyzheng0831)
  • Kubernetes v1.3 introduces a new Attach/Detach Controller. This controller manages attaching and detaching of volumes on-behalf of nodes. (#26351, @saad-ali)
    • This ensures that attachment and detachment of volumes is independent of any single nodes’ availability. Meaning, if a node or kubelet becomes unavailable for any reason, the volumes attached to that node will be detached so they are free to be attached to other nodes.
    • Specifically the new controller watches the API server for scheduled pods. It processes each pod and ensures that any volumes that implement the volume Attacher interface are attached to the node their pod is scheduled to.
    • When a pod is deleted, the controller waits for the volume to be safely unmounted by kubelet. It does this by waiting for the volume to no longer be present in the nodes Node.Status.VolumesInUse list. If the volume is not safely unmounted by kubelet within a pre-configured duration (3 minutes in Kubernetes v1.3), the controller unilaterally detaches the volume (this prevents volumes from getting stranded on nodes that become unavailable).
    • In order to remain backwards compatible, the new controller only manages attach/detach of volumes that are scheduled to nodes that opt-in to controller management. Nodes running v1.3 or higher of Kubernetes opt-in to controller management by default by setting the "volumes.kubernetes.io/controller-managed-attach-detach" annotation on the Node object on startup. This behavior is gated by a new kubelet flag, "enable-controller-attach-detach,” (default true).
    • In order to safely upgrade an existing Kubernetes cluster without interruption of volume attach/detach logic:
      • First upgrade the master to Kubernetes v1.3.
        • This will start the new attach/detach controller.
        • The new controller will initially ignore volumes for all nodes since they lack the "volumes.kubernetes.io/controller-managed-attach-detach" annotation.
      • Then upgrade nodes to Kubernetes v1.3.
        • As nodes are upgraded, they will automatically, by default, opt-in to attach/detach controller management, which will cause the controller to start managing attaches/detaches for volumes that get scheduled to those nodes.
  • Added DNS Reverse Record logic for service IPs (#26226, @ArtfulCoder)
  • read gluster log to surface glusterfs plugin errors properly in describe events (#24808, @screeley44)

v1.3.0-alpha.5

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 724bf5a4437ca9dc75d9297382f47a179e8dc5a6 2a8b4a5297df3007fce69f1e344fd87e

Changes since v1.3.0-alpha.4

Action Required

Other notable changes

  • Fix a bug with pluralization of third party resources (#25374, @brendandburns)
  • Run l7 controller on master (#26048, @bprashanth)
  • AWS: ELB proxy protocol support via annotation service.beta.kubernetes.io/aws-load-balancer-proxy-protocol (#24569, @williamsandrew)
  • kubectl run --restart=Never creates pods (#25253, @soltysh)
  • Add LabelSelector to PersistentVolumeClaimSpec (#25917, @pmorie)
  • Removed metrics api group (#26073, @piosz)
  • Fixed check in kubectl autoscale: cpu consumption can be higher than 100%. (#26162, @jszczepkowski)
  • Add support for 3rd party objects to kubectl label (#24882, @brendandburns)
  • Move shell completion generation into 'kubectl completion' command (#23801, @sttts)
  • Fix strategic merge diff list diff bug (#26418, @AdoHe)
  • Setting TLS1.2 minimum because TLS1.0 and TLS1.1 are vulnerable (#26169, @victorgp)
  • Kubelet: Periodically reporting image pulling progress in log (#26145, @Random-Liu)
  • Federation service controller is one key component of federation controller manager, it watches federation service, creates/updates services to all registered clusters, and update DNS records to global DNS server. (#26034, @mfanjie)
  • Stabilize map order in kubectl describe (#26046, @timoreimann)
  • Google Cloud DNS dnsprovider - replacement for #25389 (#26020, @quinton-hoole)
  • Fix system container detection in kubelet on systemd. (#25982, @derekwaynecarr)
    • This fixed environments where CPU and Memory Accounting were not enabled on the unit
    • that launched the kubelet or docker from reporting the root cgroup when
    • monitoring usage stats for those components.
  • Added pods-per-core to kubelet. #25762 (#25813, @rrati)
  • promote sourceRange into service spec (#25826, @freehan)
  • kube-controller-manager: Add configure-cloud-routes option (#25614, @justinsb)
  • kubelet: reading cloudinfo from cadvisor (#21373, @enoodle)
  • Disable cAdvisor event storage by default (#24771, @timstclair)
  • Remove docker-multinode (#26031, @luxas)
  • nodecontroller: Fix log message on successful update (#26207, @zmerlynn)
  • remove deprecated generated typed clients (#26336, @caesarxuchao)
  • Kubenet host-port support through iptables (#25604, @freehan)
  • Add metrics support for a GCE PD, EC2 EBS & Azure File volumes (#25852, @vishh)
  • Bump cAdvisor to v0.23.2 - See changelog for details (#25914, @timstclair)
  • Alpha version of "Role Based Access Control" API. (#25634, @ericchiang)
  • Add Seccomp API (#25324, @jfrazelle)
  • AWS: Fix long-standing bug in stringSetToPointers (#26331, @therc)
  • Add dnsmasq as a DNS cache in kube-dns pod (#26114, @ArtfulCoder)
  • routecontroller: Add wait.NonSlidingUntil, use it (#26301, @zmerlynn)
  • Attempt 2: Bump GCE containerVM to container-v1-3-v20160517 (Docker 1.11.1) again. (#26001, @dchen1107)
  • Downward API implementation for resources limits and requests (#24179, @aveshagarwal)
  • GCE clusters start using GCI as the default OS image for masters (#26197, @wonderfly)
  • Add a 'kubectl clusterinfo dump' option (#20672, @brendandburns)
  • Fixing heapster memory requirements. (#26109, @Q-Lee)
  • Handle federated service name lookups in kube-dns. (#25727, @madhusudancs)
  • Support sort-by timestamp in kubectl get (#25600, @janetkuo)
  • vSphere Volume Plugin Implementation (#24947, @abithap)
  • ResourceQuota controller uses rate limiter to prevent hot-loops in error situations (#25748, @derekwaynecarr)
  • Fix hyperkube flag parsing (#25512, @colhom)
  • Add a kubectl create secret tls command (#24719, @bprashanth)
  • Introduce a new add-on pod NodeProblemDetector. (#25986, @Random-Liu)
    • NodeProblemDetector is a DaemonSet running on each node, monitoring node health and reporting
    • node problems as NodeCondition and Event. Currently it already supports kernel log monitoring, and
    • will support more problem detection in the future. It is enabled by default on gce now.
  • Handle cAdvisor partial failures (#25933, @timstclair)
  • Use SkyDNS as a library for a more integrated kube DNS (#23930, @ArtfulCoder)
  • Introduce node memory pressure condition to scheduler (#25531, @ingvagabund)
  • Fix detection of docker cgroup on RHEL (#25907, @ncdc)
  • Kubelet evicts pods when available memory falls below configured eviction thresholds (#25772, @derekwaynecarr)
  • Use protobufs by default to communicate with apiserver (still store JSONs in etcd) (#25738, @wojtek-t)
  • Implement NetworkPolicy v1beta1 API object / client support. (#25638, @caseydavenport)
  • Only expose top N images in NodeStatus (#25328, @resouer)
  • Extend secrets volumes with path control (#25285, @ingvagabund)
  • With this PR, kubectl and other RestClient's using the AuthProvider framework can make OIDC authenticated requests, and, if there is a refresh token present, the tokens will be refreshed as needed. (#25270, @bobbyrullo)
  • Make addon-manager cross-platform and use it with hyperkube (#25631, @luxas)
  • kubelet: Optionally, have kubelet exit if lock file contention is observed, using --exit-on-lock-contention flag (#25596, @derekparker)
  • Bump up glbc version to 0.6.2 (#25446, @bprashanth)
  • Add "kubectl set image" for easier updating container images (for pods or resources with pod templates). (#25509, @janetkuo)
  • NodeController doesn't evict Pods if no Nodes are Ready (#25571, @gmarek)
  • Incompatible change of kube-up.sh: (#25734, @jszczepkowski)
    • when turning on cluster autoscaler by setting KUBE_ENABLE_NODE_AUTOSCALER=true,
    • KUBE_AUTOSCALER_MIN_NODES and KUBE_AUTOSCALER_MAX_NODES need to be set.
  • systemd node spec proposal (#17688, @derekwaynecarr)
  • Bump GCE ContainerVM to container-v1-3-v20160517 (Docker 1.11.1) (#25843, @zmerlynn)
  • AWS: Move enforcement of attached AWS device limit from kubelet to scheduler (#23254, @jsafrane)
  • Refactor persistent volume controller (#24331, @jsafrane)
  • Add support for running GCI on the GCE cloud provider (#25425, @andyzheng0831)
  • Implement taints and tolerations (#24134, @kevin-wangzefeng)
  • Add init containers to pods (#23567, @smarterclayton)

v1.3.0-alpha.4

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 758e97e7e50153840379ecd9f8fda1869543539f 4e18ae6a428c99fcc30e2137d7c41854

Changes since v1.3.0-alpha.3

Action Required

Other notable changes

  • Fix hyperkube's layer caching, and remove --make-symlinks at build time (#25693, @luxas)
  • AWS: More support for ap-northeast-2 region (#24464, @matthewrudy)
  • Make bigger master root disks in GCE for large clusters (#25670, @gmarek)
  • AWS kube-down: don't fail if ELB not in VPC - #23784 (#23785, @ajohnstone)
  • Build hyperkube in hack/local-up-cluster instead of separate binaries (#25627, @luxas)
  • enable recursive processing in kubectl rollout (#25110, @metral)
  • Support struct,array,slice types when sorting kubectl output (#25022, @zhouhaibing089)
  • federated api servers: Adding a discovery summarizer server (#20358, @nikhiljindal)
  • AWS: Allow cross-region image pulling with ECR (#24369, @therc)
  • Automatically add node labels beta.kubernetes.io/{os,arch} (#23684, @luxas)
  • kubectl "rm" will suggest using "delete"; "ps" and "list" will suggest "get". (#25181, @janetkuo)
  • Add IPv6 address support for pods - does NOT include services (#23090, @tgraf)
  • Use local disk for ConfigMap volume instead of tmpfs (#25306, @pmorie)
  • Alpha support for scheduling pods on machines with NVIDIA GPUs whose kubelets use the --experimental-nvidia-gpus flag, using the alpha.kubernetes.io/nvidia-gpu resource (#24836, @therc)
  • AWS: SSL support for ELB listeners through annotations (#23495, @therc)
  • Implement kubectl rollout status that can be used to watch a deployment's rollout status (#19946, @janetkuo)
  • Webhook Token Authenticator (#24902, @cjcullen)
  • Update PodSecurityPolicy types and add admission controller that could enforce them (#24600, @pweil-)
  • Introducing ScheduledJobs as described in the proposal as part of batch/v2alpha1 version (experimental feature). (#24970, @soltysh)
  • kubectl now supports validation of nested objects with different ApiGroups (e.g. objects in a List) (#25172, @pwittrock)
  • Change default clusterCIDRs from /16 to /14 in GCE configs allowing 1000 Node clusters by default. (#25350, @gmarek)
  • Add 'kubectl set' (#25444, @janetkuo)
  • vSphere Cloud Provider Implementation (#24703, @dagnello)
  • Added JobTemplate, a preliminary step for ScheduledJob and Workflow (#21675, @soltysh)
  • Openstack provider (#21737, @zreigz)
  • AWS kube-up: Allow VPC CIDR to be specified (experimental) (#23362, @miguelfrde)
  • Return "410 Gone" errors via watch stream when using watch cache (#25369, @liggitt)
  • GKE provider: Add cluster-ipv4-cidr and arbitrary flags (#25437, @zmerlynn)
  • AWS kube-up: Increase timeout waiting for docker start (#25405, @justinsb)
  • Sort resources in quota errors to avoid duplicate events (#25161, @derekwaynecarr)
  • Display line number on JSON errors (#25038, @mfojtik)
  • If the cluster node count exceeds the GCE TargetPool maximum (currently 1000), (#25178, @zmerlynn)
    • randomly select which nodes are members of Kubernetes External Load Balancers.
  • Clarify supported version skew between masters, nodes, and clients (#25087, @ihmccreery)
  • Move godeps to vendor/ (#24242, @thockin)
  • Introduce events flag for describers (#24554, @ingvagabund)
  • run kube-addon-manager in a static pod (#23600, @mikedanese)
  • Reimplement 'pause' in C - smaller footprint all around (#23009, @uluyol)
  • Add subPath to mount a child dir or file of a volumeMount (#22575, @MikaelCluseau)
  • Handle image digests in node status and image GC (#25088, @ncdc)
  • PLEG: reinspect pods that failed prior inspections (#25077, @ncdc)
  • Fix kubectl create secret/configmap to allow = values (#24989, @derekwaynecarr)
  • Upgrade installed packages when building hyperkube to improve the security profile (#25114, @aaronlevy)
  • GCI/Trusty: Support ABAC authorization (#24950, @andyzheng0831)
  • fix cinder volume dir umount issue #24717 (#24718, @chengyli)
  • Inter pod topological affinity and anti-affinity implementation (#22985, @kevin-wangzefeng)
  • start etcd compactor in background (#25010, @hongchaodeng)
  • GCI: Add two GCI specific metadata pairs (#25105, @andyzheng0831)
  • Ensure status is not changed during an update of PV, PVC, HPA objects (#24924, @mqliang)
  • GCE: Prefer preconfigured node tags for firewalls, if available (#25148, @a-robinson)
  • kubectl rolling-update support for same image (#24645, @jlowdermilk)
  • Add an entry to the salt config to allow Debian jessie on GCE. (#25123, @jlewi)
    • As with the existing Wheezy image on GCE, docker is expected
    • to already be installed in the image.
  • Mark kube-push.sh as broken (#25095, @ihmccreery)
  • AWS: Add support for ap-northeast-2 region (Seoul) (#24457, @leokhoa)
  • GCI: Update the command to get the image (#24987, @andyzheng0831)
  • Port-forward: use out and error streams instead of glog (#17030, @csrwng)
  • Promote Pod Hostname & Subdomain to fields (were annotations) (#24362, @ArtfulCoder)
  • Validate deletion timestamp doesn't change on update (#24839, @liggitt)
  • Add flag -t as shorthand for --tty (#24365, @janetkuo)
  • Add support for running clusters on GCI (#24893, @andyzheng0831)
  • Switch to ABAC authorization from AllowAll (#24210, @cjcullen)
  • Fix DeletingLoadBalancer event generation. (#24833, @a-robinson)

v1.2.4

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz f3aea83f8f0e16b2b41998a2edc09eb42fd8d945 ab0aca3a20e8eba43c8ff9d672793618

Changes since v1.2.3

Other notable changes

v1.3.0-alpha.3

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 01e0dc68653173614dc99f44875173478f837b38 ae22c35f3a963743d21daa17683e0288

Changes since v1.3.0-alpha.2

Action Required

  • Updating go-restful to generate "type":"object" instead of "type":"any" in swagger-spec (breaks kubectl 1.1) (#22897, @nikhiljindal)
  • Make watch cache treat resourceVersion consistent with uncached watch (#24008, @liggitt)

Other notable changes

v1.2.3

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz b2ce4e0c72562d09ba06e3c0913f0bd78da0285e 69e75650de30d5a52d144799e94a168d

Changes since v1.2.2

Action Required

  • Make watch cache treat resourceVersion consistent with uncached watch (#24008, @liggitt)

Other notable changes

v1.3.0-alpha.2

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 305c8c2af7e99d463dbbe4208ecfe2b50585e796 aadb8d729d855e69212008f8fda628c0

Changes since v1.3.0-alpha.1

Other notable changes

v1.2.2

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 8dede5833a1986434adea80749624f81a0db7bb4 72a5389f22827fb5133fdc3b7bfb9b3a

Changes since v1.2.1

Other notable changes

v1.2.1

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 1639807c5788e1c6b1ab51fd30b723fb5debd865 235a1da47972c96a560d718d3256ca4f

Changes since v1.2.0

Other notable changes

v1.3.0-alpha.1

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz e0041b08e220a4704ea2ad90a6ec7c8f2120c2d3 7bb2df32aea94678f72a8d1f43a12098

Changes since v1.2.0

Action Required

  • Disabling swagger ui by default on apiserver. Adding a flag that can enable it (#23025, @nikhiljindal)
  • restore ability to run against secured etcd (#21535, @AdoHe)

Other notable changes

v1.2.0

Documentation & Examples

Downloads

binary sha1 hash md5 hash
kubernetes.tar.gz 52dd998e1191f464f581a9b87017d70ce0b058d9 c0ce9e6150e9d7a19455db82f3318b4c

Changes since v1.1.1

Major Themes

  • Significant scale improvements. Increased cluster scale by 400% to 1000 nodes with 30,000 pods per cluster. Kubelet supports 100 pods per node with 4x reduced system overhead.
  • Simplified application deployment and management.
    • Dynamic Configuration (ConfigMap API in the core API group) enables application configuration to be stored as a Kubernetes API object and pulled dynamically on container startup, as an alternative to baking in command-line flags when a container is built.
    • Turnkey Deployments (Deployment API (Beta) in the Extensions API group) automate deployment and rolling updates of applications, specified declaratively. It handles versioning, multiple simultaneous rollouts, aggregating status across all pods, maintaining application availability, and rollback.
  • Automated cluster management:
    • Kubernetes clusters can now span zones within a cloud provider. Pods from a service will be automatically spread across zones, enabling applications to tolerate zone failure.
    • Simplified way to run a container on every node (DaemonSet API (Beta) in the Extensions API group): Kubernetes can schedule a service (such as a logging agent) that runs one, and only one, pod per node.
    • TLS and L7 support (Ingress API (Beta) in the Extensions API group): Kubernetes is now easier to integrate into custom networking environments by supporting TLS for secure communication and L7 http-based traffic routing.
    • Graceful Node Shutdown (aka drain) - The new “kubectl drain” command gracefully evicts pods from nodes in preparation for disruptive operations like kernel upgrades or maintenance.
    • Custom Metrics for Autoscaling (HorizontalPodAutoscaler API in the Autoscaling API group): The Horizontal Pod Autoscaling feature now supports custom metrics (Alpha), allowing you to specify application-level metrics and thresholds to trigger scaling up and down the number of pods in your application.
  • New GUI (dashboard) allows you to get started quickly and enables the same functionality found in the CLI as a more approachable and discoverable way of interacting with the system. Note: the GUI is enabled by default in 1.2 clusters.
Dashboard UI screenshot showing cards that represent applications that run inside a cluster

Other notable improvements

  • Job was Beta in 1.1 and is GA in 1.2 .
    • apiVersion: batch/v1 is now available. You now do not need to specify the .spec.selector field — a unique selector is automatically generated for you.
    • The previous version, apiVersion: extensions/v1beta1, is still supported. Even if you roll back to 1.1, the objects created using the new apiVersion will still be accessible, using the old version. You can continue to use your existing JSON and YAML files until you are ready to switch to batch/v1. We may remove support for Jobs with apiVersion: extensions/v1beta1 in 1.3 or 1.4.
  • HorizontalPodAutoscaler was Beta in 1.1 and is GA in 1.2 .
    • apiVersion: autoscaling/v1 is now available. Changes in this version are:
      • Field CPUUtilization which was a nested structure CPUTargetUtilization in HorizontalPodAutoscalerSpec was replaced by TargetCPUUtilizationPercentage which is an integer.
      • ScaleRef of type SubresourceReference in HorizontalPodAutoscalerSpec which referred to scale subresource of the resource being scaled was replaced by ScaleTargetRef which points just to the resource being scaled.
      • In extensions/v1beta1 if CPUUtilization in HorizontalPodAutoscalerSpec was not specified it was set to 80 by default while in autoscaling/v1 HPA object without TargetCPUUtilizationPercentage specified is a valid object. Pod autoscaler controller will apply a default scaling policy in this case which is equivalent to the previous one but may change in the future.
    • The previous version, apiVersion: extensions/v1beta1, is still supported. Even if you roll back to 1.1, the objects created using the new apiVersions will still be accessible, using the old version. You can continue to use your existing JSON and YAML files until you are ready to switch to autoscaling/v1. We may remove support for HorizontalPodAutoscalers with apiVersion: extensions/v1beta1 in 1.3 or 1.4.
  • Kube-Proxy now defaults to an iptables-based proxy. If the --proxy-mode flag is specified while starting kube-proxy (‘userspace’ or ‘iptables’), the flag value will be respected. If the flag value is not specified, the kube-proxy respects the Node object annotation: ‘net.beta.kubernetes.io/proxy-mode’. If the annotation is not specified, then ‘iptables’ mode is the default. If kube-proxy is unable to start in iptables mode because system requirements are not met (kernel or iptables versions are insufficient), the kube-proxy will fall-back to userspace mode. Kube-proxy is much more performant and less resource-intensive in ‘iptables’ mode.
  • Node stability can be improved by reserving resources for the base operating system using --system-reserved and --kube-reserved Kubelet flags
  • Liveness and readiness probes now support more configuration parameters: periodSeconds, successThreshold, failureThreshold
  • The new ReplicaSet API (Beta) in the Extensions API group is similar to ReplicationController, but its selector is more general (supports set-based selector; whereas ReplicationController only supports equality-based selector).
  • Scale subresource support is now expanded to ReplicaSets along with ReplicationControllers and Deployments. Scale now supports two different types of selectors to accommodate both equality-based selectors supported by ReplicationControllers and set-based selectors supported by Deployments and ReplicaSets.
  • “kubectl run” now produces Deployments (instead of ReplicationControllers) and Jobs (instead of Pods) by default.
  • Pods can now consume Secret data in environment variables and inject those environment variables into a container’s command-line args.
  • Stable version of Heapster which scales up to 1000 nodes: more metrics, reduced latency, reduced cpu/memory consumption (~4mb per monitored node).
  • Pods now have a security context which allows users to specify:
    • attributes which apply to the whole pod:
      • User ID
      • Whether all containers should be non-root
      • Supplemental Groups
      • FSGroup - a special supplemental group
      • SELinux options
    • If a pod defines an FSGroup, that Pod’s system (emptyDir, secret, configMap, etc) volumes and block-device volumes will be owned by the FSGroup, and each container in the pod will run with the FSGroup as a supplemental group
  • Volumes that support SELinux labelling are now automatically relabeled with the Pod’s SELinux context, if specified
  • A stable client library release_1_2 is added. The library is here, and detailed doc is here. We will keep the interface of this go client stable.
  • New Azure File Service Volume Plugin enables mounting Microsoft Azure File Volumes (SMB 2.1 and 3.0) into a Pod. See example for details.
  • Logs usage and root filesystem usage of a container, volumes usage of a pod and node disk usage are exposed through Kubelet new metrics API.

Experimental Features

  • Dynamic Provisioning of PersistentVolumes: Kubernetes previously required all volumes to be manually provisioned by a cluster administrator before use. With this feature, volume plugins that support it (GCE PD, AWS EBS, and Cinder) can automatically provision a PersistentVolume to bind to an unfulfilled PersistentVolumeClaim.
  • Run multiple schedulers in parallel, e.g. one or more custom schedulers alongside the default Kubernetes scheduler, using pod annotations to select among the schedulers for each pod. Documentation is here, design doc is here.
  • More expressive node affinity syntax, and support for “soft” node affinity. Node selectors (to constrain pods to schedule on a subset of nodes) now support the operators {In, NotIn, Exists, DoesNotExist, Gt, Lt} instead of just conjunction of exact match on node label values. In addition, we’ve introduced a new “soft” kind of node selector that is just a hint to the scheduler; the scheduler will try to satisfy these requests but it does not guarantee they will be satisfied. Both the “hard” and “soft” variants of node affinity use the new syntax. Documentation is here (see section “Alpha feature in Kubernetes v1.2: Node Affinity“). Design doc is here.
  • A pod can specify its own Hostname and Subdomain via annotations (pod.beta.kubernetes.io/hostname, pod.beta.kubernetes.io/subdomain). If the Subdomain matches the name of a headless service in the same namespace, a DNS A record is also created for the pod’s FQDN. More details can be found in the DNS README. Changes were introduced in PR #20688.
  • New SchedulerExtender enables users to implement custom out-of-(the-scheduler)-process scheduling predicates and priority functions, for example to schedule pods based on resources that are not directly managed by Kubernetes. Changes were introduced in PR #13580. Example configuration and documentation is available here. This is an alpha feature and may not be supported in its current form at beta or GA.
  • New Flex Volume Plugin enables users to use out-of-process volume plugins that are installed to “/usr/libexec/kubernetes/kubelet-plugins/volume/exec/” on every node, instead of being compiled into the Kubernetes binary. See example for details.
  • vendor volumes into a pod. It expects vendor drivers are installed in the volume plugin path on each kubelet node. This is an alpha feature and may change in future.
  • Kubelet exposes a new Alpha metrics API - /stats/summary in a user friendly format with reduced system overhead. The measurement is done in PR #22542.

Action required

  • Docker v1.9.1 is officially recommended. Docker v1.8.3 and Docker v1.10 are supported. If you are using an older release of Docker, please upgrade. Known issues with Docker 1.9.1 can be found below.
  • CPU hardcapping will be enabled by default for containers with CPU limit set, if supported by the kernel. You should either adjust your CPU limit, or set CPU request only, if you want to avoid hardcapping. If the kernel does not support CPU Quota, NodeStatus will contain a warning indicating that CPU Limits cannot be enforced.
  • The following applies only if you use the Go language client (/pkg/client/unversioned) to create Job by defining Go variables of type "k8s.io/kubernetes/pkg/apis/extensions".Job). We think this is not common, so if you are not sure what this means, you probably aren't doing this. If you do this, then, at the time you re-vendor the "k8s.io/kubernetes/" code, you will need to set job.Spec.ManualSelector = true, or else set job.Spec.Selector = nil. Otherwise, the jobs you create may be rejected. See Specifying your own pod selector.
  • Deployment was Alpha in 1.1 (though it had apiVersion extensions/v1beta1) and was disabled by default. Due to some non-backward-compatible API changes, any Deployment objects you created in 1.1 won’t work with in the 1.2 release.
    • Before upgrading to 1.2, delete all Deployment alpha-version resources, including the Replication Controllers and Pods the Deployment manages. Then create Deployment Beta resources after upgrading to 1.2. Not deleting the Deployment objects may cause the deployment controller to mistakenly match other pods and delete them, due to the selector API change.
    • Client (kubectl) and server versions must match (both 1.1 or both 1.2) for any Deployment-related operations.
    • Behavior change:
      • Deployment creates ReplicaSets instead of ReplicationControllers.
      • Scale subresource now has a new targetSelector field in its status. This field supports the new set-based selectors supported by Deployments, but in a serialized format.
    • Spec change:
      • Deployment’s selector is now more general (supports set-based selector; it only supported equality-based selector in 1.1).
      • .spec.uniqueLabelKey is removed -- users can’t customize unique label key -- and its default value is changed from “deployment.kubernetes.io/podTemplateHash” to “pod-template-hash”.
      • .spec.strategy.rollingUpdate.minReadySeconds is moved to .spec.minReadySeconds
  • DaemonSet was Alpha in 1.1 (though it had apiVersion extensions/v1beta1) and was disabled by default. Due to some non-backward-compatible API changes, any DaemonSet objects you created in 1.1 won’t work with in the 1.2 release.
    • Before upgrading to 1.2, delete all DaemonSet alpha-version resources. If you do not want to disrupt the pods, use kubectl delete daemonset --cascade=false. Then create DaemonSet Beta resources after upgrading to 1.2.
    • Client (kubectl) and server versions must match (both 1.1 or both 1.2) for any DaemonSet-related operations.
    • Behavior change:
      • DaemonSet pods will be created on nodes with .spec.unschedulable=true and will not be evicted from nodes whose Ready condition is false.
      • Updates to the pod template are now permitted. To perform a rolling update of a DaemonSet, update the pod template and then delete its pods one by one; they will be replaced using the updated template.
    • Spec change:
      • DaemonSet’s selector is now more general (supports set-based selector; it only supported equality-based selector in 1.1).
  • Running against a secured etcd requires these flags to be passed to kube-apiserver (instead of --etcd-config):
    • --etcd-certfile, --etcd-keyfile (if using client cert auth)
    • --etcd-cafile (if not using system roots)
  • As part of preparation in 1.2 for adding support for protocol buffers (and the direct YAML support in the API available today), the Content-Type and Accept headers are now properly handled as per the HTTP spec. As a consequence, if you had a client that was sending an invalid Content-Type or Accept header to the API, in 1.2 you will either receive a 415 or 406 error. The only client this is known to affect is curl when you use -d with JSON but don't set a content type, helpfully sends "application/x-www-urlencoded", which is not correct. Other client authors should double check that you are sending proper accept and content type headers, or set no value (in which case JSON is the default). An example using curl: curl -H "Content-Type: application/json" -XPOST -d '{"apiVersion":"v1","kind":"Namespace","metadata":{"name":"kube-system"}}' "http://127.0.0.1:8080/api/v1/namespaces"
  • The version of InfluxDB is bumped from 0.8 to 0.9 which means storage schema change. More details here.
  • We have renamed “minions” to “nodes”. If you were specifying NUM_MINIONS or MINION_SIZE to kube-up, you should now specify NUM_NODES or NODE_SIZE.

Known Issues

  • Paused deployments can't be resized and don't clean up old ReplicaSets.
  • Minimum memory limit is 4MB. This is a docker limitation
  • Minimum CPU limits is 10m. This is a Linux Kernel limitation
  • “kubectl rollout undo” (i.e. rollback) will hang on paused deployments, because paused deployments can’t be rolled back (this is expected), and the command waits for rollback events to return the result. Users should use “kubectl rollout resume” to resume a deployment before rolling back.
  • “kubectl edit ” will open the editor multiple times, once for each resource in the list.
  • If you create HPA object using autoscaling/v1 API without specifying targetCPUUtilizationPercentage and read it using kubectl it will print default value as specified in extensions/v1beta1 (see details in #23196).
  • If a node or kubelet crashes with a volume attached, the volume will remain attached to that node. If that volume can only be attached to one node at a time (GCE PDs attached in RW mode, for example), then the volume must be manually detached before Kubernetes can attach it to other nodes.
  • If a volume is already attached to a node any subsequent attempts to attach it again (due to kubelet restart, for example) will fail. The volume must either be manually detached first or the pods referencing it deleted (which would trigger automatic volume detach).
  • In very large clusters it may happen that a few nodes won’t register in API server in a given timeframe for whatever reasons (networking issue, machine failure, etc.). Normally when kube-up script will encounter even one NotReady node it will fail, even though the cluster most likely will be working. We added an environmental variable to kube-up ALLOWED_NOTREADY_NODES that defines the number of nodes that if not Ready in time won’t cause kube-up failure.
  • “kubectl rolling-update” only supports Replication Controllers (it doesn’t support Replica Sets). It’s recommended to use Deployment 1.2 with “kubectl rollout” commands instead, if you want to rolling update Replica Sets.
  • When live upgrading Kubelet to 1.2 without draining the pods running on the node, the containers will be restarted by Kubelet (see details in #23104).

Docker Known Issues

1.9.1
  • Listing containers can be slow at times which will affect kubelet performance. More information here
  • Docker daemon restarts can fail. Docker checkpoints have to deleted between restarts. More information here
  • Pod IP allocation-related issues. Deleting the docker checkpoint prior to restarting the daemon alleviates this issue, but hasn’t been verified to completely eliminate the IP allocation issue. More information here
  • Daemon becomes unresponsive (rarely) due to kernel deadlocks. More information here

Provider-specific Notes

Various

Core changes:

  • Support for load balancers with source ranges

AWS

Core changes:

  • Support for ELBs with complex configurations: better subnet selection with multiple subnets, and internal ELBs
  • Support for VPCs with private dns names
  • Multiple fixes to EBS volume mounting code for robustness, and to support mounting the full number of AWS recommended volumes.
  • Multiple fixes to avoid hitting AWS rate limits, and to throttle if we do
  • Support for the EC2 Container Registry (currently in us-east-1 only)

With kube-up:

  • Automatically install updates on boot & reboot
  • Use optimized image based on Jessie by default
  • Add support for Ubuntu Wily
  • Master is configured with automatic restart-on-failure, via CloudWatch
  • Bootstrap reworked to be more similar to GCE; better supports reboots/restarts
  • Use an elastic IP for the master by default
  • Experimental support for node spot instances (set NODE_SPOT_PRICE=0.05)

GCE

  • Ubuntu Trusty support added

Please see the Releases Page for older releases.

Analytics