Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add azuredisk PV size grow feature
**What this PR does / why we need it**:
According to kubernetes/features#284, add size grow feature for azure disk
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56463
**Special notes for your reviewer**:
- This feature is ony for azure managed disk, and if that disk is already attached to a running VM, disk resize will fail as following:
```
$ kubectl describe pvc pvc-azuredisk
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning VolumeResizeFailed 51s (x3 over 3m) volume_expand Error expanding volume "default/pvc-azuredisk" of plugin kubernetes.io/azure-disk : disk.DisksClient#CreateOrUpdate: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="OperationNotAllowed" Message="Cannot resize disk andy-mg1102-dynamic-pvc-d2d00dd9-6185-11e8-a6c3-000d3a0643a8 while it is attached to running VM /subscriptions/.../resourceGroups/.../providers/Microsoft.Compute/virtualMachines/k8s-agentpool-17607330-0."
```
**How to use this feature**
- `kubectl edit pvc pvc-azuredisk` to change azuredisk PVC size from 6GB to 10GB
```
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
...
volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/azure-disk
creationTimestamp: 2018-05-27T08:13:23Z
finalizers:
- kubernetes.io/pvc-protection
name: pvc-azuredisk
...
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 6Gi
storageClassName: hdd
volumeMode: Filesystem
volumeName: pvc-d2d00dd9-6185-11e8-a6c3-000d3a0643a8
status:
accessModes:
- ReadWriteOnce
capacity:
storage: 6Gi
conditions:
- lastProbeTime: null
lastTransitionTime: 2018-05-27T08:14:34Z
message: Waiting for user to (re-)start a pod to finish file system resize of
volume on node.
status: "True"
type: FileSystemResizePending
phase: Bound
```
- After resized, `/mnt/disk` is still 6GB
```
$ kubectl exec -it nginx-azuredisk -- bash
# df -h
Filesystem Size Used Avail Use% Mounted on
...
/dev/sdf 5.8G 15M 5.5G 1% /mnt/disk
...
```
- After user run `sudo resize2fs /dev/sdf` in agent node, `/mnt/disk` becomes 10GB now:
```
$ kubectl exec -it nginx-azuredisk -- bash
# df -h
Filesystem Size Used Avail Use% Mounted on
...
/dev/sdf 9.8G 16M 9.3G 1% /mnt/disk
...
```
**Release note**:
```
Add azuredisk size grow feature
```
/sig azure
/assign @feiskyer @karataliu @gnufied
cc @khenidak
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Modify e2e tests to use priorityClass beta client version & switch priorityClass to beta
**What this PR does / why we need it**:
/cc @bsalamat @aveshagarwal @liggitt
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #57471
**Special notes for your reviewer**:
Once #57963 merges, this could go in.
**Release note**:
```release-note
Modify e2e tests to use priorityClass beta version & switch priorityClass feature to beta
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix error handling in gc e2e test
**What this PR does / why we need it**:
Error messages were not being surfaced in log of GC e2e test, part of fixing #60463
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix test tag on dynamic config tests
The test accidentally got turned off when the NodeAlphaFeature tag was
added in #64125. This PR updates the tag to turn it back on.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adding CSI driver registration with plugin watcher
Adding CSI driver registration bits. The registration process will leverage driver-registrar side which will open the `registration` socket and will listen for pluginwatcher's GetInfo calls.
```release-note
Adding CSI driver registration code.
```
/sig sig-storage
Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix the handling of untagged images
**What this PR does / why we need it**:
This PR addresses the issue [#63354](https://github.com/kubernetes/kubernetes/issues/63354). If a container image is untagged, the default ":latest" tag will be appended to the image, preserving the semantics of the kubelet's handling and reporting the container images on a node.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63354
**Special notes for your reviewer**:
cc @resouer
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bind mount subpath with same read/write settings as underlying volume
**What this PR does / why we need it**:
https://github.com/kubernetes/kubernetes/pull/63045 broke two scenarios:
* If volumeMount path already exists in container image, container runtime will try to chown the volume
* In SELinux system, we will try to set SELinux labels when starting the container
This fix makes it so that the subpath bind mount will inherit the read/write settings of the underlying volume mount. It does this by using the "bind,remount" mount options when doing the bind mount.
The underlying volume mount is ro when the volumeSource.readOnly flag is set. This is for persistent volume types like PVC, GCE PD, NFS, etc. When this is set, we won't try to configure SELinux labels. Also in this mode, subpaths have to already exist in the volume, we cannot make new directories on a read only volume.
When volumeMount.readOnly is set, the container runtime is in charge of making the volume in the container readOnly, but the underlying volume mount on the host can be writable. This can be set for any volume type, and is permanently set for atomic volume types like configmaps, secrets. In this case, SELinux labels will be applied before the container runtime makes the volume readOnly. And subpaths don't have to exist.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#64120
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixes issue for readOnly subpath mounts for SELinux systems and when the volume mountPath already existed in the container image.
```
Automatic merge from submit-queue (batch tested with PRs 62266, 64351, 64366, 64235, 64560). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add log and fs stats for Windows containers
**What this PR does / why we need it**:
Add log and fs stats for Windows containers.
Without this, kubelet will report errors continuously:
```
Unable to fetch container log stats for path \var\log\pods\2a70ed65-37ae-11e8-8730-000d3a14b1a0\echo: Du not supported for this build.
```
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60180#62047
**Special notes for your reviewer**:
**Release note**:
```release-note
Add log and fs stats for Windows containers
```
Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Replace glog.Info{f,ln} with fmt.Print{f,ln}
Please see commit message.
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
**What this PR does / why we need it**:
This PR makes kubeadm's logging output consistent for the regular case and offers verbosity with glog.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixeskubernetes/kubeadm#852
**Special notes for your reviewer**:
Unfortunately there is one call site in a dependency that uses a bare glog.Infof which we can't change without a bigger effort.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
client-go: promote exec plugin support to beta
/sig auth
/kind feature
Adds a "v1beta1" API group for client authentication. Because of a lack of usage with the input parameters, these have been dropped for the beta. Would like to re-introduce them as users feel they require them.
updates https://github.com/kubernetes/kubernetes/issues/61796
```release-note
client-go: credential exec plugins have been promoted to beta
```
Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Revert "Remove rescheduler and corresponding tests from master"
Reverts kubernetes/kubernetes#64364
After discussing with @bsalamat on how DS controllers(ref: https://github.com/kubernetes/kubernetes/pull/63223#discussion_r192277527) cannot create pods if the cluster is at capacity and they have to rely on rescheduler for making some space, we thought it is better to
- Bring rescheduler back.
- Make rescheduler priority aware.
- If cluster is full and if **only** DS controller is not able to create pods, let rescheduler be run and let it evict some pods which have less priority.
- The DS controller pods will be scheduled now.
So, I am reverting this PR now. Step 2, 3 above are going to be in rescheduler.
/cc @bsalamat @aveshagarwal @k82cn
Please let me know your thoughts on this.
```release-note
Revert #64364 to resurrect rescheduler. More info https://github.com/kubernetes/kubernetes/issues/64725 :)
```
Automatic merge from submit-queue (batch tested with PRs 63453, 64592, 64482, 64618, 64661). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add Makefile target to push fat manifest for multi-arch images
Signed-off-by: Nitesh Konkar <niteshkonkar@in.ibm.com>
**What this PR does / why we need it**:
This PR adds support to push fat manifest for multi-arch images. We need this
so that we can seamlessly pull and run the test images on multiple platforms.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Introduce priority class in the resource quota
**What this PR does / why we need it**:
Implements https://github.com/kubernetes/community/pull/933
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#48648
**Special notes for your reviewer**:
Test Cases are still to be covered. Opening this PR to make discussion convenient with code references.
Will update test cases only after design PR has got merged.
**Release note**:
```release-note
Ability to quota resources by priority
```
/kind feature
/priority important-soon
/sig scheduling
/sig node
/cc @resouer @derekwaynecarr @sjenning @bsalamat @timstclair @aveshagarwal @ravisantoshgudimetla
Automatic merge from submit-queue (batch tested with PRs 61610, 64591, 58143, 63929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add parametrizable priority function mapping requested/capacity ratio to priority
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61610, 64591, 58143, 63929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add netd as an addon for GCP
**What this PR does / why we need it**:
Add netd as an addon for GKE.
The PR will add setup functions and var to help deploy netd daemon on GKE.
Please checkout more detail for netd at https://github.com/GoogleCloudPlatform/netd
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61610, 64591, 58143, 63929). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Create CoreDNS and kube-dns folders
**What this PR does / why we need it**:
Separate the CoreDNS and kube-dns manifests by creating their own folders (dns/coredns and dns/kube-dns)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61435
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
cc @MrHohn
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apimachinery: do not fuzz ObjectMeta.{Labels/Annotation} with empty keys
Fixes flaking `TestReplicationControllerConversion`.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubeadm: Set the kubelet `--resolv-conf` flag conditionally on init
**What this PR does / why we need it**:
`kubeadm init` detects if systemd-resolved is running and configures the kubelet to use a working resolv.conf.
This patch also removes the warning message prompting manual user action for this configuration.
/area kubeadm
/area kubelet
/area dns
/kind bug
/priority important-soon
/sig cluster-lifecycle
/assign @timothysc
**Which issue(s) this PR fixes**
Fixes https://github.com/kubernetes/kubeadm/issues/845
**Special notes for your reviewer**:
See the difference in `KUBELET_KUBEADM_ARGS` when running with this build and enabling the resolved daemon on Ubuntu 17.10:
```bash
root@vagrant:/vagrant/bin# bash << EOF
systemctl start systemd-resolved
./845_kubeadm init |& tail -n5
cat /var/lib/kubelet/kubeadm-flags.env
./845_kubeadm reset --force |& tail -n2
systemctl stop systemd-resolved
echo nameserver 8.8.8.8 > /etc/resolv.conf
./845_kubeadm init |& tail -n5
cat /var/lib/kubelet/kubeadm-flags.env
EOF
You can now join any number of machines by running the following on each node
as root:
kubeadm join 10.0.2.15:6443 --token 77q84j.0342evur7rrfrwwx --discovery-token-ca-cert-hash sha256:190040f9c3adf8410bc6766dac79f8679870190564e15e8f8d1704fafa03f678
KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni --resolv-conf=/run/systemd/resolve/resolv.conf
I0603 06:49:16.592482 14106 reset.go:276] [reset] deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
I0603 06:49:16.592858 14106 reset.go:290] [reset] deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
You can now join any number of machines by running the following on each node
as root:
kubeadm join 10.0.2.15:6443 --token 8mdart.gp67vq3nh9urq4z5 --discovery-token-ca-cert-hash sha256:da6b2e5841546eae134524b045e782f0dd91a6b53becc8d69c15d9eab9c88758
KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni
```
**Release note**:
```release-note
`kubeadm init` detects if systemd-resolved is running and configures the kubelet to use a working resolv.conf.
```
This follows the pattern `kubectl` uses for logging.
There are two remaining glog.Infof call that cannot be removed easily.
One glog call comes from kubelet validation which calls features.SetFromMap.
The other comes from test/e2e during kernel validation.
Mostly fixeskubernetes/kubeadm#852
Signed-off-by: Chuck Ha <ha.chuck@gmail.com>
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix data loss issue if using existing azure disk with partitions in disk mount
**What this PR does / why we need it**:
When use an existing azure disk(also called [static provisioning](https://github.com/andyzhangx/demo/tree/master/linux/azuredisk#static-provisioning-for-azure-disk)) in pod, if that disk has multiple partitions, the disk will be formatted in the pod mounting.
This PR removes `formatIfNotFormatted` func in `WaitForAttach` which uses `lsblk` command to check whether disk is formatted or not
b87a392b1a/pkg/volume/azure_dd/azure_common_linux.go (L213-L215)
And finally the format disk operation will happen in `MountDevice` in which it uses common k8s code(`SafeFormatAndMount.GetDiskFormat`) using `blkid` to detect disk format, `blkid` could detect multiple partitions
b87a392b1a/pkg/util/mount/mount_linux.go (L541-L543)
- so if we use common k8s code(`SafeFormatAndMount.GetDiskFormat`), following error will be returned for mulitple partition disks, which is expected:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63235
**Special notes for your reviewer**:
This PR depends on https://github.com/kubernetes/kubernetes/pull/63248
**Release note**:
```
fix data loss issue if using existing azure disk with partitions in disk mount
```
/sig azure
/assign @khenidak
Automatic merge from submit-queue (batch tested with PRs 64481, 64569). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add tallclair to milestone maintainers
For sig-auth.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64481, 64569). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Instrument envelop transformer.
**What this PR does / why we need it**:
Add metrics for envelope transformer:
transformation_operation_count
transformation_failures_count
envelope_transformation_cache_misses_count
data_key_generation_latencies_microseconds
data_key_generation_failures_count
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cinder Volume API changes for adding support for secrets in the future
**What this PR does / why we need it**:
Many of the in-tree volume sources support specification of secret(s) for specific volumes. This support is not present in cinder currently.
This PR adds just the changes in the API, so work can be done in the future in the external openstack controller manager.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Address comments in #64006.
Address comments in #64006
@tallclair @yujuhong
@kubernetes/sig-node-pr-reviews
Signed-off-by: Lantao Liu <lantaol@google.com>
**Release note**:
```release-note
none
```
lichuqiang