Automatic merge from submit-queue (batch tested with PRs 57252, 57168). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Implementing predicates ordering
**What this PR does / why we need it**: implements predicates ordering for the scheduler
**Which issue(s) this PR fixes** : Fixes#53812
**Special notes for your reviewer**:
@bsalamat @gmarek @resouer as discussed on slack, to implement ordering we have to choices:
- use a layered approach with a list that indexes the order of the predicates map
- change the underlying data structure used to represent a collection of predicates (a map in our case) into a list of predicates objects.
Going with this solution might be "cleaner" but it will require a lot of changes and will increase the cost for accessing predicates from O(1) to O(n) (n being the number of predicates used by the scheduler).
we might go with this solution for now. If the number of predicates start growing, we might switch to the second option.
**Release note**:
```release-note
adding predicates ordering for the kubernetes scheduler.
```
Automatic merge from submit-queue (batch tested with PRs 57252, 57168). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Reduce CPU and memory requests for Metrics Server Nanny
**What this PR does / why we need it**:
Metrics Server Nanny is a sidecar container that performs small computations every 5 minutes to scale Metrics Server resource requirements when cluster size changes. This change reduces the CPU and memory requests to free up unused resource.
**Release note**:
```release-note
Free up CPU and memory requested but unused by Metrics Server Pod Nanny.
```
Automatic merge from submit-queue (batch tested with PRs 56681, 57384). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Deprecate the alpha Accelerators feature gate.
Encourage people to use DevicePlugins instead.
/kind cleanup
Related to https://github.com/kubernetes/features/issues/192 and https://github.com/kubernetes/features/issues/368
**Release note**:
```release-note
The alpha Accelerators feature gate is deprecated and will be removed in v1.11. Please use device plugins instead. They can be enabled using the DevicePlugins feature gate.
```
/sig node
/sig scheduling
/area hw-accelerators
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Build etcd image for 3.1.11 version
This is fixing some bugs in HA setups.
It should be cherrypicked to 1.9 branch.
Automatic merge from submit-queue (batch tested with PRs 56600, 56814). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update systemstat9 to allow compilation on OSX
The latest version of system statadds stubbed out methods for non-Linux OSes:
https://bitbucket.org/bertimus9/systemstat/pull-requests/2
**What this PR does / why we need it**:
The current version of Systemstat is Linux-only. This pull request bumps bertimus9/systemstat to a slightly newer revision which does compile on OSX.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 56600, 56814). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix volume size allocation in gcd_pd
**What this PR does / why we need it**:
GCE PDs are allocated in chunks of GBs not GiB but CreateVolume function incorrectly creates volume in chunks of GiB.
1 GiB = 1024 * 1024 * 1024 Bytes
1 GB = 1000 * 1000 * 1000 Bytes
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56081
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixed dynamic provisioning of GCE PDs to round to the next GB instead of GiB
```
Automatic merge from submit-queue (batch tested with PRs 56716, 52371). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Separate loop and plugin control
**What this PR does / why we need it**: Separate loop and plugin control in the kube-controller-manager.
Adding an "--external-plugin" flag to specify a plugin to load when
cloud-provider is set to "external". Flag has no effect currently
when the cloud-provider is not set to external. The expectation is
that the cloud provider and external plugin flags would go away once
all cloud providers are on stage 2 cloud-controller-manager solutions.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#52369
**Special notes for your reviewer**:
**Release note**:
```release-note Added --external-plugin flag to kube-controller-manager to determine plugin when --cloud-provider is set to external.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Collect all the assorted image URLs from e2e tests in one place
utils/image/manifest has an additional `arch` parameter, which determines
whether an image ends in `-$ARCH` (like `-amd64`).
All locations that previously had gcr.io urls referenced in costants or inline
have been updated to refere test/utils/image.
**What this PR does / why we need it**:
Previously, all sorts of `gcr.io/` URLs were scattered all over the E2E tests codebase. This PR unifies them all in one place, making it easier to see what is needed to run the E2E tests, as well as making sure all tests use the same version of docker images.
Heptio is working on a project to get the E2E tests running in airgapped environments. Part of that task is collecting all images in one place. A future PR will allow modification of the registry URLs to point at a private docker registry.
**Special notes for your reviewer**:
Two images, https://github.com/kubernetes/kubernetes/pull/56716/files#diff-e3afa632328a4a5271f4b8578faa34bdL318 and https://github.com/kubernetes/kubernetes/pull/56716/files#diff-7dc1ebd3b2f2f6ca1ff248e6601016a6L104 are now pointing to slightly different images. If this breaks the E2E tests, the variables can be duplicated to restore the old dependencies, and the duplicates can be removed at a later date.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57400, 57403, 57303). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Restrict url conditions when run kubectl create with --raw
**What this PR does / why we need it**:
Currently when run kubectl create with --raw, it will raise "--raw cannot read from a url " if the filename has prefix "http", which is not proper as we may have filenames like "httptest" or similar.
This PR restrict the URL check conditions.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57370
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57400, 57403, 57303). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use old dns-ip mechanism with older cdk-addons.
**What this PR does / why we need it**: Use old dns-ip mechanism with older cdk-addons.
**Release note**:
```release-note
Use old dns-ip mechanism with older cdk-addons.
```
Automatic merge from submit-queue (batch tested with PRs 57400, 57403, 57303). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Pointing juju charms to 1.9
**What this PR does / why we need it**: Point juju charms to 1.9 k8s release
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add --retry-connrefused to all curl invocations if available.
By default 'Connection refused' error is not a transient error
and is not retried.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Retry 'connection refused' errors when setting up clusters on GCE.
```
utils/image/manifest has an additional `arch` parameter, which determines
whether an image ends in `-$ARCH` (like `-amd64`).
All locations that previously had gcr.io urls referenced in costants or inline
have been updated to refere test/utils/image.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Minor lint fix on Juju master
**What this PR does / why we need it**: Fix lint error in master
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
adds generic scaler to kubectl
**What this PR does / why we need it**:
this PR adds generic scaler to kubectl. The implementation uses a polymorphic scale client capable of operating against scale subresources which can be found here https://github.com/kubernetes/client-go/tree/master/scale
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 56403, 57243). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Register metav1 types into samplecontroller api scheme
**What this PR does / why we need it**:
Registers metav1 resource types (e.g. ListOptions) with sample-controller scheme.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57205
**Release note**:
```release-note
NONE
```
/cc @sttts @nikhita
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
code-gen: reusable main.go logic, preparation for kube-gen
Split main.go files into plumbing and calls to NewDefaults, AddFlags and Validate. This will allow us to create kube-gen without duplicating much code, at least no generator logic (with the exception of a little loop in client-gen adding the group path to the input dirs).
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Version bump to etcd v3.2.11, grpc v1.7.5
Fix https://github.com/kubernetes/kubernetes/issues/56114: Update to etcd client 3.2.11
Version bumps:
- etcd from 3.1.10 to 3.2.11
- grpc from 1.3.0 to 1.7.5
- grpc-gateway from v1.1.0-25-g84398b9 to v1.3.0
TODO:
- [x] Apply etcd [3.2 client upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md)
- [x] Apply grpc API changes in 1.6.0 and 1.7.0 [release notes](https://github.com/grpc/grpc-go/releases)
- [x] bbolt was pulled in transitively, why? We have tests that embed etcd, so we must vendor the etcd server and all it's dependencies.
- [x] Upgrade to containerd v1.0.0? Currently kubernetes depends on containerd v1.0.0-beta.2-159-g27d450a0 which depends on grpc v1.3.0, but containerd v1.0.0 depends on grpc 1.7.2. Not needed. The containerd grpc upgrade required [no code changes](ce3e32680d).
- [x] Fix all failing tests
- [x] Ensure we can safely upgrade grpc to 1.7.5 given that docker and cAdvisor still depend on grpc 1.3.0 (both in the versions we vend and on master for both projects). Should we hold off on this change until we have a docker release that uses gprc 1.7.x?
- [x] Wait for grpc 1.7.5 to be released (it will include https://github.com/grpc/grpc-go/pull/1747). Once released, bump grpc version in this PR and remove workarounds in `hack/godep-save.sh`.
Transitive dependencies on grpc:
- docker depends on grpc, but according to the package dependency graph (`go list -f '{{ .Deps }}'`) there are no dependencies from kubernetes to grpc via docker packages.
- containerd v1.0.0 depends on grpc 1.7.2, we should upgrade to containerd v1.0.0 soon, this can be done in a separate PR
- cadvisor depends on grpc 1.3.0 on master, it should upgrade it to grpc 1.7.5, this can be done in a separate PR
**Release note**:
```release-note
Upgrade to etcd client 3.2.11 and grpc 1.7.5 to improve HA etcd cluster stability.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add azure owners
**What this PR does / why we need it**:
Add azure owners and sort owner files.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
cc @khenidak @andyzhangx
**Release note**:
```release-note
NONE
```
/assign @brendandburns
Automatic merge from submit-queue (batch tested with PRs 55475, 57155, 57260, 57222). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
make sure that 'ldflags' are space-safe
**What this PR does / why we need it**:
Recently I met the problem as #56216 described, I download the source-tar of 1.8.5 and run `make` command failed because of invalid ldflag:
`-X k8s.io/kubernetes/pkg/version.gitTreeState=git archive`
Though #56249 has change version string `git archive` to `archive`, i think we should avoid this problem happen again.
cc @ixdy
**Release note**: NONE
Automatic merge from submit-queue (batch tested with PRs 55475, 57155, 57260, 57222). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improved mount/attach error logging and added attach event.
Fixed kubelet error message to be more descriptive. Added Attach success event for help in debugging.
The attach event is helpful when the node may not have the correct information about attachment status, it allows the user to see whether the Attach was run at all. If there is no success/failure attach message we can infer that there was no attach started at all.
Fixes#57217
Automatic merge from submit-queue (batch tested with PRs 55475, 57155, 57260, 57222). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
FC plugin: Update detach logic for block volume if devicePath is empty
**What this PR does / why we need it**:
Currently, FC plugin doesn't have a chance to update attached volume's devicePath into volume object.
Therefore, at the volume teardown path, FC plugin gets empty devicePath from kubelet then fails volume detach operations especially volumeMode is Block.
This PR adds logic to obtain devicePath from global map path if passed devicePath from kubelet is empty.
I think this PR isn't complete solution but is a workaround to avoid teardown failures. In order to solve the root cause, we need to discuss the way to update devicePath at kubelet side again.(https://github.com/kubernetes/kubernetes/pull/54264)
If volume is managed by multipath, FC plugin fails to detach volume from kubelet node on both Filesystem and Block cases. This PR also fix the problem.
**Which issue(s) this PR fixes** : Fixes#57153
**Special notes for your reviewer**:
@rootfs @jsafrane
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55475, 57155, 57260, 57222). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix e2e local test
**What this PR does / why we need it**:
Fix some issue on local e2e_node test: `Can't start e2e service "kubelet"`
**Which issue(s) this PR fixes**:
Fixes https://github.com/kubernetes/kubernetes/issues/54622
**Special notes for your reviewer**:
**Release note**:
```release-note
```
/sig node
Automatic merge from submit-queue (batch tested with PRs 55751, 57337, 56406, 56864, 57347). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add comment to gce config files advising to use non-empty scopes
**What this PR does / why we need it**: Add a comment in gce config files advising to use non-empty scopes, because kubelet requires a service account to initialize.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55751, 57337, 56406, 56864, 57347). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add pod-selector kubectl drain
**Release note**:
```release-note
Added the ability to select pods in a chosen node to be drained, based on given pod label-selector
```
This patch adds the ability to select pods in a chosen node to be drained, based on given pod label-selector. Related downstream issue: https://github.com/openshift/origin/issues/17554
Further, it removes explicit, specific, pod-controller check. The `drain` command currently fails if a pod has a controller of a `kind` [not explicitly handled in the command itself](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/drain.go#L331). This causes `drain` to be unusable if a node contains pods managed by third-party, or "unknown" controllers.
Based on [this comment](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/drain.go#L353), the expectation was to fail if a pod's controller was not found for whatever reason. I believe that the `drain` command should not care about the existence of a pod controller. It should only care whether a pod has one, and act according to that controller kind. This solves a downstream bug: https://github.com/openshift/origin/issues/17563
cc @fabianofranz @deads2k @kubernetes/sig-cli-misc
Automatic merge from submit-queue (batch tested with PRs 55751, 57337, 56406, 56864, 57347). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
should reuse code rather than rewrite it
**What this PR does / why we need it**:
should reuse `dc.GetDatastoreByName()`, instead of rewrite it
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55751, 57337, 56406, 56864, 57347). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Wait for kubedns to be ready when collecting the cluster IP.
**What this PR does / why we need it**: Wait for kubedns to be ready when collecting the cluster IP.
**Release note**:
```release-note
Wait for kubedns to be ready when collecting the cluster IP.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move `DefaultTerminationGracePeriodSeconds` into a separate const group
**What this PR does / why we need it**:
It seems odd to me to keep `DefaultTerminationGracePeriodSeconds` in the const group where DNSPolicies live in.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #NONE
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 56108, 56811, 57335, 57331, 54530). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
api: validate container phase transitions
https://github.com/kubernetes/kubernetes/issues/54499 exposed an issue where a container was transitioning from the terminal phases of `Succeeded` or `Failed` to `Pending`. It is due to a bug in the kubelet, but additional validation in the API server can prevent this invalid phase transition from being accepted.
@smarterclayton @derekwaynecarr @dashpole @joelsmith @frobware
I confirmed that the reproducer in https://github.com/kubernetes/kubernetes/issues/54499 does not work with this validation in place. The erroneous kubelet status update is rejected:
```
status_manager.go:437] Failed to update status for pod "test_default(2f02ecdf-b92a-11e7-a2d0-1c1b0deeddfa)": Pod "test" is invalid: status.containerStatuses[0].state: Forbidden: may not be transitioned to non-terminated state
```
However, it only works to a point with this particular issue. The termination hangs and eventually the resource is removed from etcd and the status update goes through because there is no old statuses to compare. Not exactly sure how this happens since there is no pod in etcd anymore ¯\\_(ツ)_/¯
Automatic merge from submit-queue (batch tested with PRs 56108, 56811, 57335, 57331, 54530). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
E2E autoscaling test scenario "should add node to the particular mig" fix
Resize targeted MIG to one to fix flakes when initial size of it is zero. Scale-up from zero doesn't make sense in this scenario, and the test assumes there will be always at least one node.
Automatic merge from submit-queue (batch tested with PRs 56108, 56811, 57335, 57331, 54530). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix Stackdriver Logging e2e tests
This PR fixes the problems in the SD Logging tests:
- Because of Docker problem, trimming works unpredictably on some OSs, so instead of assuming exact trimming mechanism, just assume trimming
- Increases the throughput of reading logs through Pub/Sub by increasing the parallelism level
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
e2e_node: small fixes to setup_host.sh for Ubuntu Trusty
**What this PR does / why we need it**:
Two small fixes for how nsenter is built from source and installed on Ubuntu Trusty:
1. Use mktemp for the creating the build directory instead of a hard coded name
2. Use current (2.31) util-linux instead of 3+ year old version
**Which issue(s) this PR fixes**:
Fixes#56106
**Special notes for your reviewer**:
See https://github.com/kubernetes/kubernetes/issues/56106 for some thoughts on other ways to address this. My patch for util-linux 2.31 may just be a band-aid?
**Release note**:
```release-note
NONE
```
Kubernetes Submit Queue