The file was analyzed with shellcheck, and various issues fixed. Most of
the problems were just cleanups, but also potential bugs were fixed.
Many variables were quoted with double quotes to prevent globbing. The
local_files array expansion was quoted so that any file names with
potential spaces in the filename would not be re-split. The empty
default value was removed from the list processing. POSIX standard "grep
-E" was used instead of egrep.
In code "popd 2>&1 > /dev/null" the stderr output goes to the terminal
instead of /dev/null, which is clearly the intention. Change the order
to fix this in four places. Also, while at it, change one incorrect
comparison to use "-gt" instead of ">" in the same file.
Automatic merge from submit-queue (batch tested with PRs 61453, 61393, 61379, 61373, 61494). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make `test-cmd` work with OS-X tooling
OS-X thips with the BSD versions of `date` and `grep`. Those don't have
certain features the script relies on:
- BSD date does not support nanoseconds (`%N`)
- BSD grep does not support perl RegEx (`-P`)
As we use `bash` specifically in the hashbang anyway it is probably fine
to rely on the `$RANDOM` bashism here.
**What this PR does / why we need it**:
Currently `make test-cmd` doesn't work on OSX. This PR makes `make test-cmd` work on OSX.
**Release note**:
```release-note
`make test-cmd` now works on OSX.
```
cc: @apelisse
Automatic merge from submit-queue (batch tested with PRs 61396, 61321, 61443, 60911, 61461). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add COPYING file name as valid license file
**What this PR does / why we need it**:
Some valuable libraries such as https://github.com/google/shlex which could be integrated into Kubernetes use COPYING for their License file name. This would be helpful in getting pull requests such as https://github.com/kubernetes/kubernetes/pull/59700 merged.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60980, 61273, 60811, 61021, 61367). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
prevent conformance test failure in DIND scenario
**What this PR does / why we need it**:
in the "ci-kubernetes-local-e2e" job, we are running local-up-cluster.sh
using DIND. We see failure for "HostPath should give a volume the
correct mode [Conformance]" test. The failure looks as follows:
linux mounts: path /tmp is mounted on / but it is not a shared or slave
mount
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
https://github.com/kubernetes/kubernetes/issues/61058#issuecomment-372764783
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60373, 61098, 61352, 61359, 61362). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use mktemp instead of static temporary file path in scripts.
**What this PR does / why we need it**:
Using static file names in `/tmp` is considered to be a security anti-pattern, even if the modern Linux distributions make it difficult for the attacker to use symbolic link attacks against the `/tmp` directory. We should consider changing to the pattern of safely creating a temporary directory which only the user can access and placing the temporary files there. This assumes that `mktemp` command is installed -- it should be a safe assumption since the command is already widely used in the kubernetes scripts.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60793, 61181, 61267, 61252, 61334). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
--show-all is inert in v1.11
**What this PR does / why we need it**:
`--show-all` is inert in v1.11
ref: #60210
**Special notes for your reviewer**:
/assign @deads2k
**Release note**:
```release-note
`--show-all` (which only affected pods and only for human readable/non-API printers) is inert in v1.11, and will be removed in a future release.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove YEAR field of all generated files and fix kubernetes boilerplate checker
**What this PR does / why we need it**:
Remove YEAR field of all generated files and fix kubernetes boilerplate checker
xref: [remove YEAR fileds in gengo #91](https://github.com/kubernetes/gengo/pull/91)
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes [#gengo/issues/24](https://github.com/kubernetes/gengo/issues/24)
**Special notes for your reviewer**:
/cc @thockin @lavalamp @sttts
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61124, 59537, 61235, 61258, 61114). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove outdated waiting apiserver comments in local-up-cluster.sh
**What this PR does / why we need it**:
Now secure port is used, remove outdated waiting for apiserver healthy comments in local-up-cluster.sh.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 61087, 61075, 61213, 61149, 60826). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add support for --dry-run to kubectl create job
**What this PR does / why we need it**:
Create job subcommand lacks dry-run flag, this patch fix this.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57871, 61094, 60459, 61089, 61105). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove hack/test-cmd.sh: make is the main build tool
The script suggests to use `make test-cmd` and we don't use this script anywhere.
**Release note**:
```release-note
NONE
```
/kind cleanup
/assign sttts cblecker
Automatic merge from submit-queue (batch tested with PRs 60919, 60953, 61085, 61083, 60971). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove unused `pkg/api/unversioned`
**What this PR does / why we need it**:
clean code, see #61084
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61084
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
1.add dry-run flag for create job subcommand
2.add cmd-util test for create job subcommand
modified: pkg/kubectl/cmd/create_job.go
modified: hack/make-rules/test-cmd-util.sh
Automatic merge from submit-queue (batch tested with PRs 60696, 60876, 60901, 60925, 60428). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove unused hack/lookup_pull.py
This script is not used anywhere.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add myself for sig-cli related stuff as approver
This is adding me as an approver for sig-cli related things.
/assign @pwittrock
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 60710, 60855, 60873, 60895, 60862). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
translate hack/e2e.go -v to --verbose-commands
**What this PR does / why we need it**: translates the old `go run hack/e2e.go` `-v` flag to `--verbose-commands` for kubetest. kubetest now imports client-go, which imports `glog`, which registers an incompatible `-v level` flag so kubetest now uses `--verbose-commands` instead.
This should fix existing workflows.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**: See also: https://github.com/kubernetes/community/pull/1901
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60710, 60855, 60873, 60895, 60862). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix local cluster leaking memory.
**What this PR does / why we need it**:
Local cluster is leaking memory due to mutation detector being enabled.
In addition there is no warning in the logs that this could be the
issue.
Added a log warning when this feature is enabled to make debugging this
issue easier for other cases of this.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60854
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 60710, 60855, 60873, 60895, 60862). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move `persistentvolume` into `storage` package under `plugin/pkg/admission`
Sorry for omitting this in the previous PR #60367
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Following #60367
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @jsafrane @thockin
Automatic merge from submit-queue (batch tested with PRs 60574, 60666, 60831, 60877, 60357). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove some not used imports from python codes
remove some not used imports from python codes
/assign @sttts
**Release note**:
```release-note
NONE
```
OS-X thips with the BSD versions of `date` and `grep`. Those don't have
certain features the script relies on:
- BSD date does not support nanoseconds (`%N`)
- BSD grep does not support perl RegEx (`-P`)
As we use `bash` specifically in the hashbang anyway it is probably fine
to rely on the `$RANDOM` bashism here.
in the "ci-kubernetes-local-e2e" job, we are running local-up-cluster.sh
using DIND. We see failure for "HostPath should give a volume the
correct mode [Conformance]" test. The failure looks as follows:
linux mounts: path /tmp is mounted on / but it is not a shared or slave
mount
Using static paths for temporary files has been a security problem in
the past. Use kube::util::ensure-temp-dir to create a safe temporary
directory and place the temporary file there.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bump to etcd 3.1.12 to pick up critical fix
etcd [3.1.12](https://github.com/coreos/etcd/releases/tag/v3.1.12) (as well as 3.2.17 and 3.3.2) was released yesterday to fix a bug critical to kubernetes:
Fix [mvcc "unsynced" watcher restore operation](https://github.com/coreos/etcd/pull/9297).
- "unsynced" watcher is watcher that needs to be in sync with events that have happened.
- That is, "unsynced" watcher is the slow watcher that was requested on old revision.
- "unsynced" watcher restore operation was not correctly populating its underlying watcher group.
- Which possibly causes [missing events from "unsynced" watchers](https://github.com/coreos/etcd/issues/9086).
This will be backported to 1.9 as well.
Release note:
```release-note
Upgrade the default etcd server version to 3.1.12 to pick up critical etcd "mvcc "unsynced" watcher restore operation" fix.
```
cc @gyuho @wojtek-t @shyamjvs @timothysc @jdumars
Local cluster is leaking memory due to mutation detector being enabled.
In addition there is no warning in the logs that this could be the
issue.
Added a log warning when this feature is enabled to make debugging this
issue easier for other cases of this.
Fixed import ordering as per @liggitt
Automatic merge from submit-queue (batch tested with PRs 60530, 60618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add support for `make verify WHAT=typecheck`
This will be used to make the typecheck job emit junit and avoids duplicating the rest of the verify logic.
**Release note**:
```release-note
NONE
```
Most of the time spent compiling is spent optimizing and linking
binary code. Most errors occur at the syntax or semantic (type) layers.
Go's compiler is importable as a normal package, so we can do fast
syntax and type checking for the 10 platforms we build on.
This currently takes ~6 minutes of CPU time (parallelized).
This makes presubmit cross builds superfluous, since it should catch
most cross-build breaks (generally Unix and 64-bit assumptions).
Example output:
$ time go run test/typecheck/main.go
type-checking: linux/amd64, windows/386, darwin/amd64, linux/arm, linux/386, windows/amd64, linux/arm64, linux/ppc64le, linux/s390x, darwin/386
ERROR(windows/amd64) pkg/proxy/ipvs/proxier.go:1708:27: ENXIO not declared by package unix
ERROR(windows/386) pkg/proxy/ipvs/proxier.go:1708:27: ENXIO not declared by package unix
real 0m45.083s
user 6m15.504s
sys 1m14.000s
Automatic merge from submit-queue (batch tested with PRs 59365, 60446, 60448, 55019, 60431). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Use feature-gates command line for kube-proxy
**What this PR does / why we need it**:
Avoid throwing `featureGates:` into the kube-proxy yaml, since it's
not in the right format. It should be
```
featureGates:
AllAlpha: true
```
Since FEATURES_GATES is set to `AllAlpha=true`, we can't just throw that
into the yaml file. Just pass along FEATURES_GATES in the command line
instead just like we do for all other services
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60426
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59365, 60446, 60448, 55019, 60431). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove dep-reviewers
**What this PR does / why we need it**:
The dep-reviewers group seems to get assigned PRs early the the review process. However, most code changes should be reviewed in the importing part of the code base first, and then assigned to an approver after.
By removing the reviewers group, the approvers plugin will still suggest assigning to an approver, but won't assign for review when the PR is initially opened.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59310, 60424, 60308, 60436, 60020). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix potential issues and bugs in hack/lib/*.sh scripts using shellcheck
**What this PR does / why we need it**:
This PR is continuing the work for cleaning up the shell scripts using shellcheck and manual inspection. The plan is to make the scripts more robust in case for unexpected input and also to fix potential bugs.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
The changes are for scripts in `hack/lib/` directory, and as library scripts are used from multiple places. It is not trivial to see all possible places from where the scripts are accessed, so careful review is needed. I tried to make sure that the changes would be compatible for older bash versions too.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
code-gen: output golint compliant 'Generated by' comment
New PR instead of reopening #58115 because /reopen did not work.
This won't be ready to merge until the upstream https://github.com/kubernetes/gengo/pull/94 merges. Once that merges, the second commit will be changed to godep-save.sh and update-staging-godeps.sh, and the last commit will be changed to update-all.sh
The failing test is due to the upstream changes not being merged yet
```devel-release-note
Go code generated by the code generators will now have a comment which allows them to be easily identified by golint
```
Fixes#56489
Automatic merge from submit-queue (batch tested with PRs 60011, 59256, 59293, 60328, 60367). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move `storageclass` into `storage` package under `plugin/pkg/admission`
We have two directories `storage` and `storageclass` under package `plugin/pkg/admission` and one subdir `storageobjectinuseprotection` of `storage` now.
- 1. Move `storageclass` into `storage` being `storage`'s sub-package since storageclass is one of the storage objects
- 2. fix golint failures
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50724, 59025, 59710, 59404, 59958). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Require boilerplate on Bazel Skylark source files
**What this PR does / why we need it**: `.bzl` files are also source code, so they should probably have the boilerplate text too.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60435, 60334, 60458, 59301, 60125). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Increase timeout of integration tests
**What this PR does / why we need it**:
In #24509 we intended to increase the timeout to 600s, but it was reverted by accident in #57521 when overriding of the value was enabled. The jenkins job should honor the default of 600s instead of continuing to override it to 300s.
This lead to flaky integration tests, specifically TestCRDDeletionCascading
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59426
**Special notes for your reviewer**:
**Release note**:
```release-note
Increase timeout of integration tests
```
Avoid throwing `featureGates:` into the kube-proxy yaml, since it's
not in the right format. It should be
```
featureGates:
AllAlpha: true
```
Since FEATURES_GATES is set to `AllAlpha=true`, we can't just throw that
into the yaml file. Just pass along FEATURES_GATES in the command line
instead just like we do for all other services
Do not bother with pushd/popd construct -- there is no need to go back
to original directory in a subshell. Use "find --exec" instead of "find
| xargs" to handle special file names better.
You can test this change like this:
$ mkdir -p '/tmp/foo bar/x'
$ cd $(dirname "/tmp/foo bar/x")
bash: cd: too many arguments
$ cd "$(dirname "/tmp/foo bar/x")"
$ pwd
/tmp/foo bar
It's admittedly extremely unlikely that the host platform name would
contain special characters, but still use double quotes to pattern
matching.
Consider this script:
#!/bin/bash
bar="foobar"
foo="foo*"
[[ $bar == $foo ]] && echo "first true"
[[ "$bar" == "$foo" ]] && echo "second true"
We get the output:
first true
The plan is to move from first case to the second case to prevent
pattern match where there shouldn't be any.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support Running local-up-cluster in CI
**What this PR does / why we need it**:
We have a new CI job for running conformance tests with local-up-cluster.sh:
https://k8s-testgrid.appspot.com/sig-testing-misc#ci-kubernetes-local-e2e
In this PR, we do some additional tasks to be able to setup and run kubetest
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59159, 60318, 60079, 59371, 57415). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Introduce External Metrics API
**What this PR does / why we need it**:
Introduce External Metrics API
This PR introduces External Metrics API which has been proposed: xref https://github.com/kubernetes/community/pull/1802
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60302, 57921, 59042, 60126, 59561). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove pkg/client/unversioned
**What this PR does / why we need it**:
This is removing unused package, and moves the used bits into appropriate placeholders.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#25442
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @deads2k
/assign @sttts
Automatic merge from submit-queue (batch tested with PRs 60196, 59844, 60255, 60254, 60251). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Unset CDPATH in build script to fix path generation
**What this PR does / why we need it**:
In #52255, it was determined that if the user has set the `CDPATH` environment variable, running `make` will fail. This is because using the `KUBE_ROOT` variable results in invalid paths when the CDPATH environment variable is set.
Fixes#52255
**Release note**:
```release-note
NONE
```
/area build-release
/sig testing
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apiextensions: add subresources for custom resources
Fixes#38113Fixes#58778
**Related**:
- Proposal: https://github.com/kubernetes/community/pull/913
- For custom resources to work with `kubectl scale`: https://github.com/kubernetes/kubernetes/pull/58283
**Add types**:
- Add `CustomResourceSubResources` type to CRD.
- Fix proto generation for `CustomResourceSubResourceStatus`: https://github.com/kubernetes/kubernetes/pull/55970.
- Add feature gate for `CustomResourceSubResources`.
- Update CRD strategy: if feature gate is disabled, this feature is dropped (i.e. set to `nil`).
- Add validation for `CustomResourceSubResources`:
- `SpecReplicasPath` should not be empty and should be a valid json path under `.spec`. If there is no value under the given path in the CustomResource, the `/scale` subresource will return an error on GET.
- `StatusReplicasPath` should not be empty and should be a valid json path under `.status`. If there is no value under the given path in the CustomResource, the status replica value in the /scale subresource will default to 0.
- If present, `LabelSelectorPath` should be a valid json path. If there is no value under `LabelSelectorPath` in the CustomResource, the status label selector value in the `/scale` subresource will default to the empty string.
- `ScaleGroupVersion` should be `autoscaling/v1`.
- If `CustomResourceSubResources` is enabled, only `properties` is allowed under the root schema for CRD validation.
**Add status and scale subresources**:
- Use helper functions from `apimachinery/pkg/apis/meta/v1/unstructured/helpers.go`.
- Improve error handling: https://github.com/kubernetes/kubernetes/pull/56563, https://github.com/kubernetes/kubernetes/pull/58215.
- Introduce Registry interface for storage.
- Update storage:
- Introduce `CustomResourceStorage` which acts as storage for the custom resource and its status and scale subresources. Note: storage for status and scale is only enabled when the feature gate is enabled _and_ the respective fields are enabled in the CRD.
- Introduce `StatusREST` and its `New()`, `Get()` and `Update()` methods.
- Introduce `ScaleREST` and its `New()`, `Get()` and `Update()` methods.
- Get and Update use the json paths from the CRD and use it to return an `autoscaling/v1.Scale` object.
- Update strategy:
- In `PrepareForCreate`,
- Clear `.status`.
- Set `.metadata.generation` = 1
- In `PrepareForUpdate`,
- Do not update `.status`.
- If both the old and new objects have `.status` and it is changed, set it back to its old value.
- If the old object has a `.status` but the new object doesn't, set it to the old value.
- If old object did not have a `.status` but the new object does, delete it.
- Increment generation if spec changes i.e. in the following cases:
- If both the old and new objects had `.spec` and it changed.
- If the old object did not have `.spec` but the new object does.
- If the old object had a `.spec` but the new object doesn't.
- In `Validate` and `ValidateUpdate`,
- ensure that values at `specReplicasPath` and `statusReplicasPath` are >=0 and < maxInt32.
- make sure there are no errors in getting the value at all the paths.
- Introduce `statusStrategy` with its methods.
- In `PrepareForUpdate`:
- Do not update `.spec`.
- If both the old and new objects have `.spec` and it is changed, set it back to its old value.
- If the old object has a `.spec` but the new object doesn't, set it to the old value.
- If old object did not have a `.spec` but the new object does, delete it.
- Do not update `.metadata`.
- In `ValidateStatusUpdate`:
- For CRD validation, validate only under `.status`.
- Validate value at `statusReplicasPath` as above. If `labelSelectorPath` is a path under `.status`, then validate it as well.
- Plug into the custom resource handler:
- Store all three storage - customResource, status and scale in `crdInfo`.
- Use the storage as per the subresource in the request.
- Use the validator as per the subresource (for status, only use the schema for `status`, if present).
- Serve the endpoint as per the subresource - see `serveResource`, `serveStatus` and `serveScale`.
- Update discovery by adding the `/status` and `/scale` resources, if enabled.
**Add tests**:
- Add unit tests in `etcd_test.go`.
- Add integration tests.
- In `subresources_test.go`, use the [polymporphic scale client](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/client-go/scale) to get and update `Scale`.
- Add a test to check everything works fine with yaml in `yaml_test.go`.
**Release note**:
```release-note
`/status` and `/scale` subresources are added for custom resources.
```
In https://github.com/kubernetes/kubernetes/issues/52255, it was
determined that if the user has a CDPATH, running `make` will fail. This
is because using the KUBE_ROOT variable results in invalid paths when
the CDPATH environment variable is set.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Bring PVC Protection feature to Beta
**What this PR does / why we need it**:
Brings PVC Protection feature to Beta.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # N/A
**Special notes for your reviewer**:
xref https://github.com/kubernetes/features/issues/498
**Release note**:
```release-note
PVC Protection alpha feature was renamed to Storage Protection. Storage Protection feature is beta.
```
Automatic merge from submit-queue (batch tested with PRs 58716, 59977, 59316, 59884, 60117). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
collapse printing paths
This collapses our printers into:
1. `PrintSuccess` - which was special cased before and should really just be a flavor of printer
2. `PrintObject` - prints an object using all the defaults. This should go away once we have options on all our commands.
3. `PrinterForOptions` - which gives back a printer based on the options. This should really be a method on the options themselves.
And none of these methods have any factory dependencies anymore.
@kubernetes/sig-cli-maintainers
@soltysh @juanvallejo You two have followed the work. Let's tie this off.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 58716, 59977, 59316, 59884, 60117). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove deprecated /proxy paths
These were deprecated in v1.2.
ref https://github.com/kubernetes/kubernetes/issues/59885
```release-note
kube-apiserver: the root /proxy paths have been removed (deprecated since v1.2). Use the /proxy subresources on objects that support HTTP proxying.
```
@kubernetes/sig-api-machinery-api-reviews
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
oidc authentication: switch to v2 of coreos/go-oidc
Switch to v2 of [coreos/go-oidc](https://github.com/coreos/go-oidc), which uses square/go-jose to verify tokens and supports more signing algorithms.
Most of this PR removes dependencies used by the older version of github.com/coreos/go-oidc, and updates vendor files.
This PR has been tested against tokens issued by Okta, Google, and CoreOS's dex.
Closes https://github.com/kubernetes/kubernetes/issues/57806
```release-note
kube-apiserver: the OpenID Connect authenticator can now verify ID Tokens signed with JOSE algorithms other than RS256 through the --oidc-signing-algs flag.
kube-apiserver: the OpenID Connect authenticator no longer accepts tokens from the Google v3 token APIs, users must switch to the "https://www.googleapis.com/oauth2/v4/token" endpoint.
```
cc @rithujohn191 @liggitt
cc @kubernetes/sig-auth-pr-reviews
PVCProtection feature was renamed to Storage Protection in:
https://github.com/kubernetes/kubernetes/pull/58743
That's why it's renamed when brought into beta.
In addition, StorageProtection feature is brought into beta in 1.10 release.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix golint warnings in daemon controller
This fixes the golint errors in the daemon controller package.
The only on remaining asks to rename DaemonSetsController, which is a public interface and would need proper deprecation first.
**Special notes for your reviewer**:
**Release note**:
I don't believe a release note is required. It only fixes code comments.
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Return information about which int tests failed in the summary
**What this PR does / why we need it**:
Currently if integration tests fail they will print the name of the failing tests as it goes, but the summary only tells that it failed. You need to know what to look for to be able to find all these occurrences. This PR provides a summary next to the failure information, eg:
```
FAILED TESTS: run_cluster_management_tests, run_plugins_tests,
```
Which should greatly simplify developers integration debugging.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add cblecker to vendor OWNERS
**What this PR does / why we need it**:
Adds myself to vendor OWNERS. I can help approve dep bumps of existing deps, and refer to Tim and new deps for license review.
**Release note**:
```release-note
NONE
```
/assign thockin
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update bazelbuild/rules_go, kubernetes/repo-infra, and gazelle dependencies
**What this PR does / why we need it**: updates our bazelbuild/rules_go dependency in order to bump everything to go1.9.4. I'm separating this effort into two separate PRs, since updating rules_go requires a large cleanup, removing an attribute from most build rules.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59398, 59881). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Pipe error message from openapi/swaggerspec verify checks to stderr
**What this PR does / why we need it**:
This pipes the error messages from openapi and swagger spec verify jobs to stderr so they show up in junit reports.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
CSI 0.2.0 breaking changes
Refactor kubernetes CSI bits to support CSI version 0.2.0
```release-note
Addressing breaking changes introduced by new 0.2.0 release of CSI spec
```
Automatic merge from submit-queue (batch tested with PRs 59683, 59964, 59841, 59936, 59686). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move ipvs module loading logic in local-cluster-up.sh
**What this PR does / why we need it**:
This PR makes the module loading logic for ipvs kube-proxy a little more robust. Previously we were attempting to load the modules and not checking that it succeeded. Now we make sure the loading was successful before proceeding with using ipvs as the proxier.
/assign @cblecker
Release Note
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 59353, 59905, 53833). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Graduate kubeletconfig API group to beta
Regarding https://github.com/kubernetes/features/issues/281, this PR moves the kubeletconfig API group to beta.
After #53088, the KubeletConfiguration type should not contain any deprecated or experimental fields, and we should not have to remove any more fields from the type before graduating it to beta.
We need the community to double check for two things, however:
1. Are there any fields currently in the KubeletConfiguration type that you were going to mark deprecated this quarter, but haven't yet?
2. Are there any fields currently in the KubeletConfiguration type that are experimental or alpha, but were not explicitly denoted as such?
Please comment on this PR if you can answer "yes" to either of those two questions. Please cc anyone with a stake in the kubeletconfig API, so we get as much coverage as possible.
/cc @thockin @dchen1107 @Random-Liu @yujuhong @dashpole @tallclair @vishh @abw @freehan @dnardo @bowei @MrHohn @luxas @liggitt @ncdc @derekwaynecarr @mikedanese
@kubernetes/sig-network-pr-reviews, @kubernetes/sig-node-pr-reviews
```release-note
action required: The `kubeletconfig` API group has graduated from alpha to beta, and the name has changed to `kubelet.config.k8s.io`. Please use `kubelet.config.k8s.io/v1beta1`, as `kubeletconfig/v1alpha1` is no longer available.
```
**TODO:**
- [x] Move experimental/non-gated-alpha/soon-to-be-deprecated fields to `KubeletFlags`
- [x] #53088
- [x] #54154
- [x] #54160
- [x] #55562
- [x] #55983
- [x] #57851
- [x] Lift embedded structure out of strings
- [x] #53025
- [x] #54643
- [x] #54823
- [x] #55254
- [x] Resolve relative paths against the location config files are loaded from
- [x] #55648
- [x] Rename to `kubelet.config.k8s.io`
- [x] Comments
- [x] Make sure existing comments at least read sensibly.
- [x] Note default values in comments on the versioned struct.
- [x] Remove any reference to default values in comments on the internal struct.
- [x] Most fields should be `+optional` and `omitempty`. Add where necessary. ~Where omitted, explicitly comment.~ Edit: We should not distinguish between nil and empty, see below items.
- [x] Ensure defaults are specified via `pkg/kubelet/apis/kubelet.config.k8s.io/v1beta1/defaults.go`, not `cmd/kubelet/app/options/options.go`.
- [x] #57770
- [x] Ensure kubeadm does not persist v1alpha1 KubeletConfiguration objects (or feature-gates this functionality)
- [x] Don't make a distinction between empty and nil, because of #43203.
- [x] #59515
- [x] #59681
- [x] Take the opportunity to fix insecure Kubelet defaults @tallclair
- [x] #59666
- [x] Remove CAdvisorPort from KubeletConfiguration wrt #56523.
- [x] #59580
- [x] Hide `ConfigTrialDuration` until we're more sure what to do with it.
- [x] #59628
- [x] Fix `// default: x` comments after rebasing on recent changes.
Automatic merge from submit-queue (batch tested with PRs 59877, 59886, 59892). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Partial revert of #59797
**What this PR does / why we need it**:
Need to do a partial revert of #59797. This PR introduced two problems:
- `INTERNAL_DIRS` has an unbound variable error
- The script introduces the use of mapfile which doesn't exist in bash versions prior to bash 4.x. Darwin (OS X) uses bash 3.x by default.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Standardize on KUBE_PROXY_MODE (not KUBEPROXY_MODE)
**What this PR does / why we need it**:
There's a disconnect between 12d4eac038
and the earlier a6af827898. One says
KUBEPROXY_MODE and another says KUBE_PROXY_MODE. Let's just pick one.
Currently we fail with the following error:
hack/local-up-cluster.sh: line 808: KUBEPROXY_MODE: unbound variable
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Create pkg/kubelet/apis/deviceplugin/v1beta1 directory.
The proto stays the same as v1alpha. Only changes Version in
constants.go to "v1beta1" and the BUILD file to pick up the new dir.
```release-note
Adding pkg/kubelet/apis/deviceplugin/v1beta1 API.
```
Automatic merge from submit-queue (batch tested with PRs 59878, 59852). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Check if netstat or iproute2 is available
**What this PR does / why we need it**:
In https://github.com/kubernetes/kubernetes/pull/59655, I incorrectly assumed that the kubetest image would have net-tools. It doesn't. This checks and uses `ss` from iproute2, *or* `netstat` from net-tools. The kubetest image does appear to have iproute2.
**Release note**:
```release-note
NONE
```
There's a disconnect between 12d4eac038
and the earlier a6af827898. One says
KUBEPROXY_MODE and another says KUBE_PROXY_MODE. Let's just pick one.
Currently we fail with the following error:
hack/local-up-cluster.sh: line 808: KUBEPROXY_MODE: unbound variable
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Build Kubernetes binaries with valid Semantic Version
**What this PR does / why we need it**:
Refuse to build Kubernetes when a version string like `v1.8.3+xx+xx`.
This PR is to restrict the `KUBE_GIT_VERSION`, if the version string does match the rule of Semantic Version, then refuse to build.
Since Kubernetes Conformance test needs the `KUBE_GIT_VERSION` to be a valid Semantic Version, so I think it's better to restrict the version string.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59532, 59685, 59797). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Continue fixing bash scripts
**What this PR does / why we need it**:
This PR continues the work of fixing bugs in shell scripts (see https://github.com/kubernetes/kubernetes/pull/59572 for the previous PR). The fixed scripts are `hack/grab-profiles.sh` and `hack/update-codegen.sh`. Most of the issues are found by using `shellcheck`. The goal is to make the scripts work and behave as they did previously, but fix obvious bugs and make them more robust and resilient for irregular input such as file names with special characters or whitespace in them.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Route verify script output to stderr
**What this PR does / why we need it**:
This changes a few of the review scripts that weren't routing their output to stderr, to route there error output there. This allows the junit report to pick up on the error message.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
sort of fixes#59518
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a new environment variable to the gce startup scripts called KUBE_PROXY_MODE
**What this PR does / why we need it**:
This PR adds a new environment variable called KUBE_PROXY_MODE to the startup scripts for gce. This variable will allow a user to specify the kube-proxy implementation they want to use, with the choices being ipvs or iptables (iptables is default).
Next steps:
1. Need to remove use of feature gateway when IPVS goes GA
2. Need to add logic of loading required ipvs kernel modules in the scripts
Question: If the proxier is IPVS, is it necessary to have the iptables sync period flags?
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove experimental keystone authenticator
**What this PR does / why we need it**:
experimental-keystone-url and experimental-keystone-ca-file were always
experimental. So we don't need a deprecation period.
KeystoneAuthenticator was on the server side and needed userid/password
to be passed in and used that to authenticate with Keystone. We now
have authentication and authorization web hooks that can be used. There
is a external repo with a webook for keystone which works fine along
with the kubectl auth provider that was added in:
a0cebcb559
So we don't need this older style / hard coded / experimental code
anymore.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
kube-apiserver: the experimental in-tree Keystone password authenticator has been removed in favor of extensions that enable use of Keystone tokens.
```
rename ALLOW_SECURITY_CONTEXT to DENY_SECURITY_CONTEXT_ADMISSION to be
in line with the other admission plugins (like PSP_ADMISSION). Make
sure by default, this plugin is not enabled as well.
Use "find -exec" instead of plain "find | xargs" to fix handling of
difficult file names (such as those containing spaces). Also, use
"mapfile" for creating the array from the output instead of letting the
shell split the results into the array. Add double quotes to places
where variable handling needs it to prevent splitting and globbing.
The current code
if ! [[ " ${INTERNAL_DIRS[@]:-} " =~ " ${int_group} " ]]; then
is broken because the array is concatenated in [[ .. ]] structure. This
means that the match will be done to any substring in the resulting
string which just happens to include ${int_group}. Fix this to use a
loop instead, and do exact matching. Also make tabs consistent in the
for loop.
Use "mapfile" and "read" to split the $KUBE_AVAILABLE_GROUP_VERSIONS
string into an array using space as the delimiter. This prevents shell
from globbing and splitting the string in potentially wrong places.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove /ui/ redirect
The existing kube-apiserver hard-codes `/ui` to redirect to an optional add-on, which is not appropriate. It does not work in the following scenarios:
* https-enabled dashboards
* the dashboard is deployed to a different namespace or service name
* the dashboard is not installed at all
* authorization is enabled and does not allow access to /ui
This PR removes the hard-coded `/ui` redirect.
```release-note
apiserver: the /ui kube-dashboard redirect has been removed. Follow instructions specific to your deployment to access kube-dashboard
```
Use double quotes around variables in places where they might be
used in globbing. Also replace an "echo | sed" construct with bash
variable substitution.
You can compare the variable substitution with this example:
$ addresses="foo;bar,test"
$ for token in $(echo $addresses | sed 's/[,;]/\n/g'); do echo "token: $token"; done
token: foo
token: bar
token: test
$ for token in ${addresses//[,;]/' '}; do echo "token: $token" ; done
token: foo
token: bar
token: test
The SSH_PID variable doesn't get expanded in the trap because of the
single quotes. Change to double quotes.
This is an example how the change works:
$ FOO="bar"
$ echo '$FOO'
$FOO
$ echo "$FOO"
bar
Ismo Puustinen