Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)
Enable RBAC by default in hack/local-up-cluster.sh
**What this PR does / why we need it**:
Since hack/local-up-cluster.sh checks https API port after commit 413ab26df9, RBAC should be enabled by default to avoid TLS port access denied.
**Which issue this PR fixes** : Fixes#49319
**Special notes for your reviewer**:
@liggitt
**Release note**:
```release-note
hack/local-up-cluster.sh now enables RBAC authorization by default
```
Automatic merge from submit-queue (batch tested with PRs 49316, 46117, 49064, 48073, 49323)
add e2e tests for the bootstrapsigner and tokencleaner controllers, integration testing for bootstrap token auth
**What this PR does / why we need it**:
Add e2e test for bootstrap signer
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
None
```
Since hack/local-up-cluster.sh checks https API port
after commit 413ab26df9,
RBAC should be enabled by default to avoid TLS port
access denied.
Fixes#49319
Automatic merge from submit-queue (batch tested with PRs 49114, 48810)
Unify fuzzers and roundtrip tests
- reorganize apimachinery/pkg/api/testing package to avoid circular imports
- make fuzzers modular per apigroup
- add roundtrip test for meta types
- add roundtrip test for a couple of staging apigroups
Follow-up:
- split `pkg/api` tests by apigroup
Automatic merge from submit-queue (batch tested with PRs 49083, 45540, 46862)
Respect KUBE_BUILD_PLATFORMS set by user
**What this PR does / why we need it**:
Currently the only(?) toggle available for custom arch builds is
KUBE_FASTBUILD. We should allow the user to specify a list
of arch(es) in the environment variable KUBE_BUILD_PLATFORMS.
Example:
KUBE_BUILD_PLATFORMS="linux/amd64 linux/arm64" hack/build-cross.sh
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#20365
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49083, 45540, 46862)
update golang version to go1.8
Signed-off-by: sakeven <jc5930@sina.cn>
**What this PR does / why we need it**:
There are some code breaking the compatibility with golang 1.6. Now kubernetes only support golang 1.8.
We need to update golang version check script.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49082
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48377, 48940, 49144, 49062, 49148)
fixit: break sig-cluster-lifecycle tests into subpackage
this is part of fixit week. ref #49161
@kubernetes/sig-cluster-lifecycle-misc
Automatic merge from submit-queue
Add PriorityClass API object under new "scheduling" API group
**What this PR does / why we need it**: This PR is a part of a series of PRs to add pod priority to Kubernetes. This PR adds a new API group called "scheduling" with a new API object called "PriorityClass". PriorityClass maps the string value of priority to its integer value.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: Given the size of this PR, I will add the admission controller for the PriorityClass in a separate PR.
**Release note**:
```release-note
Add PriorityClass API object under new "scheduling" API group
```
ref/ #47604
ref/ #48646
Currently the only(?) toggle available for custom arch builds is
KUBE_FASTBUILD. We should allow the user to specify a list
of arch(es) in the environment variable KUBE_BUILD_PLATFORMS.
Example:
KUBE_BUILD_PLATFORMS="linux/amd64 linux/arm64" hack/build-cross.sh
Fixes#20365
Automatic merge from submit-queue
Switch from gazel to kazel
**What this PR does / why we need it**: switches to `kazel`, where all new development will be focused. Part of the effort for switching from `gazel` to `gazelle`.
I'm imagining that we'll use `kube::util::go_install_from_commit` to version `gazelle` as well.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: x-ref #47558
**Release note**:
```release-note
NONE
```
/release-note-none
/assign @mikedanese @spxtr
Automatic merge from submit-queue (batch tested with PRs 49058, 49072, 49137, 49182, 49045)
*: remove --insecure-allow-any-token option
~Since the authenticator is still used in e2e tests, don't remove
the actual package. Maybe a follow up?~
edit: e2e and integration tests have been switched over to the tokenfile
authenticator instead.
```release-note
The --insecure-allow-any-token flag has been removed from kube-apiserver. Users of the flag should use impersonation headers instead for debugging.
```
closes#49031
cc @kubernetes/sig-auth-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 49058, 49072, 49137, 49182, 49045)
use https to check healthz in hack/local-up-cluster.sh
**What this PR does / why we need it**:
```
# PSP_ADMISSION=true ALLOW_PRIVILEGED=true ALLOW_SECURITY_CONTEXT=true ALLOW_ANY_TOKEN=true ENABLE_RBAC=true RUNTIME_CONFIG="extensions/v1beta1=true,extensions/v1beta1/podsecuritypolicy=true" hack/local-up-cluster.sh
...
Waiting for apiserver to come up
+++ [0718 09:34:38] On try 5, apiserver: :
Cluster "local-up-cluster" set.
use 'kubectl --kubeconfig=/var/run/kubernetes/admin-kube-aggregator.kubeconfig' to use the aggregated API server
Creating kube-system namespace
clusterrolebinding "system:kube-dns" created
serviceaccount "kube-dns" created
configmap "kube-dns" created
error: unable to recognize "kubedns-deployment.yaml": no matches for extensions/, Kind=Deployment
service "kube-dns" created
Kube-dns deployment and service successfully deployed.
kubelet ( 10952 ) is running.
Create podsecuritypolicy policies for RBAC.
unable to recognize "/home/nfs/mygo/src/k8s.io/kubernetes/examples/podsecuritypolicy/rbac/policies.yaml": no matches for extensions/, Kind=PodSecurityPolicy
unable to recognize "/home/nfs/mygo/src/k8s.io/kubernetes/examples/podsecuritypolicy/rbac/policies.yaml": no matches for extensions/, Kind=PodSecurityPolicy
unable to recognize "/home/nfs/mygo/src/k8s.io/kubernetes/examples/podsecuritypolicy/rbac/roles.yaml": no matches for rbac.authorization.k8s.io/, Kind=ClusterRole
unable to recognize "/home/nfs/mygo/src/k8s.io/kubernetes/examples/podsecuritypolicy/rbac/roles.yaml": no matches for rbac.authorization.k8s.io/, Kind=ClusterRole
unable to recognize "/home/nfs/mygo/src/k8s.io/kubernetes/examples/podsecuritypolicy/rbac/bindings.yaml": no matches for rbac.authorization.k8s.io/, Kind=ClusterRoleBinding
unable to recognize "/home/nfs/mygo/src/k8s.io/kubernetes/examples/podsecuritypolicy/rbac/bindings.yaml": no matches for rbac.authorization.k8s.io/, Kind=ClusterRoleBinding
unable to recognize "/home/nfs/mygo/src/k8s.io/kubernetes/examples/podsecuritypolicy/rbac/bindings.yaml": no matches for rbac.authorization.k8s.io/, Kind=ClusterRoleBinding
Create default storage class for
error: unable to recognize "/home/nfs/mygo/src/k8s.io/kubernetes/cluster/addons/storage-class/local/default.yaml": no matches for storage.k8s.io/, Kind=StorageClass
Local Kubernetes cluster is running. Press Ctrl-C to shut it down.
Logs:
/tmp/kube-apiserver.log
/tmp/kube-controller-manager.log
/tmp/kube-proxy.log
/tmp/kube-scheduler.log
/tmp/kubelet.log
...
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47739
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 49116, 49095)
Move pkg/api/v1/ref -> client-go/tools/reference
`pkg/api/v1/ref` is the only remaining package copied from pkg/api/v1 to client-go via staging/copy.sh.
Automatic merge from submit-queue (batch tested with PRs 48481, 48256)
Refactor: pkg/util into sub-pkgs
**What this PR does / why we need it**:
- move code in pkg/util into sub-pkgs
- delete some unused funcs
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#15634
**Special notes for your reviewer**:
This is the final work of #15634. It will close that issue.
/cc @thockin
**Release note**:
```release-note
NONE
```
Add PriorityClass to pkg/registry
Add PriorityClass to pkg/master/master.go
Add PriorityClass to import_know_versions.go
Update linted packages
minor fix
e2e and integration tests have been switched over to the tokenfile
authenticator instead.
```release-note
The --insecure-allow-any-token flag has been removed from kube-apiserver. Users of the flag should use impersonation headers instead for debugging.
```
Automatic merge from submit-queue (batch tested with PRs 49043, 49001, 49057, 49066, 48102)
Make doc generation on cherry-picks optional
Follow up of https://github.com/kubernetes/kubernetes/pull/46993, xref #44533
Most cherry-picks don't require doc changes, and doing it unconditionally actually breaks picking to the release-1.6 branch
Automatic merge from submit-queue (batch tested with PRs 49043, 49001, 49057, 49066, 48102)
cluster/gke: If NODE_INSTANCE_GROUP is set, don't execute any bash
Transitional part of kubernetes/test-infra#3307, should be eliminated by kubernetes/test-infra#3330:
Allow NODE_INSTANCE_GROUP to be set externally from `hack/ginkgo-e2e.sh`, which eliminates any cluster/gke use if KUBERNETES_CONFORMANCE_PROVIDER is set to `gke`.
```release-note
NONE
```
Transitional part of kubernetes/test-infra#3307, should be eliminated
by kubernetes/test-infra#3330: Allow NODE_INSTANCE_GROUP to be set
before we get here, which eliminates any cluster/gke use if
KUBERNETES_CONFORMANCE_PROVIDER is set to "gke".
Automatic merge from submit-queue (batch tested with PRs 48494, 48733)
Move test-webserver from contrib/for-demos to kubernetes/test/images
**What this PR does / why we need it**:
This PR is for
- Moving the https://github.com/kubernetes/contrib/tree/master/for-demos/test-webserver to kubernetes/test/images - Refer https://github.com/kubernetes/contrib/pull/2544 for more information
- Multi architecture support for test-webserver image
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 46091, 48280)
allow output patch string in edit command
**What this PR does / why we need it**:
allow user to get the patch from edit command if user is not familiar with the patch format.
```
# ./cluster/kubectl.sh create role a --verb=get,list --resource=no
role "a" created
# ./cluster/kubectl.sh edit role a --output-patch=true
Patch: {"rules":[{"apiGroups":[""],"resources":["nodes"],"verbs":["get","list","delete"]}]}
role "a" edited
# ./cluster/kubectl.sh create role b --verb=get,list --resource=no
role "b" created
# ./cluster/kubectl.sh patch role b -p '{"rules":[{"apiGroups":[""],"resources":["nodes"],"verbs":["get","list","delete"]}]}'
role "b" patched
```
**Which issue this PR fixes**: fixes#47173
**Special notes for your reviewer**:
**Release note**:
```release-note
Could get the patch from kubectl edit command
```
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)
Support customized system spec in the node conformance test and create the GKE system spec
ref: https://github.com/kubernetes/kubernetes/issues/46891
- System specs are located in `test/e2e_node/system/specs`. Created one for validating GKE images in `test/e2e_node/system/specs/gke.yaml`.
- `--image-spec-name` can be used to specify a system spec in node e2e and conformance tests. This option maps to `SYSTEM_SPEC_NAME` in a test properties file, which is the user facing configuration. So, users can specify `SYSTEM_SPEC_NAME=gke` to run the image validation using the GKE system spec.
- If `SYSTEM_SPEC_NAME` is unspecified, the default spec (`system.DefaultSysSpec`) will be used.
- We can also use `make test-e2e-node SYSTEM_SPEC_NAME=gke` to run tests using GKE image spec.
**Release note**:
`None`
Automatic merge from submit-queue
Add quick-verify make rule.
This is useful for humans to run to catch obvious problems before
pushing commits and waiting for CI to run verify checks.
Quick mode only runs a whitelist of verify scripts that are reasonably fast.
I set the initial bar arbitrarily at <10s each on my workstation.
The whole set runs in <30s for me, assuming I had already run `make` and
`hack/godep-restore.sh`. This is compared to the full `make verify`
which takes [I don't know how long because I gave up after 45min].
Automatic merge from submit-queue (batch tested with PRs 47066, 48892, 48933, 48854, 48894)
azure: msi: add managed identity field, logic
**What this PR does / why we need it**: Enables managed service identity support for the Azure cloudprovider. "Managed Service Identity" allows us to ask the Azure Compute infra to provision an identity for the VM. Users can then retrieve the identity and assign it RBAC permissions to talk to Azure ARM APIs for the purpose of the cloudprovider needs.
Per the commit text:
```
The azure cloudprovider will now use the Managed Service Identity
to retrieve access tokens for the Azure ARM APIs, rather than
requiring hard-coded, user-specified credentials.
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: n/a
**Special notes for your reviewer**: none
**Release note**:
```release-note
azure: support retrieving access tokens via managed identity extension
```
cc: @brendandburns @jdumars @anhowe
Automatic merge from submit-queue
Azure PD (Managed/Blob)
This is exactly the same code as this [PR](https://github.com/kubernetes/kubernetes/pull/41950). It has a clean set of generated items. We created a separate PR to accelerate the accept/merge the PR
CC @colemickens
CC @brendandburns
**What this PR does / why we need it**:
1. Adds K8S support for Azure Managed Disks.
2. Adds support for dedicated blob disks (1:1 to storage account) in addition to shared blob disks (n:1 to storage account).
3. Automatically manages the underlying storage accounts. New storage accounts are created at 50% utilization. Max is 100 disks, 60 disks per storage account.
2. Addresses the current issues with Blob Disks:
..* Significantly faster attach process. Disks are now usually available for pods on nodes under 30 sec if formatted, under a min if not formatted.
..* Adds support to move disks between nodes.
..* Adds consistent attach/detach behavior, checks if the disk is leased/attached on a different node before attempting to attach to target nodes.
..* Fixes a random hang behavior on Azure VMs during mount/format (for both blob + managed disks).
..* Fixes a potential conflict by avoiding the use of disk names for mount paths. The new plugin uses hashed disk uri for mount path.
The existing AzureDisk is used as is. Additional "kind" property was added allowing the user to decide if the pd will be shared, dedicated or managed (Azure Managed Disks are used).
Due to the change in mounting paths, existing PDs need to be recreated as PV or PVCs on the new plugin.
Automatic merge from submit-queue (batch tested with PRs 48864, 48651, 47703)
Enable logexporter mechanism to dump logs from k8s nodes to GCS directly
Ref https://github.com/kubernetes/kubernetes/issues/48513
This adds support for logexporter from k8s side. Next I'll send a PR adding support from test-infra side.
/cc @kubernetes/sig-scalability-misc @kubernetes/test-infra-maintainers @fejta @wojtek-t @gmarek
Automatic merge from submit-queue
add dockershim checkpoint node e2e test
Add a bunch of disruptive cases to test kubelet/dockershim's checkpoint work flow.
Some steps are quite hacky. Not sure if there is better ways to do things.
This is useful for humans to run to catch obvious problems before
pushing commits and waiting for CI to run verify checks.
Quick mode only runs a whitelist of verify scripts that are reasonably fast.
I set the initial bar arbitrarily at <10s each on my workstation.
The whole set runs in <30s for me, assuming I had already run `make` and
`hack/godep-restore.sh`. This is compared to the full `make verify`
which takes [I don't know how long because I gave up after 45min].
Automatic merge from submit-queue (batch tested with PRs 48594, 47042, 48801, 48641, 48243)
Prepare to introduce websockets for exec and portforward
Refactor the code in remotecommand to better represent the structure of
what is common between portforward and exec.
Ref #48633
Automatic merge from submit-queue (batch tested with PRs 48279, 48566, 48319, 48794, 47952)
Removed mesos flags from known-flags.txt.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
none
```
Automatic merge from submit-queue (batch tested with PRs 48698, 48712, 48516, 48734, 48735)
Name change: s/timstclair/tallclair/
I changed my name, and I'm migrating my user name to be consistent.
Automatic merge from submit-queue (batch tested with PRs 47948, 48631, 48693, 48549, 47593)
add generated clients. modify codegen script
**What this PR does / why we need it**:
Adds in the generated clientsets for the sample apiserver. Modifies the update-codgen script to copy over the client from the generated location. We need it in order to be able to add integrations and unit tests that make use of the clientsets and their fakes.
**Special notes for your reviewer**:
@deads2k @p0lyn0mial hopefully done the correct thing here. Not 100% sure on needing the copy but it seemed what was needed to get the import paths correct?
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
kube-apiserver: tests for aggregation and CRDs via delegation
In our integration tests we do not use the real kube-apiserver setup code, but mock our own. Here I use the actual `cmd/kube-apiserver/app.Run()` func with an testing etcd server. This can test the whole delegation chain of aggregator, apiextensions and kube-apiserver.
Automatic merge from submit-queue (batch tested with PRs 47232, 48625, 48613, 48567, 39173)
Make kube-proxy's MetricsBindAddress configurable via flag
**What this PR does / why we need it**: From #48600, `MetricsBindAddress` used to be configurable through flag (coupled with `HealthzBindAddress`). But this functionality went away after #44968 separated healthz server from metrics server. This PR adds that functionality back.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:
/assign @ncdc
cc @bowei @r0bj
**Release note**:
```release-note
none
```
Automatic merge from submit-queue (batch tested with PRs 48196, 42783, 48507, 47719, 46138)
fix parse resource in setting selector
**What this PR does / why we need it**:
**Which issue this PR fixes**: fixes#47718
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
fix issue where binaries not being removed. Add new client packages to lint
put new linted packages in correct location. remove flag from variables
Add all pkgs that should be linted and run sort
Add output from hack/update-bazel.sh
remove genclient=true from FischerList and regenerate client
re update bazel build files
fix missing resource method in register.go
Automatic merge from submit-queue (batch tested with PRs 47040, 48597, 48608, 48653)
Fix godep verify to use godep restore script
**What this PR does / why we need it**:
A bug was introduced in #48615. `hack/verify-godeps.sh` only downloads and compares if godeps have changed, so it wasn't caught on the original PR. However, when it does run (e.g. https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/48630/pull-kubernetes-verify/38350/) it fails because the godep-save script now doesn't permit a compex GOPATH. verify-godeps.sh actually sets one because it restores godeps not using the `hack/godep-restore.sh` script.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @sttts
/priority failing-test
Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)
Move pkg/apimachinery/test to apimachinery
Fixes https://github.com/kubernetes/kubernetes/issues/48265
for circular dependency reason, all the test file contain both `"k8s.io/apimachinery/pkg/apis/meta/v1"` and `"k8s.io/apimachinery/pkg/runtime"` is hard to move to a ideal location, so I create a separated test package for those files
I also bump the example package :https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/apiserver/pkg/apis/example here for apimachinery, @sttts told me it's ok if to bump file if only for test EDIT: seems it's no need to bump, will update soon
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47234, 48410, 48514, 48529, 48348)
Remove unused sub-pkgs in pkg/util
**What this PR does / why we need it**:
Remove no longer used sug-pkgs in pkg/util
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48386
**Special notes for your reviewer**:
/cc @deads2k
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47435, 46044)
kubeadm: Make self-hosting work and split it out to a phase
**What this PR does / why we need it**:
- Removes the old self-hosting code
- Puts the new self-hosting code in `phases/selfhosting`
- The new code reads manifests from disk (static pods)...
- ...mutates the PodSpec as necessary...
- ...and posts the DaemonSet to the API Server...
- ...and waits for it to come up
- Uses DaemonSets for all control plane components
- Creates a `kubeadm alpha phase selfhosting` command that can be invoked against any kubeadm-cluster after install.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes: https://github.com/kubernetes/kubeadm/issues/127
(large part of at least)
**Special notes for your reviewer**:
Please only review the fourth commit, based on https://github.com/kubernetes/kubernetes/pull/47345
**Release note**:
```release-note
kubeadm: Make self-hosting work by using DaemonSets and split it out to a phase that can be invoked via the CLI
```
@kubernetes/sig-cluster-lifecycle-pr-reviews @jbeda
Automatic merge from submit-queue (batch tested with PRs 48309, 48489)
Check if golint exists first in hack/verify-golint.sh
**What this PR does / why we need it**:
Check if golint exists first in hack/verify-golint.sh
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48488
**Special notes for your reviewer**:
nope.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48480, 48353)
remove tpr api access
xref https://github.com/kubernetes/kubernetes/issues/48152
TPR tentacles go pretty deep. This gets us started by removing API access and we'll move down from there.
@kubernetes/sig-api-machinery-misc
@ironcladlou this should free up the GC implementation since TPRs will no longer be present and failing.
```release-note
Removing TPR api access per https://github.com/kubernetes/kubernetes/issues/48152
```
Automatic merge from submit-queue (batch tested with PRs 47918, 47964, 48151, 47881, 48299)
Add ApiEndpoint support to GCE config.
**What this PR does / why we need it**:
Add the ability to change ApiEndpoint for GCE.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 48295, 48298, 47339, 44910, 48037)
Make the `--controllers` flag configurable in hack/local-up-cluster.sh
**What this PR does / why we need it**:
add options to enable tokencleaner,bootstrapsigner controller for bootstrap token testing
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 48295, 48298, 47339, 44910, 48037)
Make Makefiles in `test/images/` compatible with multiple architectures
**What this PR does / why we need it**:
This PR is for making test images multi architecture for different platforms like amd64, arm, arm64, ppc64le
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #31331
**Special notes for your reviewer**:
- Actual tests need to be modified to use these images based on the architecture later.
- Not covering the cross building of docker images for `s390x` platform due to problem faced while running containers with `qemu-s390x-static`
- Will submit separate PR for `volume and pet` test images
- This PR depends on - https://github.com/kubernetes/ingress/pull/587
**Release note**:
```NONE```
Automatic merge from submit-queue (batch tested with PRs 46850, 47984)
Enable vmodule support for all components
Support vmodule (via `LOG_SPEC`) for all local cluster components.
```release-note
NONE
```
When $GO_OUT is not set, line 152 will output the error:
[: ==: unary operator expected. This occurs because the if condition becomes
if [ == "" ]. This results in an error because == is a binary operator.
Automatic merge from submit-queue (batch tested with PRs 47650, 47936, 47939, 47986, 48006)
External dependency of k8s.io/api
Fix https://github.com/kubernetes/kubernetes/issues/48007
It's unfortunate that k8s.io/api has external dependencies.
Most of the dependencies are introduced by "k8s.io/apimachinery/pkg/util/intstr" and ugorji.
Automatic merge from submit-queue
correct the name of the script to generate swagger doc
**What this PR does / why we need it**: The name of the script to generate swagger doc is not correct, this PR is to fix it.
**Which issue this PR fixes**: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47921, 45984, 46829, 46896, 46923)
Remove unnecessary wrapper flags
**What this PR does / why we need it**:
Drop KUBE_GOFLAGS, KUBE_GOGCFLAGS, KUBE_GOLDFLAGS references
from the build infrastructure
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47296
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47921, 45984, 46829, 46896, 46923)
Create output_dir if doesn't exist
**What this PR does / why we need it**:
Minor fix to the swagger spec gen, that if the directory doesn't exist, create it before copying.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47921, 45984, 46829, 46896, 46923)
Move govet verify into rest of verify*-.sh scripts
**What this PR does / why we need it**:
Instead of having two govet scripts, consolidate them to into one and have both the Makefile and verify.sh scripts target the same script. This also will allow proper syntax highlighting and timing when the vet script is run as part of `make verify`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/kubernetes/test-infra/issues/2725
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/cc @fejta @rmmh
Automatic merge from submit-queue
Give developer more help info when running integration tests without etcd found.
**What this PR does / why we need it**:
When running integration tests, if etcd version on env is lower than expected, developer will get info `You can use 'hack/install-etcd.sh' to install a copy in third_party/.` It would be better to have the same info if no etcd installed on env.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Don't revert, change boundingdirs in Makefile for deepcopy-gen
manually fix pkg/client/clientset_generated/clientset/typed/core/v1/pod_expansion.go because external policy types are moved now
Automatic merge from submit-queue
Regenerate docs (if necessary) during cherry-pick operations.
This change ensures that cherrypicks are coupled with any related doc generation required.
closes#44533
This commit ensures that:
- go vet will be run as part of the make verify target
- the vet make-rule script won't be run directly, as generated_files won't be run in that case
- that go vet errors show up in the build log with a start time, finish time, and SUCCESS/FAILED message as part of the verify make rule script
Automatic merge from submit-queue
Fixed the issue with formatting issues not being reported when verification fails
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
If there are 2 or more PR(s) in the queue, they will end up with
conflicts (and rechecks). So let us remove the timestamp entirely
when we generate the files.
Fixes#46814
Automatic merge from submit-queue (batch tested with PRs 46327, 47166)
mark --network-plugin-dir deprecated for kubelet
**What this PR does / why we need it**:
**Which issue this PR fixes** : fixes#43967
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47530, 47679)
Use cos-stable-59-9460-64-0 instead of cos-beta-59-9460-20-0.
Remove dead code that has now moved to another repo as part of #47467
**Release note**:
```release-note
NONE
```
/sig node
Automatic merge from submit-queue
clientgo/examples: add ToC for examples
Also add authenticate- prefix to auth samples. This patch could use some
improvement explaining workqueue and TPR examples as I'm not entirely sure.
/assign @caesarxuchao
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
- It contains a fix for ipaliasing.
- It contains a fix which decouples GPU driver installation from kernel
version.
Remove dead code that has now moved to another repo as part of #47467
Automatic merge from submit-queue
fix --local flag for kubectl commands
Fixes https://github.com/kubernetes/kubernetes/issues/47079
**Release note**:
```release-note
NONE
```
Fixes the `--local` flag for `kubectl set ...` sub-commands.
**As of the 1.7 release**, `PrinterForCommand` was updated to [use a mapper and typer for unstructured objects](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/util/factory_builder.go#L52), which further prevented the use of `--local` when there was no connection to an api server.
**before** (with no connection to a server)
```
$ kubectl set resources -f pod.json --limits=cpu=200m,memory=512Mi --local
error: unable to connect to a server to handle "pods": Get https://10.13.137.149:8443/api: dial tcp 10.13.137.149:8443: getsockopt: connection refused
```
**after** (with no connection to a server)
```
$ kubectl set resources -f pod.json --limits=cpu=200m,memory=512Mi --local
NAME READY STATUS RESTARTS AGE
mypod 0/1 0 <unknown>
```
cc @smarterclayton @fabianofranz
Automatic merge from submit-queue (batch tested with PRs 47492, 47542, 46800, 47545, 45764)
Fix TPR watches.
Fixes#47027
TPR watch has been broken since #44350.
Automatic merge from submit-queue (batch tested with PRs 47204, 46808, 47432, 47400, 47099)
support junit in test-cmd
Output junit result for cmd tests.
Fixes#45196
There will be a followup PR to refactor the cmd test into functions as test cases there is not a lot changes in `hack/make-rules/test-cmd-util.sh`
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47204, 46808, 47432, 47400, 47099)
Make kubectl apply add change-cause before patching
**What this PR does / why we need it**: We shouldn't patch the project with applied change, and then patch again with the change cause. Otherwise, DaemonSet change cause wouldn't be copied to its history (after the first patch, history will be created with the old change cause).
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47210
**Special notes for your reviewer**:
/assign @mengqiy
@kubernetes/sig-apps-bugs @kubernetes/sig-cli-maintainers
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47470, 47260, 47411, 46852, 46135)
Write reports for each upgrade test
Due to the way Ginkgo runs individual test cases and the level of coordination required for the upgrade tests, they were all run under a single Ginkgo test case. This PR generates and auxiliary report that break out the results of each upgrade test. This is accomplished by:
1) Wrapping `ginkgo.Fail` and `ginkgo.Skip` to get the actual failure or skip messages.
2) Recovering that info in the upgrade test to generate an auxiliary report.
I suggest reviewing commit by commit.
Sample report: https://storage.googleapis.com/krouseytestreports/logs/results/1/artifacts/junit_upgrades.xmlFixes: #47371
Automatic merge from submit-queue (batch tested with PRs 46441, 43987, 46921, 46823, 47276)
Azure plugin for client auth
This is an Azure Active Directory plugin for client authentification. It provides an integration with Azure CLI 2.0 login command. It can also be used standalone, in that case it will use the device code flow to acquire an access token.
More details are provided in the README.md file.
https://github.com/kubernetes/kubectl/issues/29
cc @brendandburns @colemickens
Automatic merge from submit-queue
Change what is stored in DaemonSet history `.data`
**What this PR does / why we need it**:
In DaemonSet history `.data`, store a strategic merge patch that can be applied to restore a DaemonSet. Only PodSpecTemplate is saved.
This will become consistent with the data stored in StatefulSet history.
Before this fix, a serialized pod template is stored in `.data`; however, seriazlized pod template isn't a `runtime.RawExtension`, and caused problems when controllers try to patch the history's controller ref.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47008
**Special notes for your reviewer**: @kubernetes/sig-apps-bugs @erictune @kow3ns @kargakis @lukaszo @mengqiy
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
bazel: stamp multiple packages by using x_defs instead of linkstamp in go_binary rules
**What this PR does / why we need it**: Fixes regression introduced sometime in the last few months that prevented bazel-built clusters from identifying version properly.
It does so by updating the bazelbuild/rules_go and kubernetes/repo-infra dependencies to support using stamp values in `go_binary` `x_defs`, and then changing our `go_binary` rules to use `x_defs` instead of `linkstamp`.
This whole charade is necessary because we need to stamp version information in multiple packages.
This pretty much only affects the bazel build, so it should be low risk.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45298
**Special notes for your reviewer**: depends on https://github.com/kubernetes/repo-infra/pull/18; should not be merged before it.
**Release note**:
```release-note
NONE
```
/assign @spxtr @mikedanese
Automatic merge from submit-queue (batch tested with PRs 46979, 47078, 47138, 46916)
Add a secretbox and AES-CBC path for encrypt at rest
Add a secretbox and AES-CBC encrypt at rest provider and alter the config, based on feedback from security review. AES-CBC is more well reviewed and generally fits better with common criteria and FIPS, secretbox is newer and faster than CBC.
```release-note
Add secretbox and AES-CBC encryption modes to at rest encryption. AES-CBC is considered superior to AES-GCM because it is resistant to nonce-reuse attacks, and secretbox uses Poly1305 and XSalsa20.
```
Automatic merge from submit-queue (batch tested with PRs 47024, 47050, 47086, 47081, 47013)
apiextensions-apiserver: Fix decoding of DeleteOptions.
Fixes#47072 by making apiextensions-apiserver capable of decoding unversioned DeleteOptions, rather than only handling Unstructured objects (i.e. Custom Resources).
This also closes#46736 and #37554 since the added regression test works for TPR as well.
Automatic merge from submit-queue (batch tested with PRs 47024, 47050, 47086, 47081, 47013)
client-go: deprecate TPR example and add CRD example
/cc @nilebox
Part of https://github.com/kubernetes/kubernetes/issues/46702
Drop KUBE_GOFLAGS, KUBE_GOGCFLAGS, KUBE_GOLDFLAGS references
from the build infrastructure. There are some usages still
for KUBE_GOFLAGS, so we should print a deprecation message
until all those are fixed. If both KUBE_GOFLAGS and GOFLAGS
are just then we just bail out.
Automatic merge from submit-queue (batch tested with PRs 46235, 44786, 46833, 46756, 46669)
implements StatefulSet update
**What this PR does / why we need it**:
1. Implements rolling update for StatefulSets
2. Implements controller history for StatefulSets.
3. Makes StatefulSet status reporting consistent with DaemonSet and ReplicaSet.
https://github.com/kubernetes/features/issues/188
**Special notes for your reviewer**:
**Release note**:
```release-note
Implements rolling update for StatefulSets. Updates can be performed using the RollingUpdate, Paritioned, or OnDelete strategies. OnDelete implements the manual behavior from 1.6. status now tracks
replicas, readyReplicas, currentReplicas, and updatedReplicas. The semantics of replicas is now consistent with DaemonSet and ReplicaSet, and readyReplicas has the semantics that replicas did prior to this release.
```
Automatic merge from submit-queue (batch tested with PRs 46235, 44786, 46833, 46756, 46669)
federation: Add admission controller for policy-based placement
@nikhiljindal
Here's the initial version of the scheduling policy admission controller. It's at the point where it would benefit from having another pair of eyes look at it. The main thing I'm unsure of is the serialization of Kube resources for the webhook/query call.
Release Note:
```
The federation-apiserver now supports a SchedulingPolicy admission controller that enables policy-based control over placement of federated resources.
```
Ref #39982
Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)
Add group alias names to API resources to allow discovery
**What this PR does / why we need it**:
Adds `GroupNames []string` to API resources, which represents the list of group aliases that every resource belongs to.
Partially fixes https://github.com/kubernetes/kubernetes/issues/41353
This moves the logic of "all" (which currently [translates](fbc94c0896/pkg/kubectl/cmd/util/shortcut_restmapper.go (L106)) to "pods,replicationcontrollers,services,...") to the server-side. Will allow clients like `kubectl` to discover group aliases instead of having it hardcoded and the API server to better handle consistency across multiple clients, version skew, etc; and will make "all" un-special and allow other groups to be created.
As a follow-up we'll patch `kubectl` to make groups aliases discoverable and the hardcoded list a fallback while we still have to support it.
Related to https://github.com/kubernetes/kubernetes/pull/42595#issuecomment-286839885.
**Release note**:
```release-note
Adds the `Categories []string` field to API resources, which represents the list of group aliases (e.g. "all") that every resource belongs to.
```
@kubernetes/sig-api-machinery-misc @deads2k @bgrant0607
Automatic merge from submit-queue (batch tested with PRs 46972, 42829, 46799, 46802, 46844)
promote tls-bootstrap to beta
last commit of this PR.
Towards https://github.com/kubernetes/kubernetes/issues/46999
```release-note
Promote kubelet tls bootstrap to beta. Add a non-experimental flag to use it and deprecate the old flag.
```
Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)
Add configuration for encryption providers
## Additions
Allows providing a configuration file (using flag `--experimental-encryption-provider-config`) to use the existing AEAD transformer (with multiple keys) by composing mutable transformer, prefix transformer (for parsing providerId), another prefix transformer (for parsing keyId), and AES-GCM transformers (one for each key). Multiple providers can be configured using the configuration file.
Example configuration:
```
kind: EncryptionConfig
apiVersion: v1
resources:
- resources:
- namespaces
providers:
- aes:
keys:
- name: key1
secret: c2vjcmv0iglzihnly3vyzq==
- name: key2
secret: dghpcybpcybwyxnzd29yza==
- identity: {}
```
Need for configuration discussed in:
#41939
[Encryption](3418b4e4c6/contributors/design-proposals/encryption.md)
**Pathway of a read/write request**:
1. MutableTransformer
2. PrefixTransformer reads the provider-id, and passes the request further if that matches.
3. PrefixTransformer reads the key-id, and passes the request further if that matches.
4. GCMTransformer tries decrypting and authenticating the cipher text in case of reads. Similarly for writes.
## Caveats
1. To keep the command line parameter parsing independent of the individual transformer's configuration, we need to convert the configuration to an `interface{}` and manually parse it in the transformer. Suggestions on better ways to do this are welcome.
2. Flags `--encryption-provider` and `--encrypt-resource` (both mentioned in [this document](3418b4e4c6/contributors/design-proposals/encryption.md) ) are not supported in this because they do not allow more than one provider, and the current format for the configuration file possibly supersedes their functionality.
3. Currently, it can be tested by adding `--experimental-encryption-provider-config=config.yml` to `hack/local-up-cluster.sh` on line 511, and placing the above configuration in `config.yml` in the root project directory.
Previous discussion on these changes:
https://github.com/sakshamsharma/kubernetes/pull/1
@jcbsmpsn @destijl @smarterclayton
## TODO
1. Investigate if we need to store keys on disk (per [encryption.md](3418b4e4c6/contributors/design-proposals/encryption.md (option-1-simple-list-of-keys-on-disk)))
2. Look at [alpha flag conventions](https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go)
3. Need to reserve `k8s:enc` prefix formally for encrypted data. Else find a better way to detect transformed data.
Automatic merge from submit-queue
Add initializer support to admission and uninitialized filtering to rest storage
Initializers are the opposite of finalizers - they allow API clients to react to object creation and populate fields prior to other clients seeing them.
High level description:
1. Add `metadata.initializers` field to all objects
2. By default, filter objects with > 0 initializers from LIST and WATCH to preserve legacy client behavior (known as partially-initialized objects)
3. Add an admission controller that populates .initializer values per type, and denies mutation of initializers except by certain privilege levels (you must have the `initialize` verb on a resource)
4. Allow partially-initialized objects to be viewed via LIST and WATCH for initializer types
5. When creating objects, the object is "held" by the server until the initializers list is empty
6. Allow some creators to bypass initialization (set initializers to `[]`), or to have the result returned immediately when the object is created.
The code here should be backwards compatible for all clients because they do not see partially initialized objects unless they GET the resource directly. The watch cache makes checking for partially initialized objects cheap. Some reflectors may need to change to ask for partially-initialized objects.
```release-note
Kubernetes resources, when the `Initializers` admission controller is enabled, can be initialized (defaulting or other additive functions) by other agents in the system prior to those resources being visible to other clients. An initialized resource is not visible to clients unless they request (for get, list, or watch) to see uninitialized resources with the `?includeUninitialized=true` query parameter. Once the initializers have completed the resource is then visible. Clients must have the the ability to perform the `initialize` action on a resource in order to modify it prior to initialization being completed.
```
Automatic merge from submit-queue (batch tested with PRs 46239, 46627, 46346, 46388, 46524)
Dynamic webhook admission control plugin
Unit tests pass.
Needs plumbing:
* [ ] service resolver (depends on @wfender PR)
* [x] client cert (depends on ????)
* [ ] hook source (depends on @caesarxuchao PR)
Also at least one thing will need to be renamed after Chao's PR merges.
```release-note
Allow remote admission controllers to be dynamically added and removed by administrators. External admission controllers make an HTTP POST containing details of the requested action which the service can approve or reject.
```
Automatic merge from submit-queue (batch tested with PRs 46801, 45184, 45930, 46192, 45563)
Exclude dockerized verify patterns
**What this PR does / why we need it**:
Change some of the `make verify` logic to allow excluding based on a pattern. Add the `verify-*-dockerized.sh` pattern to the excluded list.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46801, 45184, 45930, 46192, 45563)
Tweak client-gen to support metrics API
This PR adds support to client-gen for readonly APIs as well as for customizing the resource name used for a given type.
This is required to support the clients generated for `k8s.io/metrics/pkg/apis/metrics`.
cc @caesarxuchao
Currently based on #45180
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46432, 46701, 46326, 40848, 46396)
Add a server side Get operation
Implement proposal kubernetes/community#363
```release-note
The Kubernetes API supports retrieving tabular output for API resources via a new mime-type `application/json;as=Table;v=v1alpha1;g=meta.k8s.io`. The returned object (if the server supports it) will be of type `meta.k8s.io/v1alpha1` with `Table`, and contain column and row information related to the resource. Each row will contain information about the resource - by default it will be the object metadata, but callers can add the `?includeObject=Object` query parameter and receive the full object. In the future kubectl will use this to retrieve the results of `kubectl get`.
```
Add location transformer, config for transformers
Location transformer helps choose the most specific transformer for
read/write operations depending on the path of resource being accessed.
Configuration allows use of --experimental-encryption-provider-config
to set up encryption providers. Only AEAD is supported at the moment.
Add new files to BUILD, AEAD => k8s-aes-gcm
Use group resources to select encryption provider
Update tests for configuration parsing
Remove location transformer
Allow specifying providers per resource group in configuration
Add IdentityTransformer configuration option
Fix minor issues with initial AEAD implementation
Unified parsing of all configurations
Parse configuration using a union struct
Run configuration parsing in APIserver, refactor parsing
More gdoc, fix minor bugs
Add test coverage for combined transformers
Use table driven tests for encryptionconfig
Automatic merge from submit-queue
apiserver: add a webhook implementation of the audit backend
This builds off of #45315 and is intended to implement an interfaced defined in #45766.
TODO:
- [x] Rebase on top of API types PR.
- [x] Rebase on top of API types updates (#46065)
- [x] Rebase on top of feature flag (#46009)
- [x] Rebase on top of audit instrumentation.
- [x] Hook up API server flag or register plugin (depending on #45766)
Features issue https://github.com/kubernetes/features/issues/22
Design proposal https://github.com/kubernetes/community/blob/master/contributors/design-proposals/auditing.md
```release-notes
Webhook added to the API server which omits structured audit log events.
```
/cc @soltysh @timstclair @soltysh @deads2k
Automatic merge from submit-queue
Set Kubelet Disk Defaults for the 1.7 release
The `--low-diskspace-threshold-mb` flag has been depreciated since 1.6.
This PR sets the default to `0`, and sets defaults for disk eviction based on the values used for our [e2e tests](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/services/kubelet.go#L145).
This also removes the custom defaults for vagrant, as the new defaults should work for it as well.
/assign @derekwaynecarr
cc @vishh
```release-note
By default, --low-diskspace-threshold-mb is not set, and --eviction-hard includes "nodefs.available<10%,nodefs.inodesFree<5%"
```
Automatic merge from submit-queue
Disable all alpha feature gates by default in local-up-cluster.sh
**What this PR does / why we need it**:
Disable all alpha feature gates by default in local-up-cluster.sh
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#46691
**Special notes for your reviewer**:
**Release note**:
```
None
```
Automatic merge from submit-queue
hack/verify-staging-imports.sh: simplify by negating package list
The forbidden list of repos got lengthy. The inverse is much shorter and easiert to maintain.
Automatic merge from submit-queue (batch tested with PRs 43505, 45168, 46439, 46677, 46623)
Add TPR to CRD migration helper.
This is a helper for migrating TPR data to CustomResource. It's rather hacky because it requires crossing apiserver boundaries, but doing it this way keeps the mess contained to the TPR code, which is scheduled for deletion anyway.
It's also not completely hands-free because making it resilient enough to be completely automated is too involved to be worth it for an alpha-to-beta migration, and would require investing significant effort to fix up soon-to-be-deleted TPR code. Instead, this feature will be documented as a best-effort helper whose results should be verified by hand.
The intended benefit of this over a totally manual process is that it should be possible to copy TPR data into a CRD without having to tear everything down in the middle. The process would look like this:
1. Upgrade to k8s 1.7. Nothing happens to your TPRs.
1. Create CRD with group/version and resource names that match the TPR. Still nothing happens to your TPRs, as the CRD is hidden by the overlapping TPR.
1. Delete the TPR. The TPR data is converted to CustomResource data, and the CRD begins serving at the same REST path.
Note that the old TPR data is left behind by this process, so watchers should not receive DELETE events. This also means the user can revert to the pre-migration state by recreating the TPR definition.
Ref. https://github.com/kubernetes/kubernetes/issues/45728
Automatic merge from submit-queue (batch tested with PRs 46686, 45049, 46323, 45708, 46487)
[Federation][kubefed]: Use StorageClassName for etcd pvc
This PR updates kubefed to use the StorageClassName field [added in 1.6](http://blog.kubernetes.io/2017/03/dynamic-provisioning-and-storage-classes-kubernetes.html
) for etcd's pvc to allow the user to specify which storage class they want to use. If no value is provided to ``kubefed init``, the field will not be set, and initialization of the pvc may fail on a cluster without a default storage class configured.
The alpha annotation that was previously used (``volume.alpha.kubernetes.io/storage-class``) was deprecated as of 1.4 according to the following blog post:
http://blog.kubernetes.io/2016/10/dynamic-provisioning-and-storage-in-kubernetes.html
**Release note**:
```
'kubefed init' has been updated to support specification of the storage class (via --etcd-pv-storage-class) for the Persistent Volume Claim (PVC) used for etcd storage. If --etcd-pv-storage-class is not specified, the default storage class configured for the cluster will be used.
```
cc: @kubernetes/sig-federation-pr-reviews
To properly register the types in the admission API group we need to
create an "install" package and wire it up. This is required by the
webhook admission controller being developed as part of
https://github.com/kubernetes/community/pull/132
Automatic merge from submit-queue
Node authorizer
This PR implements the authorization portion of https://github.com/kubernetes/community/blob/master/contributors/design-proposals/kubelet-authorizer.md and kubernetes/features#279:
* Adds a new authorization mode (`Node`) that authorizes requests from nodes based on a graph of related pods,secrets,configmaps,pvcs, and pvs:
* Watches pods, adds edges (secret -> pod, configmap -> pod, pvc -> pod, pod -> node)
* Watches pvs, adds edges (secret -> pv, pv -> pvc)
* When both Node and RBAC authorization modes are enabled, the default RBAC binding that grants the `system:node` role to the `system:nodes` group is not automatically created.
* Tightens the `NodeRestriction` admission plugin to require identifiable nodes for requests from users in the `system:nodes` group.
This authorization mode is intended to be used in combination with the `NodeRestriction` admission plugin, which limits the pods and nodes a node may modify. To enable in combination with RBAC authorization and the NodeRestriction admission plugin:
* start the API server with `--authorization-mode=Node,RBAC --admission-control=...,NodeRestriction,...`
* start kubelets with TLS boostrapping or with client credentials that place them in the `system:nodes` group with a username of `system:node:<nodeName>`
```release-note
kube-apiserver: a new authorization mode (`--authorization-mode=Node`) authorizes nodes to access secrets, configmaps, persistent volume claims and persistent volumes related to their pods.
* Nodes must use client credentials that place them in the `system:nodes` group with a username of `system:node:<nodeName>` in order to be authorized by the node authorizer (the credentials obtained by the kubelet via TLS bootstrapping satisfy these requirements)
* When used in combination with the `RBAC` authorization mode (`--authorization-mode=Node,RBAC`), the `system:node` role is no longer automatically granted to the `system:nodes` group.
```
```release-note
RBAC: the automatic binding of the `system:node` role to the `system:nodes` group is deprecated and will not be created in future releases. It is recommended that nodes be authorized using the new `Node` authorization mode instead. Installations that wish to continue giving all members of the `system:nodes` group the `system:node` role (which grants broad read access, including all secrets and configmaps) must create an installation-specific ClusterRoleBinding.
```
Follow-up:
- [ ] enable e2e CI environment with admission and authorizer enabled (blocked by kubelet TLS bootstrapping enablement in https://github.com/kubernetes/kubernetes/pull/40760)
- [ ] optionally enable this authorizer and admission plugin in kubeadm
- [ ] optionally enable this authorizer and admission plugin in kube-up
Automatic merge from submit-queue
Support grabbing test suite metrics
**What this PR does / why we need it**:
Add support for grabbing metrics that cover the entire test suite's execution.
Update the "interesting" controller-manager metrics to match the
current names for the garbage collector, and add namespace controller
metrics to the list.
If you enable `--gather-suite-metrics-at-teardown`, the metrics file is written to a file with a name such as `MetricsForE2ESuite_2017-05-25T20:25:57Z.json` in the `--report-dir`. If you don't specify `--report-dir`, the metrics are written to the test log output.
I'd like to enable this for some of the `pull-*` CI jobs, which will require a separate PR to test-infra.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-testing-pr-reviews @smarterclayton @wojtek-t @gmarek @derekwaynecarr @timothysc
This sample adds sample code for create/list/replace/delete operations for
Deployment resource for client-go library. This patch also contains
documentation explaining what the example application does and how it can be
executed.
This patch is part of the body of work improving the client library samples.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Automatic merge from submit-queue
Support storageclass storage updates to v1
**What this PR does / why we need it**: enable cluster administrators to update storageclasses stored in etcd from storage.k8s.io/v1beta1 to storage.k8s.io/v1.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: I had a hard time getting the test to work with how it was handling KUBE_API_VERSIONS and RUNTIME_CONFIG. I would appreciate some extra review attention there. Also, I had to hack in a `cluster-scoped` "namespace" to get the verification portions of the test script to work. I'm definitely open to ideas for how to improve that if needed.
**Release note**:
```release-note
Support updating storageclasses in etcd to storage.k8s.io/v1. You must do this prior to upgrading to 1.8.
```
cc @kubernetes/sig-storage-pr-reviews @kubernetes/sig-api-machinery-pr-reviews @jsafrane @deads2k @saad-ali @enj
Automatic merge from submit-queue (batch tested with PRs 46561, 46566)
Fix golint errors in cmd/kube-apiserver
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 46561, 46566)
Return nil when err is nil
**What this PR does / why we need it**:
Return nil directly when err is nil
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Fix golint error in apimachinery/pkg/{fields,lables}
Also add these two packages to linted packages lists
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
no-snat test
This test checks that Pods can communicate with each other in the same cluster without SNAT.
I intend to create a job that runs this in small clusters (\~3 nodes) at a low frequency (\~once per day) so that we have a signal as we work on allowing multiple non-masquerade CIDRs to be configured (see [kubernetes-incubator/ip-masq-agent](https://github.com/kubernetes-incubator/ip-masq-agent), for example).
/cc @dnardo
Automatic merge from submit-queue
Protobuf generation for k8s.io/metrics
This PR introduces protobuf generation for k8s.io/metrics. Doing so required:
- fixing a bug in `go-to-protobuf` causing the `cast{key,value,type}` values to not be quoted when coming from struct tags (and not auto-injection by `go-to-protobuf` itself).
- Making sure the proto IDL in k8s.io/client-go had a package name of `k8s.io.client_go.xyz` and not `k8s.io.kubernetes.xyz`.
Additionally, I updated `go-to-protobuf` to skip functions and non-public types when composing the import list, which cuts down on the more bizarre imports in the IDL (like importing the sample API package in every IDL file because it contained `addToScheme`, like every other API package).
We use `castvalue` to force gogo-proto to realize that it should consider the value of the map which underlies `ResourceList` when calculating which imports need to be named. Otherwise, it ignores the value's type, leading to compilation errors when it later can't find an import it assumed existed. We accidentally didn't hit this in `k8s.io/kubernetes/pkg/api/v1` since another field coincidentally happens to directly use `resource.Quantity` (the value type of `ResourceList`).
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46302, 44597, 44742, 46554)
Change to aggregator so it calls a user apiservice via its pod IP.
proxy_handler now does a sideways call to lookup the pod IPs for aservice.
It will then pick a random pod IP to forward the use apiserver request to.
**What this PR does / why we need it**: It allows the aggregator to work without setting up the full network stack on the kube master (i.e. with kube-dns or kube-proxy)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44619
**Special notes for your reviewer**:
**Release note**:
```release-note
```
This commit adds the `k8s.io/metrics` APIs to the list of packages for
which to generate protobuf. Additionally, it adds
`k8s.io/client-go/pkg/apis/v1` as a non-generated (referenced) package.
Automatic merge from submit-queue
support NonResourceURL for kubectl create clusterrole
Release note:
```release-note
add --non-resource-url to kubectl create clusterrole
```
proxy_handler now uses the endpoint router to map the cluster IP to
appropriate endpoint (Pod) IP for the given resource.
Added code to allow aggregator routing to be optional.
Updated bazel build.
Fixes to cover JLiggit comments.
Added util ResourceLocation method based on Listers.
Fixed issues from verification steps.
Updated to add an interface to obfuscate some of the routing logic.
Collapsed cluster IP resolution in to the aggregator routing
implementation.
Added 2 simple unit tests for ResolveEndpoint
Automatic merge from submit-queue (batch tested with PRs 46383, 45645, 45923, 44884, 46294)
Dynamic registration prototype
Implementing the api proposed in https://github.com/kubernetes/community/pull/611.
Wiring the code to serve the api via apiserver.
```release-note
Adding admissionregistration API group which enables dynamic registration of initializers and external admission webhooks. It is an alpha feature.
```
Update the "interesting" controller-manager metrics to match the
current names for the garbage collector, and add namespace controller
metrics to the list.
Automatic merge from submit-queue (batch tested with PRs 46429, 46308, 46395, 45867, 45492)
Controller history
**What this PR does / why we need it**:
Implements the ControllerRevision API object and clientset to allow for the implementation of StatefulSet update and DaemonSet history
```release-note
ControllerRevision type added for StatefulSet and DaemonSet history.
```
Automatic merge from submit-queue (batch tested with PRs 46429, 46308, 46395, 45867, 45492)
Bump Go version to 1.8.3
This PR also removed this patched version of Go 1.8.1 which we used to use to workaround performance problem of Go 1.8.1.
Fix https://github.com/kubernetes/kubernetes/issues/45216
Ref #46391
@timothysc @bradfitz
This change makes it so that errors during watch decoding panic the
server if it is in a test environment. This allows us to catch coder
errors related to storing incompatible types at the same location in
etcd.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Automatic merge from submit-queue (batch tested with PRs 46149, 45897, 46293, 46296, 46194)
check flag format in file known-flags.txt
All flags in file hack/verify-flags/known-flags.txt should contain
character -, this change check it to prevent adding useless flags
to known-flags.txt
ref #45948
**Release note**:
```
NONE
```
Automatic merge from submit-queue
Allow the /logs handler on the apiserver to be toggled.
Adds a flag to kube-apiserver, and plumbs through en environment variable in configure-helper.sh
Automatic merge from submit-queue
Enable "kick the tires" support for Nvidia GPUs in COS
This PR provides an installation daemonset that will install Nvidia CUDA drivers on Google Container Optimized OS (COS).
User space libraries and debug utilities from the Nvidia driver installation are made available on the host in a special directory on the host -
* `/home/kubernetes/bin/nvidia/lib` for libraries
* `/home/kubernetes/bin/nvidia/bin` for debug utilities
Containers that run CUDA applications on COS are expected to consume the libraries and debug utilities (if necessary) from the host directories using `HostPath` volumes.
Note: This solution requires updating Pod Spec across distros. This is a known issue and will be addressed in the future. Until then CUDA workloads will not be portable.
This PR updates the COS base image version to m59. This is coupled with this PR for the following reasons:
1. Driver installation requires disabling a kernel feature in COS.
2. The kernel API for disabling this interface changed across COS versions
3. If the COS image update is not handled in this PR, then a subsequent COS image update will break GPU integration and will require an update to the installation scripts in this PR.
4. Instead of having to post `3` PRs, one each for adding the basic installer, updating COS to m59, and then updating the installer again, this PR combines all the changes to reduce review overhead and latency, and additional noise that will be created when GPU tests break.
**Try out this PR**
1. Get Quota for GPUs in any region
2. `export `KUBE_GCE_ZONE=<zone-with-gpus>` KUBE_NODE_OS_DISTRIBUTION=gci`
3. `NODE_ACCELERATORS="type=nvidia-tesla-k80,count=1" cluster/kube-up.sh`
4. `kubectl create -f cluster/gce/gci/nvidia-gpus/cos-installer-daemonset.yaml`
5. Run your CUDA app in a pod.
**Another option is to run a e2e manually to try out this PR**
1. Get Quota for GPUs in any region
2. export `KUBE_GCE_ZONE=<zone-with-gpus>` KUBE_NODE_OS_DISTRIBUTION=gci
3. `NODE_ACCELERATORS="type=nvidia-tesla-k80,count=1"`
4. `go run hack/e2e.go -- --up`
5. `hack/ginkgo-e2e.sh --ginkgo.focus="\[Feature:GPU\]"`
The e2e will install the drivers automatically using the daemonset and then run test workloads to validate driver integration.
TODO:
- [x] Update COS image version to m59 release.
- [x] Remove sleep from the install script and add it to the daemonset
- [x] Add an e2e that will run the daemonset and run a sample CUDA app on COS clusters.
- [x] Setup a test project with necessary quota to run GPU tests against HEAD to start with https://github.com/kubernetes/test-infra/pull/2759
- [x] Update node e2e serial configs to install nvidia drivers on COS by default
Automatic merge from submit-queue (batch tested with PRs 46201, 45952, 45427, 46247, 46062)
kubectl: fix deprecation warning bug
**What this PR does / why we need it**:
Some kubectl commands were deprecated but would fail to print the
correct warning message when a flag was given before the command name.
# Correctly prints the warning that "resize" is deprecated and
# "scale" is now preferred.
kubectl resize [...]
# Should print the same warning but no warning is printed.
kubectl --v=1 resize [...]
This was due to a fragile check on os.Args[1].
This commit implements a new function deprecatedCmd() that is used to
construct new "passthrough" commands which are marked as deprecated and
hidden.
Note that there is an existing "filters" system that may be preferable
to the system created in this commit. I'm not sure why the "filters"
array was not used for all deprecated commands in the first place.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46201, 45952, 45427, 46247, 46062)
[Federation][kubefed]: Add support for etcd image override
This PR adds support for overriding the default etcd image used by ``kubefed init`` by providing an argument to ``--etcd-image``. This is primarily intended to allow consumers like openshift to provide a different default, but as a nice side-effect supports code-free validation of non-default etcd images.
**Release note**:
```release-note
'kubefed init' now supports overriding the default etcd image name with the --etcd-image parameter.
```
cc: @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 46022, 46055, 45308, 46209, 43590)
More cli sanity verifications
Adds some more `kubectl` command sanity checks to improve consistency and avoid the need of code reviews for some of our CLI style and standards.
**Release note**:
```release-note
NONE
```
@kubernetes/sig-cli-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 38990, 45781, 46225, 44899, 43663)
migrate set generation to go_genrule
Depends on https://github.com/kubernetes/release/pull/238
Kubernetes Submit Queue