Automatic merge from submit-queue
Job failure policy controller support
**What this PR does / why we need it**:
Start implementing the support of the "Backoff policy and failed pod limit" in the ```JobController``` defined in https://github.com/kubernetes/community/pull/583.
This PR depends on a previous PR #48075 that updates the K8s API types.
TODO:
* [X] Implement ```JobSpec.BackoffLimit``` support
* [x] Rebase when #48075 has been merged.
* [X] Implement end2end tests
implements https://github.com/kubernetes/community/pull/583
**Special notes for your reviewer**:
**Release note**:
```release-note
Add backoff policy and failed pod limit for a job
```
Automatic merge from submit-queue (batch tested with PRs 51805, 51725, 50925, 51474, 51638)
Allow custom client verbs to be generated using client-gen
This change will allow to define custom verbs for resources using the following new tag:
```
// +genclient:method=Foo,verb=create,subresource=foo,input=Bar,output=k8s.io/pkg/api.Blah
```
This will generate client method `Foo(bar *Bar) (*api.Blah, error)` (format depends on the particular verb type)
With this change we can add `UpdateScale()` and `GetScale()` into all scalable resources. Note that intention of this PR is not to fix the Scale(), but that is used as an example of this new capability.
Additionally this will also allow us to get rid of `// +genclient:noStatus` and fix guessing of the "updateStatus" subresource presence based on the existence of '.Status' field.
Basically you will have to add following into all types you want to generate `UpdateStatus()` for:
```
// +genclient:method=UpdateStatus,verb=update,subresource=status
```
This allows further extension of the client without writing an expansion (which proved to be pain to maintain and copy...). Also allows to customize native CRUD methods if needed (input/output types).
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51805, 51725, 50925, 51474, 51638)
Flexvolume dynamic plugin discovery: Prober unit tests and basic e2e test.
**What this PR does / why we need it**: Tests for changes introduced in PR #50031 .
As part of the prober unit test, I mocked filesystem, filesystem watch, and Flexvolume plugin initialization.
Moved the filesystem event goroutine to watcher implementation.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#51147
**Special notes for your reviewer**:
First commit contains added functionality of the mock filesystem.
Second commit is the refactor for moving mock filesystem into a common util directory.
Third commit is the unit and e2e tests.
**Release note**:
```release-note
NONE
```
/release-note-none
/sig storage
/assign @saad-ali @liggitt
/cc @mtaufen @chakri-nelluri @wongma7
Automatic merge from submit-queue (batch tested with PRs 51805, 51725, 50925, 51474, 51638)
Limit events accepted by API Server
**What this PR does / why we need it**:
This PR adds the ability to limit events processed by an API server. Limits can be set globally on a server, per-namespace, per-user, and per-source+object. This is needed to prevent badly-configured or misbehaving players from making a cluster unstable.
Please see https://github.com/kubernetes/community/pull/945.
**Release Note:**
```release-note
Adds a new alpha EventRateLimit admission control that is used to limit the number of event queries that are accepted by the API Server.
```
Automatic merge from submit-queue
kubeadm: Add omitempty tags to nullable values and use metav1.Duration
**What this PR does / why we need it**:
From @sttts review of https://github.com/kubernetes/kubernetes/pull/49959; we found some shortcomings of the current JSON tags in the kubeadm config file.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Note that https://github.com/kubernetes/kubernetes/pull/49959 will not be merged for v1.8, but this at least improves the state in v1.8 without changing anything really.
**Release note**:
```release-note
NONE
```
@kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 50579, 50875, 51797, 51807, 51803)
make url parsing in apiserver configurable
We have known cases where the attributes for a request are assigned differently. The kubelet is one example. This makes the value an interface, not a struct, and provides a hook for (non-default) users to override it.
Automatic merge from submit-queue (batch tested with PRs 50579, 50875, 51797, 51807, 51803)
oidc auth: make the OIDC claims prefix configurable
Add the following flags to control the prefixing of usernames and
groups authenticated using OpenID Connect tokens.
--oidc-username-prefix
--oidc-groups-prefix
```release-note
The OpenID Connect authenticator can now use a custom prefix, or omit the default prefix, for username and groups claims through the --oidc-username-prefix and --oidc-groups-prefix flags. For example, the authenticator can map a user with the username "jane" to "google:jane" by supplying the "google:" username prefix.
```
Closes https://github.com/kubernetes/kubernetes/issues/50408
Ref https://github.com/kubernetes/kubernetes/issues/31380
cc @grillz @kubernetes/sig-auth-pr-reviews @thomastaylor312 @gtaylor
Automatic merge from submit-queue
Fixes kubernetes/kubernetes#29271: accept prefixed namespaces
**What this PR does / why we need it**: `kubectl get namespaces -o name` outputs the names of all namespaces, prefixed with `namespaces/`. This changeset allows these namespace names to be passed directly back in to `kubectl` via the `-n` flag without reprocessing them to remove `namespaces/`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#29271
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue
Implement the `kubeadm upgrade` command
**What this PR does / why we need it**:
Implements the kubeadm upgrades proposal: https://docs.google.com/document/d/1PRrC2tvB-p7sotIA5rnHy5WAOGdJJOIXPPv23hUFGrY/edit#
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes: https://github.com/kubernetes/kubeadm/issues/14
**Special notes for your reviewer**:
I'm gonna split out changes not directly related to the upgrade procedure into separate PRs as dependencies as we go.
**Now ready for review. Please look at `cmd/kubeadm/app/phases/upgrade` first and give feedback on that. The rest kind of follows.**
**Release note**:
```release-note
Implemented `kubeadm upgrade plan` for checking whether you can upgrade your cluster to a newer version
Implemented `kubeadm upgrade apply` for upgrading your cluster from one version to an other
```
cc @fabriziopandini @kubernetes/sig-cluster-lifecycle-pr-reviews @craigtracey @mattmoyer
Automatic merge from submit-queue
Switch audit output to v1beta1
This PR adds two switches to pick preferred version for webhook and log backends, and it switches to use `audit.k8s.io/v1beta1` as default for both.
@sttts @crassirostris ptal
**Release note**:
```release-note
Switch to audit.k8s.io/v1beta1 in audit.
```
Automatic merge from submit-queue
bump QEMU version to v2.9.1
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
xref #38067
**Special notes for your reviewer**:
/assign @luxas
**Release note**:
```release-note
update QEMU version to v2.9.1
```
Job failure policy integration in JobController. From the
JobSpec.BackoffLimit the JobController will define the backoff
duration between Job retry.
It use the ```workqueue.RateLimitingInterface``` to store the number of
"retry" as "requeue" and the default Job backoff initial duration is set
during the initialization of the ```workqueue.RateLimiter.
Since the number of retry for each job is store in a local structure
"JobController.queue" if the JobController restarts the number of retries
will be lost and the backoff duration will be reset to 0.
Add e2e test for Job backoff failure policy
Remove APIService OpenAPI spec when it is deleted
Add eTag support and returning httpStatus to OpenAPI spec downloader
Update aggregated OpenAPI spec periodically
Use delegate chain
Refactor OpenAPI aggregator to have separate controller and aggregation function
Enable OpenAPI spec for extensions api server
Do not filter paths. higher priority specs wins the conflicting paths
Move OpenAPI aggregation controller to pkg/controller/openapi
Automatic merge from submit-queue (batch tested with PRs 50602, 51561, 51703, 51748, 49142)
Slow-start batch pod creation of rs, rc, ds, jobs
Prevent too-large replicas from generating enormous numbers
of events by creating only a few pods at a time, then increasing
the batch size when pod creations succeed. Stop creating batches
of pods when any pod creation errors are encountered.
Todo:
- [x] Add automated tests
- [x] Test ds
Fixes https://github.com/kubernetes/kubernetes/issues/49145
**Release note**:
```release-note
controllers backoff better in face of quota denial
```
Automatic merge from submit-queue (batch tested with PRs 50602, 51561, 51703, 51748, 49142)
Simplify describe events table
The describe table for events is not easy to read and violates other
output guidelines. Change to use spaces (we don't use tabs in formal
output for tables). Remove columns that are not normally needed or
available on events.
Example for pods:
```
...
QoS Class: BestEffort
Node-Selectors: role=app
Tolerations: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Pulling 1h (x51 over 5h) kubelet, origin-ci-ig-n-gj0x pulling image "registry.svc.ci.openshift.org/experiment/commenter:latest"
Normal BackOff 8m (x1274 over 5h) kubelet, origin-ci-ig-n-gj0x Back-off pulling image "registry.svc.ci.openshift.org/experiment/commenter:latest"
Warning FailedSync 3m (x1359 over 5h) kubelet, origin-ci-ig-n-gj0x Error syncing pod
```
Puts the type first (separate important from not), then reason (which is
the most impactful scanning field). Collapses first seen, last seen, and
times into a single field, since most of the time you care about the
last time the event happened, not the first time.
@kubernetes/sig-cli-pr-reviews sorry for the last minute drop, but the usability of this is driving me up the wall and I can't take it anymore. Would like to slip this into 1.8 so that I can debug things without dying a little inside.
Fixes#47715
```release-note
The event table output under `kubectl describe` has been simplified to show only the most essential info.
```
Automatic merge from submit-queue (batch tested with PRs 50602, 51561, 51703, 51748, 49142)
expose discovery information on scalable resources
Builds on https://github.com/kubernetes/kubernetes/pull/49971 and provides the GroupVersion information that can be used by a dynamic scale client.
@kubernetes/sig-api-machinery-pr-reviews
@foxish @DirectXMan12 since you both asked for it.
Automatic merge from submit-queue (batch tested with PRs 50602, 51561, 51703, 51748, 49142)
Implement GetZoneByProviderID & GetZoneByNodeName
Adding an implementation of GetZoneByProviderID & GetZoneByNodeName for
GCE.
This is related to ticket 50926.
This was tested as part of the ongoing separate GCE cloud provider work.
**What this PR does / why we need it**: It implements GCE methods needed by the cloud provider work.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50926
**Special notes for your reviewer**: Tested with pull/50811
**Release note**:
<!-- Steps to write your release note:
```release-note NONE
```
Automatic merge from submit-queue (batch tested with PRs 50602, 51561, 51703, 51748, 49142)
Use arm32v7|arm64v8 images instead of the deprecated armhf|aarch64 image organizations
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50601
**Special notes for your reviewer**:
/assign @ixdy @jbeda @zmerlynn
**Release note**:
```release-note
Use arm32v7|arm64v8 images instead of the deprecated armhf|aarch64 image organizations
```
Automatic merge from submit-queue (batch tested with PRs 51301, 50497, 50112, 48184, 50993)
Replace the deprecated function with the suggest function in aws module
**What this PR does / why we need it**:
There are some deprecated function and I replace the deprecated function with the suggest function in aws module.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51301, 50497, 50112, 48184, 50993)
audit newest impersonated user info in the ResponseStarted, ResponseComplete audit stage
Impersonation will automatically add system:authenticated, system:serviceaccounts group to the impersonated user info. This pr use the newest impersonated user info in the second audit event. This will help users to debug rbac problems.
**Release note**:
```
[advanced audit] audit newest impersonated user info in the ResponseStarted, ResponseComplete audit stage
```
@liggitt @sttts
Automatic merge from submit-queue (batch tested with PRs 51301, 50497, 50112, 48184, 50993)
AWS: handle multiple IPs when using more than 1 network interface per ec2 instance
**What this PR does / why we need it**:
Adds support for kubelets running with the AWS cloud provider on ec2 instances with multiple network interfaces. If the active interface is not eth0, the AWS cloud provider currently reports the wrong node IP.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44686
**Special notes for your reviewer**:
There is also some work necessary for handling multiple DNS names and such but I didn't fix them in this PR.
**Release note**:
```release-note
Fixed bug in AWS provider to handle multiple IPs when using more than 1 network interface per ec2 instance.
```
Automatic merge from submit-queue (batch tested with PRs 51301, 50497, 50112, 48184, 50993)
Introduce new flag "--include-uninitialized" to kubectl
**What this PR does / why we need it**:
Introduce `--include-uninitialized` as a global flag to kubectl
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49035
**Special notes for your reviewer**:
/assign @caesarxuchao @smarterclayton @ahmetb @deads2k
**Release note**:
```release-note
Add flag "--include-uninitialized" to kubectl annotate, apply, edit-last-applied, delete, describe, edit, get, label, set. "--include-uninitialized=true" makes kubectl commands apply to uninitialized objects, which by default are ignored if the names of the objects are not provided. "--all" also makes kubectl commands apply to uninitialized objects. Please see the [initializer](https://kubernetes.io/docs/admin/extensible-admission-controllers/) doc for more details.
```
Automatic merge from submit-queue
GCE: Add Alpha feature "Network Tiers" for external L4 load balancers
**Special notes for your reviewer**:
The PR has been manually tested in a GCE e2e cluster for the following conditions:
1. When `network-tier` is not enabled in gce.conf, network tier annotations are completely ignored by the controller.
2. When `network-tier` is enabled in gce.conf:
* Service w/ Standard tier: create a standard-tier LB.
* Update Service to use a different tier: tear down the existing forwarding rule and release the IP before creating a new LB.
* Service w/ an invalid tier value: `ensureExternalLoadBalancer()` returns an error, and controller emits an event.
* Service w/ a user-owned static IP: check if the tier matches, if not, returns an error and emits an event.
I uploaded an e2e test #51483. You're welcome to review that one too.
**Release note**:
```release-note
GCE: Service object now supports "Network Tiers" as an Alpha feature via annotations.
```
Automatic merge from submit-queue (batch tested with PRs 51666, 49829, 51058, 51004, 50938)
Fix threshold notifier build tags
**What this PR does / why we need it**:
Cross building from darwin is currently broken on the following error:
```
# k8s.io/kubernetes/pkg/kubelet/eviction
pkg/kubelet/eviction/threshold_notifier_unsupported.go:25: NewMemCGThresholdNotifier redeclared in this block
previous declaration at pkg/kubelet/eviction/threshold_notifier_linux.go:38
```
It looks like #49300 broke the build tags introduced in #38630 and #37384. This fixes the build tag on `threshold_notifier_unsupported.go` as the cgo requirement was removed from `threshold_notifier_linux.go`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50935
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51666, 49829, 51058, 51004, 50938)
add the caller ip into rsync hosts allow list
fixes https://github.com/kubernetes/kubernetes/issues/51003.
This patch allows running build in pods.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51666, 49829, 51058, 51004, 50938)
Add debugging to the codegen process
This was useful when debugging a different problem.
Automatic merge from submit-queue (batch tested with PRs 51666, 49829, 51058, 51004, 50938)
Add test items for job utils
**What this PR does / why we need it**:
Add test item for job util
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
NONE
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51666, 49829, 51058, 51004, 50938)
Fixed integer overflow when matching PVPVC claims
Fixes#49911
Fixed integer overflow when matching PVPVC claims. Added test to guard this behavior.
Automatic merge from submit-queue (batch tested with PRs 51553, 51538, 51663, 51069, 51737)
Edit owner files for kube-proxy manifests
**What this PR does / why we need it**: We should have owner file for kube-proxy daemonset manifest.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #NONE
**Special notes for your reviewer**:
/assign @bowei @thockin
cc @dnardo @freehan @nicksardo
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51553, 51538, 51663, 51069, 51737)
Allow enable pod priority feature gate for GCE and configure priority for kube-proxy
**What this PR does / why we need it**:
From #23225, this PR adds an option for user to enable pod priority feature gate using GCE startup scripts, and configure pod priority for kube-proxy when enabled.
The setup `priorityClassName: system` derives from: ce1485c626/staging/src/k8s.io/api/core/v1/types.go (L2536-L2542)
The plan is to configure pod priority for kube-proxy daemonset (https://github.com/kubernetes/kubernetes/pull/50705) in the same way.
**Special notes for your reviewer**:
cc @bsalamat @davidopp @thockin
**Release note**:
```release-note
When using kube-up.sh on GCE, user could set env `ENABLE_POD_PRIORITY=true` to enable pod priority feature gate.
```
Kubernetes Submit Queue