Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)
Generate protobuf for the audit API
Add protobuf generation & annotations for the advanced auditing API. We're not using them directly now, but settling on the field numbering will help with some 3rd party integrations.
For kubernetes/features#22
/cc @ihmccreery
Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)
Add group alias names to API resources to allow discovery
**What this PR does / why we need it**:
Adds `GroupNames []string` to API resources, which represents the list of group aliases that every resource belongs to.
Partially fixes https://github.com/kubernetes/kubernetes/issues/41353
This moves the logic of "all" (which currently [translates](fbc94c0896/pkg/kubectl/cmd/util/shortcut_restmapper.go (L106)) to "pods,replicationcontrollers,services,...") to the server-side. Will allow clients like `kubectl` to discover group aliases instead of having it hardcoded and the API server to better handle consistency across multiple clients, version skew, etc; and will make "all" un-special and allow other groups to be created.
As a follow-up we'll patch `kubectl` to make groups aliases discoverable and the hardcoded list a fallback while we still have to support it.
Related to https://github.com/kubernetes/kubernetes/pull/42595#issuecomment-286839885.
**Release note**:
```release-note
Adds the `Categories []string` field to API resources, which represents the list of group aliases (e.g. "all") that every resource belongs to.
```
@kubernetes/sig-api-machinery-misc @deads2k @bgrant0607
Automatic merge from submit-queue (batch tested with PRs 46967, 46992, 43338, 46717, 46672)
deflake CRD watch tests
Fixes https://github.com/kubernetes/kubernetes/issues/46446. Again...
This flake window is caused by the watch cache starting late. This pull updates the test to do fancy list/create/watch/delete semantics to catch the problem. In the field, this should be treated the same as any other "resourceVersion tool old" error and handled with a list/watch. The test cannot be level driven, it is actually testing the edge behavior, so we have to do something weird like this.
@sttts @liggitt let's try this again...
Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)
Fix round-trip of Unstructured.OwnerReferences
**What this PR does / why we need it**:
Previously `setOwnerReference()` was storing pointers but `extractOwnerReference()` is expecting pointer fields as plain values so it cannot read those pointers. And hence you cannot read what you've just stored.
**Which issue this PR fixes**
#46817
**Special notes for your reviewer**:
This is similar to #43346.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)
While deleting a namespace, the TPR instances under this ns should be…
… deleted.
While deleting a namespace, the TPR instances under this ns should be deleted.
Fixed#46736
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)
update copyed doc for advanced audit
doc for WithAudit is copyed from WithLegacyAudit, it's out of date.
This change update doc for these two functions.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 44883, 46836, 46765, 46683, 46050)
Added Bind method to Scheduler Extender
- only one extender can support the bind method
- if an extender supports bind, scheduler delegates the pod binding to the extender
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#41235
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 46972, 42829, 46799, 46802, 46844)
promote tls-bootstrap to beta
last commit of this PR.
Towards https://github.com/kubernetes/kubernetes/issues/46999
```release-note
Promote kubelet tls bootstrap to beta. Add a non-experimental flag to use it and deprecate the old flag.
```
Automatic merge from submit-queue (batch tested with PRs 46972, 42829, 46799, 46802, 46844)
Multizone static pv test
**What this PR does / why we need it**:
Adds an e2e test for checking that pods get scheduled to the same zone as statically created PVs. This tests the PersistentVolumeLabel admission controller, which adds zone and region labels when PVs are created. As part of this, I also had to make changes to volume test utility code to pass in a zone parameter for creating PDs, and also had to add an argument to the e2e test program to accept a list of zones.
Fixes#46995
**Special notes for your reviewer**:
It's probably easier to review each commit separately.
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)
Add configuration for encryption providers
## Additions
Allows providing a configuration file (using flag `--experimental-encryption-provider-config`) to use the existing AEAD transformer (with multiple keys) by composing mutable transformer, prefix transformer (for parsing providerId), another prefix transformer (for parsing keyId), and AES-GCM transformers (one for each key). Multiple providers can be configured using the configuration file.
Example configuration:
```
kind: EncryptionConfig
apiVersion: v1
resources:
- resources:
- namespaces
providers:
- aes:
keys:
- name: key1
secret: c2vjcmv0iglzihnly3vyzq==
- name: key2
secret: dghpcybpcybwyxnzd29yza==
- identity: {}
```
Need for configuration discussed in:
#41939
[Encryption](3418b4e4c6/contributors/design-proposals/encryption.md)
**Pathway of a read/write request**:
1. MutableTransformer
2. PrefixTransformer reads the provider-id, and passes the request further if that matches.
3. PrefixTransformer reads the key-id, and passes the request further if that matches.
4. GCMTransformer tries decrypting and authenticating the cipher text in case of reads. Similarly for writes.
## Caveats
1. To keep the command line parameter parsing independent of the individual transformer's configuration, we need to convert the configuration to an `interface{}` and manually parse it in the transformer. Suggestions on better ways to do this are welcome.
2. Flags `--encryption-provider` and `--encrypt-resource` (both mentioned in [this document](3418b4e4c6/contributors/design-proposals/encryption.md) ) are not supported in this because they do not allow more than one provider, and the current format for the configuration file possibly supersedes their functionality.
3. Currently, it can be tested by adding `--experimental-encryption-provider-config=config.yml` to `hack/local-up-cluster.sh` on line 511, and placing the above configuration in `config.yml` in the root project directory.
Previous discussion on these changes:
https://github.com/sakshamsharma/kubernetes/pull/1
@jcbsmpsn @destijl @smarterclayton
## TODO
1. Investigate if we need to store keys on disk (per [encryption.md](3418b4e4c6/contributors/design-proposals/encryption.md (option-1-simple-list-of-keys-on-disk)))
2. Look at [alpha flag conventions](https://github.com/kubernetes/kubernetes/blob/master/pkg/features/kube_features.go)
3. Need to reserve `k8s:enc` prefix formally for encrypted data. Else find a better way to detect transformed data.
Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)
bazel: base kube-proxy image on debian-iptables instead of busybox + iptables
**What this PR does / why we need it**: the bazel-built kube-proxy image currently uses a custom base image made up of scratch + busybox + iptables + a few dependencies, while the official kube-proxy image is based off of the debian-iptables image.
This difference seems to cause some weird issues such as #46103, since the container layout doesn't look the same.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#46103, probably?
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @mikedanese @spxtr @pipejakob
/cc @Q-Lee @thockin @cblecker
Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)
Update cAdvisor version to v0.26.0
issue: #46658
I have requested a 1 day exception for code freeze.
/assign @dchen1107
```release-note
Fix disk partition discovery for brtfs
Add ZFS support
Add overlay2 storage driver support
```
Automatic merge from submit-queue (batch tested with PRs 46550, 46663, 46816, 46820, 46460)
[GCE] Support internal load balancers
**What this PR does / why we need it**:
Allows users to expose K8s services externally of the K8s cluster but within their GCP network.
Fixes#33483
**Important User Notes:**
- This is a beta feature. ILB could be enabled differently in the future.
- Requires nodes having version 1.7.0+ (ILB requires health checking and a health check endpoint on kube-proxy has just been exposed)
- This cannot be used for intra-cluster communication. Do not call the load balancer IP from a K8s node/pod.
- There is no reservation system for private IPs. You can specify a RFC 1918 address in `loadBalancerIP` field, but it could be lost to another VM or LB if service settings are modified.
- If you're running an ingress, your existing loadbalancer backend service must be using BalancingMode type `RATE` - not `UTILIZATION`.
- Option 1: With a 1.5.8+ or 1.6.4+ version master, delete all your ingresses, and re-create them.
- Option 2: Migrate to a new cluster running 1.7.0. Considering ILB requires nodes with 1.7.0, this isn't a bad idea.
- Option 3: Possible migration opportunity, but use at your own risk. More to come later.
**Reviewer Notes**:
Several files were renamed, so github thinks ~2k lines have changed. Review commits one-by-one to see the actual changes.
**Release note**:
```release-note
Support creation of GCP Internal Load Balancers from Service objects
```
Handle failure cases on startup gracefully to avoid causing cascading
errors and poor initialization in other components. Initial errors from
config load cause the initializer to pause and hold requests. Return
typed errors to better communicate failures to clients.
Add code to handle two specific cases - admin wants to bypass
initialization defaulting, and mirror pods (which want to bypass
initialization because the kubelet owns their lifecycle).
Automatic merge from submit-queue
HPA Status Conditions
This PR introduces conditions to the status of the HorizontalPodAutoscaler (in autoscaling/v2alpha1).
The conditions whether or not the autoscaler is actively scaling, and why. This gives greater visibility
into the *current* status of the autoscaler, similarly to how conditions work for pods, nodes, etc.
`kubectl describe` has been updated to the display the conditions affecting a given HPA.
Implements kubernetes/features#264 (alpha in 1.7)
**Release note**:
```release-note
Introduces status conditions to the HorizontalPodAutoscaler in autoscaling/v2alpha1, indicating the current status of a given HorizontalPodAutoscaler, and why it is or is not scaling.
```
This commit updates `kubectl describe` to display the new HPA
status conditions. This should make it easier for users to discern
the current state of the HPA.
This commit causes the HPA controller to set a variety of status
conditions using the new `Status.Conditions` field of
autoscaling/v2alpha1. These provide insight into the current state
of the HPA, and generally correspond to similar events being emitted.
This commit adds the new API status conditions to the API types.
The field exists as a field in autoscaling/v2alpha1, and is
round-tripped through an annotation in autoscaling/v1.
Automatic merge from submit-queue (batch tested with PRs 46734, 46810, 46759, 46259, 46771)
Added node to persistent-volume-binder clusterrole
**What this PR does / why we need it**: Added missing permission to volume-binder clusterrole
**Which issue this PR fixes**: fixes#46770
**Special notes for your reviewer**: Non
**Release note**: Non
Automatic merge from submit-queue (batch tested with PRs 46734, 46810, 46759, 46259, 46771)
Add iptables lock-file mount to kube-proxy manifest
**What this PR does / why we need it**: kube-proxy is broken in make bazel-release. The new iptables binary uses a lockfile in "/run", but the directory doesn't exist. This causes iptables-restore to fail. We need to share the same lock-file amongst all containers, so mount the host /run dir.
This is similar to #46132 but expediency matters, since builds are broken.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#46103
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 46734, 46810, 46759, 46259, 46771)
Improve code coverage for pkg/kubelet/images/image_gc_manager
**What this PR does / why we need it**:
#39559#40780
code coverage from 74.5% to 77.4%
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 46734, 46810, 46759, 46259, 46771)
OpenAPI aggregation for kube-aggregator
This PR implements OpenAPI aggregation layer for kube-aggregator. On each API registration, it tries to download swagger.spec of the user api server. On failure it will try again next time (either on another add or get /swagger.* on aggregator server) up to five times. To merge specs, it first remove all unrelated paths from the downloaded spec (anything other than group/version of the API service) and then remove all unused definitions. Adding paths are straightforward as they won't have any conflicts, but definitions will most probably have conflicts. To resolve that, we would reused any definition that is not changed (documentation changes are fine) and rename the definition otherwise.
To use this PR, kube aggregator should have nonResourceURLs (for get verb) to user apiserver.
```release-note
Support OpenAPI spec aggregation for kube-aggregator
```
fixes: #43717
Automatic merge from submit-queue (batch tested with PRs 45871, 46498, 46729, 46144, 46804)
PD e2e test: Ready node check now uses the most up-to-date node count.
Follow-up to PR #46746
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
Automatic merge from submit-queue (batch tested with PRs 45871, 46498, 46729, 46144, 46804)
Implement kubectl rollout undo and history for DaemonSet
~Depends on #45924, only the 2nd commit needs review~ (merged)
Ref https://github.com/kubernetes/community/pull/527/
TODOs:
- [x] kubectl rollout history
- [x] sort controller history, print overview (with revision number and change cause)
- [x] print detail view (content of a history)
- [x] print template
- [x] ~(do we need to?) print labels and annotations~
- [x] kubectl rollout undo:
- [x] list controller history, figure out which revision to rollback to
- if toRevision == 0, rollback to the latest revision, otherwise choose the history with matching revision
- [x] update the ds using the history to rollback to
- [x] replace the ds template with history's
- [x] ~(do we need to?) replace the ds labels and annotations with history's~
- [x] test-cmd.sh
@kubernetes/sig-apps-pr-reviews @erictune @kow3ns @lukaszo @kargakis @kubernetes/sig-cli-maintainers
---
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 45871, 46498, 46729, 46144, 46804)
Enable some pod-related admission plugins for kubemark
Ref https://github.com/kubernetes/kubernetes/issues/44701
This should help reduce discrepancy in "list pods" latency wrt real cluster. Let's see.
/cc @wojtek-t @gmarek
Automatic merge from submit-queue (batch tested with PRs 45871, 46498, 46729, 46144, 46804)
Fix some comments in dnsprovider
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
delete the useless "gv" in Errorf
Signed-off-by: yupengzte <yu.peng36@zte.com.cn>
**What this PR does / why we need it**:
Fix "no formatting directive in Errorf call"
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 43852, 44255)
Bump github.com/mitchellh/mapstructure
**What this PR does / why we need it**:
This PR bump revision of github.com/mitchellh/mapstructure.
The library is required by Gophercloud, also they has passed tests with the newer revision.
So, since Gophercloud is update, please also renew this library.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue
Add SuccessfulMountVolume message to the events of pod
**What this PR does / why we need it:**
When creating a pod with volume, the volume mount may failed at first, but eventually succeed after retry several times. kubectl describe pod can only see the failed messages, so i think it will be better to add the SuccessfulMountVolume message to the pod events too.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#42867
Kubernetes Submit Queue