Automatic merge from submit-queue (batch tested with PRs 51824, 50476, 52451, 52009, 52237)
Plumbing the proxy dialer to the webhook admission plugin
* Fixing https://github.com/kubernetes/kubernetes/issues/49987. Plumb the `Dial` function to the `transport.Config`
* Fixing https://github.com/kubernetes/kubernetes/issues/52366. Let the webhook admission plugin sets the `TLSConfg.ServerName`.
I tested it in my gke setup. I don't have time to implement an e2e test before 1.8 release. I think it's ok to add the test later, because *i)* the change only affects the alpha webhook admission feature, and *ii)* the webhook feature is unusable without the fix. That said, it's up to my reviewer to decide.
Filed https://github.com/kubernetes/kubernetes/issues/52368 for the missing e2e test.
( The second commit is https://github.com/kubernetes/kubernetes/pull/52372, which is just a cleanup of client configuration in e2e tests. It removed a function that marshalled the client config to json and then unmarshalled it. It is a prerequisite of this PR, because this PR added the `Dial` function to the config which is not json marshallable.)
```release-note
Fixed the webhook admission plugin so that it works even if the apiserver and the nodes are in two networks (e.g., in GKE).
Fixed the webhook admission plugin so that webhook author could use the DNS name of the service as the CommonName when generating the server cert for the webhook.
Action required:
Anyone who generated server cert for admission webhooks need to regenerate the cert. Previously, when generating server cert for the admission webhook, the CN value doesn't matter. Now you must set it to the DNS name of the webhook service, i.e., `<service.Name>.<service.Namespace>.svc`.
```
Automatic merge from submit-queue
use specified discovery information if possible
Fixes https://github.com/kubernetes/kubernetes/issues/49948
This uses the available discovery information if available, but it seems we never updated "normal" resources to show the singular name, so its often not available. I've left this code compatible.
@enisoc @ash2k
@kubernetes/sig-api-machinery-misc
```release-note
custom resources that use unconventional pluralization now work properly with kubectl and garbage collection
```
Automatic merge from submit-queue (batch tested with PRs 52227, 52120)
Fix discovery restmapper finding resources in non-preferred versions
Fixes: #52219
Also reverts behavioral changes to tests that version-qualified cronjobs to work around this issue.
The discovery rest mapper was only populating the priority rest mapper's search list with preferred groupversions.
That meant that if a resource existed in multiple non-preferred versions, AND did not exist in the preferred version (like cronjob, which only exists in v1beta2.batch and v2alpha1.batch, but not v1.batch), the priority restmapper would not find it in its group/version priority list, and would return an error.
```release-note
Fixed an issue looking up cronjobs when they existed in more than one API version
```
Automatic merge from submit-queue
Restore OWNERS file for k8s.io/metrics
The owners file for k8s.io/metrics somehow got lost. This restores it
to its contents on the "legacy" branch of k8s.io/metrics.
```release-note
NONE
```
Automatic merge from submit-queue
newline to separate unimplemented TaintEffectNoScheduleNoAdmit
**What this PR does / why we need it**:
Unimplemented `TaintEffectNoScheduleNoAdmit ` should not be treated as comments of `TaintEffectNoExecute `
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
xref #49530
**Special notes for your reviewer**:
/assign @k82cn
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 52047, 52063, 51528)
Improve dynamic kubelet config e2e node test and fix bugs
Rather than just changing the config once to see if dynamic kubelet
config at-least-sort-of-works, this extends the test to check that the
Kubelet reports the expected Node condition and the expected configuration
values after several possible state transitions.
Additionally, this adds a stress test that changes the configuration 100
times. It is possible for resource leaks across Kubelet restarts to
eventually prevent the Kubelet from restarting. For example, this test
revealed that cAdvisor's leaking journalctl processes (see:
https://github.com/google/cadvisor/issues/1725) could break dynamic
kubelet config. This test will help reveal these problems earlier.
This commit also makes better use of const strings and fixes a few bugs
that the new testing turned up.
Related issue: #50217
I had been sitting on this until the cAdvisor fix merged in #51751, as these tests fail without that fix.
**Release note**:
```release-note
NONE
```
Currently setting watch cache size for a given resource does not disable
the watch cache. This commit adds a new `default-watch-cache-size` flag
to map to the existing field, and refactors how watch cache sizes are
calculated to bring all of the code into one place. It also adds debug
logging to startup to allow us to verify watch cache enablement in
production.
Trying to make sure we always log the flags an instance of apiserver
starts with.
This can be especially valuable for emailed logs or e2e/kubemark tests.
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
Fill in creationtimestamp in audit events
**What this PR does / why we need it**:
This is fixing null creationtimestamp in audit events.
@sttts @crassirostris like we've talked earlier today
**Release note**:
```release-note
none
```
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
A policy with 0 rules should return an error
**Which issue this PR fixes**
[isuue#51565](https://github.com/kubernetes/kubernetes/issues/51565)
**Release note**:
```
An audit policy file with 0 rule returns an error.
```
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
apiservers: stratify versioned informer construction
The versioned share informer factory has been part of the GenericApiServer config,
but its construction depended on other fields of that config (e.g. the loopback
client config). Hence, the order of changes to the config mattered.
This PR stratifies this by moving the SharedInformerFactory from the generic Config
to the CompleteConfig struct. Hence, it is only filled during completion when it is
guaranteed that the loopback client config is set.
While doing this, the CompletedConfig construction is made more type-safe again,
i.e. the use of SkipCompletion() is considereably reduced. This is archieved by
splitting the derived apiserver Configs into the GenericConfig and the ExtraConfig
part. Then the completion is structural again because CompleteConfig is again
of the same structure: generic CompletedConfig and local completed ExtraConfig.
Fixes#50661.
Automatic merge from submit-queue (batch tested with PRs 52091, 52071)
Log a warning when --audit-policy-file not passed to apiserver
When audit backend provided, but --audit-policy-file is not passed, leave a info in log.
**Release note**:
```
Log a warning when --audit-policy-file not passed to apiserver
```
Automatic merge from submit-queue
fix format of forbidden messages
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#51813
**Special notes for your reviewer**:
/assign @deads2k @liggitt
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 48552, 51876)
Disable default paging in list watches
For 1.8 this will be off by default. In 1.9 it will be on by default.
Add tests and rename some fields to use the `chunking` terminology.
Note that the pager may be used for other things besides chunking.
Follow on to #48921, we left the field on to get some exercise in the normal code paths, but needs to be disabled for 1.8.
@liggitt let's merge on wednesday.
Rather than just changing the config once to see if dynamic kubelet
config at-least-sort-of-works, this extends the test to check that the
Kubelet reports the expected Node condition and the expected configuration
values after several possible state transitions.
Additionally, this adds a stress test that changes the configuration 100
times. It is possible for resource leaks across Kubelet restarts to
eventually prevent the Kubelet from restarting. For example, this test
revealed that cAdvisor's leaking journalctl processes (see:
https://github.com/google/cadvisor/issues/1725) could break dynamic
kubelet config. This test will help reveal these problems earlier.
This commit also makes better use of const strings and fixes a few bugs
that the new testing turned up.
Related issue: #50217
Automatic merge from submit-queue (batch tested with PRs 52097, 52054)
Provide field info in storage configuration
Makes debugging how storage was configured difficult
@kubernetes/sig-api-machinery-bugs
Automatic merge from submit-queue (batch tested with PRs 51239, 51644, 52076)
code-generator/protobuf: cut-off kubernetes specifics
The package list moved to hack/update-generated-protobuf-dockerized.sh.
This make the protobuf generator usable outside of kube.
Automatic merge from submit-queue
Fix proxied request-uri to be valid HTTP requests
Fixes#52022, introduced in 1.7. Stringifying/re-parsing the URL masked that the path was not constructed with a leading `/` in the first place.
This makes upgrade requests proxied to pods/services via the API server proxy subresources be valid HTTP requests
```release-note
Fixes an issue with upgrade requests made via pod/service/node proxy subresources sending a non-absolute HTTP request-uri to backends
```
For 1.8 this will be off by default. In 1.9 it will be on by default.
Add tests and rename some fields to use the `chunking` terminology.
Note that the pager may be used for other things besides chunking.
Automatic merge from submit-queue (batch tested with PRs 51956, 50708)
Move autoscaling/v2 from alpha1 to beta1
This graduates autoscaling/v2alpha1 to autoscaling/v2beta1. The move is more-or-less just a straightforward rename.
Part of kubernetes/features#117
```release-note
v2 of the autoscaling API group, including improvements to the HorizontalPodAutoscaler, has moved from alpha1 to beta1.
```
Automatic merge from submit-queue (batch tested with PRs 49133, 51557, 51749, 50842, 52018)
enhance unit tests of advance audit feature
This change addresses comments from @crassirostris
https://github.com/kubernetes/kubernetes/pull/49115#discussion_r133416080
It does three things:
1. use auditinternal for unit test in filter stage
2. add a seperate unit test for Audit-ID http header
3. add unit test for audit log backend
**Release note**:
```
NONE
```
Charge object count when object is created, no matter if the object is
initialized or not.
Charge the remaining quota when the object is initialized.
Also, checking initializer.Pending and initializer.Result when
determining if an object is initialized. We didn't need to check them
because before 51082, having 0 pending initializer and nil
initializers.Result is invalid.
Automatic merge from submit-queue
set AdvancedAuditing feature gate to true by default
All feature commits are merged. The types are updated already to beta. This only enable the feature gate by default.
**Release note**:
```
Promote the AdvancedAuditing feature to beta and enable the feature gate by default.
```
This change does three things:
1. use auditinternal for unit test in filter stage
2. add a seperate unit test for Audit-ID http header
3. add unit test for audit log backend
Automatic merge from submit-queue (batch tested with PRs 51603, 51653)
Graduate metrics/v1alpha1 to v1beta1
This introduces v1beta1 of the resource metrics API, previously in alpha.
The v1alpha1 version remains for compatibility with the Heapster legacy version
of the resource metrics API, which is compatible with the v1alpha1 version. It also
renames the v1beta1 version to `resource-metrics.metrics.k8s.io`.
The HPA controller's REST clients (but not the legacy client) have been migrated as well.
Part of kubernetes/features#118.
```release-note
Migrate the metrics/v1alpha1 API to metrics/v1beta1. The HorizontalPodAutoscaler
controller REST client now uses that version. For v1beta1, the API is now known as
resource-metrics.metrics.k8s.io.
```
Automatic merge from submit-queue (batch tested with PRs 51733, 51838)
Relax update validation of uninitialized pod
Split from https://github.com/kubernetes/kubernetes/pull/50344
Fix https://github.com/kubernetes/kubernetes/issues/47837
* Let the podStrategy to only call `validation.ValidatePod()` if the old pod is not initialized, so fields are mutable.
* Let the podStatusStrategy refuse updates if the old pod is not initialized.
cc @smarterclayton
```release-note
Pod spec is mutable when the pod is uninitialized. The apiserver requires the pod spec to be valid even if it's uninitialized. Updating the status field of uninitialized pods is invalid.
```
Automatic merge from submit-queue
Improve APIService auto-registration for HA/upgrade scenarios
Fixes#51912
Required for 1.8 due to impact on HA upgrades.
/assign @deads2k
cc @kubernetes/sig-api-machinery-bugs
```release-note
Fixes an issue with APIService auto-registration affecting rolling HA apiserver restarts that add or remove API groups being served.
```
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Bug Fix - Adding an allowed address pair wipes port security groups
**What this PR does / why we need it**:
Fix for cloud routes enabled instances will have their security groups
removed when the allowed address pair is added to the instance's port.
Upstream bug report is in:
https://github.com/gophercloud/gophercloud/issues/509
Upstream bug fix is in:
https://github.com/gophercloud/gophercloud/pull/510
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#51755
**Special notes for your reviewer**:
Just an fix in vendored code. minimal changes needed in OpenStack cloud provider
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Build test targets for all server platforms
**What this PR does / why we need it**:
🤦
I really should have checked this before code freeze, but tbh forgot it in the rush. Also I thought this was the case already...
As part of https://github.com/kubernetes/features/issues/288; these binaries should be built for all server platforms indeed.
This is just a straightforward add to that list.
Can we please get this into v1.8?
There is virtually no risk involved here really...
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Build test targets for all server platforms
```
@ixdy @jdumars @mkumatag
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Add EgressRule to NetworkPolicy
**What this PR does / why we need it**:
Add EgressRule to NetworkPolicy
**Which issue this PR fixes**: fixes#50453
**Special notes for your reviewer**:
- Please take a look at the comments for the various types. I tried to mimic some of the language used in the Ingress comments, but I may have mangled some sentences.
- Let me know if I should add some test cases for validation. I have 2-3, and did not think it was necessary to replicate each case already covered in ingress.
**Release note**:
```
Add egress policies to NetworkPolicy
```
Automatic merge from submit-queue (batch tested with PRs 51186, 50350, 51751, 51645, 51837)
Update Cadvisor Dependency
Fixes: https://github.com/kubernetes/kubernetes/issues/51832
This is the worst dependency update ever...
The root of the problem is the [name change of Sirupsen -> sirupsen](https://github.com/sirupsen/logrus/issues/570#issuecomment-313933276). This means that in order to update cadvisor, which venders the lowercase, we need to update all dependencies to use the lower-cased version. With that being said, this PR updates the following packages:
`github.com/docker/docker`
- `github.com/docker/distribution`
- `github.com/opencontainers/go-digest`
- `github.com/opencontainers/image-spec`
- `github.com/opencontainers/runtime-spec`
- `github.com/opencontainers/selinux`
- `github.com/opencontainers/runc`
- `github.com/mrunalp/fileutils`
- `golang.org/x/crypto`
- `golang.org/x/sys`
- `github.com/docker/go-connections`
- `github.com/docker/go-units`
- `github.com/docker/libnetwork`
- `github.com/docker/libtrust`
- `github.com/sirupsen/logrus`
- `github.com/vishvananda/netlink`
`github.com/google/cadvisor`
- `github.com/euank/go-kmsg-parser`
`github.com/json-iterator/go`
Fixed https://github.com/kubernetes/kubernetes/issues/51832
```release-note
Fix journalctl leak on kubelet restart
Fix container memory rss
Add hugepages monitoring support
Fix incorrect CPU usage metrics with 4.7 kernel
Add tmpfs monitoring support
```
Automatic merge from submit-queue
Add client side event spam filtering
**What this PR does / why we need it**:
Add client side event spam filtering to stop excessive traffic to api-server from internal cluster components.
this pr defines a per source+object event budget of 25 burst with refill of 1 every 5 minutes.
i tested this pr on the following scenarios:
**Scenario 1: Node with 50 crash-looping pods**
```
$ create 50 crash-looping pods on a single node
$ kubectl run bad --image=busybox --replicas=50 --command -- derekisbad
```
Before:
* POST events with peak of 1.7 per second, long-tail: 0.2 per second
* PATCH events with peak of 5 per second, long-tail: 5 per second
After:
* POST events with peak of 1.7 per second, long-tail: 0.2 per second
* PATCH events with peak of 3.6 per second, long-tail: 0.2 per second
Observation:
* https://github.com/kubernetes/kubernetes/pull/47462 capped the number of total events in the long-tail as expected, but did nothing to improve total spam of master.
**Scenario 2: replication controller limited by quota**
```
$ kubectl create quota my-quota --hard=pods=1
$ kubectl run nginx --image=nginx --replicas=50
```
Before:
* POST events not relevant as aggregation worked well here.
* PATCH events with peak and long-tail of 13.6 per second
After:
* POST events not relevant as aggregation worked well here.
* PATCH events with peak: .35 per second, and long-tail of 0
**Which issue this PR fixes**
fixes https://github.com/kubernetes/kubernetes/issues/47366
**Special notes for your reviewer**:
this was a significant problem in a kube 1.5 cluster we are running where events were co-located in a single etcd. this cluster was normal to have larger numbers of unhealty pods as well as denial by quota.
**Release note**:
```release-note
add support for client-side spam filtering of events
```
Automatic merge from submit-queue
Provide a way to omit Event stages in audit policy
This provide a way to omit some stages for each audit policy rule.
For example:
```
apiVersion: audit.k8s.io/v1beta1
kind: Policy
- level: Metadata
resources:
- group: "rbac.authorization.k8s.io"
resources: ["roles"]
omitStages:
- "RequestReceived"
```
RequestReceived stage will not be emitted to audit backends with previous config.
**Release note**:
```
None
```
#
Updates https://github.com/kubernetes/kubernetes/issues/48561
This provide a way to omit some stages for each audit policy rule.
For example:
apiVersion: audit.k8s.io/v1beta1
kind: Policy
- level: Metadata
resources:
- group: "rbac.authorization.k8s.io"
resources: ["roles"]
omitStages:
- "RequestReceived"
RequestReceived stage will not be emitted to audit backends with
previous config.
Introduce feature gate for expanding PVs
Add a field to SC
Add new Conditions and feature tag pvc update
Add tests for size update via feature gate
register the resize admission plugin
Update golint failures
Automatic merge from submit-queue
Remove deprecated init-container in annotations
fixes#50655fixes#51816closes#41004fixes#51816
Builds on #50654 and drops the initContainer annotations on conversion to prevent bypassing API server validation/security and targeting version-skewed kubelets that still honor the annotations
```release-note
The deprecated alpha and beta initContainer annotations are no longer supported. Init containers must be specified using the initContainers field in the pod spec.
```
Automatic merge from submit-queue
Workloads deprecation 1.8
**What this PR does / why we need it**: This PR deprecates the Deployment, ReplicaSet, and DaemonSet kinds in the extensions/v1beta1 group version and the StatefulSet, Deployment, and ControllerRevision kinds in the apps/v1beta1 group version. The Deployment, ReplicaSet, DaemonSet, StatefuSet, and ControllerRevision kinds in the apps/v1beta2 group version are now the current version.
xref kubernetes/features#353
```release-note
The Deployment, DaemonSet, and ReplicaSet kinds in the extensions/v1beta1 group version are now deprecated, as are the Deployment, StatefulSet, and ControllerRevision kinds in apps/v1beta1. As they will not be removed until after a GA version becomes available, you may continue to use these kinds in existing code. However, all new code should be developed against the apps/v1beta2 group version.
```
Automatic merge from submit-queue (batch tested with PRs 51819, 51706, 51761, 51818, 51500)
fix kube-proxy panic because of nil sessionAffinityConfig
**What this PR does / why we need it**:
fix kube-proxy panic because of nil sessionAffinityConfig
**Which issue this PR fixes**: closes#51499
**Special notes for your reviewer**:
I apology that this bug is introduced by #49850 :(
@thockin @smarterclayton @gnufied
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51805, 51725, 50925, 51474, 51638)
Allow custom client verbs to be generated using client-gen
This change will allow to define custom verbs for resources using the following new tag:
```
// +genclient:method=Foo,verb=create,subresource=foo,input=Bar,output=k8s.io/pkg/api.Blah
```
This will generate client method `Foo(bar *Bar) (*api.Blah, error)` (format depends on the particular verb type)
With this change we can add `UpdateScale()` and `GetScale()` into all scalable resources. Note that intention of this PR is not to fix the Scale(), but that is used as an example of this new capability.
Additionally this will also allow us to get rid of `// +genclient:noStatus` and fix guessing of the "updateStatus" subresource presence based on the existence of '.Status' field.
Basically you will have to add following into all types you want to generate `UpdateStatus()` for:
```
// +genclient:method=UpdateStatus,verb=update,subresource=status
```
This allows further extension of the client without writing an expansion (which proved to be pain to maintain and copy...). Also allows to customize native CRUD methods if needed (input/output types).
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51805, 51725, 50925, 51474, 51638)
Limit events accepted by API Server
**What this PR does / why we need it**:
This PR adds the ability to limit events processed by an API server. Limits can be set globally on a server, per-namespace, per-user, and per-source+object. This is needed to prevent badly-configured or misbehaving players from making a cluster unstable.
Please see https://github.com/kubernetes/community/pull/945.
**Release Note:**
```release-note
Adds a new alpha EventRateLimit admission control that is used to limit the number of event queries that are accepted by the API Server.
```
Automatic merge from submit-queue (batch tested with PRs 50579, 50875, 51797, 51807, 51803)
make url parsing in apiserver configurable
We have known cases where the attributes for a request are assigned differently. The kubelet is one example. This makes the value an interface, not a struct, and provides a hook for (non-default) users to override it.
Kubernetes Submit Queue