Automatic merge from submit-queue (batch tested with PRs 51438, 52182, 51607, 47912, 51595). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Ensure we log the flag apiserver starts with.
Trying to make sure we always log the flags an instance of apiserver
starts with.
This can be especially valuable for emailed logs or e2e/kubemark tests.
**What this PR does / why we need it**: Ensures we log the flags an apiserver was started with.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #none
**Special notes for your reviewer**:
**Release note**:```release-note NONE
```
Automatic merge from submit-queue (batch tested with PRs 50392, 52108, 52083, 52134, 51526). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Do deep copy instead of to and from JSON encoding
**What this PR does / why we need it**:
Unstructured converter encodes to JSON and then parses the result into a new object. For `Unstructured` this can be avoided by directly doing a deep copy. It is an optimization.
**Special notes for your reviewer**:
#47889 is somewhat related.
**Release note**:
```release-note
NONE
```
/sig api-machinery
Automatic merge from submit-queue (batch tested with PRs 51929, 52015, 51906, 52069, 51542). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
move specialDefaultResourcePrefixes out of vendor/k8s.io/apiserver
just a clean-up, fixes TODO: move out of this package, it is not generic
@sttts PTAL
/assign @sttts
Automatic merge from submit-queue (batch tested with PRs 51031, 51705, 51888, 51727, 51684). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Avoid hard-coding location for auto-generated cert files
**What this PR does / why we need it**:
This fixes the confusing/incorrect help message for "--tls-cert-file" in apiserver package.
**Which issue this PR fixes**: fixes#51887
**Special notes for your reviewer**:
Please advise if a release note is needed.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52843, 52710, 52821, 52844). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
update-staging-godeps: only mangle staging repos in staging Godeps.json
- this re-adds non-staging k8s.io/* repos to the staging Godeps.jsons
- x-outs instead of removing of staging dependencies in order to get a
precise trigger for a complete godep restore+save run in the publisher
bot.
The first breaks k8s.io/kube-aggregator's staging export.
The second potentially leads to inconsistent godeps in our exported staging repos.
Automatic merge from submit-queue (batch tested with PRs 48406, 52819). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Made image as deliberately optional in v1 Container struct.
**What this PR does / why we need it**:
Revert https://github.com/kubernetes/kubernetes/pull/47246/
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Mark image as deliberately optional in v1 Container struct. Many objects in the Kubernetes API inherit the container struct and only Pods require the field to be set.
```
Automatic merge from submit-queue (batch tested with PRs 52477, 52790, 52798). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
conversion-gen: make staging dirs independent of living in vendor/
The `+k8s:conversion-gen` tags included the package directory inside of kube's vendor dir. This
makes them invalid when we publish staging repos.
Without this PR our sample-apiserver example code-generation is broken once published.
Automatic merge from submit-queue (batch tested with PRs 51869, 52281). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
apiextensions: fix conversion of CRD schema
- [x] Fix conversion of CRD schema to go-openapi types.
- [x] Add roundtrip tests for this conversion: https://github.com/kubernetes/kubernetes/pull/52793. Split into another PR since it touches godeps.
**Release note**:
```release-note
NONE
```
/cc @sttts
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
k8s.io/code-generator: hide gen test output from go tools
The underscore in front of _test makes the go tools (including godep) to
ignore those packages by default. We do not want to create dependencies
to client-go and friends through the _test output.
The underscore in front of _test makes the go tools (including godep) to
ignore those packages by default. We do not want to create dependencies
to client-go and friend through the _test output.
Automatic merge from submit-queue (batch tested with PRs 52500, 52533). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Cronjob beta defaults
**What this PR does / why we need it**:
I forget to set the defaults in beta for (successful|failed)JobsHistoryLimit so doing this now. It's already too late for 1.8 so this goes into 1.9.
**Release note**:
```release-note
Set defaults for successfulJobsHistoryLimit (3) and failedJobsHistoryLimit (1) in batch/v1beta1.CronJobs
```
Remove the MonitorRequest method and replace with a method that takes
request.RequestInfo, which is our default way to talk about API objects.
Preserves existing semantics for calls.
Default http.ProxyFromEnvironment uses uppper case proxy
environment variables first, and if they are not defined,
tries lower case. For NewProxierWithNoProxyCIDR we should
provide similar user experience.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
api: update progressdeadlineseconds comment for deployments
@kubernetes/sig-apps-api-reviews we may never end up doing autorollback - this drops the comment from the pds field for now
Automatic merge from submit-queue (batch tested with PRs 52176, 43152). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
etcd3 store: retry with live object on conflict if there was a suggestion
Retry with a live object instead of the cached version if the watch
cache receives a conflict trying to do the update.
Fixes#41892
In GuaranteedUpdate, if it was called with a suggestion (e.g. via the
watch cache), and the suggested object is stale, perform a live lookup
and then retry the update.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
Automatic merge from submit-queue
Increase sliding window to 5hr for request_latencies metric
We're seeing high latency values for a couple of types of api calls in our density test (ref https://github.com/kubernetes/kubernetes/issues/51899). And we're recording values from only the last 1 hour in the metric (as @wojtek-t told me offline) - so our test result is pretty much counting only the calls during the delete phase.
cc @kubernetes/sig-scalability-misc @kubernetes/sig-api-machinery-misc @gmarek
Automatic merge from submit-queue (batch tested with PRs 51824, 50476, 52451, 52009, 52237)
Improve apiserver metrics reporting
Normalize "WATCHLIST" to "WATCH", add "scope" to the other metrics (listing 50k pods is != listing pods in a namespace), and add a new scope "resource" to cover individual resource calls.
This roughly aligns metrics with our ACL model (technically resource scope is GET, but POST to a subresource and POST to a namespace are not the same thing).
```release-note
WATCHLIST calls are now reported as WATCH verbs in prometheus for the apiserver_request_* series. A new "scope" label is added to all apiserver_request_* values that is either 'cluster', 'resource', or 'namespace' depending on which level the query is performed at.
```
Automatic merge from submit-queue (batch tested with PRs 51824, 50476, 52451, 52009, 52237)
Plumbing the proxy dialer to the webhook admission plugin
* Fixing https://github.com/kubernetes/kubernetes/issues/49987. Plumb the `Dial` function to the `transport.Config`
* Fixing https://github.com/kubernetes/kubernetes/issues/52366. Let the webhook admission plugin sets the `TLSConfg.ServerName`.
I tested it in my gke setup. I don't have time to implement an e2e test before 1.8 release. I think it's ok to add the test later, because *i)* the change only affects the alpha webhook admission feature, and *ii)* the webhook feature is unusable without the fix. That said, it's up to my reviewer to decide.
Filed https://github.com/kubernetes/kubernetes/issues/52368 for the missing e2e test.
( The second commit is https://github.com/kubernetes/kubernetes/pull/52372, which is just a cleanup of client configuration in e2e tests. It removed a function that marshalled the client config to json and then unmarshalled it. It is a prerequisite of this PR, because this PR added the `Dial` function to the config which is not json marshallable.)
```release-note
Fixed the webhook admission plugin so that it works even if the apiserver and the nodes are in two networks (e.g., in GKE).
Fixed the webhook admission plugin so that webhook author could use the DNS name of the service as the CommonName when generating the server cert for the webhook.
Action required:
Anyone who generated server cert for admission webhooks need to regenerate the cert. Previously, when generating server cert for the admission webhook, the CN value doesn't matter. Now you must set it to the DNS name of the webhook service, i.e., `<service.Name>.<service.Namespace>.svc`.
```
Automatic merge from submit-queue
use specified discovery information if possible
Fixes https://github.com/kubernetes/kubernetes/issues/49948
This uses the available discovery information if available, but it seems we never updated "normal" resources to show the singular name, so its often not available. I've left this code compatible.
@enisoc @ash2k
@kubernetes/sig-api-machinery-misc
```release-note
custom resources that use unconventional pluralization now work properly with kubectl and garbage collection
```
This commit adds an option for controlling request of compaction to
etcd3 from apiserver. There is a situation that apiserver cannot fully
own its etcd cluster (e.g. sharing it with canal). In such a case,
apiserver should have limited access in terms of etcd's auth
functionality so it don't have a priviledge to issue compaction
requests. It means that the compaction requests should be issued by
other component and apiserver's compaction requests are needless.
For such use cases, this commit adds a new flag
`storagebackend.Config.CompactionInterval`. If the flag is non 0,
apiserver issues the compaction requests like current behaviour (the
default is 5 minutes). If it is 0, apiserver doesn't issue the
requests. It can be configured with a newly added option of apiserver
`--etcd-compaction-interval`.
Automatic merge from submit-queue (batch tested with PRs 52227, 52120)
Fix discovery restmapper finding resources in non-preferred versions
Fixes: #52219
Also reverts behavioral changes to tests that version-qualified cronjobs to work around this issue.
The discovery rest mapper was only populating the priority rest mapper's search list with preferred groupversions.
That meant that if a resource existed in multiple non-preferred versions, AND did not exist in the preferred version (like cronjob, which only exists in v1beta2.batch and v2alpha1.batch, but not v1.batch), the priority restmapper would not find it in its group/version priority list, and would return an error.
```release-note
Fixed an issue looking up cronjobs when they existed in more than one API version
```
The first one being RecommendedPluginOrder the second one being DefaultOffPlugins.
In case a cluster-admin did not provide plugin names they will be derived from these fields.
Automatic merge from submit-queue
Restore OWNERS file for k8s.io/metrics
The owners file for k8s.io/metrics somehow got lost. This restores it
to its contents on the "legacy" branch of k8s.io/metrics.
```release-note
NONE
```
Automatic merge from submit-queue
newline to separate unimplemented TaintEffectNoScheduleNoAdmit
**What this PR does / why we need it**:
Unimplemented `TaintEffectNoScheduleNoAdmit ` should not be treated as comments of `TaintEffectNoExecute `
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
xref #49530
**Special notes for your reviewer**:
/assign @k82cn
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 52047, 52063, 51528)
Improve dynamic kubelet config e2e node test and fix bugs
Rather than just changing the config once to see if dynamic kubelet
config at-least-sort-of-works, this extends the test to check that the
Kubelet reports the expected Node condition and the expected configuration
values after several possible state transitions.
Additionally, this adds a stress test that changes the configuration 100
times. It is possible for resource leaks across Kubelet restarts to
eventually prevent the Kubelet from restarting. For example, this test
revealed that cAdvisor's leaking journalctl processes (see:
https://github.com/google/cadvisor/issues/1725) could break dynamic
kubelet config. This test will help reveal these problems earlier.
This commit also makes better use of const strings and fixes a few bugs
that the new testing turned up.
Related issue: #50217
I had been sitting on this until the cAdvisor fix merged in #51751, as these tests fail without that fix.
**Release note**:
```release-note
NONE
```
Currently setting watch cache size for a given resource does not disable
the watch cache. This commit adds a new `default-watch-cache-size` flag
to map to the existing field, and refactors how watch cache sizes are
calculated to bring all of the code into one place. It also adds debug
logging to startup to allow us to verify watch cache enablement in
production.
Trying to make sure we always log the flags an instance of apiserver
starts with.
This can be especially valuable for emailed logs or e2e/kubemark tests.
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
Fill in creationtimestamp in audit events
**What this PR does / why we need it**:
This is fixing null creationtimestamp in audit events.
@sttts @crassirostris like we've talked earlier today
**Release note**:
```release-note
none
```
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
A policy with 0 rules should return an error
**Which issue this PR fixes**
[isuue#51565](https://github.com/kubernetes/kubernetes/issues/51565)
**Release note**:
```
An audit policy file with 0 rule returns an error.
```
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
apiservers: stratify versioned informer construction
The versioned share informer factory has been part of the GenericApiServer config,
but its construction depended on other fields of that config (e.g. the loopback
client config). Hence, the order of changes to the config mattered.
This PR stratifies this by moving the SharedInformerFactory from the generic Config
to the CompleteConfig struct. Hence, it is only filled during completion when it is
guaranteed that the loopback client config is set.
While doing this, the CompletedConfig construction is made more type-safe again,
i.e. the use of SkipCompletion() is considereably reduced. This is archieved by
splitting the derived apiserver Configs into the GenericConfig and the ExtraConfig
part. Then the completion is structural again because CompleteConfig is again
of the same structure: generic CompletedConfig and local completed ExtraConfig.
Fixes#50661.
Automatic merge from submit-queue (batch tested with PRs 52091, 52071)
Log a warning when --audit-policy-file not passed to apiserver
When audit backend provided, but --audit-policy-file is not passed, leave a info in log.
**Release note**:
```
Log a warning when --audit-policy-file not passed to apiserver
```
Automatic merge from submit-queue
fix format of forbidden messages
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#51813
**Special notes for your reviewer**:
/assign @deads2k @liggitt
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 48552, 51876)
Disable default paging in list watches
For 1.8 this will be off by default. In 1.9 it will be on by default.
Add tests and rename some fields to use the `chunking` terminology.
Note that the pager may be used for other things besides chunking.
Follow on to #48921, we left the field on to get some exercise in the normal code paths, but needs to be disabled for 1.8.
@liggitt let's merge on wednesday.
Rather than just changing the config once to see if dynamic kubelet
config at-least-sort-of-works, this extends the test to check that the
Kubelet reports the expected Node condition and the expected configuration
values after several possible state transitions.
Additionally, this adds a stress test that changes the configuration 100
times. It is possible for resource leaks across Kubelet restarts to
eventually prevent the Kubelet from restarting. For example, this test
revealed that cAdvisor's leaking journalctl processes (see:
https://github.com/google/cadvisor/issues/1725) could break dynamic
kubelet config. This test will help reveal these problems earlier.
This commit also makes better use of const strings and fixes a few bugs
that the new testing turned up.
Related issue: #50217
Automatic merge from submit-queue (batch tested with PRs 52097, 52054)
Provide field info in storage configuration
Makes debugging how storage was configured difficult
@kubernetes/sig-api-machinery-bugs
Automatic merge from submit-queue (batch tested with PRs 51239, 51644, 52076)
code-generator/protobuf: cut-off kubernetes specifics
The package list moved to hack/update-generated-protobuf-dockerized.sh.
This make the protobuf generator usable outside of kube.
Automatic merge from submit-queue
Fix proxied request-uri to be valid HTTP requests
Fixes#52022, introduced in 1.7. Stringifying/re-parsing the URL masked that the path was not constructed with a leading `/` in the first place.
This makes upgrade requests proxied to pods/services via the API server proxy subresources be valid HTTP requests
```release-note
Fixes an issue with upgrade requests made via pod/service/node proxy subresources sending a non-absolute HTTP request-uri to backends
```
For 1.8 this will be off by default. In 1.9 it will be on by default.
Add tests and rename some fields to use the `chunking` terminology.
Note that the pager may be used for other things besides chunking.
Automatic merge from submit-queue (batch tested with PRs 51956, 50708)
Move autoscaling/v2 from alpha1 to beta1
This graduates autoscaling/v2alpha1 to autoscaling/v2beta1. The move is more-or-less just a straightforward rename.
Part of kubernetes/features#117
```release-note
v2 of the autoscaling API group, including improvements to the HorizontalPodAutoscaler, has moved from alpha1 to beta1.
```
Automatic merge from submit-queue (batch tested with PRs 49133, 51557, 51749, 50842, 52018)
enhance unit tests of advance audit feature
This change addresses comments from @crassirostris
https://github.com/kubernetes/kubernetes/pull/49115#discussion_r133416080
It does three things:
1. use auditinternal for unit test in filter stage
2. add a seperate unit test for Audit-ID http header
3. add unit test for audit log backend
**Release note**:
```
NONE
```
Charge object count when object is created, no matter if the object is
initialized or not.
Charge the remaining quota when the object is initialized.
Also, checking initializer.Pending and initializer.Result when
determining if an object is initialized. We didn't need to check them
because before 51082, having 0 pending initializer and nil
initializers.Result is invalid.
Automatic merge from submit-queue
set AdvancedAuditing feature gate to true by default
All feature commits are merged. The types are updated already to beta. This only enable the feature gate by default.
**Release note**:
```
Promote the AdvancedAuditing feature to beta and enable the feature gate by default.
```
This change does three things:
1. use auditinternal for unit test in filter stage
2. add a seperate unit test for Audit-ID http header
3. add unit test for audit log backend
Automatic merge from submit-queue (batch tested with PRs 51603, 51653)
Graduate metrics/v1alpha1 to v1beta1
This introduces v1beta1 of the resource metrics API, previously in alpha.
The v1alpha1 version remains for compatibility with the Heapster legacy version
of the resource metrics API, which is compatible with the v1alpha1 version. It also
renames the v1beta1 version to `resource-metrics.metrics.k8s.io`.
The HPA controller's REST clients (but not the legacy client) have been migrated as well.
Part of kubernetes/features#118.
```release-note
Migrate the metrics/v1alpha1 API to metrics/v1beta1. The HorizontalPodAutoscaler
controller REST client now uses that version. For v1beta1, the API is now known as
resource-metrics.metrics.k8s.io.
```
Automatic merge from submit-queue (batch tested with PRs 51733, 51838)
Relax update validation of uninitialized pod
Split from https://github.com/kubernetes/kubernetes/pull/50344
Fix https://github.com/kubernetes/kubernetes/issues/47837
* Let the podStrategy to only call `validation.ValidatePod()` if the old pod is not initialized, so fields are mutable.
* Let the podStatusStrategy refuse updates if the old pod is not initialized.
cc @smarterclayton
```release-note
Pod spec is mutable when the pod is uninitialized. The apiserver requires the pod spec to be valid even if it's uninitialized. Updating the status field of uninitialized pods is invalid.
```
Automatic merge from submit-queue
Improve APIService auto-registration for HA/upgrade scenarios
Fixes#51912
Required for 1.8 due to impact on HA upgrades.
/assign @deads2k
cc @kubernetes/sig-api-machinery-bugs
```release-note
Fixes an issue with APIService auto-registration affecting rolling HA apiserver restarts that add or remove API groups being served.
```
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Bug Fix - Adding an allowed address pair wipes port security groups
**What this PR does / why we need it**:
Fix for cloud routes enabled instances will have their security groups
removed when the allowed address pair is added to the instance's port.
Upstream bug report is in:
https://github.com/gophercloud/gophercloud/issues/509
Upstream bug fix is in:
https://github.com/gophercloud/gophercloud/pull/510
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#51755
**Special notes for your reviewer**:
Just an fix in vendored code. minimal changes needed in OpenStack cloud provider
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Build test targets for all server platforms
**What this PR does / why we need it**:
🤦
I really should have checked this before code freeze, but tbh forgot it in the rush. Also I thought this was the case already...
As part of https://github.com/kubernetes/features/issues/288; these binaries should be built for all server platforms indeed.
This is just a straightforward add to that list.
Can we please get this into v1.8?
There is virtually no risk involved here really...
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Build test targets for all server platforms
```
@ixdy @jdumars @mkumatag
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Add EgressRule to NetworkPolicy
**What this PR does / why we need it**:
Add EgressRule to NetworkPolicy
**Which issue this PR fixes**: fixes#50453
**Special notes for your reviewer**:
- Please take a look at the comments for the various types. I tried to mimic some of the language used in the Ingress comments, but I may have mangled some sentences.
- Let me know if I should add some test cases for validation. I have 2-3, and did not think it was necessary to replicate each case already covered in ingress.
**Release note**:
```
Add egress policies to NetworkPolicy
```
Automatic merge from submit-queue (batch tested with PRs 51186, 50350, 51751, 51645, 51837)
Update Cadvisor Dependency
Fixes: https://github.com/kubernetes/kubernetes/issues/51832
This is the worst dependency update ever...
The root of the problem is the [name change of Sirupsen -> sirupsen](https://github.com/sirupsen/logrus/issues/570#issuecomment-313933276). This means that in order to update cadvisor, which venders the lowercase, we need to update all dependencies to use the lower-cased version. With that being said, this PR updates the following packages:
`github.com/docker/docker`
- `github.com/docker/distribution`
- `github.com/opencontainers/go-digest`
- `github.com/opencontainers/image-spec`
- `github.com/opencontainers/runtime-spec`
- `github.com/opencontainers/selinux`
- `github.com/opencontainers/runc`
- `github.com/mrunalp/fileutils`
- `golang.org/x/crypto`
- `golang.org/x/sys`
- `github.com/docker/go-connections`
- `github.com/docker/go-units`
- `github.com/docker/libnetwork`
- `github.com/docker/libtrust`
- `github.com/sirupsen/logrus`
- `github.com/vishvananda/netlink`
`github.com/google/cadvisor`
- `github.com/euank/go-kmsg-parser`
`github.com/json-iterator/go`
Fixed https://github.com/kubernetes/kubernetes/issues/51832
```release-note
Fix journalctl leak on kubelet restart
Fix container memory rss
Add hugepages monitoring support
Fix incorrect CPU usage metrics with 4.7 kernel
Add tmpfs monitoring support
```
Automatic merge from submit-queue
Add client side event spam filtering
**What this PR does / why we need it**:
Add client side event spam filtering to stop excessive traffic to api-server from internal cluster components.
this pr defines a per source+object event budget of 25 burst with refill of 1 every 5 minutes.
i tested this pr on the following scenarios:
**Scenario 1: Node with 50 crash-looping pods**
```
$ create 50 crash-looping pods on a single node
$ kubectl run bad --image=busybox --replicas=50 --command -- derekisbad
```
Before:
* POST events with peak of 1.7 per second, long-tail: 0.2 per second
* PATCH events with peak of 5 per second, long-tail: 5 per second
After:
* POST events with peak of 1.7 per second, long-tail: 0.2 per second
* PATCH events with peak of 3.6 per second, long-tail: 0.2 per second
Observation:
* https://github.com/kubernetes/kubernetes/pull/47462 capped the number of total events in the long-tail as expected, but did nothing to improve total spam of master.
**Scenario 2: replication controller limited by quota**
```
$ kubectl create quota my-quota --hard=pods=1
$ kubectl run nginx --image=nginx --replicas=50
```
Before:
* POST events not relevant as aggregation worked well here.
* PATCH events with peak and long-tail of 13.6 per second
After:
* POST events not relevant as aggregation worked well here.
* PATCH events with peak: .35 per second, and long-tail of 0
**Which issue this PR fixes**
fixes https://github.com/kubernetes/kubernetes/issues/47366
**Special notes for your reviewer**:
this was a significant problem in a kube 1.5 cluster we are running where events were co-located in a single etcd. this cluster was normal to have larger numbers of unhealty pods as well as denial by quota.
**Release note**:
```release-note
add support for client-side spam filtering of events
```
Automatic merge from submit-queue
Provide a way to omit Event stages in audit policy
This provide a way to omit some stages for each audit policy rule.
For example:
```
apiVersion: audit.k8s.io/v1beta1
kind: Policy
- level: Metadata
resources:
- group: "rbac.authorization.k8s.io"
resources: ["roles"]
omitStages:
- "RequestReceived"
```
RequestReceived stage will not be emitted to audit backends with previous config.
**Release note**:
```
None
```
#
Updates https://github.com/kubernetes/kubernetes/issues/48561
This provide a way to omit some stages for each audit policy rule.
For example:
apiVersion: audit.k8s.io/v1beta1
kind: Policy
- level: Metadata
resources:
- group: "rbac.authorization.k8s.io"
resources: ["roles"]
omitStages:
- "RequestReceived"
RequestReceived stage will not be emitted to audit backends with
previous config.
Introduce feature gate for expanding PVs
Add a field to SC
Add new Conditions and feature tag pvc update
Add tests for size update via feature gate
register the resize admission plugin
Update golint failures
Automatic merge from submit-queue
Remove deprecated init-container in annotations
fixes#50655fixes#51816closes#41004fixes#51816
Builds on #50654 and drops the initContainer annotations on conversion to prevent bypassing API server validation/security and targeting version-skewed kubelets that still honor the annotations
```release-note
The deprecated alpha and beta initContainer annotations are no longer supported. Init containers must be specified using the initContainers field in the pod spec.
```
Automatic merge from submit-queue
Workloads deprecation 1.8
**What this PR does / why we need it**: This PR deprecates the Deployment, ReplicaSet, and DaemonSet kinds in the extensions/v1beta1 group version and the StatefulSet, Deployment, and ControllerRevision kinds in the apps/v1beta1 group version. The Deployment, ReplicaSet, DaemonSet, StatefuSet, and ControllerRevision kinds in the apps/v1beta2 group version are now the current version.
xref kubernetes/features#353
```release-note
The Deployment, DaemonSet, and ReplicaSet kinds in the extensions/v1beta1 group version are now deprecated, as are the Deployment, StatefulSet, and ControllerRevision kinds in apps/v1beta1. As they will not be removed until after a GA version becomes available, you may continue to use these kinds in existing code. However, all new code should be developed against the apps/v1beta2 group version.
```
Automatic merge from submit-queue (batch tested with PRs 51819, 51706, 51761, 51818, 51500)
fix kube-proxy panic because of nil sessionAffinityConfig
**What this PR does / why we need it**:
fix kube-proxy panic because of nil sessionAffinityConfig
**Which issue this PR fixes**: closes#51499
**Special notes for your reviewer**:
I apology that this bug is introduced by #49850 :(
@thockin @smarterclayton @gnufied
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51805, 51725, 50925, 51474, 51638)
Allow custom client verbs to be generated using client-gen
This change will allow to define custom verbs for resources using the following new tag:
```
// +genclient:method=Foo,verb=create,subresource=foo,input=Bar,output=k8s.io/pkg/api.Blah
```
This will generate client method `Foo(bar *Bar) (*api.Blah, error)` (format depends on the particular verb type)
With this change we can add `UpdateScale()` and `GetScale()` into all scalable resources. Note that intention of this PR is not to fix the Scale(), but that is used as an example of this new capability.
Additionally this will also allow us to get rid of `// +genclient:noStatus` and fix guessing of the "updateStatus" subresource presence based on the existence of '.Status' field.
Basically you will have to add following into all types you want to generate `UpdateStatus()` for:
```
// +genclient:method=UpdateStatus,verb=update,subresource=status
```
This allows further extension of the client without writing an expansion (which proved to be pain to maintain and copy...). Also allows to customize native CRUD methods if needed (input/output types).
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51805, 51725, 50925, 51474, 51638)
Limit events accepted by API Server
**What this PR does / why we need it**:
This PR adds the ability to limit events processed by an API server. Limits can be set globally on a server, per-namespace, per-user, and per-source+object. This is needed to prevent badly-configured or misbehaving players from making a cluster unstable.
Please see https://github.com/kubernetes/community/pull/945.
**Release Note:**
```release-note
Adds a new alpha EventRateLimit admission control that is used to limit the number of event queries that are accepted by the API Server.
```
Automatic merge from submit-queue (batch tested with PRs 50579, 50875, 51797, 51807, 51803)
make url parsing in apiserver configurable
We have known cases where the attributes for a request are assigned differently. The kubelet is one example. This makes the value an interface, not a struct, and provides a hook for (non-default) users to override it.
Automatic merge from submit-queue (batch tested with PRs 50579, 50875, 51797, 51807, 51803)
oidc auth: make the OIDC claims prefix configurable
Add the following flags to control the prefixing of usernames and
groups authenticated using OpenID Connect tokens.
--oidc-username-prefix
--oidc-groups-prefix
```release-note
The OpenID Connect authenticator can now use a custom prefix, or omit the default prefix, for username and groups claims through the --oidc-username-prefix and --oidc-groups-prefix flags. For example, the authenticator can map a user with the username "jane" to "google:jane" by supplying the "google:" username prefix.
```
Closes https://github.com/kubernetes/kubernetes/issues/50408
Ref https://github.com/kubernetes/kubernetes/issues/31380
cc @grillz @kubernetes/sig-auth-pr-reviews @thomastaylor312 @gtaylor
Automatic merge from submit-queue
Fixes kubernetes/kubernetes#29271: accept prefixed namespaces
**What this PR does / why we need it**: `kubectl get namespaces -o name` outputs the names of all namespaces, prefixed with `namespaces/`. This changeset allows these namespace names to be passed directly back in to `kubectl` via the `-n` flag without reprocessing them to remove `namespaces/`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#29271
**Special notes for your reviewer**:
**Release note**:
```NONE
```
Automatic merge from submit-queue
Switch audit output to v1beta1
This PR adds two switches to pick preferred version for webhook and log backends, and it switches to use `audit.k8s.io/v1beta1` as default for both.
@sttts @crassirostris ptal
**Release note**:
```release-note
Switch to audit.k8s.io/v1beta1 in audit.
```
Remove APIService OpenAPI spec when it is deleted
Add eTag support and returning httpStatus to OpenAPI spec downloader
Update aggregated OpenAPI spec periodically
Use delegate chain
Refactor OpenAPI aggregator to have separate controller and aggregation function
Enable OpenAPI spec for extensions api server
Do not filter paths. higher priority specs wins the conflicting paths
Move OpenAPI aggregation controller to pkg/controller/openapi
Automatic merge from submit-queue (batch tested with PRs 50602, 51561, 51703, 51748, 49142)
expose discovery information on scalable resources
Builds on https://github.com/kubernetes/kubernetes/pull/49971 and provides the GroupVersion information that can be used by a dynamic scale client.
@kubernetes/sig-api-machinery-pr-reviews
@foxish @DirectXMan12 since you both asked for it.
Automatic merge from submit-queue (batch tested with PRs 51301, 50497, 50112, 48184, 50993)
audit newest impersonated user info in the ResponseStarted, ResponseComplete audit stage
Impersonation will automatically add system:authenticated, system:serviceaccounts group to the impersonated user info. This pr use the newest impersonated user info in the second audit event. This will help users to debug rbac problems.
**Release note**:
```
[advanced audit] audit newest impersonated user info in the ResponseStarted, ResponseComplete audit stage
```
@liggitt @sttts
Automatic merge from submit-queue (batch tested with PRs 51301, 50497, 50112, 48184, 50993)
Introduce new flag "--include-uninitialized" to kubectl
**What this PR does / why we need it**:
Introduce `--include-uninitialized` as a global flag to kubectl
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#49035
**Special notes for your reviewer**:
/assign @caesarxuchao @smarterclayton @ahmetb @deads2k
**Release note**:
```release-note
Add flag "--include-uninitialized" to kubectl annotate, apply, edit-last-applied, delete, describe, edit, get, label, set. "--include-uninitialized=true" makes kubectl commands apply to uninitialized objects, which by default are ignored if the names of the objects are not provided. "--all" also makes kubectl commands apply to uninitialized objects. Please see the [initializer](https://kubernetes.io/docs/admin/extensible-admission-controllers/) doc for more details.
```
Automatic merge from submit-queue (batch tested with PRs 51553, 51538, 51663, 51069, 51737)
Consistent Names for ControllerRevisions, ReplicaSets, and objects using GenerateName
**What this PR does / why we need it**:
Adds the rand.SafeEncodeString function and uses this function to generate names for ReplicaSets and ControllerRevisions.
```release-note
The names generated for ControllerRevision and ReplicaSet are consistent with the GenerateName functionality of the API Server and will not contain "bad words".
```
Automatic merge from submit-queue (batch tested with PRs 51583, 51283, 51374, 51690, 51716)
Unify initializer name validation
Unify the validation rules on initializer names. Fix https://github.com/kubernetes/kubernetes/issues/51843.
```release-note
Action required: validation rule on metadata.initializers.pending[x].name is tightened. The initializer name needs to contain at least three segments separated by dots. If you create objects with pending initializers, (i.e., not relying on apiserver adding pending initializers according to initializerconfiguration), you need to update the initializer name in existing objects and in configuration files to comply to the new validation rule.
```
Automatic merge from submit-queue
Do not mutate original UnstructuredList even temporarily to avoid data races
**What this PR does / why we need it**:
See #51521.
**Which issue this PR fixes**:
Fixes#51521.
**Release note**:
```release-note
NONE
```
/kind bug
/sig api-machinery
Automatic merge from submit-queue (batch tested with PRs 50832, 51119, 51636, 48921, 51712)
apiextensions: add maximum for validation
Missed the `Maximum` field for validation. Adding it now.
**Release note**:
```release-note
NONE
```
/cc @sttts
Automatic merge from submit-queue (batch tested with PRs 51335, 51364, 51130, 48075, 50920)
Graduate custom metrics API to v1beta1
This graduates custom-metrics.metrics.k8s.io/v1alpha1 to custom-metrics.metrics.k8s.io/v1beta1. The move is more-or-less just a straightforward rename.
Part of kubernetes/features#117 and kubernetes/features#118
```release-note
the custom metrics API (custom-metrics.metrics.k8s.io) has moved from v1alpha1 to v1beta1
```
Automatic merge from submit-queue (batch tested with PRs 51335, 51364, 51130, 48075, 50920)
[API] Feature/job failure policy
**What this PR does / why we need it**: Implements the Backoff policy and failed pod limit defined in https://github.com/kubernetes/community/pull/583
**Which issue this PR fixes**:
fixes#27997, fixes#30243
**Special notes for your reviewer**:
This is a WIP PR, I updated the api batchv1.JobSpec in order to prepare the backoff policy implementation in the JobController.
**Release note**:
```release-note
Add backoff policy and failed pod limit for a job
```
Automatic merge from submit-queue (batch tested with PRs 51335, 51364, 51130, 48075, 50920)
Add the possibility to set return values for the FakeDiscovery implementation
**What this PR does / why we need it**:
As an user of the fake clientset (with the fake discovery), I want to be able to set the fake server's version on demand like this for example:
```go
func TestFakingServerVersion(t *testing.T) {
client := fakeclientset.NewSimpleClientset()
fakeDiscovery, ok := client.Discovery().(*fakediscovery.FakeDiscovery)
if !ok {
t.Fatalf("couldn't convert Discovery() to *FakeDiscovery")
}
testGitCommit := "v1.0.0"
fakeDiscovery.FakedServerVersion = &version.Info{
GitCommit: testGitCommit,
}
sv, err := client.Discovery().ServerVersion()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if sv.GitCommit != testGitCommit {
t.Fatalf("unexpected faked discovery return value: %q", sv.GitCommit)
}
}
```
This PR makes that possible, in wait for a more sophisticated FakeDiscovery implementation generally.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue
basic logging and additional documentation for healthz installer
@pmorie upstream logging
- InstallHandler is the public interface through which all interaction
occurs.
- It is good to know whether the default ping is occurring to know due
to manual installation or automatic installation.
- It is good to know how many handlers are installed to see whether
code changes are taking effect.
**What this PR does / why we need it**:
add logging for external users of the genericapiserver
**Special notes for your reviewer**:
This should run exactly once at startup.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45724, 48051, 46444, 51056, 51605)
Add v1 API as a default conversion peer
Some of the APIs call do not this out, and a partial build produces
wrong results.
Automatic merge from submit-queue (batch tested with PRs 45724, 48051, 46444, 51056, 51605)
Mount propagation in kubelet
Together with #45724 it implements mount propagation as proposed in https://github.com/kubernetes/community/pull/659
There is:
- New alpha annotation that allows user to explicitly set propagation mode for each `VolumeMount` in pod containers (to be replaced with real `VolumeMount.Propagation` field during beta) + validation + tests. "Private" is the default one (= no change to existing pods).
I know about proposal for real API fields for alpha feature in https://docs.google.com/document/d/1wuoSqHkeT51mQQ7dIFhUKrdi3-1wbKrNWeIL4cKb9zU/edit, but it seems it's not implemented yet. It would save me quite lot of code and ugly annotation.
- Updated CRI API to transport chosen propagation to Docker.
- New `kubelet --experimental-mount-propagation` option to enable the previous bullet without modifying types.go (worked around with changing `KubeletDeps`... not nice, but it's better than adding a parameter to `NewMainKubelet` and removing it in the next release...)
```release-note
kubelet has alpha support for mount propagation. It is disabled by default and it is there for testing only. This feature may be redesigned or even removed in a future release.
```
@derekwaynecarr @dchen1107 @kubernetes/sig-node-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 45724, 48051, 46444, 51056, 51605)
Add selfsubjectrulesreview in authorization
**What this PR does / why we need it**:
**Which issue this PR fixes**: fixes#47834#31292
**Special notes for your reviewer**:
**Release note**:
```release-note
Add selfsubjectrulesreview API for allowing users to query which permissions they have in a given namespace.
```
/cc @deads2k @liggitt
The EventRateLimit plug-in limits the number of events that the API Server
will accept in a given time period. It allows for server-wide, per-namespace,
per-user,and per-source+object rate limiting.
Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)
Split APIVersion into APIGroup and APIVersion in audit events
audit.Event.ObjectRef.APIVersion currently holds both the the API group and
version, separated by a /. This change break these out into separate fields.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51632, 51055, 51676, 51560, 50007)
fixing a typo in staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go
**What this PR does / why we need it**:
Fix a typo in `staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
`NONE`
Add a feature gate in the apiserver to control whether paging can be
used. Add controls to the storage factory that allow it to be disabled
per resource. Use a JSON encoded continuation token that can be
versioned. Create a 410 error if the continuation token is expired.
Adds GetContinue() to ListMeta.
Clayton Coleman