Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
k8s.io/code-generator: hide gen test output from go tools
The underscore in front of _test makes the go tools (including godep) to
ignore those packages by default. We do not want to create dependencies
to client-go and friends through the _test output.
The underscore in front of _test makes the go tools (including godep) to
ignore those packages by default. We do not want to create dependencies
to client-go and friend through the _test output.
Automatic merge from submit-queue (batch tested with PRs 52500, 52533). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Cronjob beta defaults
**What this PR does / why we need it**:
I forget to set the defaults in beta for (successful|failed)JobsHistoryLimit so doing this now. It's already too late for 1.8 so this goes into 1.9.
**Release note**:
```release-note
Set defaults for successfulJobsHistoryLimit (3) and failedJobsHistoryLimit (1) in batch/v1beta1.CronJobs
```
Remove the MonitorRequest method and replace with a method that takes
request.RequestInfo, which is our default way to talk about API objects.
Preserves existing semantics for calls.
Default http.ProxyFromEnvironment uses uppper case proxy
environment variables first, and if they are not defined,
tries lower case. For NewProxierWithNoProxyCIDR we should
provide similar user experience.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
api: update progressdeadlineseconds comment for deployments
@kubernetes/sig-apps-api-reviews we may never end up doing autorollback - this drops the comment from the pds field for now
Automatic merge from submit-queue (batch tested with PRs 52176, 43152). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
etcd3 store: retry with live object on conflict if there was a suggestion
Retry with a live object instead of the cached version if the watch
cache receives a conflict trying to do the update.
Fixes#41892
In GuaranteedUpdate, if it was called with a suggestion (e.g. via the
watch cache), and the suggested object is stale, perform a live lookup
and then retry the update.
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
Automatic merge from submit-queue
Increase sliding window to 5hr for request_latencies metric
We're seeing high latency values for a couple of types of api calls in our density test (ref https://github.com/kubernetes/kubernetes/issues/51899). And we're recording values from only the last 1 hour in the metric (as @wojtek-t told me offline) - so our test result is pretty much counting only the calls during the delete phase.
cc @kubernetes/sig-scalability-misc @kubernetes/sig-api-machinery-misc @gmarek
Automatic merge from submit-queue (batch tested with PRs 51824, 50476, 52451, 52009, 52237)
Improve apiserver metrics reporting
Normalize "WATCHLIST" to "WATCH", add "scope" to the other metrics (listing 50k pods is != listing pods in a namespace), and add a new scope "resource" to cover individual resource calls.
This roughly aligns metrics with our ACL model (technically resource scope is GET, but POST to a subresource and POST to a namespace are not the same thing).
```release-note
WATCHLIST calls are now reported as WATCH verbs in prometheus for the apiserver_request_* series. A new "scope" label is added to all apiserver_request_* values that is either 'cluster', 'resource', or 'namespace' depending on which level the query is performed at.
```
Automatic merge from submit-queue (batch tested with PRs 51824, 50476, 52451, 52009, 52237)
Plumbing the proxy dialer to the webhook admission plugin
* Fixing https://github.com/kubernetes/kubernetes/issues/49987. Plumb the `Dial` function to the `transport.Config`
* Fixing https://github.com/kubernetes/kubernetes/issues/52366. Let the webhook admission plugin sets the `TLSConfg.ServerName`.
I tested it in my gke setup. I don't have time to implement an e2e test before 1.8 release. I think it's ok to add the test later, because *i)* the change only affects the alpha webhook admission feature, and *ii)* the webhook feature is unusable without the fix. That said, it's up to my reviewer to decide.
Filed https://github.com/kubernetes/kubernetes/issues/52368 for the missing e2e test.
( The second commit is https://github.com/kubernetes/kubernetes/pull/52372, which is just a cleanup of client configuration in e2e tests. It removed a function that marshalled the client config to json and then unmarshalled it. It is a prerequisite of this PR, because this PR added the `Dial` function to the config which is not json marshallable.)
```release-note
Fixed the webhook admission plugin so that it works even if the apiserver and the nodes are in two networks (e.g., in GKE).
Fixed the webhook admission plugin so that webhook author could use the DNS name of the service as the CommonName when generating the server cert for the webhook.
Action required:
Anyone who generated server cert for admission webhooks need to regenerate the cert. Previously, when generating server cert for the admission webhook, the CN value doesn't matter. Now you must set it to the DNS name of the webhook service, i.e., `<service.Name>.<service.Namespace>.svc`.
```
Automatic merge from submit-queue
use specified discovery information if possible
Fixes https://github.com/kubernetes/kubernetes/issues/49948
This uses the available discovery information if available, but it seems we never updated "normal" resources to show the singular name, so its often not available. I've left this code compatible.
@enisoc @ash2k
@kubernetes/sig-api-machinery-misc
```release-note
custom resources that use unconventional pluralization now work properly with kubectl and garbage collection
```
Automatic merge from submit-queue (batch tested with PRs 52227, 52120)
Fix discovery restmapper finding resources in non-preferred versions
Fixes: #52219
Also reverts behavioral changes to tests that version-qualified cronjobs to work around this issue.
The discovery rest mapper was only populating the priority rest mapper's search list with preferred groupversions.
That meant that if a resource existed in multiple non-preferred versions, AND did not exist in the preferred version (like cronjob, which only exists in v1beta2.batch and v2alpha1.batch, but not v1.batch), the priority restmapper would not find it in its group/version priority list, and would return an error.
```release-note
Fixed an issue looking up cronjobs when they existed in more than one API version
```
The first one being RecommendedPluginOrder the second one being DefaultOffPlugins.
In case a cluster-admin did not provide plugin names they will be derived from these fields.
Automatic merge from submit-queue
Restore OWNERS file for k8s.io/metrics
The owners file for k8s.io/metrics somehow got lost. This restores it
to its contents on the "legacy" branch of k8s.io/metrics.
```release-note
NONE
```
Automatic merge from submit-queue
newline to separate unimplemented TaintEffectNoScheduleNoAdmit
**What this PR does / why we need it**:
Unimplemented `TaintEffectNoScheduleNoAdmit ` should not be treated as comments of `TaintEffectNoExecute `
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
xref #49530
**Special notes for your reviewer**:
/assign @k82cn
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 52047, 52063, 51528)
Improve dynamic kubelet config e2e node test and fix bugs
Rather than just changing the config once to see if dynamic kubelet
config at-least-sort-of-works, this extends the test to check that the
Kubelet reports the expected Node condition and the expected configuration
values after several possible state transitions.
Additionally, this adds a stress test that changes the configuration 100
times. It is possible for resource leaks across Kubelet restarts to
eventually prevent the Kubelet from restarting. For example, this test
revealed that cAdvisor's leaking journalctl processes (see:
https://github.com/google/cadvisor/issues/1725) could break dynamic
kubelet config. This test will help reveal these problems earlier.
This commit also makes better use of const strings and fixes a few bugs
that the new testing turned up.
Related issue: #50217
I had been sitting on this until the cAdvisor fix merged in #51751, as these tests fail without that fix.
**Release note**:
```release-note
NONE
```
Currently setting watch cache size for a given resource does not disable
the watch cache. This commit adds a new `default-watch-cache-size` flag
to map to the existing field, and refactors how watch cache sizes are
calculated to bring all of the code into one place. It also adds debug
logging to startup to allow us to verify watch cache enablement in
production.
Trying to make sure we always log the flags an instance of apiserver
starts with.
This can be especially valuable for emailed logs or e2e/kubemark tests.
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
Fill in creationtimestamp in audit events
**What this PR does / why we need it**:
This is fixing null creationtimestamp in audit events.
@sttts @crassirostris like we've talked earlier today
**Release note**:
```release-note
none
```
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
A policy with 0 rules should return an error
**Which issue this PR fixes**
[isuue#51565](https://github.com/kubernetes/kubernetes/issues/51565)
**Release note**:
```
An audit policy file with 0 rule returns an error.
```
Automatic merge from submit-queue (batch tested with PRs 51900, 51782, 52030)
apiservers: stratify versioned informer construction
The versioned share informer factory has been part of the GenericApiServer config,
but its construction depended on other fields of that config (e.g. the loopback
client config). Hence, the order of changes to the config mattered.
This PR stratifies this by moving the SharedInformerFactory from the generic Config
to the CompleteConfig struct. Hence, it is only filled during completion when it is
guaranteed that the loopback client config is set.
While doing this, the CompletedConfig construction is made more type-safe again,
i.e. the use of SkipCompletion() is considereably reduced. This is archieved by
splitting the derived apiserver Configs into the GenericConfig and the ExtraConfig
part. Then the completion is structural again because CompleteConfig is again
of the same structure: generic CompletedConfig and local completed ExtraConfig.
Fixes#50661.
Automatic merge from submit-queue (batch tested with PRs 52091, 52071)
Log a warning when --audit-policy-file not passed to apiserver
When audit backend provided, but --audit-policy-file is not passed, leave a info in log.
**Release note**:
```
Log a warning when --audit-policy-file not passed to apiserver
```
Automatic merge from submit-queue
fix format of forbidden messages
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#51813
**Special notes for your reviewer**:
/assign @deads2k @liggitt
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 48552, 51876)
Disable default paging in list watches
For 1.8 this will be off by default. In 1.9 it will be on by default.
Add tests and rename some fields to use the `chunking` terminology.
Note that the pager may be used for other things besides chunking.
Follow on to #48921, we left the field on to get some exercise in the normal code paths, but needs to be disabled for 1.8.
@liggitt let's merge on wednesday.
Rather than just changing the config once to see if dynamic kubelet
config at-least-sort-of-works, this extends the test to check that the
Kubelet reports the expected Node condition and the expected configuration
values after several possible state transitions.
Additionally, this adds a stress test that changes the configuration 100
times. It is possible for resource leaks across Kubelet restarts to
eventually prevent the Kubelet from restarting. For example, this test
revealed that cAdvisor's leaking journalctl processes (see:
https://github.com/google/cadvisor/issues/1725) could break dynamic
kubelet config. This test will help reveal these problems earlier.
This commit also makes better use of const strings and fixes a few bugs
that the new testing turned up.
Related issue: #50217
Automatic merge from submit-queue (batch tested with PRs 52097, 52054)
Provide field info in storage configuration
Makes debugging how storage was configured difficult
@kubernetes/sig-api-machinery-bugs
Automatic merge from submit-queue (batch tested with PRs 51239, 51644, 52076)
code-generator/protobuf: cut-off kubernetes specifics
The package list moved to hack/update-generated-protobuf-dockerized.sh.
This make the protobuf generator usable outside of kube.
Automatic merge from submit-queue
Fix proxied request-uri to be valid HTTP requests
Fixes#52022, introduced in 1.7. Stringifying/re-parsing the URL masked that the path was not constructed with a leading `/` in the first place.
This makes upgrade requests proxied to pods/services via the API server proxy subresources be valid HTTP requests
```release-note
Fixes an issue with upgrade requests made via pod/service/node proxy subresources sending a non-absolute HTTP request-uri to backends
```
For 1.8 this will be off by default. In 1.9 it will be on by default.
Add tests and rename some fields to use the `chunking` terminology.
Note that the pager may be used for other things besides chunking.
Automatic merge from submit-queue (batch tested with PRs 51956, 50708)
Move autoscaling/v2 from alpha1 to beta1
This graduates autoscaling/v2alpha1 to autoscaling/v2beta1. The move is more-or-less just a straightforward rename.
Part of kubernetes/features#117
```release-note
v2 of the autoscaling API group, including improvements to the HorizontalPodAutoscaler, has moved from alpha1 to beta1.
```
Automatic merge from submit-queue (batch tested with PRs 49133, 51557, 51749, 50842, 52018)
enhance unit tests of advance audit feature
This change addresses comments from @crassirostris
https://github.com/kubernetes/kubernetes/pull/49115#discussion_r133416080
It does three things:
1. use auditinternal for unit test in filter stage
2. add a seperate unit test for Audit-ID http header
3. add unit test for audit log backend
**Release note**:
```
NONE
```
Charge object count when object is created, no matter if the object is
initialized or not.
Charge the remaining quota when the object is initialized.
Also, checking initializer.Pending and initializer.Result when
determining if an object is initialized. We didn't need to check them
because before 51082, having 0 pending initializer and nil
initializers.Result is invalid.
Automatic merge from submit-queue
set AdvancedAuditing feature gate to true by default
All feature commits are merged. The types are updated already to beta. This only enable the feature gate by default.
**Release note**:
```
Promote the AdvancedAuditing feature to beta and enable the feature gate by default.
```
This change does three things:
1. use auditinternal for unit test in filter stage
2. add a seperate unit test for Audit-ID http header
3. add unit test for audit log backend
Automatic merge from submit-queue (batch tested with PRs 51603, 51653)
Graduate metrics/v1alpha1 to v1beta1
This introduces v1beta1 of the resource metrics API, previously in alpha.
The v1alpha1 version remains for compatibility with the Heapster legacy version
of the resource metrics API, which is compatible with the v1alpha1 version. It also
renames the v1beta1 version to `resource-metrics.metrics.k8s.io`.
The HPA controller's REST clients (but not the legacy client) have been migrated as well.
Part of kubernetes/features#118.
```release-note
Migrate the metrics/v1alpha1 API to metrics/v1beta1. The HorizontalPodAutoscaler
controller REST client now uses that version. For v1beta1, the API is now known as
resource-metrics.metrics.k8s.io.
```
Automatic merge from submit-queue (batch tested with PRs 51733, 51838)
Relax update validation of uninitialized pod
Split from https://github.com/kubernetes/kubernetes/pull/50344
Fix https://github.com/kubernetes/kubernetes/issues/47837
* Let the podStrategy to only call `validation.ValidatePod()` if the old pod is not initialized, so fields are mutable.
* Let the podStatusStrategy refuse updates if the old pod is not initialized.
cc @smarterclayton
```release-note
Pod spec is mutable when the pod is uninitialized. The apiserver requires the pod spec to be valid even if it's uninitialized. Updating the status field of uninitialized pods is invalid.
```
Automatic merge from submit-queue
Improve APIService auto-registration for HA/upgrade scenarios
Fixes#51912
Required for 1.8 due to impact on HA upgrades.
/assign @deads2k
cc @kubernetes/sig-api-machinery-bugs
```release-note
Fixes an issue with APIService auto-registration affecting rolling HA apiserver restarts that add or remove API groups being served.
```
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Bug Fix - Adding an allowed address pair wipes port security groups
**What this PR does / why we need it**:
Fix for cloud routes enabled instances will have their security groups
removed when the allowed address pair is added to the instance's port.
Upstream bug report is in:
https://github.com/gophercloud/gophercloud/issues/509
Upstream bug fix is in:
https://github.com/gophercloud/gophercloud/pull/510
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Fixes#51755
**Special notes for your reviewer**:
Just an fix in vendored code. minimal changes needed in OpenStack cloud provider
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Build test targets for all server platforms
**What this PR does / why we need it**:
🤦
I really should have checked this before code freeze, but tbh forgot it in the rush. Also I thought this was the case already...
As part of https://github.com/kubernetes/features/issues/288; these binaries should be built for all server platforms indeed.
This is just a straightforward add to that list.
Can we please get this into v1.8?
There is virtually no risk involved here really...
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Build test targets for all server platforms
```
@ixdy @jdumars @mkumatag
Automatic merge from submit-queue (batch tested with PRs 51984, 51351, 51873, 51795, 51634)
Add EgressRule to NetworkPolicy
**What this PR does / why we need it**:
Add EgressRule to NetworkPolicy
**Which issue this PR fixes**: fixes#50453
**Special notes for your reviewer**:
- Please take a look at the comments for the various types. I tried to mimic some of the language used in the Ingress comments, but I may have mangled some sentences.
- Let me know if I should add some test cases for validation. I have 2-3, and did not think it was necessary to replicate each case already covered in ingress.
**Release note**:
```
Add egress policies to NetworkPolicy
```
Automatic merge from submit-queue (batch tested with PRs 51186, 50350, 51751, 51645, 51837)
Update Cadvisor Dependency
Fixes: https://github.com/kubernetes/kubernetes/issues/51832
This is the worst dependency update ever...
The root of the problem is the [name change of Sirupsen -> sirupsen](https://github.com/sirupsen/logrus/issues/570#issuecomment-313933276). This means that in order to update cadvisor, which venders the lowercase, we need to update all dependencies to use the lower-cased version. With that being said, this PR updates the following packages:
`github.com/docker/docker`
- `github.com/docker/distribution`
- `github.com/opencontainers/go-digest`
- `github.com/opencontainers/image-spec`
- `github.com/opencontainers/runtime-spec`
- `github.com/opencontainers/selinux`
- `github.com/opencontainers/runc`
- `github.com/mrunalp/fileutils`
- `golang.org/x/crypto`
- `golang.org/x/sys`
- `github.com/docker/go-connections`
- `github.com/docker/go-units`
- `github.com/docker/libnetwork`
- `github.com/docker/libtrust`
- `github.com/sirupsen/logrus`
- `github.com/vishvananda/netlink`
`github.com/google/cadvisor`
- `github.com/euank/go-kmsg-parser`
`github.com/json-iterator/go`
Fixed https://github.com/kubernetes/kubernetes/issues/51832
```release-note
Fix journalctl leak on kubelet restart
Fix container memory rss
Add hugepages monitoring support
Fix incorrect CPU usage metrics with 4.7 kernel
Add tmpfs monitoring support
```
Nikhita Raghunath