Automatic merge from submit-queue (batch tested with PRs 48583, 48605, 48601)
apimachinery+apiserver: separate test types in their own packages
Preparation for static deepcopy https://github.com/kubernetes/kubernetes/pull/48544
and its use of package-global deepcopy-gen tags for all runtime.Objects.
- [x] wait for #48497
Automatic merge from submit-queue (batch tested with PRs 48583, 48605, 48601)
support json output for log backend of advanced audit
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
Add json format support for advanced audit in apiserver. Use --audit-log-format=json to emit json to log backend.
```
Automatic merge from submit-queue
Record 429 and timeout errors to prometheus
Allows gathering of load being shed.
Fixes#48559
@deads2k please review, there was a logic error in apiserver RequestInfo (minor, fortunately)
```release-note
Requests with the query parameter `?watch=` are treated by the API server as a request to watch, but authorization and metrics were not correctly identifying those as watch requests, instead grouping them as list calls.
```
Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)
remove dead code
This removes the dead code cruft since we stopped serving TPRs.
ref #48152
Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)
audit: fix deepcopy registration
Remove manual registration into scheme by fixing the deepcopy tag of the api group.
Automatic merge from submit-queue (batch tested with PRs 47435, 46044)
IPv6 support for getting node IP
As part of ChooseHostInterface(), it will call a function to try to get the
global IP for the host, by looking at all the system interfaces and select
the first IP that is not a loopback, link-local, or point-to-point IP.
This commit does the following:
- Allows IPv6 non-local IPs to be selected.
- IPv4 takes priority (checks all interfaces for IPv4 addresses and
then checks all interfaces for IPv6), for backward compatibility.
- Adds UTs for code coverage (was no coverage of underlying function),
increasing from 62% to 85%.
- Improved logging and reporting for error conditions.
- Minor renaming of functions and variables for readability.
**What this PR does / why we need it**:
This will be part of several PRs to add IPv6 support in apimachinery area for use by Kubernetes. It partially fixes the issue.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: xref #44848
**Special notes for your reviewer**:
The intent is to break up the PR 45116 into multiple PRs to take on this change a piece at a time.
**Release note**:
```release-noteNONE
```
Automatic merge from submit-queue
Fix invalid Content-Type for 403 error
https://github.com/kubernetes/kubernetes/pull/47384 makes 403 errors return Status Object. However the Content-Type is still "text/plain"
This change fixes it.
Before this change:
kubectl get pods --as=tom
Error from server (Forbidden): {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"\" is forbidden: User \"tom\" cannot list pods in the namespace \"default\".","reason":"Forbidden","details":{"kind":"pods"},"code":403} (get pods)
After this change:
$ kubectl get pods --as=tom
Error from server (Forbidden): pods "" is forbidden: User "tom" cannot list pods in the namespace "default".
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Now http header key "Audit-ID" doesn't have effect, because golang
automaticly transforms "Audit-ID" into "Audit-Id". This change use
http.Header.Get() function to canonicalize "Audit-ID" to "Audit-Id".
Automatic merge from submit-queue (batch tested with PRs 47700, 48464, 48502)
Add a refreshing discovery client
Introduce a discovery client (implementing `CachedDiscoveryInterface`) which caches discovery information in memory and which can be actively refreshed by the user.
This implementation fetches from discovery upon refresh and could later be improved to maintain updates from a watch.
Extracted from https://github.com/kubernetes/kubernetes/pull/47665 and https://github.com/kubernetes/kubernetes/pull/46000 to help reduce the scope of https://github.com/kubernetes/kubernetes/pull/48065.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47700, 48464, 48502)
fix error type
**What this PR does / why we need it**:
invalidToken should be errInvalideToken
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
No
**Release note**:
```
None
```
Automatic merge from submit-queue (batch tested with PRs 45467, 48091, 48033, 48498)
Refactor and simplify generic printer for unknown objects
The first two commits are part of other PRs
@kubernetes/sig-cli-pr-reviews part of the general refactoring for server side print
Automatic merge from submit-queue (batch tested with PRs 47162, 48444, 48445)
make the panic handler first
Move the panic handler to the beginning of the chain so that panics will get information in the log beyond a stack.
Automatic merge from submit-queue (batch tested with PRs 47162, 48444, 48445)
Introducing a cluster-scoped resource in the wardle.k8s.io group.
**What this PR does / why we need it**:
This PR adds a cluster-scoped resource to the wardle.k8s.io group.
The cluster scoped resource has a field that indicates Flunder.Names that are disallowed.
The resource is going to be used by an admission plugin.
The admission plugin will list the cluster-scope resources and check against banned names.
**Special notes for your reviewer**:
Issue: #47868
**Release note**:
```
NONE
```
The cluster scoped resource has a field that indicates Flunder.Names that are disallowed.
The resource is going to be used by an admission plugin.
The admission plugin will list the cluster-scope resources and check against banned names.
Issue: #47868
Automatic merge from submit-queue
remove unused function and variable from audit backend
auditStringSlice is not used here anymore.
sink variable is also not used.
**Release note**:
```
NONE
```
Fixes: #47114
Automatic merge from submit-queue
Follow up for https://github.com/kubernetes/kubernetes/pull/47003
**What this PR does / why we need it**:
This is addressing left comments for https://github.com/kubernetes/kubernetes/pull/47003#discussion_r120888982
/cc @aveshagarwal
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 48439, 48440, 48394)
GuaranteedUpdate must write if stored data is not canonical
An optimization added to the GuaranteedUpdate loop changed the
comparison of the current objects serialization against the stored data,
instead comparing to the in memory object, which defeated the mechanism
we use to migrate stored data (GET then PUT should update the version stored in etcd if the canonical serialization has changed)
This commit preserves that optimization but correctly verifies the in
memory serialization against the on disk serialization by fetching the
latest serialized data. Since most updates are not no-ops, this should
not regress the performance of the normal path.
Fixes#48393
```release-note
When performing a GET then PUT, the kube-apiserver must write the canonical representation of the object to etcd if the current value does not match. That allows external agents to migrate content in etcd from one API version to another, across different storage types, or across varying encryption levels. This fixes a bug introduced in 1.5 where we unintentionally stopped writing the newest data.
```
Automatic merge from submit-queue (batch tested with PRs 47784, 47793, 48334, 48435, 48354)
allow a deletestrategy to opt-out of GC
Not all resources should be GC-able and we implemented an ignore list to handle this, but at the storage layer they could still set finalizers, they just hung in a stuck state forever. This updates the strategy to allow a resource to indicate that they shouldn't be GCed.
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 47784, 47793, 48334, 48435, 48354)
update events' ResponseStatus at Metadata level
ResponseStatus is populated in MetadataLevel, so we also update it in
MetadataLevel.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47784, 47793, 48334, 48435, 48354)
add validate for advanced audit policy
This change checks group name and non-resrouce URLs format for audit
policy.
**Release note**:
```
add validate for advanced audit policy, kube-apiserver will do a stricter validation and will break existing users with invalid configs.
```
An optimization added to the GuaranteedUpdate loop changed the
comparison of the current objects serialization against the stored data,
instead comparing to the in memory object, which defeated the mechanism
we use to migrate stored data.
This commit preserves that optimization but correctly verifies the in
memory serialization against the on disk serialization by fetching the
latest serialized data. Since most updates are not no-ops, this should
not regress the performance of the normal path.
Automatic merge from submit-queue
remove useless check from impersonation filter
When groupsSpecified is false, that means no other groups are added
rather than the service account groups. So this check doesn't make
any sense.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47918, 47964, 48151, 47881, 48299)
GZip openapi schema if accepted by client
**What this PR does / why we need it**: Uses gzip "Accept-Encoding" flag rather than specific path to download gzipped openapi schema.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48375
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 43558, 48261, 42376, 46803, 47058)
OpenAPI downloads protobuf rather than Json
**What this PR does / why we need it**:
The current implementation of the OpenAPI getter fetches the swagger in a Json format from the apiserver. The Json file is big (~1.7mb), which means that it takes a long time to download, and then a long time to parse. Because that is going to be needed on each `kubectl` run later, we want this to be as fast as possible.
The apiserver has been modified to be able to return a protobuf version of the swagger, which this patch intends to use.
Note that there is currently no piece of code that exists that allows us to go from the protobuf version of the file, back into Json and/or `spec.Swagger`. Because the protobuf is not very different (but significantly different enough that it can't be translated), I've updated the code to use `openapi_v2.Document` (the protobuf type) everywhere rather than `spec.Swagger`. The behavior should be identical though.
There are more changes that are coming in follow-up pull-requests: using the gzip version (also provided by the new apiserver) to even further reduce the size of the downloaded content, and use the HTTP Etag cache mechanism to completely get rid of recurrent fetch requests. I'm currently working on these two features.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: partly #38637
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Tim Allclair