Automatic merge from submit-queue
Rewrite staging import verifier in Go
Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
**What this PR does / why we need it**:
Maintaining Bash is nobody's idea of fun. The declarative config is infinitely easier to read, as well.
**Special notes for your reviewer**:
@sttts still trying to figure out the rules you have encoded in the shell script... very confusing. Please review the `kube-gen` specifically.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
move logs to kubectl/util
Move `pkg/util/logs` to `pkg/kubectl/util/logs` per https://github.com/kubernetes/kubernetes/issues/48209#issuecomment-311730681
This will make kubeadm, kubefed, gke-certificates-controller and e2e have dependency on kubectl, which should be fine.
partially addresses: kubernetes/community#598
```release-note
NONE
```
/assign @apelisse @monopole
Automatic merge from submit-queue
Remove deprecated ESIPP beta annotations
**What this PR does / why we need it**:
Remove deprecated ESIPP beta annotations.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50187
**Special notes for your reviewer**:
/assign @MrHohn
/sig network
**Release note**:
```release-note
Beta annotations `service.beta.kubernetes.io/external-traffic` and `service.beta.kubernetes.io/healthcheck-nodeport` have been removed. Please use fields `service.spec.externalTrafficPolicy` and `service.spec.healthCheckNodePort` instead.
```
Automatic merge from submit-queue
Added field CollisionCount to StatefulSetStatus
**What this PR does / why we need it**:
This PR added a new field `CollisionCount` into `StatefulSetStatus`, similarly in terms of both name and semantics to the existing `CollisionCount` field in `DaemonSetStatus`. The field will be used for collision avoidance when the `StatefulSet` controller creates name for the newest ControllerRevision, which will be done in another PR.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: #49909.
**Special notes for your reviewer**:
A second PR will include logic that actually uses the field for collision avoidance.
**Release note**:
```release-note
Added field CollisionCount to StatefulSetStatus in both apps/v1beta1 and apps/v1beta2
```
Automatic merge from submit-queue
add newline for unimplemented RequiredDuringSchedulingRequiredDuringExecution
**What this PR does / why we need it**:
Unimplemented `RequiredDuringSchedulingRequiredDuringExecution` should not be treated as comments of `RequiredDuringSchedulingIgnoredDuringExecution`
* [pkg/api/types.go#L1937](https://github.com/kubernetes/kubernetes/blob/master/pkg/api/types.go#L1937)
* [staging/src/k8s.io/api/core/v1/types.go#L2155](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/api/core/v1/types.go#L2155)
> // NOT YET IMPLEMENTED. TODO: Uncomment field once it is implemented.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:fixes #50483
**Special notes for your reviewer**:
* Add an empty line to separate them.
/cc @aveshagarwal @bsalamat @gyliu513 @k82cn @timothysc
**Release note**:
```release-note
None
```
Automatic merge from submit-queue
FC volume plugin: Support WWID for volume identifier
**What this PR does / why we need it**:
This PR adds World Wide Identifier (WWID) parameter to FCVolumeSource as an unique volume identifier.
**Which issue this PR fixes**: fixes#48639
**Special notes for your reviewer**:
/cc @rootfs @jsafrane @msau42
**Release note**:
```
FC volume plugin: Support WWID for volume identifier
```
Automatic merge from submit-queue
Migrate to controller references helpers in meta/v1
**What this PR does / why we need it**:
This is a follow up for #48319 that migrates all method usages to new methods in meta/v1.
**Special notes for your reviewer**:
Looking at each commit individually might be easier.
**Release note**:
```release-note
NONE
```
/sig api-machinery
/kind cleanup
Automatic merge from submit-queue
Fix my incorrect username
My mistake - used goog username rather than github.
Again, this is for kubectl extraction, currently blocked by need for many approvers in, e.g. #48580#48581#47011, etc.
Automatic merge from submit-queue
kubeadm: enhanced TLS validation for token-based discovery in `kubeadm join`
**What this PR does / why we need it**:
This PR implements enhanced TLS validation for `kubeadm join` when using token-based TLS discovery. Without this enhancement, `kubeadm join` has some less-than-ideal security properties. Specifically, in the case where a bootstrap token is compromised, the attacker can impersonate the API server to newly bootstrapping clients ([more discussion in the design proposal](https://docs.google.com/document/d/1SP4P7LJWSA8vUXj27UvKdVEdhpo5Fp0QHNo4TXvLQbw/edit?ts=5971498a)).
The gist of this enhancement is to support public key pinning in the style of [RFC7469](https://tools.ietf.org/html/rfc7469#section-2.4). When bootstrapping, `kubeadm` can now be configured with a whitelist of root CA public keys. It can then validate that the cluster it connects to is operated by the owner of one of those public keys.
These public key hashes are short enough that the entire `kubeadm join` command can still be copy-pasted relatively easily (not as easily as before, but ~160 characters). Using a public key hash rather than a hash over the entire certificate allows certificates to be reissued with updated expirations without invalidating existing key pins.
This change adds two new command line flags (and associated config parameters):
- **`--discovery-token-ca-cert-hash sha256:<hash>`:**
Validates that the cluster root CA has a public key fingerprint that matches one of the specified values. If this flag is not passed when token-based discovery is being used, a warning is printed. This warning will become an error in 1.9.
- **`--discovery-token-unsafe-skip-ca-verification`:**
Disables the warning message when no keys are pinned. In 1.9, this flag will be required _unless_ `--discovery-token-unsafe-skip-ca-verification` is used.
This is fully backwards compatible and client side (kubeadm) only. It will be a breaking change when the flag becomes required in v1.9.
This validation is done after and in addition to the existing bootstrap token signing/MAC mechanism.
#### Example from `kubeadm init`:
```
$ kubeadm init
[...]
You can now join any number of machines by running the following on each node
as root:
kubeadm join --token a66ae0.1f8a5ed9a210e187 192.168.42.10:6443 --discovery-token-ca-cert-hash sha256:547c102383c0f26387b961b4e9b8f842dc07c074c8316f238dbcf5563fc3ac35
```
#### Example from `kubeadm join`:
```
$ kubeadm join --token a66ae0.1f8a5ed9a210e187 192.168.42.10:6443 --discovery-token-ca-cert-hash sha256:547c102383c0f26387b961b4e9b8f842dc07c074c8316f238dbcf5563fc3ac35
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[discovery] Trying to connect to API Server "192.168.42.10:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.42.10:6443"
[discovery] Requesting info from "https://192.168.42.10:6443" again to validate TLS against the pinned public key
[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.42.10:6443"
[discovery] Successfully established connection with API Server "192.168.42.10:6443"
Node join complete:
* Certificate signing request sent to master and response
received.
* Kubelet informed of new secure connection details.
Run 'kubectl get nodes' on the master to see this machine join.
```
**Which issue this PR fixes**:
ref https://github.com/kubernetes/features/issues/130
fixes: https://github.com/kubernetes/kubeadm/issues/365
**Special notes for your reviewer**:
This was proposed and discussed briefly by SIG-cluster-lifecycle and SIG-auth. The design proposal is [in Google Docs](https://docs.google.com/document/d/1SP4P7LJWSA8vUXj27UvKdVEdhpo5Fp0QHNo4TXvLQbw/edit?ts=5971498a).
There is a documentation change needed to explain the security properties of `kubeadm join` with and without `--discovery-token-ca-cert-hash`. This page should be linked by to by the warning message when you don't pass either of the new flags (I have it pointing [here](https://kubernetes.io/docs/admin/kubeadm/#kubeadm-join) for now, which I think will be the right place). I will follow up with this documentation shortly.
**Release note**:
```release-note
kubeadm: added enhanced TLS validation for token-based discovery in `kubeadm join` using a new `--discovery-token-ca-cert-hash` flag.
```
/cc @luxas @jbeda @ericchiang
This change adds the `--discovery-token-ca-cert-hash` and `--discovery-token-unsafe-skip-ca-verification` flags for `kubeadm join` and corresponding fields on the kubeadm NodeConfiguration struct. These flags configure enhanced TLS validation for token-based discovery.
The enhanced TLS validation works by pinning the public key hashes of the cluster CA. This is done by connecting to the `cluster-info` endpoint initially using an unvalidated/unsafe TLS connection. After the cluster info has been loaded, parsed, and validated with the existing symmetric signature/MAC scheme, the root CA is validated against the pinned public key set. A second request is made using validated/safe TLS using the newly-known CA and the result is validated to make sure the same `cluster-info` was returned from both requests.
This validation prevents a class of attacks where a leaked bootstrap token (such as from a compromised worker node) allows an attacker to impersonate the API server.
This change also update `kubeadm init` to print the correct `--discovery-token-ca-cert-hash` flag in the example `kubeadm join` command it prints at the end of initialization.
This change adds a `k8s.io/kubernetes/cmd/kubeadm/app/util/pubkeypin` package which implements x509 public key pinning in the style of RFC7469. This is the public key hash format used by the new `kubeadm join --discovery-token-ca-cert-hash` flag.
Hashes are namespaced with a short type, with "sha256" being the only currently-supported format. Type "sha256" is a hex-encoded SHA-256 hash over the Subject Public Key Info (SPKI) object in DER-encoded ASN.1.
Automatic merge from submit-queue
Add Cluster Autoscaler scalability test suite
This suite is intended for manually testing Cluster Autoscaler on large clusters. It isn't supposed to be run automatically (at least for now).
It can be run on Kubemark (with #50440) with the following setup:
- start Kubemark with NUM_NODES=1 (as we require there to be exactly 1 replica per hollow-node replication controller in this setup)
- set kubemark-master machine type manually to appropriate type for the Kubemark cluster size. Maximum Kubemark cluster size reached in test run is defined by maxNodes constant, so for maxNodes=1000, please upgrade to n1-standard-32. Adjust if modifying maxNodes.
- start Cluster Autoscaler pod in the external cluster using image built from version with Kubemark cloud provider (release pending)
- for grabbing metrics from ClusterAutoscaler (with #50382), add "--include-cluster-autoscaler=true" parameter in addition to regular flags for gathering components' metrics/resource usage during e2e tests
cc @bskiba
Automatic merge from submit-queue
Make ClusterID required for AWS.
**What this PR does / why we need it**:
Makes ClusterID required for AWS and provides a flag to run in un-tagged mode
fixes#48954
**Release note**:
```release-note
A cluster using the AWS cloud provider will need to label existing nodes and resources with a ClusterID or the kube-controller-manager will not start. To run without a ClusterID pass --allow-untagged-cloud=true to the kube-controller-manager on startup.
```
Automatic merge from submit-queue (batch tested with PRs 45186, 50440)
Add functionality needed by Cluster Autoscaler to Kubemark Provider.
Make adding nodes asynchronous. Add method for getting target
size of node group. Add method for getting node group for node.
Factor out some common code.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 45186, 50440)
Retry fed-svc creation on diff NodePort during e2e tests
**What this PR does / why we need it**:
Currently in federated end2end tests, the creation of services are
done with a randomize NodePort selection take is causing e2e test
flakes if the creation of a federated service failed if the port is
not available.
Now the util.CreateService(...) function is retrying to create the
service on different nodePort in case of error. The method retry until
success or all possible NodePorts have been tested and also failed.
**Which issue this PR fixes**
fixes#44018
Automatic merge from submit-queue (batch tested with PRs 50306, 49624)
Add daemonset to all categories
**What this PR does / why we need it**:
We could get daemonset resource by running command `kubectl get all`.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fix https://github.com/kubernetes/kubernetes/issues/49620
Automatic merge from submit-queue (batch tested with PRs 50306, 49624)
simplify logic around LB deletion for servicecontroller
**What this PR does / why we need it**: simplify logic around LB deletio
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: discovered when aswering to #50299
**Release note**:
```release-note
None
```
Make adding nodes asynchronous. Add method for getting target
size of node group. Add method for getting node group for node.
Factor out some common code.
Automatic merge from submit-queue (batch tested with PRs 50386, 50374, 50444, 50382)
Add grabbing Cluster Autoscaler metrics in e2e tests
This adds:
- collecting metrics from Cluster Autoscaler before & after e2e test run
- --include-cluster-autoscaler opt-in flag
- passing external cluster client to MetricsGrabber (required for Kubemark setup, as Cluster Autoscaler doesn't run on master in this case)
Automatic merge from submit-queue (batch tested with PRs 50386, 50374, 50444, 50382)
jsonpath: fix comments
Minor fix to the comments. And avoid a named return value.
**Release note**:
```release-note
NONE
```
/cc @sttts
Automatic merge from submit-queue (batch tested with PRs 50386, 50374, 50444, 50382)
wires ban flunder admission plugin to the sample server
**What this PR does / why we need it**:
this PR wires ban flunder admission plugin to the sample server.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50386, 50374, 50444, 50382)
Add explicit API kind and version to the audit policy file on GCE
Adds an explicit API version and kind to the audit policy file in GCE configuration scripts. It's a prerequisite for https://github.com/kubernetes/kubernetes/pull/49115
/cc @tallclair @piosz
Most types now have valid rest mappings because
NewDefaultRESTMapperFromScheme no longer ignores certain import
paths. Thus we can no longer use the lack of a valid REST mapping
as an indicator for when to use kindWhiteList. Thus kindWhiteList
now serves as a whitelist for all kinds and not just those that
formally had no mapping. This does mean that we could whitelist
kinds due to a name conflict, but that is unlikely as names such as
GetOptions are not appropriate for new objects.
Signed-off-by: Monis Khan <mkhan@redhat.com>
Automatic merge from submit-queue
Update mrubin to matchstick in OWNERS
**What this PR does / why we need it**:
per https://github.com/kubernetes/kubernetes/issues/50048#issuecomment-320000920
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
associated with #50048
**Special notes for your reviewer**:
/assign @matchstick
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
[Federation] Implement leader election for controller-manager
**What this PR does / why we need it**:
- Add cluster-selector for namespace
- Add support for creating federation-only objects. - Ref #44631
- Implements leader election for controller-manager.
Ref: #44283
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44490
**Special notes for your reviewer**:
This PR also fixes the issue #44490, which is about delay in initializing controller-manager due to unavailability of api-server.
**Release note**:
```release-note
federation: Support for leader-election among federation controller-manager instances introduced.
```
/cc @kubernetes/sig-federation-pr-reviews
Automatic merge from submit-queue
fix error message for scale
**What this PR does / why we need it**:
ref: https://github.com/kubernetes/kubernetes/blob/master/cmd/genutils/genutils_test.go#L33
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Klaus Ma