These lines may contain information that cannot be safely logged to
disk. Only deal with known parameters, also bump to V(2) to be
consistent with other logging.
Automatic merge from submit-queue (batch tested with PRs 42044, 41694, 41927, 42050, 41987)
Simplify and fix hack/{verify,update}-staging-{client-go,godeps}.sh
- merge `hack/{verify,update}-staging-client-go.sh`
- pin godep with shared code
- remove godep-restore completely from the process and replace with a simple check that godeps are restored
- add safety check in `staging/copy.sh` that there is no lingering `k8s.io/apimachinery` in the GOPATH which would lead to inconsistent client-go builds (!)
- check that all these scripts only operate in a clean working dir.
Automatic merge from submit-queue (batch tested with PRs 35408, 41915, 41992, 41964, 41925)
add secret option to flag
To resolve the issue of security(pr #35030 ),
> @smarterclayton commented 5 days ago
> This is unfortunately not all flags that could be secrets. The best option would be to add support in spf13/pflag to tag a flag as a secret, and then use that bit to determine the list.
>
> Also, Command() could be used in contexts that need exact parameters (for subshell execution), so we would need to add a new method or extend the signature here to allow exact flags to be retrieved.
we could add a secret option to the flags.
Automatic merge from submit-queue (batch tested with PRs 41954, 40528, 41875, 41165, 41877)
Updating apiserver to return 202 when resource is being deleted asynchronously via cascading deletion
As per https://github.com/kubernetes/kubernetes/issues/33196#issuecomment-278440622.
cc @kubernetes/sig-api-machinery-pr-reviews @smarterclayton @caesarxuchao @bgrant0607 @kubernetes/api-reviewers
```release-note
Updating apiserver to return http status code 202 for a delete request when the resource is not immediately deleted because of user requesting cascading deletion using DeleteOptions.OrphanDependents=false.
```
Automatic merge from submit-queue (batch tested with PRs 41621, 41946, 41941, 41250, 41729)
Refactor printers and describers into their own package.
This sets the stage for using printer code from the server side (decoupled from kubectl) and loosens the coupling between kubectl and the printers. `pkg/printers` contains interfaces and has an import restriction against pulling in API specific code, while `pkg/printers/internalversion` can be used for internal types.
Add a method on `Factory` for retrieving PrinterForCommand which uses the Scheme and RESTMapper from the Factory, not the hardcoded ones. This further separates kubectl from the core API scheme and allows better composition.
Change NamePrinter to use RESTMapper (previously it was hardcoding those conversions). This means that we now return plural resource names (`pods/foo`) but is correct once aliases and shortnames start being returned by the mapper.
This is a prerequisite for server side get, but is pure refactor (contains no new features).
@deads2k @liggitt
Automatic merge from submit-queue
apiserver: self-signed in-memory cert for loopback
We had complicated logic before to find a matching (SNI- or server-) cert for the loopback device. In the worst case, bootstrapping failed because the provided certs didn't have the binding IP or localhost.
We now pass an artifical `ServerName` "apiserver-loopback-client" with the loopback client and install a self-signed in-memory cert+key into the server for that hostname. The client accepts the corresponding cert.
Fixing https://github.com/kubernetes/kubernetes/issues/41081
Automatic merge from submit-queue
redact detailed errors from healthz and expose in default policy
Makes `/healthz` less sensitive and exposes it by default.
@kubernetes/sig-auth-pr-reviews @kubernetes/sig-api-machinery-misc @liggitt
Automatic merge from submit-queue (batch tested with PRs 41797, 41793, 41795, 41807, 41781)
Don't use base64 encoding with etcd v3
In https://github.com/kubernetes/kubernetes/pull/36229 we changed the default storage to etcd v3.
This in fact is a fix to that PR.
With etcd v3, the base64-encoding is no longer needed - so we use it only if etcd v2 is explicitly requested as storage format.
@lavalamp
Automatic merge from submit-queue (batch tested with PRs 41797, 41793, 41795, 41807, 41781)
tweak a list to a watch verb when requested
Related to https://github.com/kubernetes/kubernetes/pull/41774
@wojtek-t does this do what's needed?
Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)
client-gen: independent scheme for clientsets
This PR adds a clientset internal scheme instead of using `pkg/api.Scheme`. **The clientset API stays the same.**
In detail:
- introduce a scheme for each clientset, i.e. do not use `pkg/api.Scheme+Registry+Codec+ParameterCodecs`.
This makes it easier to compose client-go's clientset (which is rewritten in `staging/copy.sh` and therefore hardcoded to use `k8s.io/client-go/pkg/api.Scheme+Registry+Codecs+ParameterCodecs`) with third-party clientsets (kube-aggregator, openshift, federation) which are not rewritten using `copy.sh` as all of them are self-contained and therefore relocatable.
This fixes https://github.com/kubernetes/kubernetes/pull/41403/files#diff-76edfb07dee54ff7ddeda25c33c10d29R81 and prepares client-gen for use in OpenShift.
- register types into the clientset scheme via `AddToScheme` for versioned clientsets. This decouples the client-go clients from announce+registration (internal clients continue using announce+registry and apigroup installers).
This reduces complexity for client-go, possibly remove the necessity for the announce+register machinery for many use-cases, maybe even to delete it mid-term.
- port federation and testgroup `install/install.go` to `announced.GroupMetaFactory` in order to have a proper `Install.Install(...)` func for registration.
With the first change it's easy to add the types of one clientset to the scheme of the other using the `clientset/scheme.AddToScheme` method. This allows to use cross-clientset `runtime.RawExtensions`:
```golang
import (
"k8s.io/client-go/kubernetes"
clientsetscheme "k8s.io/client-go/kuberentes/scheme"
aggregatorclientsetscheme "k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/scheme"
)
kclientset, _ := kubernetes.NewForConfig(c)
aggregatorclientsetscheme.AddToScheme(clientsetscheme.Scheme)
```
Kubernetes types with a `RawExtension` can en/decode aggregator types after this.
TODO:
- [x] fix fake clientsets
- [x] get the `*Options` types registered correctly for core, compare DO-NOT-MERGE commit.
- [x] get prefered version right in internal client. Do we need all versions registered in the internal client to support negotiation?
- [x] run `staging/copy.sh` and run tests: https://github.com/kubernetes/kubernetes/pull/41744
- ~~[ ] fixup usage through-out the code-base~~
- **Follow-up**: move `import_known_versions.go` files somewhere such that import of the `api.Scheme` package automatically installs the apigroups. It looks like we depended on the import fo the clientset for this purpose.
Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)
Fix deployment helper - no assumptions on only one new ReplicaSet
#40415
**Release note**:
```release-note
NONE
```
@kubernetes/sig-apps-bugs
Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)
Change taints/tolerations to api fields
This PR changes current implementation of taints and tolerations from annotations to API fields. Taint and toleration are now part of `NodeSpec` and `PodSpec`, respectively. The annotation keys: `scheduler.alpha.kubernetes.io/tolerations` and `scheduler.alpha.kubernetes.io/taints` have been removed.
**Release note**:
Pod tolerations and node taints have moved from annotations to API fields in the PodSpec and NodeSpec, respectively. Pod tolerations and node taints that are defined in the annotations will be ignored. The annotation keys: `scheduler.alpha.kubernetes.io/tolerations` and `scheduler.alpha.kubernetes.io/taints` have been removed.
Automatic merge from submit-queue
add godep manifest files to staging repos
The staging repos should have manifests that match the godeps of kube so we know what they build against. We don't need the actual vendored code, since a sync script on the other side needs to find the correct level of other staging directories and thus requires its own `godep restore && go get && godep save` cycle.
@sttts ptal
@lavalamp @caesarxuchao client-go needs a lot of unwinding to do something similar, but the idea is that you can run an acyclic path to get this updated by copying the types and dependencies with `go list`, then generate the clients, then generate this manifest. Then in your sync script you can pull the proper levels and finish the actual vendoring.
Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)
client-go: don't import client auth provider packages
Both of these auth providers are useful for kubectl but not so much for everyone importing client-go. Let users optionally import them (example [0]) and reduce the overall number of imports that client-go requires.
Quick grep seems to imply it wont import it after.
```
$ grep -r 'client-go/plugin/pkg/client/auth' staging/
staging/src/k8s.io/client-go/plugin/pkg/client/auth/plugins.go: _ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
staging/src/k8s.io/client-go/plugin/pkg/client/auth/plugins.go: _ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
staging/src/k8s.io/client-go/examples/third-party-resources/main.go: _ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
staging/src/k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/clientset.go: _ "k8s.io/client-go/plugin/pkg/client/auth"
staging/src/k8s.io/kube-aggregator/pkg/client/clientset_generated/internalclientset/clientset.go: _ "k8s.io/client-go/plugin/pkg/client/auth"
```
closes https://github.com/kubernetes/client-go/issues/49
updates https://github.com/kubernetes/client-go/issues/79 (removes cloud.google.com/go import)
cc @kubernetes/sig-api-machinery-pr-reviews @kubernetes/sig-auth-pr-reviews
```release-notes
client-go no longer imports GCP OAuth2 and OpenID Connect packages by default.
```
[0] 8b466d64c5/examples/third-party-resources/main.go (L34-L35)
Automatic merge from submit-queue (batch tested with PRs 41844, 41803, 39116, 41129, 41240)
Cleanup client example
**What this PR does / why we need it**:
- Package level `config` variable in `third-party-resources/main.go` is not used, it is shadowed by the one defined in `main()`. Should probably be deleted.
- Package level `kubeconfig ` variable in `out-of-cluster/main.go` is global - make it private to `main()`.
**Which issue this PR fixes**
This fixes https://github.com/kubernetes/client-go/issues/59, except the part about global `api.Scheme`, also adds test with interface check. Supersedes https://github.com/kubernetes/client-go/pull/61.
**Special notes for your reviewer**:
This is my first PR to Kubernetes :)
Automatic merge from submit-queue (batch tested with PRs 41709, 41685, 41754, 41759, 37237)
Tolerate unknown fields in strategic merge patch
When using `apply` or `edit` with an object that has a compiled-in struct, if an unknown server-side field is sent, or is present in a provided file, the strategic merge patch computation fails looking up type info from the go struct
If the field only exists in one side of the patch (is being added or removed), or is identical in both sides of the patch, we should tolerate missing type info, since it doesn't affect the patch.
Automatic merge from submit-queue (batch tested with PRs 41706, 39063, 41330, 41739, 41576)
Added bool type support for jsonpath.
Added keywords "true" and "false" for
supporting bool type in jsonpath.
Fixed#28678
Automatic merge from submit-queue (batch tested with PRs 41756, 36344, 34259, 40843, 41526)
Migrate rackspace/gophercloud -> gophercloud/gophercloud
`gophercloud` is the go library we use to interact with openstack. It has recently been renamed and incorporated a number of breaking code cleanups. This change migrates the Kubernetes openstack code (openstack provider, keystone auth, cinder volumes) to the new library.
Fixes#30404
```release-note
The openstack cloud provider config no longer supports the deprecated `api-key` option.
```
Note this PR doesn't change the rackspace provider. It uses some rackspace-specific APIs and continues to use the older rackspace/gophercloud library.
Automatic merge from submit-queue (batch tested with PRs 41421, 41440, 36765, 41722)
Use watch param instead of deprecated /watch/ prefix
Switches clients to use watch param instead of /watch/ prefix
```release-note
Clients now use the `?watch=true` parameter to make watch API calls, instead of the `/watch/` path prefix
```
Automatic merge from submit-queue
Support PATCH operation in root proxy
Not sure if we want to continue to expand the root proxy capabilities, but if we do, this fixes#40383
This change migrates the 'openstack' provider and 'keystone'
authenticator plugin to the newer gophercloud/gophercloud library.
Note the 'rackspace' provider still uses rackspace/gophercloud.
Fixes#30404
Automatic merge from submit-queue (batch tested with PRs 39373, 41585, 41617, 41707, 39958)
Feature-Gate affinity in annotations
**What this PR does / why we need it**:
Adds back basic flaggated support for alpha Affinity annotations
**Special notes for your reviewer**:
Reconcile function is placed in the lowest common denominator, which in this case is schedulercache, because you can't place flag-gated functions in apimachinery.
**Release note**:
```
NONE
```
/cc @davidopp
Automatic merge from submit-queue (batch tested with PRs 41043, 39058, 41021, 41603, 41414)
Rotate the kubelet certificate when about to expire.
Changes the kubelet so it doesn't use the cert/key files directly for
starting the TLS server. Instead the TLS server reads the cert/key from
the new CertificateManager component, which is responsible for
requesting new certificates from the Certificate Signing Request API on
the API Server.
Automatic merge from submit-queue (batch tested with PRs 41043, 39058, 41021, 41603, 41414)
Allow setting replace patchStrategy for structs
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#20208
@liggitt and @ymqytw ptal, you were in the original issue
Changes the kubelet so it doesn't use the cert/key files directly for
starting the TLS server. Instead the TLS server reads the cert/key from
the new CertificateManager component, which is responsible for
requesting new certificates from the Certificate Signing Request API on
the API Server.
Automatic merge from submit-queue (batch tested with PRs 41505, 41484, 41544, 41514, 41022)
several issues hit while trying to make it easy to register APIs
I was trying to create a script that would register all API versions on a given server and ended up hitting several problems. These are the fixes.
@sttts I suspect that I won't be able to continue down the host-network approach, since that means I won't be able to use in-cluster DNS without some finagling. It *could* be set up (and we make it work as a for instance), but the simple enablement approach will be hosted on the infrastructure. I'll go back to that.
Automatic merge from submit-queue (batch tested with PRs 41466, 41456, 41550, 41238, 41416)
Don't use json.Marshal when printing error bodies
Internal types panic when json.Marshal is called to prevent accidental
use.
Fixes#40491
Automatic merge from submit-queue
Stop controller when the stop channel is closed (when queue is empty and Pop is hanging)
Fixes: #28158
When a ``Pop`` function is invoked over empty queue, the control loop inside the functions is stacked indefinitely. In order to break the loop, introduce logic that waits for a signal to exit the loop.
Intention of the PR is not to handle situation where manipulation operations are invoked over closed queue. Intention is to break the indefinite loop.
Automatic merge from submit-queue (batch tested with PRs 41104, 41245, 40722, 41439, 41502)
add sample fuzzing tests
Make fuzzing tests as simple as possible from both the API installer and the scheme, so its easy to add for api groups and so that I can build a scheme and then make sure I got it right.
@kubernetes/sig-api-machinery-pr-reviews @sttts @mikedanese
Automatic merge from submit-queue
Move private key parsing from serviceaccount/jwt.go to client-go/util/cert
**What this PR does / why we need it**:
Unify private key parsing from serviceaccount/jwt.go into the client-go library.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Partial fix to #40807 - only private key functions.
**Special notes for your reviewer**:
**Release note**:
```release-note
Move private key parsing from serviceaccount/jwt.go to client-go/util/cert
```
Automatic merge from submit-queue (batch tested with PRs 41332, 41069, 41470, 41474)
"Avoid unnecessary copies in cacher""
This is resend of #40735 (which I reverted when I suspected it to cause issues). But the issue was a completely different. So it's safe to resubmit.
Automatic merge from submit-queue
make kube-aggregator run as static pod for local-up-cluster
Runs the kube-aggregator as a static pod for local-up-cluster. Looks like someone broke kubectl negotiation again, so I'll fix that up separately.
@kubernetes/sig-api-machinery-misc
@lavalamp you're probably looking to run kube-aggregator as a static pod, here's an example.
@jwforres I'll make a secure variant for wiring up to openshift.
Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)
apiserver: further cleanup of apiserver storage plumbing
- move kubeapiserver`s `RESTOptionsFactory` back to EtcdOptions by adding a `AddWithStorageFactoryTo`
- factor out storage backend `Config` construction from EtcdOptions
- move all `StorageFactory` related code into server/storage subpackage.
In short: remove my stomach ache about `kubeapiserver.RESTOptionsFactory`.
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)
move kube-aggregator to staging
Straight move of kube-aggregator to staging. The scripts are the interesting bits.
@sttts @lavalamp
Automatic merge from submit-queue (batch tested with PRs 41299, 41325, 41386, 41329, 41418)
move metav1 conversions to metav1
Conversions for `metav1` types belong in metav1 and should be registered when you register the types.
@mikedanese @luxas I think this is what you just hit in your fresh scheme.
@smarterclayton @lavalamp double check the sanity, but I think this does what people expect.
Automatic merge from submit-queue (batch tested with PRs 41299, 41325, 41386, 41329, 41418)
stop senseless negotiation
Most client commands don't respect a negotiated version at all. If you request a particular version, then of course it should be respected, but if you have none to request, then the current negotiation step doesn't return anything useful so we may as well have nothing so we can at least detect the situation.
@jwforres @kubernetes/sig-cli-pr-reviews
Added a TODO to make the negotiate function useful. I think I'm inclined to remove it entirely unless someone can come up with a useful reason to have it.
Automatic merge from submit-queue
copy pkg/util/logs to apiserver
This is a copy, not a move. API servers need to be able to init the logs, but so do clients. It would be weird to have the client-side commands depending on the server side logs utilities.
I updated all the server side references, but left the client-side ones.
@sttts @kubernetes/sig-api-machinery-pr-reviews acceptable?
Automatic merge from submit-queue
updated docs in roundtrip.go to correct names
**What this PR does / why we need it**: updated docs in roundtrip.go to correct names.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)
BootstrapSigner and TokenCleaner controllers
This is part of https://github.com/kubernetes/features/issues/130 and is an implementation of https://github.com/kubernetes/community/pull/189.
Work that needs to be done yet in this PR:
* [ ] ~~e2e tests~~ Will come in new PR.
* [x] flag to disable this by default
```release-note
Native support for token based bootstrap flow. This includes signing a well known ConfigMap in the `kube-public` namespace and cleaning out expired tokens.
```
@kubernetes/sig-cluster-lifecycle @dgoodwin @roberthbailey @mikedanese
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)
Register deepcopy and defaulter funcs for metav1
Houston, we've had a problem here. ObjectMeta had used the reflection fallback path for deepcopy for probably some time.
Remark: my original plan for https://github.com/kubernetes/kubernetes/pull/36412 was/is to rebase that during the master branch freeze for 1.6 and get it merged just before master opens again. Otherwise, I fear it will be rebase hell. With https://github.com/kubernetes/kubernetes/pull/36412 these kind of errors are not possible anymore.
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)
fix bug of runServer function in server.go
What this PR does / why we need it:
use parameter network instead of hardcode 'tcp' / the parameter network here is not equal to 'tcp'
**Special notes for your reviewer**:
NONE
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
make round trip testing generic
RoundTrip testing is something associated with a scheme and everyone who writes an API will want to do it. In the end, we should wire each API group separately in a test scheme and have them all call this general function. Once `kubeadm` is out of the main scheme, we'll be able to remove the one really ugly hack.
@luxas @sttts @kubernetes/sig-apimachinery-pr-reviews @smarterclayton
Automatic merge from submit-queue
Add staging/ to gopath in copy.sh
An alternative to https://github.com/kubernetes/kubernetes/pull/40731.
Please check the last 2 commits of this branch out for an example run: https://github.com/caesarxuchao/kubernetes/tree/add-stagingto-gopath. I added package "validationchao" to apimachinery, and let k8s.io/kubernetes/pkg/client/generated_clientset/ depend on it, then I run hack/update-staging-client-go.sh, and it seems to work fine.
@sttts @deads2k @lavalamp
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Shared informer custom resync periods
**What this PR does / why we need it**: add the ability for event handlers to specify their own distinct resync periods to a shared informer
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#26406
**Special notes for your reviewer**:
This is part of my series of PRs to convert as much of the code base to using generated shared informers as possible. Assuming this merges, we'll also need #40385, and I'll probably close#40097 and replace it with a series of individual PRs (1 per controller change).
Assuming we get close to agreement on the implementation, I can look at adding more unit tests.
**Release note**:
```release-note
```
@smarterclayton @deads2k @liggitt @sttts @derekwaynecarr @pmorie @lavalamp @wojtek-t @gmarek @timothysc @jayunit100 @kubernetes/sig-api-machinery-pr-reviews @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
add k8s.io/sample-apiserver to demonstrate how to build an aggregated API server
builds on https://github.com/kubernetes/kubernetes/pull/41093
This creates a sample API server is a separate staging repo to guarantee no cheating with `k8s.io/kubernetes` dependencies. The sample is run during integration tests (simple tests on it so far) to ensure that it continues to run.
@sttts @kubernetes/sig-api-machinery-misc ptal
@pwittrock @pmorie @kris-nova an aggregated API server example that will stay up to date.
Add the ability for each event handler of a shared informer to specify
its own resync period. If not specified, a handler will resync at the
informer's default interval.
Automatic merge from submit-queue (batch tested with PRs 40873, 40948, 39580, 41065, 40815)
Make DeltaFIFO Resync atomic
Make DeltaFIFO's Resync operation atomic, so it enqueues the entire
queue before allowing adds/updates/deletes.
I'm hoping to use this to help with custom resync periods for multiple event handlers against a single shared informer (see https://github.com/kubernetes/kubernetes/pull/40759#pullrequestreview-19598213 for the motivation).
@lavalamp @smarterclayton @deads2k @liggitt @sttts @timothysc @wojtek-t @gmarek @kubernetes/sig-api-machinery-pr-reviews @kubernetes/sig-scalability-pr-reviews
Eric Chiang