Automatic merge from submit-queue (batch tested with PRs 41597, 42185, 42075, 42178, 41705)
stop spamming logs on restart of api server
**What this PR does / why we need it**:
This PR reduces an excessive amount of log spam on startup of any API server with larger amounts of content. We have experienced server restart times ~5 minutes that is a consequence of us tracing each directory in etcd during our initial decoding. This spams the journald process, and results in much of those messages getting rate limited and dropped, and making actual problems much harder to debug. This stops us logging discrete keys in favor of just knowing that an operation was slow for the particular object type (which is enough information!)
this PR also makes it so we know what trace operation was ended in our log messages!
sample log now:
```
Trace "decodeNodeList *[]api.PolicyBinding" (started 2017-02-20 22:20:33.98771112 +0000 UTC)
Decoded 10718 nodes
decodeNodeList *[]api.PolicyBinding [500ms] END
```
Automatic merge from submit-queue (batch tested with PRs 41597, 42185, 42075, 42178, 41705)
auto discovery CA for extension API servers
This is what the smaller pulls were leading to. Only the last commit is unique and I expect I'll still tweak some pod definitions, but this is where I was going.
@sttts @liggitt
Automatic merge from submit-queue (batch tested with PRs 41597, 42185, 42075, 42178, 41705)
Don't log context or full request URI due to security concerns
- Add a new type PortworxVolumeSource
- Implement the kubernetes volume plugin for Portworx Volumes under pkg/volume/portworx
- The Portworx Volume Driver uses the libopenstorage/openstorage specifications and apis for volume operations.
Changes for k8s configuration and examples for portworx volumes.
- Add PortworxVolume hooks in kubectl, kube-controller-manager and validation.
- Add a README for PortworxVolume usage as PVs, PVCs and StorageClass.
- Add example spec files
Handle code review comments.
- Modified READMEs to incorporate to suggestions.
- Add a test for ReadWriteMany access mode.
- Use util.UnmountPath in TearDown.
- Add ReadOnly flag to PortworxVolumeSource
- Use hostname:port instead of unix sockets
- Delete the mount dir in TearDown.
- Fix link issue in persistentvolumes README
- In unit test check for mountpath after Setup is done.
- Add PVC Claim Name as a Portworx Volume Label
Generated code and documentation.
- Updated swagger spec
- Updated api-reference docs
- Updated generated code under pkg/api/v1
Godeps update for Portworx Volume Driver
- Adds github.com/libopenstorage/openstorage
- Adds go.pedge.io/pb/go/google/protobuf
- Updates Godep Licenses
Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)
add aggregation integration test
Wires up an integration test which runs a full kube-apiserver, the wardle server, and the kube-aggregator and creates the APIservice object for the wardle server. Without services and DNS the aggregator doesn't proxy, but it does ensure we don't have an obvious panic or bring up failure.
@sttts @ncdc
Automatic merge from submit-queue
clean up generic apiserver options
Clean up generic apiserver options before we tag any levels. This makes them more in-line with "normal" api servers running on the platform.
Also remove dead example code.
@sttts
These lines may contain information that cannot be safely logged to
disk. Only deal with known parameters, also bump to V(2) to be
consistent with other logging.
Automatic merge from submit-queue (batch tested with PRs 42044, 41694, 41927, 42050, 41987)
Simplify and fix hack/{verify,update}-staging-{client-go,godeps}.sh
- merge `hack/{verify,update}-staging-client-go.sh`
- pin godep with shared code
- remove godep-restore completely from the process and replace with a simple check that godeps are restored
- add safety check in `staging/copy.sh` that there is no lingering `k8s.io/apimachinery` in the GOPATH which would lead to inconsistent client-go builds (!)
- check that all these scripts only operate in a clean working dir.
Automatic merge from submit-queue (batch tested with PRs 35408, 41915, 41992, 41964, 41925)
add secret option to flag
To resolve the issue of security(pr #35030 ),
> @smarterclayton commented 5 days ago
> This is unfortunately not all flags that could be secrets. The best option would be to add support in spf13/pflag to tag a flag as a secret, and then use that bit to determine the list.
>
> Also, Command() could be used in contexts that need exact parameters (for subshell execution), so we would need to add a new method or extend the signature here to allow exact flags to be retrieved.
we could add a secret option to the flags.
Automatic merge from submit-queue (batch tested with PRs 41954, 40528, 41875, 41165, 41877)
Updating apiserver to return 202 when resource is being deleted asynchronously via cascading deletion
As per https://github.com/kubernetes/kubernetes/issues/33196#issuecomment-278440622.
cc @kubernetes/sig-api-machinery-pr-reviews @smarterclayton @caesarxuchao @bgrant0607 @kubernetes/api-reviewers
```release-note
Updating apiserver to return http status code 202 for a delete request when the resource is not immediately deleted because of user requesting cascading deletion using DeleteOptions.OrphanDependents=false.
```
Automatic merge from submit-queue (batch tested with PRs 41621, 41946, 41941, 41250, 41729)
Refactor printers and describers into their own package.
This sets the stage for using printer code from the server side (decoupled from kubectl) and loosens the coupling between kubectl and the printers. `pkg/printers` contains interfaces and has an import restriction against pulling in API specific code, while `pkg/printers/internalversion` can be used for internal types.
Add a method on `Factory` for retrieving PrinterForCommand which uses the Scheme and RESTMapper from the Factory, not the hardcoded ones. This further separates kubectl from the core API scheme and allows better composition.
Change NamePrinter to use RESTMapper (previously it was hardcoding those conversions). This means that we now return plural resource names (`pods/foo`) but is correct once aliases and shortnames start being returned by the mapper.
This is a prerequisite for server side get, but is pure refactor (contains no new features).
@deads2k @liggitt
Automatic merge from submit-queue
apiserver: self-signed in-memory cert for loopback
We had complicated logic before to find a matching (SNI- or server-) cert for the loopback device. In the worst case, bootstrapping failed because the provided certs didn't have the binding IP or localhost.
We now pass an artifical `ServerName` "apiserver-loopback-client" with the loopback client and install a self-signed in-memory cert+key into the server for that hostname. The client accepts the corresponding cert.
Fixing https://github.com/kubernetes/kubernetes/issues/41081
Automatic merge from submit-queue
redact detailed errors from healthz and expose in default policy
Makes `/healthz` less sensitive and exposes it by default.
@kubernetes/sig-auth-pr-reviews @kubernetes/sig-api-machinery-misc @liggitt
Automatic merge from submit-queue (batch tested with PRs 41797, 41793, 41795, 41807, 41781)
Don't use base64 encoding with etcd v3
In https://github.com/kubernetes/kubernetes/pull/36229 we changed the default storage to etcd v3.
This in fact is a fix to that PR.
With etcd v3, the base64-encoding is no longer needed - so we use it only if etcd v2 is explicitly requested as storage format.
@lavalamp
Automatic merge from submit-queue (batch tested with PRs 41797, 41793, 41795, 41807, 41781)
tweak a list to a watch verb when requested
Related to https://github.com/kubernetes/kubernetes/pull/41774
@wojtek-t does this do what's needed?
Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)
client-gen: independent scheme for clientsets
This PR adds a clientset internal scheme instead of using `pkg/api.Scheme`. **The clientset API stays the same.**
In detail:
- introduce a scheme for each clientset, i.e. do not use `pkg/api.Scheme+Registry+Codec+ParameterCodecs`.
This makes it easier to compose client-go's clientset (which is rewritten in `staging/copy.sh` and therefore hardcoded to use `k8s.io/client-go/pkg/api.Scheme+Registry+Codecs+ParameterCodecs`) with third-party clientsets (kube-aggregator, openshift, federation) which are not rewritten using `copy.sh` as all of them are self-contained and therefore relocatable.
This fixes https://github.com/kubernetes/kubernetes/pull/41403/files#diff-76edfb07dee54ff7ddeda25c33c10d29R81 and prepares client-gen for use in OpenShift.
- register types into the clientset scheme via `AddToScheme` for versioned clientsets. This decouples the client-go clients from announce+registration (internal clients continue using announce+registry and apigroup installers).
This reduces complexity for client-go, possibly remove the necessity for the announce+register machinery for many use-cases, maybe even to delete it mid-term.
- port federation and testgroup `install/install.go` to `announced.GroupMetaFactory` in order to have a proper `Install.Install(...)` func for registration.
With the first change it's easy to add the types of one clientset to the scheme of the other using the `clientset/scheme.AddToScheme` method. This allows to use cross-clientset `runtime.RawExtensions`:
```golang
import (
"k8s.io/client-go/kubernetes"
clientsetscheme "k8s.io/client-go/kuberentes/scheme"
aggregatorclientsetscheme "k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/scheme"
)
kclientset, _ := kubernetes.NewForConfig(c)
aggregatorclientsetscheme.AddToScheme(clientsetscheme.Scheme)
```
Kubernetes types with a `RawExtension` can en/decode aggregator types after this.
TODO:
- [x] fix fake clientsets
- [x] get the `*Options` types registered correctly for core, compare DO-NOT-MERGE commit.
- [x] get prefered version right in internal client. Do we need all versions registered in the internal client to support negotiation?
- [x] run `staging/copy.sh` and run tests: https://github.com/kubernetes/kubernetes/pull/41744
- ~~[ ] fixup usage through-out the code-base~~
- **Follow-up**: move `import_known_versions.go` files somewhere such that import of the `api.Scheme` package automatically installs the apigroups. It looks like we depended on the import fo the clientset for this purpose.
Kubernetes Submit Queue