Add tests to watch behavior in both protocols (http and websocket)
against all 3 media types. Adopt the
`application/vnd.kubernetes.protobuf;stream=watch` media type for the
content that comes back from a watch call so that it can be
distinguished from a Status result.
Automatic merge from submit-queue
Make etcd cache size configurable
Instead of the prior 50K limit, allow users to specify a more sensible size for their cluster.
I'm not sure what a sensible default is here. I'm still experimenting on my own clusters. 50 gives me a 270MB max footprint. 50K caused my apiserver to run out of memory as it exceeded >2GB. I believe that number is far too large for most people's use cases.
There are some other fundamental issues that I'm not addressing here:
- Old etcd items are cached and potentially never removed (it stores using modifiedIndex, and doesn't remove the old object when it gets updated)
- Cache isn't LRU, so there's no guarantee the cache remains hot. This makes its performance difficult to predict. More of an issue with a smaller cache size.
- 1.2 etcd entries seem to have a larger memory footprint (I never had an issue in 1.1, even though this cache existed there). I suspect that's due to image lists on the node status.
This is provided as a fix for #23323
Automatic merge from submit-queue
genericapiserver: Moving InstallSwaggerAPI to Run
Ref https://github.com/kubernetes/kubernetes/pull/21190#discussion_r57494673
Moving InstallSwaggerAPI() from InstallAPIGroups() to Run(). This allows the use of InstallAPIGroups() multiple times or using InstallAPIGroup() directly.
cc @jianhuiz @kubernetes/sig-api-machinery
Added selector generation to Job's
strategy.Validate, right before validation.
Can't do in defaulting since UID is not known.
Added a validation to Job to ensure that the generated
labels and selector are correct when generation was requested.
This happens right after generation, but validation is in a better
place to return an error.
Adds "manualSelector" field to batch/v1 Job to control selector generation.
Adds same field to extensions/__internal. Conversion between those two
is automatic.
Adds "autoSelector" field to extensions/v1beta1 Job. Used for storing batch/v1 Jobs
- Default for v1 is to do generation.
- Default for v1beta1 is to not do it.
- In both cases, unset == false == do the default thing.
Release notes:
Added batch/v1 group, which contains just Job, and which is the next
version of extensions/v1beta1 Job.
The changes from the previous version are:
- Users no longer need to ensure labels on their pod template are unique to the enclosing
job (but may add labels as needed for categorization).
- In v1beta1, job.spec.selector was defaulted from pod labels, with the user responsible for uniqueness.
In v1, a unique label is generated and added to the pod template, and used as the selector (other
labels added by user stay on pod template, but need not be used by selector).
- a new field called "manualSelector" field exists to control whether the new behavior is used,
versus a more error-prone but more flexible "manual" (not generated) seletor. Most users
will not need to use this field and should leave it unset.
Users who are creating extensions.Job go objects and then posting them using the go client
will see a change in the default behavior. They need to either stop providing a selector (relying on
selector generation) or else specify "spec.manualSelector" until they are ready to do the former.
Recycle controller tries to recycle or delete a PV several times.
It stores count of failed attempts and timestamp of the last attempt in
annotations of the PV.
By default, the controller tries to recycle/delete a PV 3 times in
10 minutes interval. These values are configurable by
kube-controller-manager --pv-recycler-maximum-retry=X --pvclaimbinder-sync-period=Y
arguments.
Fixes#19860 (it may be easier to look at the issue to see exact sequence
to reproduce the bug and understand the fix).
When PersistentVolumeProvisionerController.reconcileClaim() is called with the
same claim in short succession (e.g. the claim is created by an user and
at the same time periodic check of all claims is scheduled), the second
reconcileClaim() call gets an old copy of the claim as its parameter.
The method should always reload the claim to get a fresh copy with all
annotations, possibly added by previous reconcileClaim() call.
The same applies to PersistentVolumeClaimBinder.syncClaim().
Also update all the test to store claims in "fake" API server before calling
syncClaim and reconcileClaim.
Combine the fields that will be used for content transformation
(content-type, codec, and group version) into a single struct in client,
and then pass that struct into the rest client and request. Set the
content-type when sending requests to the server, and accept the content
type as primary.
Will form the foundation for content-negotiation via the client.
Pass down into the server initialization the necessary interface for
handling client/server content type negotiation. Add integration tests
for the negotiation.
A NegotiatedSerializer is passed into the API installer (and
ParameterCodec, which abstracts conversion of query params) that can be
used to negotiate client/server request/response serialization. All
error paths are now negotiation aware, and are at least minimally
version aware.
Watch is specially coded to only allow application/json - a follow up
change will convert it to use negotiation.
Ensure the swagger scheme will include supported serializations - this
now includes application/yaml as a negotiated option.
Public utility methods and JWT parsing, and controller specific logic.
Also remove the coupling between ServiceAccountTokenGetter and the
authenticator class.
1. Name default scheduler with name `kube-scheduler`
2. The default scheduler only schedules the pods meeting the following condition:
- the pod has no annotation "scheduler.alpha.kubernetes.io/name: <scheduler-name>"
- the pod has annotation "scheduler.alpha.kubernetes.io/name: kube-scheduler"
update gofmt
update according to @david's review
run hack/test-integration.sh, hack/test-go.sh and local e2e.test
We wrap TimeoutHandler with RecoverPanics, but httplog does not
implement http.CloseNotifier, which causes a naive watch from curl
against the insecure port to fail.
For now, implement CloseNotifier (but we should consider removing
httplog now that we have other tools in the stack to manage it).
Before we tried both groups, first api then experimental. If
experimental failed, the error message would be discarded and the
message for api would be shown. This commit changes this behavior so
that we detect which api group we should be using and only show the
error for that group.
A lot of packages use StringSet, but they don't use anything else from
the util package. Moving StringSet into another package will shrink
their dependency trees significantly.
Avoid TTL by deleting pods immediately when they aren't
scheduled, and letting the Kubelet delete them otherwise.
Ensure the Kubelet uses pod.Spec.TerminationGracePeriodSeconds
when no pod.DeletionGracePeriodSeconds is available.
These are "Bootstrap Controllers" as distinct from
the controllers in the controller-manager binary - they
are necessary for the cluster to start running.
* Add an allocator which saves state in etcd
* Perform PortalIP allocation check on startup and periodically afterwards
Also expose methods in master for downstream components to handle IP allocation
/ master registration themselves.
This commit adds support to core resources to enable deferred deletion
of resources. Clients may optionally specify a time period after which
resources must be deleted via an object sent with their DELETE. That
object may define an optional grace period in seconds, or allow the
default "preferred" value for a resource to be used. Once the object
is marked as pending deletion, the deletionTimestamp field will be set
and an etcd TTL will be in place.
Clients should assume resources that have deletionTimestamp set will
be deleted at some point in the future. Other changes will come later
to enable graceful deletion on a per resource basis.
In order to support graceful deletion, the resource object will
need access to the TTL value in etcd. Also, in the future we
may want to get the creation index (distinct from modifiedindex)
and expose it to clients. Change EtcdResourceVersioner to be
more type specific (objects vs lists) and provide a default
implementation that relies on the internal API convention.
Also, rename etcd_tools.go to etcd_helper.go and split a few
things up.
Also make sure all POST operations return 201 by default.
Removes the remainder of the asych logic in RESTStorage and
leaves it up to the API server to expose that behavior.
Added function to read basic ACL from a CSV file.
Added implementation of Authorize based on that file's policies.
Added docs on authentication and authorization.
Added example file and tested it.
Without this, tests which create a master
will panic with a nil pointer when the periodic
podCache update runs.
Deleted unused FakePodInfoGetter.
Added FakeKubeletClient.
Passed to master.New.
Required a KubeletClient in master.New.
Use some constants for tokens.
Refactor tokenfile creation to function.
Reorder some test cases to make lookups follow creates so they succeed.
Add expected status code to test cases (some are not quite what expected,
so filed bugs #2112, #2113, #2114)
Check expected status codes.
Close Body after each iterations so that we don't run out of file handles
when I add even more test cases in the next PR.
Handle that it is unpredictable whether status 200 or 202 is returned.
Added basic interface for authorizer implementations.
Added default "authorize everything" and "authorize nothing
implementations.
Added authorization check immediately after authentication check.
Added an integration test of authorization at the HTTP level of
abstraction.
Callsites no longer allocate a mux.
Master now exposes method to install handlers
which use the master's auth code. Not used
but forks (openshift) are expected to use these
methods. These methods will later be a point
for additional plug-in functionality.
Integration tests now use the master-provided
handler which has auth, rather than using the mux,
which didn't. Fix TestWhoAmI now that /_whoami
sits behind auth.
Added new endpoint /_whoami for debugging authentication.
Added integration test which checks that a user is authenticated
using token authentication.
Rearranged initialization of authenticator to support preceeding.
Moved code from cmd/apiserver to pkg/master.
test/integration/client_test made to use a master object,
instead of an apiserver.Handle.
Subsequent PRs will move more handler-installation into
pkg/master, with the goal that every http.Handler of a
standalone apiserver process can also be tested
in a "testing"-style go test.
In particular, a subsequent PR will test
authorization.
Allows us to define different watch versioning regimes in the future
as well as to encode information with the resource version.
This changes /watch/resources?resourceVersion=3 to start the watch at
4 instead of 3, which means clients can read a resource version and
then send it back to the server. Clients should no longer do math on
resource versions.
* Allows consumers to provide their own transports for common cases.
* Supports KUBE_API_VERSION on test cases for controlling which
api version they test against
* Provides a common flag registration method for CLIs that need
to connect to an API server (to avoid duplicating flags)
* Ensures errors are properly returned by the server
* Add a Context field to client.Config
* Defaults to v1beta1
* apiserver takes -storage_version which controls etcd storage version
and the version of the client used to connect to other apiservers
* Changed signature of client.New to add version parameter
* All controller code and component code prefers the oldest (most common)
server version
* Make Codec separate from Scheme
* Move EncodeOrDie off Scheme to take a Codec
* Make Copy work without a Codec
* Create a "latest" package that imports all versions and
sets global defaults for "most recent encoding"
* v1beta1 is the current "latest", v1beta2 exists
* Kill DefaultCodec, replace it with "latest.Codec"
* This updates the client and etcd to store the latest known version
* EmbeddedObject is per schema and per package now
* Move runtime.DefaultScheme to api.Scheme
* Split out WatchEvent since it's not an API object today, treat it
like a special object in api
* Kill DefaultResourceVersioner, instead place it on "latest" (as the
package that understands all packages)
* Move objDiff to runtime.ObjectDiff
Convert host:port and URLs passed to client.New() into the proper
values, and return an error if the value is invalid. Change CLI
to return an error if -master is invalid. Remove Client.rawRequest
which was not in use, and fix the involved tests. Add NewOrDie
Preserves the behavior of the client to not auth when a non-https
URL is passed (although in the future this should be corrected).