Automatic merge from submit-queue
Specific error message on failed rolling update issued by older kubectl against 1.4 master
Fix#32706
`kubernetes-e2e-gke-1.4-1.3-kubectl-skew` (1.3 kubectl and 1.4 master) test suite failed with:
```
k8s.io] Kubectl client [k8s.io] Kubectl rolling-update should support rolling-update to same image [Conformance]
...
Error from server: object is being deleted: replicationcontrollers "e2e-test-nginx-rc" already exists error: exit status 1 not to have occurred
```
It's because the old RC had an orphanFinalizer, so it is not deleted from the key-value store immediately. In turn, the creation of the new RC of the same name failed.
In this failure, the RC and pods are updated, it's just that the RC is of different name, i.e., original name + a hash generated based on podTemplate. The error is confusing to user, but not that bad. So this PR just prints a warning message to instruct users how to work around.
1.4 kubectl rolling-update uses different logic so it's working.
@lavalamp @gmarek @janetkuo @pwittrock
cc @liggitt for the ctx changes.
Automatic merge from submit-queue
Centralize install code
Trying to figure out a way to do this that makes the changes as painless to roll out as possible. This is going to be a multi-step process...
Automatic merge from submit-queue
Allow webhook authenticator to use TokenReviewsInterface
Refactors the authentication webhook to be able to be fed a kubeconfig file or a TokenReviewsInterface
Fixes a bug with webhooks not retrying on 429/500 errors
Related to https://github.com/kubernetes/kubernetes/pull/32518#discussion_r78450170
Automatic merge from submit-queue
Change rbac roleref to reflect the information we want
@liggitt @ericchiang This is a version of https://github.com/kubernetes/kubernetes/pull/31359 which updates the `RoleRef` to be (I think) the type that we want, with a group, resource, and name.
This is **not** backwards compatible with any existing data. I'm ok with doing this since rbac was considered alpha, but its something to consider.
If we want this instead, I'll close the previous pull (or update it with this content).
Automatic merge from submit-queue
update taints e2e, restrict taints operation with key, effect
Since taints are now unique by key, effect on a node, this PR is to restrict existing taints adding/removing/updating operations in taints e2e.
Also fixes https://github.com/kubernetes/kubernetes/issues/31066#issuecomment-242870101
Related prior Issue/PR #29362 and #30590
Automatic merge from submit-queue
add group prefix matching for kubectl usage
Adds prefix matching for groups so that `kubectl get storageclass.storage` matches `storageclass.storage.k8s.io`.
@kubernetes/kubectl
Automatic merge from submit-queue
Split path validation into a separate library
This PR splits path segment validation into it's own package. This cuts off one of the restclient's dependency paths to some docker packages, and completely eliminates its dependency on go-restful swagger validation.
cc @kubernetes/sig-api-machinery
Automatic merge from submit-queue
Generate versioned client for pvc
I noticed while trying to use client-go that there is no way to get PVC with versioned clientset. I don't know why it is excluded, I cannot find any discussion about this, so I am creating this assuming it is just an oversight...
Automatic merge from submit-queue
Add AppArmor feature gate
Add option to disable AppArmor via a feature gate. This PR treats AppArmor as Beta, and thus depends on https://github.com/kubernetes/kubernetes/pull/31471 (I will remove `do-not-merge` once that merges).
Note that disabling AppArmor means that pods with AppArmor annotations will be rejected in validation. It does not mean that the components act as though AppArmor was never implemented. This is by design, because we want to make it difficult to accidentally run a Pod with an AppArmor annotation without AppArmor protection.
/cc @dchen1107
Automatic merge from submit-queue
Add validation preventing recycle of / in a hostPath PV
Adds a validation that prevents a user from recycling `/` when it is used in a hostPath PV
cc @kubernetes/sig-storage
Automatic merge from submit-queue
add validation for PV spec to ensure correct values are used for ReclaimPolicy on initial create
k8 currently allows invalid values for ReclaimPolicy (i.e. "scotto") - this allows the PV to be created and even bound, however, when the pvc or pod is deleted and the recycler is triggered, an error is thrown
```
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
36s 36s 1 {persistentvolume-controller } Warning VolumeUnknownReclaimPolicy Volume has unrecognized PersistentVolumeReclaimPolicy
```
New behavior will not allow the user to create the PV:
```
[root@k8dev nfs]# kubectl create -f nfs-pv-bad.yaml
The PersistentVolume "pv-gce" is invalid: spec.persistentVolumeReclaimPolicy: Unsupported value: "scotto": supported values: Delete, Recycle, Retain
```
Automatic merge from submit-queue
support Azure data disk volume
This is a WIP of supporting azure data disk volume. Will add test and dynamic provisioning support once #29006 is merged
replace #25915fix#23259
@kubernetes/sig-storage
@colemickens @brendandburns
Automatic merge from submit-queue
[GarbageCollector] Allow per-resource default garbage collection behavior
What's the bug:
When deleting an RC with `deleteOptions.OrphanDependents==nil`, garbage collector is supposed to treat it as `deleteOptions.OrphanDependents==true", and orphan the pods created by it. But the apiserver is not doing that.
What's in the pr:
Allow each resource to specify the default garbage collection behavior in the registry. For example, RC registry's default GC behavior is Orphan, and Pod registry's default GC behavior is CascadingDeletion.
Automatic merge from submit-queue
Add readyReplicas to replica sets
@bgrant0607 for the api changes
@bprashanth for the controllers changes
@deads2k fyi
Automatic merge from submit-queue
Move init-container feature from alpha to beta.
```release-note
Moved init-container feature from alpha to beta.
In 1.3, an init container is specified with this annotation key
on the pod or pod template: `pods.alpha.kubernetes.io/init-containers`.
In 1.4, either that key or this key: pods.beta.kubernetes.io/init-containers`,
can be used.
When you GET an object, you will see both annotation keys with the same values.
You can safely roll back from 1.4 to 1.3, and things with init-containers
will still work (pods, deployments, etc).
If you are running 1.3, only use the alpha annotation, or it may be lost when
rolling forward.
The status has moved from annotation key
`pods.beta.kubernetes.io/init-container-statuses` to
`pods.beta.kubernetes.io/init-container-statuses`.
Any code that inspects this annotation should be changed to use the new key.
State of Initialization will continue to be reported in both pods.alpha.kubernetes.io/initialized
and in `podStatus.conditions.{status: "True", type: Initialized}`
```
Mini-design for this change:
Goals:
1. A user can create an object with the beta annotation
on 1.4, and it works. The fact that the annotation has beta
in it communicates to the user that the feature is beta,
and so the user should have confidence in using it. Preferably,
when the user gets the annotation back, he see the beta
annotation.
1) If someone had an existing alpha object in their apiserver,
such as a RS with a pod template with an init-containers
annotation on it, it should continue to work (init containers
run) when stack upgraded to 1.4.
2) If someone is using a chart or blog post that has alpha
annotation on it and they create it on a 1.4 cluster, it should
work.
3) If someone had something with an init container in 1.4
and they roll back stack to 1.3, it should not silently stop
working (init containers don't run anymore).
To meet all these, we mirror an absent beta label from the alpha
key and vice versa. If they are out of sync, we use the alpha
one. We do this in conversion since there was already logic there.
In 1.3 code, all annotations are preserved across a round trip
(v1 -> api -> v1), and the alpha annotation turns into the internal
field that kubelet uses.
In 1.4 code, the alpha annotation is always preserved across
a round trip, and a beta annotation is always set equal to
the alpha one, after a round trip.
Currently, the kubelet always sees the object after a round trip
when it GETs it. But, we don't want to rely on that behavior,
since it will break when fastpath is implemented.
So, we rely on this:
all objects either are created with an alpha annotation (1.3 or 1.4
code) or are created with a beta annotation under 1.4. In the later
case, they are round tripped at creation time, and so get both
annotations. So all subsequent GETs see both labels.
The serviceAccountName is occasionally useful for clients running on
Kube that need to know who they are when talking to other components.
The nodeName is useful for PetSet or DaemonSet pods that need to make
calls back to the API to fetch info about their node.
Both fields are immutable, and cannot easily be retrieved in another
way.
Automatic merge from submit-queue
Make labels, fields expose selectable requirements
What?
This is to change the labels/fields Selector interface and make them expose selectable requirements. We reuse labels.Requirement struct for label selector and add fields.Requirement for field selector.
Why?
In order to index labels/fields, we need them to tell us three things: index key (a field or a label), operator (greater, less, or equal), and value (string, int, etc.). By getting selectable requirements, we are able to pass them down and use them for indexing in storage layer.
Automatic merge from submit-queue
Basic scaler/reaper for petset
Currently scaling or upgrading a petset is more complicated than it should be. Would be nice if this made code freeze on friday. I'm planning on a follow up change with generation number and e2es post freeze.
Automatic merge from submit-queue
change all PredicateFunc to use SelectionPredicate
What?
- This PR changes all PredicateFunc in registry to return SelectionPredicate instead of Matcher interface.
Why?
- We want to pass SelectionPredicate to storage layer. Matcher interface did not expose enough information for indexing.
```relnote
Moved init-container feature from alpha to beta.
In 1.3, an init container is specified with this annotation key
on the pod or pod template: `pods.alpha.kubernetes.io/init-containers`.
In 1.4, either that key or this key: pods.beta.kubernetes.io/init-containers`,
can be used.
When you GET an object, you will see both annotation keys with the same values.
You can safely roll back from 1.4 to 1.3, and things with init-containers
will still work (pods, deployments, etc).
If you are running 1.3, only use the alpha annotation, or it may be lost when
rolling forward.
The status has moved from annotation key
`pods.beta.kubernetes.io/init-container-statuses` to
`pods.beta.kubernetes.io/init-container-statuses`.
Any code that inspects this annotation should be changed to use the new key.
State of Initialization will continue to be reported in both pods.alpha.kubernetes.io/initialized
and in `podStatus.conditions.{status: "True", type: Initialized}`
```
Mini-design for this change:
Goals:
1. A user can create an object with the beta annotation
on 1.4, and it works. The fact that the annotation has beta
in it communicates to the user that the feature is beta,
and so the user should have confidence in using it. Preferably,
when the user gets the annotation back, he see the beta
annotation.
1) If someone had an existing alpha object in their apiserver,
such as a RS with a pod template with an init-containers
annotation on it, it should continue to work (init containers
run) when stack upgraded to 1.4.
2) If someone is using a chart or blog post that has alpha
annotation on it and they create it on a 1.4 cluster, it should
work.
3) If someone had something with an init container in 1.4
and they roll back stack to 1.3, it should not silently stop
working (init containers don't run anymore).
To meet all these, we mirror an absent beta label from the alpha
key and vice versa. If they are out of sync, we use the alpha
one. We do this in conversion since there was already logic there.
In 1.3 code, all annotations are preserved across a round trip
(v1 -> api -> v1), and the alpha annotation turns into the internal
field that kubelet uses.
In 1.4 code, the alpha annotation is always preserved across
a round trip, and a beta annotation is always set equal to
the alpha one, after a round trip.
Currently, the kubelet always sees the object after a round trip
when it GETs it. But, we don't want to rely on that behavior,
since it will break when fastpath is implemented.
So, we rely on this:
all objects either are created with an alpha annotation (1.3 or 1.4
code) or are created with a beta annotation under 1.4. In the later
case, they are round tripped at creation time, and so get both
annotations. So all subsequent GETs see both labels.
Automatic merge from submit-queue
Fix pvc requests.storage validation
A `PersistentVolumeClaim` should not be able to request a negative amount of storage.
/cc @kubernetes/sig-storage @kubernetes/rh-cluster-infra @deads2k
Convert single GV and lists of GVs into an interface that can handle
more complex scenarios (everything internal, nothing supported). Pass
the interface down into conversion.
Automatic merge from submit-queue
Allow setting permission mode bits on secrets, configmaps and downwardAPI files
cc @thockin @pmorie
Here is the first round to implement: https://github.com/kubernetes/kubernetes/pull/28733.
I made two commits: one with the actual change and the other with the auto-generated code. I think it's easier to review this way, but let me know if you prefer in some other way.
I haven't written any tests yet, I wanted to have a first glance and not write them till this (and the API) are more close to the "LGTM" :)
There are some things:
* I'm not sure where to do the "AND 0777". I'll try to look better in the code base, but suggestions are always welcome :)
* The write permission on group and others is not set when you do an `ls -l` on the running container. It does work with write permissions to the owner. Debugging seems to show that is something happening after this is correctly set on creation. Will look closer.
* The default permission (when the new fields are not specified) are the same that on kubernetes v1.3
* I do realize there are conflicts with master, but I think this is good enough to have a look. The conflicts is with the autog-enerated code, so the actual code is actually the same (and it takes like ~30 minutes to generate it here)
* I didn't generate the docs (`generated-docs` and `generated-swagger-docs` from `hack/update-all.sh`) because my machine runs out of mem. So that's why it isn't in this first PR, will try to investigate and see why it happens.
Other than that, this works fine here with some silly scripts I did to create a secret&configmap&downwardAPI, a pod and check the file permissions. Tested the "defaultMode" and "mode" for all. But of course, will write tests once this is looking fine :)
Thanks a lot again!
Rodrigo
Automatic merge from submit-queue
Change kubectl create to use dynamic client
https://github.com/kubernetes/kubernetes/issues/16764https://github.com/kubernetes/kubernetes/issues/3955
This is a series of changes to allow kubectl create to use discovery-based REST mapping and dynamic clients.
cc @kubernetes/sig-api-machinery
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
kubectl will no longer do client-side defaulting on create and replace.
```
This implements the proposal in:
docs/proposals/secret-configmap-downwarapi-file-mode.md
Fixes: #28317.
The mounttest image is updated so it returns the permissions of the linked file
and not the symlink itself.
Automatic merge from submit-queue
Add NodeName to EndpointAddress object
Adding a new string type `nodeName` to api.EndpointAddress.
We could also do *ObjectReference to the api.Node object instead, which would be more precise for the future.
```
type ObjectReference struct {
Kind string `json:"kind,omitempty"`
Namespace string `json:"namespace,omitempty"`
Name string `json:"name,omitempty"`
UID types.UID `json:"uid,omitempty"`
APIVersion string `json:"apiVersion,omitempty"`
ResourceVersion string `json:"resourceVersion,omitempty"`
// Optional. If referring to a piece of an object instead of an entire object, this string
// should contain information to identify the sub-object. For example, if the object
// reference is to a container within a pod, this would take on a value like:
// "spec.containers{name}" (where "name" refers to the name of the container that triggered
// the event) or if no container name is specified "spec.containers[2]" (container with
// index 2 in this pod). This syntax is chosen only to have some well-defined way of
// referencing a part of an object.
// TODO: this design is not final and this field is subject to change in the future.
FieldPath string `json:"fieldPath,omitempty"`
}
```
Automatic merge from submit-queue
Simplify canonical element term in deepcopy
Replace the old functional canonical element term in deepcopy registration with direct struct instantiation.
The old way was an artifact of non-uniform pointer/non-pointer types in the signature of deepcopy function. Since we changed that to always be a pointer, we can simplify the code.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30341)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Run goimport for the whole repo
While removing GOMAXPROC and running goimports, I noticed quite a lot of other files also needed a goimport format. Didn't commit `*.generated.go`, `*.deepcopy.go` or files in `vendor`
This is more for testing if it builds.
The only strange thing here is the gopkg.in/gcfg.v1 => github.com/scalingdata/gcfg replace.
cc @jfrazelle @thockin
Automatic merge from submit-queue
Validate list objects when building scheme
The first commit is #29468. This PR is a followup.
Modified the description of **List** in api-convention.md to:
`The name of a list kind must end with "List". Lists have a limited set of common metadata. All lists use the required "items" field to contain the array of objects they return. Any kind that has the "items" field must be a list kind.`
Also added code to enforce the above rules during scheme registration.
@kubernetes/sig-api-machinery @bgrant0607 @erictune @lavalamp @smarterclayton
Automatic merge from submit-queue
Add support to quota pvc storage requests
Adds support to quota cumulative `PersistentVolumeClaim` storage requests in a namespace.
Per our chat today @markturansky @abhgupta - this is not done (lacks unit testing), but is functional.
This lets quota enforcement for `PersistentVolumeClaim` to occur at creation time. Supporting bind time enforcement would require substantial more work. It's possible this is sufficient for many, so I am opening it up for feedback.
In the future, I suspect we may want to treat local disk in a special manner, but that would have to be a different resource altogether (i.e. `requests.disk`) or something.
Example quota:
```
apiVersion: v1
kind: ResourceQuota
metadata:
name: quota
spec:
hard:
persistentvolumeclaims: "10"
requests.storage: "40Gi"
```
/cc @kubernetes/rh-cluster-infra @deads2k
Automatic merge from submit-queue
Initial support for pod eviction based on disk
This PR adds the following:
1. node reports disk pressure condition based on configured thresholds
1. scheduler does not place pods on nodes reporting disk pressure
1. kubelet will not admit any pod when it reports disk pressure
1. kubelet ranks pods for eviction when low on disk
1. kubelet evicts greediest pod
Follow-on PRs will need to handle:
1. integrate with new image gc PR (https://github.com/kubernetes/kubernetes/pull/27199)
1. container gc policy should always run (will not be launched from eviction, tbd who does that)
1. this means kill pod is fine for all eviction code paths since container gc will remove dead container
1. min reclaim support will just poll summary provider (derek will do follow-on)
1. need to know if imagefs is same device as rootfs from summary (derek follow-on)
/cc @vishh @kubernetes/sig-node
Automatic merge from submit-queue
storage error: precondition failure should return invalid object error
In introducing the preconditions by @caesarxuchao , if check preconditions failed, it returns resource version conflict error. This is the wrong error to return, and instead it should return invalid object error. We need to separate these two types of errors.
See the implementation in etcd3 [https://github.com/kubernetes/kubernetes/blob/master/pkg/storage/etcd3/store.go#L467].
Also renaming "ErrCodeResourceVersionConflicts" to "ErrCodeVersionConflicts" for simpler reading.
Automatic merge from submit-queue
Fix panic in schema test
If the swagger files for testing are lost, the func `loadSchemaForTest` or `NewSwaggerSchemaFromBytes` will return a non-nil error and a nil schema. In this case, the calling for `ValidateBytes` will result in panic. So, call Fatalf instead of Errorf.
Also fix minor typos.
Test logs:
```
--- FAIL: TestLoad (0.01s)
schema_test.go:131: Failed to load: open ../../../api/swagger-spec/v1.json: no such file or directory
--- FAIL: TestValidateOk (0.00s)
schema_test.go:138: Failed to load: open ../../../api/swagger-spec/v1.json: no such file or directory
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x20 pc=0x4d52df]
goroutine 10 [running]:
panic(0x15fffa0, 0xc8200100a0)
/usr/local/go/src/runtime/panic.go:481 +0x3e6
testing.tRunner.func1(0xc820085a70)
/usr/local/go/src/testing/testing.go:467 +0x192
panic(0x15fffa0, 0xc8200100a0)
/usr/local/go/src/runtime/panic.go:443 +0x4e9
k8s.io/kubernetes/pkg/api/validation.TestValidateOk(0xc820085a70)
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/api/validation/schema_test.go:159 +0x79f
testing.tRunner(0xc820085a70, 0x22aad68)
/usr/local/go/src/testing/testing.go:473 +0x98
created by testing.RunTests
/usr/local/go/src/testing/testing.go:582 +0x892
FAIL k8s.io/kubernetes/pkg/api/validation 0.048s
```
Automatic merge from submit-queue
Fix init container update validation for pods
Partial fix#26840
The remaining issues with `kubectl apply` on pods with init containers
are caused by temporary annotation-based representation and
will resolve themselves once init containers leave alpha state.
Also, this PR makes sure internal and external objects don't get mixed up by the
PATCH handler (see related issue #25106).
This PR is an alternative for #28557 which met criticism from @smarterclayton
and @liggitt for working around the temporary issue with annotations.
#28557 is a full fix for #26840 and contains an e2e test that cannot pass
without the `VolumeMounts` workaround. As there appears to be no
good way to include an e2e test that's known to be failing in k8s source,
I've removed it from this PR.
Either this PR or #28557 should be applied, but not both.
Automatic merge from submit-queue
Allow handlers earlier in a request flow to inject a UID for an object
This lets admission controllers specify a stable UID for an object prior to its creation. That lets the admission controller then record a reference to the object on another resource using that stable UID prior to the object being created. This would be a prerequisite for supporting quota reservations.
/cc @smarterclayton @lavalamp @deads2k
Automatic merge from submit-queue
Custom conversions between RCs/RSs
Needed for building storages that still need to work with RCs but also want to facilitate RSs.
cc: @deads2k @smarterclayton @liggitt
Automatic merge from submit-queue
Fix GPU resource validation
This fixes scheduling of pods with GPU resources. The change was never upstreamed during the 1.3 beta period, as it got lost in the noise of other changes in our fork. Ooops. I'll submit a cherry-pick request for 1.3.1 as soon as this lands in master.
Because of defaulting, requests are always set if limits are. Thus, the check can never succeed. Instead, make sure that the two values are equal.
Also, remove a few other error messages and remove unnecessary Sprintf calls.
Kubernetes Submit Queue
