Automatic merge from submit-queue (batch tested with PRs 40497, 39769, 40554, 40569, 40597)
NetworkPolicy validation improvements
I noticed while implementing NetworkPolicy that I we weren't validating the "Ports" field at all.
The docs are actually completely silent about what a string-valued Port field is supposed to mean. I had guessed it meant to call `net.LookupPort()` on it (ie, map it from /etc/services) but in every other case where we have an IntOrString-valued Port field in an API struct, it refers to a named ContainerPort. But that would be extremely awkward to implement in this case; a policy specifying a named port could end up mapping to a different numeric port on every container in the namespace... Do other people actually implement string-valued ports that way? Or, for that matter, implement string-valued ports at all? (Related: I hadn't noticed until now that you can leave the Port value unspecified, allowing you to say "allow to all UDP ports, but no TCP ports" or "allow to all TCP ports, but no UDP ports". That seems like something that ended up in the spec just because it was possible, not because it was actually useful...)
@kubernetes/sig-network-misc
Automatic merge from submit-queue (batch tested with PRs 37845, 39439, 39514, 39457, 38866)
Move watch/versioned to pkg/apis/meta/v1/watch.go
Watch is a part of the server API
Automatic merge from submit-queue (batch tested with PRs 39466, 39490, 39527)
bump gengo to latest
bumping gengo to limit surprises while working on https://github.com/kubernetes/kubernetes/pull/39475
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 38920, 38090)
Improve error message for name/label validation.
Instead of just providing regex in name/label validation error output, we need to add the naming rules of the name/label, which is more end-user readable.
Fixed#37654
Automatic merge from submit-queue (batch tested with PRs 35300, 36709, 37643, 37813, 37697)
Add generated informers
Add informer-gen and the informers it generates. We'll do follow-up PRs to convert everything currently using the hand-written informers to the generated ones.
TODO:
- [x] switch to `GroupVersionResource`
- [x] finish godoc
@deads2k @caesarxuchao @sttts @liggitt
Automatic merge from submit-queue
Remove ExportOptions from api/internal and use unversioned
Should only have one internal object in use
Part of #37530
Automatic merge from submit-queue
Curating Owners: pkg/apis
cc @lavalamp @smarterclayton @erictune @thockin @bgrant0607
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the
future in the **reviewers** section. You probably do NOT need to modify
the **approvers** section. Names asre sorted by relevance, using some
secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue
Remove last probe time from replica sets
While experimenting with Deployment conditions I found out that if we are going to use lastProbeTime as we are supposed to be using it then we hotloop between updates (see https://github.com/kubernetes/kubernetes/pull/19343#issuecomment-255096778 for more info)
cc: @smarterclayton @soltysh
Automatic merge from submit-queue
Add PSP support for seccomp profiles
Seccomp support for PSP. There are still a couple of TODOs that need to be fixed but this is passing tests.
One thing of note, since seccomp is all being stored in annotations right now it breaks some of the assumptions we've stated for the provider in terms of mutating the passed in pod. I've put big warning comments around the pieces that do that to make sure it's clear and covered the rollback in admission if the policy fails to validate.
@sttts @pmorie @erictune @smarterclayton @liggitt
Automatic merge from submit-queue
Add test cases to test the default value of HorizontalPodAutoscaler
Add test cases to test the default value of HorizontalPodAutoscaler.
Previously we refused to emit 'autoConvert_*' functions if any field was not
convertible. The way around this was to write manual Conversion functions, but
to do so safely you must handle every fields. Huge opportunity for errors.
This PR cleans up the filtering such that it only operates on types that should
be converted (remove a lot of code) and tracks when fields are skipped. In
that case, it emits an 'autoConvert' function but not a public 'Convert'
function. If there is no manual function, the compile will fail.
This also means that manual conversion functions can call autoConvert functions
and then "patch up" what they need.
Automatic merge from submit-queue
Improvements on OpenAPI spec generation
- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation
Reference: #13414
**Release note**:
```release-note
Generate separate OpenAPI spec for each API GroupVersion on /<Group>/<Version>/swagger.json
```
Automatic merge from submit-queue
Set Job's labels from PodTemplateSpec if none are specified
Fixes#31009 and replaces #32033.
@AdoHe I've picked your changes, added the additional ones for 'batch/v2alpha1' and most importantly added tests for job's defaulting. ptal
Automatic merge from submit-queue
Add Cluster field in ObjectMeta
There will be no sub-rs, but add `Cluster` field to the ObjectMeta (for all the objects)
"To distinguish the object at the federation level from it's constituents at the cluster level we will add a "Cluster" field to the metadata of all objects (where the federation itself will also have a cluster identifier). That way it is possible to list, interact with, and distinguish between the objects either at the federation level or at the individual cluster level based on the cluster identifier. "
@quinton-hoole @nikhiljindal @deepak-vij @mfanjie @huangyuqi
[]()
Automatic merge from submit-queue
AppArmor PodSecurityPolicy support
Implements the AppArmor PodSecurityPolicy support based on the alpha API proposed [here](https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/apparmor.md#pod-security-policy)
This implementation deviates from the original proposal in one way: it adds a separate option for specifying a default profile:
```
apparmor.security.alpha.kubernetes.io/defaultProfileName
```
This has several advantages over the original proposal:
- The default is explicit, rather than implicit on the ordering
- The default can be specified without constraining the allowed profiles
- The allowed profiles can be restricted without specifying a default (requires every pod to explicitly set a profile)
The E2E cluster does not currently enable the PodSecurityPolicy, so I will submit E2E tests in a separate PR.
/cc @dchen1107 @pweil- @sttts @jfrazelle @Amey-D
There's been enough people broken by not committing generated code, that we
should undo that until we have a proper client that is `go get` compatible.
This is temporary.
This mostly takes the previously checked in files and removes them, and moves
the generation to be on-demand instead of manual. Manually verified no change
in generated output.
This drives conversion generation from file tags like:
// +conversion-gen=k8s.io/my/internal/version
.. rather than hardcoded lists of packages.
The only net change in generated code can be explained as correct. Previously
it didn't know that conversion was available.
This is the last piece of Clayton's #26179 to be implemented with file tags.
All diffs are accounted for. Followup will use this to streamline some
packages.
Also add some V(5) debugging - it was helpful in diagnosing various issues, it
may be helpful again.
This drives most of the logic of deep-copy generation from tags like:
// +deepcopy-gen=package
..rather than hardcoded lists of packages. This will make it possible to
subsequently generate code ONLY for packages that need it *right now*, rather
than all of them always.
Also remove pkgs that really do not need deep-copies (no symbols used
anywhere).
This is in prep to simplify tag logic. Don't rely on processing commas as new
tag delimiters. Put new tags on new lines. This had zero effect on generated
code (as intended).
In bringing back Clayton's PR piece-by-piece this was almost as easy to
implement as his version, and is much more like what I think we should be
doing.
Specifically, any time which defines a .DeepCopy() method will have that method
called preferentially. Otherwise we generate our own functions for
deep-copying. This affected exactly one type - resource.Quantity. In applying
this heuristic, several places in the generated code were simplified.
To achieve this I had to convert types.Type.Methods from a slice to a map,
which seems correct anyway (to do by-name lookups).
His PR cam during the middle of this development cycle, and it was easier to
burn it down and recreate it than try to patch it into an existing series and
re-test every assumption. This behavior will be re-introduced in subsequent
commits.
Automatic merge from submit-queue
ObjectMeta, ListMeta, and TypeMeta should implement their interfaces
Make unversioned.ListMeta implement List. Update all the *List types so they implement GetListMeta.
This helps avoid using reflection to get list information.
Remove all unnecessary boilerplate, move the interfaces to the right
places, and add a test that verifies that objects implement one, the
other, but never both.
@ncdc @lavalamp this supercedes #26964 with the boilerplate removed. Added tests
Make unversioned.ListMeta implement List. Update all the *List types so they implement GetListMeta.
This helps avoid using reflection to get list information.
Remove all unnecessary boilerplate, move the interfaces to the right
places, and add a test that verifies that objects implement one, the
other, but never both.
Automatic merge from submit-queue
Round should avoid clearing s, save a string
Instead of saving bytes, save a string, which makes String() faster
and does not unduly penalize marshal. During parse, save the string
if it is in canonical form.
@wojtek-t @lavalamp this makes quantity.String() faster for a few cases
where it matters. We were also not clearing s properly before on Round()
Automatic merge from submit-queue
Make name validators return string slices
Part of the larger validation PR, broken out for easier review and merge. Builds on previous PRs in the series.
Automatic merge from submit-queue
validate third party resources
addresses validation portion of https://github.com/kubernetes/kubernetes/issues/22768
* ThirdPartyResource: validates name (3 segment DNS subdomain) and version names (single segment DNS label)
* ThirdPartyResourceData: validates objectmeta (name is validated as a DNS label)
* removes ability to use GenerateName with thirdpartyresources (kind and api group should not be randomized, in my opinion)
test improvements:
* updates resttest to clean up after create tests (so the same valid object can be used)
* updates resttest to take a name generator (in case "foo1" isn't a valid name for the object under test)
action required for alpha thirdpartyresource users:
* existing thirdpartyresource objects that do not match these validation rules will need to be removed/updated (after removing thirdpartyresourcedata objects stored under the disallowed versions, kind, or group names)
* existing thirdpartyresourcedata objects that do not match the name validation rule will not be able to be updated, but can be removed
Automatic merge from submit-queue
PSP admission
```release-note
Update PodSecurityPolicy types and add admission controller that could enforce them
```
Still working on removing the non-relevant parts of the tests but I wanted to get this open to start soliciting feedback.
- [x] bring PSP up to date with any new features we've added to SCC for discussion
- [x] create admission controller that is a pared down version of SCC (no ns based strategies, no user/groups/service account permissioning)
- [x] fix tests
@liggitt @pmorie - this is the simple implementation requested that assumes all PSPs should be checked for each requests. It is a slimmed down version of our SCC admission controller
@erictune @smarterclayton
Automatic merge from submit-queue
Move internal types of hpa from pkg/apis/extensions to pkg/apis/autoscaling
ref #21577
@lavalamp could you please review or delegate to someone from CSI team?
@janetkuo could you please take a look into the kubelet changes?
cc @fgrzadkowski @jszczepkowski @mwielgus @kubernetes/autoscaling
Automatic merge from submit-queue
Added JobTemplate, a preliminary step for ScheduledJob and Workflow
@sdminonne as promised, sorry it took this long 😊
@erictune fyi though it does not have to be in for 1.2
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/21675)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Make ThirdPartyResource a root scoped object
ThirdPartyResource (the registration of a third party type) belongs at the cluster scope. It results in resource handlers installed in every namespace, and the same name in two namespaces collides (namespace is ignored when determining group/kind).
ThirdPartyResourceData (an actual instance of that type) is still namespace-scoped.
This PR moves ThirdPartyResource to be a root scope object. Someone previously using ThirdPartyResource definitions in alpha should be able to move them from namespace to root scope like this:
setup (run on 1.2):
```
kubectl create ns ns1
echo '{"kind":"ThirdPartyResource","apiVersion":"extensions/v1beta1","metadata":{"name":"foo.example.com"},"versions":[{"name":"v8"}]}' | kubectl create -f - --namespace=ns1
echo '{"kind":"Foo","apiVersion":"example.com/v8","metadata":{"name":"MyFoo"},"testkey":"testvalue"}' | kubectl create -f - --namespace=ns1
```
export:
```
kubectl get thirdpartyresource --all-namespaces -o yaml > tprs.yaml
```
remove namespaced kind registrations (this shouldn't remove the data of that type, which is another possible issue):
```
kubectl delete -f tprs.yaml
```
... upgrade ...
re-register the custom types at the root scope:
```
kubectl create -f tprs.yaml
```
Additionally, pre-1.3 clients that expect to read/write ThirdPartyResource at a namespace scope will not be compatible with 1.3+ servers, and 1.3+ clients that expect to read/write ThirdPartyResource at a root scope will not be compatible with pre-1.3 servers.
Automatic merge from submit-queue
API changes for Cascading deletion
This PR includes the necessary API changes to implement cascading deletion with finalizers as proposed is in #23656. Comments are welcome.
@lavalamp @derekwaynecarr @bgrant0607 @rata @hongchaodeng
Having internal and external integer types being different hides
potential conversion problems. Propagate that out further (which will
also allow us to better optimize conversion).
Automatic merge from submit-queue
Make all defaulters public
Will allow for generating direct accessors in conversion code instead of using reflection.
@wojtek-t
Here are a list of changes along with an explanation of how they work:
1. Add a new string field called TargetSelector to the external version of
extensions Scale type (extensions/v1beta1.Scale). This is a serialized
version of either the map-based selector (in case of ReplicationControllers)
or the unversioned.LabelSelector struct (in case of Deployments and
ReplicaSets).
2. Change the selector field in the internal Scale type (extensions.Scale) to
unversioned.LabelSelector.
3. Add conversion functions to convert from two external selector fields to a
single internal selector field. The rules for conversion are as follows:
i. If the target resource that this scale targets supports LabelSelector
(Deployments and ReplicaSets), then serialize the LabelSelector and
store the string in the TargetSelector field in the external version
and leave the map-based Selector field as nil.
ii. If the target resource only supports a map-based selector
(ReplicationControllers), then still serialize that selector and
store the serialized string in the TargetSelector field. Also,
set the the Selector map field in the external Scale type.
iii. When converting from external to internal version, parse the
TargetSelector string into LabelSelector struct if the string isn't
empty. If it is empty, then check if the Selector map is set and just
assign that map to the MatchLabels component of the LabelSelector.
iv. When converting from internal to external version, serialize the
LabelSelector and store it in the TargetSelector field. If only
the MatchLabel component is set, then also copy that value to
the Selector map field in the external version.
4. HPA now just converts the LabelSelector field to a Selector interface
type to list the pods.
5. Scale Get and Update etcd methods for Deployments and ReplicaSets now
return extensions.Scale instead of autoscaling.Scale.
6. Consequently, SubresourceGroupVersion override and is "autoscaling"
enabled check is now removed from pkg/master/master.go
7. Other small changes to labels package, fuzzer and LabelSelector
helpers to piece this all together.
8. Add unit tests to HPA targeting Deployments and ReplicaSets.
9. Add an e2e test to HPA targeting ReplicaSets.
In podSecurityPolicy:
1. Rename .seLinuxContext to .seLinux
2. Rename .seLinux.type to .seLinux.rule
3. Rename .runAsUser.type to .runAsUser.rule
4. Rename .seLinux.SELinuxOptions
1,2,3 as suggested by thockin in #22159.
I added 3 for consistency with 2.
Added selector generation to Job's
strategy.Validate, right before validation.
Can't do in defaulting since UID is not known.
Added a validation to Job to ensure that the generated
labels and selector are correct when generation was requested.
This happens right after generation, but validation is in a better
place to return an error.
Adds "manualSelector" field to batch/v1 Job to control selector generation.
Adds same field to extensions/__internal. Conversion between those two
is automatic.
Adds "autoSelector" field to extensions/v1beta1 Job. Used for storing batch/v1 Jobs
- Default for v1 is to do generation.
- Default for v1beta1 is to not do it.
- In both cases, unset == false == do the default thing.
Release notes:
Added batch/v1 group, which contains just Job, and which is the next
version of extensions/v1beta1 Job.
The changes from the previous version are:
- Users no longer need to ensure labels on their pod template are unique to the enclosing
job (but may add labels as needed for categorization).
- In v1beta1, job.spec.selector was defaulted from pod labels, with the user responsible for uniqueness.
In v1, a unique label is generated and added to the pod template, and used as the selector (other
labels added by user stay on pod template, but need not be used by selector).
- a new field called "manualSelector" field exists to control whether the new behavior is used,
versus a more error-prone but more flexible "manual" (not generated) seletor. Most users
will not need to use this field and should leave it unset.
Users who are creating extensions.Job go objects and then posting them using the go client
will see a change in the default behavior. They need to either stop providing a selector (relying on
selector generation) or else specify "spec.manualSelector" until they are ready to do the former.
Dr. Stefan Schimanski
