Automatic merge from submit-queue
Fix image verification when hostname is present in image
Deal better with the situation where a image name contains
a hostname as well.
Fixes#30580
Automatic merge from submit-queue
Add volume reconstruct/cleanup logic in kubelet volume manager
Currently kubelet volume management works on the concept of desired
and actual world of states. The volume manager periodically compares the
two worlds and perform volume mount/unmount and/or attach/detach
operations. When kubelet restarts, the cache of those two worlds are
gone. Although desired world can be recovered through apiserver, actual
world can not be recovered which may cause some volumes cannot be cleaned
up if their information is deleted by apiserver. This change adds the
reconstruction of the actual world by reading the pod directories from
disk. The reconstructed volume information is added to both desired
world and actual world if it cannot be found in either world. The rest
logic would be as same as before, desired world populator may clean up
the volume entry if it is no longer in apiserver, and then volume
manager should invoke unmount to clean it up.
Fixes https://github.com/kubernetes/kubernetes/issues/27653
Automatic merge from submit-queue
CRI: remove pod sandbox resources
The pod-level resources need further discussion. Remove it from CRI for now.
See the original discussion in #29871
Currently kubelet volume management works on the concept of desired
and actual world of states. The volume manager periodically compares the
two worlds and perform volume mount/unmount and/or attach/detach
operations. When kubelet restarts, the cache of those two worlds are
gone. Although desired world can be recovered through apiserver, actual
world can not be recovered which may cause some volumes cannot be cleaned
up if their information is deleted by apiserver. This change adds the
reconstruction of the actual world by reading the pod directories from
disk. The reconstructed volume information is added to both desired
world and actual world if it cannot be found in either world. The rest
logic would be as same as before, desired world populator may clean up
the volume entry if it is no longer in apiserver, and then volume
manager should invoke unmount to clean it up.
Automatic merge from submit-queue
pkg/apiserver/authenticator: reorder oidc plugin to auth after service accounts
Both plugins verify JWTs, but the OpenID Connect plugin performs
much worse when faced with cache misses. Reorder the plugins so
the service account plugin tries to authenticate a bearer token
first.
I had a fun time with this by writing an OpenID Connect provider that stores its data in third party resources. When it's running in the cluster it uses a service account and caused some interesting behavior when the keys expired.
Our OpenID Connect plugin needs a more sophisticated caching model to avoid continuously re-requesting keys when seeing a lot of tokens it doesn't recognize. However, I feel this reordering is generally useful since service accounts will be more common than OpenID Connect tokens.
cc @kubernetes/sig-auth
Automatic merge from submit-queue
Remove pods along with jobs when Replace ConcurrentPolicy is set
Fixes#30442
This builds on #30327 and needs a bit more love in tests.
@janetkuo @erictune fyi
Automatic merge from submit-queue
Add CloudStack cloud provider (extended and refactored)
This PR is superseding PR #26165 is which some groundwork for this PR has been done. So this PR now fixes#26165 and fixes#26045.
I've been in contact with @ngtuna about this updated version of his earlier work (which is still in this PR as one squashed commit) and he has given his 👍 for this 😉
This PR adds additional logic for allocating and associating a public IP, if the `—load-balancer-ip` option is not used. It will do proper management of public IP’s that are allocated by this provider ( so IP’s that are no longer needed/used will also be released again).
Additionally the provider can now also work with CloudStack projects and advanced (VPC) networks. And lastly the Zone interface now returns an actual zone (supplied by the cloud config), a few logical errors are fixed and the first few tests are added.
All the functionality is extensively tested against both basic and advanced (VPC) networks and of course all new and existing (integration) tests are all passing.
Automatic merge from submit-queue
Move new etcd storage (low level storage) into cacher
In an effort for #29888, we are pushing forward this:
What?
- It changes creating etcd storage.Interface impl into creating config
- In creating cacher storage (StorageWithCacher), it passes config created above and new etcd storage inside.
Why?
- We want to expose the information of (etcd) kv client to cacher. Cacher storage uses this information to talk to remote storage.
Automatic merge from submit-queue
move syncNetworkConfig to Init for cni network plugin
start syncNetworkConfig routine in `Init` instead of probing. This fixes a bug where the syncNetworkConfig runs periodically even `cni` network plugin is not in use.
Automatic merge from submit-queue
add RequiresExactMatch test for empty andterm
What?
Add a test path for empty andterm.
Why?
fields.Everything() returns empty andterm.
fields.SelectorFromSet() returns empty andterm.
Automatic merge from submit-queue
Remove resource specifications from CRI until further notice
See #29871 for the discussion issue.
cc @dchen1107 @vishh @yujuhong @euank @yifan-gu @feiskyer
Automatic merge from submit-queue
Endpoint controller logs errors during normal cluster behavior
The endpoint controller logs an error when its forbidden from creating new endpoints during namespace termination. This is normal cluster behavior, and therefore should not be logged. This confuses operators administrating the cluster.
Updated to log at a lower level in response to a forbidden message when performing a create operation. In case of an error on the API server side of the house, I continue to requeue the key. It should be ignored in a future syncService call once the service is deleted as part of namespace termination.
See https://bugzilla.redhat.com/show_bug.cgi?id=1347425
/cc @kubernetes/rh-cluster-infra
This commit adds logic for allocating and associating a public IP, if the `—load-balancer-ip` option is not used. It will do proper management of IP’s that are allocated by this provider, so IP’s that are no longer needed/used will also be released again.
Additionally the provider can now also work with CloudStack projects and advanced (VPC) networks.
Lastly the Zone interface now returns an actual zone (supplied by the cloud config), a few logical errors are fixed and the first few tests are added.
All the functionality is extensively tested against both basic and advanced (VPC) networks.
Automatic merge from submit-queue
speed up RC scaler
The RC scaler was waiting before starting the scale and then didn't use a watch to observe the result. That led to longer than expected wait times.
@fabianofranz ptal. You may want to sweep the rest of the file. It could use some tidying with `RetryOnConflict` and `watch.Until`.
Automatic merge from submit-queue
Validate SHA/Tag when checking docker images
Docker API does not validate the tag/sha, for example, all the following
calls work say for a alpine image with short SHA "4e38e38c8ce0"
echo -e "GET /images/alpine:4e38e38c8ce0/json HTTP/1.0\r\n" | nc -U /var/run/docker.sock
echo -e "GET /images/alpine:4e38e38c/json HTTP/1.0\r\n" | nc -U /var/run/docker.sock
echo -e "GET /images/alpine:4/json HTTP/1.0\r\n" | nc -U /var/run/docker.sock
So we should check the response from the Docker API and look for the tags or SHA explicitly.
Fixes#30355
Automatic merge from submit-queue
Make more messages respect --quiet flag
Make following two messages respect `--quiet` in `kubectl run`
- `If you don't see a command prompt, try pressing enter.`
- `Pod "name" deleted`
Ref #28695
Automatic merge from submit-queue
Set pod state as "unknown" when CNI plugin fails
Before this change, CNI plugin failure didn't change anything in the pod status, so pods having containers without requested network were "running".
Fixes#29148
Automatic merge from submit-queue
Remove kubelet pkill dependency
Issue #26093 identified pkill as one of the dependencies of kublet
which could be worked around. Build on the code introduced for pidof
and regexp for the process(es) we need to send a signal to.
Related to #26093
Automatic merge from submit-queue
Adding events to federation control plane
Adding events to federation control plane.
Apart from the standard changes to add a resource to `federation/apis/core/v1`, other changes are:
* Adding a new `federationoptions.ServerRunOptions` which includes `genericoptions.ServerRunOptions` and EventsTTL.
* Added a new method in `pkg/api/mapper` to build a RestMapper based on the passed Scheme rather than using `api.Scheme`. Updated `federation/apis/core/install` to use this new method. Without this change, if `federation/apis/core/install.init()` is called before `pkg/api/install.init()` then the registered RESTMapper in `pkg/apimachinery/registered` will have no resources. This second problem will be fixed once we have instances of `pkg/apimachinery/registered` instead of a single global singleton (generated clientset which imports `pkg/api/install` will have a different instance of registered, than federation-apiserver which imports `federation/apis/core/install`).
cc @kubernetes/sig-cluster-federation @lavalamp
Automatic merge from submit-queue
[kubelet] Introduce --protect-kernel-defaults flag to make the tunable behaviour configurable
Let's make the default behaviour of kernel tuning configurable. The default behaviour is kept modify as has been so far.
Automatic merge from submit-queue
Fix TestPidOf {procfs} - Take #2
We should not bailout when we get an error. We should continue
processing other files/directories. We were returning the
err passed in which was causing the processing to stop.
Fixes#30377
Automatic merge from submit-queue
Kubelet: generate sandbox/container config for new runtime API
Generate sandbox/container config for new runtime API. Part of #28789 .
CC @yujuhong @Random-Liu @dchen1107
Automatic merge from submit-queue
Add zsh compatibility note `completion` cmd help
zsh completions are not supported on zsh versions < 5.2.
This patch advices user on supported versions of zsh when using the `completion`
command to avoid potential UX failure.
##### After
`$ kubectl completion -h`
```
Output shell completion code for the given shell (bash or zsh).
This command prints shell code which must be evaluation to provide interactive
completion of kubectl commands.
Examples:
$ source <(kubectl completion bash)
will load the kubectl completion code for bash. Note that this depends on the
bash-completion framework. It must be sourced before sourcing the kubectl
completion, e.g. on the Mac:
$ brew install bash-completion
$ source $(brew --prefix)/etc/bash_completion
$ source <(kubectl completion bash)
If you use zsh*, the following will load kubectl zsh completion:
$ source <(kubectl completion zsh)
* zsh completions are only supported in versions of zsh >= 5.2
```
```release-note
release-note-none
```
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30460)
<!-- Reviewable:end -->
Automatic merge from submit-queue
openstack: Autodetect LBaaS v1 vs v2
```release-note
* openstack: autodetect LBaaS v1/v2 by querying for available extensions. For most installs, this effectively changes the default from v1 to v2. Existing installs can add "lb-version = v1" to the provider config file to continue to use v1.
```
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29726)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Implement 'kubectl top' command
```release-note
Added 'kubectl top' command showing the resource usage metrics.
```
Sample output:
Nodes:
```
$ kubectl top node
NAME CPU MEMORY STORAGE TIMESTAMP
kubernetes-minion-group-xxxx 76m 1468 Mi 0 Mi Tue, 12 Jul 2016 17:37:00 +0200
kubernetes-minion-group-yyyy 73m 1511 Mi 0 Mi Tue, 12 Jul 2016 17:37:00 +0200
kubernetes-minion-group-zzzz 46m 1506 Mi 0 Mi Tue, 12 Jul 2016 17:37:00 +0200
kubernetes-master 76m 2059 Mi 0 Mi Tue, 12 Jul 2016 17:37:00 +0200
```
Pods in all namespaces:
```
$ kubectl top pod --all-namespaces
NAMESPACE NAME CPU MEMORY STORAGE TIMESTAMP
default nginx-1111111111-zzzzz 0m 1 Mi 0 Mi Tue, 12 Jul 2016 17:49:00 +0200
kube-system etcd-server-kubernetes-master 4m 116 Mi 0 Mi Tue, 12 Jul 2016 17:49:00 +0200
kube-system fluentd-cloud-logging-kubernetes-minion-group-xxxx 14m 110 Mi 0 Mi Tue, 12 Jul 2016 17:49:00 +0200
kube-system kube-dns-v18-zzzzz 1m 6 Mi 0 Mi Tue, 12 Jul 2016 17:49:00 +0200
...
```
Pod with containers:
```
$ kubectl top pod heapster-v1.1.0-1111111111-miail --namespace=kube-system --containers
NAMESPACE NAME CPU MEMORY STORAGE TIMESTAMP
kube-system heapster-v1.1.0-1111111111-miail 1m 42 Mi 0 Mi Tue, 12 Jul 2016 17:52:00 +0200
heapster 1m 26 Mi 0 Mi
eventer 0m 3 Mi 0 Mi
heapster-nanny 0m 6 Mi 0 Mi
eventer-nanny 0m 6 Mi 0 Mi
```
ref #11382
[]()
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/28844)
<!-- Reviewable:end -->
Automatic merge from submit-queue
return err on `kubectl run --image` with invalid value
When running `kubectl run <configname> --image="Invalid$$%ImageValue%%__"`, a configuration is successfully created with an image name that is not a valid value for an image reference.
This patch validates that the image name is a valid image reference, and returns an error before creating a config if an invalid value is passed.
`$ kubectl run test --image="Invalid__%imagename"`
```
error: Invalid image name "Invalid__%imagename": invalid reference format
```
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30162)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Basic audit log
Fixes#2203 by introducing simple audit logging, including the information about impersonation. We currently have something identical in openshift, but I'm open to any suggestions. Sample logs look like that:
as `<self>`:
```
AUDIT: id="75114bb5-970a-47d5-a5f1-1e99cea0574c" ip="127.0.0.1" method="GET" user="test-admin" as="<self>" namespace="openshift" uri="/api/v1/namespaces/openshift/pods/python"
AUDIT: id="75114bb5-970a-47d5-a5f1-1e99cea0574c" response=200
```
as user:
```
AUDIT: id="b0a443ae-f7d8-408c-a355-eb9501fd5c59" ip="192.168.121.118" method="GET" user="system:admin" as="test-admin" namespace="openshift" uri="/api/v1/namespaces/openshift/pods/python"
AUDIT: id="b0a443ae-f7d8-408c-a355-eb9501fd5c59" response=200
```
```release-note
* Add basic audit logging
```
@ericchiang @smarterclayton @roberthbailey @erictune @ghodss
[]()
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/27087)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Let kubectl delete rc and rs with DeleteOptions.OrphanDependents=false
so that when the garbage collector is enabled, RC and RS are deleted immediately without waiting for the garbage collector to orphan the pods.
There is no user visible changes, so we don't need a release note.
cc @fabioy
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30461)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Name jobs created by sj deterministically
```release-note
Name the job created by scheduledjob (sj) deterministically with sj's name and a hash of job's scheduled time.
```
@erictune @soltysh
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30420)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Fix code generators-- make scheme building composable
I needed to make some changes to make my other refactoring possible and this got rather large.
We now provide a "SchemeBuilder" to help all of the api packages provide their scheme-building functions (addKnownTypes and friends) in a standardized way. This also allows generated deepcopies & conversions to be entirely self contained, the project will now build without them being present (as they can add themselves to the SchemeBuilder). (Although if you actually build without them, you will get reduced performance!)
Previously, there was no way to construct your own runtime.Scheme (e.g., to test), you had to use the api.Scheme object, which has all sorts of non-hermetic cruft in it. Now you can get everything from a package by calling the scheme builder's AddToScheme, including the generated functions, if they are present.
Next steps are to allow for declaring dependencies, and to standardize the registration & install code. (#25434)
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/25526)
<!-- Reviewable:end -->
Automatic merge from submit-queue
the observed usage should match those that have hard constraints
in the sync process, the quota will be replenished, the new observed usage will be sumed from each evaluator, if the previousUsed set is not be cleared, the new usage will be dirty, maybe some unusage resource still in , as the code below
newUsage = quota.Mask(newUsage, matchedResources)
for key, value := range newUsage {
usage.Status.Used[key] = value
}
so i think here shoul not set value previousUsed
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29653)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Quota admission errors if usage is negative
If quota observes negative usage for an artifact, that artifact could game the quota system.
This adds a global check in the quota system to catch this scenario for all evaluators.
/cc @deads2k
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30396)
<!-- Reviewable:end -->
This removes the need to manually specify the version in all but unusual
cases.
For most installs this will effectively flip the default from
v1 (deprecated) to v2 so conservative existing installs may want to
manually configure "lb-version = v1" before upgrading.
Automatic merge from submit-queue
Start verifying golint on a per-package basis as packages are fixed
<!--
Checklist for submitting a Pull Request
Please remove this comment block before submitting.
1. Please read our [contributor guidelines](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md).
2. See our [developer guide](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md).
3. If you want this PR to automatically close an issue when it is merged,
add `fixes #<issue number>` or `fixes #<issue number>, fixes #<issue number>`
to close multiple issues (see: https://github.com/blog/1506-closing-issues-via-pull-requests).
4. Follow the instructions for [labeling and writing a release note for this PR](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes) in the block below.
-->
```release-note
Added `golint` for pkg/security/podsecuritypolicy/capabilities` along with validation.
```
[]()
This is a POC to start enabling `golint` checks on a per-package basis, we did this on the docker project and it was a great way for new contributors to help and it benefits the project overall. All they have to do is add the package they fixed to the bash array in `hack/verify-golint.sh` and fix all the lint errors.
Eventually when all the packages have been fixed we can change the function to `find_files`. Or something based off which files are changed in a patch set to verify `golint`.
Now I used this specific package as the POC because I wanted to show the downside of this changing the api of the package.
Most of the times this arose in docker/docker we decided that if someone wasn't importing their deps locally then it was their loss, but I'm not sure if you all will agree.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/27911)
<!-- Reviewable:end -->
We should not bailout when we get an error. We should continue
processing other files/directories. We were returning the
err passed in which was causing the processing to stop.
Fixes#30377
Automatic merge from submit-queue
check validation with no apps client in kubectl util factory
autoscaling client already exist:
if c.c.AutoscalingClient == nil {
return errors.New("unable to validate: no autoscaling client")
so following autoscaling client should be apps client:
if c.c.AppsClient == nil {
return errors.New("unable to validate: no autoscaling client")
}
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30362)
<!-- Reviewable:end -->
Both plugins verify JWTs, but the OpenID Connect plugin performs
much worse when faced with cache misses. Reorder the plugins so
the service account plugin tries to authenticate a bearer token
first.
Automatic merge from submit-queue
[Kubelet] Rename `--config` to `--pod-manifest-path`. `--config` is deprecated.
This field holds the location of a manifest file or directory of manifest
files for pods the Kubelet is supposed to run. The name of the field
should reflect that purpose. I didn't change the flag name because that
API should remain stable.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29999)
<!-- Reviewable:end -->
Docker API does not validate the tag/sha, for example, all the following
calls work say for a alpine image with short SHA "4e38e38c8ce0"
echo -e "GET /images/alpine:4e38e38c8ce0/json HTTP/1.0\r\n" | nc -U /var/run/docker.sock
echo -e "GET /images/alpine:4e38e38c/json HTTP/1.0\r\n" | nc -U /var/run/docker.sock
echo -e "GET /images/alpine:4/json HTTP/1.0\r\n" | nc -U /var/run/docker.sock
So we should check the response from the Docker API and look for the
tags or SHA explicitly.
Fixes#30355
Automatic merge from submit-queue
[GarbageCollector] measure latency
First commit is #27600.
In e2e tests, I measure the average time an item spend in the eventQueue(~1.5 ms), dirtyQueue(~13ms), and orphanQueue(~37ms). There is no stress test in e2e yet, so the number may not be useful.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/28387)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Simplify canonical element term in deepcopy
Replace the old functional canonical element term in deepcopy registration with direct struct instantiation.
The old way was an artifact of non-uniform pointer/non-pointer types in the signature of deepcopy function. Since we changed that to always be a pointer, we can simplify the code.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30341)
<!-- Reviewable:end -->
Issue #26093 identified pkill as one of the dependencies of kublet
which could be worked around. Build on the code introduced for pidof
and regexp for the process(es) we need to send a signal to.
Related to #26093
Automatic merge from submit-queue
add metrics for workqueues
Adds prometheus metrics to work queues and enables them for the resourcequota controller. It would be easy to add this to all other workqueue based controllers and gather basic responsiveness metrics.
@kubernetes/rh-cluster-infra helps debug quota controller responsiveness problems.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30296)
<!-- Reviewable:end -->
Also provide a new --pod-manifest-path flag and deprecate the old
--config one.
This field holds the location of a manifest file or directory of manifest
files for pods the Kubelet is supposed to run. The name of the field
should reflect that purpose.
Automatic merge from submit-queue
kube-proxy: Propagate hostname to iptables proxier
Need to propagate the hostname (i.e. Nodename) from kube-proxy to the iptables proxier to allow kube-proxy to determine local endpoints.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30293)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Remove kubelet dependency on pidof
Issue #26093 identified pidof as one of the dependencies of kublet
which could be worked around. In this PR, we just look at /proc
to construct the list of pids we need for a specified process
instead of running "pidof" executable
Related to #26093
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30002)
<!-- Reviewable:end -->
Automatic merge from submit-queue
pkg/controller/garbagecollector: simplify mutexes.
pkg/controller/garbagecollector: simplified synchronization and made idiomatic.
Similar to #29598, we can rely on the zero-value construction behavior
to embed `sync.Mutex` into parent structs.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29898)
<!-- Reviewable:end -->
Automatic merge from submit-queue
apiserver: fix timeout handler
Protect access of the original writer. Panics if anything has wrote
into the original writer or the writer is hijacked when times out.
Fix#29001
/cc @smarterclayton @lavalamp
The next step would be respect the request context once 1.7 is out.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29594)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Cut the client repo, staging it in the main repo
Tracking issue: #28559
ref: https://github.com/kubernetes/kubernetes/pull/25978#issuecomment-232710174
This PR implements the plan a few of us came up with last week for cutting client into its own repo:
1. creating "_staging" (name is tentative) directory in the main repo, using a script to copy the client and its dependencies to this directory
2. periodically publishing the contents of this staging client to k8s.io/client-go repo
3. converting k8s components in the main repo to use the staged client. They should import the staged client as if the client were vendored. (i.e., the import line should be `import "k8s.io/client-go/<pacakge name>`). This requirement is to ease step 4.
4. In the future, removing the staging area, and vendoring the real client-go repo.
The advantage of having the staging area is that we can continuously run integration/e2e tests with the latest client repo and the latest main repo, without waiting for the client repo to be vendored back into the main repo. This staging area will exist until our test matrix is vendoring both the client and the server.
In the above plan, the tricky part is step 3. This PR achieves it by creating a symlink under ./vendor, pointing to the staging area, so packages in the main repo can refer to the client repo as if it's vendored. To prevent the godep tool from messing up the staging area, we export the staged client to GOPATH in hack/godep-save.sh so godep will think the client packages are local and won't attempt to manage ./vendor/k8s.io/client-go.
This is a POC. We'll rearrange the directory layout of the client before merge.
@thockin @lavalamp @bgrant0607 @kubernetes/sig-api-machinery
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29147)
<!-- Reviewable:end -->
Issue #26093 identified pidof as one of the dependencies of kublet
which could be worked around. In this PR, we just look at /proc
to construct the list of pids we need for a specified process
instead of running "pidof" executable
Related to #26093
Automatic merge from submit-queue
HPA: ignore scale targets whose replica count is 0
Disable HPA when the user (or another component) explicitly sets the replicas to 0.
Fixes#28603
@kubernetes/autoscaling @fgrzadkowski @kubernetes/rh-cluster-infra @smarterclayton @ncdc
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29212)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Modify predicate() interface to return all failed predicates
As stated in the comments below, this is the first step of showing the user all predicates that failed for a given node when scheduling of a given pod failed on every node.
ref #20064
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29272)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Remove default etcd validation in generic apiserver
Moving verification of `--etcd-servers` to the concrete apiserver instead of checking during defaulting in generic apiserver.
The context for this change is that heapster (will be another apiserver) doesn't need to have etcd underneath.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29993)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Kubelet: Add container ports label.
Addresses https://github.com/kubernetes/kubernetes/pull/30049#discussion_r73983952.
Add container ports label, although we don't use it now, it will make it easier to switch to new runtime interface in the future.
@yujuhong @feiskyer
Automatic merge from submit-queue
[GarbageCollector] only store typeMeta and objectMeta in the gc store
GC only needs to know the apiVersion, kind, and objectMeta of an object. This PR makes the stores of GC only save these fields.
cc @kubernetes/sig-api-machinery
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/28480)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Update scale cmd help to display duration "units" in --timeout usage
release-note-none
`kubectl scale` has `--timeout` option, but the help info does not give accepted time units.
The help info gives default value 0 without a unit, potentially misleading a user into thinking they need to pass the numbered time only.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30022)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Add service and deployment create sub-commands to kubectl
```release-note
Added new commands kubectl create service & kubectl create deployment
```
Partial #25382
Automatic merge from submit-queue
Add total inodes to kubelet summary api
Needed to support inode based eviction thresholds as a percentage.
/cc @ronnielai @vishh @kubernetes/rh-cluster-infra
Before this change, CNI plugin failure didn't change anything in
the pod status, so pods having containers without requested
network were "running".
Fixes#29148
`ex.Command()` already searches the binary in PATH, no need to manually
specify it. `pkg/util/exec` tests fail in non-conventional environments
due to this (e.g. NixOS).
Automatic merge from submit-queue
Install authorization group in 1.4 client
By the way, buried in my other PR there is an e2e test which will remind people adding this line.
@deads2k
Also, fix unit tests to have the same claim and volume sizes in most of the
tests where we don't test matching based on size and test for a specific size
when we do actually test the matching.
Automatic merge from submit-queue
make correction to two variables in inverse order
from Fprintf info ""Renaming %s to %s\n", it should be renamed from old to new.
Automatic merge from submit-queue
[kubelet] Auto-discover node IP if neither cloud provider exists and IP is not explicitly specified
One example where the earlier implementation failed is when running kubelet on CoreOS (bare-metal), where the nameserver is set to `8.8.8.8`. kubelet tries to lookup the node name agains Google DNS, which obviously fails. The kubelet won't recover after that.
The workaround hsa been to set `--hostname-override` to an IP address, but it's quite annoying to try to make a multi-distro way of getting the IP in bash for example. This way is much cleaner.
Refactored the function a little bit at the same time
@vishh @yujuhong @resouer @Random-Liu
Automatic merge from submit-queue
Verify volume.GetPath() never returns ""
Add a new helper method volume.GetPath(Mounter) instead of calling
the GetPath() of the Mounter directly. Check if GetPath() is returning
a "" and convert that into an error.
Fixes#23163
Automatic merge from submit-queue
kubectl config set-crentials: add arguments for auth providers
This PR adds `--auth-provider` and `--auth-provider-arg` flags to the
`kubectl config set-credentials` sub-command.
There's currently no way of interacting with the new auth provider framework added in #23066 through kubectl. You have to render a custom kubeconfig to use them. Additionally `kubectl config set` just sort of craps out when attempting to interact with authentication info objects (#29312).
This is a minimal implementation of allowing `kubect config set-credentials` to set fields for client auth providers.
cc @cjcullen @kubernetes/kubectl
Automatic merge from submit-queue
Run goimport for the whole repo
While removing GOMAXPROC and running goimports, I noticed quite a lot of other files also needed a goimport format. Didn't commit `*.generated.go`, `*.deepcopy.go` or files in `vendor`
This is more for testing if it builds.
The only strange thing here is the gopkg.in/gcfg.v1 => github.com/scalingdata/gcfg replace.
cc @jfrazelle @thockin
Automatic merge from submit-queue
optimize conditions of ServiceReplenishmentUpdateFunc to replenish service
Originally, the replenishQuota method didn't focus on the third parameter object even if others transfered to it, i think the function is not efficient and perfect. then i use the third param to get MatchResources, it will be more exact. for example, if the old pod was quota tracked and the new was not, the replenishQuota only focus on usage resource of the old pod, still if the third parameter object is nil, the process will be same as before
Automatic merge from submit-queue
add subjectaccessreviews resource
Adds a subjectaccessreviews endpoint that uses the API server's authorizer to determine if a subject is allowed to perform an action.
Part of kubernetes/features#37
Automatic merge from submit-queue
Fix Incorrect response model for pods/{name}/log
The swagger spec for pods/{name}/log does not include
"text/plain" as a possible content-type for the the response.
So we implement ProducesMIMETypes to make sure "text/plain"
gets added to the default list ot content-types.
the v1.json was generated by running:
hack/update-generated-swagger-docs.sh;./hack/update-swagger-spec.sh;
The swagger spec for pods/{name}/log does not include
"text/plain" as a possible content-type for the the response.
So we implement ProducesMIMETypes to make sure "text/plain"
gets added to the default list ot content-types.
the v1.json was generated by running:
hack/update-generated-swagger-docs.sh;./hack/update-swagger-spec.sh;
Fixes#14071
Add a new helper method volume.GetPath(Mounter) instead of calling
the GetPath() of the Mounter directly. Check if GetPath() is returning
a "" and convert that into an error. At this point, we only have
information about the type of the Mounter, so let's log that if
there is a problem
Fixes#23163
Automatic merge from submit-queue
replace improper create quota alias
`q` is not a shortname for `resourcequota`, so it shouldn't be an alias for `kubectl create resourcequota`.
@sttts ptal
Automatic merge from submit-queue
add asserts for RecognizingDecoder and update protobuf serializer
Ensure that serializers are asserting that they implement the `RecognizingDecoder` interface and update the protobuf serializer to add the `unknown` parameter.
Manifested in: https://github.com/openshift/origin/issues/9739
@smarterclayton @deads2k
Automatic merge from submit-queue
make reousrce prefix consistent with other registries
Storage class registry was added recently in #29694.
In other registries, resource prefix was changed to a more generic way using RESTOptions.ResourcePrefix.
This PR is an attempt to make both consistent.
Automatic merge from submit-queue
Node disk pressure should induce image gc
If the node reports disk pressure, prior to evicting pods, the node should clean up unused images.
Automatic merge from submit-queue
refactor quota calculation for re-use
Refactors quota calculation to allow reuse. This will allow us to do "punch through" calculation inside of admission if a particular quota needs usage stats and allows downstream re-use by moving calculation closer to the evaluators and separating "needs calculation" logic from "do calculation".
@derekwaynecarr
Automatic merge from submit-queue
Bump Libcontainer to latest head
@Random-Liu or @yujuhong Can any one of you please do a quick review.
I updated libcontainer in a previous PR but #29492 reverted those changes. This is needed for #27204.
Signed-off-by: Buddha Prakash <buddhap@google.com>
Automatic merge from submit-queue
Rewrite service controller to apply best controller pattern
This PR is a long term solution for #21625:
We apply the same pattern like replication controller to service controller to avoid the potential process order messes in service controller, the change includes:
1. introduce informer controller to watch service changes from kube-apiserver, so that every changes on same service will be kept in serviceStore as the only element.
2. put the service name to be processed to working queue
3. when process service, always get info from serviceStore to ensure the info is up-to-date
4. keep the retry mechanism, sleep for certain interval and add it back to queue.
5. remote the logic of reading last service info from kube-apiserver before processing the LB info as we trust the info from serviceStore.
The UT has been passed, manual test passed after I hardcode the cloud provider as FakeCloud, however I am not able to boot a k8s cluster with any available cloudprovider, so e2e test is not done.
Submit this PR first for review and for triggering a e2e test.
Automatic merge from submit-queue
fix annotate.go single resource check
```release-note
Fix issue with kubectl annotate when --resource-version is provided.
```
When using `kubectl annotate` with a `--resource-version` on a resource, such as `kubectl annotate pod <pod_name> --resource-version=1820 description='myannotation'`, the command fails with the error: `error: --resource-version may only be used with a single resource`.
Upon printing the output of `resources` that the annotate command receives from cli args, it prints: `Resources:[pod <pod_name>]`. In other words, it treats the name of the resource as a second resource. This PR addresses this issue by using the resource builder `Singular` flag to determine if only a single resource was passed.
Automatic merge from submit-queue
Handle container terminated but pod still running in conditions
Sometimes when you have a pod with more than one container, and the container runs and terminates really fast, `PodContainerRunning` can go into a state where the pod indicates it's still running, but the container is already terminated. Handle that condition by returning `ErrContainerTerminated` when it happens.
Automatic merge from submit-queue
rbac validation: rules can't combine non-resource URLs and regular resources
This PR updates the validation used for RBAC to prevent rules from mixing non-resource URLs and regular resources.
For example the following is no longer valid
```yml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
name: admins
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
nonResourceURLs: ["*"]
```
And must be rewritten as so.
```yml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
name: admins
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
- nonResourceURLs: ["*"]
verbs: ["*"]
```
It also:
* Mandates non-zero length arrays for required resources.
* Mandates non-resource URLs only be used for ClusterRoles (not namespaced Roles).
* Updates the swagger validation so `verbs` are the only required field in a rule. Further validation is done by the server.
Also, do we need to bump the API version?
Discussed by @erictune and @liggitt in #28304
Updates kubernetes/features#2
cc @kubernetes/sig-auth
Edit:
* Need to update the RBAC docs if this change goes in.
Automatic merge from submit-queue
Fix deployment e2e test: waitDeploymentStatus should error when entering an invalid state
Follow up #28162
1. We should check that max unavailable and max surge aren't violated at all times in e2e tests (didn't check this in deployment scaled rollout yet, but we should wait for it to become valid and then continue do the check until it finishes)
2. Fix some minor bugs in e2e tests
@kubernetes/deployment
Automatic merge from submit-queue
Verify if Secure and InsecurePort are equal for apiserver
If we specify the kube-apiserver flags `--secure-port` and `--insecure-port` with the same port, the server will print the below error info repeatedly. In fact, it's meaningless to do this. It should verify if the two flags are equal. If so, the server should give error info and exit directly.
```
root@vm:~# [restful] 2016/08/01 13:11:52 log.go:30: [restful/swagger] listing is available at https://172.16.1.11:8888/swaggerapi/
[restful] 2016/08/01 13:11:52 log.go:30: [restful/swagger] https://172.16.1.11:8888/swaggerui/ is mapped to folder /swagger-ui/
E0801 13:11:53.102232 2481 genericapiserver.go:733] Unable to listen for secure (listen tcp 0.0.0.0:8888: bind: address already in use); will try again.
E0801 13:12:08.116463 2481 genericapiserver.go:733] Unable to listen for secure (listen tcp 0.0.0.0:8888: bind: address already in use); will try again.
```
After fixing, the print is like this:
```
root@vm:~# F0801 11:51:44.308180 1921 genericapiserver.go:599] --secure-port and --insecure-port cannot use the same port.
goroutine 1 [running]:
k8s.io/kubernetes/vendor/github.com/golang/glog.stacks(0x4871d00, 0x0, 0x0, 0x0)
/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:766 +0xb8
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).output(0x48517c0, 0xc800000003, 0xc820368000, 0x470aeab, 0x13, 0x257, 0x0)
/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:717 +0x259
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).printf(0x48517c0, 0xc800000003, 0x3518280, 0x3b, 0x0, 0x0, 0x0)
/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:655 +0x1d4
k8s.io/kubernetes/vendor/github.com/golang/glog.Fatalf(0x3518280, 0x3b, 0x0, 0x0, 0x0)
/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:1145 +0x5d
k8s.io/kubernetes/pkg/genericapiserver.verifySecureAndInsecurePort(0xc820132800)
/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:599 +0xad
k8s.io/kubernetes/pkg/genericapiserver.ValidateRunOptions(0xc820132800)
/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:607 +0x4b
k8s.io/kubernetes/pkg/genericapiserver.DefaultAndValidateRunOptions(0xc820132800)
/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:611 +0x4e
k8s.io/kubernetes/cmd/kube-apiserver/app.Run(0xc8202c9560, 0x0, 0x0)
/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:84 +0x8e
main.main()
/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/apiserver.go:48 +0x111
[2]+ Exit 255
```
This will fix the same issue of federation-apiserver.
cc @lavalamp @quinton-hoole
Automatic merge from submit-queue
VSAN support for VSphere Volume Plugin
This PR does the following,
- Fixes#28625 (VSphere Volume doesn't unmount): modified vmdk namespace path parsing so it accurately handles VMs in folders. See file pkg/volume/vsphere_volume/vsphere_volume.go.
- Updates vmware/govmomi dependency. It was quite behind. The majority of files in the change are in this category.
- Adds support for VSAN datastore. Handle namespace to uuid mapping to assist unmount and detach in VSAN case as well. See file pkg/cloudprovider/providers/vsphere/vsphere.go.
Tested:
- Created a K8s cluster on VSphere with VSAN datastore. Created a vmdk in VSAN datastore and created pod which uses this vmdk. Before fix (VSphere Volume doesn't unmount) it failed. After fix the volume gets successfully unmounted and detached.
- Created a K8s cluster on VSphere with VMFS datastore. Created a vmdk in subdirectory of root in VMFS datastore and created pod which uses this vmdk. Before fix (VSphere Volume doesn't unmount) it failed. After fix the volume gets successfully unmounted and detached.
Automatic merge from submit-queue
Consolidating image pulling logic
Moving image puller logic into image manager by consolidating 2 pullers into one implementation.
Automatic merge from submit-queue
add DefaultServiceNodePortRange const
The PR add DefaultServiceNodePortRange const in server_run_options.go, and use it in genericapiserver.go and server_run_options.go.
Automatic merge from submit-queue
Kubelet: add kubeGenericRuntimeManager for new runtime API
Part of #28789. Add `kubeGenericRuntimeManager` for kubelet new runtime API #17048.
Note that:
- To facilitate code reviewing, #28396 is splited into a few small PRs. This is the first part.
- This PR also fixes some syntax errors in `api.proto`.
- This PR is depending on #29811 (already merged).
CC @yujuhong @Random-Liu @kubernetes/sig-node
Tamer Tas
