Automatic merge from submit-queue (batch tested with PRs 55050, 53464, 54936, 55028, 54928). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix panic in kubelet because of uninitialized map
**What this PR does / why we need it**:
Initialized the uninitialized map in kubelet
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes [#54927](https://github.com/kubernetes/kubernetes/issues/54927)
**Special notes for your reviewer**:
The default value of --enable-controller-attach-detach is true, map will be initialized like:
```
if kl.enableControllerAttachDetach {
if node.Annotations == nil {
node.Annotations = make(map[string]string)
}
...
}
```
if set --enable-controller-attach-detach to false, map will have no Initialized.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55050, 53464, 54936, 55028, 54928). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: dockershim: remove orphaned checkpoint files
Fixes https://github.com/kubernetes/kubernetes/issues/55070
Currently, `ListPodSandbox()` returns a combined list of sandboxes populated from both the runtime and the dockershim checkpoint files. However the sandboxes in the checkpoint files might not exist anymore.
The kubelet sees the sandbox returned by `ListPodSandbox()` and determines it shouldn't be running and calls `StopPodSandbox()` on it. This generates an error when `StopContainer()` is called as the container does not exist. However the checkpoint file is not cleaned up. This leads to subsequent calls to `StopPodSandbox()` that fail in the same way each time.
This PR removes the checkpoint file if StopContainer fails due to container not found.
The only other place `RemoveCheckpoint()` is called, except if it is corrupt, is from `RemoveSandbox()`. If the container does not exist, what `RemoveSandbox()` would have done has been effectively been done already. So this is just clean up.
@derekwaynecarr @eparis @freehan @dcbw
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
StopPodSandbox should not log when container is already removed
**What this PR does / why we need it**:
StopPodSandbox should not log when a container is already gone. It should only log if it could not stop and the container was still present.
Fixes https://github.com/kubernetes/kubernetes/issues/55021
**Special notes for your reviewer**:
This was seen in our production logs, need to eliminate spam.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add admission handler for device resources allocation
**What this PR does / why we need it**:
Add admission handler for device resources allocation to fail fast during pod creation
**Which issue this PR fixes**
fixes#51592
**Special notes for your reviewer**:
@jiayingz Sorry, there is something wrong with my branch in #51895. And I think the existing comments in the PR might be too long for others to view. So I closed it and opened the new one, as we have basically reach an agreement on the implement :)
I have covered the functionality and unit test part here, and would set about the e2e part ASAP
/cc @jiayingz @vishh @RenaudWasTaken
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a file store utility package in kubelet
More and more components checkpoints (i.e., persist their states) in
kubelet. Refurbish and move the implementation in dockershim to a
utility package to improve code reusability.
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Lift embedded structure out of ManifestURLHeader field
Related: #53833
```release-note
It is now possible to set multiple manifest url headers via the Kubelet's --manifest-url-header flag. Multiple headers for the same key will be added in the order provided. The ManifestURLHeader field in KubeletConfiguration object (kubeletconfig/v1alpha1) is now a map[string][]string, which facilitates writing JSON and YAML files.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubenet: yield lock while executing CNI plugin.
The CNI plugin can take up to 3 seconds to execute. CNI plugins can safely be
executed in parallel, so yield the lock to speed up pod creation.
This caused problems with the pod latency tests - previously, CNI plugins executed
in under 20ms. Now they must wait for DAD to finish and addresses to leave
tentative state.
Fixes: #54651
**What this PR does / why we need it**:
After upgrading CNI plugins to v0.6 in #51250, the pod latency tests began failing. This is because the plugins, in order to support IPv6, need to wait for DAD to finish. Because this
delay is while the kubenet lock is held, it significantly slows down the pod creation rate.
**Special notes for your reviewer**:
The CNI plugins also do locking for their critical paths, so it is safe to run them concurrently.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54894, 54630, 54828, 54926, 54865). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
set leveled logging (v=4) for 'updating container' message
**What this PR does / why we need it**:
Currently cpu_manager.go logs a line for every pod at every reconcilePeriod (10 sec default) when it reconciles and updates the pod's cpuset setting. This creates a lot of logging information that is not very interesting and we should suppress that by default by increasing the logging level.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54804
**Special notes for your reviewer**:
I chose V(4) because that seems to be a popular level for messages at this detail. Happy to follow logging guideline if there is any.
**Release note**:
``` kubelet: cpu_manager logs informative reconcile message at V(4) to reduce clutter ```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update volume OWNERS to reflect active sig-storage reviewers
**What this PR does / why we need it**:
Update sig-storage reviewers to add new members and remove those that don't have as much time to review storage PRs. Approvers are unchanged.
**Special notes for your reviewer**:
For all those that have been removed, please approve. If you want to remain as a reviewer, let me know and I will add you back.
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 53962, 54708). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Prevent successful containers from restarting with OnFailure restart policy
**What this PR does / why we need it**:
This is a follow-on to #54597 which makes sure that its validation
also applies to pods with a restart policy of OnFailure. This
deficiency was pointed out by @smarterclayton here:
https://github.com/kubernetes/kubernetes/pull/54530#discussion_r147226458
**Which issue this PR fixes** This is another fix to address #54499
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
improve the relation of ExecInContainer and Exec
keep the relation between ExecInContainer and Exec be consistence with PortForward in streaming server
fix#54903
Signed-off-by: yanxuean <yan.xuean@zte.com.cn>
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
The CNI plugin can take up to 3 seconds to execute. CNI plugins can safely be
executed in parallel, so yield the lock to speed up pod creation.
Fixes: #54651
Automatic merge from submit-queue (batch tested with PRs 49762, 52256). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add fake remote runtime service
**What this PR does / why we need it**:
Add fake remote runtime service.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
First step of #45206.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54656, 54552, 54389, 53634, 54408). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add file backed state to cpu manager
**What this PR does / why we need it**:
Adds file backed `State` implementation to cpu manger with tests.
Reads from `State` are done from memory, while each write triggers state save to a file.
Any failure in reading the state file results in empty state
Next PR: #54409
Automatic merge from submit-queue (batch tested with PRs 54593, 54607, 54539, 54105). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Removed containers are not always waiting
fixes#54499
The issue was that a container that is removed (during pod deletion, for example), is assumed to be in a "waiting" state.
Instead, we should use the previous container state.
Fetching the most recent status is required to ensure that we accurately reflect the previous state. The status attached to a pod object is often stale.
I verified this by looking through the kubelet logs during a deletion, and verifying that the status updates do not transition from terminated -> pending.
cc @kubernetes/sig-node-bugs @sjenning @smarterclayton @derekwaynecarr @dchen1107
```release-note
Fix an issue where pods were briefly transitioned to a "Pending" state during the deletion process.
```
Automatic merge from submit-queue (batch tested with PRs 54597, 54593, 54081, 54271, 54600). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: check for illegal container state transition
supersedes https://github.com/kubernetes/kubernetes/pull/54530
Puts a state transition check in the kubelet status manager to detect and block illegal transitions; namely from terminated to non-terminated.
@smarterclayton @derekwaynecarr @dashpole @joelsmith @frobware
I confirmed that the reproducer in #54499 does not work with this check in place. The erroneous kubelet status update is rejected:
```
status_manager.go:301] Status update on pod default/test aborted: terminated container test-container attempted illegal transition to non-terminated state
```
After fix https://github.com/kubernetes/kubernetes/pull/54593, I do not see the message with the above mentioned reproducer.
While moving device_plugin_handler_test.go from pkg/kubelet/cm/ to
pkg/kubelet/cm/deviceplugin/, we can no longer uses cm in its tests
because that would cause a cycle dependency. To solve this problem,
I moved the main cm GetResources functionality as well as part of the
current device plugin handler Allocate functionality into a new device
plugin handler function, GetDeviceRunContainerOptions(). This
refactoring is also needed by another PR 51895 that moves device
allocation into admission phase. Now device plugin handler Allocate()
first checks whether there is cached device runtime state and only
issues Allocate grpc call if there is no cached state available.
The new GetDeviceRunContainerOptions() function simply returns device
runtime config from the cached state. To support this change, extended the
podDevices struct and checkpoint data structure with device runtime state.
Automatic merge from submit-queue (batch tested with PRs 53743, 53564). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: remove the --network-plugin-dir flag
**What this PR does / why we need it**:
This flag has been replaced with `--cni-bin-dir`, and has been deprecated in Kubernetes 1.7.
It is safe to remove in Kubernetes 1.9 according to the deprecation policy.
**Which issue this PR fixes**: fixes#46410
**Special notes for your reviewer**:
/assign @mtaufen @freehan @dchen1107
**Release note**:
```release-note
Remove the --network-plugin-dir flag.
```
Automatic merge from submit-queue (batch tested with PRs 52747, 54329). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Device Plugin Endpoints correctly closes client connexion
**What this PR does / why we need it**:
Endpoints in the device plugin may unexpectedly end. Currently the connexion will not be properly closed.
This commit aims to fix this
Related issues #51993
**Special notes for your reviewer**: @jiayingz @mindprince @vishh
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52147, 54309). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Deviceplugin refactoring: cleanup some unnecessary functions
**What this PR does / why we need it**:
cleanup some of unnecessary functions of deviceplugin to improve code readability
**Which issue this PR fixes**
fixes#51993
Part1
**Special notes for your reviewer**:
Currently, it seems that func `IsResourceNameValid` is not used outside and could be changed to internal.
But as Renaud commented, it is designed for potential usage from users outside.
And I could not find a better position to place the func, except for file `utils.go`. So, shall we just retain the `utils.go` file, and maybe for possible expansion in the future?
/cc @jiayingz @RenaudWasTaken @vishh
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
avoid kubelet converts and validates pods multiple times
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#53171
**Special notes for your reviewer**:
/assign @yujuhong @caesarxuchao
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix dockershim panic when listing images
**What this PR does / why we need it**:
dockershim panic when listing containers because of `opts.Filters` not initialized:
505ccb88da/pkg/kubelet/dockershim/docker_image.go (L35-L39)
Also when imgSpec.Image is empty string, dockershim returns an empty image list which is not expected. (We should not set opts.Filters in this case).
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#54122
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make AllocateResponse artifacts global across all devices per container in device plugin API
The current version of Device plugins returns artifacts (env vars, mounts, devices) per device, per container. This is not necessary and results in complex merging issues on kubelet side
There can still be a conflict if the artifacts returned by device plugin conflicts with the pod spec. In that case, I'd recommend failing pods in kubelet. This is yet to be addressed.
The go package name for device plugin apis is updated from `pkg/kubelet/apis/deviceplugin/v1alpha1` to `pkg/kubelet/apis/deviceplugin/v1alpha` (sub-version dropped) because we expect the alpha version to change until it graduates to beta and changing the go package everytime the actual alpha version changes is too tedious.
```release-note
Device plugin Alpha API no longer supports returning artifacts per device as part of AllocateResponse.
```
TODO:
- [x] Bump kubelet side API version
- [X] Post an updated device plugin image that works with the new API version
- [ ] Stabilize e2es (This PR needs to be merged since there is a dependency on the plugin side for vendoring)
Kubernetes Submit Queue