Automatic merge from submit-queue (batch tested with PRs 54824, 55911, 55730, 55979, 55961). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add kubeletconfig round trip test
I noticed we were missing one of these.
fixes#55959
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54824, 55911, 55730, 55979, 55961). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Device plugin API merge of handler and manager
**What this PR does / why we need it**: We are trying different approaches to make the device plugin implementation simpler and more robust. One option is to merge the notion of the `device_plugin_handler` into the `device_manager`. This is for several reasons:
1) Some calls go directly from handler to manager, adding little value.
2) The separation of concern is not clear between the two components. They have a 1:1 relationship.
3) The separation and abstractions needed are at a different level. Code that can be refactored will most likely live in abstractions which hide details around lock acquisition and check pointing.
In this PR, we will **just** merge the two interfaces. After this, there is several opportunities for simplifying and cleaning up the device plugin.
Fixes#55180
**Special notes for your reviewer**: This is a WIP. May very well get dropped, but keeping up for the sake of early sharing and showing the progress of the code move.
**Release note**:
```release-note
NONE
```
This PR adds pod-level ephemeral storage metric into Summary API.
Pod-level ephemeral storage usage is the sum of all containers and local
ephemeral volume including EmptyDir (if not backed up by memory or
hugepages), configueMap, and downwardAPI.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix cadvisor.New signature for cross build
**What this PR does / why we need it**: fixes the `pkg/kubelet/cadvisor.New` signature on non-linux platforms to match the new one on linux. This should fix the cross build
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#56002
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig release
Automatic merge from submit-queue (batch tested with PRs 55839, 54495, 55884, 55983, 56069). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
seccomp is an alpha feature and not feature gated
Move SeccompProfileRoot to KubeletFlags and document flag as alpha.
wrt https://github.com/kubernetes/kubernetes/pull/53833#issuecomment-345396575, seccomp is an alpha feature, but this isn't clearly documented anywhere (the annotation just has the word "alpha" in it, and that's your signal that it's alpha).
Since seccomp was around before feature gates, it doesn't have one.
Thus SeccompProfileRoot should not be part of KubeletConfiguration, and this PR moves it to KubeletFlags, and amends the help text to note the alpha state of the feature.
fixes: #56087
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55839, 54495, 55884, 55983, 56069). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
deviceplugin: fix race when multiple plugins are registered
**What this PR does / why we need it**:
When registering multiple device plugins to Kubelet concurrently, there exists a race that crashes the Kubelet.
Consider two plugins: D1 and D2. The call order method is roughly
D1 -> manager.go:register -> endpoint.go:listAndWatch -> device_plugin_handler.go:(*D1).callback
D2 -> manager.go:register -> endpoint.go:listAndWatch -> device_plugin_handler.go:(*D2).callback
The callback function accesses HandlerImpl's allDevices map that maps (resourceName -> DeviceID). If both plugins reach these accesses at the same time, Kubelet crashes with "fatal error: concurrent map read and map write".
This can be solved by making sure handler is locked when allDevices are being updated. The functionality is needed to avoid Kubelet crashes when multiple device plugins are trying to register with Kubelet at the same moment. Occurs frequently when single binary tries to register itself as multiple plugins.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55841, 55948, 55945). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
CPU Manager: file state for all policies
**What this PR does / why we need it**:
Before this change, the new file-backed state was only enabled for the static CPU manager policy. This patch enables persistent state for all policies.
This PR fixes#55736 and the potential CPU resource leak described in that issue.
**Release note**:
```release-note
NONE
```
/kind bug
/sig node
/assign @balajismaniam
Automatic merge from submit-queue (batch tested with PRs 54837, 55970, 55912, 55898, 52977). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improve kubelet cgroup
**What this PR does / why we need it**:
1.Use arg cgroupRoot,not nodeConfig.CgroupRoot
Using both arg cgroupRoot and nodeConfig.CgroupRoot is confused in function NewQOSContainerManager
2.improve cgroupmanager in qosContainerManager
3. improve arg "cgroupRoot" type in NewQOSContainerManager
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 50457, 55558, 53483, 55731, 52842). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
improve the logic setting cgroupparent in RunPodSandbox
Signed-off-by: yanxuean <yan.xuean@zte.com.cn>
**What this PR does / why we need it**:
The setting of cgroupparent is too confused!
The old logic is:
1. set CgroupParent correctly
2. reset CgroupParent incorrectly
3. set CgroupParent again (refer to #42055 )
The login is too confused, and It is sure that there are many people who drop in trap.
We only need to set it in one place.
kubernetes/pkg/kubelet/dockershim/docker_sandbox.go
```
func (ds *dockerService) makeSandboxDockerConfig(c *runtimeapi.PodSandboxConfig, image string) (*dockertypes.ContainerCreateConfig, error) {
....
// Apply linux-specific options.
if lc := c.GetLinux(); lc != nil {
if err := ds.applySandboxLinuxOptions(hc, lc, createConfig, image, securityOptSep); err != nil {
return nil, err
}
}
// Apply resource options.
setSandboxResources(hc) **<-- reset the CgroupParent incorrectly**
// Apply cgroupsParent derived from the sandbox config.
if lc := c.GetLinux(); lc != nil {
// Apply Cgroup options.
cgroupParent, err := ds.GenerateExpectedCgroupParent(lc.CgroupParent)
if err != nil {
return nil, fmt.Errorf("failed to generate cgroup parent in expected syntax for container %q: %v", c.Metadata.Name, err)
}
hc.CgroupParent = cgroupParent
}
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55642, 55897, 55835, 55496, 55313). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix issue #55031 to remove dependence on package lxn/win
**What this PR does / why we need it**:
This PR fixes issue #55031 where kubelet.exe crashes on Windows Server Core. The root cause is that kubelet.exe depends on package lxn/win pdh and kernel32 wrapper for node metrics. However, opengl32.dll is not available in Server Core and lxn/win requires the presence of all win32 DLLs.
This PR uses a slim win32 package JeffAshton/win_pdh since most win32 APIs needed are PDH API. Also this PR makes own implementation of GetPhysicallyInstalledSystemMemory until golang Windows syscall has it or lxn/win fixes opengl32 issue. Also this PR modifies the way to get Windows version.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55031
**Special notes for your reviewer**:
**Release note**:
```release-note
```
/sig windows
/sig node
Automatic merge from submit-queue (batch tested with PRs 55642, 55897, 55835, 55496, 55313). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Disable container disk metrics when using the CRI stats integration
Issue: https://github.com/kubernetes/kubernetes/issues/51798
As explained in the issue, runtimes which make use of the CRI Stats API still have the performance overhead of collecting those same stats through cAdvisor.
The CRI Stats API has metrics for CPU, Memory, and Disk. This PR significantly reduces the added overhead due to collecting these stats in both cAdvisor and in the runtime.
This PR disables container disk metrics, which are very expensive to collect.
This PR does not disable node-level disk stats, as the "Raw" container handler does not currently respect ignoring DiskUsageMetrics.
This PR factors out the logic for determining whether or not to use the CRI stats provider into a helper function, as cAdvisor is instantiated before it is passed to the kubelet as a dependency.
cc @kubernetes/sig-node-pr-reviews @derekwaynecarr
/kind feature
/sig node
/assign @Random-Liu @derekwaynecarr
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Filter out duplicated container stats
**What this PR does / why we need it**:
**Which issue this PR fixes** *
fixes#53514
**Special notes for your reviewer**:
/cc @Random-Liu
Signed-off-by: Yanqiang Miao <miao.yanqiang@zte.com.cn>
Automatic merge from submit-queue (batch tested with PRs 55908, 55829, 55293, 55653, 55665). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
If mountPath is missing, prefix with root dir.
If `mountPath` is not absolute, add the os-specific root directory as a prefix.
Ref: https://github.com/kubernetes/kubernetes/pull/51240https://groups.google.com/forum/#!topic/kubernetes-sig-storage/k_0Wr2kYkpU
@thockin @saad-ali @andyzhangx
Note to @thockin I left the validation in place, in seems like it is a net win, since it will give a decent
error message to most people, but the fall-back defaulting is there if it doesn't catch 'c:'
I'm happy to rip out the validation if that is preferable to everyone. Let me know.
```release-note
If a non-absolute mountPath is passed to the kubelet, prefix it with the appropriate root path.
```
Automatic merge from submit-queue (batch tested with PRs 55908, 55829, 55293, 55653, 55665). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix accelerator stats API to follow API conventions.
Introduced in #55188
**Release note**:
```release-note
None
```
The first call to Set will clear the map before adding entries;
subsequent calls will simply append to the map.
This makes it possible to override default values with a command-line
option rather than appending to defaults,
while still allowing the distribution of key-value pairs across
multiple flag invocations.
For example: `--flag "a:hello" --flag "b:again" --flag "b:beautiful"
--flag "c:world"` results in `{"a": ["hello"], "b": ["again",
"beautiful"], "c": ["world"]}`
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Lift embedded structure out of eviction-related KubeletConfiguration fields
- Changes the following KubeletConfiguration fields from `string` to
`map[string]string`:
- `EvictionHard`
- `EvictionSoft`
- `EvictionSoftGracePeriod`
- `EvictionMinimumReclaim`
- Adds flag parsing shims to maintain Kubelet's public flags API, while
enabling structured input in the file API.
- Also removes `kubeletconfig.ConfigurationMap`, which was an ad-hoc flag
parsing shim living in the kubeletconfig API group, and replaces it
with the `MapStringString` shim introduced in this PR. Flag parsing
shims belong in a common place, not in the kubeletconfig API.
I manually audited these to ensure that this wouldn't cause errors
parsing the command line for syntax that would have previously been
error free (`kubeletconfig.ConfigurationMap` was unique in that it
allowed keys to be provided on the CLI without values. I believe this was
done in `flags.ConfigurationMap` to facilitate the `--node-labels` flag,
which rightfully accepts value-free keys, and that this shim was then
just copied to `kubeletconfig`). Fortunately, the affected fields
(`ExperimentalQOSReserved`, `SystemReserved`, and `KubeReserved`) expect
non-empty strings in the values of the map, and as a result passing the
empty string is already an error. Thus requiring keys shouldn't break
anyone's scripts.
- Updates code and tests accordingly.
Regarding eviction operators, directionality is already implicit in the
signal type (for a given signal, the decision to evict will be made when
crossing the threshold from either above or below, never both). There is
no need to expose an operator, such as `<`, in the API. By changing
`EvictionHard` and `EvictionSoft` to `map[string]string`, this PR
simplifies the experience of working with these fields via the
`KubeletConfiguration` type. Again, flags stay the same.
Other things:
- There is another flag parsing shim, `flags.ConfigurationMap`, from the
shared flag utility. The `NodeLabels` field still uses
`flags.ConfigurationMap`. This PR moves the allocation of the
`map[string]string` for the `NodeLabels` field from
`AddKubeletConfigFlags` to the defaulter for the external
`KubeletConfiguration` type. Flags are layered on top of an internal
object that has undergone conversion from a defaulted external object,
which means that previously the mere registration of flags would have
overwritten any previously-defined defaults for `NodeLabels` (fortunately
there were none).
Related: #53833 (lifting embedded structures out of string fields is part of getting this API to beta)
```release-note
The EvictionHard, EvictionSoft, EvictionSoftGracePeriod, EvictionMinimumReclaim, SystemReserved, and KubeReserved fields in the KubeletConfiguration object (kubeletconfig/v1alpha1) are now of type map[string]string, which facilitates writing JSON and YAML files.
```
- Changes the following KubeletConfiguration fields from `string` to
`map[string]string`:
- `EvictionHard`
- `EvictionSoft`
- `EvictionSoftGracePeriod`
- `EvictionMinimumReclaim`
- Adds flag parsing shims to maintain Kubelet's public flags API, while
enabling structured input in the file API.
- Also removes `kubeletconfig.ConfigurationMap`, which was an ad-hoc flag
parsing shim living in the kubeletconfig API group, and replaces it
with the `MapStringString` shim introduced in this PR. Flag parsing
shims belong in a common place, not in the kubeletconfig API.
I manually audited these to ensure that this wouldn't cause errors
parsing the command line for syntax that would have previously been
error free (`kubeletconfig.ConfigurationMap` was unique in that it
allowed keys to be provided on the CLI without values. I believe this was
done in `flags.ConfigurationMap` to facilitate the `--node-labels` flag,
which rightfully accepts value-free keys, and that this shim was then
just copied to `kubeletconfig`). Fortunately, the affected fields
(`ExperimentalQOSReserved`, `SystemReserved`, and `KubeReserved`) expect
non-empty strings in the values of the map, and as a result passing the
empty string is already an error. Thus requiring keys shouldn't break
anyone's scripts.
- Updates code and tests accordingly.
Regarding eviction operators, directionality is already implicit in the
signal type (for a given signal, the decision to evict will be made when
crossing the threshold from either above or below, never both). There is
no need to expose an operator, such as `<`, in the API. By changing
`EvictionHard` and `EvictionSoft` to `map[string]string`, this PR
simplifies the experience of working with these fields via the
`KubeletConfiguration` type. Again, flags stay the same.
Other things:
- There is another flag parsing shim, `flags.ConfigurationMap`, from the
shared flag utility. The `NodeLabels` field still uses
`flags.ConfigurationMap`. This PR moves the allocation of the
`map[string]string` for the `NodeLabels` field from
`AddKubeletConfigFlags` to the defaulter for the external
`KubeletConfiguration` type. Flags are layered on top of an internal
object that has undergone conversion from a defaulted external object,
which means that previously the mere registration of flags would have
overwritten any previously-defined defaults for `NodeLabels` (fortunately
there were none).
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move 'alpha' KubeletConfiguration fields that aren't feature-gated and self-registration fields to KubeletFlags
Some of these fields are marked "alpha" in help text. They cannot be in the KubeletConfiguration object unless they are feature gated or graduated from alpha.
Others relate to Kubelet self-registration, and given https://github.com/kubernetes/community/pull/911 I think its prudent to wait and see if these really should be in the KubeletConfiguration type.
For now we just leave them all as flags.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55697, 55631, 51905, 55647, 55826). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GetPodPortMappings should return nil error if checkpoint returns with KeyNotFound error
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55819
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55657, 54758, 47584, 55758, 55651). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Move DNS related kubelet codes into its own package
**What this PR does / why we need it**:
Ref https://github.com/kubernetes/features/issues/504, this PR rearranges DNS related kubelet codes into its own pacakge and adds an OWNERS file.
Again, there is no functional changes, just that codes are moved around and couple fields (`clusterDomain`, `clusterDNS`, `resolverConfig`) are replaced with a `dnsConfigurer` struct.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #NONE
**Special notes for your reviewer**:
/assign @bowei @thockin
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55764, 55683, 55468, 54409, 55546). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Enable file back state in static policy
**What this PR does / why we need it**:
Enables file back `State` in `static policy` and cpu manager + tests.
Upon policy start, state read from file is validated whether it meets the policy assumption. In case of any error, state is cleared.
Previous PR: #54408
Next PR: #54409
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Deviceplugin refactoring: merge func list and listwatch in endpoint into one
**What this PR does / why we need it**:
merge func list and listwatch in endpoint into one, since we won't call list func individually
**Which issue this PR fixes**
fixes#51993
Part2
**Special notes for your reviewer**:
/cc @jiayingz @RenaudWasTaken @vishh
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54436, 53148, 55153, 55614, 55484). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix hostport duplicate chain names
Fixes bad conversion from int32 to string. Without this patch, getHostportChain/hostportChainName generates the same chain names for ports 57119/55429/56833 of the same pod.
closes#55771
```release-note
Fixes bad conversion in host port chain name generating func which leads to some unreachable host ports.
```
Automatic merge from submit-queue (batch tested with PRs 54436, 53148, 55153, 55614, 55484). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
ip6tables should be set in the noop plugin
**What this PR does / why we need it**:
The noop plugin currently sets the iptables for IPv4.
This updates that to also set the iptables for IPv6 so
IPv6 can have parity with IPv4.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#53147
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55648, 55274, 54982, 51955, 55639). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubelet: Relative paths in local config file
Resolve relative paths against the config file's location.
Issue: #55644
Related comment: https://github.com/kubernetes/kubernetes/pull/53833#issuecomment-344009912
Will add the same behavior for dynamic Kubelet config in a future PR, see issue #55645.
```release-note
Relative paths in the Kubelet's local config files (--init-config-dir) will be resolved relative to the location of the containing files.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adds jiayingz@ and vish@ as approvers for pkg/kubelet/cm/deviceplugin/.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 53780, 55663, 55321, 52421, 55659). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove duplicated import
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Complete test case for kubeletconfig api validation
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55664
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Adds Support for Node Resource IPv6 Addressing
**What this PR does / why we need it**:
This PR adds support for the following:
1. A node resource to be assigned an IPv6 address.
2. Expands IPv4/v6 address validation checks.
**Which issue this PR fixes**:
Fixes Issue #44848 (in combination with PR #45116).
**Special notes for your reviewer**:
This PR is part of a larger effort, Issue #1443 to add IPv6 support to k8s.
**Release note**:
```
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
dockershim: remove corrupt checkpoints immediately upon detection
Fixes https://github.com/kubernetes/kubernetes/issues/55620
The current checkpoint abstraction leaks the handling of corrupt checkpoints to the user.
If the user does a `GetCheckpoint()` and the checkpoint is corrupt, the corrupt checkpoint is returned to the user (why??) with an error indicating the corruption. It is then up to the user to detect the corruption via the error msg and call `RemoveCheckpoint()` to remove the corrupted checkpoint.
The checkpoint abstraction should not expose corruption to the user. If it is corrupt, it is as good as not found to the user. The checkpoint code should handle cleanup of corrupt entries and report "not found" to the user.
@derekwaynecarr @eparis @dcbw @freehan
Automatic merge from submit-queue (batch tested with PRs 54005, 55127, 53850, 55486, 53440). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Containerized mount utilities
This is implementation of https://github.com/kubernetes/community/pull/589
@tallclair @vishh @dchen1107 PTAL
@kubernetes/sig-node-pr-reviews
**Release note**:
```release-note
Kubelet supports running mount utilities and final mount in a container instead running them on the host.
```
Automatic merge from submit-queue (batch tested with PRs 55283, 55461, 55288, 53970, 55487). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add more comments for DevicePluginHandlerImpl struct
**What this PR does / why we need it**:
Add more comments
**Special notes for your reviewer**:
@jiayingz PTAL.
**Release note**:
```
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[Part 1] Remove docker dep in kubelet startup
**What this PR does / why we need it**:
Remove dependency of docker during kubelet start up.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Part 1 of #54090
**Special notes for your reviewer**:
Changes include:
1. Move docker client initialization into dockershim pkg.
2. Pass a docker `ClientConfig` from kubelet to dockershim
3. Pass parameters needed by `FakeDockerClient` thru `ClientConfig` to dockershim
(TODO, the second part) Make dockershim tolerate when dockerd is down, otherwise it will still fail kubelet
Please note after this PR, kubelet will still fail if dockerd is down, this will be fixed in the subsequent PR by making dockershim tolerate dockerd failure (initializing docker client in a separate goroutine), and refactoring cgroup and log driver detection.
**Release note**:
```release-note
Remove docker dependency during kubelet start up
```
Automatic merge from submit-queue (batch tested with PRs 54460, 55258, 54858, 55506, 55510). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
redendancy code and error log message in cni
**What this PR does / why we need it**:
redendancy code and error log message in cni
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig-node
Adds support for the following:
1. A node resource to be assigned an IPv6 address.
2. Expands IPv4/v6 address validation checks.
Which issue this PR fixes:
fixes#44848 in combination with PR #45116
Special notes for your reviewer:
Release note:
With this PR, nodes can be assigned an IPv6 address. An IPv4 address is
preferred over an IPv6 address. IP address validation has been expanded
to check for multicast, link-local and unspecified addresses.
This PR uses a slim win32 package JeffAshton/win_pdh since most win32 APIs needed are PDH API. Also this PR makes own implementation of GetPhysicallyInstalledSystemMemory until golang Windows syscall has it or lxn/win fixes opengl32 issue. Also this PR modifies the way to get Windows version.
assert.Contains() checks if its second argument (which is supposed to be
a single element) is contained in its first argument (which is supposed
to be a slice/map etc.) The third and following arguments are supposed
to be message and args for the output in case of failure.
Because of this bad form, a failure was hidden, the system container is
named "misc", not "system".
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubenet: disable DAD in the container.
Since kubenet externally guarantees that IP address will not conflict, we can short-circuit the kernel's normal wait. This lets us avoid the 1 second network wait.
**What this PR does / why we need it**:
Fixes the pod startup latency identified in #54651 and #55060
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Support copying "options" in resolv.conf into pod sandbox when dnsPolicy is Default
**What this PR does / why we need it**:
This PR adds support for copying "options" from host's /etc/resolv.conf (or --resolv-conf) into pod's resolv.conf when dnsPolicy is Default. Being able to customize options is important because it is common to leverage options to fine-tune the behavior of DNS client.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#42542
**Special notes for your reviewer**:
I originally wanted to also tackle the issue of copying options for when dnsPolicy is ClusterFirst, but with ability to "merge" with default options (ndots:5 more specifically) when it makes sense. I decided to leave it off for now because the "merging" may need more discussions. Happy to add that to this PR or create another PR for that if it makes sense and is clear what should be done. I think even when dnsPolicy is ClusterFirst it is important to allow customization.
**Release note**:
```kubelet: add support for copying "options" from /etc/resolv.conf (or --resolv-conf if it is used) into pod's /etc/resolv.conf when dnsPolicy is Default.```
Automatic merge from submit-queue (batch tested with PRs 53747, 54528, 55279, 55251, 55311). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
a typo in dockershim.cm.containerManager.doWork
Signed-off-by: yanxuean <yan.xuean@zte.com.cn>
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/sig node
Automatic merge from submit-queue (batch tested with PRs 54177, 55203, 55120, 55275, 55260). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
GCE: provide an option to disable docker's live-restore
**What this PR does / why we need it**:
Provide an option to disable docker's live-restore for COS/ubuntu images on GCE. Some newer COS images have live-restore enabled by default. This allows users to override the option if needed.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
GCE: provide an option to disable docker's live-restore on COS/ubuntu
```
Automatic merge from submit-queue (batch tested with PRs 55331, 55272, 55228, 49763, 55242). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
use versiond group clients from client-go
**What this PR does / why we need it**:
Some **Deprecated** group clients are still used, replace them with versioned group clients.
**Which issue this PR fixes**: fixes#49760
**Special notes for your reviewer**:
/assign @caesarxuchao
**Release note**:
```release-note
NONE
```
Since kubenet externally guarantees that IP address will not conflict,
we can short-circuit the kernel's normal wait. This lets us avoid the 1
second network wait.
Automatic merge from submit-queue (batch tested with PRs 55114, 52976, 54871, 55122, 55140). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make CRI logs parsing to a library
**What this PR does / why we need it**:
Make CRI logs parsing to a library.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55136
**Special notes for your reviewer**:
**Release note**:
```release-note
Add CRI log parsing library at pkg/kubelet/apis/cri/logs
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Clean up redundant DNS related codes
**What this PR does / why we need it**:
As https://github.com/kubernetes/kubernetes/pull/54773#discussion_r148904955 described, resolv.conf setup for pod is handled by `generatePodSandboxConfig()`, though we have some redundant DNS related codes in `GenerateRunContainerOptions()` which seems to have no effect.
This PR cleans up the ineffective codes and rearranges the cluster DNS unit test and hopefully it would be less confusing.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#55201
**Special notes for your reviewer**:
cc @Random-Liu @phsiao
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55034, 55068). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Close the file before renaming in FileStore
Also change the unit test to use a real file system to detect errors
like this.
Automatic merge from submit-queue (batch tested with PRs 53679, 51063). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes to enable Windows CNI
**What this PR does / why we need it**:
This PR has fixed which enables Kubelet to use Windows CNI plugin.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
#49646
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 55050, 53464, 54936, 55028, 54928). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix panic in kubelet because of uninitialized map
**What this PR does / why we need it**:
Initialized the uninitialized map in kubelet
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes [#54927](https://github.com/kubernetes/kubernetes/issues/54927)
**Special notes for your reviewer**:
The default value of --enable-controller-attach-detach is true, map will be initialized like:
```
if kl.enableControllerAttachDetach {
if node.Annotations == nil {
node.Annotations = make(map[string]string)
}
...
}
```
if set --enable-controller-attach-detach to false, map will have no Initialized.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55050, 53464, 54936, 55028, 54928). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: dockershim: remove orphaned checkpoint files
Fixes https://github.com/kubernetes/kubernetes/issues/55070
Currently, `ListPodSandbox()` returns a combined list of sandboxes populated from both the runtime and the dockershim checkpoint files. However the sandboxes in the checkpoint files might not exist anymore.
The kubelet sees the sandbox returned by `ListPodSandbox()` and determines it shouldn't be running and calls `StopPodSandbox()` on it. This generates an error when `StopContainer()` is called as the container does not exist. However the checkpoint file is not cleaned up. This leads to subsequent calls to `StopPodSandbox()` that fail in the same way each time.
This PR removes the checkpoint file if StopContainer fails due to container not found.
The only other place `RemoveCheckpoint()` is called, except if it is corrupt, is from `RemoveSandbox()`. If the container does not exist, what `RemoveSandbox()` would have done has been effectively been done already. So this is just clean up.
@derekwaynecarr @eparis @freehan @dcbw
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
StopPodSandbox should not log when container is already removed
**What this PR does / why we need it**:
StopPodSandbox should not log when a container is already gone. It should only log if it could not stop and the container was still present.
Fixes https://github.com/kubernetes/kubernetes/issues/55021
**Special notes for your reviewer**:
This was seen in our production logs, need to eliminate spam.
**Release note**:
```release-note
NONE
```
Following are part of this commit
+++++++++++++++++++++++++++++++++
* Windows CNI Support
(1) Support to use --network-plugin=cni
(2) Handled platform requirement of calling CNI ADD for all the containers.
(2.1) For POD Infra container, netNs has to be empty
(2.2) For all other containers, sharing the network namespace of POD container,
should pass netNS name as "container:<Pod Infra Container Id>", same as the
NetworkMode of the current container
(2.3) The Windows CNI plugin has to handle this to call into Platform.
Sample Windows CNI Plugin code to be shared soon.
* Sandbox support for Windows
(1) Sandbox support for Windows. Works only with Docker runtime.
(2) Retained CONTAINER_NETWORK as a backward compatibilty flag,
to not break existing deployments using it.
(3) Works only with CNI plugin enabled.
(*) Changes to reinvoke CNI ADD for every new container created. This is hooked up with PodStatus,
but would be ideal to move it outside of this, once we have CNI GET support
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add admission handler for device resources allocation
**What this PR does / why we need it**:
Add admission handler for device resources allocation to fail fast during pod creation
**Which issue this PR fixes**
fixes#51592
**Special notes for your reviewer**:
@jiayingz Sorry, there is something wrong with my branch in #51895. And I think the existing comments in the PR might be too long for others to view. So I closed it and opened the new one, as we have basically reach an agreement on the implement :)
I have covered the functionality and unit test part here, and would set about the e2e part ASAP
/cc @jiayingz @vishh @RenaudWasTaken
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a file store utility package in kubelet
More and more components checkpoints (i.e., persist their states) in
kubelet. Refurbish and move the implementation in dockershim to a
utility package to improve code reusability.
Automatic merge from submit-queue (batch tested with PRs 52367, 53363, 54989, 54872, 54643). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Lift embedded structure out of ManifestURLHeader field
Related: #53833
```release-note
It is now possible to set multiple manifest url headers via the Kubelet's --manifest-url-header flag. Multiple headers for the same key will be added in the order provided. The ManifestURLHeader field in KubeletConfiguration object (kubeletconfig/v1alpha1) is now a map[string][]string, which facilitates writing JSON and YAML files.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubenet: yield lock while executing CNI plugin.
The CNI plugin can take up to 3 seconds to execute. CNI plugins can safely be
executed in parallel, so yield the lock to speed up pod creation.
This caused problems with the pod latency tests - previously, CNI plugins executed
in under 20ms. Now they must wait for DAD to finish and addresses to leave
tentative state.
Fixes: #54651
**What this PR does / why we need it**:
After upgrading CNI plugins to v0.6 in #51250, the pod latency tests began failing. This is because the plugins, in order to support IPv6, need to wait for DAD to finish. Because this
delay is while the kubenet lock is held, it significantly slows down the pod creation rate.
**Special notes for your reviewer**:
The CNI plugins also do locking for their critical paths, so it is safe to run them concurrently.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54894, 54630, 54828, 54926, 54865). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
set leveled logging (v=4) for 'updating container' message
**What this PR does / why we need it**:
Currently cpu_manager.go logs a line for every pod at every reconcilePeriod (10 sec default) when it reconciles and updates the pod's cpuset setting. This creates a lot of logging information that is not very interesting and we should suppress that by default by increasing the logging level.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54804
**Special notes for your reviewer**:
I chose V(4) because that seems to be a popular level for messages at this detail. Happy to follow logging guideline if there is any.
**Release note**:
``` kubelet: cpu_manager logs informative reconcile message at V(4) to reduce clutter ```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update volume OWNERS to reflect active sig-storage reviewers
**What this PR does / why we need it**:
Update sig-storage reviewers to add new members and remove those that don't have as much time to review storage PRs. Approvers are unchanged.
**Special notes for your reviewer**:
For all those that have been removed, please approve. If you want to remain as a reviewer, let me know and I will add you back.
**Release note**:
NONE
Automatic merge from submit-queue (batch tested with PRs 53962, 54708). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Prevent successful containers from restarting with OnFailure restart policy
**What this PR does / why we need it**:
This is a follow-on to #54597 which makes sure that its validation
also applies to pods with a restart policy of OnFailure. This
deficiency was pointed out by @smarterclayton here:
https://github.com/kubernetes/kubernetes/pull/54530#discussion_r147226458
**Which issue this PR fixes** This is another fix to address #54499
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
improve the relation of ExecInContainer and Exec
keep the relation between ExecInContainer and Exec be consistence with PortForward in streaming server
fix#54903
Signed-off-by: yanxuean <yan.xuean@zte.com.cn>
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
The CNI plugin can take up to 3 seconds to execute. CNI plugins can safely be
executed in parallel, so yield the lock to speed up pod creation.
Fixes: #54651
Automatic merge from submit-queue (batch tested with PRs 49762, 52256). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add fake remote runtime service
**What this PR does / why we need it**:
Add fake remote runtime service.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
First step of #45206.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 54656, 54552, 54389, 53634, 54408). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add file backed state to cpu manager
**What this PR does / why we need it**:
Adds file backed `State` implementation to cpu manger with tests.
Reads from `State` are done from memory, while each write triggers state save to a file.
Any failure in reading the state file results in empty state
Next PR: #54409
Automatic merge from submit-queue (batch tested with PRs 54593, 54607, 54539, 54105). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Removed containers are not always waiting
fixes#54499
The issue was that a container that is removed (during pod deletion, for example), is assumed to be in a "waiting" state.
Instead, we should use the previous container state.
Fetching the most recent status is required to ensure that we accurately reflect the previous state. The status attached to a pod object is often stale.
I verified this by looking through the kubelet logs during a deletion, and verifying that the status updates do not transition from terminated -> pending.
cc @kubernetes/sig-node-bugs @sjenning @smarterclayton @derekwaynecarr @dchen1107
```release-note
Fix an issue where pods were briefly transitioned to a "Pending" state during the deletion process.
```
Automatic merge from submit-queue (batch tested with PRs 54597, 54593, 54081, 54271, 54600). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: check for illegal container state transition
supersedes https://github.com/kubernetes/kubernetes/pull/54530
Puts a state transition check in the kubelet status manager to detect and block illegal transitions; namely from terminated to non-terminated.
@smarterclayton @derekwaynecarr @dashpole @joelsmith @frobware
I confirmed that the reproducer in #54499 does not work with this check in place. The erroneous kubelet status update is rejected:
```
status_manager.go:301] Status update on pod default/test aborted: terminated container test-container attempted illegal transition to non-terminated state
```
After fix https://github.com/kubernetes/kubernetes/pull/54593, I do not see the message with the above mentioned reproducer.
While moving device_plugin_handler_test.go from pkg/kubelet/cm/ to
pkg/kubelet/cm/deviceplugin/, we can no longer uses cm in its tests
because that would cause a cycle dependency. To solve this problem,
I moved the main cm GetResources functionality as well as part of the
current device plugin handler Allocate functionality into a new device
plugin handler function, GetDeviceRunContainerOptions(). This
refactoring is also needed by another PR 51895 that moves device
allocation into admission phase. Now device plugin handler Allocate()
first checks whether there is cached device runtime state and only
issues Allocate grpc call if there is no cached state available.
The new GetDeviceRunContainerOptions() function simply returns device
runtime config from the cached state. To support this change, extended the
podDevices struct and checkpoint data structure with device runtime state.
Automatic merge from submit-queue (batch tested with PRs 53743, 53564). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet: remove the --network-plugin-dir flag
**What this PR does / why we need it**:
This flag has been replaced with `--cni-bin-dir`, and has been deprecated in Kubernetes 1.7.
It is safe to remove in Kubernetes 1.9 according to the deprecation policy.
**Which issue this PR fixes**: fixes#46410
**Special notes for your reviewer**:
/assign @mtaufen @freehan @dchen1107
**Release note**:
```release-note
Remove the --network-plugin-dir flag.
```
Automatic merge from submit-queue (batch tested with PRs 52747, 54329). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Device Plugin Endpoints correctly closes client connexion
**What this PR does / why we need it**:
Endpoints in the device plugin may unexpectedly end. Currently the connexion will not be properly closed.
This commit aims to fix this
Related issues #51993
**Special notes for your reviewer**: @jiayingz @mindprince @vishh
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52147, 54309). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Deviceplugin refactoring: cleanup some unnecessary functions
**What this PR does / why we need it**:
cleanup some of unnecessary functions of deviceplugin to improve code readability
**Which issue this PR fixes**
fixes#51993
Part1
**Special notes for your reviewer**:
Currently, it seems that func `IsResourceNameValid` is not used outside and could be changed to internal.
But as Renaud commented, it is designed for potential usage from users outside.
And I could not find a better position to place the func, except for file `utils.go`. So, shall we just retain the `utils.go` file, and maybe for possible expansion in the future?
/cc @jiayingz @RenaudWasTaken @vishh
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
avoid kubelet converts and validates pods multiple times
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#53171
**Special notes for your reviewer**:
/assign @yujuhong @caesarxuchao
**Release note**:
```release-note
None
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix dockershim panic when listing images
**What this PR does / why we need it**:
dockershim panic when listing containers because of `opts.Filters` not initialized:
505ccb88da/pkg/kubelet/dockershim/docker_image.go (L35-L39)
Also when imgSpec.Image is empty string, dockershim returns an empty image list which is not expected. (We should not set opts.Filters in this case).
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#54122
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make AllocateResponse artifacts global across all devices per container in device plugin API
The current version of Device plugins returns artifacts (env vars, mounts, devices) per device, per container. This is not necessary and results in complex merging issues on kubelet side
There can still be a conflict if the artifacts returned by device plugin conflicts with the pod spec. In that case, I'd recommend failing pods in kubelet. This is yet to be addressed.
The go package name for device plugin apis is updated from `pkg/kubelet/apis/deviceplugin/v1alpha1` to `pkg/kubelet/apis/deviceplugin/v1alpha` (sub-version dropped) because we expect the alpha version to change until it graduates to beta and changing the go package everytime the actual alpha version changes is too tedious.
```release-note
Device plugin Alpha API no longer supports returning artifacts per device as part of AllocateResponse.
```
TODO:
- [x] Bump kubelet side API version
- [X] Post an updated device plugin image that works with the new API version
- [ ] Stabilize e2es (This PR needs to be merged since there is a dependency on the plugin side for vendoring)
There is no use case known for passing artifacts per device as it currently exists. The current API is also
complex to use for simple clients. Hence this PR creates a flat namespace where artifacts like environment variables
and mount points apply globally to all devices returned as part of AllocateResponse proto.
Signed-off-by: Vishnu kannan <vishnuk@google.com>
Automatic merge from submit-queue (batch tested with PRs 43661, 54062). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix typo in function name.
Also remove a superfluous comment.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 43661, 54062). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix#43583 (kubenet: remove code forcing bridge MAC address)
**What this PR does / why we need it**:
*kubenet: remove code forcing bridge MAC address*
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#43583
**Special notes for your reviewer**:
**Release note**:
```release-note
```
cc @dcbw @freehan
Automatic merge from submit-queue (batch tested with PRs 47717, 53896). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Delete the redundant parameter flag
What this PR does / why we need it:
Delete redundant parameter flag, otherwise the log will be show like:
Warning: path "/var/lib/kubelet/pods/3c6c4869-4d02-11e7-9685-fa163eeda0fa/volumes" does not exist: %!q(MISSING)
thank you!
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
pkg/api: extract Scheme/Registry/Codecs into pkg/api/legacyscheme
This serves as
- a preparation for the pkg/api->pkg/apis/core move
- and makes the dependency to the scheme explicit when vizualizing
left depenncies.
The later helps with our our efforts to split up the monolithic repo
into self-contained sub-repos, e.g. for kubectl, controller-manager
and kube-apiserver in the future.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
CRI: Add extra information in status functions in CRI.
Fixes https://github.com/kubernetes/kubernetes/issues/53757.
@yujuhong @feiskyer @mrunalp
/cc @kubernetes/sig-node-api-reviews
```release-note
Verbose option is added to each status function in CRI. Container runtime could return extra information in status response for debugging.
```
Automatic merge from submit-queue (batch tested with PRs 53696, 54059). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix lint warnings for useless err checks.
**What this PR does / why we need it**:
This check was recently added to golint.
**Which issue this PR fixes**
Related to #37254
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Increases test coverage for kubelet/kuberuntime
What this PR does / why we need it:
Increases test coverage for kubelet/kuberuntime
#46123
Which issue this PR fixes:
#46123
/assign @feiskyer
Automatic merge from submit-queue (batch tested with PRs 54040, 52503). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Get fallback termination msg from docker when using journald log driver
**What this PR does / why we need it**:
When using the legacy docker container runtime and when a container has `terminationMessagePolicy=FallbackToLogsOnError` and when docker is configured with a log driver other than `json-log` (such as `journald`), the kubelet should not try to get the container's log from the json log file (since it's not there) but should instead ask docker for the logs.
**Which issue this PR fixes** fixes#52502
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixed log fallback termination messages when using docker with journald log driver
```
Automatic merge from submit-queue (batch tested with PRs 54040, 52503). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
falls back to parse Docker runtime version as generic if not semver
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#54039
**Special notes for your reviewer**:
/assign @tallclair @vishh
**Release note**:
```release-note
falls back to parse Docker runtime version as generic if not semver
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Do not remove kubelet labels during startup
Fixes#54070
```release-note
kubelet: prevent removal of default labels from Node API objects on startup
```
More and more components checkpoints (i.e., persist their states) in
kubelet. Refurbish and move the implementation in dockershim to a
utility package to improve code reusability.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Clean up kublet secret and configmap unit test
**What this PR does / why we need it**:
These changes are clean-up items to fix confusing code encountered while investigating #52043. No actual bugs are fixed here (except, maybe, correcting unit tests that had actual/expected swapped).
A summary of the changes, as listed in the commit:
* Expected value comes before actual value in assert.Equal()
* Use `assert.Equal()` instead of `assert.True()` when possible
* Add a unit test that verifies no-op pod updates to the `secret_manager` and the `configmap_manager`
* Add a clarifying comment about why it's good to seemingly delete a secret on updates.
* Fix (for now, non-buggy) variable shadowing issue
**Special notes for your reviewer**:
N/A
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix typos: remove duplicated word in comments
**What this PR does / why we need it**: Remove the duplicated word `the` in comments
**Which issue this PR fixes** : fixes #
**Special notes for your reviewer**:
```release-note
NONE
```
Prevent a Kubelet from shutting down when the server isn't responding to
us but we cannot get a new certificate. This allows a cluster to coast
if the master is unresponsive or a node is partitioned and their client
cert expires.
Clients shouldn't have to know about watch.ErrWatchClosed, which is
typically a server side decision to close and always means "Timeout" in
this conetxt.
The client cert manager uses the most recent cert to request new
certificates. If that certificate is expired, it will be unable to
complete new CSR requests. This commit alters the manager to force
process exit if no further client cert rotation is possible, which
is expected to trigger a restart of the kubelet and either a
re-bootstrap from the bootstrap kubeconfig or a re-read of the
current disk state (assuming that some other agent is managing the
bootstrap configuration).
This prevents the Kubelet from wedging in a state where it cannot make
API calls.
Ensures that in a crash loop state we can make forward progress by
generating a new key and hence new CSR. If we do not delete the key, an
expired CSR may block startup.
Also more aggressively delete a bad cert path
Before the bootstrap client is used, check a number of conditions that
ensure it can be safely loaded by the server. If any of those conditions
are invalid, re-bootstrap the node. This is primarily to force
bootstrapping without human intervention when a certificate is expired,
but also handles partial file corruption.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix pkg/ depends on cmd/ problems
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Partial fix for https://github.com/kubernetes/kubernetes/issues/53341
**Special notes for your reviewer**:
No logic changes, Just moving things around
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52959, 53790). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
remove redundancy code in setCPUCgroupConfig
fix#53925
Signed-off-by: yanxuean <yan.xuean@zte.com.cn>
**What this PR does / why we need it**:
The check of burstableCPUShares is redundancy. We have done it in MilliCPUToShares. It is responsibility of MilliCPUToShares.
```
func (m *qosContainerManagerImpl) setCPUCgroupConfig(configs map[v1.PodQOSClass]*CgroupConfig) error {
........
// set burstable shares based on current observe state
burstableCPUShares := MilliCPUToShares(burstablePodCPURequest)
if burstableCPUShares < uint64(MinShares) {
burstableCPUShares = uint64(MinShares)
}
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Improveing code.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 51840, 53542, 53857, 53831, 53702). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet sync pod throws more detailed events
**What this PR does / why we need it**:
If there are errors in the kubelet sync pod iteration, it is difficult to determine the problem.
This provides more specific events for errors that occur in the syncPod iteration to help perform problem isolation.
Fixes https://github.com/kubernetes/kubernetes/issues/53900
**Special notes for your reviewer**:
It is safer to dispatch more specific events now that we have an event budget per object enforced via https://github.com/kubernetes/kubernetes/pull/47367
**Release note**:
```release-note
kubelet provides more specific events when unable to sync pod
```
Automatic merge from submit-queue (batch tested with PRs 51840, 53542, 53857, 53831, 53702). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Kubelet Evictions take Priority into account
Issue: https://github.com/kubernetes/kubernetes/issues/22212
This implements the eviction strategy documented here: https://github.com/kubernetes/community/pull/1162, and discussed here: https://github.com/kubernetes/community/pull/846.
When priority is not enabled, all pods are treated as equal priority.
This PR makes the following changes:
1. Changes the eviction ordering strategy to (usage < requests, priority, usage - requests)
2. Changes unit testing to account for this change in eviction strategy (including tests where priority is disabled).
3. Adds a node e2e test which tests the eviction ordering of pods with different priorities.
/assign @dchen1107 @vishh
cc @bsalamat @derekwaynecarr
```release-note
Kubelet evictions take pod priority into account
```
Automatic merge from submit-queue (batch tested with PRs 53119, 53753, 53795, 52981). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Reduce log spam in qos container manager
**What this PR does / why we need it**:
excessive log stmts make it hard to debug actual problems.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add feature gate for allocatable disk eviction
Issue: #52336
This PR adds the local storage feature gate to local storage allocatable eviction.
cc @kubernetes/sig-node-bugs
/assign @jingxu97 @dchen1107
we should target this for 1.7 if possible.
```release-note
fix a bug where disk pressure could trigger prematurely
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kubelet/cm: remove unneeded fork of 'cat'
Reading a file in Go is perfectly possible without invoking cat.
I also removed an outdated comment.
This is meant to be a trivial/minor code cleanup, nothing more.
```release-note
NONE
```
Revert "Merge pull request #51857 from kubernetes/revert-51307-kc-type-refactor"
This reverts commit 9d27d92420, reversing
changes made to 2e69d4e625.
See original: #51307
We punted this from 1.8 so it could go through an API review. The point
of this PR is that we are trying to stabilize the kubeletconfig API so
that we can move it out of alpha, and unblock features like Dynamic
Kubelet Config, Kubelet loading its initial config from a file instead
of flags, kubeadm and other install tools having a versioned API to rely
on, etc.
We shouldn't rev the version without both removing all the deprecated
junk from the KubeletConfiguration struct, and without (at least
temporarily) removing all of the fields that have "Experimental" in
their names. It wouldn't make sense to lock in to deprecated fields.
"Experimental" fields can be audited on a 1-by-1 basis after this PR,
and if found to be stable (or sufficiently alpha-gated), can be restored
to the KubeletConfiguration without the "Experimental" prefix.
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Make feature gates loadable from a map[string]bool
Command line flag API remains the same. This allows ComponentConfig
structures (e.g. KubeletConfiguration) to express the map structure
behind feature gates in a natural way when written as JSON or YAML.
For example:
KubeletConfiguration Before:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates: "DynamicKubeletConfig=true,Accelerators=true"
```
KubeletConfiguration After:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates:
DynamicKubeletConfig: true
Accelerators: true
```
Fixes: #53024
```release-note
The Kubelet's feature gates are now specified as a map when provided via a JSON or YAML KubeletConfiguration, rather than as a string of key-value pairs.
```
/cc @mikedanese @jlowdermilk @smarterclayton
Automatic merge from submit-queue (batch tested with PRs 53297, 53328). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Cpu Manager - make CoreID's platform unique
**What this PR does / why we need it**:
Cpu Manager uses topology from cAdvisor(`/proc/cpuinfo`) where coreID's are socket unique - not platform unique - this causes problems on multi-socket platforms.
All code assumes unique coreID's (on platform) - `Discovery` function has been changed to assign CoreID as the lowest cpuID from all cpus belonging to the same core. This can be expressed as:
`CoreID=min(cpuID's on the same core)`
Since cpuID's are platform unique - above gives us guarantee that CoreID's will also be platform unique.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#53323
Automatic merge from submit-queue (batch tested with PRs 53297, 53328). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
format some code in dockershim
**What this PR does / why we need it**:
format some code in dockershim
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Command line flag API remains the same. This allows ComponentConfig
structures (e.g. KubeletConfiguration) to express the map structure
behind feature gates in a natural way when written as JSON or YAML.
For example:
KubeletConfiguration Before:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates: "DynamicKubeletConfig=true,Accelerators=true"
```
KubeletConfiguration After:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates:
DynamicKubeletConfig: true
Accelerators: true
```
Automatic merge from submit-queue (batch tested with PRs 53525, 53652). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apimachinery: remove ObjectCopier interface(s)
The big commit is a mechanical, transitive removal of the copier interfaces in all structs and function calls.
Automatic merge from submit-queue (batch tested with PRs 53444, 52067, 53571, 53182). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Don't skip mounts if we can't find the volume
**What this PR does / why we need it**:
Return an error instead of skipping the volume while constructing the list of volume mounts for the container runtime. This prevents the scenario of a container writing data to an ephemeral volume when it expects the volume to be persistent.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#53421
**Release note**:
NONE
@kubernetes/sig-storage-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 52662, 53547, 53588, 53573, 53599). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
In DevicePluginHandlerImpl.Allocate(), skips untracked extended resou…
…rces.
Otherwise, we would fail a Pod allocation request that has an extended
resource not managed by any device plugin.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/53548
**Special notes for your reviewer**:
**Release note**:
```release-note
Ignore extended resources that are not registered with kubelet
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor nsenter
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#51273
**Special notes for your reviewer**:
/assign @jsafrane
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 53044, 52956, 53512, 53028). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes the flaky TestDevicePluginReRegistration.
In the current test, there is a race that the new device plugin endpoint
may not be added to the device plugin manager endpoints at the time when
we call manager.Devices(). Added the checking and waiting for endpoint
updates before calling manager.Devices() in the test.
Tested:
go test -race -count 500 k8s.io/kubernetes/pkg/kubelet/deviceplugin -run
TestDevicePluginReRegistration -timeout 5h
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/52560
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Kubernetes Submit Queue