Automatic merge from submit-queue
CRI: add Message in ContainerStatus
Kubernetes expects a brief CamelCase string as "Reason" and a human-readable
string as "Message" in the conatiner status. It is difficult for kubelet to
derive the other one if the runtime only provides one of the two fields.
Augment the API by adding the "Message" field.
/cc @kubernetes/sig-node
Kubernetes expects a brief CamelCase string as "Reason" and a human-readable
string as "Message" in the conatiner status. It is difficult for kubelet to
derive the other one if the runtime only provides one of the two fields.
Augment the API by adding the "Message" field.
Automatic merge from submit-queue
Add positive logging for GC events
We have no positive logging for GC events. This PR:
1. Adds positive logging at V(4) for success cases
2. Adds positive logging at V(1) for the first successful GC after a failure
Automatic merge from submit-queue
Fake container exec/logs support for in-process docker CRI integration
This is necessary to unblock other work on docker integration, while we are addressing
`logs` and `exec` in the meantime.
This is part of #31459 and #33189
/cc @kubernetes/sig-node
This is a temporary hack to bypass CRI when getting container logs or
running exec in a container. This is necessary to unblock testing and adding
other features in the integration.
Automatic merge from submit-queue
Add port forwarding for rkt with kvm stage1
Port forwarding for rkt kvm using `socat`.
cc @yifan-gu @euank @pskrzyns @lukasredynk
Automatic merge from submit-queue
Move image pull throttling logic to pkg/kubelet/images
This is part of #31458
This allows runtimes in different packages (dockertools, rkt, kuberuntime) to
share the same logic. Before this change, only dockertools support this
feature. Now all three packages support image pull throttling.
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Refactor cert utils into one pkg, add funcs from bootkube for kubeadm to use
**What this PR does / why we need it**:
We have ended-up with rather incomplete and fragmented collection of utils for handling certificates. It may be worse to consider using `cfssl` for doing all of these things, but for now there is some functionality that we need in `kubeadm` that we can borrow from bootkube. It makes sense to move the utils from bookube into core, as discussed in #31221.
**Special notes for your reviewer**: I've taken the opportunity to review names of existing funcs and tried to make some improvements in that area (with help from @peterbourgon).
**Release note**:
```release-note
NONE
```
This allows runtimes in different packages (dockertools, rkt, kuberuntime) to
share the same logic. Before this change, only dockertools support this
feature. Now all three packages support image pull throttling.
Automatic merge from submit-queue
simplify RC and SVC listers
Make the RC and SVC listers use the common list functions that more closely match client APIs, are consistent with other listers, and avoid unnecessary copies.
Automatic merge from submit-queue
kubenet: SyncHostports for both running and ready to run pods.
This fixes the race that happens in rktnetes when pod B invokes
'kubenet.SetUpPod()' before another pod A becomes actually running.
The second 'kubenet.SetUpPod()' call will not pick up the pod A
and thus overwrite the host port iptable rules that breaks pod A.
This PR fixes the case by listing all 'active pods' (all non-exited
pods) instead of only running pods.
Fix https://github.com/kubernetes/kubernetes/issues/27975
Originally discussed in https://github.com/kubernetes/kubernetes/pull/27914#issuecomment-228140108
cc @euank @freehan @dcbw
Automatic merge from submit-queue
CRI: Add missing sandbox in runningPod.
Add a container that represents pod sandbox when converting the
pod status to runningPod.
Without the change, `kubeGenericRuntimeManager.killPodWithSyncResult()` will not kill any sandboxes.
cc @feiskyer @Random-Liu @yujuhong
The new flag, if specified, and if --container-runtime=docker, switches
kubelet to use the new CRI implementation for testing. This is hidden flag
since the feature is still under heavy development and the flag may be changed
in the near future.
This fixes the race that happens in rktnetes when pod B invokes
'kubenet.SetUpPod()' before another pod A becomes actually running.
The second 'kubenet.SetUpPod()' call will not pick up the pod A
and thus overwrite the host port iptable rules that breaks pod A.
This PR fixes the case by listing all 'active pods' (all non-exited
pods) instead of only running pods.
Automatic merge from submit-queue
Disambiguate unsupported metrics from metrics errors
**What this PR does / why we need it**:
Stop logging "metrics are not supported for MetricsNil Volumes" as it spams the log.
**Which issue this PR fixes**
fixes#20676, fixes#27373
**Special notes for your reviewer**:
None
**Release note**:
```release-note
Don't log "metrics are not supported for MetricsNil Volumes"
```
Automatic merge from submit-queue
Kubelet: add SyncPod for new runtime API
This PR adds implements of `SyncPod` for new runtime API. Note that init containers is not included, it will in another following PR.
**DO-NOT-MERGE**. Based on #31322 and #31395, only last commit is for review.
CC @yujuhong @Random-Liu and @kubernetes/sig-node @kubernetes/sig-rktnetes
Automatic merge from submit-queue
Check kubeClient nil in Kubelet and bugfix
1. check kubeClient nil first before using as it maybe nil
2. configMaps and secrets map do not be used properly and should use it as cache
Automatic merge from submit-queue
Kubelet: add garbage collection for new runtime API
This PR adds garbage collection for new runtime API.
Note that this PR also adds `CreatedAt` and `PodSandboxId` to `ListContainers()` result.
CC @yujuhong @Random-Liu @kubernetes/sig-node @kubernetes/sig-rktnetes
Docker 1.10 does not guarantee that the pulled digest matches the digest
on disk when dealing with v1 schemas stored in a Docker registry. This
is the case for images like
centos/ruby-23-centos7@sha256:940584acbbfb0347272112d2eb95574625c0c60b4e2fdadb139de5859cf754bf
which as a result of #30366 cannot be pulled by Kube from a Docker 1.10
system.
Instead, use RepoDigests field as the primary match, validating the
digest, and then fall back to ID (also validating the match). Adds more
restrictive matching.
Automatic merge from submit-queue
Kubelet: remove name filter from CRI
Since #30753 and #30463, `name` is not used to identify the container/sandbox, so remove it from CRI.
cc @yujuhong @kubernetes/sig-node @kubernetes/sig-rktnetes
Automatic merge from submit-queue
Add flag to set CNI bin dir, and use it on gci nodes
**What this PR does / why we need it**:
When using `kube-up` on GCE, following #31023 which moved the workers from debian to gci, CNI just isn't working. The root cause is basically as discussed in #28563: one flag (`--network-plugin-dir`) means two different things, and the `configure-helper` script uses it for the wrong purpose.
This PR adds a new flag `--cni-bin-dir`, then uses it to configure CNI as desired.
As discussed at #28563, I have also added a flag `--cni-conf-dir` so users can be explicit
**Which issue this PR fixes** : fixes#28563
**Special notes for your reviewer**:
I left the old flag largely alone for backwards-compatibility, with the exception that I stop setting the default when CNI is in use. The value of `"/usr/libexec/kubernetes/kubelet-plugins/net/exec/"` is unlikely to be what is wanted there.
**Release note**:
```release-note
Added new kubelet flags `--cni-bin-dir` and `--cni-conf-dir` to specify where CNI files are located.
Fixed CNI configuration on GCI platform when using CNI.
```
Automatic merge from submit-queue
Improvements on OpenAPI spec generation
- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation
Reference: #13414
**Release note**:
```release-note
Generate separate OpenAPI spec for each API GroupVersion on /<Group>/<Version>/swagger.json
```
Automatic merge from submit-queue
Revert "Merge pull request #30513 from tmrts/kubelet-rkt-cri/use-imag…
This reverts commit aff7dfcaab, reversing
changes made to 7a4d81ea43.
See https://github.com/kubernetes/kubernetes/pull/30513#issuecomment-245949664, this breaks tests for the rkt runtime.
The original PR also changed the source of image truth from the api-service to cli, which could have further implications which @yifan-gu could speak better to, so I think it's safer to just revert for now and discuss further in the rktlet repo / pr. The reverted code effectively already exists (with the bug in question already fixed) here https://github.com/kubernetes-incubator/rktlet/pull/5 .. once that's merged, we can vendor and call it over here in place of #30513.
@yifan-gu / @tmrts if you think continuing with the changes and rolling forwards (just adding the `--full` flag to list to fix the immediate bug) is better, feel free to close this and open a PR with that change, but I think this approach is overall better for the reasons in the previous paragraph.
cc @tmrts @yifan-gu @kubernetes/sig-rktnetes @pskrzyns
- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation
Automatic merge from submit-queue
Fix race condition in updating attached volume between master and node
This PR tries to fix issue #29324. The cause of this issue is that a race
condition happens when marking volumes as attached for node status. This
PR tries to clean up the logic of when and where to mark volumes as
attached/detached. Basically the workflow as follows,
1. When volume is attached sucessfully, the volume and node info is
added into nodesToUpdateStatusFor to mark the volume as attached to the
node.
2. When detach request comes in, it will check whether it is safe to
detach now. If the check passes, remove the volume from volumesToReportAsAttached
to indicate the volume is no longer considered as attached now.
Afterwards, reconciler tries to update node status and trigger detach
operation. If any of these operation fails, the volume is added back to
the volumesToReportAsAttached list showing that it is still attached.
These steps should make sure that kubelet get the right (might be
outdated) information about which volume is attached or not. It also
garantees that if detach operation is pending, kubelet should not
trigger any mount operations.
This PR tries to fix issue #29324. This cause of this issue is a race
condition happens when marking volumes as attached for node status. This
PR tries to clean up the logic of when and where to mark volumes as
attached/detached. Basically the workflow as follows,
1. When volume is attached sucessfully, the volume and node info is
added into nodesToUpdateStatusFor to mark the volume as attached to the
node.
2. When detach request comes in, it will check whether it is safe to
detach now. If the check passes, remove the volume from volumesToReportAsAttached
to indicate the volume is no longer considered as attached now.
Afterwards, reconciler tries to update node status and trigger detach
operation. If any of these operation fails, the volume is added back to
the volumesToReportAsAttached list showing that it is still attached.
These steps should make sure that kubelet get the right (might be
outdated) information about which volume is attached or not. It also
garantees that if detach operation is pending, kubelet should not
trigger any mount operations.
Automatic merge from submit-queue
Fixed TODO: move predicate check into a pod admitter
refractoring AdmitPod func to move predicate check into a pod admitter
Automatic merge from submit-queue
Redundant code process for container_mananger start
1. need not sum the total numEnsureStateFuncs
2. numEnsureStateFuncs should > 0, otherwise, calculate numEnsureStateFuncs would be not neccessary
Automatic merge from submit-queue
Move CSR helper for nodes out of kubelet
**What this PR does / why we need it**:
Including `cmd/kubelet/app` in kubeadm causes flag leakage.
Namelly, the problem is with `pkg/credentialprovider/gcp`, which
leaks `--google-json-key` and changing the behaviour of `init()`
doesn't sound reasonable, given kubelet is the only one who uses
this packages and obviously the flag is part of the functionality.
The helper is already generic enough, it has already been exported
and works well for kubeadm, so moving it should be fine.
**Special notes for your reviewer**: cc @mikedanese @yifan-gu @gtank
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Kubelet: add KillPod for new runtime API
This PR adds implements of KillPod for new runtime API.
CC @yujuhong @Random-Liu @kubernetes/sig-node @kubernetes/sig-rktnetes
Automatic merge from submit-queue
Kubelet: rename CreatePodSandbox to RunPodSandbox in CRI
As @yifan-gu pointed out in #31847, the name `CreatePodSandbox` doesn't reflect that the sandbox is running after the API succeeds. This PR renames it to `RunPodSandbox` to make this clear.
CC @yujuhong @yifan-gu @kubernetes/sig-node
Automatic merge from submit-queue
Kubelet: implement GetPodContainerID for new runtime API
Add implements of `GetPodContainerID` interface for new runtime API.
CC @yujuhong @kubernetes/sig-node @kubernetes/sig-rktnetes
Automatic merge from submit-queue
correct imagefs inodes value in kubelet summary stats
Fix https://github.com/kubernetes/kubernetes/issues/31501
Correct get imagefs inodes value from imageFsInfo.Inodes in kubelet summary stats api.
@derekwaynecarr
Automatic merge from submit-queue
Avoid unnecessary status update when there is no corresponding mirror pod
Fixes https://github.com/kubernetes/kubernetes/issues/32191.
This PR changes status manager to skip update when there is no mirror pod for a static pod.
We need this because:
1) When static pod terminates and mirror pod is deleted, this will avoid extra `syncPod`.
2) During mirror pod creation and recreation, this will avoid unnecessary `syncPod`.
Mark P1 to match the original issue.
@wojtek-t @yujuhong
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Instruct PLEG to detect pod sandbox state changes
This PR adds a Sandboxes list in `kubecontainer.Pod`, so that PLEG can check
sandbox changes using `GetPods()` . The sandboxes are treated as regular
containers (type `kubecontainer.Container`) for now to avoid additional
changes in PLEG.
/cc @feiskyer @yifan-gu @euank
Including `cmd/kubelet/app` in kubeadm causes flag leakage.
Namelly, the problem is with `pkg/credentialprovider/gcp`, which
leaks `--google-json-key` and changing the behaviour of `init()`
doesn't sound reasonable, given kubelet is the only one who uses
this packages and obviously the flag is part of the functionality.
The helper is already generic enough, it has already been exported
and works well for kubeadm, so moving it should be fine.
Automatic merge from submit-queue
Rename ConnectToDockerOrDie to CreateDockerClientOrDie
This function does not actually attempt to connect to the docker daemon, it just creates a client object that can be used to do so later. The old name was confusing, as it implied that a failure to touch the docker daemon could cause program termination (rather than just a failure to create the client).
Automatic merge from submit-queue
Log an event when container runtime exceeds grace-period during eviction
While debugging flakes in eviction, I encountered scenarios where the container run-time did not evict a pod within the allowed grace period. This could result in situations where a BE pod would not get killed fast enough and therefore a Bu pod was killed next (assuming there were no other BE pods)
/cc @mtaufen @vishh
Automatic merge from submit-queue
Kubelet: implement GetPodStatus for new runtime API
Implement `GetPodStatus()` for new runtime API. Part of #28789 .
CC @yujuhong @Random-Liu @dchen1107
Automatic merge from submit-queue
rkt: Update kube-up rkt version to v1.14.0
cc @kubernetes/sig-rktnetes
This should have been included in #31286 (whoops).
This is a bugfix that I propose for v1.4 inclusion.
Automatic merge from submit-queue
Remove net.ipv4.tcp_max_syn_backlog from sysctl whitelist
Remove `net.ipv4.tcp_max_syn_backlog` from sysctl whitelist. This is not namespaced in today's kernels, but must be set on node-level.
Having this on the whitelist, wouldn't harm because the kernel only offers namespaced `net.*` sysctls in the `/proc/sys` tree. But having a sysctl on the whitelist, which cannot be used, doesn't make sense either.
#### 1.4 justification:
- Risk: the whitelist is a published API. We shouldn't have sysctls on there which do not work.
- Rollback: nothing should depend on this behavior.
- Cost: the cost of this is relatively low, as no pod with this sysctl will launch.
Automatic merge from submit-queue
kubelet_test.go: use assertions
Switch most of the tests in this file to using the assert library
(`github.com/stretchr/testify/assert`) in the tests for better readability and
less code in general.
Automatic merge from submit-queue
Update node status instead of node in kubelet
#31730 added code for the Kubelet to reconcile the existing and new nodes in order to annotate existing nodes with the annotation for controller-managed attach-detach. However, it used `Update` instead of `UpdateStatus`, which changes the operations the node's token needs to be permitted to use. Using `UpdateStatus` is functionally equivalent and maintains the same set of permissions nodes need to have today.
I'm adding this with the 1.4 milestone because it is a follow-on to a 1.4 PR and fixes a downstream bug (which won't surface to Kube).
Switch most of the tests in this file to using the assert/require library
(in `github.com/stretchr/testify`) in the tests for better readability and
less code in general.
Automatic merge from submit-queue
Make it possible to enable controller-managed attach-detach on existing nodes
Fixes#31673. Now, if a node already exists with the given name on Kubelet startup, the Kubelet will reconcile the value of the controller-managed-attach-detach annotation so that existing nodes can have this feature turned on and off by changing the Kubelet configuration.
cc @kubernetes/sig-storage @kubernetes/rh-cluster-infra
PLEG will treat them as if they are regular containers and detect changes the
same manner. Note that this makes an assumption that container IDs will not
collide with the podsandbox IDs.
Automatic merge from submit-queue
Include security options in the container created event
New container creation events look like:
```
Created container with docker id /k8s_bar2.a4; Security:[seccomp=sub/subtest(md5:07c9bcb4db631f7ca191d6e0bca49f76)]
Created container with docker id /k8s_bar2.a4; Security:[seccomp=unconfined apparmor=foo-profile]
```
The goal is to provide enough information to confirm that the requseted security constraints were honored.
For https://github.com/kubernetes/kubernetes/issues/31284
/cc @dchen1107 @thockin @jfrazelle @pweil- @pmorie
---
Justification for v1.4:
- Risk: low. This appends some additional information to a human readable message. A bug here would probably not break any functionality
- Roll-back: I don't anticipate any more changes to this area of the code. No functionality depends on this change.
- Cost of not including: Users don't get any (positive) confirmation that the AppArmor or Seccomp profile they requested were actually enabled.
Automatic merge from submit-queue
Add log message in Kubelet when controller attach/detach is enabled
Adds a message to the Kubelet log indicating whether controller attach/detach is enabled for a node.
cc @kubernetes/sig-storage
Kubernetes Submit Queue