Automatic merge from submit-queue (batch tested with PRs 60054, 60202, 60219, 58090, 60275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Disable mount propagation for windows containers
**What this PR does / why we need it**:
Windows containers don't support mount propagation. This PR disables it for windows containers.
Without this PR, windows containers creation would fail with error:
Error: Error response from daemon: invalid bind mount spec "c:\\var\\lib\\kubelet\\pods\\a260a7c4-1852-11e8-bb1d-000d3a19c1da\\volumes\\kubernetes.io~secret\\default-token-rj7qv:c:/var/run/secrets/kubernetes.io/serviceaccount:ro,rslave": invalid volume specification: 'c:\var\lib\kubelet\pods\a260a7c4-1852-11e8-bb1d-000d3a19c1da\volumes\kubernetes.io~secret\default-token-rj7qv:c:\var\run\secrets\kubernetes.io\serviceaccount:ro,rslave'
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60274
**Special notes for your reviewer**:
**Release note**:
```release-note
Disable mount propagation for windows containers.
```
Automatic merge from submit-queue (batch tested with PRs 60054, 60202, 60219, 58090, 60275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Pass location parameter to event exporter.
**What this PR does / why we need it**:
This PR makes event-exporter export cluster location together with events.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60054, 60202, 60219, 58090, 60275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Improves backoff policy in JobController
**What this PR does / why we need it**:
This PR is fixing the issue: #56853, It improves the "Job backoff policy" when Job is configure to allow parallelism and few pods' Jobs failed but others succeed.
Now, it checks if the number of pods succeeded increased since the last check. If yes the backoff delay is cleared.
**Which issue(s) this PR fixes**:
Fixes#56853
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60054, 60202, 60219, 58090, 60275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fixes for HTTP/2 max streams per connection setting
**What this PR does / why we need it**:
This PR makes two changes. One is to introduce a parameter
for the HTTP/2 setting that an api-server sends to its clients
telling them how many streams they may have concurrently open in
an HTTP/2 connection. If left at its default value of zero,
this means to use the default in golang's HTTP/2 code (which
is currently 250; see https://github.com/golang/net/blob/master/http2/server.go).
The other change is to make the recommended options for an aggregated
api-server set this limit to 1000. The limit of 250 is annoyingly low
for the use case of many controllers watching objects of Kinds served
by an aggregated api-server reached through the main api-server (in
its mode as a proxy for the aggregated api-server, in which it uses a
single HTTP/2 connection for all calls proxied to that aggregated
api-server).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60042
**Special notes for your reviewer**:
**Release note**:
```release-note
Introduced `--http2-max-streams-per-connection` command line flag on api-servers and set default to 1000 for aggregated API servers.
```
Automatic merge from submit-queue (batch tested with PRs 57672, 60299, 59757, 60283, 60265). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add description of MountOptions to StorageClass describe printer
**What this PR does / why we need it**:
Print `StorageClass.MountOptions` for `kubectl describe` command.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57672, 60299, 59757, 60283, 60265). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
initialize all known client auth plugins
**What this PR does / why we need it**:
`k8s.io/client-go/plugin/pkg/client/auth/plugins.go` has already initialized all known client auth plugins.
ec77ddfe19/staging/src/k8s.io/client-go/plugin/pkg/client/auth/plugins.go (L19-L25)
We just need import `k8s.io/client-go/plugin/pkg/client/auth` instead.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 57672, 60299, 59757, 60283, 60265). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add a metric exposing number of objects per type
Fix#51953
Adds a goroutine that periodically checks the count of objects in etcd and publishes a metric with this data.
```release-note
APIserver backed by etcdv3 exports metric showing number of resources per kind
```
Automatic merge from submit-queue (batch tested with PRs 57672, 60299, 59757, 60283, 60265). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apiserver: fix testing etcd config in preparation for etcd 3.2.16+
The AuthToken must be set, also in non-https mode. Otherwise, etcd refuses to start.
Automatic merge from submit-queue (batch tested with PRs 57672, 60299, 59757, 60283, 60265). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
[kube-proxy]enhance kubeproxy init flag
**What this PR does / why we need it**:
remove `TODO: once we switch everything over to Cobra commands, we can go back to calling
utilflag.InitFlags() (by removing its pflag.Parse() call). For now, we have to set the
normalize func and add the go flag set by hand.`
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59286, 59743, 59883, 60190, 60165). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Updating GCE PD StorageClass parameters
**What this PR does / why we need it**: New parameter to improve support of multi-zone PDs.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: partially addresses #59988
/cc @msau42 @saad-ali
Automatic merge from submit-queue (batch tested with PRs 59286, 59743, 59883, 60190, 60165). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
audit/request.go remove inaccurate TODO and fix typo
**What this PR does / why we need it**:
remove one inaccurate `TODO` and fix a typo
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 59286, 59743, 59883, 60190, 60165). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
DaemonSet Controller and tests to apps/v1
**What this PR does / why we need it**:
Updates the DaemonSet controller, its integration tests, and its e2e tests to use the apps/v1 API.
**Release note**:
```release-note
The DaemonSet controller, its integration tests, and its e2e tests, have been updated to use the apps/v1 API.
```
Automatic merge from submit-queue (batch tested with PRs 59286, 59743, 59883, 60190, 60165). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix image file system stats for windows nodes
**What this PR does / why we need it**:
Kubelet is reporting `invalid capacity 0 on image filesystem` on windows nodes and image GC always fails.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59742
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix image file system stats for windows nodes
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Delete stale UDP conntrack entries that use hostPort
**What this PR does / why we need it**:
This PR introduces a change to delete stale conntrack entries for UDP connections, specifically for udp connections that use hostPort. When the pod listening on that udp port get updated/restarted(and gets a new ip address), these entries need to be flushed so that ongoing udp connections can recover once the pod is back and the new iptables rules have been installed.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59033
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
cmd/controller-manager: add OWNERS for generic controller-manager code
This should have the same owners as kube-controller-manager.
Automatic merge from submit-queue (batch tested with PRs 60302, 57921, 59042, 60126, 59561). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Allow to specify categories for custom resources
Allow to specify categories for custom resources so that we can get it working with `kubectl get all`.
Adds a new field `Categories` in the CRD spec.
**Release note**:
```release-note
Custom resources can be listed with a set of grouped resources (category) by specifying the categories in the CustomResourceDefinition spec. Example: They can be used with `kubectl get all`, where `all` is a category.
```
/cc sttts liggitt deads2k
Automatic merge from submit-queue (batch tested with PRs 60302, 57921, 59042, 60126, 59561). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove pkg/client/unversioned
**What this PR does / why we need it**:
This is removing unused package, and moves the used bits into appropriate placeholders.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#25442
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
/assign @deads2k
/assign @sttts
Automatic merge from submit-queue (batch tested with PRs 60302, 57921, 59042, 60126, 59561). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Removing Flexvolume feature tag in e2e tests because Flexvolume is now GA
**What this PR does / why we need it**: Flexvolume e2e tests should be run in serial test suite after the feature was marked GA.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57466
/release-note-none
/sig storage
/cc @wongma7
Automatic merge from submit-queue (batch tested with PRs 60302, 57921, 59042, 60126, 59561). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Revert "Allow env to be updated via specific key in resource"
This introduced an unstable test that is failing in our queue.
/assign @soltysh
I'm trying to find a real fix, but let's get the revert tested and ready.
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Critical pod priorityClass addition
**What this PR does / why we need it**:
@bsalamat - Apologies for the delay. This PR is to ensure that all pods with priorityClassName `system-node-critical` and `system-cluster-critical` will be critical pods while preserving backwards compatibility.
**Special notes for your reviewer**:
- Moved some constants and other data structures to scheduler/api/types.go where other constants are present.
- An automatic assignment of critical priorities to pods based on critical pod annotation for backwards compatibility including some unit tests.
xref: https://github.com/kubernetes/kubernetes/issues/57471
**Release note**:
```release-note
Critical pods to use priorityClasses.
```
Automatic merge from submit-queue (batch tested with PRs 55637, 57461, 60268, 60290, 60210). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
deprecate --show-all
`--show-all` is a pod-only filter that only affects human-readable printing of pods and only from `kubectl get`. It hides pods which are in a terminal state. Even at the beginning this was questionable, since you often (usually?) want to see the pods that have failed: all happy pods are alike, but every unhappy pod is unhappy in its own way. In addition, it only worked on human-readable printers. Doing a `-o name` or `-o yaml` showed a different set of results!
Per the mailing list discussion here: https://groups.google.com/forum/#!topic/kubernetes-sig-cli/0SxgDxObxD0
```release-note
`--show-all` (which only affected pods and only for human readable/non-API printers) is now defaulted to true and deprecated. It will be inert in 1.11 and removed in a future release.
```
/assign @adohe
/assign @pwittrock
/assign @soltysh
@kubernetes/sig-cli-maintainers
Automatic merge from submit-queue (batch tested with PRs 55637, 57461, 60268, 60290, 60210). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Autoscaler e2e - fix getting initial pool size
**What this PR does / why we need it**:
Fixes gcloud commands when getting initial pool size in e2e autoscaler tests. Adds logging to make test easier to debug.
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55637, 57461, 60268, 60290, 60210). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
kms: rename KMSService to KeyManagementService
KMSService is redundant. Introduced in #55684
@kubernetes/sig-auth-api-reviews
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55637, 57461, 60268, 60290, 60210). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Don't create no-op iptables rules for services with no endpoints
Currently for all services we create `-t nat -A KUBE-SERVICES` rules that match the destination IPs (ClusterIP, ExternalIP, NodePort IPs, etc) and then jump to the appropriate `KUBE-SVC-XXXXXX` chain. But if the service has no endpoints then the `KUBE-SVC-XXXXXX` chain will be empty and so nothing happens except that we wasted time (a) forcing iptables-restore to parse the match rules, and (b) forcing the kernel to test matches that aren't going to have any effect.
This PR gets rid of the match rules in this case. Which is to say, it changes things so that every incoming service packet is matched *either* by nat rules to rewrite it *or* by filter rules to ICMP reject it, but not both. (Actually, that's not quite true: there are no filter rules to reject Ingress-addressed packets, and I *think* that's a bug?)
I also got rid of some comments that seemed redundant.
The patch is mostly reindentation, so best viewed with `diff -w`.
Partial fix for #56842 / Related to #56164 (which it conflicts with but I'll fix that after one or the other merges).
**Release note**:
```release-note
Removed some redundant rules created by the iptables proxier, to improve performance on systems with very many services.
```
Automatic merge from submit-queue (batch tested with PRs 55637, 57461, 60268, 60290, 60210). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update get.go to use server-side printing
Addresses part of https://github.com/kubernetes/kubernetes/issues/58536
Adds support for server-side changes implemented in https://github.com/kubernetes/kubernetes/pull/40848 and updated in https://github.com/kubernetes/kubernetes/pull/59059
@deads2k per our discussion, opening this as a separate PR.
This wires through a per-request use of `as=Table;...` header parameters
using the resource builder from the `kube get` command.
#### Items to consider going forward:
- [ ] Figure out how to handle sorting when dealing with multiple Table objects from the server
- [ ] Figure out sorting when dealing with a mixed response from the server consisting of Tables and normal resources (`--sort-by` is handled in this PR by falling back to old behavior)
- [ ] Filtering: How should we filter Table objects? Separate filter for rows? Filter on jsonpath? We have access to partial object metadata for each table row - not enough to know how to filter pods, for example but we could request that the original object be included along with each Table.Row by adding an `includeObject` param in the client request.
#### Resources that do not yet support Table output
- [ ] Namespaces
- [ ] Services
- [ ] Service catalog resources: https://github.com/kubernetes-incubator/service-catalog/blob/master/pkg/apis/servicecatalog/v1beta1/types.go
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
PSP plugin: allow authorizing via "use" verb in policy API group
**What this PR does / why we need it**:
In order to determine whether a service account/user has access to PSP, PodSecurityPolicy admission plugin tests whether a service account/user is authorized for "use" verb in `extensions` API group. As PSP is being migrated to `policy` API group, we need to support its new location. This PR adds such a support by checking in both API groups.
**Which issue(s) this PR fixes**:
Addressed to: https://github.com/kubernetes/features/issues/5
Follow-up to: https://github.com/kubernetes/kubernetes/pull/54933
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
simplify kubectl testing factory
This simplifies and strips down the fake factory and collapses us down to a single fake!
@kubernetes/sig-cli-maintainers @juanvallejo
/assign @pwittrock
/assign @soltysh
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 57550, 60089). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove subnet size restriction for IPv6
RangeSize was restricting IPv6 subnets to a /66 due to the
logic using a uint64. This is not practical for IPv6.
This change removes the /64 restriction, but also sets a limit
on the range that can be allocated, so that the bitmap will not grow too large.
**What this PR does / why we need it**:
This PR removes the /66 restriction in ipallocator for IPv6. It is not practical to restrict
IPv6 to /66. Currently a /64 or /48 is not allowed. The problem with removing the restriction is
the bitmap that tracks the subnets can grow really large, so a limit
on the max size of the subnet was set to 65536.
Setting the max size will have a side-effect with larger subnets that the allocator
will allocate in a smaller section of IP's, this will need to be addressed in a follow-on PR.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60081
**Special notes for your reviewer**:
**Release note**:
```release-note-none
```
Automatic merge from submit-queue (batch tested with PRs 60106, 59510, 60263, 60063, 59088). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
CodeClean, merge Logf And FailNow to Fatalf
**What this PR does / why we need it**:
Trivial changes to clean code, merge Logf And FailNow to Fatalf.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
"NONE"
```
Automatic merge from submit-queue (batch tested with PRs 60106, 59510, 60263, 60063, 59088). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
clean up KubeletConfigOk condition construction
This PR cleans up the construction of the node condition and also fixes
a small bug where the last transition time could be updated incorrectly
when the sync failure overlay was present.
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 60106, 59510, 60263, 60063, 59088). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Reuse the `min*Nodes` slices in order to save GC time
**What this PR does / why we need it**:
Reuse the `min*Nodes` slices to save GC time when executing `pickOneNodeForPreemption`.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59748
**Special notes for your reviewer**:
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 60106, 59510, 60263, 60063, 59088). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Refactor service storage to remove registry wrapper
This exposes the correct table exporter to the API endpoint, which is a prereq for server side GET to beta. Removing the use of the registry simplifies a few complex changes but results in test abstractions changing.
Part of #58536
Automatic merge from submit-queue (batch tested with PRs 60106, 59510, 60263, 60063, 59088). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update cadvisor godeps to v0.29.0 and ignore per-cpu metrics
**What this PR does / why we need it**:
Updates the cAdvisor dependency to the cAdvisor release associated with the kubernetes 1.10 release.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#60052
**Special notes for your reviewer**:
This PR also adds per-cpu metrics to the ignoreMetrics list. This is a new metric that can be ignored in the most recent cAdvisor release.
The reason for not collecting per-cpu metrics is that it can cause severe scalability issues.
For example, if using a 128 core machine, and running 100 containers, we have 12800 different streams of metrics just for per-cpu metrics which cAdvisor needs to process and transmit.
Additionally, per-cpu metrics are not used by any kubernetes components, and if a user needs these metrics, they can run cAdvisor as a daemonset.
**Release note**:
```release-note
Disable per-cpu metrics by default for scalability.
Fix inaccurate disk usage monitoring of overlayFs.
Retry docker connection on startup timeout to avoid permanent loss of metrics.
```
/assign @dchen1107
Kubernetes Submit Queue