github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
k3s/staging
History
Kubernetes Submit Queue 870406bec5 Merge pull request #47367 from derekwaynecarr/event-spam 
Automatic merge from submit-queue

Add client side event spam filtering

**What this PR does / why we need it**:
Add client side event spam filtering to stop excessive traffic to api-server from internal cluster components.

this pr defines a per source+object event budget of 25 burst with refill of 1 every 5 minutes.

i tested this pr on the following scenarios:

**Scenario 1: Node with 50 crash-looping pods**
```
$ create 50 crash-looping pods on a single node
$ kubectl run bad --image=busybox --replicas=50 --command -- derekisbad
```
Before:
* POST events with peak of 1.7 per second, long-tail: 0.2 per second
* PATCH events with peak of 5 per second, long-tail: 5 per second

After:
* POST events with peak of 1.7 per second, long-tail: 0.2 per second
* PATCH events with peak of 3.6 per second, long-tail: 0.2 per second

Observation:
* https://github.com/kubernetes/kubernetes/pull/47462 capped the number of total events in the long-tail as expected, but did nothing to improve total spam of master.

**Scenario 2: replication controller limited by quota**
```
$ kubectl create quota my-quota --hard=pods=1
$ kubectl run nginx --image=nginx --replicas=50
```
Before:
* POST events not relevant as aggregation worked well here.
* PATCH events with peak and long-tail of 13.6 per second

After:
* POST events not relevant as aggregation worked well here.
* PATCH events with peak: .35 per second, and long-tail of 0

**Which issue this PR fixes**
fixes https://github.com/kubernetes/kubernetes/issues/47366

**Special notes for your reviewer**:
this was a significant problem in a kube 1.5 cluster we are running where events were co-located in a single etcd.  this cluster was normal to have larger numbers of unhealty pods as well as denial by quota.

**Release note**:
```release-note
add support for client-side spam filtering of events
```
 		2017-09-04 16:13:47 -07:00
..
src/k8s.io Merge pull request #47367 from derekwaynecarr/event-spam 2017-09-04 16:13:47 -07:00
BUILD Add a new paging utility for client side ranging 2017-09-01 10:46:57 -04:00
OWNERS
…
README.md Fixup after k8s.io/{kube-gen -> code-generator} rename 2017-08-17 17:55:12 +02:00
godeps-json-updater.go
…
prime-apimachinery.sh Fixup after k8s.io/{kube-gen -> code-generator} rename 2017-08-17 17:55:12 +02:00

README.md

External Repository Staging Area

This directory is the staging area for packages that have been split to their own repository. The content here will be periodically published to respective top-level k8s.io repositories.

Repositories currently staged here:

The code in the staging/ directory is authoritative, i.e. the only copy of the code. You can directly modify such code.

Using staged repositories from Kubernetes code

Kubernetes code uses the repositories in this directory via symlinks in the vendor/k8s.io directory into this staging area. For example, when Kubernetes code imports a package from the k8s.io/client-go repository, that import is resolved to staging/src/k8s.io/client-go relative to the project root:

// pkg/example/some_code.go
package example

import (
  "k8s.io/client-go/dynamic" // resolves to staging/src/k8s.io/client-go/dynamic
)

Once the change-over to external repositories is complete, these repositories will actually be vendored from k8s.io/<package-name>.