Commit Graph

202 Commits (4c874dbefe2bd1f06a9358146be106c87b4858ce)

Author SHA1 Message Date
Mike Danese a05c3c0efd autogenerated 2017-04-14 10:40:57 -07:00
Andy Goldstein a81e5807b6 Update bazel 2017-04-14 07:27:45 -04:00
Andy Goldstein e63fcf708d Make controller Run methods consistent
- startup/shutdown logging
- wait for cache sync logging
- defer utilruntime.HandleCrash()
- wait for stop channel before exiting
2017-04-14 07:27:45 -04:00
Wei Wei 3329446c0e add Stringer interface for eventType 2017-04-11 10:36:52 +08:00
Christoph Blecker 6681835b0c
Fix gofmt errors 2017-03-28 17:12:04 -07:00
deads2k 8e26fa25da wire in aggregation 2017-03-27 09:44:10 -04:00
Chao Xu 33da82bc67 construction of GC should not fail for restmapper error caused by tpr 2017-03-16 14:19:17 -07:00
Chao Xu 0605ba7a6d wait for garbagecollector to be synced in test 2017-03-14 16:19:33 -07:00
Chao Xu d7aef0a338 Let GC print specific message for RESTMapping failure 2017-03-10 11:38:57 -08:00
Chao Xu c3baf402f5 gc changes 2017-02-28 23:05:41 -08:00
Chao Xu e4aa9db258 generated 2017-02-28 23:05:40 -08:00
Dr. Stefan Schimanski 395be3b401 apimachinery: handle duplicated and conflicting type registration 2017-02-27 21:58:21 +01:00
Jordan Liggitt 7b6ef2cedc
Switch watch prefixes to params 2017-02-22 08:54:08 -05:00
Jordan Liggitt e6dbe5f57e
Pass typed options to dynamic client 2017-02-22 08:53:57 -05:00
Wojciech Tyczynski a21b08d00f Revert "Use watch param instead of deprecated /watch/ prefix" 2017-02-21 08:37:51 +01:00
Jordan Liggitt f950171003
Switch watch prefixes to params 2017-02-19 23:51:58 -05:00
Jordan Liggitt 308fdcd13f
Pass typed options to dynamic client 2017-02-19 22:12:55 -05:00
Jordan Liggitt 09b5d7279f
install authentication.k8s.io/v1, add tests 2017-02-07 01:44:56 -05:00
Jordan Liggitt 8706bc9617
install authorization.k8s.io/v1, add tests
Use specific v1beta1 Authorization client

Add client expansions
2017-02-06 18:16:51 -05:00
Dr. Stefan Schimanski de2eee391a Update bazel 2017-02-02 15:19:04 +01:00
Dr. Stefan Schimanski ef8bb4e7e7 Move pkg/api.FinanlizerOrphan into metav1 2017-02-02 15:17:34 +01:00
deads2k 2c1c0f3f72 move workqueue to client-go 2017-01-30 09:08:21 -05:00
deads2k 9488e2ba30 move testing/core to client-go 2017-01-26 13:54:40 -05:00
deads2k 2734f8f892 move dynamic and discovery clients 2017-01-26 08:37:06 -05:00
deads2k b0b156b381 make tools/cache authoritative 2017-01-25 08:29:45 -05:00
Clayton Coleman be6d2933df
refactor: Move *Options references to metav1 2017-01-24 13:41:51 -05:00
deads2k 5a8f075197 move authoritative client-go utils out of pkg 2017-01-24 08:59:18 -05:00
Clayton Coleman 469df12038
refactor: move ListOptions references to metav1 2017-01-23 17:52:46 -05:00
deads2k ee6752ef20 find and replace 2017-01-20 08:04:53 -05:00
deads2k c587b8a21e re-run client-gen 2017-01-20 08:02:36 -05:00
deads2k c47717134b move utils used in restclient to client-go 2017-01-19 07:55:14 -05:00
Clayton Coleman 9a2a50cda7
refactor: use metav1.ObjectMeta in other types 2017-01-17 16:17:19 -05:00
Clayton Coleman 36acd90aba
Move APIs and core code to use metav1.ObjectMeta 2017-01-17 16:17:18 -05:00
deads2k f31ecdd0f7 generated changes 2017-01-17 08:32:05 -05:00
deads2k 26c46971f2 move PatchType to apimachinery 2017-01-17 08:32:05 -05:00
deads2k 77b4d55982 mechanical 2017-01-16 09:35:12 -05:00
Kubernetes Submit Queue 6b5d82b512 Merge pull request #37505 from k82cn/use_controller_inf
Automatic merge from submit-queue (batch tested with PRs 39807, 37505, 39844, 39525, 39109)

Made cache.Controller to be interface.

**What this PR does / why we need it**:

#37504
2017-01-13 13:40:41 -08:00
deads2k f1176d9c5c mechanical repercussions 2017-01-13 08:27:14 -05:00
Klaus Ma 25fe1e0d82 Made cache.Controller to be interface. 2017-01-13 13:33:23 +08:00
Dr. Stefan Schimanski 2741eb7fdb Update generated files 2017-01-11 21:54:07 +01:00
deads2k 6a4d5cd7cc start the apimachinery repo 2017-01-11 09:09:48 -05:00
Kubernetes Submit Queue 3c56041384 Merge pull request #39457 from zhouhaibing089/gcc
Automatic merge from submit-queue (batch tested with PRs 37845, 39439, 39514, 39457, 38866)

Log a warning message when failed to find kind for resource in garbage collector controller

at this time, I do not think thirdparty api group version resources should be taken care by garbage collector controllers, and this line of call will fail actually: https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/garbagecollector/garbagecollector.go#L565, and as a result, the garbagecollector controller failed to start.
2017-01-09 13:15:19 -08:00
zhouhaibing089 b25873a4bc warning instead error when search kinds for resources 2017-01-08 09:47:43 +08:00
Jeff Grafton 20d221f75c Enable auto-generating sources rules 2017-01-05 14:14:13 -08:00
Mike Danese 161c391f44 autogenerated 2016-12-29 13:04:10 -08:00
Kubernetes Submit Queue 8abbedae54 Merge pull request #38315 from mikedanese/pin-gazel
Automatic merge from submit-queue

Pin gazel to a version and support cgo

This fixes the bazel build.

@krousey who is buildcop
2016-12-12 19:32:29 -08:00
Kubernetes Submit Queue f45e918b8b Merge pull request #35833 from apelisse/owners-pkg-controller
Automatic merge from submit-queue

Curating Owners: pkg/controller

cc @jsafrane @mikedanese @bprashanth @derekwaynecarr @thockin @saad-ali

In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone **lgtms** and then someone
experienced in the project **approves**), we are adding new reviewers to
existing owners files.
## If You Care About the Process:

We did this by algorithmically figuring out who’s contributed code to
the project and in what directories.  Unfortunately, that doesn’t work
perfectly: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.

Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).

At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
## TLDR:

As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the OWNERS file to add
   the names of people that should be reviewing code in the future in the **reviewers** section. You probably do NOT need to modify the **approvers** section.
3. Notify me if you want some OWNERS file to be removed.  Being an approver or reviewer
   of a parent directory makes you a reviewer/approver of the subdirectories too, so not all
   OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
   over again (don't hesitate to ask me for help, or use the pull-request
   above as an example)
2016-12-12 18:51:33 -08:00
Mike Danese c87de85347 autoupdate BUILD files 2016-12-12 13:30:07 -08:00
Clayton Coleman c52d510a24
refactor: generated 2016-12-10 18:05:53 -05:00
Clayton Coleman 42d410fdde
Switch to use pkg/apis/meta/v1/unstructured and the new interfaces
Avoid directly accessing an unstructured type if it is not required.
2016-12-10 18:05:28 -05:00
Clayton Coleman c30862a488
Move OwnerReference to pkg/apis/meta/v1 and remove metatypes pkg
OwnerReference is common.
2016-12-10 18:05:28 -05:00
Dr. Stefan Schimanski 24e24fc7bb Add verb support to gc and namespace controllers 2016-12-05 12:36:05 +01:00
Dr. Stefan Schimanski 458d2b2fe4 Add verb support for discovery client 2016-12-05 12:36:05 +01:00
Clayton Coleman 3454a8d52c
refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
Clayton Coleman 5df8cc39c9
refactor: generated 2016-12-03 19:10:46 -05:00
Clayton Coleman 35a6bfbcee
generated: refactor 2016-11-23 22:30:47 -06:00
Chao Xu bcc783c594 run hack/update-all.sh 2016-11-23 15:53:09 -08:00
Chao Xu 7eeb71f698 cmd/kube-controller-manager 2016-11-23 15:53:09 -08:00
Chao Xu 9aa1049a03 Update OWNERS 2016-11-02 16:19:22 -07:00
Antoine Pelisse db35acde19 Update OWNERS: Remove reviewers: pkg/controller 2016-11-02 16:19:19 -07:00
Antoine Pelisse c695a54c1c Update OWNERS approvers and reviewers: pkg/controller 2016-11-02 16:19:18 -07:00
Clayton Coleman ca2f1b87ad
Replace negotiation with a new method that can extract info
Alter how runtime.SerializeInfo is represented to simplify negotiation
and reduce the need to allocate during negotiation. Simplify the dynamic
client's logic around negotiating type. Add more tests for media type
handling where necessary.
2016-10-28 11:30:11 -04:00
Mike Danese 3b6a067afc autogenerated 2016-10-21 17:32:32 -07:00
mbohlool 25afcc5522 Add +optional tag to all fields with omitempty json tag 2016-10-17 08:52:13 -07:00
Chao Xu 7249c9bd8a fix TestCreateWithNonExistentOwner
remove the use of gc.QueuesDrained
2016-09-26 16:51:56 -07:00
Clayton Coleman 97c35fcc67
Allow garbage collection to work against different API prefixes
The GC needs to build clients based only on Resource or Kind. Hoist the
restmapper out of the controller and the clientpool, support a new
ClientForGroupVersionKind and ClientForGroupVersionResource, and use the
appropriate one in both places.
2016-09-22 15:00:58 -04:00
Wojciech Tyczynski 27d75054b3 Avoid unnecessary API calls from GC 2016-09-18 07:09:11 +02:00
Kubernetes Submit Queue 0d9685b0b5 Merge pull request #32805 from caesarxuchao/more-gc-optimization
Automatic merge from submit-queue

Add the uid in a delete event to the absentOwnerCache

This is a small optimization to further reduce the traffic sent by the GC.

In #31167, GC caches the non-existent owners when it processes the dirtyQueue. As discovered in #32571, there is still small inefficiency, because there are multiple goroutines processing the dirtyQueue, many of them might send a GET to the apiserver before the cache gets populated.

This PR populates the cache when GC observes an object gets deleted, which happens before the processing of the dirtyQueue, so it avoids the simultaneous GET sent by the GC workers.

cc @lavalamp
2016-09-16 00:40:24 -07:00
Chao Xu d122de5371 add the uid in a delete event to the absentOwnerCache 2016-09-15 13:53:47 -07:00
Mike Danese a765d59932 move informer and controller to pkg/client/cache
Signed-off-by: Mike Danese <mikedanese@google.com>
2016-09-15 12:50:08 -07:00
Chao Xu 21896dac4b add the uid in a delete event to the absentOwnerCache 2016-09-15 11:22:22 -07:00
deads2k 8fac64b43f add localSAR 2016-09-13 08:54:23 -04:00
Kubernetes Submit Queue 7a4d81ea43 Merge pull request #31271 from deads2k/self-sar
Automatic merge from submit-queue

add selfsubjectaccessreview API

Exposes the REST API for self subject access reviews.  This allows a user to see whether or not they can perform a particular action.

@kubernetes/sig-auth
2016-09-08 01:29:48 -07:00
Chao Xu 25966e1148 gc retries failed garbage collection 2016-09-02 14:17:42 -07:00
deads2k 97529e3687 add selfsubjectaccessreview API 2016-08-29 09:51:54 -04:00
Chao Xu 9ac91e5172 debugging gc 2016-08-26 15:58:33 -07:00
Kubernetes Submit Queue 8f431e4af8 Merge pull request #31167 from caesarxuchao/gc-absent-owner-cache
Automatic merge from submit-queue

[GarbageCollector] add absent owner cache

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:
Reducing the Request sent to the API server by the garbage collector to check if an owner exists.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

#26120

**Special notes for your reviewer**:

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
```

Currently when processing an item in the dirtyQueue, the garbage collector issues GET to check if any of its owners exist. If the owner is a replication controller with 1000 pods, the garbage collector sends a GET for the RC 1000 times. This PR caches the owner's UID if it does not exist according to the API server. This cuts 1/3 of the garbage collection time of the density test in the gce-500 and gce-scale, where the QPS is the bottleneck.
2016-08-25 07:39:31 -07:00
Chao Xu f2d0f1e3f0 add absent owner cache 2016-08-23 17:21:15 -07:00
jianhuiz 494129b089 add generated files 2016-08-22 23:00:23 -07:00
Chao Xu 10a4e912a8 gc never resync 2016-08-22 11:32:37 -07:00
Chao Xu 992afd9c45 let rate_limiter_helper use sync.Once 2016-08-19 20:32:28 -07:00
Chao Xu c790773018 temporarlily stop register RateLimiter metrics in the garbage collector 2016-08-19 17:31:32 -07:00
Chao Xu 65d1dbe8d9 fix memory leak in gc 2016-08-18 21:54:44 -07:00
Kubernetes Submit Queue 035ec518af Merge pull request #28387 from caesarxuchao/gc-latency-measure
Automatic merge from submit-queue

[GarbageCollector] measure latency

First commit is #27600.

In e2e tests, I measure the average time an item spend in the eventQueue(~1.5 ms), dirtyQueue(~13ms), and orphanQueue(~37ms). There is no stress test in e2e yet, so the number may not be useful.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/28387)
<!-- Reviewable:end -->
2016-08-11 02:33:55 -07:00
Chao Xu 41572cb22d add metrics to gc 2016-08-10 12:45:06 -07:00
Kubernetes Submit Queue 01d6ac64b4 Merge pull request #29898 from matttproud/refactor/simplify/garbagecollector
Automatic merge from submit-queue

pkg/controller/garbagecollector: simplify mutexes.

pkg/controller/garbagecollector: simplified synchronization and made idiomatic.


Similar to #29598, we can rely on the zero-value construction behavior
to embed `sync.Mutex` into parent structs.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29898)
<!-- Reviewable:end -->
2016-08-09 21:39:26 -07:00
Timothy St. Clair 24993b0343 Update generated files due to dependency shift.
Also minor bug fix in federation api.
2016-08-09 10:41:09 -05:00
Chao Xu 4d2350632c only store typeMeta and objectMeta in the gc store 2016-08-08 17:23:13 -07:00
deads2k 32920b5617 add subjectaccessreviews resource 2016-08-05 11:20:56 -04:00
deads2k 60dd4a5d26 interesting changes to add tokenreviews endpoint to implement webhook 2016-08-03 08:37:45 -04:00
Matt T. Proud 1ca99119fd pkg/controller/garbagecollector: simplify mutexes.
Similar to #29598, we can rely on the zero-value construction behavior
to embed `sync.Mutex` into parent structs.
2016-08-02 08:23:27 +02:00
k8s-merge-robot a4442250f3 Merge pull request #29550 from caesarxuchao/fix-28713
Automatic merge from submit-queue

[Garbage Collector] GC waits for all reflectors to be synced before starting workers

Fix 28713. See https://github.com/kubernetes/kubernetes/issues/28713#issuecomment-234826276 for details on the cause of the flake. The test has passed 100 times locally.
2016-08-01 18:41:12 -07:00
Chao Xu d077db366a GC waits for all reflectors to be synced before starts workers 2016-07-25 14:03:30 -07:00
lixiaobing10051267 57eabf2b7e Garbagecollector.go 2016-07-21 22:31:08 +08:00
Wojciech Tyczynski f71244975d Revert "[garbage collector] add e2e test" 2016-07-15 10:10:02 +02:00
Chao Xu 7a5b3c43a0 plumb --enable_garbage_collector from environment variable;
adding a simple e2e test
2016-07-08 10:27:16 -07:00
David McMahon ef0c9f0c5b Remove "All rights reserved" from all the headers. 2016-06-29 17:47:36 -07:00
Chao Xu d9f07925be let dynamic client handle non-registered ListOptions;
register ListOptions for apis/policy
2016-06-22 13:18:50 -07:00
Chao Xu c15c10f312 fix a log line 2016-06-10 09:58:27 -07:00
Chao Xu 06f49f7ca7 Let the dynamic client take a customized parameter codec for List, Watch, and DeleteCollection.
Let the gc's ListWatcher use api.ParameterCodec. Fixes 25890.
2016-06-03 11:22:51 -07:00
Chao Xu 1665546d2d add finalizer logics to the API server and the garbage collector; handling DeleteOptions.OrphanDependents in the API server 2016-05-24 13:07:28 -07:00
Chao Xu c73406bcfe the garbage collector controller 2016-05-15 16:04:19 -07:00