Commit Graph

357 Commits (dcf296a97a53aad4d95eb328c5e7c283eeba2fb1)

Author SHA1 Message Date
Kubernetes Submit Queue f2db955b9d
Merge pull request #64363 from idealhack/sub-benchmarks/scheduler/schedulercache
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

scheduler: update tests to use sub-benchmarks (pkg/scheduler/cache)

**What this PR does / why we need it**:

Go 1.7 added the subtest feature which can make table-driven tests much easier to run and debug. Some tests are not using this feature.

Further reading: [Using Subtests and Sub-benchmarks](https://blog.golang.org/subtests)

/kind cleanup

**Release note**:

```release-note
NONE
```
2018-07-01 19:04:19 -07:00
Yang Li d7e12ce453 scheduler: update tests to use sub-benchmarks (pkg/scheduler/cache) 2018-07-01 00:51:42 +08:00
Kubernetes Submit Queue ea3451f83e
Merge pull request #65188 from aveshagarwal/master-rhbz-1555057
Automatic merge from submit-queue (batch tested with PRs 65188, 65541, 65534). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Increase glog level of some scheduling errors.

In our production environments, we are noticing that for every scheduling error, we are logging 3 errors at following lines:

1. https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/scheduler.go#L194

2. https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/factory/factory.go#L1416

3. https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/factory/factory.go#L1323

This PR increases log levels of the last 2 errors to V(3).Infof. We can discuss if it would be helpful to increase the log level of the first error too.

@kubernetes/sig-scheduling-pr-reviews 
@bsalamat @k82cn @liggitt @sjenning 

```release-note
None.
```
2018-06-29 21:42:07 -07:00
Dr. Stefan Schimanski f8de7cea40 Update generated files 2018-06-29 20:36:17 +02:00
Avesh Agarwal c0cffb8a34 Increase glog level of some scheduling errors. 2018-06-28 23:34:29 -04:00
Kubernetes Submit Queue a13fe4d15d
Merge pull request #65424 from liggitt/scheduler-config
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix scheduler config decoding

Fixes #65413

Implements a custom unmarshaler for a single scheduler config type which did not correctly specify JSON tags until https://github.com/kubernetes/kubernetes/issues/65414 is resolved

Adds missing compatibility tests for scheduler extenders back to 1.7

```release-note
Fixes incompatibility with custom scheduler extender configurations specifying `bindVerb`
```
2018-06-25 00:21:35 -07:00
Jordan Liggitt fcaaf59359
Fix scheduler config decoding 2018-06-24 23:28:56 -04:00
Kubernetes Submit Queue f0311d8232
Merge pull request #65396 from bsalamat/sched_no_sort
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improve scheduler's performance by eliminating sorting of nodes by their score

**What this PR does / why we need it**:
Profiling scheduler, I noticed that scheduler spends a significant amount of time in sorting the nodes after we score them to find nodes with the highest score. Finding nodes with the highest score does not need sorting the array. This PR replaces the sort with a linear scan.

Eliminating the sort results in over 10% improvement in throughput of the scheduler.

Before (3 runs for 5000 nodes, scheduling 1000 pods in a cluster running 2000 pods):
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  20682552 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  20464729 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  21188906 ns/op

After:
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  18485866 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  18457749 ns/op
BenchmarkScheduling/5000Nodes/2000Pods-12         	    1000	  18418200 ns/op

**Release note**:

```release-note
Improve scheduler's performance by eliminating sorting of nodes by their score.
```
2018-06-23 20:12:01 -07:00
Bobby (Babak) Salamat ffc8cc2f50 Improve scheduler's performance by eliminating sorting when finding the host with the highest score 2018-06-23 11:24:43 -07:00
Kubernetes Submit Queue 582b88c879
Merge pull request #64995 from bsalamat/preempt_opt
Automatic merge from submit-queue (batch tested with PRs 65388, 64995). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add more conditions to the list of predicate failures that won't be resolved by preemption

**What this PR does / why we need it**:
Adds more conditions to the list of predicate failures that won't be resolved by preemption. This change can potentially improve performance of preemption by avoiding the nodes that won't be able to schedule the pending pod no matter how many other pods are removed from them.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
Add more conditions to the list of predicate failures that won't be resolved by preemption.
```

/sig scheduling
2018-06-23 05:52:07 -07:00
Bobby (Babak) Salamat 8cdf83ed1e Add tests to cover newly added unresolvable failures 2018-06-22 17:06:19 -07:00
Bobby (Babak) Salamat fab26e470c Add more unresolvable conditions to optimize preemption logic 2018-06-22 17:04:55 -07:00
Jeff Grafton 23ceebac22 Run hack/update-bazel.sh 2018-06-22 16:22:57 -07:00
Jeff Grafton a725660640 Update to gazelle 0.12.0 and run hack/update-bazel.sh 2018-06-22 16:22:18 -07:00
Kubernetes Submit Queue b45ba959c0
Merge pull request #64693 from xiechengsheng/fix-typos
Automatic merge from submit-queue (batch tested with PRs 65024, 65287, 65345, 64693, 64941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix some typos in code comments.

Signed-off-by: xiechengsheng <XIE1995@whut.edu.cn>



**What this PR does / why we need it**:
Fix some typos in code comments.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
NONE

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-06-22 06:10:21 -07:00
xiechengsheng 66e0b53c3c fix some typos
Signed-off-by: xiechengsheng <XIE1995@whut.edu.cn>
2018-06-22 09:26:14 +08:00
Kubernetes Submit Queue 23b4690d00
Merge pull request #65306 from shyamjvs/fine-grained-scheduler-metric
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Split scheduler latency metric to fine-grained steps

This splits the summary metric we recently added into finer steps. It should be very useful for performance experiments.

/cc @wojtek-t 
fyi - @bsalamat @misterikkit 

Strictly speaking this is a breaking change, but since this metric was added only ~week ago I think it should fine (we should port this change to 1.11).

```release-note
Split 'scheduling_latency_seconds' metric into finer steps (predicate, priority, premption)
```
2018-06-21 09:11:58 -07:00
Shyam Jeedigunta b9ae20c99e Split scheduler latency metric to fine-grained steps 2018-06-21 14:19:39 +02:00
Kubernetes Submit Queue d1f5cb2348
Merge pull request #65050 from sttts/sttts-deepcopy-update
Automatic merge from submit-queue (batch tested with PRs 64895, 64938, 63700, 65050, 64957). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump gengo to include uniform pointer deepcopy

This bumps k8s.io/gengo with uniform pointer support in deepcopy-gen.

Fixes https://github.com/kubernetes/code-generator/issues/45.
2018-06-21 04:15:16 -07:00
Kubernetes Submit Queue 15902d9113
Merge pull request #63662 from xchapter7x/pkg-scheduler-algorithmprovider
Automatic merge from submit-queue (batch tested with PRs 64285, 63660, 63661, 63662, 64883). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg/scheduler/algorithmprovider)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-21 01:19:26 -07:00
Kubernetes Submit Queue 58574021a7
Merge pull request #63661 from xchapter7x/pkg-scheduler
Automatic merge from submit-queue (batch tested with PRs 64285, 63660, 63661, 63662, 64883). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg/scheduler)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-21 01:19:22 -07:00
Kubernetes Submit Queue 1712073165
Merge pull request #63660 from xchapter7x/pkg-scheduler-algorithm-predicates
Automatic merge from submit-queue (batch tested with PRs 64285, 63660, 63661, 63662, 64883). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg/scheduler/algorithm/predicates)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-21 01:19:18 -07:00
Jonathan Basseri 56b941f3df scheduler: fix equiv. cache logging.
Change the log messages to accurately reflect the cache behavior.
2018-06-20 10:52:33 -07:00
Jonathan Basseri c24842d806 Add a package docstring to core/equivalence. 2018-06-20 10:52:33 -07:00
Jonathan Basseri b571065bc4 Clean up names in equivalence package.
Remove stutter from names and provide more idiomatic patterns.

This makes call sites that use equivalence cache easier to read.
2018-06-20 10:52:33 -07:00
Jonathan Basseri 31c746d960 Move equivalence cache into new package.
This moves the equivalence cache implementation out of the 'core'
package and into k8s.io/kubernetes/pkg/scheduler/core/equivalence.

Separating the equiv. cache from the genericScheduler implementation
make their interaction points easier to follow, and prevents us from
accidentally accessing unexported fields.
2018-06-20 10:52:32 -07:00
Kubernetes Submit Queue bb6270bd92
Merge pull request #64692 from adohe/scheduler_cache
Automatic merge from submit-queue (batch tested with PRs 64882, 64692, 64389, 60626, 64840). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

update pod state of scheduler cache when UpdatePod

update pod state map in scheduler cache when call UpdatePod. @k82cn @bsalamat 

```release-note
keep pod state consistent when scheduler cache UpdatePod
```
2018-06-20 10:03:23 -07:00
Kubernetes Submit Queue dd040d6010
Merge pull request #63942 from misterikkit/ecache-cleanup
Automatic merge from submit-queue (batch tested with PRs 64142, 64426, 62910, 63942, 64548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

scheduler: further cleanup of equivalence cache

**What this PR does / why we need it**:
This improves comments and simplifies some names/logic in equivalence_cache.go, as well as changing the order of some items in the file.


**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
/kind cleanup
2018-06-20 00:05:18 -07:00
Kubernetes Submit Queue 53d03c58cd
Merge pull request #64179 from wgliang/master.scheduler-unused-para
Automatic merge from submit-queue (batch tested with PRs 64252, 64307, 64163, 64378, 64179). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove unused parameter (pod) in `pkg/scheduler/core/generic_scheduler`

**What this PR does / why we need it**:

Remove unused parameter (pod) in `pkg/scheduler/core/generic_scheduler`

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-06-19 21:45:21 -07:00
Kubernetes Submit Queue e486e8f1f9
Merge pull request #63659 from xchapter7x/pkg-scheduler-algorithm-priorities-util
Automatic merge from submit-queue (batch tested with PRs 65230, 57355, 59174, 63698, 63659). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use subtest for table units (pkg-scheduler-algorithm-priorities-util)

**What this PR does / why we need it**: Update scheduler's unit table tests to use subtest

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:
breaks up PR: https://github.com/kubernetes/kubernetes/pull/63281
/ref #63267

**Release note**:

```release-note
This PR will leverage subtests on the existing table tests for the scheduler units.
Some refactoring of error/status messages and functions to align with new approach.

```
2018-06-19 20:19:25 -07:00
Krzysztof Siedlecki e32910a544 Readding summary metrics 2018-06-14 15:05:12 +02:00
Krzysztof Siedlecki 0547bbf744 Revert "Fixing scheduling latency metrics"
This reverts commit 0e833bfc83.
2018-06-14 14:50:12 +02:00
Dr. Stefan Schimanski 1208437f84 Update generated files 2018-06-13 12:35:13 +02:00
John Calabrese f415558c30 use subtest for table units
implement PR feedback

replace errorf + return with fatalf

  https://github.com/kubernetes/kubernetes/pull/63662#discussion_r192540370
  https://github.com/kubernetes/kubernetes/pull/63662#discussion_r192540457
2018-06-08 11:44:24 -04:00
John Calabrese 7735dd6843 use subtest for table units
apply consistent format to name strings

  - #63661 (comment)
  - #63661 (comment)

inline subtest logic

  https://github.com/kubernetes/kubernetes/pull/63661/files#r192540031

remove duplicate messaging in subtest errors
2018-06-08 10:52:28 -04:00
AdoHe 6116c64f64 keep pod state consistent when UpdatePod 2018-06-06 21:13:12 +08:00
xiechengsheng cd65afdf05 fix some typos
Signed-off-by: xiechengsheng <XIE1995@whut.edu.cn>
2018-06-06 13:11:26 +08:00
John Calabrese 8bbf182695 use subtest for table units
apply subtest to predicates
2018-06-05 09:07:57 -04:00
Silvery Fu 8792f99bf8 Fix the handling of untagged images 2018-06-04 13:57:09 -07:00
Silvery Fu 73970a5027 Update generated bazel 2018-06-04 13:56:51 -07:00
Łukasz Osipiuk 98041d0925 Allow parametrization of RequestedToCapacityRatio priority function via policy config 2018-06-04 12:28:54 +02:00
Łukasz Osipiuk 4684d5aac6 Register RequestedToCapacityRatioPriority priority function 2018-06-04 11:30:57 +02:00
Hemant Kumar a0a9ccfa87 Implement scheduler changes for volume limits 2018-06-01 19:17:31 -04:00
Guoliang Wang 761cf41427 Move pkg/scheduler/schedulercache -> pkg/scheduler/cache 2018-05-31 22:55:34 +08:00
Kubernetes Submit Queue b5d21a92ad
Merge pull request #64339 from liztio/pronouns
Automatic merge from submit-queue (batch tested with PRs 61803, 64305, 64170, 64361, 64339). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove some unnecessarily gendered pronouns in comments

**What this PR does / why we need it**:

A bunch of comments are unnecessarily gendered. I've changed them to gender-neutral they/theirs.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-05-30 17:34:25 -07:00
Krzysztof Siedlecki 0e833bfc83 Fixing scheduling latency metrics 2018-05-30 11:20:12 +02:00
Jonathan Basseri ba08b05e28 Rename equiv. class invalidation functions.
Change the invalidation functions to have cleaner and more consistent
names.
2018-05-29 11:23:13 -07:00
Jonathan Basseri 5d13798e5c Change the return of EquivalenceClass.lookupResult.
This makes the lookup behave like a normal map lookup, so it is easier
for readers to follow the logic. It also inverts the "invalid" bool to
an "ok" bool because `!invalid` is a double negative.
2018-05-29 11:22:02 -07:00
Jonathan Basseri 9b06870620 Clean up names and comments in equivalence cache. 2018-05-29 11:22:02 -07:00
John Calabrese d1abc5c824 use subtest for table units
employ consitent table var naming

  - https://github.com/kubernetes/kubernetes/pull/63659#discussion_r187736533
2018-05-28 09:33:42 -04:00