Commit Graph

596 Commits (b06462f0407b2e4ebbe214f4bbb4b6cd6bb5c82d)

Author SHA1 Message Date
Jonathan Basseri b864a5a808 Remove scheduler "TestGroup" utility.
This util was used to fake certain aspects of apiserver behavior, such
as resource paths and JSON encoding. Our unit tests have been refactored
so they don't rely on the REST or JSON aspects of apiserver. This util
is no longer needed.
2018-12-03 16:28:21 -08:00
Kubernetes Prow Robot 2c322a2ff5
Merge pull request #70227 from bsalamat/reserve
Add plugin interfaces for reserve and prebind extension points of the scheduling framework
2018-12-01 05:25:12 -08:00
Kubernetes Prow Robot 82abbdc11a
Merge pull request #71488 from bsalamat/queue-sort
Change sort function of the scheduling queue to avoid starvation
2018-12-01 03:59:51 -08:00
Bobby (Babak) Salamat e60f510e38 Autogenerated files 2018-11-30 16:03:47 -08:00
Bobby (Babak) Salamat f74b30868c Add plugin invocation for 'reserve' and 'prebind' plugins to the scheduler. 2018-11-30 16:03:47 -08:00
Bobby (Babak) Salamat f6f81fb1f3 Add Context as a mechanism to pass data between plugins. 2018-11-30 16:02:09 -08:00
Bobby (Babak) Salamat 7221589dde Add "reserve" and "prebind" plugin interfaces for the scheduling framework. 2018-11-30 16:02:09 -08:00
k8s-ci-robot 79e5cb2cb7
Merge pull request #71302 from liggitt/verify-unit-test-feature-gates
Split mutable and read-only access to feature gates, limit tests to readonly access
2018-11-29 21:45:12 -08:00
k8s-ci-robot 527d1c34cc
Merge pull request #70947 from Adirio/nodetree-thread-safety
Scheduler internal NodeTree thread-safe NumNodes
2018-11-29 07:36:48 -08:00
k8s-ci-robot 6d87c074e2
Merge pull request #63664 from xchapter7x/pkg-scheduler-api-validation
use subtest for table units (pkg/scheduler/api/validation)
2018-11-29 02:00:43 -08:00
Bobby (Babak) Salamat 36f8859fa0 autogenerated files 2018-11-27 17:08:58 -08:00
Bobby (Babak) Salamat 554acf2b38 Change sort function of the scheduling queue to avoid starvation 2018-11-27 17:08:40 -08:00
Jordan Liggitt 2498ca7606 drop VerifyFeatureGatesUnchanged 2018-11-21 11:51:33 -05:00
Yecheng Fu 8fc00ebda6 Clear pod binding cache. 2018-11-21 11:24:53 +08:00
k8s-ci-robot 7e621ccb08
Merge pull request #71063 from Huang-Wei/nodeinfo-clone-panic
fix a scheduler panic due to internal cache inconsistency
2018-11-16 20:27:44 -08:00
k8s-ci-robot 1f3057b7fb
Merge pull request #70898 from Huang-Wei/preemption-issue
ensure scheduler preemptor behaves in an efficient/correct path
2018-11-16 20:27:35 -08:00
Wei Huang b4fd11512a
ensure scheduler preemptor behaves in an efficient/correct path
- don't update nominatedMap cache when Pop() an element from activeQ
- instead, delete the nominated info from cache when it's "assumed"
- unit test behavior adjusted
- expose SchedulingQueue in factory.Config
2018-11-16 14:22:15 -08:00
Wei Huang a86ba8b3c4
fix a scheduler panic due to internal cache inconsistency 2018-11-16 13:02:13 -08:00
Jordan Liggitt 733dd9dfd7 Add tests to ensure feature gate changes don't escape kubelet/scheduler packages 2018-11-16 10:52:53 -05:00
Jordan Liggitt de8bf9b63d fix scheduler and kubelet unit tests leaking feature flag changes 2018-11-16 10:52:53 -05:00
Jordan Liggitt 358c092abe fix storage unit tests leaking feature flag changes 2018-11-16 10:52:52 -05:00
PingWang 9e760732c3 Refactor New function
Signed-off-by: PingWang <wang.ping5@zte.com.cn>

add comments for InitPolicyFromFile

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

make the methods package private

Signed-off-by: PingWang <wang.ping5@zte.com.cn>
2018-11-15 14:30:19 +08:00
Adrián Orive c7cba7370f Scheduler internal NodeTree thread-safe NumNodes
Signed-off-by: Adrián Orive <adrian.orive.oneca@gmail.com>
2018-11-13 08:40:48 +01:00
Davanum Srinivas 954996e231
Move from glog to klog
- Move from the old github.com/golang/glog to k8s.io/klog
- klog as explicit InitFlags() so we add them as necessary
- we update the other repositories that we vendor that made a similar
change from glog to klog
  * github.com/kubernetes/repo-infra
  * k8s.io/gengo/
  * k8s.io/kube-openapi/
  * github.com/google/cadvisor
- Entirely remove all references to glog
- Fix some tests by explicit InitFlags in their init() methods

Change-Id: I92db545ff36fcec83afe98f550c9e630098b3135
2018-11-10 07:50:31 -05:00
k8s-ci-robot b8fece50f5
Merge pull request #70892 from mikedanese/schedrace
Fix a race in the scheduler.
2018-11-09 23:01:15 -08:00
Mike Danese 62c3ec969d Fix a race in the scheduler.
Loop over priorityConfigs seperately. The node loop can only safely
modify result[i][index]. Before this change it sometimes modified
result[i] concurrently with other loops.

Fixes: 7164967662

==================== Test output for //pkg/scheduler/core:go_default_test:
==================
WARNING: DATA RACE
Read at 0x00c0005e8ed0 by goroutine 22:
  k8s.io/kubernetes/pkg/scheduler/core.PrioritizeNodes.func2()
      pkg/scheduler/core/generic_scheduler.go:667 +0x2ea
  k8s.io/kubernetes/vendor/k8s.io/client-go/util/workqueue.ParallelizeUntil.func1()
      staging/src/k8s.io/client-go/util/workqueue/parallelizer.go:65 +0x9e

Previous write at 0x00c0005e8ed0 by goroutine 21:
  k8s.io/kubernetes/pkg/scheduler/core.PrioritizeNodes.func2()
      pkg/scheduler/core/generic_scheduler.go:668 +0x450
  k8s.io/kubernetes/vendor/k8s.io/client-go/util/workqueue.ParallelizeUntil.func1()
      staging/src/k8s.io/client-go/util/workqueue/parallelizer.go:65 +0x9e

Goroutine 22 (running) created at:
  k8s.io/kubernetes/vendor/k8s.io/client-go/util/workqueue.ParallelizeUntil()
      staging/src/k8s.io/client-go/util/workqueue/parallelizer.go:57 +0x1a3
  k8s.io/kubernetes/pkg/scheduler/core.PrioritizeNodes()
      pkg/scheduler/core/generic_scheduler.go:682 +0x592
  k8s.io/kubernetes/pkg/scheduler/core.(*genericScheduler).Schedule()
      pkg/scheduler/core/generic_scheduler.go:186 +0x77d
  k8s.io/kubernetes/pkg/scheduler/core.TestGenericScheduler.func1()
      pkg/scheduler/core/generic_scheduler_test.go:464 +0x91f
  testing.tRunner()
      GOROOT/src/testing/testing.go:827 +0x162

Goroutine 21 (running) created at:
  k8s.io/kubernetes/vendor/k8s.io/client-go/util/workqueue.ParallelizeUntil()
      staging/src/k8s.io/client-go/util/workqueue/parallelizer.go:57 +0x1a3
  k8s.io/kubernetes/pkg/scheduler/core.PrioritizeNodes()
      pkg/scheduler/core/generic_scheduler.go:682 +0x592
  k8s.io/kubernetes/pkg/scheduler/core.(*genericScheduler).Schedule()
      pkg/scheduler/core/generic_scheduler.go:186 +0x77d
  k8s.io/kubernetes/pkg/scheduler/core.TestGenericScheduler.func1()
      pkg/scheduler/core/generic_scheduler_test.go:464 +0x91f
  testing.tRunner()
      GOROOT/src/testing/testing.go:827 +0x162
==================
--- FAIL: TestGenericScheduler (0.01s)
    --- FAIL: TestGenericScheduler/test_6 (0.00s)
        testing.go:771: race detected during execution of test
    testing.go:771: race detected during execution of test
FAIL
2018-11-09 15:21:22 -08:00
Babak "Bobby" Salamat a2c0958428
Revert "Hold mutex lock shorter when processing inter-pod affinity/anti-affin…" 2018-11-08 18:26:26 -08:00
k8s-ci-robot be800e623a
Merge pull request #69663 from sttts/sttts-scheduler-secure-serving
scheduler: enable secure port and authn/z
2018-11-08 17:36:14 -08:00
Dr. Stefan Schimanski d91feb6d18 kube-scheduler: move stopCh creation out of scheduler factory code
Enforces clean ownership of the channel.
2018-11-08 16:43:59 +01:00
tanshanshan cb95edafe8 kube-scheduler: enable secure ports 10259 2018-11-08 16:43:59 +01:00
k8s-ci-robot c5b453b717
Merge pull request #70783 from hex108/debug_extender
Add debug info: scheduler extenders's score and its name for each pod
2018-11-08 02:51:12 -08:00
Jun Gong 9fc369dd0d Add debug info: scheduler extenders's score and its name for each pod 2018-11-08 13:02:57 +08:00
Babak "Bobby" Salamat 2c8e73a16b
Revert "Activate unschedulable pods only if the node became more schedulable" 2018-11-07 16:57:47 -08:00
k8s-ci-robot ed06cbe3e3
Merge pull request #70500 from bsalamat/scheduler_debuger
Add a scheduler cache dumper for debugging purposes
2018-11-06 16:12:54 -08:00
Bobby (Babak) Salamat 48557a163a fixup! Add a scheduler cache dumper 2018-11-06 10:08:22 -08:00
k8s-ci-robot 7984a2bf60
Merge pull request #70564 from KevinWang15/master
Fix typos
2018-11-05 19:04:45 -08:00
Bobby (Babak) Salamat 4bb57c440e Autogenerated files 2018-11-05 13:31:51 -08:00
Bobby (Babak) Salamat 7ce3245ca9 Add a scheduler cache dumper 2018-11-05 13:31:51 -08:00
k8s-ci-robot c0daab0e03
Merge pull request #70274 from zhangmingld/combinesimilercode
combine similar code where calucate schedule priority
2018-11-05 08:14:05 -08:00
Ke Wang 946c701b05 Fix Typo: mataData -> metaData; masquared -> masquerade 2018-11-05 21:19:25 +08:00
k8s-ci-robot 774b18491f
Merge pull request #70605 from bsalamat/affinity_lock_opt
Hold mutex lock shorter when processing inter-pod affinity/anti-affin…
2018-11-04 11:59:05 -08:00
Bobby (Babak) Salamat aa8b5b431b Hold mutex lock shorter when processing inter-pod affinity/anti-affinity priority function 2018-11-02 20:58:07 -07:00
Bobby (Babak) Salamat 7a352b2b92 Do not allocate memory for pods that do not have inter-pod affinity/anti-affinity 2018-11-02 15:15:45 -07:00
k8s-ci-robot b53edbc695
Merge pull request #70348 from zhangmingld/unnecessaryglogv10
duplicated glog.V(10) when had a if glog.V(10)
2018-10-31 01:07:32 -07:00
zhangmingld 7164967662 combine similar code where calucate schedule priority 2018-10-31 08:59:53 +08:00
k8s-ci-robot fda41d14c4
Merge pull request #70366 from mlmhl/scheduler_optimization
Activate unschedulable pods only if the node became more schedulable
2018-10-30 04:57:12 -07:00
k8s-ci-robot 2f175c1b41
Merge pull request #70290 from tossmilestone/scheduler-test-refactor
Refactor scheduler_test.go to use Clientset
2018-10-29 22:07:52 -07:00
mlmhl c50f89dd43 activate unschedulable pods only if the node became more schedulable 2018-10-30 10:45:59 +08:00
zhangmingld 429e67a12f duplicated glog.V(10) when had a if glog.V(10) 2018-10-29 11:30:16 +08:00
He Xiaoxi 12634bf136 Refactor scheduler_test.go to use Clientset
Signed-off-by: He Xiaoxi <xxhe@alauda.io>
2018-10-29 09:51:07 +08:00
k8s-ci-robot 7b5705c619
Merge pull request #70203 from ravisantoshgudimetla/fix-e2e-resource-limits
Fix e2e resource limits
2018-10-26 19:13:41 -07:00
ravisantoshgudimetla fad6b326e3 Fix default algorithm provider priority insertion 2018-10-26 13:48:44 -04:00
zhangmingld cbfaf3856f fix typo in predicates_test.go 2018-10-26 09:59:40 +08:00
k8s-ci-robot 101d26c613
Merge pull request #59529 from wackxu/addmetricvol
Add metrics for volume scheduling operations
2018-10-23 13:52:29 -07:00
wackxu d5edcd3dc3 Add metrics to volume scheduling operations 2018-10-23 20:59:12 +08:00
k8s-ci-robot 060218a862
Merge pull request #69412 from tossmilestone/scheduler-factory-test
Refactor scheduler factory test
2018-10-15 13:52:41 -07:00
Bobby (Babak) Salamat 141b55abf5 Fix a bug in node tree when all nodes in a zone are removed 2018-10-12 21:39:38 -07:00
tanshanshan b7c7966b9f Move pkg/scheduler/algorithm/well_known_labels.go out 2018-10-13 09:10:00 +08:00
He Xiaoxi a96a390d92 Refactor scheduler factory test
Use `k8s.io/client-go/kubernetes/fake.Clientset` as the fake k8s client.

Signed-off-by: He Xiaoxi <xxhe@alauda.io>
2018-10-12 14:39:08 +08:00
k8s-ci-robot 94306c12f5
Merge pull request #69057 from denkensk/create-a-new-scheduler-constructor
create-a-new-scheduler-constructor
2018-10-11 13:45:02 -07:00
k8s-ci-robot 539bdbc355
Merge pull request #69495 from wgliang/feature/movenodoinfofunctions
[scheduler cleanup phase 1]: Move NodeInfo utils into pkg/scheduler/cache
2018-10-11 07:12:40 -07:00
Guoliang Wang a50404d441 [scheduler cleanup phase 1]: Move NodeInfo utils into pkg/scheduler/cache 2018-10-11 11:04:23 +08:00
Guoliang Wang f44f55cdd7 [scheduler cleanup phase 1]: Move CacheComparer to pkg/scheduler/internal/cache/comparer 2018-10-10 21:28:14 +08:00
wangqingcan 608911d5ac add test for new constructor 2018-10-10 17:15:10 +08:00
wangqingcan a74fd15e62 create a new scheduler constructor 2018-10-10 17:10:10 +08:00
k8s-ci-robot d3fe0ea7ff
Merge pull request #69318 from wgliang/feature/move-fakecache
[scheduler cleanup phase 1]: Move FakeCache to pkg/scheduler/internal…
2018-10-09 23:21:49 -07:00
Bobby (Babak) Salamat 5f2555e3ad Remove pod status.phase check from pod event handlers 2018-10-09 13:39:04 -07:00
k8s-ci-robot c00f19bd15
Merge pull request #68403 from wgliang/master.deprecate-Parallelize
Replace Parallelize with function ParallelizeUntil and formally depre…
2018-10-06 09:40:07 -07:00
Guoliang Wang 558e1960b8 [scheduler cleanup phase 1]: Move FakeCache to pkg/scheduler/internal/cache/fake 2018-10-06 23:58:53 +08:00
Guoliang Wang 187e2e01c9 Move scheduler cache interface and implementation to pkg/scheduler/internal/cache 2018-10-06 20:48:59 +08:00
Christoph Blecker 97b2992dc1
Update gofmt for go1.11 2018-10-05 12:59:38 -07:00
Guoliang Wang c2622dd9d8 Replace Parallelize with function ParallelizeUntil and formally deprecate the Parallelize 2018-10-05 17:56:56 +08:00
Bobby (Babak) Salamat 2d9d8c405d Add validation for percentage-of-nodes-to-score of the scheduler config 2018-10-02 17:09:47 -07:00
k8s-ci-robot 14949b302f
Merge pull request #68724 from krmayankk/sched1
scheduler: improve readability of bind function
2018-10-01 18:04:04 -07:00
k8s-ci-robot 5b658bb65e
Merge pull request #69136 from zhangmingld/deletecommentgetPodsMatchingAffinity
func getPodsMatchingAffinity no longer exists,dont need it in comment
2018-09-30 02:37:03 -07:00
Mayank Kumar 1ae12337db scheduler: add error handling for bind 2018-09-28 21:13:21 -07:00
Wei Huang 9da576f03c
move SchedulingQueue to pkg/scheduler/internal/queue 2018-09-28 11:51:02 -07:00
Wei Huang 2e7461c087
auto-generated files 2018-09-28 11:51:01 -07:00
k8s-ci-robot db1d1c8674
Merge pull request #68700 from Huang-Wei/schedulingQ-graceful-shutdown
shutdown schedulingQueue gracefully
2018-09-28 00:46:14 -07:00
Wei Huang be661fddb4
shutdown schedulingQueue gracefully
- add Close() to interface SchedulingQueue
- implement Close() for FIFO and PriorityQueue
- add unit test
2018-09-27 14:32:58 -07:00
Wei Huang 56fcfc5fee
cleanup duplicate assignment logic in scheduler 2018-09-27 11:38:05 -07:00
k8s-ci-robot 762ddf792d
Merge pull request #68984 from k82cn/k8s_68899_m
Keep backward compatibility for 'node.Spec.Unschedulable'.
2018-09-27 04:48:49 -07:00
zhangmingld 061e0dbb7e func getPodsMatchingAffinity no longer exists,dont need it in comment 2018-09-27 14:42:49 +08:00
k8s-ci-robot 823530d4db
Merge pull request #68917 from zhangmingld/fixwordserr
Fix some typo err
2018-09-26 22:59:32 -07:00
k8s-ci-robot a6bc5aa49e
Merge pull request #68563 from DylanBLE/dev
fix scheduler crash when Prioritize Map function failed
2018-09-26 22:59:04 -07:00
Bobby (Babak) Salamat c051f0d31a autogenerated files 2018-09-26 14:22:21 -07:00
Bobby (Babak) Salamat f340f8baf8 Remove PDB and its event handlers from the scheduler cache 2018-09-26 14:22:21 -07:00
hongjian.sun f33c2c11f2 fix scheduler crash when Prioritize Map function failed 2018-09-26 20:16:05 +08:00
k8s-ci-robot 28d86ac47d
Merge pull request #67308 from cofyc/fix67260
Use monotonically increasing generation to prevent equivalence cache race
2018-09-25 00:18:00 -07:00
Jonathan Basseri b0a8dbbc9d Add scheduler throughput metric.
This adds a counter to the scheduler that can be used to calculate
throughput and error ratio. Pods which fail to schedule are not counted
as errors, but can still be tracked separately from successes.

We already measure scheduler latency, but throughput was missing. This
should be considered a key metric for the scheduler.
2018-09-24 14:38:39 -07:00
Da K. Ma 78f6484e14 Keep backward compatibility for 'node.Spec.Unschedulable'.
Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com>
2018-09-23 10:33:51 +08:00
Yecheng Fu b3f1e1200b Update notes to document why invalidation order is important. 2018-09-22 12:09:24 +08:00
Yecheng Fu 2f46bc8a18 Use seqeuence number to represent generation of equivalence cache.
- snapshot equivalence cache generation numbers before snapshotting the
scheduler cache
- skip update when generation does not match live generation
- keep the node and increment its generation to invalidate it instead of
deletion
- use predicates order ID as key to improve performance
2018-09-22 12:08:21 +08:00
Yecheng Fu a2cc1b1a20 Revert "Use sync.map to scale ecache better"
This reverts commit 17d0190706.
2018-09-22 11:33:06 +08:00
zhangmingld 6aaeb209eb fix some typo 2018-09-21 10:19:35 +08:00
Wei Huang 7490542156
fix PodAntiAffinity issues
- update logic of verifying incoming pod's anti-affinity
- rename podMatchesAffinityTermProperties to podMatchesAllAffinityTermProperties
- add podMatchesAnyAffinityTermProperties which is used in some PodAntiAffinity cases
- rename some functions to make it more readable
- add unit tests to verify correctness of PodAffinity and PodAntiAffinity
  - verifying "Existing pod anti-affinity"
  - verifying "incoming pod's anti-affinity"
  - verifying "incoming pod's affinity"
2018-09-13 18:32:33 -07:00
Kubernetes Submit Queue a6eb49f0dc
Merge pull request #68195 from luxas/consolidate_componentconfig_code_standards
Automatic merge from submit-queue (batch tested with PRs 67950, 68195). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Consolidate componentconfig code standards

**What this PR does / why we need it**:

This PR fixes a bunch of very small misalignments in ComponentConfig packages:
 - Add sane comments to all functions/variables in componentconfig `register.go` files
 - Make the `register.go` files of componentconfig pkgs follow the same pattern and not differ from each other like they do today.
 - Register the `openapi-gen` tag in all `doc.go` files where the pkg contains _external_ types.
 - Add the `groupName` tag where missing
 - Fix cases where `addKnownTypes` was registered twice in the `SchemeBuilder`
 - Add `Readme` and `OWNERS` files to `Godeps` directories if missing.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:


**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

/assign @sttts @thockin
2018-09-07 11:19:40 -07:00
Lucas Käldström 83d53ea1c2
Standardize componentconfig code/comment patterns 2018-09-06 13:42:02 +03:00
Stephen Cuppett d85daf0f4c Resolves #59015, extends existing regex to cover t3, r5(d) & z1d instance types
From current AWS documentation:

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/volume_limits.html

T3, C5, C5d, M5, M5d, R5, R5d, and z1d instances support a maximum of
28 attachments, and every instance has at least one network interface
attachment. If you have no additional network interface attachments on
these instances, you could attach 27 EBS volumes.
2018-09-05 21:24:09 -04:00
Kubernetes Submit Queue ca43f007a3
Merge pull request #67731 from gnufied/fix-csi-attach-limit
Automatic merge from submit-queue (batch tested with PRs 68161, 68023, 67909, 67955, 67731). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Fix csi attach limit

Add support for volume limits for CSI.

xref: https://github.com/kubernetes/community/pull/2051

```release-note
Add support for volume attach limits for CSI volumes
```
2018-09-05 14:51:55 -07:00