Commit Graph

68930 Commits (d017bebf6bee94b7ed885cfaa090a189bb23eae2)

Author SHA1 Message Date
Kubernetes Submit Queue d017bebf6b
Merge pull request #67145 from jiayingz/reboot-fix
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fail container start if its requested device plugin resource is unknown.

With the change, Kubelet device manager now checks whether it has cached option state for the requested device plugin resource to make sure the resource is in ready state when we start the container.



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes https://github.com/kubernetes/kubernetes/issues/67107

**Special notes for your reviewer**:

**Release note**:

```release-note
Fail container start if its requested device plugin resource hasn't registered after Kubelet restart.
```
2018-08-21 01:48:54 -07:00
Kubernetes Submit Queue 54dbbc41df
Merge pull request #67618 from liztio/fix-bazel
Automatic merge from submit-queue (batch tested with PRs 66862, 67618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix bazel

**What this PR does / why we need it**:

`bazel test //cmd/kubeadm/...` has been broken for a while. This is incredibly annoying to me, because I use it constantly during my workflow. This should fix it.

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2018-08-21 00:24:04 -07:00
Kubernetes Submit Queue 4cca6a89a0
Merge pull request #66862 from resouer/sync-map
Automatic merge from submit-queue (batch tested with PRs 66862, 67618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use sync.map to scale equiv class cache better

**What this PR does / why we need it**:

Change the current lock in first level ecache into  `sync.Map`, which is known for scaling better than `sync. Mutex ` on machines with >8 CPUs

ref: https://golang.org/pkg/sync/#Map
 
And the code is much cleaner in this way.

5k Nodes, 10k Pods benchmark with ecache enabled in 64 cores VM:

```bash
// before
BenchmarkScheduling/5000Nodes/0Pods-64             10000          17550089 ns/op

// after
BenchmarkScheduling/5000Nodes/0Pods-64             10000          16975098 ns/op
```
Comparing to current implementation, the improvement after this change is noticeable, and the test is stable in 8, 16, 64 cores VM.

**Special notes for your reviewer**:

**Release note**:

```release-note
Use sync.map to scale ecache better
```
2018-08-21 00:24:01 -07:00
Kubernetes Submit Queue 5d8a79f2e1
Merge pull request #67337 from linyouchong/pr-0813-issue67225
Automatic merge from submit-queue (batch tested with PRs 67493, 67617, 67582, 67337). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix bug:DaemonSet didn't create pod after node have enough resource

**What this PR does / why we need it**:
Fix bug:DaemonSet didn't create pod after node have enough resource

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #67225 

**Special notes for your reviewer**:
NONE

**Release note**:
```release-note
DaemonSet: Fix bug- daemonset didn't create pod after node have enough resource
```
2018-08-20 20:39:17 -07:00
Kubernetes Submit Queue 3f6717a097
Merge pull request #67582 from monotek/kibana
Automatic merge from submit-queue (batch tested with PRs 67493, 67617, 67582, 67337). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

updates kibana to 6.3.2

**What this PR does / why we need it**:
* updates kibana to 6.3.2 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
needed as es was updated to 6.3.2 too in https://github.com/kubernetes/kubernetes/pull/67484


**Release note**:

```release-note
updates kibana to 6.3.2 
```
2018-08-20 20:39:13 -07:00
Kubernetes Submit Queue b8c1e34df1
Merge pull request #67617 from liggitt/authz-details
Automatic merge from submit-queue (batch tested with PRs 67493, 67617, 67582, 67337). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

limit forbidden error to details of what was forbidden

cleans up output from https://github.com/kubernetes/kubernetes/pull/65906#discussion_r210048853

/assign @smarterclayton

```release-note
NONE
```
2018-08-20 20:39:10 -07:00
Kubernetes Submit Queue 7c4cbbb0a0
Merge pull request #67493 from soltysh/nil_int_percent
Automatic merge from submit-queue (batch tested with PRs 67493, 67617, 67582, 67337). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Tolerate nil input in GetValueFromIntOrPercent

**What this PR does / why we need it**:
`GetValueFromIntOrPercent` accepts pointer argument but does not validate it. This PR fixes that problem preventing from panics.

/assign @deads2k @sttts 

**Release note**:
```release-note
NONE
```
2018-08-20 20:39:07 -07:00
Kubernetes Submit Queue 80382044ce
Merge pull request #67351 from mkimuram/issue/67269
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Change rbd base image back to fedora 26

Due to the compatiblility issue for ceph server, it needs to change
back to fedora 26 from fedora 28, as a workaround.

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #67269

**Special notes for your reviewer**:
/sig storage
@dims @rootfs

**Release note**:
```release-note
NONE
```
2018-08-20 19:13:52 -07:00
Kubernetes Submit Queue 6d76e35b39
Merge pull request #67605 from liztio/cert-list-2
Automatic merge from submit-queue (batch tested with PRs 67596, 67520, 67605). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Cert list 2

**What this PR does / why we need it**:

Continuation of #67208. Uses the newly created declarative list of certificates kubeadm requires for the certs phase and upgrade steps.


**Special notes for your reviewer**:

**Release note**:

```release-note

```
2018-08-20 15:03:02 -07:00
Kubernetes Submit Queue 61aea3fab0
Merge pull request #67520 from oomichi/issue/67519
Automatic merge from submit-queue (batch tested with PRs 67596, 67520, 67605). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Wait for Available in PV protection test

**What this PR does / why we need it**:

Just after creating a PV, its phase is "Pending" and its finalizers
doesn't contain "kubernetes.io/pv-protection". If the e2e test performs
so faster than the target k8s cluster, the test fails because the PV is
not ready. This adds WaitForPersistentVolumePhase() for waiting the
phase "Available" to avoid such situation.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: 
Fixes #67519

**Release note**: NONE
2018-08-20 15:02:58 -07:00
liz b17830a312
Fix `bazel test //cmd/kubeadm/...`
kubeadm tests no longer change the default directory, they simply use the
correct absolute path
2018-08-20 17:11:13 -04:00
Kubernetes Submit Queue 8f4ab6fe76
Merge pull request #67596 from nikhita/add-apimachinery-label-owners
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add sig/api-machinery label to apimachinery OWNERS files

Inspired by https://github.com/kubernetes/kubernetes/pull/67548. List of OWNERS files taken from https://github.com/kubernetes/community/blob/master/sig-api-machinery/README.md#subprojects.

**Release note**:

```release-note
NONE
```
2018-08-20 14:02:17 -07:00
Jordan Liggitt ecbd013795
limit forbidden error to details of what was forbidden 2018-08-20 16:40:29 -04:00
liz 394e6b554a
Yank out a bunch of manual tests and prose
`phase certs` and upgrade commands now all use certslist infra
2018-08-20 15:21:08 -04:00
Kubernetes Submit Queue e9de06d4df
Merge pull request #67544 from monotek/fluentd-elasticsearch
Automatic merge from submit-queue (batch tested with PRs 67593, 67544). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fixes json logging in fluentd-elasticsearch image

**What this PR does / why we need it**: 
* fixes json logging in fluentd-elasticsearch image by downgrading fluent-plugin-kubernetes_metadata_filter plugin to version 2.0.0
* adds missing pos files to fluentd config 
* adds mapping type name 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
* https://github.com/helm/charts/issues/7168

**Special notes for your reviewer**:
* https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/issues/145
* https://github.com/uken/fluent-plugin-elasticsearch/issues/442

**Release note**:

```release-note
fixes json logging in fluentd-elasticsearch image by downgrading fluent-plugin-kubernetes_metadata_filter plugin to version 2.0.0
```
2018-08-20 11:50:13 -07:00
Kubernetes Submit Queue 76d1418d5d
Merge pull request #67593 from AdamDang/patch-13
Automatic merge from submit-queue (batch tested with PRs 67593, 67544). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Typo fix in returned message: ClusteRole->ClusterRole

There is a typo in returned message: 
Line 143: ClusteRole->ClusterRole
2018-08-20 11:50:09 -07:00
Kubernetes Submit Queue 4c5e615652
Merge pull request #67386 from CaoShuFeng/audit_annotation_object
Automatic merge from submit-queue (batch tested with PRs 55600, 67386). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

update Annotations description about audit.Event

ref: https://github.com/kubernetes/kubernetes/pull/58679#discussion_r209375632

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/assign @liggitt 

**Release note**:

```release-note
NONE
```
2018-08-20 08:46:04 -07:00
Kubernetes Submit Queue 83db13a952
Merge pull request #55600 from yanxuean/api-unit-test
Automatic merge from submit-queue (batch tested with PRs 55600, 67386). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove deplicate code for PodRequestsAndLimits

remove deplicate code for PodRequestsAndLimits
add unit test for PodRequestsAndLimits 

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-08-20 08:46:00 -07:00
Nikhita Raghunath 6e47ba1fde Add sig/api-machinery label to apimachinery OWNERS files 2018-08-20 18:46:47 +05:30
Maciej Szulik 53b4c6387a
Tolarate nil inputs on maxSurge and maxUnavailable when parsing IntOrString 2018-08-20 15:15:57 +02:00
Maciej Szulik 3d7c589f05
Return error on input in GetValueFromIntOrPercent 2018-08-20 15:15:53 +02:00
Anago GCB 6d51311735 Update CHANGELOG-1.10.md for v1.10.7. 2018-08-20 11:37:08 +00:00
AdamDang b938d49571
Typo fix in returned message: ClusteRole->ClusterRole
ClusteRole->ClusterRole
2018-08-20 19:33:03 +08:00
André Bauer f8cea1b028 updates kibana to 6.3.2 2018-08-20 10:00:01 +02:00
André Bauer 6390e590cc added missing pos files & added mapping type name in fluentd config 2018-08-20 09:55:38 +02:00
Kubernetes Submit Queue 39e341c11f
Merge pull request #66639 from hanxiaoshuai/fix07251
Automatic merge from submit-queue (batch tested with PRs 66592, 66639). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

refactor some hard code in pkg/util/ipset/ipset.go

**What this PR does / why we need it**:
refactor some hard code in pkg/util/ipset/ipset.go
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-08-19 22:40:00 -07:00
Kubernetes Submit Queue 4c08bd9abc
Merge pull request #66592 from hanxiaoshuai/addut0723
Automatic merge from submit-queue (batch tested with PRs 66592, 66639). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add unit test for func EntryString in util/ipset

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
add unit test for func EntryString in util/ipset
**Release note**:

```release-note
NONE
```
2018-08-19 22:39:57 -07:00
Kubernetes Submit Queue 31420467ae
Merge pull request #67302 from dixudx/dnsmasq_add_loop_detect
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add --dns-loop-detect option to dnsmasq run by kube-dns

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #67299

**Special notes for your reviewer**:
/cc @kubernetes/sig-network-pr-reviews 

**Release note**:

```release-note
add --dns-loop-detect option to dnsmasq run by kube-dns
```
2018-08-18 08:47:18 -07:00
linyouchong c42439c676 Fix bug:DaemonSet didn't create pod after node have enough resource 2018-08-18 16:15:15 +08:00
Kubernetes Submit Queue b44a768052
Merge pull request #67549 from yliaog/istio
Automatic merge from submit-queue (batch tested with PRs 60790, 66023, 67549). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Removed istio related addon manifests, as the directory is deprecated.

**What this PR does / why we need it**:
the cluster/addon directory is deprecated, hence removing istio related addon manifests underneath.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
None
```
2018-08-17 20:37:01 -07:00
Kubernetes Submit Queue ae9beb4663
Merge pull request #66023 from ibrasho/change-TRUNCATED-to-DATA+OMITTED
Automatic merge from submit-queue (batch tested with PRs 60790, 66023, 67549). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

change TRUNCATED to DATA+OMITTED in kubectl config view

**What this PR does / why we need it**:
Based on the discussion in #61573, this PR switches the replacement text for CA certificate data and client certificates and secrets printed using `kubectl config view`. Currently, `REDACTED` is used, which might give a false impression that the data is a secret (which is not true for the public certificates).

This PR changes `REDACTED` to `DATA+OMITTED`. The printed string is the base64 encoded string on the byte stream. Some trickery is involved to print a readable string (refer to [this comment](https://github.com/kubernetes/kubernetes/pull/66023/files#diff-aec000ca3f293c94dcd99b4a9d1c5c3cL86) for more info).

**Which issue(s) this PR fixes**:
Fixes #61573

**Special notes for your reviewer**:


**Release note**:
```release-note
Switched certificate data replacement from "REDACTED" to "DATA+OMITTED"
```
2018-08-17 20:36:57 -07:00
Kubernetes Submit Queue 7fa1b77a1f
Merge pull request #60790 from hanxiaoshuai/addut03051
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add unit test for function AsScaledInt64

**What this PR does / why we need it**:
add unit test for function AsScaledInt64
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-08-17 20:22:16 -07:00
Kubernetes Submit Queue ef388fee53
Merge pull request #66948 from mohamed-mehany/anti-affinity-optimization
Automatic merge from submit-queue (batch tested with PRs 67041, 66948). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Anti affinity optimization

**What this PR does / why we need it**:
This pull request aims to optimize the performance of anti-affinity rules lookup of existing pods
This optimization maps the topology values to a list of pods running on nodes that match this value and store that map in the pod metadata. Accordingly, when validating anti-affinity rules of existing pods we will only check those running on nodes with similar topology values to the current candidate (node) for scheduling.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #63937

**Special notes for your reviewer**:
/sig scalability
/sig scheduling
**Release note**:

```release-note
improve performance of anti-affinity predicate of default scheduler.
```
2018-08-17 19:14:08 -07:00
Kubernetes Submit Queue 363e34124d
Merge pull request #67041 from tristanburgess/50342_error_handling_refinement_for_serialization_encode
Automatic merge from submit-queue (batch tested with PRs 67041, 66948). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

50342: Establish '406 Not Acceptable' response for protobuf serializa…

…tion 'errNotMarshalable'

     - Added metav1.Status() that enforces '406 Not Acceptable' response if
    protobuf serialization is not fully supported for the API resource type.
     - JSON and YAML serialization are supposed to be more completely baked
    in, so serialization involving those, and general errors with seralizing
    protobuf, will return '500 Internal Server Error'.
	- If serialization failure occurs and original HTTP status code is
    error, use the original status code, else use the serialization failure
    status code.
     - Write encoded API responses to intermediate buffer
     - Use apimachinery/runtime::Encode() instead of
    apimachinery/runtime/protocol::Encode() in
    apiserver/endpoints/handlers/responsewriters/writers::SerializeObject()
     - This allows for intended encoder error handling to fully work, facilitated by
    apiserver/endpoints/handlers/responsewriters/status::ErrorToAPIResponse() before officially
    writing to the http.ResponseWriter
     - The specific part that wasn't working by ErrorToAPIResponse() was the
    HTTP status code set. A direct call to
    http.ResponseWriter::WriteHeader(statusCode) was made in
    SerializeObject() with the original response status code, before
    performing the encode. Once this
    method is called, it can not again update the status code at a later
    time, with say, an erro status code due to encode failure.
     - Updated relevant apiserver unit test to reflect the new behavior
    (TestWriteJSONDecodeError())
     - Add build deps from make update for protobuf serializer



**What this PR does / why we need it**:
This PR fixes a bug that was blocking extensible error handling in the case that serializing response data fails, and implements a '406 Not Acceptable' status code response if protobuf marshal definitions are not implemented for an API resource type. See commit message for further details.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #50342 

**Special notes for your reviewer**:

**Release note**:

```release-note
Fixed a bug that was blocking extensible error handling when serializing API responses error out. Previously, serialization failures always resulted in the status code of the original response being returned. Now, the following behavior occurs:
   - If the serialization type is application/vnd.kubernetes.protobuf, and protobuf marshaling is not implemented for the requested API resource type, a '406 Not Acceptable is returned'.
   - If the serialization type is 'application/json':
        - If serialization fails, and the original status code was an failure (e.g. 4xx or 5xx), the original status code will be returned.
        - If serialization fails, and the original status code was not a failure (e.g. 2xx), the status code of the serialization failure will be returned. By default, this is '500 Internal Server Error', because JSON serialization is our default, and not supposed to be implemented on a type-by-type basis.

```
2018-08-17 19:14:04 -07:00
yanxuean efca28f8a5 remove deplicate code for PodRequestsAndLimits
Signed-off-by: yanxuean <yan.xuean@zte.com.cn>
2018-08-18 09:22:28 +08:00
Kubernetes Submit Queue c2c0d9e441
Merge pull request #67472 from islinwb/pkg_proxy_util-add-ut
Automatic merge from submit-queue (batch tested with PRs 66733, 67472). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add ut for PortPart()

**What this PR does / why we need it**:
improve ut coverage

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-08-17 17:50:05 -07:00
Kubernetes Submit Queue 8c1bfeb0cf
Merge pull request #66733 from bsalamat/subset_nodes
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add a feature to the scheduler to score fewer than all nodes in every scheduling cycle

**What this PR does / why we need it**:
Today, the scheduler scores all the nodes in the cluster in every scheduling cycle (every time a posd is attempted). This feature implements a mechanism in the scheduler that allows scoring fewer than all nodes in the cluster. The scheduler stops searching for more nodes once the configured number of feasible nodes are found. This can help improve the scheduler's performance in large clusters (several hundred nodes and larger).
This PR also adds a new structure to the scheduler's cache, called NodeTree, that allows scheduler to iterate over various nodes in different zones in a cluster. This is needed to avoid scoring the same set of nodes in every scheduling cycle.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #66627 

**Special notes for your reviewer**:
This is a large PR, but broken into a few logical commits. Reviewing would be easier if you review by commits.

**Release note**:

```release-note
Add a feature to the scheduler to score fewer than all nodes in every scheduling cycle. This can improve performance of the scheduler in large clusters.
```
2018-08-17 17:21:32 -07:00
Ahmad Diaa b4c7d190cd using set instead of lists for topologyPairsMaps attributes 2018-08-18 01:02:48 +02:00
Ahmad Diaa 0f4c3064fd created struct for topologyPairs maps 2018-08-18 01:02:48 +02:00
Ahmad Diaa f6659e4543 further enhancements removing matchingTerms from metadata 2018-08-18 01:02:47 +02:00
Mohamed Mehany 3fb6912d08 add topologyValue map to reduce search space 2018-08-18 01:02:46 +02:00
Kubernetes Submit Queue 8b52ca1ab9
Merge pull request #67433 from deads2k/controller-02-quotadiscovery
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

allow failed discovery on initial quota controller start

Fixes #65005

Aggregated API servers now correctly provide 503s on discovery endpoints for groups that cannot be reached.  This means that the kube-controller-manager process is now sensitive to discovery failures in the quota controller.  This change allows discovery failures in the initial quota replenishment controller resource discovery.

@liggitt suspects similar races exist to those he found GC last release, but this pull doesn't make that better or worse.

@kubernetes/sig-api-machinery-bugs
2018-08-17 16:01:31 -07:00
Kubernetes Submit Queue 6b41352679
Merge pull request #65779 from cblecker/mergo-update
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

update github.com/imdario/mergo to v0.3.5

**What this PR does / why we need it**:
Updates github.com/imdario/mergo library to v0.3.5. We were pinned because of a functionality change in the dependency, however, a new function was introduced with similar functionality to the old.

There is apparently some Debian packaging issues (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=878254) because of this. I'm still not clear what those are, unless they are forcing the library to update, as opposed to using our `vendor/`.

That said, this will allow for some of those vendor conflicts to resolve for anyone else who is using client-go, so that's at least worthwhile.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
fixes #27543, fixes https://github.com/kubernetes/client-go/issues/431

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2018-08-17 14:43:48 -07:00
Kubernetes Submit Queue 008165027b
Merge pull request #67540 from mborsz/wait3
Automatic merge from submit-queue (batch tested with PRs 67194, 67540). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix framework.WaitForDaemonSets

* when daemonsets are not ready, wait for really them
* swap parameters in Logf so that they are more readable.



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-08-17 13:23:09 -07:00
Kubernetes Submit Queue 567b3025ce
Merge pull request #67194 from yue9944882/refactor/storage-object-inuse-protection-externalize
Automatic merge from submit-queue (batch tested with PRs 67194, 67540). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Externalize PV/PVC informer for StorageObjectInUseProtection & NodeAuthorizer

**What this PR does / why we need it**:

/sig api-machinery

ref: #66680

We move on and make the change happen for PV/PVC. 

> PV: NodeAuthorizer, StorageObjectInUseProtection
> PVC: StorageObjectInUseProtection

1. Externalize PV and PVC informers for `StorageObjectInUseProtection`
2. Copy utility functions for PV from `pkg/api/persistentvolume` to `pkg/api/v1/persistentvolume` and make it accept external types.
3. Bump PV informer in NodeAuthorizer

**Release note**:

```release-note
NONE
```
2018-08-17 13:23:06 -07:00
Tristan Burgess bcdf3bb643 50342: Establish '406 Not Acceptable' response for protobuf serialization 'errNotMarshalable'
- Added metav1.Status() that enforces '406 Not Acceptable' response if
    protobuf serialization is not fully supported for the API resource type.
     - JSON and YAML serialization are supposed to be more completely baked
    in, so serialization involving those, and general errors with seralizing
    protobuf, will return '500 Internal Server Error'.
	- If serialization failure occurs and original HTTP status code is
    error, use the original status code, else use the serialization failure
    status code.
     - Write encoded API responses to intermediate buffer
     - Use apimachinery/runtime::Encode() instead of
    apimachinery/runtime/protocol::Encode() in
    apiserver/endpoints/handlers/responsewriters/writers::SerializeObject()
     - This allows for intended encoder error handling to fully work, facilitated by
    apiserver/endpoints/handlers/responsewriters/status::ErrorToAPIResponse() before officially
    writing to the http.ResponseWriter
     - The specific part that wasn't working by ErrorToAPIResponse() was the
    HTTP status code set. A direct call to
    http.ResponseWriter::WriteHeader(statusCode) was made in
    SerializeObject() with the original response status code, before
    performing the encode. Once this
    method is called, it can not again update the status code at a later
    time, with say, an erro status code due to encode failure.
     - Updated relevant apiserver unit test to reflect the new behavior
    (TestWriteJSONDecodeError())
     - Add build deps from make update for protobuf serializer

50342: Code review suggestion impl
 - Ensure that http.ResponseWriter::Header().Set() is called before http.ResponseWriter::WriteHeader()
    - This will avert a potential issue where changing the response media type to text/plain wouldn't work.
    - We want to respond with plain text if serialization fails of the original response, and serialization also fails for the resultant error response.

50342: wrapper for http.ResponseWriter
  - Prevent potential performance regression caused by modifying encode to use a buffer instead of streaming
    - This is achieved by creating a wrapper type for http.ResponseWriter that will use WriteHeader(statusCode) on the first
    call to Write(). Thus, on encode success, Write() will write the original statusCode. On encode failure, we pass control
    onto responsewriters::errSerializationFatal(), which will process the error to obtain potentially a new status code, depending
    on whether or not the original status code was itself an error.

50342: code review suggestions
  - Remove historical note from unit test comment
  - Don't export httpResponseWriterWithInit type (for now)
2018-08-17 16:21:07 -04:00
Yu Liao 77d76c05b4 Removed istio related addon manifests, as the directory is deprecated. 2018-08-17 13:04:26 -07:00
André Bauer 9b5c5650eb use image 2.2.0 again 2018-08-17 21:49:44 +02:00
André Bauer 6ab218d4dd fixed typo 2018-08-17 21:48:44 +02:00
Kubernetes Submit Queue 1737a43324
Merge pull request #66876 from juanvallejo/jvallejo/prototype-plugins
Automatic merge from submit-queue (batch tested with PRs 67062, 67169, 67539, 67504, 66876). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update the kubectl plugin mechanism

**Release note**:
```release-note
The plugin mechanism functionality to closely follow the git plugin design
```

Replace the existing plugin mechanism with the design proposed in https://github.com/kubernetes/community/pull/2437.

~~_The full implementation of the plugin mechanism itself is entirely contained within the first commit._~~

## Walkthrough

Under the new design, there is no plugin installation or loading required to use plugins.
A plugin is simply any executable file on a user's PATH whose name begins with `kubectl-`.
- Plugins receive the inherited environment from the `kubectl` binary. All environment variables
accessible by `kubectl` become accessible by the plugin.
- Plugins decide which command path they wish to implement based on their name. For example, a plugin wanting to provide a new command `foo`, would simply be named `kubectl-foo`.

### Creating a plugin

Below is an example plugin, that we will use for this walkthrough. Plugins may be written in any language, and handle arguments and flags in any way, optionally (as a convention) providing a way to retrieve their version via a `version` subcommand.

```bash
#!/bin/bash

# optional argument handling
if [[ "$1" == "version" ]]
then
    echo "1.0.0"
    exit 0
fi

# optional argument handling
if [[ "$1" == "config" ]]
then
    echo $KUBECONFIG
    exit 0
fi

echo "I am a plugin named kubectl-foo"
```

### Using a plugin

To use a plugin, simply make it executable:

```bash
sudo chmod +x ./kubectl-foo
```

and place it anywhere in your PATH:

```bash
sudo mv ./kubectl-foo /usr/local/bin
```

You may now invoke your plugin as a `kubectl` command:

```bash
$ kubectl foo
I am a plugin named kubectl-foo
```

All args and flags are passed as-is to the executable:

```bash
$ kubectl foo version
1.0.0
```

All environment variables are also passed as-is to the executable:

```bash
$ export KUBECONFIG=~/.kube/config
$ kubectl foo config
/home/<user>/.kube/config

$ KUBECONFIG=/etc/kube/config kubectl foo config
/etc/kube/config
```

Additionally, the first argument that is passed to a plugin will always be the full path to the location where it was invoked (`$0` would equal `/usr/local/bin/kubectl-foo` in our example above).

### Plugin discoverability

Seeing as how the `kubectl plugin` command is left as a no-op with this PR (perhaps it could serve as an entrypoint towards additional plugin functionality in the future), a small subcommand has been included that _lists all available plugin executables on a user's PATH_, along with any warnings it finds.

Example usage of this new subcommand is included below:

```bash
$ kubectl plugin list
The following kubectl-compatible plugins are available:

test/fixtures/pkg/kubectl/plugins/kubectl-foo
plugins/kubectl-foo
  - warning: plugins/kubectl-foo is overshadowed by a similarly named plugin: test/fixtures/pkg/kubectl/plugins/kubectl-foo
plugins/kubectl-invalid
  - warning: plugins/kubectl-invalid identified as a kubectl plugin, but it is not executable
plugins/kubectl-bar

error: 2 plugin warnings were found
```

cc @kubernetes/kubectl-maintainers @kubernetes/sig-cli-pr-reviews @soltysh @seans3 @mengqiy
2018-08-17 11:58:22 -07:00