Commit Graph

65028 Commits (3989ec66eba9a9da9ae85fb981ee776e4579ec41)

Author SHA1 Message Date
yankaiz 3989ec66eb Add MAX_PODS_PER_NODE env allowing kubelet to be max-pods aware. 2018-05-04 11:09:55 -07:00
Kubernetes Submit Queue 484f62a568
Merge pull request #63333 from deads2k/api-14-snip
Automatic merge from submit-queue (batch tested with PRs 63421, 63432, 63333). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

update tests to be specific about the versions they are testing

When setting up tests, you want to rely on your own scheme.  This eliminates coupling to floating versions which gives unnecessary flexibility in most cases and prevents testing all the versions you need.

@liggitt  scrubs unnecessary deps.

```release-note
NONE
```
2018-05-04 10:52:10 -07:00
Kubernetes Submit Queue 3945a241aa
Merge pull request #63432 from deads2k/cli-46-printerinterface
Automatic merge from submit-queue (batch tested with PRs 63421, 63432, 63333). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

 slim printer interface down to printing

The printer interface only needs to know how to print.  This reduces it's interface to scope it to printing.

@kubernetes/sig-cli-maintainers 
@soltysh @juanvallejo  you've been active in the area

/assign @juanvallejo 

```release-note
NONE
```
2018-05-04 10:52:07 -07:00
Kubernetes Submit Queue 068b7befa9
Merge pull request #63421 from liggitt/discover-completions
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Cache preferred resources, use in kubectl resource name autocomplete

Fixes #63145
Fixes https://github.com/kubernetes/kubectl/issues/357
Alternative to #61928 

* starts to unify preferred resource logic on top of ServerGroups()/ServerResourcesForGroupVersion() methods
* allows indicating a cached list of resources is acceptable when calling `kubectl api-resources` (default is still to rediscover)
* uses `kubectl api-resources` in bash completion

```sh
$ kubectl get [TAB][TAB]
apiservices.apiregistration.k8s.io                            networkpolicies.extensions
certificatesigningrequests.certificates.k8s.io                networkpolicies.networking.k8s.io
clusterrolebindings.rbac.authorization.k8s.io                 nodes
clusterroles.rbac.authorization.k8s.io                        persistentvolumeclaims
componentstatuses                                             persistentvolumes
configmaps                                                    poddisruptionbudgets.policy
controllerrevisions.apps                                      pods
cronjobs.batch                                                podsecuritypolicies.extensions
customresourcedefinitions.apiextensions.k8s.io                podsecuritypolicies.policy
daemonsets.apps                                               podtemplates
daemonsets.extensions                                         replicasets.apps
deployments.apps                                              replicasets.extensions
deployments.extensions                                        replicationcontrollers
endpoints                                                     resourcequotas
events                                                        rolebindings.rbac.authorization.k8s.io
events.events.k8s.io                                          roles.rbac.authorization.k8s.io
horizontalpodautoscalers.autoscaling                          secrets
ingresses.extensions                                          serviceaccounts
initializerconfigurations.admissionregistration.k8s.io        services
jobs.batch                                                    statefulsets.apps
limitranges                                                   storageclasses.storage.k8s.io
mutatingwebhookconfigurations.admissionregistration.k8s.io    validatingwebhookconfigurations.admissionregistration.k8s.io
namespaces                                                    volumeattachments.storage.k8s.io
```


```release-note
NONE
```
2018-05-04 10:08:53 -07:00
Kubernetes Submit Queue bc56947e8d
Merge pull request #63405 from dims/update-to-latest-gophercloud-may-3-2018
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update to latest Gophercloud

**What this PR does / why we need it**:

periodic update to latest gophercloud. The changes are as follows:
6da026c32e...781450b3c4

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-05-04 09:03:44 -07:00
David Eads e1fca70506 slim down printer interface 2018-05-04 11:15:35 -04:00
Jordan Liggitt 0dd652cffd
Fix bash completion with --all-namespaces 2018-05-04 09:57:57 -04:00
Jordan Liggitt 4dfd26a4f2
Switch kubectl resource completion to discovery 2018-05-04 09:57:36 -04:00
Kubernetes Submit Queue 54cf942a05
Merge pull request #63399 from andyzhangx/mount-windows-test-fix
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

use TempDir func in mount_windows_test.go

**What this PR does / why we need it**:
Use `c:\tmp` dir is not correct in windows test, this PR use `ioutil.TempDir("", xx)` to create temp dir instead.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```
none
```

/assign @jsafrane @msau42
2018-05-04 05:30:42 -07:00
Kubernetes Submit Queue 194e640bf1
Merge pull request #63339 from sjenning/remove-mount-success-event
Automatic merge from submit-queue (batch tested with PRs 63298, 63339). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubelet: volume: do not create event on mount success

xref https://github.com/openshift/origin/issues/19497

There is an argument to be made that we should not emit events on volume mount successes.  Many pods have lots of volumes that generate lots of events, especially if the pods are short lived (batch, etc). This creates load on the API server and add noise in the events.

Creating events on volume mount failures is obviously a good idea.  Seems like the absence of such events or the "Started container" event implies mount success for all volumes.

/sig node
/sig storage
/release-note-none

@smarterclayton @jsafrane @saad-ali
2018-05-04 04:42:06 -07:00
Kubernetes Submit Queue 1929e0d86d
Merge pull request #63298 from dims/kubelet-remove-unused-code
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubelet - Remove unused code

**What this PR does / why we need it**:

Looks like we have a bunch of unused methods. Let's clean them up

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-05-04 04:20:06 -07:00
Jordan Liggitt 918df97a0f
Allow api-resources to return cached data 2018-05-04 00:06:39 -04:00
Jordan Liggitt 74b7cec0c6
Make ServerPreferred[Namespaced]Resources logic and caches consistent 2018-05-04 00:06:39 -04:00
Kubernetes Submit Queue bb4745da99
Merge pull request #63407 from nicksardo/change-setsslcert
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

GCE: Pass certificate URLs instead of the certificate structs

**What this PR does / why we need it**:
Simplify callers by passing in only what's necessary - a slice of URLs.

**Special notes for your reviewer**:
/assign @MrHohn 
/cc @MrHohn 

**Release note**:
```release-note
NONE
```
2018-05-03 18:41:39 -07:00
Kubernetes Submit Queue b52ebfa28e
Merge pull request #63312 from deads2k/cli-44-encoder
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove unnecessary encoder

Followups to https://github.com/kubernetes/kubernetes/pull/63105
2018-05-03 16:11:17 -07:00
Kubernetes Submit Queue 2f694e8fa1
Merge pull request #63404 from kubernetes/revert-60372-sttts-scheduler-option-config
Automatic merge from submit-queue (batch tested with PRs 61455, 63346, 63130, 63404). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "scheduler: align with ctrl-managers and apiservers, add https+auth in options"

This PR completely broke scheduler performance:
https://k8s-testgrid.appspot.com/sig-scalability-kubemark#kubemark-5000

Pod startup time exploded to ~20s.

Ref: https://github.com/kubernetes/kubernetes/pull/60372
2018-05-03 14:24:13 -07:00
Kubernetes Submit Queue a244d8a48f
Merge pull request #63130 from vikaschoudhary16/dp_e2e_alloc
Automatic merge from submit-queue (batch tested with PRs 61455, 63346, 63130, 63404). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[Device-Plugin]: Extend e2e test to cover node allocatables

**What this PR does / why we need it**:
 Extends device plugin e2e to cover node allocatable
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
None
```
/sig node
/area hw-accelerators
/cc @jiayingz @vishh @RenaudWasTaken
2018-05-03 14:24:10 -07:00
Kubernetes Submit Queue 89e6895e1c
Merge pull request #63346 from roycaihw/bump-kube-openapi
Automatic merge from submit-queue (batch tested with PRs 61455, 63346, 63130, 63404). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump kube-openapi dependency

**What this PR does / why we need it**:
Pick up https://github.com/kubernetes/kube-openapi/pull/64

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #63218 

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

/sig api-machinery
/cc @mbohlool @liggitt
2018-05-03 14:24:07 -07:00
Kubernetes Submit Queue 4e3efbe364
Merge pull request #61455 from liggitt/uid-conflict
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Deprecate repair-malformed-updates flag, move object meta mutation into BeforeCreate

closes #23297

```release-note
NONE
```
2018-05-03 13:41:07 -07:00
Davanum Srinivas ddfcf1e193 Update to latest Gophercloud
6da026c32e...781450b3c4
2018-05-03 16:37:01 -04:00
Nick Sardo f9d1f7eb75 Pass certificate URLs instead of the certificate structs 2018-05-03 13:14:55 -07:00
Kubernetes Submit Queue 88c25ca2d9
Merge pull request #63403 from deads2k/server-14-creationpower
Automatic merge from submit-queue (batch tested with PRs 63258, 63398, 63403). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

don't block creation on lack of delete powers

Create and delete aren't the same thing, but the alternatives seem worse. This stops checking for deletion powers on create.  You still have to know the UID to create an effective ownerref, so name hunting is unrealistic.

@kubernetes/sig-api-machinery-pr-reviews 

```release-note
owner references can be set during creation without deletion power
```
2018-05-03 12:48:16 -07:00
Kubernetes Submit Queue eaa3aa382a
Merge pull request #63398 from deads2k/cli-45-conversion
Automatic merge from submit-queue (batch tested with PRs 63258, 63398, 63403). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove unnessary kubectl conversions

Working with unstructured types means that conversion can be eliminated.  This pulls  it from places I noticed.

/assign @juanvallejo 

@kubernetes/sig-cli-misc 

```release-note
NONE
```
2018-05-03 12:48:13 -07:00
Kubernetes Submit Queue a29b4607df
Merge pull request #63258 from deads2k/cli-42-simplify-buidler
Automatic merge from submit-queue (batch tested with PRs 63258, 63398, 63403). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

stop using Info.Mappings when they may not be present

On a resource builder, you cannot logically have a RESTMapping or a Client if you're running a local action.  Reliance on the `info.Client` and `info.Mapping` is a bug we need to fix.  This updates the docs and eliminates unnecessary reliance.  Other hits I found didn't have `--local` options, so we're safe or had them and and were already broken.  I think we'll be able to help them after making our creation flow obvious.

@kubernetes/sig-cli-maintainers 

```release-note
NONE
```
2018-05-03 12:48:10 -07:00
Wojciech Tyczynski 7b93d81a97
Revert "scheduler: align with ctrl-managers and apiservers, add https+auth in options" 2018-05-03 21:05:00 +02:00
vikaschoudhary16 b953f852f5 [Device-Plugin]: Extend e2e test to cover node allocatables 2018-05-03 14:19:29 -04:00
Kubernetes Submit Queue 33d85b01fe
Merge pull request #63098 from shubheksha/fix/62916-replace-path-with-filepath-aws-azure
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

aws, azure: replace path with filepath

**What this PR does / why we need it**:
This PR replaces usage of `path` with `filepath` as it uses OS-specific path separators.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #62916 

**Special notes for your reviewer**:
This PR addresses the following volume plugins:
- `pkg/volume/aws_ebs`
- `pkg/volume/azure_dd`
- `pkg/volume/azure_file`

**Release note**:

```release-note
NONE
```
2018-05-03 11:16:10 -07:00
Kubernetes Submit Queue 865321c2d6
Merge pull request #61940 from alinbalutoiu/master
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add support for CNI on Windows Server 2016 RTM

**What this PR does / why we need it**:
Windows Server 2016 RTM has limited CNI support. This PR makes it possible for the CNI plugin to be used to setup POD networking on Windows Server 2016 RTM (build number 14393).

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #61939

**Special notes for your reviewer**:
The old mode is not supported and tested on Windows Server 2016 RTM. This change allows the CNI plugin to be used on Windows Server 2016 RTM to retrieve the container IP instead of using workarounds (docker inspect).

CNI support has been added for Windows Server 2016 version 1709 (build number 16299), this patch will just allow the same support for older build numbers.

Windows Server 2016 RTM has a longer lifecycle (LTS) than Windows Server 2016 version 1709.
https://support.microsoft.com/en-us/lifecycle/search/19761 vs https://support.microsoft.com/en-us/lifecycle/search/20311


**Release note**:

```release-note
NONE
```
2018-05-03 10:17:03 -07:00
David Eads 1f4f22f72d don't block creation on lack of delete powers 2018-05-03 12:04:04 -04:00
Kubernetes Submit Queue 592c39bccc
Merge pull request #62541 from filbranden/cgroupname1
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use a []string for CgroupName, which is a more accurate internal representation

**What this PR does / why we need it**:

This is purely a refactoring and should bring no essential change in behavior.

It does clarify the cgroup handling code quite a bit.

It is preparation for further changes we might want to do in the cgroup hierarchy. (But it's useful on its own, so even if we don't do any, it should still be considered.)

**Special notes for your reviewer**:

The slice of strings more precisely captures the hierarchic nature of the cgroup paths we use to represent pods and their groupings.

It also ensures we're reducing the chances of passing an incorrect path format to a cgroup driver that requires a different path naming, since now explicit conversions are always needed.

The new constructor `NewCgroupName` starts from an existing `CgroupName`, which enforces a hierarchy where a root is always needed. It also performs checking on the component names to ensure invalid characters ("/" and "_") are not in use.

A `RootCgroupName` for the top of the cgroup hierarchy tree is introduced.

This refactor results in a net reduction of around 30 lines of code,
mainly with the demise of ConvertCgroupNameToSystemd which had fairly
complicated logic in it and was doing just too many things.

There's a small TODO in a helper `updateSystemdCgroupInfo` that was introduced to make this commit possible. That logic really belongs in libcontainer, I'm planning to send a PR there to include it there. (The API already takes a field with that information, only that field is only processed in cgroupfs and not systemd driver, we should fix that.)

Tested: By running the e2e-node tests on both Ubuntu 16.04 (with cgroupfs driver) and CentOS 7 (with systemd driver.)

**NOTE**: I only tested this with dockershim, we should double-check that this works with the CRI endpoints too, both in cgroupfs and systemd modes.

/assign @derekwaynecarr 
/assign @dashpole 
/assign @Random-Liu 

**Release note**:

```release-note
NONE
```
2018-05-03 08:16:45 -07:00
Kubernetes Submit Queue a16c348acf
Merge pull request #56137 from containscafeine/remove-punctuation-from-error
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove punctuation from the end of an error string

```release-note
NONE
```
2018-05-03 06:37:57 -07:00
andyzhangx 3db442bcd0 use TempDir func in mount_windows_test.go
fix comments
2018-05-03 12:58:32 +00:00
David Eads abe9e0d25b stop using Info.Mappings when they may not be present 2018-05-03 08:12:05 -04:00
David Eads a9a99681e4 remove unnessary kubectl conversions 2018-05-03 08:06:53 -04:00
Kubernetes Submit Queue 398ced1dcd
Merge pull request #63392 from cofyc/fixlocaltest
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix test "should fail due to non-existent path" under gce-slow

**What this PR does / why we need it**:

PR #62903 changed error string GetMountRefs() returned, which broke test `should fail due to non-existent path`.
Remove error string check to fix test.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

```
go run hack/e2e.go -- \
    --provider=local \ 
    --test \
    --test_args="--ginkgo.focus=PersistentVolumes-local.*should\sfail\sdue\sto\snon-existent\spath --clean-start=true"
```

passed now.

**Release note**:

```release-note
NONE
```
2018-05-03 04:33:22 -07:00
Kubernetes Submit Queue fb85e69b9b
Merge pull request #63376 from liggitt/another-mapper-err
Automatic merge from submit-queue (batch tested with PRs 63073, 63376). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

don't reuse resource builder in describe

fixes #63363 

```release-note
NONE
```
2018-05-03 01:49:06 -07:00
Yecheng Fu a84ebb5193 PR #62903 changed error string GetMountRefs() returned, which broke test
`should fail due to non-existent path`.
Remove error string check to fix test.
2018-05-03 16:33:28 +08:00
Kubernetes Submit Queue 4f56127582
Merge pull request #63073 from andyxning/refactor_grpc_dial_with_dialcontext
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

refactor device plugin grpc dial with dialcontext

**What this PR does / why we need it**:
Refactor grpc `dial` with `dialContext` as `grpc.WithTimeout` has been deprecated by:
> use DialContext and context.WithTimeout instead.

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-05-03 01:16:34 -07:00
Kubernetes Submit Queue 456b56a2fb
Merge pull request #63227 from karataliu/nodec
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix ensure by provider id

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #63226

**Special notes for your reviewer**:

cc @adnavare 

**Release note**:

```release-note
NONE
```
2018-05-03 00:08:48 -07:00
Kubernetes Submit Queue 51e97bfa6b
Merge pull request #63377 from Cynerva/gkk/fix-status-messages
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

juju: Make kubernetes-master status handling more robust

**What this PR does / why we need it**:

This fixes a charm hook failure that can occur after restarting kubernetes-master units in Juju-based Kubernetes deployments.

This should also hopefully make our status handling in kubernetes-master more robust in general, by making sure all of our `blocked`, `waiting`, and `active` status messages are handled in one place, on hook exit.

**Which issue(s) this PR fixes**:

https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/523

**Release note**:

```release-note
NONE
```
2018-05-02 23:06:41 -07:00
Kubernetes Submit Queue 4299a0518c
Merge pull request #63261 from dougm/cloudprovider-notimplemented
Automatic merge from submit-queue (batch tested with PRs 61725, 63261). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use cloudprovider.NotImplemented in AddSSHKeyToAllInstances

**What this PR does / why we need it**:

Minor cleanup: In looking at implementations of this method,
noticed a few providers created their own errors instead of using cloudprovider.NotImplemented

**Release note**:

```release-note
NONE
```
2018-05-02 22:16:06 -07:00
Kubernetes Submit Queue 1e9acb63f1
Merge pull request #61725 from liggitt/pod-deletion-contention
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Pod deletion can be contended, causing test failure

Observed this running e2e tests downstream

```release-note
NONE
```
2018-05-02 21:31:33 -07:00
Kubernetes Submit Queue ab9f64afa9
Merge pull request #63213 from filmil/oidc-dist-claims
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Implements OIDC distributed claims.

Next step to enable this feature is to enable claim caching.

A distributed claim allows the OIDC provider to delegate a claim to a
separate URL.  Distributed claims are of the form as seen below, and are
defined in the OIDC Connect Core 1.0, section 5.6.2.

See: https://openid.net/specs/openid-connect-core-1_0.html#AggregatedDistributedClaims

Example claim:

```
{
  ... (other normal claims)...
  "_claim_names": {
    "groups": "src1"
  },
  "_claim_sources": {
    "src1": {
      "endpoint": "https://www.example.com",
      "access_token": "f005ba11"
    },
  },
}
```

Example response to a followup request to https://www.example.com is a
JWT-encoded claim token:

```
{
  "iss": "https://www.example.com",
  "aud": "my-client",
  "groups": ["team1", "team2"],
  "exp": 9876543210
}
```

Apart from the indirection, the distributed claim behaves exactly
the same as a standard claim.  For Kubernetes, this means that the
token must be verified using the same approach as for the original OIDC
token.  This requires the presence of "iss", "aud" and "exp" claims in
addition to "groups".

All existing OIDC options (e.g. groups prefix) apply.

Any claim can be made distributed, even though the "groups" claim is
the primary use case.

Allows groups to be a single string due to
https://github.com/kubernetes/kubernetes/issues/33290, even though
OIDC defines "groups" claim to be an array of strings. So, this will
be parsed correctly:

```
{
  "iss": "https://www.example.com",
  "aud": "my-client",
  "groups": "team1",
  "exp": 9876543210
}
```

Expects that distributed claims endpoints return JWT, per OIDC specs.

In case both a standard and a distributed claim with the same name
exist, standard claim wins.  The specs seem undecided about the correct
approach here.

Distributed claims are resolved serially.  This could be parallelized
for performance if needed.

Aggregated claims are silently skipped.  Support could be added if
needed.



**What this PR does / why we need it**: Makes it possible to retrieve many group memberships by offloading to a dedicated backend for groups resolution.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #62920

**Special notes for your reviewer**:
There are a few TODOs that seem better handled in separate commits.

**Release note**:

```release-note
Lays groundwork for OIDC distributed claims handling in the apiserver authentication token checker.

A distributed claim allows the OIDC provider to delegate a claim to a
separate URL.  Distributed claims are of the form as seen below, and are
defined in the OIDC Connect Core 1.0, section 5.6.2.

For details, see: 
http://openid.net/specs/openid-connect-core-1_0.html#AggregatedDistributedClaims
```
2018-05-02 20:41:51 -07:00
Kubernetes Submit Queue 692b34825f
Merge pull request #63375 from liggitt/diff-limit
Automatic merge from submit-queue (batch tested with PRs 62657, 63278, 62903, 63375). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

ensure diff output includes the portion that differs

When using ObjectReflectDiff() on objects with long string fields, the
80 character limit on diffs will commonly hide the actual difference
between the fields and require that the dev change which diff function
is used to see what the issue was. This defeats the purpose of printing
the diff between objects.
2018-05-02 20:13:13 -07:00
Kubernetes Submit Queue 186dd7beb1
Merge pull request #62903 from cofyc/fixfsgroupcheckinlocal
Automatic merge from submit-queue (batch tested with PRs 62657, 63278, 62903, 63375). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add more volume types in e2e and fix part of them.

**What this PR does / why we need it**:

- Add dir-link/dir-bindmounted/dir-link-bindmounted/bockfs volume types for e2e tests.
- Fix fsGroup related e2e tests partially.
- Return error if we cannot resolve volume path.
  - Because we should not fallback to volume path, if it's a symbolic link, we may get wrong results.

To safely set fsGroup on local volume, we need to implement these two methods correctly for all volume types both on the host and in container:

- get volume path kubelet can access
  - paths on the host and in container are different
- get mount references
  - for directories, we cannot use its mount source (device field) to identify mount references, because directories on same filesystem have same mount source (e.g. tmpfs), we need to check filesystem's major:minor and directory root path on it

Here is current status:

| | (A) volume-path (host) | (B) volume-path (container) | (C) mount-refs (host) | (D) mount-refs (container) |
| --- | --- | --- | --- | --- |
| (1) dir | OK | FAIL | FAIL | FAIL |
| (2) dir-link | OK | FAIL | FAIL | FAIL |
| (3) dir-bindmounted | OK | FAIL | FAIL | FAIL |
| (4) dir-link-bindmounted | OK | FAIL | FAIL | FAIL |
| (5) tmpfs| OK | FAIL | FAIL | FAIL |
| (6) blockfs| OK | FAIL | OK | FAIL |
| (7) block| NOTNEEDED | NOTNEEDED | NOTNEEDED | NOTNEEDED |
| (8) gce-localssd-scsi-fs| NOTTESTED | NOTTESTED | NOTTESTED | NOTTESTED |

- This PR uses `nsenter ... readlink` to resolve path in container as @msau42  @jsafrane [suggested](https://github.com/kubernetes/kubernetes/pull/61489#pullrequestreview-110032850). This fixes B1:B6 and D6, , the rest will be addressed in https://github.com/kubernetes/kubernetes/pull/62102.
- C5:D5 marked `FAIL` because `tmpfs` filesystems can share same mount source, we cannot rely on it to check mount references. e2e tests passes due to we use unique mount source string in tests.
- A7:D7 marked `NOTNEEDED` because we don't set fsGroup on block devices in local plugin. (TODO: Should we set fsGroup on block device?)
- A8:D8 marked `NOTTESTED` because I didn't test it, I leave it to `pull-kubernetes-e2e-gce`. I think it should be same as `blockfs`.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-05-02 20:13:11 -07:00
Kubernetes Submit Queue 4018211800
Merge pull request #63278 from feiskyer/azure-lb-new-interface
Automatic merge from submit-queue (batch tested with PRs 62657, 63278, 62903, 63375). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use new Azure SDK APIs for load balancer and public IP operations

**What this PR does / why we need it**:

#63063 updated Azure SDK to a stable version. After that, we should also update existing clients to use new SDK APIs.

Without this, public IP listing will be blocked forever in some case.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

A following PR will be sent for other interfaces, e.g. routes and NSGs.

**Release note**:

```release-note
NONE
```
2018-05-02 20:13:07 -07:00
Kubernetes Submit Queue b5f61ac129
Merge pull request #62657 from matthyx/master
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update all script shebangs to use /usr/bin/env interpreter instead of /bin/interpreter

This is required to support systems where bash doesn't reside in /bin (such as NixOS, or the *BSD family) and allow users to specify a different interpreter version through $PATH manipulation.
https://www.cyberciti.biz/tips/finding-bash-perl-python-portably-using-env.html
```release-note
Use /usr/bin/env in all script shebangs to increase portability.
```
2018-05-02 19:44:32 -07:00
Dong Liu 9a6319b23c Update error assertation 2018-05-03 10:22:20 +08:00
Kubernetes Submit Queue 4558e419bc
Merge pull request #62892 from liggitt/node-authorizer-index
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add index to node-authorizer for high cardinality vertices

follow-up to https://github.com/kubernetes/kubernetes/pull/62856#issuecomment-382788780

explores adding an index to high-cardinality vertices in the node authorizer to reduce CPU usage for high density namespaces

* first commit is a refactor only - cc @mtaufen 
* second commit adds an optional per-vertex index we can maintain when there are sufficient outgoing edges.

benchmark results:
* shared_secret_via_pod cases are ~1000x faster
* throughput on processing of graph modifications is 50% higher
* there is more variance on graph modifications requiring index updates (though the 100 index-impacting graph modifications per second might be a higher-than-realistic write rate)

data profile (5000 pods per namespace, assigned to 5000 nodes, shared service account and secret):
```
        opts := sampleDataOpts{
                // To simulate high replication in a small number of namespaces:
               nodes:       5000,
               namespaces:  10,
               podsPerNode: 10,
...
```

command:
```
$ go test ./plugin/pkg/auth/authorizer/node/  -bench Authorization  -benchmem -v 
```

before
```
BenchmarkAuthorization/allowed_node_configmap-8                                  557 ns/op   530 B/op   11 allocs/op   3000000
BenchmarkAuthorization/allowed_configmap-8                                       539 ns/op   530 B/op   11 allocs/op   3000000
BenchmarkAuthorization/allowed_secret_via_pod-8                                  605 ns/op   529 B/op   11 allocs/op   3000000
BenchmarkAuthorization/allowed_shared_secret_via_pod-8                        215974 ns/op   792 B/op   19 allocs/op      5000
BenchmarkAuthorization/disallowed_node_configmap-8                               823 ns/op   694 B/op   17 allocs/op   2000000
BenchmarkAuthorization/disallowed_configmap-8                                    888 ns/op   691 B/op   17 allocs/op   2000000
BenchmarkAuthorization/disallowed_secret_via_pod-8                               868 ns/op   694 B/op   17 allocs/op   2000000
BenchmarkAuthorization/disallowed_shared_secret_via_pvc-8                       1216 ns/op   948 B/op   22 allocs/op   1000000
BenchmarkAuthorization/disallowed_pvc-8                                          918 ns/op   691 B/op   17 allocs/op   2000000
BenchmarkAuthorization/disallowed_pv-8                                          1095 ns/op   839 B/op   19 allocs/op   2000000
BenchmarkAuthorization/disallowed_attachment_-_no_relationship-8                 867 ns/op   677 B/op   16 allocs/op   2000000
BenchmarkAuthorization/disallowed_attachment_-_feature_disabled-8                220 ns/op   208 B/op    2 allocs/op  10000000
BenchmarkAuthorization/allowed_attachment_-_feature_enabled-8                    687 ns/op   594 B/op   12 allocs/op   2000000
BenchmarkAuthorization/contentious_allowed_node_configmap-8                      656 ns/op   530 B/op   11 allocs/op   3000000
BenchmarkAuthorization/contentious_allowed_configmap-8                           659 ns/op   529 B/op   11 allocs/op   2000000
BenchmarkAuthorization/contentious_allowed_secret_via_pod-8                      654 ns/op   529 B/op   11 allocs/op   2000000
BenchmarkAuthorization/contentious_allowed_shared_secret_via_pod-8            234308 ns/op  1022 B/op   22 allocs/op      5000
BenchmarkAuthorization/contentious_disallowed_node_configmap-8                  1118 ns/op   692 B/op   17 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_configmap-8                       1054 ns/op   692 B/op   17 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_secret_via_pod-8                  1059 ns/op   691 B/op   17 allocs/op   2000000
BenchmarkAuthorization/contentious_disallowed_shared_secret_via_pvc-8           1403 ns/op   949 B/op   22 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_pvc-8                             1058 ns/op   692 B/op   17 allocs/op   2000000
BenchmarkAuthorization/contentious_disallowed_pv-8                              1237 ns/op   838 B/op   19 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_attachment_-_no_relationship-8    1022 ns/op   676 B/op   16 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_attachment_-_feature_disabled-8    260 ns/op   209 B/op    2 allocs/op   5000000
BenchmarkAuthorization/contentious_allowed_attachment_-_feature_enabled-8        793 ns/op   594 B/op   12 allocs/op   2000000
--- BENCH: BenchmarkAuthorization
   node_authorizer_test.go:596: graph modifications during non-contention test: 0
   node_authorizer_test.go:593: graph modifications during contention test: 961
   node_authorizer_test.go:594: <1ms=774, <10ms=32, <25ms=14, <50ms=29, <100ms=62, <250ms=46, <500ms=2, <1000ms=1, >1000ms=1
```

after
```
BenchmarkAuthorization/allowed_node_configmap-8                                  629 ns/op   530 B/op   11 allocs/op   3000000
BenchmarkAuthorization/allowed_configmap-8                                       641 ns/op   530 B/op   11 allocs/op   3000000
BenchmarkAuthorization/allowed_secret_via_pod-8                                  591 ns/op   530 B/op   11 allocs/op   3000000
BenchmarkAuthorization/allowed_shared_secret_via_pod-8                           217 ns/op   160 B/op    1 allocs/op  10000000
BenchmarkAuthorization/disallowed_node_configmap-8                               912 ns/op   693 B/op   17 allocs/op   2000000
BenchmarkAuthorization/disallowed_configmap-8                                    913 ns/op   694 B/op   17 allocs/op   2000000
BenchmarkAuthorization/disallowed_secret_via_pod-8                               881 ns/op   691 B/op   17 allocs/op   2000000
BenchmarkAuthorization/disallowed_shared_secret_via_pvc-8                       1271 ns/op   952 B/op   22 allocs/op   1000000
BenchmarkAuthorization/disallowed_pvc-8                                          903 ns/op   694 B/op   17 allocs/op   2000000
BenchmarkAuthorization/disallowed_pv-8                                          1024 ns/op   836 B/op   19 allocs/op   1000000
BenchmarkAuthorization/disallowed_attachment_-_no_relationship-8                1187 ns/op   678 B/op   16 allocs/op   2000000
BenchmarkAuthorization/disallowed_attachment_-_feature_disabled-8                250 ns/op   209 B/op    2 allocs/op  10000000
BenchmarkAuthorization/allowed_attachment_-_feature_enabled-8                    694 ns/op   594 B/op   12 allocs/op   2000000
BenchmarkAuthorization/contentious_allowed_node_configmap-8                      732 ns/op   530 B/op   11 allocs/op   2000000
BenchmarkAuthorization/contentious_allowed_configmap-8                           820 ns/op   530 B/op   11 allocs/op   2000000
BenchmarkAuthorization/contentious_allowed_secret_via_pod-8                     1082 ns/op   531 B/op   11 allocs/op   1000000
BenchmarkAuthorization/contentious_allowed_shared_secret_via_pod-8               274 ns/op   160 B/op    1 allocs/op   5000000
BenchmarkAuthorization/contentious_disallowed_node_configmap-8                  1332 ns/op   693 B/op   17 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_configmap-8                       1534 ns/op   693 B/op   17 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_secret_via_pod-8                  1077 ns/op   692 B/op   17 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_shared_secret_via_pvc-8           1976 ns/op   949 B/op   22 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_pvc-8                             1297 ns/op   694 B/op   17 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_pv-8                              1632 ns/op   837 B/op   19 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_attachment_-_no_relationship-8    1394 ns/op   677 B/op   16 allocs/op   1000000
BenchmarkAuthorization/contentious_disallowed_attachment_-_feature_disabled-8    320 ns/op   209 B/op    2 allocs/op   5000000
BenchmarkAuthorization/contentious_allowed_attachment_-_feature_enabled-8       1055 ns/op   595 B/op   12 allocs/op   2000000
--- BENCH: BenchmarkAuthorization
    node_authorizer_test.go:629: graph modifications during non-contention test: 0
    node_authorizer_test.go:626: graph modifications during contention test: 1424
    node_authorizer_test.go:627: <1ms=0, <10ms=569, <25ms=340, <50ms=145, <100ms=101, <250ms=160, <500ms=61, <1000ms=42, >1000ms=6
```

```release-note
NONE
```
2018-05-02 18:54:46 -07:00
Kubernetes Submit Queue c968d99ee5
Merge pull request #62516 from nicksardo/expand-id
Automatic merge from submit-queue (batch tested with PRs 62060, 62516). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[GCE] Add new formats to resource parser and self link creator

**What this PR does / why we need it**:
- Expands the set of parse-able strings for resource IDs, while also simplifying the code. Note that these are acceptable values for some fields in GCP API.
  - global/networks/my-network
  - regions/us-central1/subnetworks/my-sub
  - zones/us-central1-a/instances/my-hacker-instance
- Fixes the SelfLink function to return links for regions and zones:
  - https://www.googleapis.com/compute/v1/projects/proj4/regions/us-central1
- Generates helper functions to create a ResourceID for each resource
- Generates a unit test that ensures all links can be generated and all generated links can be parsed.
- Fixes an ILB test which creates a malformed URL.

**Special notes for your reviewer**:
/assign rramkumar1

**Release note**:
```release-note
NONE
```
2018-05-02 18:12:06 -07:00