Automatic merge from submit-queue
Fix share name generation in azure file provisioner.
**What this PR does / why we need it**:
Azure file share name has a length limit of 63, but currently the code is using data volume's limit 75.
**Which issue this PR fixes**
With current implementation, when the cluster name is long, the resulting file share name could have a length of 75.
Also function `GenerateVolumeName` would produce double '-' when clusterName containing '-' is to be truncated.
In both cases, service would reject the creating file share request.
**Special notes for your reviewer**:
Please refer to:
https://docs.microsoft.com/en-us/rest/api/storageservices/naming-and-referencing-shares--directories--files--and-metadata#share-names
- Share names must be from 3 through 63 characters long.
- The name cannot contain two consecutive hyphens.
@rootfs @colemickens
**Release note**:
Automatic merge from submit-queue
jsonpath filter: allow intermediate missing keys
**What this PR does / why we need it**:
In jsonpath, when filtering a list, if allowMissingKeys is true, skip
over any items that are missing an intermediate key in the filter,
instead of returning a confusing error.
For example, if the filter is
{.items[?(@.metadata.annotations.foo=="bar")].metadata.name}
we should return all items where metadata.annotations.foo == bar, but if
an item in the list does not have metadata, metadata.annotations, or
metadata.annotations.foo, skip it instead of erroring.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#45546
**Special notes for your reviewer**:
**Release note**:
```release-note
Fixed a bug where a jsonpath filter would return an error if one of the items being evaluated did not contain all of the nested elements in the filter query.
```
cc @timothysc @smarterclayton @stevekuznetsov @mengqiy @liggitt @kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)
Fix the order of deletion
1. EnsureLoadBalancer can't delete pool without deleting members,
just let EnsureLoadBalancerDeleted do it.
2. Add some friendly error message
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48890, 46893, 48872, 48896)
Support customized system spec in the node conformance test and create the GKE system spec
ref: https://github.com/kubernetes/kubernetes/issues/46891
- System specs are located in `test/e2e_node/system/specs`. Created one for validating GKE images in `test/e2e_node/system/specs/gke.yaml`.
- `--image-spec-name` can be used to specify a system spec in node e2e and conformance tests. This option maps to `SYSTEM_SPEC_NAME` in a test properties file, which is the user facing configuration. So, users can specify `SYSTEM_SPEC_NAME=gke` to run the image validation using the GKE system spec.
- If `SYSTEM_SPEC_NAME` is unspecified, the default spec (`system.DefaultSysSpec`) will be used.
- We can also use `make test-e2e-node SYSTEM_SPEC_NAME=gke` to run tests using GKE image spec.
**Release note**:
`None`
Automatic merge from submit-queue
Move api-machinery related e2e tests to a 'api-machinery' e2e test su…
…bdirectory.
**What this PR does / why we need it**:
Moves all e2e tests belonging to sig-api-machinery into a dedicated `test/e2e/apimachinery ` directory and updates the tests to use `SigDescribe` to prepend `[sig-api-machinery]` to the testnames of all apimachinery owned tests.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48842, 48891)
[Federation] Handle federation up timeouts
Instead of relying on external timeout command.
First raised in #48756
/assign madhusudancs
Automatic merge from submit-queue
Add quick-verify make rule.
This is useful for humans to run to catch obvious problems before
pushing commits and waiting for CI to run verify checks.
Quick mode only runs a whitelist of verify scripts that are reasonably fast.
I set the initial bar arbitrarily at <10s each on my workstation.
The whole set runs in <30s for me, assuming I had already run `make` and
`hack/godep-restore.sh`. This is compared to the full `make verify`
which takes [I don't know how long because I gave up after 45min].
Automatic merge from submit-queue (batch tested with PRs 48578, 48895, 48958)
use port configuration
**What this PR does / why we need it**: Uses the `port` config option in the kubeapi-load-balancer charm.
**Release note**:
```release-note
Uses the port config option in the kubeapi-load-balancer charm.
```
Automatic merge from submit-queue (batch tested with PRs 48578, 48895, 48958)
move sig-apps upgrade tests to its directory
**What this PR does / why we need it**: This PR moves sig-apps upgrade tests to its directory in accord to fixit requirements.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48839
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 48578, 48895, 48958)
run must output message on container error
**What this PR does / why we need it**: `kubectl run` must output a message (instead of just exiting with an error code) on container error.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/openshift/origin/issues/15031 found in OpenShift
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 47066, 48892, 48933, 48854, 48894)
replace hardcoded use of "kubectl" in apply warning msg
**Release note**:
```release-note
NONE
```
Removes use of hardcoded "kubectl" in the `kubectl apply ...` warning message that is printed when the `last-applied-configuration` annotation is missing on a resource. This is useful for downstream wrappers around the `apply` command.
cc @stevekuznetsov @fabianofranz
Automatic merge from submit-queue (batch tested with PRs 47066, 48892, 48933, 48854, 48894)
azure: msi: add managed identity field, logic
**What this PR does / why we need it**: Enables managed service identity support for the Azure cloudprovider. "Managed Service Identity" allows us to ask the Azure Compute infra to provision an identity for the VM. Users can then retrieve the identity and assign it RBAC permissions to talk to Azure ARM APIs for the purpose of the cloudprovider needs.
Per the commit text:
```
The azure cloudprovider will now use the Managed Service Identity
to retrieve access tokens for the Azure ARM APIs, rather than
requiring hard-coded, user-specified credentials.
```
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: n/a
**Special notes for your reviewer**: none
**Release note**:
```release-note
azure: support retrieving access tokens via managed identity extension
```
cc: @brendandburns @jdumars @anhowe
Automatic merge from submit-queue (batch tested with PRs 47066, 48892, 48933, 48854, 48894)
Fix windows build by altering go-ansiterm version
**What this PR does / why we need it**:
An attempt to fix windows build break in #48887, moves `github.com/Azure/go-ansiterm` to 70b2c90b26.
We can upgrade to new version when we pick docker 1.12
See also:
fab67ab677 (diff-c36e523b64f1ca41293e912113063a92L9)
**Which issue this PR fixes**
Fixes#48887
**Special notes for your reviewer**:
**Release note**:
Automatic merge from submit-queue
examples/volumes/flexvolume/nfs: check for jq and simplify quoting
**What this PR does / why we need it**:
This PR improves error reporting of the nfs flex plugin script. Before it output non-JSON error, when `jq` wasn't installed on the server:
```
$ /usr/libexec/kubernetes/kubelet-plugins/volume/exec/k8s~nfs/nfs getvolumename test
/usr/libexec/kubernetes/kubelet-plugins/volume/exec/k8s~nfs/nfs: line 87: jq: command not found
/usr/libexec/kubernetes/kubelet-plugins/volume/exec/k8s~nfs/nfs: line 88: jq: command not found
{"status": "Success", "volumeName": "/"}
```
K8s fails to unmarshal such output to show an error in the logs:
>E0606 19:32:30.196262 25700 driver-call.go:212] Failed to unmarshal output for command: getvolumename, output: "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/k8s\~nfs/nfs: line 87: jq: command not found\n/usr/libexec/kubernetes/kubelet-plugins/volume/exec/k8s\~nfs/nfs: line 88: jq: command not found\n{\"status\": \"Success\", \"volumeName\": \"/\"}", error: invalid character '/' looking for beginning of value
After this change:
```sh
$ /usr/libexec/kubernetes/kubelet-plugins/volume/exec/k8s~nfs/nfs getvolumename test
{ "status": "Failure", "message": "'jq' binary not found. Please install jq package before using this driver"}
```
and
> E0606 19:52:07.915594 25700 driver-call.go:219] getvolumename command failed, status: Failure, reason: 'jq' binary not found. Please install jq package before using this driver
Also this PR improves quoting a bit by using single quotes where it possible.
**Release note**:
```release-note
NONE
```
CC @mfojtik
Automatic merge from submit-queue
remove some people from OWNERS so they don't get reviews anymore
These are googlers who don't work on the project anymore but are still
getting reviews assigned to them:
- @bprashanth
- @rjnagal
- @vmarmol
Automatic merge from submit-queue
Correct the comment in PSP examples.
**What this PR does / why we need it**:
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
none
```
Automatic merge from submit-queue (batch tested with PRs 48572, 48838, 48931, 48783, 47090)
kubeadm: change the default bootstrap token TTL to 24 hours
**What this PR does / why we need it**:
This PR changes the TTL for the default bootstrap token generated by `kubeadm init` (without the `--token-ttl` parameter) and `kubeadm token create` (without the `--ttl` flag). Previously, the default TTL was infinite. After this change it is 24 hours.
~~The reasoning for 2 hours as a default is that it's 1) long enough that someone manually using kubeadm (copy-pasting) shouldn't have any issues and 2) short enough that if something is going to break, it should break while the user/admin is still paying attention to the cluster. I'm open to bikeshedding about the exact value, 2 hours is a bit of a strawman.~~
**Edit: updated this to 24 hours instead of 2 hours.**
This is a breaking change if you rely on infinite TTL tokens (e.g., if you had an ASG group of worker nodes). The old behavior is easily restored by passing `--token-ttl 0` to `kubeadm init` or the `--ttl 0` flag to `kubeadm token create`.
**Which issue this PR fixes**: fixes https://github.com/kubernetes/kubeadm/issues/343
**Special notes for your reviewer**:
This was discussed earlier today in SIG-cluster-lifecycle
**Release note**:
```release-note
Change the default kubeadm bootstrap token TTL from infinite to 24 hours. This is a breaking change. If you require the old behavior, use `kubeadm init --token-ttl 0` / `kubeadm token create --ttl 0`.
```
cc @jbeda
Automatic merge from submit-queue (batch tested with PRs 48572, 48838, 48931, 48783, 47090)
remove duplicated word file in error
Signed-off-by: allencloud <allen.sun@daocloud.io>
**What this PR does / why we need it**:
remove duplicated word file in error
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
NONE
**Special notes for your reviewer**:
NONE
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48572, 48838, 48931, 48783, 47090)
kubeadm: add a warning about the default token TTL changing in 1.8
**What this PR does / why we need it**:
This adds a warning to `kubeadm init` and `kubeadm token create` if they are run without the `--token-ttl` / `--ttl` flags. In 1.7 and before, the tokens generated by these commands defaulted to an infinite TTL (no expiration) in 1.8, they will generate a token with a 24 hour TTL.
The actual default change is in https://github.com/kubernetes/kubernetes/pull/48783. This change is separate so we can cherry pick the warning into the `release-1.7` branch.
**Which issue this PR fixes**: ref https://github.com/kubernetes/kubeadm/issues/343
**Special notes for your reviewer**:
This change is blocked on https://github.com/kubernetes/kubeadm/issues/343. These warnings should probably be removed in the 1.9 cycle.
**Release note**:
```release-note
Add a runtime warning about the kubeadm default token TTL changes in 1.8.
```
/assign @luxas
Automatic merge from submit-queue
Azure PD (Managed/Blob)
This is exactly the same code as this [PR](https://github.com/kubernetes/kubernetes/pull/41950). It has a clean set of generated items. We created a separate PR to accelerate the accept/merge the PR
CC @colemickens
CC @brendandburns
**What this PR does / why we need it**:
1. Adds K8S support for Azure Managed Disks.
2. Adds support for dedicated blob disks (1:1 to storage account) in addition to shared blob disks (n:1 to storage account).
3. Automatically manages the underlying storage accounts. New storage accounts are created at 50% utilization. Max is 100 disks, 60 disks per storage account.
2. Addresses the current issues with Blob Disks:
..* Significantly faster attach process. Disks are now usually available for pods on nodes under 30 sec if formatted, under a min if not formatted.
..* Adds support to move disks between nodes.
..* Adds consistent attach/detach behavior, checks if the disk is leased/attached on a different node before attempting to attach to target nodes.
..* Fixes a random hang behavior on Azure VMs during mount/format (for both blob + managed disks).
..* Fixes a potential conflict by avoiding the use of disk names for mount paths. The new plugin uses hashed disk uri for mount path.
The existing AzureDisk is used as is. Additional "kind" property was added allowing the user to decide if the pd will be shared, dedicated or managed (Azure Managed Disks are used).
Due to the change in mounting paths, existing PDs need to be recreated as PV or PVCs on the new plugin.
Automatic merge from submit-queue
Redirect all files in /examples folder to kubernetes/examples repo
**What this PR does / why we need it**:
Examples are being moved to their own repository: https://github.com/kubernetes/examples
We need to remove them from the main repo , but first we need to keep a redirect.
This is a *big* organizational change, but nothing technical (aside from e2e tests)
**Which issue this PR fixes**
fixes part of #24343
**Special notes for your reviewer**:
WIP, I still need to figure out what to do with the BUILD script and tests, plus take care of the e2e tests that use some of these examples.
**release notes**
```release-note
Redirect all examples README to the the kubernetes/examples repo
```
Automatic merge from submit-queue
kubeadm: fix broken `kubeadm init --config` flag
**What this PR does / why we need it**:
This code was changed in ea196490a0 (https://github.com/kubernetes/kubernetes/pull/43558) to validate that `--config` wasn't passed along with other flags. Unfortunately, the implementation was checking `PersistentFlags()`, which was not parsed at the point it was being validated.
The fix is to use `Flags()` instead, which contains the expected data.
**Which issue this PR fixes** fixes https://github.com/kubernetes/kubeadm/issues/345
**Special notes for your reviewer**:
This is a regression that was cherry picked (https://github.com/kubernetes/kubernetes/pull/48577) into the `release-1.7` branch. We should fix this before the next 1.7 point release.
This is awkward to unit test without restructuring the code. I think this command parsing code would be a good candidate for some higher level tests.
**Release note**:
```release-note
Fix a regression that broke the `--config` flag for `kubeadm init`.
```
/assign @luxas
This adds a warning to `kubeadm init` and `kubeadm token create` if they are run without the `--token-ttl` / `--ttl` flags. In 1.7 and before, the tokens generated by these commands defaulted to an infinite TTL (no expiration) in 1.8, they will generate a token with a 24 hour TTL.
The actual default change is in https://github.com/kubernetes/kubernetes/pull/48783. This change is separate so we can cherry pick the warning into the release-1.7 branch.
Automatic merge from submit-queue
Split out max_density test and add owners to other perf tests
@rrati - IIRC you're the one who added max-density test and promised to look after it. I split it out of the main density.go, so it'd be clear that this one is not owned by us.
cc @kubernetes/sig-scalability-misc
Automatic merge from submit-queue (batch tested with PRs 48812, 48276)
client-go support IPv6 addresses for PortForwarder getListener()
Currently, client-go requires that an IPv6 address string for hostname has
square brackets surrounding, so that it can be used with address:port in
an API request.
This change, removes that requirement, and has getListener() add the
square brackets, if needed. If the hostname is a name, an IPv4 address
string, or an IPv6 address already with square brackets, no change is made.
Decided to change here, rather than everywhere client-go is used (thinking
that there may be places where we DON'T want the square brackets applied).
This issue was found in kubelet, which, at startup, creates a listener for
services and nodes. If an IPv6 address is used, the URI was malformed.
**What this PR does / why we need it**:
Ensures that client-go will handle IPv6 addresses as hostnames, without any special encoding by caller.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#48225
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48812, 48276)
Change fluentd-gcp monitoring to use metrics exposed by SD plugin
Following https://github.com/GoogleCloudPlatform/fluent-plugin-google-cloud/pull/135, make fluentd-gcp expose metrics in Prometheus registry and use them instead of counting records in the pipeline.
/cc @piosz @igorpeshansky
```release-note
Fluentd-gcp DaemonSet exposes different set of metrics.
```
Automatic merge from submit-queue (batch tested with PRs 48864, 48651, 47703)
Do not persist SelfLink into etcd storage
This behavior regressed in an earlier release. Clearing the self link reduces the size of the stored object by a small amount and keeps etcd marginally cleaner. Add tests to verify that Create and Update result in no SelfLink stored in etcd.
Automatic merge from submit-queue (batch tested with PRs 48864, 48651, 47703)
Enable logexporter mechanism to dump logs from k8s nodes to GCS directly
Ref https://github.com/kubernetes/kubernetes/issues/48513
This adds support for logexporter from k8s side. Next I'll send a PR adding support from test-infra side.
/cc @kubernetes/sig-scalability-misc @kubernetes/test-infra-maintainers @fejta @wojtek-t @gmarek
Automatic merge from submit-queue
Fixed cluster validation for multizonal clusters.
Fixed cluster validation for multizonal clusters.
This should fix HA master e2e tests.
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 46748, 48826)
Added `CriticalAddonsOnly` toleration for npd.
**What this PR does / why we need it**:
We should add `CriticalAddonsOnly` toleration to make sure the daemonset can be scheduled on the node even if already planned to run critical pod.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47015
**Special notes for your reviewer**:
**Release note**:
```release-note
none
```
Automatic merge from submit-queue
Revert workaround in PR 46246 as APIs have been consistent
**What this PR does / why we need it**:
No need to convert v1.ObjectReference as APIs have been consistent in `k8s.io/api/core/v1`.
**Which issue this PR fixes** : fixes#48668
**Special notes for your reviewer**:
/assign @derekwaynecarr @caesarxuchao
**Release note**:
```release-note
NONE
```
Kubernetes Submit Queue