Automatic merge from submit-queue (batch tested with PRs 38445, 40292)
Add the ability to edit fields within a config map.
Addresses part of https://github.com/kubernetes/kubernetes/issues/36222
Example command:
```console
$ kubectl edit configmap foo --config-map-data=bar
```
Will open the data element named `bar` in the `ConfigMap` named `foo` in `$EDITOR`, the edited contents are then updated back to the config map.
@kubernetes/sig-cli
```release-note
Add a special purpose tool for editing individual fields in a ConfigMap with kubectl
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubeadm: add optional self-hosted deployment
**What this PR does / why we need it**: add an optional self-hosted deployment type to `kubeadm`, for master components only, namely `apiserver`, `controller-manager` and `scheduler`.
**Which issue this PR fixes**: closes#38407
**Special notes for your reviewer**: /cc @aaronlevy @luxas @dgoodwin
**Release note**:
```release-note
kubeadm: add optional self-hosted deployment for apiserver, controller-manager and scheduler.
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubelet: storage: teardown terminated pod volumes
This is a continuation of the work done in https://github.com/kubernetes/kubernetes/pull/36779
There really is no reason to keep volumes for terminated pods attached on the node. This PR extends the removal of volumes on the node from memory-backed (the current policy) to all volumes.
@pmorie raised a concern an impact debugging volume related issues if terminated pod volumes are removed. To address this issue, the PR adds a `--keep-terminated-pod-volumes` flag the kubelet and sets it for `hack/local-up-cluster.sh`.
For consideration in 1.6.
Fixes#35406
@derekwaynecarr @vishh @dashpole
```release-note
kubelet tears down pod volumes on pod termination rather than pod deletion
```
Automatic merge from submit-queue (batch tested with PRs 39486, 37288, 39477, 39455, 39542)
Allow missing keys in templates by default
Switch to allowing missing keys in jsonpath templates by default.
Add support for allowing/disallowing missing keys in go templates
(default=allow).
Add --allow-missing-template-keys flag to control this behavior (default=true /
allow missing keys).
Fixes#37991
@kubernetes/sig-cli-misc @kubernetes/api-reviewers @smarterclayton @fabianofranz @liggitt @pwittrock
Switch to allowing missing keys in jsonpath templates by default.
Add support for allowing/disallowing missing keys in go templates
(default=allow).
Add --allow-missing-template-keys flag to control this behavior
(default=true / allow missing keys).
Automatic merge from submit-queue (batch tested with PRs 36229, 39450)
Bump etcd to 3.0.14 and switch to v3 API in etcd.
Ref #20504
**Release note**:
```release-note
Switch default etcd version to 3.0.14.
Switch default storage backend flag in apiserver to `etcd3` mode.
```
Automatic merge from submit-queue
Coreos kube-up now with less cloud init
This update includes significant refactoring. It moves almost all of the
logic into bash scripts, modeled after the `gci` cluster scripts.
The reason to do this is:
1. Avoid duplicating the saltbase manifests by reusing gci's parsing logic (easier maintenance)
2. Take an incremental step towards sharing more code between gci/trusty/coreos, again for better maintenance
3. Pave the way for making future changes (e.g. improved rkt support, kubelet support) easier to share
The primary differences from the gci scripts are the following:
1. Use of the `/opt/kubernetes` directory over `/home/kubernetes`
2. Support for rkt as a runtime
3. No use of logrotate
4. No use of `/etc/default/`
5. No logic related to noexec mounts or gci-specific firewall-stuff
It will make sense to move 2 over to gci, as well as perhaps a few other small improvements. That will be a separate PR for ease of review.
Ref #29720, this is a part of that because it removes a copy of them.
Fixes#24165
cc @yifan-gu
Since this logic largely duplicates logic from the gci folder, it would be nice if someone closely familiar with that gave an OK or made sure I didn't fall into any gotchas related to that, so cc @andyzheng0831
Automatic merge from submit-queue (batch tested with PRs 38426, 38917, 38891, 38935)
Remove cluster/mesos from hack/verify-flags/exceptions.txt
`cluster/mesos` scripts was removed; so remove it from `hack/verify-flags/exceptions.txt`.
The diff was generated by `hack/verify-flags-underscore.py -e > hack/verify-flags/exceptions.txt`.
Automatic merge from submit-queue
conversion-gen: add --skip-unsafe flag
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
Automatic merge from submit-queue
[Federation] Make federation etcd PVC size configurable
This one implements one of the many TODO items pending in the previous set of kubefed PRs.
The design doc PR is at https://github.com/kubernetes/kubernetes/pull/34484
cc @kubernetes/sig-cluster-federation @madhusudancs
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
[Federation] kubefed init now has a new flag, --etcd-pv-capacity, which can be used to configure the persistent volume capacity for etcd.
```
Automatic merge from submit-queue
kubedns: use initial resource listing as ready signal
Fix#35140.
Set up the ready signal after the first resource listing finished for both endpoints and services instead of listen on kubernetes service.
@bprashanth @bowei @thockin
**Release note**:
```
```
Automatic merge from submit-queue
add a configuration for kubelet to register as a node with taints
and deprecate --register-schedulable
ref #28687#29178
cc @dchen1107 @davidopp @roberthbailey
Automatic merge from submit-queue
Split inflight requests into read-only and mutating groups
cc @smarterclayton @lavalamp @caesarxuchao
```release-note
API server have two separate limits for read-only and mutating inflight requests.
```
Automatic merge from submit-queue
plumb in front proxy group header
Builds on https://github.com/kubernetes/kubernetes/pull/36662 and https://github.com/kubernetes/kubernetes/pull/36774, so only the last commit is unique.
This completes the plumbing for front proxy header information and makes it possible to add just the front proxy header authenticator.
WIP because I'm going to assess it in use downstream.
Automatic merge from submit-queue
kubectl top pod|node should handle when Heapster is somewhere else
OpenShift runs Heapster on HTTPS, which means `top node` and `top pod`
are broken because they hardcode 'http' as the scheme. Provide an
options struct allowing users to specify `--heapster-namespace`,
`--heapster-service`, `--heapster-scheme`, and `--heapster-port` to the
commands (leveraging the existing defaults).
@kubernetes/sig-metrics makes top a little more useful in other spots
Automatic merge from submit-queue (batch tested with PRs 35300, 36709, 37643, 37813, 37697)
Add generated informers
Add informer-gen and the informers it generates. We'll do follow-up PRs to convert everything currently using the hand-written informers to the generated ones.
TODO:
- [x] switch to `GroupVersionResource`
- [x] finish godoc
@deads2k @caesarxuchao @sttts @liggitt
Automatic merge from submit-queue
Add kubernetes-anywhere as a new e2e deployment option.
This change adds support for using `kubernetes-anywhere` as a deployment option for hack/e2e.go. This work is toward the larger goal of being able to run e2e tests against `kubeadm` clusters, which `kubernetes-anywhere` supports.
**Release note**:
```release-note
Add kubernetes-anywhere as a new e2e deployment option
```
The configuration in `getConfig()` comes mostly from the defaults in `kubernetes-anywhere`. In the future, we can add more plumbing to override them via CLI flags.
CC @mikedanese
Automatic merge from submit-queue
create service add create ExternalName service implementation
@kubernetes/kubectl create service add ExternalName support, refer #34731 for more detail.
```release-note
kubectl create service externalname
```
OpenShift runs Heapster on HTTPS, which means `top node` and `top pod`
are broken because they hardcode 'http' as the scheme. Provide an
options struct allowing users to specify `--heapster-namespace`,
`--heapster-service`, `--heapster-scheme`, and `--heapster-port` to the
commands (leveraging the existing defaults).
Automatic merge from submit-queue
move parts of the mega generic run struct out
This splits the main `ServerRunOptions` into composeable pieces that are bindable separately and adds easy paths for composing servers to run delegating authentication and authorization.
@sttts @ncdc alright, I think this is as far as I need to go to make the composing servers reasonable to write. I'll try leaving it here
Automatic merge from submit-queue
[Federation][join-flags] Add flags for cluster context and secret names while joining clusters to federation.
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make
`kubefed join` consistent with `kubefed unjoin`. This also means
`--cluster-context` is now made a flag and defaults to cluster name if
unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Fixes: Issue #35954
cc @kubernetes/sig-cluster-federation @quinton-hoole @irfanurrehman
- Adds command line flags --config-map, --config-map-ns.
- Fixes 36194 (https://github.com/kubernetes/kubernetes/issues/36194)
- Update kube-dns yamls
- Update bazel (hack/update-bazel.sh)
- Update known command line flags
- Temporarily reference new kube-dns image (this will be fixed with
a separate commit when the DNS image is created)
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make `kubefed
join` consistent with `kubefed unjoin`. This also means `--cluster-
context` is now made a flag and defaults to cluster name if unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Automatic merge from submit-queue
Add a flag allowing contention profiling of the API server
Useful for performance debugging.
cc @smarterclayton @timothysc @lavalamp
```release-note
Add a flag allowing contention profiling of the API server
```
Automatic merge from submit-queue
Make GCI nodes mount non tmpfs, ext* & bind mounts using an external mounter
This PR downloads the stage1 & gci-mounter ACIs as part of cluster bring up instead of downloading them dynamically from gcr.io, which was the cause for #36206.
I have also optimized the containerized mounter to pre-load the mounter image once to avoid fetch latency while using it.
Original PR which got reverted: https://github.com/kubernetes/kubernetes/pull/35821
```release-note
GCI nodes use an external mounter script to mount NFS & GlusterFS storage volumes
```
@mtaufen Node e2e is not re-enabled in this PR.
cc @jingxu97
Automatic merge from submit-queue
specify custom ca file to verify the keystone server
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Sometimes the keystone server's certificate is self-signed, mainly used for internal development, testing and etc.
For this kind of ca, we need a way to verify the keystone server.
Otherwise, below error will occur.
> x509: certificate signed by unknown authority
This patch provide a way to pass in a ca file to verify the keystone server when starting `kube-apiserver`.
**Which issue this PR fixes** : fixes#22695, #24984
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
``` release-note
```
Provides an opt-in flag, --experimental-fail-swap-on (and corresponding
KubeletConfiguration value, ExperimentalFailSwapOn), which is false by default.
If we have a public & private zone with the same name (which is common
on AWS), this means we can still create records. Also tighten up some
of the logic to allow for zones with duplicate names.
Automatic merge from submit-queue
Node Conformance Test: Add system verification
For #30122 and #29081.
This PR introduces system verification test in node e2e and conformance test. It will run before the real test. Once the system verification fails, the test will just fail. The output of the system verification is like this:
```
I0909 23:33:20.622122 2717 validators.go:45] Validating os...
OS: Linux
I0909 23:33:20.623274 2717 validators.go:45] Validating kernel...
I0909 23:33:20.624037 2717 kernel_validator.go:79] Validating kernel version
KERNEL_VERSION: 3.16.0-4-amd64
I0909 23:33:20.624146 2717 kernel_validator.go:93] Validating kernel config
CONFIG_NAMESPACES: enabled
CONFIG_NET_NS: enabled
CONFIG_PID_NS: enabled
CONFIG_IPC_NS: enabled
CONFIG_UTS_NS: enabled
CONFIG_CGROUPS: enabled
CONFIG_CGROUP_CPUACCT: enabled
CONFIG_CGROUP_DEVICE: enabled
CONFIG_CGROUP_FREEZER: enabled
CONFIG_CGROUP_SCHED: enabled
CONFIG_CPUSETS: enabled
CONFIG_MEMCG: enabled
I0909 23:33:20.679328 2717 validators.go:45] Validating cgroups...
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
I0909 23:33:20.679454 2717 validators.go:45] Validating docker...
DOCKER_GRAPH_DRIVER: aufs
```
It verifies the system following a predefined `SysSpec`:
``` go
// DefaultSysSpec is the default SysSpec.
var DefaultSysSpec = SysSpec{
OS: "Linux",
KernelVersion: []string{`3\.[1-9][0-9].*`, `4\..*`}, // Requires 3.10+ or 4+
// TODO(random-liu): Add more config
KernelConfig: KernelConfig{
Required: []string{
"NAMESPACES", "NET_NS", "PID_NS", "IPC_NS", "UTS_NS",
"CGROUPS", "CGROUP_CPUACCT", "CGROUP_DEVICE", "CGROUP_FREEZER",
"CGROUP_SCHED", "CPUSETS", "MEMCG",
},
Forbidden: []string{},
},
Cgroups: []string{"cpu", "cpuacct", "cpuset", "devices", "freezer", "memory"},
RuntimeSpec: RuntimeSpec{
DockerSpec: &DockerSpec{
Version: []string{`1\.(9|\d{2,})\..*`}, // Requires 1.9+
GraphDriver: []string{"aufs", "overlay", "devicemapper"},
},
},
}
```
Currently, it only supports:
- Kernel validation: version validation and kernel configuration validation
- Cgroup validation: validating whether required cgroups subsystems are enabled.
- Runtime Validation: currently, only validates docker graph driver.
The validating framework is ready. The specific validation items could be added over time.
@dchen1107
/cc @kubernetes/sig-node
Automatic merge from submit-queue
New command: "kubeadm token generate"
As part of #33930, this PR adds a new top-level command to kubeadm to just generate a token for use with the init/join commands. Otherwise, users are left to either figure out how to generate a token on their own, or let `kubeadm init` generate a token, capture and parse the output, and then use that token for `kubeadm join`.
At this point, I was hoping for feedback on the CLI experience, and then I can add tests. I spoke with @mikedanese and he didn't like the original propose of `kubeadm util generate-token`, so here are the runners up:
```
$ kubeadm generate-token # <--- current implementation
$ kubeadm generate token # in case kubeadm might generate other things in the future?
$ kubeadm init --generate-token # possibly as a subcommand of an existing one
```
Currently, the output is simply the token on one line without any padding/formatting:
```
$ kubeadm generate-token
1087fd.722b60cdd39b1a5f
```
CC: @kubernetes/sig-cluster-lifecycle
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
``` release-note
New kubeadm command: generate-token
```
Automatic merge from submit-queue
make using service account credentials from controllers optional
Adds a flag to make the "SA per controller" behavior optional.
Automatic merge from submit-queue
[Kubelet] Use the custom mounter script for Nfs and Glusterfs only
This patch reduces the scope for the containerized mounter to NFS and GlusterFS on GCE + GCI clusters
This patch also enabled the containerized mounter on GCI nodes
Shepherding multiple PRs through the submit queue is painful. Hence I combined them into this PR. Please review each commit individually.
cc @jingxu97 @saad-ali
https://github.com/kubernetes/kubernetes/pull/35652 has also been reverted as part of this PR
Automatic merge from submit-queue
[Federation][init] Implement `kubefed init` command that performs federation control plane bootstrap.
Please review only the last commit here. This is based on PR #35593 which will be reviewed independently.
I am intentionally not including the unit tests in this PR to better distribute and parallelize reviews. This PR is already big.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
Automatic merge from submit-queue
[Federation][unjoin-00] Implement `kubefed unjoin` command.
Please review only the last commit here. This is based on PR #35493 which will be reviewed independently.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
Automatic merge from submit-queue
Move .gitattributes annotation to the root, so GitHub will respect them.
This should fix the merge conflicts by letting GitHub use the simpler line-by-line algorithm for this file. Having .gitattributes in a sub-directory would work for local merging, but would show conflicts on the web UI.
Automatic merge from submit-queue
[Federation][join-01] Implement `kubefed join` command.
Supersedes PR #35155.
Please review only the last commit here. This is based on PR #35492 which will be reviewed independently.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @mwielgus
Along the way: Fix ginkgo-e2e.sh. This change got dropped in the
original PR, but it was meant to allow a conformance-style
kubectl-auth test and still have a legit cloud provider.
Kubernetes Submit Queue