Automatic merge from submit-queue (batch tested with PRs 41919, 41149, 42350, 42351, 42285)
Juju: Disable anonymous auth on kubelet
**What this PR does / why we need it**:
This disables anonymous authentication on kubelet when deployed via Juju.
I've also adjusted a few other TLS options for kubelet and kube-apiserver. The end result is that:
1. kube-apiserver can now authenticate with kubelet
2. kube-apiserver now verifies the integrity of kubelet
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/219
**Special notes for your reviewer**:
This is dependent on PR #41251, where the tactics changes are being merged in separately.
Some useful pages from the documentation:
* [apiserver -> kubelet](https://kubernetes.io/docs/admin/master-node-communication/#apiserver---kubelet)
* [Kubelet authentication/authorization](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/)
**Release note**:
```release-note
Juju: Disable anonymous auth on kubelet
```
Automatic merge from submit-queue
Protect kubeproxy deployed via kube-up from system OOMs
This change is necessary until it can be moved to Guaranteed QoS Class.
For #40573
Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)
kubeadm: Add a --ca-cert-path flag to kubeadm join
**What this PR does / why we need it**:
This PR makes it possible to customize where the CA file is written
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
```
@pires @mikedanese @dmmcquay @jbeda @errordeveloper
Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)
Lint fixes for the master and worker Python code.
**What this PR does / why we need it**: lint fixes for the python code.
**Which issue this PR fixes** none
**Special notes for your reviewer**: This is lint fixes for the Juju python code.
**Release note**:
```release-note
NONE
```
Please consider these changes so we can pass flake8 lint tests in our build process.
Automatic merge from submit-queue
Added a basic monitor for providing etcd version related info
Fixes#41071
This tool scrapes metrics partly from etcd's /version and /metrics endpoints and partly using etcdctl and exposes them as prometheus metrics at `http://localhost:9101/metrics` endpoint on the master. Here is a summary of the metrics it exposes (self-explanatory from the code):
- etcdVersionFetchCount = prometheus.NewCounterVec(
prometheus.CounterOpts{
Namespace: "etcd",
Name: "version_info_fetch_count",
Help: "Number of times etcd's version info was fetched, labeled by etcd's server binary and cluster version",
},
[]string{"serverversion", "clusterversion"})
- etcdGRPCRequestsTotal = prometheus.NewCounterVec(
prometheus.CounterOpts{
Namespace: namespace,
Name: "grpc_requests_total",
Help: "Counter of received grpc requests, labeled by grpc method and grpc service names",
},
[]string{"grpc_method", "grpc_service"})
For further info on how to run this as a binary/docker-container/kubernetes-pod and checking the metrics, have a look at the README.md file.
cc @fgrzadkowski @wojtek-t @piosz
For now, this is focused on a fixed set of flags that makes the audit
log show up under /var/log/kube-apiserver-audit.log and behave similarly
to /var/log/kube-apiserver.log. Allowing other customization would
require significantly more complex changes.
Audit log rotation is handled externally by the wildcard /var/log/*.log
already configured in configure-helper.sh.
Automatic merge from submit-queue
Add flag to node e2e test specifying location of ssh privkey
**What this PR does / why we need it**: in CI, the ssh private key is not always located at `$HOME/.ssh`, so it's helpful to be able to override it.
@krzyzacy here's my resurrected change. I'm not sure why I neglected to follow-through on it originally.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Able to quick create a HA cluster by kube-up.sh centos provider
Make `kube-up.sh` `centos provider` support quick create a HA cluster, as I said above [#39430](https://github.com/kubernetes/kubernetes/issues/39430), it's more flexible than `kops` or `kubeadm` for some people in a limited network region.
I'm new to k8s dev, so if this pull request need to change, please let me know.
```release-note
Added support for creating HA clusters for centos using kube-up.sh.
```
Fix: cannot get default master advertise address correctly
Set default value of NUM_MASTERS and NUM_NODES by MASTERS and NODES themself
Code cleanup and documented
Using runtime reconfiguration for etcd cluster instead of etcd discovery
Add exceptions for verify-flags
Automatic merge from submit-queue (batch tested with PRs 40335, 40320, 40324, 39103, 40315)
Splitting master/node services into separate charm layers
**What this PR does / why we need it**:
This branch includes a roll-up series of commits from a fork of the
Kubernetes repository pre 1.5 release because we didn't make the code freeze.
This additional effort has been fully tested and has results submit into
the gubernator to enhance confidence in this code quality vs. the single
layer, posing as both master/node.
To reference the gubernator results, please see:
https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/
Apologies in advance for the large commit however, we did not want to
submit without having successful upstream automated testing results.
This commit includes:
- Support for CNI networking plugins
- Support for durable storage provided by Ceph
- Building from upstream templates (read: kubedns - no more template
drift!)
- An e2e charm-layer to make running validation tests much simpler/repeatable
- Changes to support the 1.5.x series of Kubernetes
**Special notes for your reviewer**:
Additional note: We will be targeting -all- future work against upstream
so large pull requests of this magnitude will not occur again.
**Release note**:
```release-note
- Splits Juju Charm layers into master/worker roles
- Adds support for 1.5.x series of Kubernetes
- Introduces a tactic for keeping templates in sync with upstream eliminating template drift
- Adds CNI support to the Juju Charms
- Adds durable storage support to the Juju Charms
- Introduces an e2e Charm layer for repeatable testing efforts and validation of clusters
```
This branch includes a rollup series of commits from a fork of the
kubernetes repository pre 1.5 release because we didn't make the code freeze.
This additional effort has been fully tested and has results submit into
the gubernator to enhance confidence in this code quality vs. the single
layer, posing as both master/node.
To reference the gubernator results, please see:
https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/
Apologies in advance for the large commit, however we did not want to
submit without having successful upstream automated testing results.
This commit includes:
- Support for CNI networking plugins
- Support for durable storage provided by ceph
- Building from upstream templates (read: kubedns - no more template
drift!)
- An e2e charm-layer to make running validation tests much simpler/repeatable
- Changes to support the 1.5.x series of kubernetes
Additional note: We will be targeting -all- future work against upstream
so large pull requests of this magnitude will not occur again.
Automatic merge from submit-queue (batch tested with PRs 36229, 39450)
Bump etcd to 3.0.14 and switch to v3 API in etcd.
Ref #20504
**Release note**:
```release-note
Switch default etcd version to 3.0.14.
Switch default storage backend flag in apiserver to `etcd3` mode.
```
Automatic merge from submit-queue
Coreos kube-up now with less cloud init
This update includes significant refactoring. It moves almost all of the
logic into bash scripts, modeled after the `gci` cluster scripts.
The reason to do this is:
1. Avoid duplicating the saltbase manifests by reusing gci's parsing logic (easier maintenance)
2. Take an incremental step towards sharing more code between gci/trusty/coreos, again for better maintenance
3. Pave the way for making future changes (e.g. improved rkt support, kubelet support) easier to share
The primary differences from the gci scripts are the following:
1. Use of the `/opt/kubernetes` directory over `/home/kubernetes`
2. Support for rkt as a runtime
3. No use of logrotate
4. No use of `/etc/default/`
5. No logic related to noexec mounts or gci-specific firewall-stuff
It will make sense to move 2 over to gci, as well as perhaps a few other small improvements. That will be a separate PR for ease of review.
Ref #29720, this is a part of that because it removes a copy of them.
Fixes#24165
cc @yifan-gu
Since this logic largely duplicates logic from the gci folder, it would be nice if someone closely familiar with that gave an OK or made sure I didn't fall into any gotchas related to that, so cc @andyzheng0831
Automatic merge from submit-queue
Add kubernetes-anywhere as a new e2e deployment option.
This change adds support for using `kubernetes-anywhere` as a deployment option for hack/e2e.go. This work is toward the larger goal of being able to run e2e tests against `kubeadm` clusters, which `kubernetes-anywhere` supports.
**Release note**:
```release-note
Add kubernetes-anywhere as a new e2e deployment option
```
The configuration in `getConfig()` comes mostly from the defaults in `kubernetes-anywhere`. In the future, we can add more plumbing to override them via CLI flags.
CC @mikedanese
Automatic merge from submit-queue
add UpdateRuntimeConfig interface
Expose UpdateRuntimeConfig interface in RuntimeService for kubelet to pass a set of configurations to runtime. Currently it only takes PodCIDR.
The use case is for kubelet to pass configs to runtime. Kubelet holds some config/information which runtime does not have, such as PodCIDR. I expect some of kubelet configurations will gradually move to runtime, but I believe cases like PodCIDR, which dynamically assigned by k8s master, need to stay for a while.
Automatic merge from submit-queue
Add gcl cluster logging test
This PR changes default logging destination for tests to gcp and adds test for cluster logging using google cloud logging
Fix#20760
Automatic merge from submit-queue
log-dump.sh: Add a LOG_DUMP_USE_KUBECTL mode, refactor
**What this PR does / why we need it**: This refactors log-dump.sh so that it can optionally just use `"kubectl get node"` output, for dumping logs off a cluster that was brought up using something other than kube-up.sh (e.g. kops or kubeadm).
Note that for obvious reasons, if the cluster is fried, it'll probably fail to hit the apiserver and all bets are off, but for the 95% case where you're debugging tests, this is good. To cover the remaining 5%, you could actually squirrel away the relevant external IPs after the e2e.go:Up stage (because we better be able to hit the apiserver then) and fetch them from cache later.
**Special notes for your reviewer**: Sorry for so many changes. :)
This refactors log-dump.sh so that it can optionally just use "kubectl
get node" output, for dumping logs off a cluster that was brought up
using sometihng other than kube-up.sh (e.g. kops or kubeadm).
Note that for obvious reasons, if the cluster is fried, it'll probably
fail to hit the apiserver and all bets are off, but for the 95% case
where you're debugging tests, this is good. To cover the remaining 5%,
you could actually squirrel away the relevant external IPs after the
e2e.go:Up stage (because we better be able to hit the apiserver then)
and fetch them from cache later.
Michael Taufen