Automatic merge from submit-queue (batch tested with PRs 47850, 47835, 46197, 47250, 48284)
Securing the cluster created by Juju
**What this PR does / why we need it**: This PR secures the deployments done with Juju master. Works around certain security issues inherent to kubernetes (see for example dashboard access)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
Securing Juju kubernetes dashboard
```
Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)
Add a failsafe for etcd not returning a connection string
**What this PR does / why we need it**: Removing a kubernetes-master will fail as described on this issue: https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/311
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/311
**Special notes for your reviewer**: This is a two liner defensive code. I am not totally sold on this patch. I might not be the right place to address the above issue. However, solving the problem on the etcd side and updating the interface scope to be unit (as suggested) seems much more involving.
**Release note**:
```
Fix error when removing juju kubernetes-master unit
```
Automatic merge from submit-queue
Insert Cynerva and Kjackal to approvers list
**What this PR does / why we need it**:
Per the membership reviews, we're looking to promote Konstantinos and
George to approvers to help distribute the review/bug load for the `cluster/juju` code
tree.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:
cc @marcoceppi and @tvansteenburgh
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 48012, 47443, 47702, 47178)
Fix setting juju worker labels during deployment
**What this PR does / why we need it**: Allows for setting the labels of juju workers during deployment (eg inside a bundle)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47176
**Special notes for your reviewer**:
**Release note**:
```
Fix bug in setting Juju kubernetes-worker labels in bundle.yaml files.
```
Automatic merge from submit-queue (batch tested with PRs 47860, 47170)
Fix restart action on juju kubernetes-master
**What this PR does / why we need it**: Restart action of kubernetes-master of Juju is not functioning.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/299
**Special notes for your reviewer**:
**Release note**:
```
Fix: Restart action of juju's kubernetes-master restarts the respective snap based services
```
Per the membership reviews we're looking to promote Konstantinos and
George to approvers to help distribute the review/bug load for the juju code
tree.
Automatic merge from submit-queue
Remove Initializers from admission-control in kubernetes-master charm for pre-1.7
**What this PR does / why we need it**:
This fixes a problem with the kubernetes-master charm where kube-apiserver never comes up:
```
failed to initialize admission: Unknown admission plugin: Initializers
```
The Initializers plugin does not exist before Kubernetes 1.7. The charm needs to support 1.6 as well.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#47062
**Special notes for your reviewer**:
This fixes a problem introduced by https://github.com/kubernetes/kubernetes/pull/36721
**Release note**:
```release-note
Remove Initializers from admission-control in kubernetes-master charm for pre-1.7
```
This adds additional state guardsof the idle_status method, which will
prevent it from being run until a worker has joined the relationship.
Previous invocations may have some message artifacting, but will reach
eventual consistency once a worker has joined.
This prevents the polling loop from executing too soon, bloating the
installation time by bare-minimum an additional 10 minutes.
Automatic merge from submit-queue (batch tested with PRs 45070, 45821, 45732, 45494, 45789)
Fix lint errors in juju kubernetes master and e2e charms
**What this PR does / why we need it**: Fixes style error in the Juju charms
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
Code style fixes in Juju charms
```
Automatic merge from submit-queue (batch tested with PRs 45283, 45289, 45248, 44295)
Remove offending code due to bad rebase
**What this PR does / why we need it**: Fix bug introduced by bad rebasing
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue
README.md: Update outdated links
**What this PR does / why we need it**:
the PR aims to update some links.
Some links with "#" would not redirect to right point of pages.
Other links without "#" can work, but they are outdated. I change them by the way.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
none
Automatic merge from submit-queue
Retry calls we report config changes quickly.
**What this PR does / why we need it**: In Juju deployments of Kubernetes the status of the charms is updated when a status-update is triggered periodically. As a result changes in config variables may take up to 10 minutes to be reflected on the charms status. See bug below.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/263
**Special notes for your reviewer**:
**Release note**:
```
Kubernetes clusters deployed with Juju pick up config changes faster.
```
Automatic merge from submit-queue (batch tested with PRs 41530, 44814, 43620, 41985)
Fixes juju kubernetes master: 1. Get certs from a dead leader. 2. Append tokens.
**What this PR does / why we need it**:
Fixes two issues with the Juju kubernetes master.
1. Grab certificates from a leader that is already removed.
2. Append (not truncate) auth tokens
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
fixes#43563fixes#43519
**Special notes for your reviewer**:
**Release note**:
```
Recover certificates from leadership context in case all masters die in a Juju deployment
```
Automatic merge from submit-queue (batch tested with PRs 42432, 44628, 45101, 44921)
Use correct option name in the kubernetes-worker layer registry action
**What this PR does / why we need it**: It fixes#44920
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#44920
**Special notes for your reviewer**:
**Release note**:
```
Ensure kubernetes-worker juju layer registry action uses correct ingress controller option name
```
Automatic merge from submit-queue (batch tested with PRs 42740, 44980, 45039, 41627, 45044)
Update kubernetes-e2e charm to use snaps
**What this PR does / why we need it**:
This updates the kubernetes-e2e charm to use snaps instead of Juju resources for payload delivery.
The main advantage of this is that it decouples the charm from the e2e payload, allowing us to support multiple versions of Kubernetes with a single release of the charm.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Update kubernetes-e2e charm to use snaps
```
The master-components started state triggers a daemon recycle. The guard
was to prevent the daemons from being cycled too often and interrupting
normal workflow. This additional state check is guarded against the etcd
connection string from changing, allowing the current behavior but
triggers a re-configure and recycle of the api-control plane when etcd
units are scaling up and down.
Automatic merge from submit-queue
Send dns details only after cdk-addons are configured
**What this PR does / why we need it**: This is a bugfix on the deployment of Kubernetes via Juju. See issue below.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40386 and
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/262
**Special notes for your reviewer**:
**Release note**:
```
Fix KubeDNS issue in Juju deployments.
```
Fix the pause action with regard to the new behavior where
--delete-local-data=false by default. Historically --force was all that
was required, this flag has changed to be more descriptive of the
actions it's taking.
Automatic merge from submit-queue
Add namespace-{list, create, delete} actions to the kubernetes-master layer
**What this PR does / why we need it**:
This PR adds namespace-{list,create,delete} actions to the juju kubernetes-master layer.
**Which issue this PR fixes**: fixes#43712
**Special notes for your reviewer**:
Original PR https://github.com/juju-solutions/kubernetes/pull/109
**Release note**:
```
Add namespace-{list,create,delete} actions to the juju kubernetes-master layer
```
This patch sets the hostname to a unique identifier (the juju unit name)
during pre-deployment of the charm. This may not be a FQDN resolveable
hostname but will prevent hostname collision.
Automatic merge from submit-queue (batch tested with PRs 44722, 44704, 44681, 44494, 39732)
prevent installation of docker from upstream
**What this PR does / why we need it**: Disallows installation of upstream docker from PPA in the Juju kubernetes-worker charm.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Disallows installation of upstream docker from PPA in the Juju kubernetes-worker charm.
```
Automatic merge from submit-queue
select one api endpoint at random when deploying kubernetes-core charm
**What this PR does / why we need it**: Fixes a bug in the kubernetes-worker Juju charm code that attempted to give kube-proxy more than one api endpoint.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**: https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/255
**Release note**:
```release-note
Fixes a bug in the kubernetes-worker Juju charm code that attempted to give kube-proxy more than one api endpoint.
```
Automatic merge from submit-queue
Fix ceph-secret type to kubernetes.io/rbd in kubernetes-master charm
**What this PR does / why we need it**:
This fixes the type of the ceph-secret secret that's created by the kubernetes-master charm.
Without the `kubernetes.io/rbd` type, automatic provisioning of PVCs doesn't work.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix ceph-secret type to kubernetes.io/rbd in kubernetes-master charm
```
Automatic merge from submit-queue (batch tested with PRs 43000, 44500, 44457, 44553, 44267)
Add Kubernetes 1.6 support to Juju charms
**What this PR does / why we need it**:
This adds Kubernetes 1.6 support to Juju charms.
This includes some large architectural changes in order to support multiple versions of Kubernetes with a single release of the charms. There are a few bug fixes in here as well, for issues that we discovered during testing.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Thanks to @marcoceppi, @ktsakalozos, @jacekn, @mbruzek, @tvansteenburgh for their work in this feature branch as well!
**Release note**:
```release-note
Add Kubernetes 1.6 support to Juju charms
Add metric collection to charms for autoscaling
Update kubernetes-e2e charm to fail when test suite fails
Update Juju charms to use snaps
Add registry action to the kubernetes-worker charm
Add support for kube-proxy cluster-cidr option to kubernetes-worker charm
Fix kubernetes-master charm starting services before TLS certs are saved
Fix kubernetes-worker charm failures in LXD
Fix stop hook failure on kubernetes-worker charm
Fix handling of juju kubernetes-worker.restart-needed state
Fix nagios checks in charms
```
Automatic merge from submit-queue (batch tested with PRs 44047, 43514, 44037, 43467)
Juju: Enable GPU mode if GPU hardware detected
**What this PR does / why we need it**:
Automatically configures kubernetes-worker node to utilize GPU hardware when such hardware is detected.
layer-nvidia-cuda does the hardware detection, installs CUDA and Nvidia
drivers, and sets a state that the k8s-worker can react to.
When gpu is available, worker updates config and restarts kubelet to
enable gpu mode. Worker then notifies master that it's in gpu mode via
the kube-control relation.
When master sees that a worker is in gpu mode, it updates to privileged
mode and restarts kube-apiserver.
The kube-control interface has subsumed the kube-dns interface
functionality.
An 'allow-privileged' config option has been added to both worker and
master charms. The gpu enablement respects the value of this option;
i.e., we can't enable gpu mode if the operator has set
allow-privileged="false".
**Special notes for your reviewer**:
Quickest test setup is as follows:
```bash
# Bootstrap. If your aws account doesn't have a default vpc, you'll need to
# specify one at bootstrap time so that juju can provision a p2.xlarge.
# Otherwise you can leave out the --config "vpc-id=vpc-xxxxxxxx" bit.
juju bootstrap --config "vpc-id=vpc-xxxxxxxx" --constraints "cores=4 mem=16G root-disk=64G" aws/us-east-1 k8s
# Deploy the bundle containing master and worker charms built from
# https://github.com/tvansteenburgh/kubernetes/tree/gpu-support/cluster/juju/layers
juju deploy cs:~tvansteenburgh/bundle/kubernetes-gpu-support-3
# Setup kubectl locally
mkdir -p ~/.kube
juju scp kubernetes-master/0:config ~/.kube/config
juju scp kubernetes-master/0:kubectl ./kubectl
# Download a gpu-dependent job spec
wget -O /tmp/nvidia-smi.yaml https://raw.githubusercontent.com/madeden/blogposts/master/k8s-gpu-cloud/src/nvidia-smi.yaml
# Create the job
kubectl create -f /tmp/nvidia-smi.yaml
# You should see a new nvidia-smi-xxxxx pod created
kubectl get pods
# Wait a bit for the job to run, then view logs; you should see the
# nvidia-smi table output
kubectl logs $(kubectl get pods -l name=nvidia-smi -o=name -a)
```
kube-control interface: https://github.com/juju-solutions/interface-kube-control
nvidia-cuda layer: https://github.com/juju-solutions/layer-nvidia-cuda
(Both are registered on http://interfaces.juju.solutions/)
**Release note**:
```release-note
Juju: Enable GPU mode if GPU hardware detected
```
Automatic merge from submit-queue
Adding more proxy options and header to nginx load-balancer.
**What this PR does / why we need it**: The kubeapi-load-balancer uses nginx to proxy commands to the kube-apiserver. It currently does not support SPDY and therefore the `kubectl exec` command is broken.
**Which issue this PR fixes** :
fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/226
fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/201
**Special notes for your reviewer**: This only changes the nginx configuration no code change was required.
**Release note**:
```release-note
Using http2 in kubeapi-load-balancer to fix kubectl exec uses
```
layer-nvidia-cuda does the hardware detection and sets a state that the
worker can react to.
When gpu is available, worker updates config and restarts kubelet to
enable gpu mode. Worker then notifies master that it's in gpu mode via
the kube-control relation.
When master sees that a worker is in gpu mode, it updates to privileged
mode and restarts kube-apiserver.
The kube-control interface has subsumed the kube-dns interface
functionality.
An 'allow-privileged' config option has been added to both worker and
master charms. The gpu enablement respects the value of this option;
i.e., we can't enable gpu mode if the operator has set
allow-privileged="false".
Automatic merge from submit-queue (batch tested with PRs 41919, 41149, 42350, 42351, 42285)
Juju: Disable anonymous auth on kubelet
**What this PR does / why we need it**:
This disables anonymous authentication on kubelet when deployed via Juju.
I've also adjusted a few other TLS options for kubelet and kube-apiserver. The end result is that:
1. kube-apiserver can now authenticate with kubelet
2. kube-apiserver now verifies the integrity of kubelet
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/219
**Special notes for your reviewer**:
This is dependent on PR #41251, where the tactics changes are being merged in separately.
Some useful pages from the documentation:
* [apiserver -> kubelet](https://kubernetes.io/docs/admin/master-node-communication/#apiserver---kubelet)
* [Kubelet authentication/authorization](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/)
**Release note**:
```release-note
Juju: Disable anonymous auth on kubelet
```
Automatic merge from submit-queue (batch tested with PRs 42162, 41973, 42015, 42115, 41923)
add nrpe-external-master relation to kubernetes-master and kubernetes-worker
**What this PR does / why we need it**:
This PR adds an an nrpe-external-master relation to the kubernetes-worker, kubernetes-master and kubeapi-load-balancer charms. This is needed to monitor the state of the workers, the masters and the load-balancers via Nagios.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/165
**Special notes for your reviewer**:
Original work by @axinojolais in PR #40897. All I've done is squash commits on his behalf.
**Release note**:
```release-note
The kubernetes-master, kubernetes-worker and kubeapi-load-balancer charms have gained an nrpe-external-master relation, allowing the integration of their monitoring in an external Nagios server.
```
Automatic merge from submit-queue (batch tested with PRs 42058, 41160, 42065, 42076, 39338)
Juju: Fix shebangs in charm actions to use python3
**What this PR does / why we need it**:
This fixes the microbot and create-rbd-pv actions by reverting them back to python3. We accidentally switched them to python2 to match the boilerplate checker's expectations for python files.
It looks like hack/verify-boilerplate.sh does not check these since they don't have the .py extension, so we should be good with no changes there.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/212
**Special notes for your reviewer**:
**Release note**:
```release-note
Juju: Fix shebangs in charm actions to use python3
```
Automatic merge from submit-queue (batch tested with PRs 40932, 41896, 41815, 41309, 41628)
enable DefaultTolerationSeconds admission controller by default
**What this PR does / why we need it**:
Continuation of PR #41414, enable DefaultTolerationSeconds admission controller by default.
**Which issue this PR fixes**:
fixes: #41860
related Issue: #1574, #25320
related PRs: #34825, #41133, #41414
**Special notes for your reviewer**:
**Release note**:
```release-note
enable DefaultTolerationSeconds admission controller by default
```
Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)
Multi master patch
**What this PR does / why we need it**: Corrects a sync files issue present when running in a HA Master configuration. This PR adds logic to syncronize on first deployment for `/etc/kubernetes/serviceaccount.key` which will cause cypto verification failure if not 1:1 on each master unit. Additionally syncs basic_auth and additional files in /srv/kubernetes.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#41019
**Special notes for your reviewer**: This requires PR #41251 as a dependency before merging.
**Release note**:
```release-note
Juju - K8s master charm now properly keeps distributed master files in sync for an HA control plane.
```
This branch sync's the crypto keys, and flat-files used for auth with
all the masters when scaling the kubernetes-master units. This should
fix the mis-matched crypto keys seen when rebooting units after first
deploy.
Automatic merge from submit-queue (batch tested with PRs 40691, 40551, 40683, 40700, 40702)
Juju kubernetes-master charm: improve status messages
**What this PR does / why we need it**:
This update to the kubernetes-master charm does the following:
1. Remove "Kubernetes master services ready" status which was occurring too early
2. Add "Waiting for kube-system pods to start" status
3. Replace "Rendering the Kubernetes DNS files." status with "Deploying KubeDNS"
4. Add "Waiting to retry KubeDNS deployment" status
The purpose of this is to give better feedback to the operator during cluster deployment.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/143, which we are tracking in a separate repository
**Special notes for your reviewer**:
This is a rebase of https://github.com/juju-solutions/kubernetes/pull/103, where prior review was done, though it was targeted against a fork.
**Release note**:
```release-note
Juju kubernetes-master charm: improve status messages
```
George Kraft