It is also possible to deploy Helm charts. k3s supports a CRD controller for installing charts. A YAML file specification can look as following (example taken from `/var/lib/rancher/k3s/server/manifests/traefik.yaml`):
Keep in mind that `namespace` in your HelmChart resource metadata section should always be `kube-system`, because k3s deploy controller is configured to watch this namespace for new HelmChart resources. If you want to specify the namespace for the actual helm release, you can do that using `targetNamespace` key in the spec section:
```
apiVersion: k3s.cattle.io/v1
kind: HelmChart
metadata:
name: grafana
namespace: kube-system
spec:
chart: stable/grafana
targetNamespace: monitoring
set:
adminPassword: "NotVerySafePassword"
valuesContent: |-
image:
tag: master
env:
GF_EXPLORE_ENABLED: true
adminUser: admin
sidecar:
datasources:
enabled: true
```
Also note that besides `set` you can use `valuesContent` in the spec section. And it's okay to use both of them.
As of version 0.6.0, k3s can support various storage backends including: SQLite (default), MySQL, Postgres, and etcd, this enahancement depends on the following arguments that can be passed to k3s server:
```
--storage-backend value Specify storage type etcd3 or kvsql [$K3S_STORAGE_BACKEND]
--storage-endpoint value Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$K3S_STORAGE_ENDPOINT]
--storage-cafile value SSL Certificate Authority file used to secure storage backend communication [$K3S_STORAGE_CAFILE]
--storage-certfile value SSL certification file used to secure storage backend communication [$K3S_STORAGE_CERTFILE]
--storage-keyfile value SSL key file used to secure storage backend communication [$K3S_STORAGE_KEYFILE]
```
## MySQL
To use k3s with MySQL storage backend, you can specify the following for insecure connection:
```
k3s server --storage-endpoint="mysql://"
```
By default the server will attempt to connect to mysql using the mysql socket at `/var/run/mysqld/mysqld.sock` using the root user and with no password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN.
To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example:
```
k3s server --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest"
```
This command will attempt to connect to MySQL on host `192.168.1.100` on port `3306` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the MySQL driver data source name, please refer to https://github.com/go-sql-driver/mysql#dsn-data-source-name
To connect to MySQL securely, you can use the following example:
```
k3s server --storage-endpoint="mysql://k3suser:k3spass@tcp(192.168.1.100:3306)/k3stest" --storage-cafile ca.crt --storage-certfile mysql.crt --storage-keyfile mysql.key
```
The above command will use these certificates to generate the tls config to communicate with mysql securely.
## Postgres
Connection to postgres can be established using the following command:
```
k3s server --storage-endpoint="postgres://"
```
By default the server will attempt to connect to postgres on localhost with using the `postgres` user and with `postgres` password, k3s will also create a database with the name `kubernetes` if the database is not specified in the DSN.
To override the method of connection, user/pass, and database name, you can provide a custom DSN, for example:
```
k3s server --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest"
```
This command will attempt to connect to Postgres on host `192.168.1.100` on port `5432` with username `k3suser` and password `k3spass` and k3s will automatically create a new database with the name `k3stest` if it doesn't exist, for more information about the Postgres driver data source name, please refer to https://godoc.org/github.com/lib/pq
To connect to Postgres securely, you can use the following example:
```
k3s server --storage-endpoint="postgres://k3suser:k3spass@192.168.1.100:5432/k3stest?sslmode=verify-full" --storage-certfile postgres.crt --storage-keyfile postgres.key --storage-cafile ca.crt
```
The above command will use these certificates to generate the tls config to communicate with postgres securely, note that the `sslmode` in the example is `verify-full` which verify that the certification presented by the server was signed by a trusted CA and the server host name matches the one in the certificate.
## etcd
Connection to postgres can be established using the following command:
```
k3s server --storage-backend=etcd3 --storage-endpoint="https://127.0.0.1:2379"
```
The above command will attempt to connect insecurely to etcd on localhost with port `2379`, you can connect securely to etcd using the following command:
```
k3s server --storage-backend=etcd3 --storage-endpoint="https://127.0.0.1:2379" --storage-cafile ca.crt --storage-certfile etcd.crt --storage-keyfile etcd.key
If you installed your k3s server with the help of `install.sh` script from the root directory, you may use the uninstall script generated during installation, which will be created on your server node at `/usr/local/bin/k3s-uninstall.sh`
k3s will generate config.toml for containerd in `/var/lib/rancher/k3s/agent/etc/containerd/config.toml`, for advanced customization for this file you can create another file called `config.toml.tmpl` in the same directory and it will be used instead.
The `config.toml.tmpl` will be treated as a Golang template file, and the `config.Node` structure is being passed to the template,the following is an example on how to use the structure to customize the configuration file https://github.com/rancher/k3s/blob/master/pkg/agent/templates/templates.go#L16-L32
In this setup you will still be required to install your own CNI driver. More info [here](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network)
CoreDNS
-------
CoreDNS is deployed on start of the agent, to disable add `--no-deploy coredns` to the server
Images needed for a base install are provided through the releases page, additional images can be created with the `docker save` command.
Offline Helm charts are served from the `/var/lib/rancher/k3s/server/static` directory, and Helm chart manifests may reference the static files with a `%{KUBERNETES_API}%` templated variable. For example, the default traefik manifest chart installs from `https://%{KUBERNETES_API}%/static/charts/traefik-X.Y.Z.tgz`.
If networking is completely disabled k3s may not be able to start (ie ethernet unplugged or wifi disconnected), in which case it may be necessary to add a default route. For example:
```sh
sudo ip -c address add 192.168.123.123/24 dev eno1
sudo ip route add default via 192.168.123.1
```
k3s additionally provides a `--resolv-conf` flag for kubelets, which may help with configuring DNS in air-gap networks.
k3s server and agent can be configured with options `--node-label` and `--node-taint` which adds set of Labels and Taints to kubelet, the two options only adds labels/taints at registration time, so they can only be added once and not changed after that, an example to add new label is:
```
k3s server --node-label foo=bar --node-label hello=world --node-taint key1=value1:NoExecute
1. Metrics API ([fixed](https://github.com/rancher/k3s/issues/252): use `k3s server --kubelet-arg="address=0.0.0.0"` and apply `recipes/metrics-server`)
