diff --git a/docs/proposals/local-cluster-ux.md b/docs/proposals/local-cluster-ux.md new file mode 100644 index 0000000000..39e4d74e12 --- /dev/null +++ b/docs/proposals/local-cluster-ux.md @@ -0,0 +1,190 @@ + + + + +WARNING +WARNING +WARNING +WARNING +WARNING + +

PLEASE NOTE: This document applies to the HEAD of the source tree

+ +If you are using a released version of Kubernetes, you should +refer to the docs that go with that version. + +Documentation for other releases can be found at +[releases.k8s.io](http://releases.k8s.io). + +-- + + + + + +# Kubernetes Local Cluster Experience + +This proposal attempts to improve the existing local cluster experience for kubernetes. +The current local cluster experience is sub-par and often not functional. +There are several options to setup a local cluster (docker, vagrant, linux processes, etc) and we do not test any of them continuously. +Here are some highlighted issues: +- Docker based solution breaks with docker upgrades, does not support DNS, and many kubelet features are not functional yet inside a container. +- Vagrant based solution are too heavy and have mostly failed on OS X. +- Local linux cluster is poorly documented and is undiscoverable. +From an end user perspective, they want to run a kubernetes cluster. They care less about *how* a cluster is setup locally and more about what they can do with a functional cluster. + + +## Primary Goals + +From a high level the goal is to make it easy for a new user to run a Kubernetes cluster and play with curated examples that require least amount of knowledge about Kubernetes. +These examples will only use kubectl and only a subset of Kubernetes features that are available will be exposed. + +- Works across multiple OSes - OS X, Linux and Windows primarily. +- Single command setup and teardown UX. +- Unified UX across OSes +- Minimal dependencies on third party software. +- Minimal resource overhead. +- Eliminate any other alternatives to local cluster deployment. + +## Secondary Goals + +- Enable developers to use the local cluster for kubernetes development. + +## Non Goals + +- Simplifying kubernetes production deployment experience. [Kube-deploy](https://github.com/kubernetes/kube-deploy) is attempting to tackle this problem. +- Supporting all possible deployment configurations of Kubernetes like various types of storage, networking, etc. + + +## Local cluster requirements + +- Includes all the master components & DNS (Apiserver, scheduler, controller manager, etcd and kube dns) +- Basic auth +- Service accounts should be setup +- Kubectl should be auto-configured to use the local cluster +- Tested & maintained as part of Kubernetes core + +## Existing solutions + +Following are some of the existing solutions that attempt to simplify local cluster deployments. + +### [Spread](https://github.com/redspread/spread) + +Spread's UX is great! +It is adapted from monokube and includes DNS as well. +It satisfies almost all the requirements, excepting that of requiring docker to be pre-installed. +It has a loose dependency on docker. +New releases of docker might break this setup. + +### [Kmachine](https://github.com/skippbox/kmachine) + +Kmachine is adapted from docker-machine. +It exposes the entire docker-machine CLI. +It is possible to repurpose Kmachine to meet all our requirements. + +### [Monokube](https://github.com/polvi/monokube) + +Single binary that runs all kube master components. +Does not include DNS. +This is only a part of the overall local cluster solution. + +### Vagrant + +The kube-up.sh script included in Kubernetes release supports a few Vagrant based local cluster deployments. +kube-up.sh is not user friendly. +It typically takes a long time for the cluster to be set up using vagrant and often times is unsuccessful on OS X. +The [Core OS single machine guide](https://coreos.com/kubernetes/docs/latest/kubernetes-on-vagrant-single.html) uses Vagrant as well and it just works. +Since we are targeting a single command install/teardown experience, vagrant needs to be an implementation detail and not be exposed to our users. + +## Proposed Solution + +To avoid exposing users to third party software and external dependencies, we will build a toolbox that will be shipped with all the dependencies including all kubernetes components, hypervisor, base image, kubectl, etc. +*Note: Docker provides a [similar toolbox](https://www.docker.com/products/docker-toolbox).* +This "Localkube" tool will be referred to as "Minikube" in this proposal to avoid ambiguity against Spread's existing ["localkube"](https://github.com/redspread/localkube). +The final name of this tool is TBD. Suggestions are welcome! + +Minikube will provide a unified CLI to interact with the local cluster. +The CLI will support only a few operations: + - **Start** - creates & starts a local cluster along with setting up kubectl & networking (if necessary) + - **Stop** - suspends the local cluster & preserves cluster state + - **Delete** - deletes the local cluster completely + - **Upgrade** - upgrades internal components to the latest available version (upgrades are not guaranteed to preserve cluster state) + +For running and managing the kubernetes components themselves, we can re-use [Spread's localkube](https://github.com/redspread/localkube). +Localkube is a self-contained go binary that includes all the master components including DNS and runs them using multiple go threads. +Each Kubernetes release will include a localkube binary that has been tested exhaustively. + +To support Windows and OS X, minikube will use [libmachine](https://github.com/docker/machine/tree/master/libmachine) internally to create and destroy virtual machines. +Minikube will be shipped with an hypervisor (virtualbox) in the case of OS X. +Minikube will include a base image that will be well tested. + +In the case of Linux, since the cluster can be run locally, we ideally want to avoid setting up a VM. +Since docker is the only fully supported runtime as of Kubernetes v1.2, we can initially use docker to run and manage localkube. +There is risk of being incompatible with the existing version of docker. +By using a VM, we can avoid such incompatibility issues though. +Feedback from the community will be helpful here. + +If the goal is to run outside of a VM, we can have minikube prompt the user if docker is unavailable or version is incompatible. +Alternatives to docker for running the localkube core includes using [rkt](https://coreos.com/rkt/docs/latest/), setting up systemd services, or a System V Init script depending on the distro. + +To summarize the pipeline is as follows: + +##### OS X / Windows + +minikube -> libmachine -> virtualbox/hyper V -> linux VM -> localkube + +##### Linux + +minikube -> docker -> localkube + +### Alternatives considered + +#### Bring your own docker + +##### Pros + +- Kubernetes users will probably already have it +- No extra work for us +- Only one VM/daemon, we can just reuse the existing one + +##### Cons + +- Not designed to be wrapped, may be unstable +- Might make configuring networking difficult on OS X and Windows +- Versioning and updates will be challenging. We can mitigate some of this with testing at HEAD, but we'll - inevitably hit situations where it's infeasible to work with multiple versions of docker. +- There are lots of different ways to install docker, networking might be challenging if we try to support many paths. + +#### Vagrant + +##### Pros + +- We control the entire experience +- Networking might be easier to build +- Docker can't break us since we'll include a pinned version of Docker +- Easier to support rkt or hyper in the future +- Would let us run some things outside of containers (kubelet, maybe ingress/load balancers) + +##### Cons + +- More work +- Extra resources (if the user is also running docker-machine) +- Confusing if there are two docker daemons (images built in one can't be run in another) +- Always needs a VM, even on Linux +- Requires installing and possibly understanding Vagrant. + +## Releases & Distribution + +- Minikube will be released independent of Kubernetes core in order to facilitate fixing of issues that are outside of Kubernetes core. +- The latest version of Minikube is guaranteed to support the latest release of Kubernetes, including documentation. +- The Google Cloud SDK will package minikube and provide utilities for configuring kubectl to use it, but will not in any other way wrap minikube. + + + + +[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/proposals/local-cluster-ux.md?pixel)]() +