This registry can be accessed through proxies that run on each node
listening on port 5000. We send the proxy images to the nodes directly
to avoid requests that hit the network during cluster launch. For now,
we continue to pull the registry itself over the network, especially
given its large size (we should be able to dramatically shrink the
image). On GCE we create a PD and use that for storage, otherwise we
use an emptyDir. The registry is not enabled outside of GCE. All
communication is currently plain HTTP. In order to use SSL, we will
need to be able to request a certificate/key from the apiserver signed
by the apiserver's CA cert.
Instead of hard coding kube-cert and /srv/kubernetes allow these to be
overwritten by environment variables. / is immutable on some systems
and so /srv is not a possible location to store data.
This may help Salt reload services correctly, although we still
need the script until Salt's bug with reloading services on systemd
is resolved.
Salt bug: https://github.com/saltstack/salt/issues/16778
Changes include:
- Add kube-ui binary for serving static dashboard UI
- Add kube-ui docker image, replication controller, and service
- Make the kube-ui a cluster-addon (enabled by default)
- Split the compiled pkg/ui/datafile.go into separate dashboard and swagger packages
- Update docs to reflect changes
Was previously kubernetes_auth format.
Added defaults file which uses salt to fill in an env var
with the master's IP.
More thought needs to be given soon to how to make this
connection use a cert for the master, and how to support
multiple masters, and whether to use the DNS record
instead of an IP address. But this PR unblocks some other
more urgent things, so doing it this way.
These secrets will be used in subsequent PRs by:
scheduler, controller-manager, monitoring services,
logging services, and skydns.
Each of these services will then be able to stop using kubernetes-ro
or host networking.
The better solution is some fence with Salt, but the actual logs
provided in the bug don't support any race condition here, plus the
ordering in the Salt configuration seems correct.
We haven't seen this again in a while, but given the results of the
situation (a borked cluster), I'm proposing a relatively simple
workaround.
Fixes#4357 (dubiously)
This implements phase 1 of the proposal in #3579, moving the creation
of the pods, RCs, and services to the master after the apiserver is
available.
This is such a wide commit because our existing initial config story
is special:
* Add kube-addons service and associated salt configuration:
** We configure /etc/kubernetes/addons to be a directory of objects
that are appropriately configured for the current cluster.
** "/etc/init.d/kube-addons start" slurps up everything in that dir.
(Most of the difficult is the business logic in salt around getting
that directory built at all.)
** We cheat and overlay cluster/addons into saltbase/salt/kube-addons
as config files for the kube-addons meta-service.
* Change .yaml.in files to salt templates
* Rename {setup,teardown}-{monitoring,logging} to
{setup,teardown}-{monitoring,logging}-firewall to properly reflect
their real purpose now (the purpose of these functions is now ONLY to
bring up the firewall rules, and possibly to relay the IP to the user).
* Rework GCE {setup,teardown}-{monitoring,logging}-firewall: Both
functions were improperly configuring global rules, yet used
lifecycles tied to the cluster. Use $NODE_INSTANCE_PREFIX with the
rule. The logging rule needed a $NETWORK specifier. The monitoring
rule tried gcloud describe first, but given the instancing, this feels
like a waste of time now.
* Plumb ENABLE_CLUSTER_MONITORING, ENABLE_CLUSTER_LOGGING,
ELASTICSEARCH_LOGGING_REPLICAS and DNS_REPLICAS down to the master,
since these are needed there now.
(Desperately want just a yaml or json file we can share between
providers that has all this crap. Maybe #3525 is an answer?)
Huge caveats: I've gone pretty firm testing on GCE, including
twiddling the env variables and making sure the objects I expect to
come up, come up. I've tested that it doesn't break GKE bringup
somehow. But I haven't had a chance to test the other providers.
Prashanth Balasubramanian