This allows others to leverage the existing E2E code to test some
patched kube binary by simply overriding the bucket and reusing many of
the existing scripts
If "a" is a bash array, then the syntax to append the contents of $line as a
new element to the array is a+=("$line"), not messages+=$line
Using the former syntax just seems to append to the first element, creating a
long string and thus losing newline information.
Fixing this allows us to drop some empty lines from invocations of
usage_from_stdin.
Automatic merge from submit-queue
Fix GKE kube-up to correctly find an IGM from a multi-zone cluster
I've confirmed that this successfully brings up a cluster, fixing the immediate issue with the new e2e test. Sorry about not properly vetting it in the original PR (#24075).
This does cause a warning message to be printed based on the handling of the NUM_NODES variable though, which I could fix if you guys think it's worth it:
```
Detected 6 ready nodes, found 6 nodes out of expected 3. Found more nodes than expected, your cluster may not behave correctly.
```
@quinton-hoole
Automatic merge from submit-queue
Fix log dump for new gcloud
`gcloud compute instance-groups managed list-instances` at CI has self-link for instance instead of just name. Fixes#24120
Automatic merge from submit-queue
Trusty: Fixes for running GKE master
This PR includes two fixes for running GKE master on our image:
(1) The kubelet command line assembly had a missing part for cbr0. We did not catch it because the code path is not covered by OSS k8s tests;
(2) Remove the "" from the variables in the cert files. It causes a parsing issue in GKE. Again, this code path is not covered by k8s tests.
This PR also refactors the code for assembling kubelet flag. I move all logic into a single function assemble_kubelet_flags in configure-helper.sh for better readability and also simplify node.yaml and master.yaml.
@roberthbailey @dchen1107 please review it, and mark it as cherrypick-candidate. This PR is verified by @maisem. Together with his CL for GKE, we can run GKE cluster with master on our image and nodes on ContainerVM.
cc/ @maisem @fabioy @wonderfly FYI
This only applies to gce kube-up. 60 seconds of open connection should
be sufficient for anything that we should be downloading. The release
tar is currently 255M.
Automatic merge from submit-queue
Make kube2sky and skydns docker images cross-platform
ARM tracking issue: #17981
Continues on: #19216
Make it possible to create `kube2sky` and `skydns` docker images for ARM and other architectures too
Build in a container, so `golang` isn't a dependency
I've preserved the original default behaviour:
- `skydns`: It just compiles with go on host
- `kube2sky`: Build an image
@brendandburns @dchen1107 @ArtfulCoder @thockin @fgrzadkowski
Automatic merge from submit-queue
Up to golang 1.6
A second attempt to upgrade go version above `go1.4`
Merge ASAP after you've cut the `release-1.2` branch and feel ready.
`go1.6` should perform slightly better than `go1.5`, so this time it might work
@gmarek @wojtek-t @zmerlynn @mikedanese @brendandburns @ixdy @thockin
pause is built into containervm. if it's not on the machine we should just pull
it. nobody that I'm aware of uses kube-registry-proxy and it makes build/deployment
more complicated and slower.
Automatic merge from submit-queue
Cross-build hyperkube and debian-iptables for ARM. Also add a flannel image
We have to be able to build complex docker images too on `amd64` hosts.
Right now we can't build Dockerfiles with `RUN` commands when building for other architectures e.g. ARM.
Resin has a tutorial about this here: https://resin.io/blog/building-arm-containers-on-any-x86-machine-even-dockerhub/
But it's a bit clumsy syntax.
The other alternative would be running this command in a Makefile:
```
# This registers in the kernel that ARM binaries should be run by /usr/bin/qemu-{ARCH}-static
docker run --rm --privileged multiarch/qemu-user-static:register --reset
```
and
```
ADD https://github.com/multiarch/qemu-user-static/releases/download/v2.5.0/x86_64_qemu-arm-static.tar.xz /usr/bin
```
Then the kernel will be able to differ ARM binaries from amd64. When it finds a ARM binary, it will invoke `/usr/bin/qemu-arm-static` first and lets `qemu` translate the ARM syscalls to amd64 ones.
Some code here: https://github.com/multiarch
WDYT is the best approach? If registering `binfmt_misc` in the kernels of the machines is OK, then I think we should go with that.
Otherwise, we'll have to wait for resin's patch to be merged into mainline qemu before we may use the code I have here now.
@fgrzadkowski @david-mcmahon @brendandburns @zmerlynn @ixdy @ihmccreery @thockin
Automatic merge from submit-queue
support NETWORK_PROVIDER=cni for KUBERNETES_PROVIDER=vagrant
While trying to develop CNI plugins for K8's, I found the docs referenced the support of --network-plugin=cni for kubelet, but this wasn't surfaced up via salt to support env NETWORK_PROVIDER=cni before a kube-up deployment.
This PR is my attempt at adding CNI support to the kube-up happy path, following a lot of similar work for NETWORK_PROVIDER=kubenet which already exists.
Also, I've added the ability to consume CNI plugin's (binaries) and configuration files from the local cluster/network-plugins directory into the necessary locations as referenced here for CNI:
http://kubernetes.io/docs/admin/network-plugins
This allows a local developer to easily work on CNI plugin development while following the existing kube-up.sh docs and process.
In general, i've struggled to find any authoritative information or answers to my questions in slack regarding CNI progress / correct integration, so comments encouraged here!
Files are taken from cluster/network-plugins/{bin,conf} to be consumed within a vagrant kube-up.sh environment.
Paths used for configuration files and the 'cni' name of the network provider are all from the kubernetes documentation, but the actual implementation in the salt automation doesn't seem to exist.
Use of NETWORK_PROVIDER=cni is documented as useable (as well as it's affects on the runtime args of kubelet),
however the actual implimentation in the salt automation doesnt seem to exist.
this change attempts to fix that for the vagrant usecase.
Automatic merge from submit-queue
use apply instead of create to setup namespaces and tokens in addon manager
when the addon manager restarts, it takes ~15 minutes (1000 seconds) to start the sync loop because it retries creation of namespace and tokens 100 times. Create fails if the tokens already exist. Just use apply.
Automatic merge from submit-queue
Juju kube up
I found some problems with the kube-up script that this pull request addresses. We didn't have the kubectl binary in the correct location.
Just changing where we download the package from the master, and fixing the kube-down.sh script to remove those files.
We rename it to EPHEMERAL_BLOCK_DEVICE_MAPPINGS, and we also change the value
so that it starts with a `,`, instead of always inserting a comma before it.
In this way the value can be empty.
Also, if the user sets the (currently experimental) KUBE_AWS_STORAGE
environment variable to be "ebs", then we will not mount any instance storage
which will cause the machines to use EBS storage instead.
format-disks used to run with non-strict bash semantics, but this changed in
1.2 as we now merge it into the GCE script, so pipefail and errexit are both
set.
However, the way we list the ephemeral disks, by piping to grep, would cause an
exit code of 2 if there were no ephemeral disks.
Tolerate failure here by add `|| true`. The metadata service call is unlikely
to fail, so we continue to ignore that possibility.
Automatic merge from submit-queue
Fix so setup-files don't recreate/invalidate certificates that already exist
Fixes: #23197 and a lot of other DNS and dashboard issues
This is quite critical for `docker`-based users and should be considered as a **cherrypick-candidate** as it makes a lot of people wonder why Dashboard and/or DNS doesn't work. Example: https://github.com/kubernetes/dashboard/issues/374
Earlier when you shut your `docker.md` cluster down and started it again, all ServiceAccounts became invalidated by `setup-files` that happily ran once again and replaced all files. That made `apiserver` and `controller-manager` pick up the new certs (or there was a race condition, they _could_ have picked up the old certs too, but that's unlikely) and the old certs were put into `/var/run/secrets` because the ServiceAccount's Secrets were stored in etcd, which `setup-files` didn't touch.
@fgrzadkowski @huggsboson @thockin @mikedanese @vishh @pwittrock @eparis @bgrant0607
Automatic merge from submit-queue
Trusty: Regional release .tar.gz support
@zmerlynn and @roberthbailey please review it. This change is to support the feature added in PR #22234. The entire logic is pretty much the same as in #22234, with only few minor changes in implementation.
I had manually run e2e tests with "export RELEASE_REGION_FALLBACK=true" on two clusters: (1) Trusty on master nodes on ContainerVM; (2) Master and nodes all on trusty. All tests are green. I don't figure out a way to simulate regional fallback. But I did test the function download_or_bust() out-of-box.
cc/ @wonderfly @dchen1107 @fabioy FYI.
Sadly, kube-system isn't automatically created, so people need to make
sure to create it in their turnup scripts. Also after creating
kube-system it can take 10+ seconds for master and proxy to show up.
I tested the equivalent of these changes locally, but not these changes
themselves as I don't have a dev/build env up, so please read carefully
and maybe try them out!
Use kubectl create ns
Automatic merge from submit-queue
Create a new Deployment in kube-system for every version.
It appears that version numbers have already been properly added to these files. Small change to delete an old deployment entirely, so we can make a new one per version (like replication controllers).
We'll want to change this back once the kube-addons support deployments in a later version.
Mostly doc updates and cruft removal
- describe conformance test policy and howto in e2e-tests.md
- rm e2e test info from testing.md in the name of DRY
- rm cluster/test-conformance.sh; unusable in release tar, not e2e.go
- update e2e test link in write-a-getting-started-guide.md
There are actually two `roles` setting in ubuntu installation scripts.
One is roles as string, which can be set as env and then used in scripts.
The other is roles as array, which is used by internal handling to
locate specific role by offset.
This patch tries to distinguish roles meaning by declearing the second
as roles_array, thus eliminating its ambiguity.
When using this flag, this error is shown:
Flag --api-version has been deprecated, flag is no longer respected and will be deleted in the next release
Stop using the flag in the validate-cluster.sh script and avoid the warning.
This commit imports the latest development focus from the Charmer team
working to deliver Kubernetes charms with Juju.
Notable Changes:
- The charm is now assembled from layers in $JUJU_ROOT/layers
- Prior, the juju provider would compile and fat-pack the charms, this
new approach delivers the entirety of Kubernetes via hyperkube.
- Adds Kubedns as part of `cluster/kube-up.sh` and verification
- Removes the hard-coded port 8080 for the Kubernetes Master
- Includes TLS validation
- Validates kubernetes config from leader charm
- Targets Juju 2.0 commands
This should allow allow the non_masquerade_cidr option to get configured
in /etc/salt/minion.d/grains.conf, allowing the flag to used by kubelet
in /etc/sysconfig/kubelet. Default configuration is set in pillar
If we deleted an ELB, we often fail to delete the security group,
because deleting the ELB is invisibly asynchronous.
Add a retry loop around delete-security-group to work around this.
Fix#21147
The only tested-working distros are vivid, wily & jessie.
vivid should not really be used because it is no longer supported, so
recommend wily or jessie instead.
For other distros, recommend jessie instead.
Fix#21218
The previous jessie image had a broken cloud-init, which would use an
Ubuntu-specific 'nobootwait' argument when mounting disks. We now
override that in the image.
Fix#22549
also:
- adds a mechanism to build and upload hyperkube for non-official
releases
- adds a mechanism for proxying azkube's traffic
- --no-cloud-provider for now
- support specifying the resource group for CI scenarios
Allow the gcr.io/google_containers registry to be overridden
regionally by just blasting a new KUBE_ADDON_REGISTRY out. Instead of
adding every addon to Salt and asking all of the other consumers
(Trusty, Juju, Mesos, etc) to change, just script the sed ourselves.
This is probably the 9th grossest thing I've ever done, but it works
well, and it works quickly. I kind of wish it didn't.
It includes some performance improvements for parsing JSON (which is
very important for us, since all Docker logs are JSON) as well as a
couple new settings, like forcing of a flush of multiline logs after a
time period rather than having to wait until a new log is seen before
feeling confident flushing the previous one.
-Remove CPU limits to enable CPU bursting once 1.2 begins enforcing CPU limits.
-Add a memory limit for fluentd-es to match fluentd-gcp.
-Explicitly set requests to match limits.
This change revises the way to provide kube-system manifests for clusters on Trusty. Originally, we maintained copies of some manifests under cluster/gce/trusty/kube-manifests, which is not scalable and hard to maintain. With this change, clusters on Trusty will use the same source of manifests as ContainerVM. This change also fixes some minor problems such as shell variables and comments to meet the style guidance better.
Starting docker through Salt has always been problematic. Kubelet or
the babysitter process should start it. We've kept it around primarily
so we have a `service: docker` node for the Salt DAG.
Instead, we enable (but do not start) the Docker service in Salt. This
lets us keep the DAG node, but won't start it.
There's another bug in Salt, where watches will start the service even
on `service.enabled`. So we remove the watches, and move them to our
existing Salt bug-fix script.
The kubelet flag "nosystem" was removed recently, which breaks kubelet in Trusty. This changes remove the flag usage accordingly. It also revises several aspects of Trusty support to make it in the same page as running on ContainerVM, such as new flags in kubelet and new logic in api-server and etcd pods.
The Docker 1.9.1 package on Debian is broken, and the service fails to
install when run unattended. This is treated as an installation failure
and causes everything to fail.
However, the service can be started by Salt once we're not installing
the package, and indeed we restart docker anyway.
So, on Debian, use a helper script to install the docker package. The
script sets up a policy-rc.d file to prevent the service starting, and
then cleanly removes it afterwards (this would be difficult to do in
Salt, I believe).
Euan Kemp