Automatic merge from submit-queue
Ubuntu: Enable ssh compression when downloading binaries during cluster creation
<!--
Checklist for submitting a Pull Request
Please remove this comment block before submitting.
1. Please read our [contributor guidelines](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md).
2. See our [developer guide](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md).
3. If you want this PR to automatically close an issue when it is merged,
add `fixes #<issue number>` or `fixes #<issue number>, fixes #<issue number>`
to close multiple issues (see: https://github.com/blog/1506-closing-issues-via-pull-requests).
4. Follow the instructions for [labeling and writing a release note for this PR](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes) in the block below.
-->
resolves#20971 by using the options provided by ssh.
Native ssh compression has existed for years, and the server is free to disregard the setting, so this should be safe.
With things like the kube binaries I see about a 2x speed increase.
```
λ time scp kubes-bin.tar 9.30.182.251:/mnt/build/kubin
kubes-bin.tar 100% 344MB 10.7MB/s 00:32
real 0m32.284s
user 0m1.679s
sys 0m1.263s
λ time scp -C kubes-bin.tar 9.30.182.251:/mnt/build/kubin
kubes-bin.tar 100% 344MB 22.9MB/s 00:15
real 0m14.810s
user 0m12.858s
sys 0m0.994s
λ ls -lah kubes-bin.tar
-rw-r--r-- 1 mhb staff 344M Jun 2 15:29 kubes-bin.tar
λ tar -tf kubes-bin.tar
kubectl
master/
master/etcd
master/etcdctl
master/flanneld
master/kube-apiserver
master/kube-controller-manager
master/kube-scheduler
node/
node/flanneld
node/kube-proxy
node/kubelet
```
Automatic merge from submit-queue
fix logrotate config (again)
we need to add the dateformat option so that the logrotate
can create unique logfiles for each rotation. Without this,
logrotation is skipped with message like (generated in
verbose mode of logrotate):
rotating log /var/log/rotate-test.log, log->rotateCount is 5
dateext suffix '-20160718'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
destination /var/log/rotate-test2.log-20160718.gz already exists, skipping rotation
Tested as follows:
# config in '/etc/logrotate.d/rotate-test':
/var/log/rotate-test.log {
rotate 5
copytruncate
missingok
notifempty
compress
maxsize 100M
daily
dateext
dateformat -%Y%m%d-%s
create 0644 root root
}
# create 150Mb of /var/log/rotate-test.log
$ dd if=/dev/zero of=/var/log/rotate-test.log bs=1048576 count=150 conv=notrunc oflag=append
# run logrotate
$ /usr/sbin/logrotate -v /etc/logrotate.conf
...
rotating pattern: /var/log/rotate-test.log after 1 days (5 rotations)
empty log files are not rotated, log files >= 104857600 are rotated earlier, old logs are removed
considering log /var/log/rotate-test.log
log needs rotating
rotating log /var/log/rotate-test.log, log->rotateCount is 5
Converted ' -%Y%m%d-%s' -> '-%Y%m%d-%s'
dateext suffix '-20160718-1468875268'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
copying /var/log/rotate-test.log to /var/log/rotate-test.log-20160718-1468875268
truncating /var/log/rotate-test.log
compressing log with: /bin/gzip
Repeating 'dd' and 'logrotate' commands now generate logfiles correctly.
#27754
@bprashanth can you please review?
Automatic merge from submit-queue
hyperkube: fix build for 3rd party registry (again)
Fixes issue #28487
This is a minor fix for the issue reported in #28487
we need to add the dateformat option so that the logrotate
can create unique logfiles for each rotation. Without this,
we logrotation is skipped with message like (generated in
verbose mode of logrotate):
rotating log /var/log/rotate-test.log, log->rotateCount is 5
dateext suffix '-20160718'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
destination /var/log/rotate-test2.log-20160718.gz already exists, skipping rotation
Tested as follows:
# config in '/etc/logrotate.d/rotate-test':
/var/log/rotate-test.log {
rotate 5
copytruncate
missingok
notifempty
compress
maxsize 100M
daily
dateext
dateformat -%Y%m%d-%s
create 0644 root root
}
# create 150Mb of /var/log/rotate-test.log
$ dd if=/dev/zero of=/var/log/rotate-test.log bs=1048576 count=150 conv=notrunc oflag=append
# run logrotate
$ /usr/sbin/logrotate -v /etc/logrotate.conf
...
rotating pattern: /var/log/rotate-test.log after 1 days (5 rotations)
empty log files are not rotated, log files >= 104857600 are rotated earlier, old logs are removed
considering log /var/log/rotate-test.log
log needs rotating
rotating log /var/log/rotate-test.log, log->rotateCount is 5
Converted ' -%Y%m%d-%s' -> '-%Y%m%d-%s'
dateext suffix '-20160718-1468875268'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
copying /var/log/rotate-test.log to /var/log/rotate-test.log-20160718-1468875268
truncating /var/log/rotate-test.log
compressing log with: /bin/gzip
Repeating 'dd' and 'logrotate' commands now generate logfiles correctly.
Automatic merge from submit-queue
Unset KUBERNETES_PROVIDER when KUBERNETES_CONFORMANCE_TEST is set
fixes#26269
same as #26530 - i accidentally lost my fork and couldn't rebase there ;)
@mikedanese PTAL
Automatic merge from submit-queue
kube-up: install new Docker pre-requisite (libltdl7) when not in image
Docker now has a dependency on libltdl7; we have to specify it manually
if we are installing docker using dpkg (vs using apt-get or similar,
which would pull it in automatically)
Fixes#28644
This allows us to start building real dependencies into Makefile.
Leave old hack/* scripts in place but advise to use 'make'. There are a few
rules that call things like 'go run' or 'build/*' that I left as-is for now.
Docker now has a dependency on libltdl7; we have to specify it manually
if we are installing docker using dpkg (vs using apt-get or similar,
which would pull it in automatically)
Fixes#28644
Automatic merge from submit-queue
Fixes#28205, Check release tar location for Openstack-Heat provider
This does a basic check to see where the release tars are located.
Allows people to use openstack-heat outside of compiling k8s.
Automatic merge from submit-queue
Enable extensions/v1beta1/NetworkPolicy by default
Fixes https://github.com/kubernetes/kubernetes/issues/28401
For some reason this also triggered an update to the swagger spec (which apparently hadn't been done before but wasn't failing validation...)
Automatic merge from submit-queue
Implementing a proper master/worker split in the juju cluster code.
```
release-note-none
```
General updates to the cluster/juju Kubernetes provider, to bring it up to date.
Updating the skydns templates to version 11
Updating the etcd container definition to include arch.
Updating the master template to include arch and version for hyperkube container.
Adding dns_domain configuration options.
Adding storage layer options.
[]()
Automatic merge from submit-queue
Check existence of kubernetes dir for get-kube.sh
[]()
There are a lot of references to https://get.k8s.io/ over the internet.
Most of such references do not describe KUBERNETES_SKIP_DOWNLOAD env variable
and newbies can get into a situation described below:
- execute `wget -q -O - https://get.k8s.io | bash`
- receive a failure due too missed packages or some configs
- fix the issue
- try again `wget -q -O - https://get.k8s.io | bash`
In this case, get-kube.sh will not check that kubernetes directory already
exist and repeat download again.
Lets make get-kube.sh more user-friendly and check existence of kubernetes dir
Updating the skydns templates to version 11
Updating the etcd container definition to include arch.
Updating the master template to include arch and version for hyperkube container.
Adding dns_domain configuration options.
Adding storage layer options.
Fixing underscore problem and adding exceptions.
Fixing the underscore flag errors.
Automatic merge from submit-queue
Make GKE detect-instance-groups work on Mac.
Make the fix from #27803 also work on mac.
The GNU `expr` command supports both the `expr match STRING REGEXP` and `expr STRING : REGEXP` command syntax.
The BSD `expr` command only has the `expr STRING : REGEXP` syntax.
@fabioy @a-robinson
Automatic merge from submit-queue
Bump skydns godeps to latest
Update Godeps for github.com/skynetservices/skydns and miekg/dns.
Bump kubedns version to 1.6 with latest skynetservices/skydns code
Built kube-dns for all architectures and pushed containers to gcr.io.
Automatic merge from submit-queue
Enhance kubedns pod health checks to cover kubedns container
The existing health check hits port 53, the dnsmasq container, with the same domain name every time. Since dnsmasq looks up and caches results from the kubedns container, running on port 10053, the health check is not covering the kubedns container after the first query (and once every TTL expiration).
This PR enhances the health check to directly hit port 10053 (kubedns) in addition to port 53.
Automatic merge from submit-queue
Substitute federation_domain_map parameter with its value in node bootstrap scripts.
This PR also removes the substitution code we added to the build scripts.
**Release Note**
```release-note
If you use one of the kube-dns replication controller manifest in `cluster/saltbase/salt/kube-dns`, i.e. `cluster/saltbase/salt/kube-dns/{skydns-rc.yaml.base,skydns-rc.yaml.in}`, either substitute one of `__PILLAR__FEDERATIONS__DOMAIN__MAP__` or `{{ pillar['federations_domain_map'] }}` with the corresponding federation name to domain name value or remove them if you do not support cluster federation at this time. If you plan to substitute the parameter with its value, here is an example for `{{ pillar['federations_domain_map'] }`
pillar['federations_domain_map'] = "- --federations=myfederation=federation.test"
where `myfederation` is the name of the federation and `federation.test` is the domain name registered for the federation.
```
cc @erictune @kubernetes/sig-cluster-federation @MikeSpreitzer @luxas
[]()
Automatic merge from submit-queue
Remove duplicated nginx image. Use nginx-slim instead
This PR removes the image `gcr.io/google_containers/nginx:1.7.9` and uses `gcr.io/google_containers/nginx-slim:0.7`.
Besides removing the duplication `1.7.9` is 16 months old.
Automatic merge from submit-queue
Making DHCP_OPTION_SET_ID creation optional
Reason: We have a pre-configured VPC in AWS. `kube-up.sh` should not making changes to the VPC DHCP option if there's already DHCP options configured.
PR Changes: When `DHCP_OPTION_SET_ID` is given in environment variable, kube-up.sh will skip the `DHCP_OPTION_SET_ID` creation.
Automatic merge from submit-queue
Enable setting up Kubernetes cluster in Ubuntu on Azure
Implement basic cloud provider functionality to deploy Kubernetes on
Azure. SaltStack is used to deploy Kubernetes on top of Ubuntu
virtual machines. OpenVpn provides network connectivity. For
kubelet authentication, we use basic authentication (username and
password). The scripts use the legacy Azure Service Management APIs.
We have set up a nightly test job in our Jenkins server for federated
testing to run the e2e test suite on Azure. With the cloud provider
scripts in this commit, 14 e2e test cases pass in this environment.
We plan to implement additional Azure functionality to support more
test cases.
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/21207)
<!-- Reviewable:end -->
Automatic merge from submit-queue
[fix] allow ALLOW_PRIVILEGED to be passed to kubelet and kube-api
This is something that we need for running docker in docker. Please let me know if you would consider this change. Thanks :)
Automatic merge from submit-queue
Add Calico as policy provider in GCE
Adds Calico as policy provider to GCE, enforcing the extensions/v1beta1 NetworkPolicy API.
Still to do:
- [x] Enable NetworkPolicy API when POLICY_PROVIDER is provided.
- [x] Fix CNI plugin, policy controller versions.
CC @thockin - does this general approach look good?
Automatic merge from submit-queue
Tracked addition of federation, sed support in kube DNS
[]()
The kube DNS app recently gained support for federation (whatever that
is), including a new Salt parameter. This broke the deployAddons.sh script for cluster ubuntu. The DNS app also gained alternate
templates, intended to be friendly to `sed`. Fortunately, those do
not demand a federation parameter.
This PR fixes up the ` cluster/ubuntu/deployAddons.sh` script to track those changes, by switching to the `sed`-friendly templates.
Automatic merge from submit-queue
mount instanceid file from config drive when using openstack cloud provider
fix https://github.com/kubernetes/kubernetes/issues/23191, the instanceid file is read however we do not mount it as a volume, and it would cause the cloud provider contacts the metadata server, in some cases, the metadata server is not able to serve, then the cloud provider would fail to initialize, we should avoid that.
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/23733)
<!-- Reviewable:end -->
Automatic merge from submit-queue
cluster/aws: Add option for kubeconfig context
Added KUBE_CONFIG_CONTEXT environment variable to customize the kubeconfig context created at the end of the aws kube-up script.
Fixes#24877
This PR does barely anything and shouldn't require e2e tests. It's just a minor convenience.
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24910)
<!-- Reviewable:end -->
The kube DNS app recently gained support for federation (whatever that
is), including a new Salt parameter. It also gained alternate
templates, intended to be friendly to `sed`. Fortunately, those do
not demand a federation parameter.
Automatic merge from submit-queue
Revert "Federation e2e supports aws"
Reverting https://github.com/kubernetes/kubernetes/pull/27791 to get our Jenkins tests green again.
cc @kubernetes/sig-cluster-federation
Filip Grzadkowski
