Commit Graph

440 Commits (9f8625fb2c2dd99017508057ac8948b2310a7349)

Author SHA1 Message Date
Joe Finney e660adbad8 Add connect timeout to ssh-to-node. 2016-03-07 13:29:04 -08:00
Justin Santa Barbara 523e1d0d8c Ubernetes-Lite: reuse existing configuration when reusing master
In particular, we need to share the kubelet cert & key, otherwise we
can't connect to the kube-api.

Fix #22593
2016-03-05 17:00:02 -05:00
Justin Santa Barbara 0d8d63c288 AWS: Update jessie image, to fix reboot issue
The previous jessie image had a broken cloud-init, which would use an
Ubuntu-specific 'nobootwait' argument when mounting disks.  We now
override that in the image.

Fix #22549
2016-03-05 16:57:57 -05:00
Abhi Shah 05ba8ca7fa Merge pull request #22170 from metmajer/master
Pass through EXTRA_DOCKER_OPTS.
2016-03-04 10:40:17 -08:00
k8s-merge-robot 2cf3c7809e Merge pull request #21471 from justinsb/kube_up_auto_upgrade
Auto commit by PR queue bot
2016-03-02 21:00:43 -08:00
Justin Santa Barbara b058c6b3e1 AWS: Add sha to download-or-bust
download-or-bust was changed on the GCE side to verify a hash if passed,
so we need to pass a hash in the AWS scripts that reuse this function.
2016-03-02 10:12:39 -05:00
Martin Etmajer f1b2fc39ba Pass through EXTRA_DOCKER_OPTS. 2016-02-29 19:46:58 +01:00
Justin Santa Barbara 8b122b102f kube-up: Only upgrade if AUTO_UPGRADE set, but set it on AWS
Installing upgrades is kind of dangerous, so make it not the default.
But also set AUTO_UPGRADE=true on AWS.
2016-02-27 20:55:33 -05:00
Justin Santa Barbara 4e64c4586e AWS: Set OS defaults for kube 1.2
Default distro is jessie, due to the support situation with Ubuntu
distros.  Default ubuntu distro is wily.

Update the docs to reflect the recommended distros with kube-up, and to
encourage contributions for other distros.
2016-02-25 00:10:33 -05:00
k8s-merge-robot 9b62123b37 Merge pull request #21908 from justinsb/aws_spot_instance_timeout
Auto commit by PR queue bot
2016-02-24 17:25:16 -08:00
Alex Robinson cbcc25802e Merge pull request #21237 from justinsb/aws_jessie_support
AWS Debian Jessie Support
2016-02-24 14:34:48 -08:00
Justin Santa Barbara 5d7ed4d6e7 AWS kube-up: Stop docker before moving /mnt/docker
On Jessie, we now preinstall Docker.  Before moving /mnt/docker, make
sure we stop the docker service.
2016-02-24 15:01:20 -05:00
Justin Santa Barbara 7f3bbe1dfe AWS kube-up: Bump to the latest Jessie image, allow overriding
Jessie image can be selected by name using AWS_IMAGE_NAME. This is a
little easier than specifying the AMI ID.
2016-02-24 14:34:40 -05:00
Justin Santa Barbara 62898319df AWS kube-up: Increase timeout for spot instances
Spot instances take a lot longer to run; wait up to 15 minutes for the
nodes to launch when we're using spot instances.  (Previously we were
waiting 5 minutes).
2016-02-24 14:22:33 -05:00
k8s-merge-robot 58783bc256 Merge pull request #21684 from justinsb/fix_21200
Auto commit by PR queue bot
2016-02-23 07:06:08 -08:00
Justin Santa Barbara aa57beb885 AWS Jessie: Find image by name
This means we don't have to code the AMI ids for each region.
2016-02-23 00:02:54 -05:00
Justin Santa Barbara 91eb370931 AWS Debian: Only install linux-image-extra on Ubuntu
It isn't available (and hopefully not needed) on Debian
2016-02-22 22:54:41 -05:00
Justin Santa Barbara 050863cf93 AWS kube-up: Build kubeconfig asap
Once we've built the master, we can build kubeconfig.  By doing so, if
we time out waiting for the nodes, the system is still configured
correctly.

In particular, spot instances can be slow to launch.

Related to issue #21200
2016-02-22 10:46:56 -05:00
Justin Santa Barbara fb12c48cb7 AWS kube-up: Allow spot instances
I think we should probably leave this undocumented for now, until we
have a better way to launch multiple sets of nodes, but it's great for
cost savings while testing!

Fix #21200
2016-02-22 10:42:49 -05:00
k8s-merge-robot 1ce188e557 Merge pull request #21430 from spxtr/e2e-internal
Auto commit by PR queue bot
2016-02-21 12:08:22 -08:00
k8s-merge-robot d6192dd152 Merge pull request #21342 from spxtr/remove-e2e-test
Auto commit by PR queue bot
2016-02-21 00:39:38 -08:00
k8s-merge-robot a12a2e984a Merge pull request #21233 from justinsb/aws_wily
Auto commit by PR queue bot
2016-02-20 13:28:08 -08:00
k8s-merge-robot fe18055adc Merge pull request #21165 from justinsb/aws_master_pd_in_fstab
Auto commit by PR queue bot
2016-02-20 00:29:46 -08:00
k8s-merge-robot 6c5bf89ed3 Merge pull request #21160 from justinsb/aws_bigger_master
Auto commit by PR queue bot
2016-02-20 00:29:45 -08:00
k8s-merge-robot 5acdb92126 Merge pull request #21177 from laushinka/spelling-fixes
Auto commit by PR queue bot
2016-02-18 10:29:49 -08:00
k8s-merge-robot bcabc096f2 Merge pull request #21130 from madhusudancs/daemonset-enable-default
Auto commit by PR queue bot
2016-02-17 22:54:35 -08:00
k8s-merge-robot 78c294a66c Merge pull request #21164 from justinsb/aws_reboot_with_ephemeral
Auto commit by PR queue bot
2016-02-17 17:12:01 -08:00
Joe Finney 2b756ca56d Move E2E kube-up call into test-setup from e2e-internal. 2016-02-17 16:49:07 -08:00
laushinka 7ef585be22 Spelling fixes inspired by github.com/client9/misspell 2016-02-18 06:58:05 +07:00
Madhusudan.C.S f06a195855 Remove all {KUBE_,}ENABLE_{DEPLOYMENT,DAEMONSET} variables from all our bash scripts. 2016-02-17 15:04:28 -08:00
Madhusudan.C.S ab1e9c200f Graduate DaemonSet to v1beta1 API, enable it by default. 2016-02-17 15:03:12 -08:00
Justin Santa Barbara 4d2c580781 AWS kube-up: Add support for wily
vivid is technically no longer supported now, so we should support
wily.
2016-02-17 10:45:21 -05:00
Justin Santa Barbara a16372ce40 AWS: Change master to m3.medium by default
m3.large for > 150 nodes.

t2.micro often runs out of memory.  The t2 class has very
difficult-to-understand behaviour when it runs out of CPU.  The
m3.medium is reasonably affordable, and avoids these problems.

Fix #21151

Issue #18975
2016-02-17 10:24:14 -05:00
Justin Santa Barbara c68c019150 AWS: Add comments to explain the symlink logic 2016-02-17 09:57:37 -05:00
Justin Santa Barbara affffdc9ab AWS kube-up: Put the master-pd into /etc/fstab, to mount it early
Otherwise we risk services coming up on the master before the backing
volume is ready.

If we then see the master-pd is already mounted, don't try to remount
it.

Issue #21155
2016-02-17 09:47:01 -05:00
k8s-merge-robot 22ad56dded Merge pull request #21161 from justinsb/aws_bootstrap_on_every_boot
Auto commit by PR queue bot
2016-02-17 06:45:24 -08:00
Joe Finney 6f7182e500 Remove hack/e2e-test.sh in favor of hack/e2e.go. 2016-02-16 14:54:50 -08:00
k8s-merge-robot 784f555fd9 Merge pull request #18821 from rastut/add_aws_detect_root_device
Auto commit by PR queue bot
2016-02-13 10:09:20 -08:00
k8s-merge-robot f2e845dd6d Merge pull request #21144 from justinsb/aws_fix_project_not_defined
Auto commit by PR queue bot
2016-02-13 08:33:53 -08:00
Justin Santa Barbara 69300c51b3 AWS kube-up: Reboot support with ephemeral storage
If the ephemeral volume is present and mounted, don't try to reinitialize
them.

Don't block the boot if the ephemeral volume is corrupt / missing -
this enables us to cope with a stop/start & presumably also corruption.
In this case, we'll reformat the ephemeral storage.

Fix #21157
2016-02-12 20:31:56 -05:00
Justin Santa Barbara 19cf622e88 AWS kube-up: make the bootstrap script run every boot
This is so we have the same behaviour as on GCE.

This also lets us change the bootstrap script or the config, which is
nice.  Instance data is immutable on AWS once it is booted.

Fix #21150
2016-02-12 20:30:47 -05:00
Janet Kuo 2874f30c05 Enable Deployments by default 2016-02-12 12:29:14 -08:00
Justin Santa Barbara 1852fab029 AWS: Fix kube-up generation of kubeconfig
We were assuming the PROJECT env var was set, which the e2e tests do.
But PROJECT is normally not set on AWS (it is set on GCE); this broke as
part of the harmonization.

Revert to the pre-existing behaviour here, where we use "aws_" as the
prefix.

Fix #21141
2016-02-12 09:53:22 -05:00
Chris Kühl 7fcc40bb27 docs: replace Rocket with rkt 2016-02-09 13:52:25 +01:00
k8s-merge-robot 68a165cda9 Merge pull request #20681 from justinsb/aws_sync_kubeup_gce_step3
Auto commit by PR queue bot
2016-02-09 02:08:48 -08:00
Michael Nikitochkin 55e2906f79 AWS: Allow to configure master ip and services mask
Fix script for case when neeed to setup cluster
in an existen VPC and subnet with ip mask example: 10.0.0.0/8.

Fixed bug to detect ip of master if provided MASTER_RESERVED_IP.

For some reason detecting master ip was moved to volumes and only when MASTER_RESERVED_IP=auto.
If specify IPv4 for MASTER_RESERVED_IP like `52.1.1.1`, than we could
not detect ip even during last steps of setuping cluster.
step the KUBE_MASTER_IP is reseted because there are no tag for the
volume.
2016-02-07 21:59:00 +01:00
Justin Santa Barbara 63743e9c1c AWS kube-up: Use the cert we generated in kubeconfig 2016-02-06 22:52:01 -05:00
Justin Santa Barbara 270fae4d66 kube-up: Support multiple additional IPs for SSL cert
And pass the public IP when setting up AWS
2016-02-06 22:52:01 -05:00
Justin Santa Barbara fd5c44c72e kube-up: Finish refactor of AWS to reuse GCE script
We override key functions in the GCE script with AWS equivalents.
2016-02-06 22:52:01 -05:00
k8s-merge-robot 9d981b696c Merge pull request #20642 from justinsb/aws_sync_kubeup_gce_step2
Auto commit by PR queue bot
2016-02-06 11:56:47 -08:00
k8s-merge-robot 5b046cccdb Merge pull request #20448 from justinsb/aws_fix_e2e_detect_master
Auto commit by PR queue bot
2016-02-04 16:24:20 -08:00
Justin Santa Barbara 732e76fb4e kube-up: Refactor SSL cert helpers into common
Next step in the the GCE/AWS harmonization
2016-02-04 09:10:41 -05:00
Justin Santa Barbara 0f6b3611d8 AWS: Fix e2e usage of find-tagged-master-ip
In the e2e tests detect-master is called directly.  In turn, it calls
find-tagged-master-ip, which assumed that find-master-pd has already already
been called.  But this wasn't true in the e2e case.

We add a call to find-master-pd; it is idempotent.
2016-02-01 22:49:31 +00:00
k8s-merge-robot 30375b21d6 Merge pull request #19776 from justinsb/aws_sync_kubeup_gce
Auto commit by PR queue bot
2016-02-01 14:44:18 -08:00
k8s-merge-robot 0aca70016c Merge pull request #20066 from justinsb/aws_move_build_runtime_config
Auto commit by PR queue bot
2016-02-01 09:48:27 -08:00
Alex Robinson f684e80b1f Merge pull request #20067 from justinsb/aws_test_defaults
AWS: Change config-test defaults to match GCE
2016-01-30 17:42:28 -08:00
Justin Santa Barbara c556e8f0f5 kube-up: Refactor kube-env into common; use in AWS
Use kube-env on the AWS master also
2016-01-28 10:31:51 -08:00
Justin Santa Barbara 4d88a4ed2c kube-up: move sha1sum-file into common
Also hash the tar files in AWS kube-up
2016-01-28 10:19:38 -08:00
Justin Santa Barbara a70412099a AWS: add retries around apt-get install
Related to issue #17309
2016-01-28 10:19:38 -08:00
Justin Santa Barbara ae75a7d259 kube-up: move build-runtime-config from AWS & GCE into common 2016-01-28 10:19:38 -08:00
Justin Santa Barbara 0df6f5d8c5 kube-up: move yaml-quote from GCE & AWS to common 2016-01-28 10:19:38 -08:00
k8s-merge-robot 6c63acdd3d Merge pull request #19398 from justinsb/aws_elasticip_for_master
Auto commit by PR queue bot
2016-01-26 04:34:28 -08:00
k8s-merge-robot 8c821700f3 Merge pull request #19389 from justinsb/internal_cidr
Auto commit by PR queue bot
2016-01-26 03:58:48 -08:00
Justin Santa Barbara a4d6801655 AWS: Change config-test defaults to match GCE
KUBE_ENABLE_DAEMONSETS & KUBE_ENABLE_DEPLOYMENTS default to true in
config-test in GCE, so we should do the same in AWS for e2e sanity.
2016-01-24 10:43:43 -05:00
Justin Santa Barbara 8a7afc459c AWS: Call build-runtime-config where it is used
build-runtime-config was being called in verify-prereqs, which didn't
match how GCE called it, and didn't seem to actually work.

Instead call it just before the master configuration is built.  Also
call it just before the node configuration is built, even though the
nodes don't _currently_ require the runtime_config.
2016-01-24 10:43:15 -05:00
Justin Santa Barbara d8460d5920 AWS: Pass non-masquerade-cidr through kube-up
Set the environment variable NON_MASQUERADE_CIDR.

Docs in cluster/aws/options.md
2016-01-22 21:01:44 -05:00
Rudi Chiarito bc0dd97a70 ECR credential provider 2016-01-22 15:03:25 -05:00
Justin Santa Barbara 274e589aa6 AWS: Use an elastic IP for the master by default
If we don't use an elastic IP, the IP address will be lost if we lose
the master for any reason, and a replacement master will not have the
same IP.  But the master IP is set both in client kubeconfig files and
the master SSL certificate.  Hence the default should be to allocate an
elastic IP for the master.

One complication: AWS doesn't allow tags on elastic IPs, so it is hard
to track the elastic IP so we can delete it as part of kube-down.
Instead, we take the master EBS volume with the elastic IP.  This is a
little odd, but works because the master volume & the master elastic IP
really need to be assigned to the same machine, so might be thought of
as a pair.

Also, we now delete the master EBS volume as part of kube-down, as
people expect kube-down to clean-up everything it creates.
2016-01-21 22:01:55 -05:00
Alex Mohr f788e1e11a Merge pull request #19446 from justinsb/aws_reboot_master_on_failure
AWS: Create a cloudwatch alarm to reboot the master on failure
2016-01-21 15:05:14 -08:00
Alex Mohr d8be60ee57 Merge pull request #19777 from justinsb/echo_sleep_is_silly
kube-up scripts: don't 'echo sleep'
2016-01-21 10:44:38 -08:00
Alex Mohr eaa61a72b0 Merge pull request #17919 from justinsb/multizone_gce
Ubernetes Lite support for GCE
2016-01-21 10:22:34 -08:00
Justin Santa Barbara 2958ea253a GCE: Allow for reuse of master
This is for internal use at the moment, for testing Ubernetes Lite, but
arguably makes the code a little cleaner.

Also rename KUBE_SHARE_MASTER -> KUBE_USE_EXISTING_MASTER
2016-01-20 15:37:00 -05:00
Justin Santa Barbara dcd7e813f7 AWS: Update AMIs for Ubuntu Vivid
Update to the latest AMIs for Vivid, as sourced from:
http://cloud-images.ubuntu.com/locator/ec2/
2016-01-18 16:47:06 -05:00
Justin Santa Barbara d2e65a89d0 kube-up scripts: don't 'echo sleep'
'echo sleep 5' just prints 'sleep 5'; replace with 'sleep 5'.
2016-01-17 21:29:01 -05:00
Josh Ellithorpe 76e8a8b08d Fix issues with Python3 and bring up a dev cluster 2016-01-12 11:08:42 -08:00
Justin Santa Barbara d91ad0bb6f AWS: Create a cloudwatch alarm to reboot the master on failure
This is an easier alternative to keep a master running than trying to
dynamically find & attach master volumes.

To actually work, it requires that users create the EC2ActionsAccess IAM
role as required by CloudWatch, see e.g.
http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/UsingIAM.html
2016-01-09 14:53:29 -05:00
Justin Santa Barbara 57265a4c74 AWS: Pass NUM_NODES in instance-data
The bootstrap scripts already assume it is set.
2016-01-05 23:10:43 -05:00
Justin Santa Barbara f9a6ac077e Ubernetes Lite: Volumes can dictate zone scheduling
For AWS EBS, a volume can only be attached to a node in the same AZ.
The scheduler must therefore detect if a volume is being attached to a
pod, and ensure that the pod is scheduled on a node in the same AZ as
the volume.

So that the scheduler need not query the cloud provider every time, and
to support decoupled operation (e.g. bare metal) we tag the volume with
our placement labels.  This is done automatically by means of an
admission controller on AWS when a PersistentVolume is created backed by
an EBS volume.

Support for tagging GCE PVs will follow.

Pods that specify a volume directly (i.e. without using a
PersistentVolumeClaim) will not currently be scheduled correctly (i.e.
they will be scheduled without zone-awareness).
2015-12-31 12:27:01 -05:00
Carles Lopez c8beaac51d Add root device detection to boot correctly 2015-12-17 13:33:00 +01:00
Michael Nikitochkin 2272de1f67 AWS: Fixed compacting script of cloud init for MacOS
In MacOS there is error during setup a new cluster:

```
+ sed -i -e 's/^[[:blank:]]*#.*$//' -e '/^[[:blank:]]*$/d' /sometmpfile
sed: -e: No such file or directory
```

Because sed version of MacOS does not support modern features.
2015-12-16 09:20:57 +01:00
Thomas Ploch 189f2436e3 [AWS] Wrong assignment of `KUBE_MASTER_IP` with Elastic IP
Currently when using a custom elastic IP, the ENV var `KUBE_MASTER_IP` gets
the output of `$(assign-elastic-ip $ip $master_id)` assigned.

This is wrong since the command returns a string:
`Attaching IP 99.999.999.999 to instance i-9999999`

This patch fixes the assignment by calling `get_instance_public_ip` again.
2015-12-12 13:53:01 +01:00
k8s-merge-robot 0f399211f7 Merge pull request #18258 from antoineco/ec2
Auto commit by PR queue bot
2015-12-10 11:14:49 -08:00
Justin Santa Barbara 428e4156e2 AWS: Strip more comments from instance user-data
To stay under the 16KB limit
2015-12-07 22:27:33 -05:00
Antoine Cotten 926148c5ee Add warning about AWS EC2 't2' instance type 2015-12-06 19:56:13 +01:00
k8s-merge-robot b69018eab2 Merge pull request #17844 from eosrei/kube-util-minion-node
Auto commit by PR queue bot
2015-12-02 03:06:27 -08:00
k8s-merge-robot bb86936765 Merge pull request #17803 from davidsiefert/master
Auto commit by PR queue bot
2015-12-01 02:15:11 -08:00
k8s-merge-robot 7bcd7759fa Merge pull request #17814 from themez/master
Auto commit by PR queue bot
2015-12-01 00:22:51 -08:00
jiangyaoguo a739fc44c4 keep kubeproxy hostname consistent with kubelet 2015-11-28 10:47:29 +08:00
Brad Erickson 0bd16e0437 Minion->Node rename: kube-util hosting provider helper functions 2015-11-26 09:31:46 -08:00
ThemeZ 3342e4e1d3 Correct s3 url base for region "cn-north-1" 2015-11-26 11:37:28 +08:00
David Siefert 94d32588f4 Extracting more functions for reuse out of kube-up 2015-11-25 17:29:30 -06:00
Brad Erickson 53172a5356 Minion->Node rename: NUM_NODES 2015-11-25 00:45:10 -08:00
Brad Erickson e67be19a5b Minion->Node rename: OLD_NODE_TAG, NODE_TAG 2015-11-25 00:45:09 -08:00
Brad Erickson bd06c19aa8 Minion->Node rename: NODE_SIZE 2015-11-25 00:45:09 -08:00
Brad Erickson 8431993a44 Minion->Node rename: NODE_SG_ID, NODE_SG_NAME 2015-11-25 00:45:09 -08:00
Brad Erickson 1846cfc129 Minion->Node rename: NODE_ROOT_DISK_SIZE, NODE_ROOT_DISK_TYPE, etc
NODE_SCOPES
2015-11-25 00:45:09 -08:00
Brad Erickson fc04b55088 Minion->Node rename: NODE_NAMES, NODE_NAME, NODE_PORT 2015-11-25 00:45:09 -08:00
Brad Erickson 83ed2fa22e Minion->Node rename: NODE_DISK_SIZE, NODE_DISK_TYPE, NODE_HOSTNAME, etc
NODE_IDS
NODE_ID
NODE_IMAGE_PROJECT
NODE_IMAGE
2015-11-25 00:45:09 -08:00
Brad Erickson a36d3390bf Minion->Node rename: KUBERNETES_NODE_MEMORY, VAGRANT_NODE_NAMES, etc
ENABLE_NODE_PUBLIC_IP
NODE_ADDRESS
NODE_BLOCK_DEVICE_MAPPINGS
NODE_CONTAINER_ADDRS
NODE_CONTAINER_NETMASKS
NODE_CONTAINER_SUBNET_BASE
NODE_CONTAINER_SUBNETS
NODE_CPU
2015-11-25 00:43:52 -08:00
Brad Erickson ae314ad246 Minion->Node rename: KUBE_NODE_IP_ADDRESSES, KUBE_NODE_IP_ADDRESS 2015-11-25 00:43:52 -08:00