Commit Graph

202 Commits (12a677876634995b08aa728b193c232a255e729d)

Author SHA1 Message Date
Justin Santa Barbara d898834037 AWS: Log the distro when we don't recognize it
Similar to #15070, we should log the distro if we're going to tell the
user we can't match it (so the user can see if they have typoed it, and
so it will hopefully be included to us in error reports)
2015-10-05 09:25:59 -04:00
Justin Santa Barbara f763c59743 AWS: Increase the timeout for S3 bucket creation
The current timeout of 5 seconds is needlessly short, given that we
fail kube-up if the (eventually consistent?) bucket creation takes
longer.

Raise it to 120 seconds.

Possibly related to issue #14278
2015-10-05 09:17:12 -04:00
Ananth Suryanarayana d50d7763da Add opencontrail networking provisioning support in kubernetes salt based provisioning
OpenContrail is an open-source based networking software which provides virtualization support for the cloud.

This change-set adds ability to install and provision opencontrail software for networking in kubernetes based cloud environment.

There are basically 3 components

o kube-network-manager -- plugin between contrail components and kubernets components
o provision_master.sh -- OpenContrail software installer and provisioner in master node
o provision_minion.sh -- OpenContrail software installer and provisioner in minion node(s)

These are driven via salt configuration files

One can provision opencontrail by just setting "export NETWORK_PROVIDER=opencontrail"
Optionally, OPENCONTRAIL_TAG, and OPENCONTRAIL_KUBERNETES_TAG can be used to
specify opencontrail and contrail-kubernetes software versions to install and provision.

Public-IP Subnet provided by contrail can be configured via OPENCONTRAIL_PUBLIC_SUBNET
environment variable

At this moment, plan is to add support for aws, gce and vagrant based platforms

For more information on contrail-kubernetes, please visit https://github.com/juniper/contrail-kubernetes For more information on opencontrail, please visit http://www.opencontrail.org
2015-10-03 08:03:02 -07:00
Brian Grant 1b8727ada1 Merge pull request #13072 from mikaelh/master
Elastic IP need to be assigned after the instance is in running state
2015-09-24 17:39:26 -07:00
Mikael Hammarin c82e645f9d Elastic IP need to be assigned after the instance is in running state. Currently always fails with "The pending instance x is not in a valid state for this operation" and defaults to public ip.
Fix: Moved assign-elastic-ip to run after wait-for-instance-running
2015-08-23 13:39:01 +02:00
Jeff Lowdermilk 438216844f Stop reusing basic auth on cluster create 2015-08-22 11:45:46 -07:00
erulabs 5ee816e905 Add config option for AWS EBS root volume size 2015-08-18 11:03:23 -07:00
Veres Lajos 9f77e49109 typofix - https://github.com/vlajos/misspell_fixer 2015-08-08 22:31:48 +01:00
Justin Santa Barbara 230de787be AWS: Query for location of S3 bucket, use that
Previously we would rely on the s3 bucket's region being configured
correctly, at least for the existence check.  By querying for the bucket
region and then going direct to the correct region, we avoid errors and
we avoid potential eventual consistency problems.

May be related to issue: #12109
2015-08-01 14:09:23 -04:00
Piotr Szczesniak f48543aba5 Made enabling Kube UI configurable 2015-07-27 08:23:04 +02:00
Justin Santa Barbara 355e922b5d AWS: Restore (erroneously) removed get_instance_private_ip
Fix #11201
2015-07-14 13:45:12 -04:00
Victor Marmol ee8d2857fc Merge pull request #9381 from justinsb/aws_support_jessie
AWS: Add support for Vivid Vervet & Debian Jessie
2015-07-09 15:20:37 -07:00
Justin Santa Barbara c78b71d399 AWS: Treat ubuntu as an alias for 'latest ubuntu', i.e. 'vivid' 2015-07-08 17:00:44 -04:00
Justin Santa Barbara bdb2196abe AWS: Initial support for Ubuntu Vivid Vervet 2015-07-08 17:00:43 -04:00
Victor Marmol 7dd0dc1786 Merge pull request #10752 from justinsb/aws_query_by_subnet_id
AWS: Query by subnet-id, if we have it
2015-07-08 13:57:33 -07:00
Victor Marmol 3dd3d6e889 Merge pull request #10751 from justinsb/aws_existing_subnet
AWS: Allow callers to specify VPC_ID and SUBNET_ID
2015-07-08 13:57:13 -07:00
Justin Santa Barbara 8e69495f7f AWS: Add jessie support to cluster script 2015-07-08 16:47:55 -04:00
Justin Santa Barbara cc68f87a7b AWS: Query by subnet-id, if we have it
If we are matching an existing subnet-id, query using that subnet-id!
2015-07-06 10:14:15 -04:00
Justin Santa Barbara 2429c7edf8 AWS: Allow callers to specify VPC_ID and SUBNET_ID
This is for people that want to run in a shared VPC/Subnet; while this should
work, we don't actively want to support it yet.  So we don't block it,
but we don't document/encourage it either!
2015-07-06 10:10:34 -04:00
Justin Santa Barbara c676c11189 AWS: Configure SSL certificate alternate-names
GCE does this in its per-provider scripts; this does the same for AWS and lets
other providers do the same; I believe kube2sky requires 10.0.0.1 as a SAN.
2015-07-03 01:18:07 -04:00
Justin Santa Barbara 2a8d46864b AWS: Use the SSH key fingerprint, not the AWS fingerprint
This is unfortunate, because it means we have two fingerprints,
although arguably the OpenSSH key fingerprint is much more common.
However, the OSX Mavericks version of ssh-keygen can't compute
the AWS fingerprint correctly (e.g. https://www.netmeister.org/blog/ssh2pkcs8.html)

So we work on OSX Mavericks, we use the more common OpenSSH fingerprint.
2015-07-01 04:10:48 -07:00
Robert Bailey e6f14a21ce Merge pull request #10403 from justinsb/ebs-for-master-data-2
AWS: Use persistent disk on master
2015-06-26 21:52:10 -07:00
Justin Santa Barbara a3cb84fa56 AWS: Wait for disk to be attached in setup-master-pd.sh 2015-06-26 11:52:45 -04:00
Justin Santa Barbara 6e09cd3ffd AWS: Create/reuse master pd directly 2015-06-26 11:52:45 -04:00
Justin Santa Barbara 7059f379a4 AWS: Mount persistent disk as sdb on master 2015-06-26 11:52:44 -04:00
Adam Sunderland 66e0c5432f Use EBS for Master Data
Signed-off-by: Adam Sunderland <iterion@gmail.com>
2015-06-26 11:52:44 -04:00
Eric Paris 58df58f3d7 Remove unused enable_node_monitoring option
Back in 1a7f7245e7 we dropped the one
place this was used, but left all of the variable and definitions and
garbage around cluster/
2015-06-25 20:57:56 -04:00
Jeff Lowdermilk e2d496e45d Merge pull request #9737 from iterion/use-openssl-rsa
AWS: Use RSA to Generate Fingerprint
2015-06-23 10:14:27 -07:00
Justin Santa Barbara 2a5ed2f086 AWS: Use auto-scaling group to run minions
This uses the dynamic CIDR work, and we set source-dest-check to false
when we configure the route (which kind-of makes sense)
2015-06-19 10:22:15 -04:00
Justin Santa Barbara a4e15cdf3e AWS: Configure minion routes dynamically
We need to implement the Routes interface, and then enable the functionality in the cluster scripts.
2015-06-18 14:59:37 -07:00
Fabio Yeon c4d040c720 Merge pull request #9745 from justinsb/aws_manage_route_table
AWS: Create our own route table & tag it for management
2015-06-15 15:18:39 -07:00
Fabio Yeon f033349bea Merge pull request #9746 from justinsb/aws_faster_tagging
AWS: delay less when creating tags
2015-06-15 13:34:44 -07:00
Justin Santa Barbara 6839ae9a44 AWS: delay less when creating tags
This is important because tag creation isn't atomic, and we want to minimize
the window in which we can leave 'dangling' resources around.
2015-06-12 18:32:55 -04:00
Justin Santa Barbara 1f04284508 AWS: Create our own route table & tag it for management 2015-06-12 18:32:10 -04:00
Adam Sunderland 7760d079ae Update SSH User For Master SSH Check 2015-06-12 15:38:48 -05:00
Adam Sunderland d82bfffe9d Use RSA to Generate Fingerprint 2015-06-12 15:29:37 -05:00
Abhi Shah 82aa8f9984 Merge pull request #9371 from justinsb/aws_support_wheezy
AWS: Support wheezy, for parity with GCE
2015-06-11 15:08:32 -07:00
Abhi Shah 064e7146e1 Merge pull request #9481 from justinsb/aws_fix_push
AWS: Include (idempotent) ensure-temp-dir in upload-server-tars
2015-06-11 11:24:18 -07:00
Justin Santa Barbara 8aae864784 AWS: Support wheezy, for parity with GCE 2015-06-09 23:45:41 -04:00
krousey e19e4bcd12 Merge pull request #9365 from justinsb/fix_9246
AWS: Only log "Starting cluster using os distro..." in kube-up
2015-06-09 09:51:22 -07:00
Justin Santa Barbara d92863523f AWS: Include (idempotent) ensure-temp-dir in upload-server-tars
This way we won't forget it.  Fixes kube-push, where I forgot it.
2015-06-09 11:10:15 -04:00
krousey e14d9038fe Merge pull request #9367 from justinsb/aws_ssh_check
AWS: Check for SSH connectivity & better logging on failure
2015-06-08 16:48:04 -07:00
Justin Santa Barbara 1c229e5284 Removed extra blank line in aws util.sh 2015-06-08 16:59:03 -04:00
Justin Santa Barbara 2619b6198a AWS: Mount ephemeral devices, even if not specified in the AMI
We mount up to 4; this covers almost all instance types.
2015-06-08 16:59:03 -04:00
Justin Santa Barbara 72496e7368 AWS: Check for SSH connectivity & better logging on failure 2015-06-06 14:27:41 -04:00
Justin Santa Barbara a9e1e1033b AWS: Only log "Starting cluster using os distro..." in kube-up
Fixes #9246
2015-06-06 12:48:49 -04:00
Justin Santa Barbara 112a013567 AWS: Support different docker storage mechanism by setting DOCKER_STORAGE
For parity with GCE, we really want to support aufs.

But we previously supported btrfs, so we want to expose that.

Most of the work here is required for aufs, and we let advanced users choose
devicemapper/btrfs if they have a setup that works for those configurations.
2015-06-06 12:13:51 -04:00
Justin Santa Barbara 48e8a8b0ec AWS: Set up security groups, to mirror GCE firewalling
Some slightly fussy code to enable load-balancers to talk to
instances, but otherwise relatively simple.
2015-06-05 16:10:08 -04:00
Justin Santa Barbara 33a3d884f2 AWS: Filter by Cluster tag, rationalize EC2 abstraction
Whenever we do a list we now filter on tags so we only see resources relating
to our cluster.

Also, rationalize all the DescribeX calls:
 * They all take a request object (so that we can pass filters)
 * They do paging if that is required (and return the underlying resources)
 * They wrap any error with a "error while listing X: %v" message
2015-06-05 16:09:01 -04:00
Quinton Hoole 710df2b619 Merge pull request #9294 from justinsb/aws_ssh_key_fingerprint
AWS: Support multiple SSH keys (embed the hash in the name)
2015-06-05 09:37:31 -07:00
Justin Santa Barbara d8dc416b5b AWS: Support multiple SSH keys (embed the hash in the name)
This should eliminate a nasty problem where the script doesn't cope well if
your keys don't match.
2015-06-04 21:40:57 -04:00
Justin Santa Barbara c92c63b3a9 AWS: Use s3 sync to optimize upload to s3 when nothing changed 2015-06-04 10:14:45 -04:00
Eric Tune 3c067b766a Merge pull request #8996 from manolitto/aws_cluster_monitoring_fix
aws: fix cluster monitoring (new option "influxdb" instead of "true")
2015-06-01 13:43:43 -07:00
Eric Tune 5b3e01d2fd Merge pull request #8653 from matschaffer/s3-creation-wait
Check that s3 bucket has been created
2015-06-01 10:29:03 -07:00
Manfred Geiler 8c8f8feb62 aws: fix cluster monitoring ("none" instead of "false") 2015-06-01 09:12:41 +02:00
Tim Hockin ac3cc3c518 Rename PORTAL_NET all over 2015-05-28 16:10:44 -07:00
Mat Schaffer e7ae425385 Colorize errors for consistency with other checks 2015-05-23 16:12:24 +09:00
Mat Schaffer 26736e494c Check that s3 bucket has been created
Fixes #8395
2015-05-22 14:12:36 +09:00
Dawn Chen 04c4d25065 Merge pull request #7905 from bakins/aws-coreos
AWS: use CoreOS for nodes
2015-05-21 09:05:56 -07:00
Daniel Smith ce4b54ec70 Merge pull request #8209 from krousey/v1beta1_cluster
Removing some v1beta1 uses in cluster/
2015-05-15 14:56:41 -07:00
Kris Rousey 98c457c397 Updating /cluster to use v1beta 3 specs, and change a lot of polling to
healthz instead of api endpoints.
2015-05-15 14:17:55 -07:00
Robert Bailey 9d6c032929 Merge pull request #7888 from madis/associate_aws_elastic_ip_with_master
Associate master instance with AWS Elastic IP
2015-05-14 13:18:19 -07:00
Madis Nõmme 15643a2c72 Add 'auto' option for MASTER_RESERVED_IP. No ElasticIP allocation by default.
Default behaviour when setting up a cluster is using the Amazon-assigned public ip.
It will change between reboots. If MASTER_RESERVED_IP is set to 'auto', new Elastic
IP will be allocated & assigned to master. If MASTER_RESERVED_IP is set to an existing
Elastic IP, it will be used. When something fails, original Amazon-given IP will be used.
2015-05-14 08:33:07 +03:00
Brian Akins fac4350fa6 Initial addition of CoreOS as minion for AWS cluster 2015-05-13 16:39:22 -04:00
Madis Nõmme eb220f05a6 Properly get return value (considering errexit). Quote variables. 2015-05-13 10:45:51 +03:00
Madis Nõmme d4d02a9028 Optionally associate master instance with AWS Elastic IP
When MASTER_RESERVED_IP is set to elastic IP from AWS, then aws/util.sh will
associate it with master instance and assign it to KUBE_MASTER_IP. If no MASTER_RESERVED_IP
is set, new elastic ip will be requested from amazon. This allows cluster certificates to
be generated for an IP that doesn't change between stopping & starting cluster instances.

The requested elastic ip is not released when kube-down.sh is run. I think it is good
because user could have created DNS records and it would be bad if the IP was removed.
He can reuse it next time through MASTER_RESERVED_IP when setting up cluster again.
2015-05-13 10:45:51 +03:00
Justin Santa Barbara ffb0e7f9b8 Install specific salt version on AWS, based on GCE
The latest salt version breaks the container_bridge.py _state function

We can lock to the same version as GCE.  This is not a full fix,
because we can't update to the latest salt without breaking GCE,
but this at least unbreaks and sync AWS with GCE.

This isn't a straight copy from GCE, because we still use
the salt master on AWS (for now)

Fixes #8114
2015-05-12 16:33:56 -04:00
Manfred Geiler c5c62f7d57 fixed second missing $ and added curly brackets 2015-05-08 17:18:52 +02:00
Manfred Geiler 1119340260 fixed missing $ 2015-05-08 16:58:49 +02:00
Manfred Geiler 205ed2bf6e AWS: make it possible to disable minion public ip association 2015-05-08 00:09:47 +02:00
Justin Santa Barbara 57f7b658bb AWS: Change apiserver to listen on 443 directly, not through nginx
Mirrors changes in GCE.  I think the same changes will be needed for vagrant.
2015-05-02 16:59:04 -04:00
Eric Paris 6b3a6e6b98 Make copyright ownership statement generic
Instead of saying "Google Inc." (which is not always correct) say "The
Kubernetes Authors", which is generic.
2015-05-01 17:49:56 -04:00
Brendan Burns 637cd57a25 Merge pull request #6606 from gust1n/aws-existing-vpc
AWS: Improving getting existing VPC and subnet
2015-04-27 11:11:25 -07:00
Joakim Gustin 645d6d1f26 Use existing subnet when launching AWS cluster in existing VPC 2015-04-23 11:01:17 +02:00
Justin Santa Barbara 9253ae6dca AWS: Fix some bash style problems: variable quoting & locals 2015-04-22 18:30:09 -07:00
Justin Santa Barbara 45b658ea7c AWS: don't try to create undefined config_dir 2015-04-20 19:14:16 -07:00
Jeff Lowdermilk 3787fc5eca set KUBECONFIG in common.sh, default to new location
and preserve value in ginkgo e2e test driver
2015-04-20 11:07:35 -07:00
Jeff Lowdermilk 4f6dc99075 Generate kubeconfig for all providers in cluster/ that use auth 2015-04-20 11:07:35 -07:00
Justin Santa Barbara 034412aff1 Support multiple k8s clusters 2015-04-10 13:25:43 -07:00
deads2k 609208b8b5 update scripts with correct templates 2015-04-07 08:07:24 -04:00
Justin Santa Barbara 2fb573c482 Create instances in the specified AZ 2015-04-03 08:04:53 -07:00
Zach Loafman fc8ba8d77b Merge pull request #6011 from justinsb/aws_specify_zone
Fix AWS region vs zone
2015-03-31 12:26:05 -07:00
Justin Santa Barbara edbebd653c Fix S3 location handling for US classic 2015-03-31 05:42:10 -07:00
Justin Santa Barbara 194143e61b Use the correct region for the S3 bucket
We may not have created the bucket!
2015-03-27 12:58:47 -07:00
Justin Santa Barbara 7179f5f004 Change AWS_ZONE to ZONE, for compatibility with e2e tests 2015-03-27 08:53:45 -07:00
Justin Santa Barbara 583892da2d Fix AWS region vs zone
We were specifying a region, but naming it as a zone in util.sh

The zone matters just as much as the region, e.g. for EBS volumes.

We also change the config to require a Zone, not a Region.
But we fallback to get the information from the metadata service.
2015-03-26 15:11:05 -07:00
Zach Loafman e5914915b8 Merge pull request #6003 from justinsb/aws_wait_for_instance
Wait for master to be started on AWS
2015-03-26 14:06:25 -07:00
Justin Santa Barbara f256d3b77a Wait for ELBs to be deleted 2015-03-26 13:50:35 -07:00
Justin Santa Barbara 53bdbec64e Launch minions in parallel 2015-03-26 10:58:18 -07:00
Justin Santa Barbara 9556318c58 Wait for master instance running before adding route 2015-03-26 10:23:25 -07:00
Saad Ali f3ebe30605 Merge pull request #5940 from justinsb/fix_bash_style
Fix bash style issues in AWS script
2015-03-25 14:40:42 -07:00
Justin Santa Barbara 06704bb94f Security group should be scoped to our vpc
Note that security group names only need to be unique with a VPC as well,
which is handy here.
2015-03-25 12:53:12 -07:00
Justin Santa Barbara ac12665f48 Quote AWS_SSH_KEY 2015-03-25 12:41:02 -07:00
Justin Santa Barbara 80ba56a371 Fix variable dollar-char errors 2015-03-25 12:36:52 -07:00
Justin Santa Barbara 505488269c Use double-square brackets in if tests 2015-03-25 12:36:20 -07:00
Alex Robinson c22d4c6b3f Merge pull request #5916 from justinsb/aws_cleanup_loadbalancers
Remove load balancers as part of kube-down.sh
2015-03-25 10:32:14 -07:00
Justin Santa Barbara 68935e0798 Remove load balancers as part of kube-down.sh 2015-03-25 05:36:47 -07:00
Brendan Burns cf72cf098a Disable validation on AWS where it doesn't work.
Also add an advanced route for the master which I forgot previously.
2015-03-24 21:08:04 -07:00
Brendan Burns 8bac135f85 Configure docker and kubelet on AWS master. 2015-03-24 17:19:06 -07:00
Robert Bailey b152d8e884 Merge pull request #5822 from justinsb/aws_idempotent_kube_down
More idempotent kube-down for AWS, and delete multiple security groups
2015-03-24 07:58:03 -07:00