Commit Graph

462 Commits (e9afbd5cdf4501e3ba4377d4032afc28b03eac93)

Author SHA1 Message Date
Justin Santa Barbara 2429c7edf8 AWS: Allow callers to specify VPC_ID and SUBNET_ID
This is for people that want to run in a shared VPC/Subnet; while this should
work, we don't actively want to support it yet.  So we don't block it,
but we don't document/encourage it either!
2015-07-06 10:10:34 -04:00
Justin Santa Barbara c676c11189 AWS: Configure SSL certificate alternate-names
GCE does this in its per-provider scripts; this does the same for AWS and lets
other providers do the same; I believe kube2sky requires 10.0.0.1 as a SAN.
2015-07-03 01:18:07 -04:00
Justin Santa Barbara 2a8d46864b AWS: Use the SSH key fingerprint, not the AWS fingerprint
This is unfortunate, because it means we have two fingerprints,
although arguably the OpenSSH key fingerprint is much more common.
However, the OSX Mavericks version of ssh-keygen can't compute
the AWS fingerprint correctly (e.g. https://www.netmeister.org/blog/ssh2pkcs8.html)

So we work on OSX Mavericks, we use the more common OpenSSH fingerprint.
2015-07-01 04:10:48 -07:00
Shaun Stanworth 74bb1b349a Respect DNS_DOMAIN setting in AWS+CoreOS
Without this change, CoreOS minion nodes are unable to perform short-name DNS lookups by default, as the default value for `DNS_DOMAIN` is `cluster.local`, which doesn't match the hard-coded value here.
2015-06-30 17:48:15 +01:00
Robert Bailey 1c0b765df6 Merge pull request #10359 from justinsb/aws_fix_options
Fix AWS options doc
2015-06-26 23:16:51 -07:00
Robert Bailey e6f14a21ce Merge pull request #10403 from justinsb/ebs-for-master-data-2
AWS: Use persistent disk on master
2015-06-26 21:52:10 -07:00
Justin Santa Barbara a3cb84fa56 AWS: Wait for disk to be attached in setup-master-pd.sh 2015-06-26 11:52:45 -04:00
Justin Santa Barbara 6e09cd3ffd AWS: Create/reuse master pd directly 2015-06-26 11:52:45 -04:00
Justin Santa Barbara 0a4f03f235 AWS: Mount ephemeral on /mnt/ephemeral
If we're going to have a persistent disk on /mnt/master-pd, it seems risky
sometimes to have /mnt be a mounted volume.

A new consistent approach: we mount volumes under /mnt/<name>.
2015-06-26 11:52:45 -04:00
Justin Santa Barbara 7059f379a4 AWS: Mount persistent disk as sdb on master 2015-06-26 11:52:44 -04:00
Adam Sunderland 66e0c5432f Use EBS for Master Data
Signed-off-by: Adam Sunderland <iterion@gmail.com>
2015-06-26 11:52:44 -04:00
Justin Santa Barbara 3ef028148d Fix AWS options doc
Fix formatting, and ZONE should have been KUBE_AWS_ZONE.
2015-06-26 11:16:44 -04:00
Eric Paris 58df58f3d7 Remove unused enable_node_monitoring option
Back in 1a7f7245e7 we dropped the one
place this was used, but left all of the variable and definitions and
garbage around cluster/
2015-06-25 20:57:56 -04:00
Tim Hockin 280f99afd9 EOL our registry caching mirror
Docker's v1 registry has gotten slower and slower, and they have no
interest in fixing it.  Using a mirror forces v1 mode.  Measurements
show that v1 with our mirror is slower than v2 with docker's registry in
just about all metrics.
2015-06-24 09:56:59 -07:00
Jeff Lowdermilk e2d496e45d Merge pull request #9737 from iterion/use-openssl-rsa
AWS: Use RSA to Generate Fingerprint
2015-06-23 10:14:27 -07:00
Justin Santa Barbara 2a5ed2f086 AWS: Use auto-scaling group to run minions
This uses the dynamic CIDR work, and we set source-dest-check to false
when we configure the route (which kind-of makes sense)
2015-06-19 10:22:15 -04:00
Justin Santa Barbara a4e15cdf3e AWS: Configure minion routes dynamically
We need to implement the Routes interface, and then enable the functionality in the cluster scripts.
2015-06-18 14:59:37 -07:00
Adam Sunderland 304d75caf9 Not about instance types 2015-06-17 11:32:22 -05:00
Saad Ali 4d25121206 Merge pull request #9769 from brendandburns/secure
Revert the revert of https://github.com/GoogleCloudPlatform/kubernetes/pull/9761
2015-06-16 14:21:54 -07:00
Fabio Yeon c4d040c720 Merge pull request #9745 from justinsb/aws_manage_route_table
AWS: Create our own route table & tag it for management
2015-06-15 15:18:39 -07:00
Fabio Yeon f033349bea Merge pull request #9746 from justinsb/aws_faster_tagging
AWS: delay less when creating tags
2015-06-15 13:34:44 -07:00
Brendan Burns 51b20b35e2 Revert "Revert "Optionalize (default false) --insecure-registry.""
This reverts commit 1645c9a9b8.
2015-06-12 21:00:16 -07:00
Dawn Chen 1645c9a9b8 Revert "Optionalize (default false) --insecure-registry." 2015-06-12 17:50:38 -07:00
Abhi Shah a6bed65f06 Merge pull request #9685 from brendandburns/secure
Optionalize (default false) --insecure-registry.
2015-06-12 17:06:32 -07:00
Justin Santa Barbara 6839ae9a44 AWS: delay less when creating tags
This is important because tag creation isn't atomic, and we want to minimize
the window in which we can leave 'dangling' resources around.
2015-06-12 18:32:55 -04:00
Justin Santa Barbara 1f04284508 AWS: Create our own route table & tag it for management 2015-06-12 18:32:10 -04:00
Adam Sunderland 7760d079ae Update SSH User For Master SSH Check 2015-06-12 15:38:48 -05:00
Adam Sunderland d82bfffe9d Use RSA to Generate Fingerprint 2015-06-12 15:29:37 -05:00
Brendan Burns 675d8378f2 Optionalize (default false) --insecure-registry. 2015-06-11 16:33:14 -07:00
Abhi Shah 82aa8f9984 Merge pull request #9371 from justinsb/aws_support_wheezy
AWS: Support wheezy, for parity with GCE
2015-06-11 15:08:32 -07:00
Abhi Shah 064e7146e1 Merge pull request #9481 from justinsb/aws_fix_push
AWS: Include (idempotent) ensure-temp-dir in upload-server-tars
2015-06-11 11:24:18 -07:00
Justin Santa Barbara 4db5b6f465 AWS: Don't change the default OS
It may be that we should change the default, but that is a big move,
late in the day, and it warrants its own discussion.
2015-06-11 13:02:33 -04:00
Justin Santa Barbara 8aae864784 AWS: Support wheezy, for parity with GCE 2015-06-09 23:45:41 -04:00
krousey fba6462c0b Merge pull request #9377 from justinsb/aws_persistent_mounts
AWS: add mounts to fstab
2015-06-09 12:57:43 -07:00
krousey 82f922b61d Merge pull request #9378 from justinsb/aws_no_thin_on_wheezy
AWS: Disable thin LVM provisioning on wheezy
2015-06-09 10:01:04 -07:00
krousey e19e4bcd12 Merge pull request #9365 from justinsb/fix_9246
AWS: Only log "Starting cluster using os distro..." in kube-up
2015-06-09 09:51:22 -07:00
krousey 37ed34261e Merge pull request #9364 from justinsb/fix_aws_local_not_in_function
AWS: Fix script issue where local used outside of function
2015-06-09 09:50:43 -07:00
Justin Santa Barbara d92863523f AWS: Include (idempotent) ensure-temp-dir in upload-server-tars
This way we won't forget it.  Fixes kube-push, where I forgot it.
2015-06-09 11:10:15 -04:00
krousey e14d9038fe Merge pull request #9367 from justinsb/aws_ssh_check
AWS: Check for SSH connectivity & better logging on failure
2015-06-08 16:48:04 -07:00
Justin Santa Barbara d955e532f9 AWS: Add mounts to fstab, to survive reboot 2015-06-08 18:07:22 -04:00
Justin Santa Barbara f6440247ca AWS: Don't thin provision LVM volume on wheezy
Thin provisioning isn't supported (unless you backport from jessie).

Just use normal LVM volumes with aufs.
2015-06-08 18:06:24 -04:00
Justin Santa Barbara 1c229e5284 Removed extra blank line in aws util.sh 2015-06-08 16:59:03 -04:00
Justin Santa Barbara 2619b6198a AWS: Mount ephemeral devices, even if not specified in the AMI
We mount up to 4; this covers almost all instance types.
2015-06-08 16:59:03 -04:00
Justin Santa Barbara 72496e7368 AWS: Check for SSH connectivity & better logging on failure 2015-06-06 14:27:41 -04:00
Justin Santa Barbara a9e1e1033b AWS: Only log "Starting cluster using os distro..." in kube-up
Fixes #9246
2015-06-06 12:48:49 -04:00
Justin Santa Barbara 24de0b4598 AWS: Fix script issue where local used outside of function 2015-06-06 12:19:30 -04:00
Justin Santa Barbara 112a013567 AWS: Support different docker storage mechanism by setting DOCKER_STORAGE
For parity with GCE, we really want to support aufs.

But we previously supported btrfs, so we want to expose that.

Most of the work here is required for aufs, and we let advanced users choose
devicemapper/btrfs if they have a setup that works for those configurations.
2015-06-06 12:13:51 -04:00
Justin Santa Barbara 48e8a8b0ec AWS: Set up security groups, to mirror GCE firewalling
Some slightly fussy code to enable load-balancers to talk to
instances, but otherwise relatively simple.
2015-06-05 16:10:08 -04:00
Justin Santa Barbara 33a3d884f2 AWS: Filter by Cluster tag, rationalize EC2 abstraction
Whenever we do a list we now filter on tags so we only see resources relating
to our cluster.

Also, rationalize all the DescribeX calls:
 * They all take a request object (so that we can pass filters)
 * They do paging if that is required (and return the underlying resources)
 * They wrap any error with a "error while listing X: %v" message
2015-06-05 16:09:01 -04:00
Quinton Hoole 710df2b619 Merge pull request #9294 from justinsb/aws_ssh_key_fingerprint
AWS: Support multiple SSH keys (embed the hash in the name)
2015-06-05 09:37:31 -07:00
Justin Santa Barbara d8dc416b5b AWS: Support multiple SSH keys (embed the hash in the name)
This should eliminate a nasty problem where the script doesn't cope well if
your keys don't match.
2015-06-04 21:40:57 -04:00
Justin Santa Barbara c92c63b3a9 AWS: Use s3 sync to optimize upload to s3 when nothing changed 2015-06-04 10:14:45 -04:00
Adam Sunderland 2b4d37427e Update Master IAM Policy to Include ELB 2015-06-03 12:20:19 -05:00
Eric Tune 3c067b766a Merge pull request #8996 from manolitto/aws_cluster_monitoring_fix
aws: fix cluster monitoring (new option "influxdb" instead of "true")
2015-06-01 13:43:43 -07:00
Eric Tune 5b3e01d2fd Merge pull request #8653 from matschaffer/s3-creation-wait
Check that s3 bucket has been created
2015-06-01 10:29:03 -07:00
Manfred Geiler 8c8f8feb62 aws: fix cluster monitoring ("none" instead of "false") 2015-06-01 09:12:41 +02:00
Justin Santa Barbara c4a2631593 Mount logic breaks if /var/lib/kubelet is a symlink
Pass the correct kubelet root-dir on AWS
2015-05-29 20:13:09 -04:00
Rohit Jnagal ff51f0b2e1 Merge pull request #8696 from derekwaynecarr/force_namespace_creation
Force explicit namespace provision, update e2e for failures
2015-05-29 09:28:47 -07:00
Manfred Geiler 635b6bc097 aws: fix cluster monitoring (new option "influxdb" instead of "true") 2015-05-29 11:15:21 +02:00
Tim Hockin ac3cc3c518 Rename PORTAL_NET all over 2015-05-28 16:10:44 -07:00
derekwaynecarr 3e8b1d5e01 Update all salt providers to force explicit namespace creation; update e2e 2015-05-28 13:45:49 -04:00
Mat Schaffer e7ae425385 Colorize errors for consistency with other checks 2015-05-23 16:12:24 +09:00
Mat Schaffer 26736e494c Check that s3 bucket has been created
Fixes #8395
2015-05-22 14:12:36 +09:00
Dawn Chen 04c4d25065 Merge pull request #7905 from bakins/aws-coreos
AWS: use CoreOS for nodes
2015-05-21 09:05:56 -07:00
Victor Marmol 4ba22e713a Merge pull request #8296 from jlowdermilk/gen-analytics
Add ga-beacon analytics to gendocs scripts
2015-05-18 08:40:02 -07:00
Justin Santa Barbara 87dfddb259 AWS: Set MASTER_RESERVED_IP in config-default.sh
Otherwise jenkins fails
2015-05-16 20:32:23 -04:00
Jeff Lowdermilk 553f9f822b Add ga-beacon analytics to gendocs scripts
hack/run-gendocs.sh puts ga-beacon analytics link into all md files,
hack/verify-gendocs.sh verifies presence of link.
2015-05-15 18:56:38 -07:00
Daniel Smith ce4b54ec70 Merge pull request #8209 from krousey/v1beta1_cluster
Removing some v1beta1 uses in cluster/
2015-05-15 14:56:41 -07:00
Kris Rousey 98c457c397 Updating /cluster to use v1beta 3 specs, and change a lot of polling to
healthz instead of api endpoints.
2015-05-15 14:17:55 -07:00
Robert Bailey 9d6c032929 Merge pull request #7888 from madis/associate_aws_elastic_ip_with_master
Associate master instance with AWS Elastic IP
2015-05-14 13:18:19 -07:00
Madis Nõmme 15643a2c72 Add 'auto' option for MASTER_RESERVED_IP. No ElasticIP allocation by default.
Default behaviour when setting up a cluster is using the Amazon-assigned public ip.
It will change between reboots. If MASTER_RESERVED_IP is set to 'auto', new Elastic
IP will be allocated & assigned to master. If MASTER_RESERVED_IP is set to an existing
Elastic IP, it will be used. When something fails, original Amazon-given IP will be used.
2015-05-14 08:33:07 +03:00
Brian Akins fac4350fa6 Initial addition of CoreOS as minion for AWS cluster 2015-05-13 16:39:22 -04:00
Derek Carr 9454d58547 Merge pull request #8127 from liggitt/service_account_admission
Add ServiceAccount admission plugin
2015-05-13 14:03:11 -04:00
Zach Loafman 02f3a32196 Merge pull request #8131 from justinsb/aws_install_salt_gce_style
Install specific salt version on AWS, based on GCE
2015-05-13 06:55:32 -07:00
Madis Nõmme eb220f05a6 Properly get return value (considering errexit). Quote variables. 2015-05-13 10:45:51 +03:00
Madis Nõmme d4d02a9028 Optionally associate master instance with AWS Elastic IP
When MASTER_RESERVED_IP is set to elastic IP from AWS, then aws/util.sh will
associate it with master instance and assign it to KUBE_MASTER_IP. If no MASTER_RESERVED_IP
is set, new elastic ip will be requested from amazon. This allows cluster certificates to
be generated for an IP that doesn't change between stopping & starting cluster instances.

The requested elastic ip is not released when kube-down.sh is run. I think it is good
because user could have created DNS records and it would be bad if the IP was removed.
He can reuse it next time through MASTER_RESERVED_IP when setting up cluster again.
2015-05-13 10:45:51 +03:00
Justin Santa Barbara 23b1a22203 AWS: Don't use policy-rc.d to prevent starting daemons until we're ready
It isn't required
2015-05-12 21:18:48 -04:00
Clayton Coleman 7d620c20b9 Merge pull request #8105 from thockin/dns-domain
Rename default DNS domain to cluster.local
2015-05-12 17:18:45 -04:00
Justin Santa Barbara ffb0e7f9b8 Install specific salt version on AWS, based on GCE
The latest salt version breaks the container_bridge.py _state function

We can lock to the same version as GCE.  This is not a full fix,
because we can't update to the latest salt without breaking GCE,
but this at least unbreaks and sync AWS with GCE.

This isn't a straight copy from GCE, because we still use
the salt master on AWS (for now)

Fixes #8114
2015-05-12 16:33:56 -04:00
Jordan Liggitt e5d47081a2 Add ServiceAccount admission plugin 2015-05-12 15:19:05 -04:00
Tim Hockin e83e49b076 rename default DNS domain to cluster.local 2015-05-11 23:00:43 -07:00
Jordan Liggitt 7e14a80f63 ServiceAccount admission plugin 2015-05-11 17:18:06 -04:00
Justin Santa Barbara 3cf8d72d96 Copy some new properties from config-default => config.test
ENABLE_MINION_PUBLIC_IP was causing a failure because the variable wasn't declared.

ADMISSION_CONTROL should just be set the same for both test & default
2015-05-08 14:30:17 -07:00
Manfred Geiler c5c62f7d57 fixed second missing $ and added curly brackets 2015-05-08 17:18:52 +02:00
Manfred Geiler 1119340260 fixed missing $ 2015-05-08 16:58:49 +02:00
Manfred Geiler 96d34c1106 AWS: added docs for KUBE_ENABLE_MINION_PUBLIC_IP option 2015-05-08 16:56:06 +02:00
Manfred Geiler 205ed2bf6e AWS: make it possible to disable minion public ip association 2015-05-08 00:09:47 +02:00
Zach Loafman 875e83a741 Revert "Revert "Security context - types, kubelet, admission"" 2015-05-05 16:02:13 -07:00
Zach Loafman f48904fd5e Revert "Security context - types, kubelet, admission" 2015-05-05 15:20:39 -07:00
Paul Weil 982bf19c20 security context initial implementation - squash 2015-05-05 13:46:13 -04:00
Robert Bailey c6d4c24f37 Merge pull request #7736 from justinsb/aws_fix_known_tokens_file
AWS: Fix variable naming that meant not all tokens were written
2015-05-04 14:34:11 -07:00
Justin Santa Barbara 96f0a39172 AWS: Fix variable naming that meant not all tokens were written
To resolve the inconsistency, chose to go closer to GCE
2015-05-04 17:28:24 -04:00
Justin Santa Barbara 57f7b658bb AWS: Change apiserver to listen on 443 directly, not through nginx
Mirrors changes in GCE.  I think the same changes will be needed for vagrant.
2015-05-02 16:59:04 -04:00
Eric Paris 6b3a6e6b98 Make copyright ownership statement generic
Instead of saying "Google Inc." (which is not always correct) say "The
Kubernetes Authors", which is generic.
2015-05-01 17:49:56 -04:00
Jan Safranek 6e810492fb Fixed name of kube-proxy path in deployment scripts. 2015-04-28 10:10:37 +02:00
CJ Cullen 39c5bf363b Merge pull request #7303 from erictune/kube_env3
kube-proxy uses token to access port 443 of apiserver
2015-04-27 14:33:53 -07:00
Brendan Burns 637cd57a25 Merge pull request #6606 from gust1n/aws-existing-vpc
AWS: Improving getting existing VPC and subnet
2015-04-27 11:11:25 -07:00
Zach Loafman 5ccfc0a225 Merge pull request #6006 from justinsb/aws_multiple_clusters
First step to supporting multiple k8s clusters
2015-04-27 10:11:03 -07:00
Eric Tune 9044177bb6 Generate a token for kube-proxy.
Tested on GCE.
Includes untested modifications for AWS and Vagrant.
No changes for any other distros.
Probably will work on other up-to-date providers
but beware.  Symptom would be that service proxying
stops working.

 1. Generates a token kube-proxy in AWS, GCE, and Vagrant setup scripts.
 1. Distributes the token via salt-overlay, and salt to /var/lib/kube-proxy/kubeconfig
 1. Changes kube-proxy args:
   - use the --kubeconfig argument
   - changes --master argument from http://MASTER:7080 to https://MASTER
     - http -> https
     - explicit port 7080 -> implied 443

Possible ways this might break other distros:

Mitigation: there is an default empty kubeconfig file.
If the distro does not populate the salt-overlay, then
it should get the empty, which parses to an empty
object, which, combined with the --master argument,
should still work.

Mitigation:
  - azure: Special case to use 7080 in
  - rackspace: way out of date, so don't care.
  - vsphere: way out of date, so don't care.
  - other distros: not using salt.
2015-04-27 08:59:57 -07:00
Joakim Gustin 645d6d1f26 Use existing subnet when launching AWS cluster in existing VPC 2015-04-23 11:01:17 +02:00
Justin Santa Barbara 9253ae6dca AWS: Fix some bash style problems: variable quoting & locals 2015-04-22 18:30:09 -07:00
CJ Cullen 924015dc94 Merge pull request #7182 from justinsb/aws_fix_hostname
AWS: Set hostname_override for minions, back to fully-qualified name
2015-04-22 14:13:53 -07:00
Justin Santa Barbara 4120849cc4 AWS: Set hostname_override for minions, back to fully-qualified name
This is a stop-gap fix; we'd really like to use EC2 instance ids, but that is
blocked by #7092 or changing that health-check to not assume that the node name
is resolvable.

This stop-gap essentially reverts #7072 for AWS
2015-04-22 11:28:53 -07:00
Zach Loafman 86468cd29d Revert "Added kube-proxy token." 2015-04-22 10:55:08 -07:00
Zach Loafman b98f93bb4b Merge pull request #7112 from erictune/kubeconfig-secrets
Extend PR#5470 for AWS and Vagrant
2015-04-22 09:25:53 -07:00
Eric Tune 2ca8a9d15d Added kube-proxy token.
Generates the new token on AWS, GCE, Vagrant.
Renames instance metadata from "kube-token" to "kubelet-token".
(Is this okay for GKE?)

Having separate tokens for kubelet and kube-proxy permits
using principle of least privilege, makes it easy to
rate limit the clients separately, allows annotation
of apiserver logs with the client identity at a finer grain
than just source-ip.
2015-04-21 09:21:31 -07:00
Eric Tune 7475efbcfb Extend PR#5470 for AWS and Vagrant 2015-04-21 08:22:31 -07:00
Justin Santa Barbara 45b658ea7c AWS: don't try to create undefined config_dir 2015-04-20 19:14:16 -07:00
Robert Bailey 82190a58b1 Merge pull request #7026 from justinsb/aws_use_ssl_for_salt
AWS: master should download salt using SSL
2015-04-20 11:58:47 -07:00
Jeff Lowdermilk 3787fc5eca set KUBECONFIG in common.sh, default to new location
and preserve value in ginkgo e2e test driver
2015-04-20 11:07:35 -07:00
Jeff Lowdermilk 4f6dc99075 Generate kubeconfig for all providers in cluster/ that use auth 2015-04-20 11:07:35 -07:00
Justin Santa Barbara 7d3fe2154e AWS: master should download salt using SSL
The minion already does, but the master was using plain http.
2015-04-18 12:38:52 -07:00
Justin Santa Barbara b2bce12b6c AWS: Move /var/lib/kubelet to /mnt
The backing for empty-dir volumes otherwise consumes all our disk space
2015-04-17 06:46:08 -07:00
Justin Santa Barbara 72687184b9 Don't always use aws.conf
We don't usually need it, and it makes it harder to put apiserver into a container.
2015-04-14 09:30:00 -07:00
Justin Santa Barbara 034412aff1 Support multiple k8s clusters 2015-04-10 13:25:43 -07:00
Justin Santa Barbara 49543aca61 Update IAM permissions for minion, to allow EBS 2015-04-10 13:25:42 -07:00
Jeff Lowdermilk ca6f1a1bc7 Merge pull request #6476 from deads2k/deads-fix-kubeconfig-serialization
fix kubeconfig serialization
2015-04-09 09:41:26 -07:00
Justin Santa Barbara 2426366ec8 Update cAdvisor with moved docker root on AWS
We set up a symlink now, and we also pass docker_root into the kubelet.

The symlink is probably sufficient, but doing both feels safer.
2015-04-07 10:40:22 -07:00
deads2k 609208b8b5 update scripts with correct templates 2015-04-07 08:07:24 -04:00
Justin Santa Barbara 5d7f86b041 More options documentation 2015-04-03 13:44:06 -07:00
Justin Santa Barbara 2fb573c482 Create instances in the specified AZ 2015-04-03 08:04:53 -07:00
Zach Loafman fc8ba8d77b Merge pull request #6011 from justinsb/aws_specify_zone
Fix AWS region vs zone
2015-03-31 12:26:05 -07:00
Justin Santa Barbara edbebd653c Fix S3 location handling for US classic 2015-03-31 05:42:10 -07:00
Justin Santa Barbara 07ffc26a61 Fix typo in docs 2015-03-30 06:01:00 -07:00
Justin Santa Barbara 194143e61b Use the correct region for the S3 bucket
We may not have created the bucket!
2015-03-27 12:58:47 -07:00
Justin Santa Barbara 7179f5f004 Change AWS_ZONE to ZONE, for compatibility with e2e tests 2015-03-27 08:53:45 -07:00
Justin Santa Barbara 583892da2d Fix AWS region vs zone
We were specifying a region, but naming it as a zone in util.sh

The zone matters just as much as the region, e.g. for EBS volumes.

We also change the config to require a Zone, not a Region.
But we fallback to get the information from the metadata service.
2015-03-26 15:11:05 -07:00
Zach Loafman e5914915b8 Merge pull request #6003 from justinsb/aws_wait_for_instance
Wait for master to be started on AWS
2015-03-26 14:06:25 -07:00
Justin Santa Barbara f256d3b77a Wait for ELBs to be deleted 2015-03-26 13:50:35 -07:00
Justin Santa Barbara 53bdbec64e Launch minions in parallel 2015-03-26 10:58:18 -07:00
Justin Santa Barbara 9556318c58 Wait for master instance running before adding route 2015-03-26 10:23:25 -07:00
Saad Ali f3ebe30605 Merge pull request #5940 from justinsb/fix_bash_style
Fix bash style issues in AWS script
2015-03-25 14:40:42 -07:00
Justin Santa Barbara 06704bb94f Security group should be scoped to our vpc
Note that security group names only need to be unique with a VPC as well,
which is handy here.
2015-03-25 12:53:12 -07:00
Justin Santa Barbara ac12665f48 Quote AWS_SSH_KEY 2015-03-25 12:41:02 -07:00
Justin Santa Barbara 80ba56a371 Fix variable dollar-char errors 2015-03-25 12:36:52 -07:00
Justin Santa Barbara 505488269c Use double-square brackets in if tests 2015-03-25 12:36:20 -07:00
Alex Robinson c22d4c6b3f Merge pull request #5916 from justinsb/aws_cleanup_loadbalancers
Remove load balancers as part of kube-down.sh
2015-03-25 10:32:14 -07:00
Justin Santa Barbara 68935e0798 Remove load balancers as part of kube-down.sh 2015-03-25 05:36:47 -07:00
Brendan Burns cf72cf098a Disable validation on AWS where it doesn't work.
Also add an advanced route for the master which I forgot previously.
2015-03-24 21:08:04 -07:00
Brendan Burns 8bac135f85 Configure docker and kubelet on AWS master. 2015-03-24 17:19:06 -07:00
Clayton Coleman 97560c409f Merge pull request #5554 from derekwaynecarr/namespace_controller
Namespace Controller to support Namespace Termination
2015-03-24 12:59:00 -04:00
Robert Bailey b152d8e884 Merge pull request #5822 from justinsb/aws_idempotent_kube_down
More idempotent kube-down for AWS, and delete multiple security groups
2015-03-24 07:58:03 -07:00
derekwaynecarr ee53dfc741 Turn on namespace lifecycle plug-in 2015-03-24 10:36:06 -04:00
Saad Ali b01f1f9baf Merge pull request #5683 from justinsb/aws_ephemeral_disks
Format ephemeral disks with btrfs, put Docker storage on there
2015-03-24 07:36:01 -07:00
Justin Santa Barbara 2a042fc423 More idempotent kube-down for AWS, and delete multiple security groups
There may be multiple security groups if we were using ELB, and
we have to delete them all apart from the default one, which EC2
prevents us from deleting.

Also use the same looping logic to clean up from partial up/downs.
2015-03-23 13:00:34 -07:00
Zach Loafman 407d1fec45 Both @satnam6502 and E2E tests confirm: This code is no longer needed
Deletion is wonderful. The only weird thing was where to put the
message about the proxy URLs. Satnam suggested kubectl clusterinfo,
which seemed like a good option to put at the end of cluster turn-up.
2015-03-19 22:23:11 -07:00
Justin Santa Barbara 6fec242f89 Format ephemeral disks with btrfs, put Docker storage on there
Docker's default storage has severe bugs with older (stable) kernels.

Use btrfs to bypass those issues.
2015-03-19 19:23:29 -04:00
Alex Robinson 2537665ecc Merge pull request #5381 from justinsb/aws_s3_in_us_east
Create S3 bucket in known region (default to us-east-1)
2015-03-18 09:34:59 -07:00
Justin Santa Barbara 8b535ff17a Fix copy-and-paste error in log message 2015-03-16 14:30:24 -07:00
Justin Santa Barbara 9361b76977 Configure separate IAM roles for master & minion, create roles automatically 2015-03-16 10:37:25 -07:00
Justin Santa Barbara 2413d25d29 Create S3 bucket in us-east-1, unless set by AWS_S3_REGION
Buckets are globally named, so we want a single bucket for all our regions.
2015-03-16 08:37:06 -07:00
Alex Robinson f6441a6d2f Merge pull request #5376 from justinsb/aws_customize_s3_bucket
Allow the S3 bucket name to be specified via AWS_S3_BUCKET
2015-03-13 13:51:37 -07:00
Justin Santa Barbara e3e6c8377f Allow the S3 bucket name to be specified via AWS_S3_BUCKET 2015-03-13 16:44:26 -04:00
Satnam Singh 4420e0452e Merge pull request #5391 from justinsb/aws_options_doc
Initial WIP on documenting options for AWS clusters
2015-03-12 11:42:19 -07:00
Justin Santa Barbara 84314d3206 Initial WIP on documenting options for AWS clusters
Very incomplete, but a start.
2015-03-12 13:54:37 -04:00
Alex Robinson 55ed70f723 Merge pull request #5382 from justinsb/aws_image_by_region
Choose AWS image by region
2015-03-12 10:31:00 -07:00
Alex Robinson 4ad0ec365e Merge pull request #5378 from justinsb/aws_instance_type
Allow the AWS instance type to be specified via MASTER_SIZE, MINION_SIZE
2015-03-12 09:55:06 -07:00
Alex Robinson 9344934637 Merge pull request #5377 from justinsb/aws_s3_not_world_readable
Don't make the s3 bucket world-readable (just the files)
2015-03-12 09:52:38 -07:00
roberthbailey d3b0c86624 Merge pull request #5380 from justinsb/clarify_aws_comment
Clarify comment
2015-03-12 17:43:43 +01:00
Justin Santa Barbara 70b448b5d0 Choose AWS image by region
Also update to the latest LTS image (fewer security updates needed)
2015-03-12 12:05:45 -04:00
Justin Santa Barbara 0330cf616a Don't make the s3 bucket world-readable (just the files) 2015-03-12 11:52:56 -04:00
Justin Santa Barbara f71e0eeb22 Allow the AWS instance type to be specified via MASTER_SIZE, MINION_SIZE 2015-03-12 11:52:06 -04:00
Justin Santa Barbara 41ed9e4fef Clarify comment 2015-03-12 11:50:12 -04:00
Justin Santa Barbara c43e8b192f Don't print a full-stop after the conf filename, it make copy&paste hard 2015-03-12 11:07:44 -04:00
Vishnu Kannan e7aca5c337 Enable heapster based cluster monitoring by default on other cluster providers, expect for GKE. 2015-03-11 23:34:04 +00:00
Vishnu Kannan 663bdb91c9 Remove monitoring firewall setup methods. Heapster anf Grafana are now accessible via the API server proxy. 2015-03-11 23:20:45 +00:00
derekwaynecarr 20853410bd Unify azure and aws admission control 2015-03-11 11:06:00 -04:00
derekwaynecarr 468bf1da75 Enable common set of admission controllers across salt providers 2015-03-11 11:06:00 -04:00
Justin Santa Barbara 9adfee5ddb Set AWS instance Names so they match the regex 2015-03-05 16:06:53 -08:00
roberthbailey 83c3c53ff6 Merge pull request #5014 from justinsb/aws_cluster_push
Copy download-or-bust helper from gce to aws
2015-03-04 16:26:01 -08:00
Justin Santa Barbara 05d4aea3bf Copy download-or-bust helper from gce to aws
During a push, we were wget-ing to <filename>.1, .2, etc, which wasn't working
2015-03-04 16:30:11 -05:00
Justin Santa Barbara 12e785c3b8 Implement restart-apiserver for AWS, fix comment 2015-03-04 15:35:29 -05:00
Brian Grant 8fd315e324 Merge pull request #4988 from justinsb/sync_aws_with_gce
Use new config file & sync with GCE script
2015-03-03 11:41:48 -08:00
Brian Grant e1ad29e071 Merge pull request #4987 from justinsb/aws_ssh
Make public-ip detection more robust for AWS, enable SSH function
2015-03-03 11:41:05 -08:00
Justin Santa Barbara 6508ca78a5 Use new config file & sync with GCE script 2015-03-03 12:24:56 -05:00
Justin Santa Barbara 6b067b3c85 Make public-ip detection more robust for AWS, enable SSH function 2015-03-03 12:21:44 -05:00
Justin Santa Barbara 94a433157e Use internal ip for connecting to salt master, with clearer env var 2015-03-03 12:18:12 -05:00
Satnam Singh 19b927ea57 Name a cluster and use it to make forwarding rules for GCE 2015-02-23 17:04:33 -08:00
Justin Santa Barbara cf470f7da4 Fixes to get AWS tests to run
They don't pass (yet), but they at least run!
2015-02-11 15:58:33 -05:00
Justin Santa Barbara d8791acdba Pause between launching master and minions
Salt does not retry if master is not ready, apparently
2015-01-29 21:08:27 -05:00
Justin Santa Barbara 3fb61912aa Quote args in /srv/salt-overlay/pillar, add new vars 2015-01-29 20:35:36 -05:00
Justin Santa Barbara bb406ce87f Allow # of minions to be configured 2015-01-29 20:35:36 -05:00
Justin Santa Barbara 8058632f5f Move tag creation & retry logic to a function 2015-01-29 20:35:36 -05:00
Justin Santa Barbara d20188c6d4 Create shared secret in AWS scripts 2015-01-29 20:35:35 -05:00
Zach Loafman a305269e18 Deferred creation of SkyDNS, monitoring and logging objects
This implements phase 1 of the proposal in #3579, moving the creation
of the pods, RCs, and services to the master after the apiserver is
available.

This is such a wide commit because our existing initial config story
is special:

* Add kube-addons service and associated salt configuration:
** We configure /etc/kubernetes/addons to be a directory of objects
that are appropriately configured for the current cluster.
** "/etc/init.d/kube-addons start" slurps up everything in that dir.
(Most of the difficult is the business logic in salt around getting
that directory built at all.)
** We cheat and overlay cluster/addons into saltbase/salt/kube-addons
as config files for the kube-addons meta-service.
* Change .yaml.in files to salt templates
* Rename {setup,teardown}-{monitoring,logging} to
{setup,teardown}-{monitoring,logging}-firewall to properly reflect
their real purpose now (the purpose of these functions is now ONLY to
bring up the firewall rules, and possibly to relay the IP to the user).
* Rework GCE {setup,teardown}-{monitoring,logging}-firewall: Both
functions were improperly configuring global rules, yet used
lifecycles tied to the cluster. Use $NODE_INSTANCE_PREFIX with the
rule. The logging rule needed a $NETWORK specifier. The monitoring
rule tried gcloud describe first, but given the instancing, this feels
like a waste of time now.
* Plumb ENABLE_CLUSTER_MONITORING, ENABLE_CLUSTER_LOGGING,
ELASTICSEARCH_LOGGING_REPLICAS and DNS_REPLICAS down to the master,
since these are needed there now.

(Desperately want just a yaml or json file we can share between
providers that has all this crap. Maybe #3525 is an answer?)

Huge caveats: I've gone pretty firm testing on GCE, including
twiddling the env variables and making sure the objects I expect to
come up, come up. I've tested that it doesn't break GKE bringup
somehow. But I haven't had a chance to test the other providers.
2015-01-21 12:25:50 -08:00
Satnam Singh 295bd3768d Launch Elasticsearch and Kibana automatically 2015-01-09 13:41:46 -08:00
Joe Beda a2e58d490e Merge pull request #2224 from thockin/dns
Enable DNS for services
2014-12-29 10:27:57 -08:00
Justin Santa Barbara e682997b3d Remove erroneous . from wget commands
wget always downloads into current directory
2014-12-29 12:19:45 -05:00
Tim Hockin 59164ca844 Make DNS an option for cluster turnup 2014-12-29 09:18:12 -08:00
Joe Beda d8efaab679 Merge pull request #3068 from brendandburns/dutchcoders-master
Fix up AWS support...
2014-12-19 15:14:26 -08:00
Brendan Burns 2369f0d50d Update aws. 2014-12-19 14:16:57 -08:00
Brendan Burns 824122f410 Merge pull request #2651 from justinsb/specify_zone_to_ec2
Specify AWS_DEFAULT_REGION to match configured region
2014-12-19 12:26:40 -08:00
Brendan Burns bd712b61e6 Merge branch 'master' of https://github.com/dutchcoders/kubernetes into dutchcoders-master
Conflicts:
	cluster/aws/config-default.sh
	cluster/saltbase/salt/kube-controller-manager/default
2014-12-19 11:45:46 -08:00
Brendan Burns 5b33534dcf Merge pull request #2652 from justinsb/default_vpc_has_no_tags
Don't assume every AWS VPC has tags
2014-12-08 15:53:47 -08:00
Justin Santa Barbara 0b05068f40 Default ENABLE_CLUSTER_MONITORING to false
Otherwise kube-down fails because the variable is not defined
2014-11-27 18:07:51 -08:00
Justin Santa Barbara a549468853 Specify AWS_DEFAULT_REGION to match configured region
The AWS CLI is configured with a default region, but it might not match
the configured Kubernetes region.  Specify the region to be sure.
2014-11-27 17:56:14 -08:00
Justin Santa Barbara f9a9c75697 Don't assume every AWS VPC has tags
The default VPC has no tags (by default), so we should use get()
2014-11-27 17:32:38 -08:00
Remco 83f3982eee fix(aws): cloud provider wasn't set, corrected cloud config syntax 2014-11-26 17:30:35 +01:00
Remco 1265db9686 fix(aws): fix with non us standard region 2014-11-26 16:18:57 +01:00
Remco 6d79fab0c2 enhancement(aws): added logging variable and more verbose output 2014-11-26 16:18:26 +01:00
Remco cbaa48bbc2 enhancement(aws): using iam role to pass aws credentials 2014-11-26 16:16:57 +01:00
Remco 2118c31bdb enhancement(aws): checks for existing kubernetes vpc first
without this check, kube-up creates a new vpc every time. Now it will
reuse the vpc tagged Kubernetes.
2014-11-26 12:17:14 +01:00
Remco 72bd5311bf fix(aws): error with other instances without tags
while starting the cluster, and having already other non-kubernetes
instances active without tags, an KeyError throwed.
2014-11-26 12:15:51 +01:00
Tim Hockin 47141f05c7 standard config opt names: logging 2014-11-25 17:41:53 -08:00
Tim Hockin 52ad94d766 standard config opt names: node monitoring 2014-11-25 17:39:28 -08:00
Tim Hockin fad806d1d1 standard config opt names: cluster monitoring 2014-11-25 17:38:30 -08:00
Brendan Burns ab3cfc5f75 Update the ubuntu AMI 2014-11-24 16:34:48 -08:00
Joe Beda ee2f030623 Give the API server access to TLS certs.
Moved the cert generation to a separate salt state and put it in a more appropriate sharable location (`/srv/kubernetes/`).
2014-11-12 18:14:24 -08:00
James Davies 945339362f Strip extra '-' from md5sum output when creating S3 bucket
md5sum prints out the hash, followed by the filename. When piped in from
stdin, this equates to a '-' character.

cluster/aws/util.sh was incorrect including this '-' character as part
of the S3 bucket name, causing the script to fail on Linux machines with
the md5sum binary.

i.e. "s3://kubernetes-staging-0ac68d8c77915cc1069a9e2f5e1f1d2d -"

Fixed by using `awk` to return only the first column (up to the space)
2014-11-12 16:56:50 +10:00
Brendan Burns 48498cf9bb Fix docs somewhat, add monitoring hooks. 2014-11-10 17:17:25 -08:00
Brendan Burns 62bd634c07 Update the AWS config. 2014-11-10 13:33:43 -08:00
Andriy Yurchuk 1ff79fdeb0 AWS support draft
Fix calling function before declaration

Set Name tags on instances

Hide import-key-pair error

Fix instances names resolution

Implement kube-down for AWS provider

Add cluster validation routines. Make changes according to #1255

Implement post-deployment cluster validation

Set proper master name in userdata scripts

Fix kube-down path in hint

Add getting started for AWS
2014-11-10 13:32:52 -08:00