Commit Graph

218 Commits (270fae4d667ca605604972aa9d362a562b4140e8)

Author SHA1 Message Date
Justin Santa Barbara 7179f5f004 Change AWS_ZONE to ZONE, for compatibility with e2e tests 2015-03-27 08:53:45 -07:00
Justin Santa Barbara 583892da2d Fix AWS region vs zone
We were specifying a region, but naming it as a zone in util.sh

The zone matters just as much as the region, e.g. for EBS volumes.

We also change the config to require a Zone, not a Region.
But we fallback to get the information from the metadata service.
2015-03-26 15:11:05 -07:00
Zach Loafman e5914915b8 Merge pull request #6003 from justinsb/aws_wait_for_instance
Wait for master to be started on AWS
2015-03-26 14:06:25 -07:00
Justin Santa Barbara f256d3b77a Wait for ELBs to be deleted 2015-03-26 13:50:35 -07:00
Justin Santa Barbara 53bdbec64e Launch minions in parallel 2015-03-26 10:58:18 -07:00
Justin Santa Barbara 9556318c58 Wait for master instance running before adding route 2015-03-26 10:23:25 -07:00
Saad Ali f3ebe30605 Merge pull request #5940 from justinsb/fix_bash_style
Fix bash style issues in AWS script
2015-03-25 14:40:42 -07:00
Justin Santa Barbara 06704bb94f Security group should be scoped to our vpc
Note that security group names only need to be unique with a VPC as well,
which is handy here.
2015-03-25 12:53:12 -07:00
Justin Santa Barbara ac12665f48 Quote AWS_SSH_KEY 2015-03-25 12:41:02 -07:00
Justin Santa Barbara 80ba56a371 Fix variable dollar-char errors 2015-03-25 12:36:52 -07:00
Justin Santa Barbara 505488269c Use double-square brackets in if tests 2015-03-25 12:36:20 -07:00
Alex Robinson c22d4c6b3f Merge pull request #5916 from justinsb/aws_cleanup_loadbalancers
Remove load balancers as part of kube-down.sh
2015-03-25 10:32:14 -07:00
Justin Santa Barbara 68935e0798 Remove load balancers as part of kube-down.sh 2015-03-25 05:36:47 -07:00
Brendan Burns cf72cf098a Disable validation on AWS where it doesn't work.
Also add an advanced route for the master which I forgot previously.
2015-03-24 21:08:04 -07:00
Brendan Burns 8bac135f85 Configure docker and kubelet on AWS master. 2015-03-24 17:19:06 -07:00
Robert Bailey b152d8e884 Merge pull request #5822 from justinsb/aws_idempotent_kube_down
More idempotent kube-down for AWS, and delete multiple security groups
2015-03-24 07:58:03 -07:00
Saad Ali b01f1f9baf Merge pull request #5683 from justinsb/aws_ephemeral_disks
Format ephemeral disks with btrfs, put Docker storage on there
2015-03-24 07:36:01 -07:00
Justin Santa Barbara 2a042fc423 More idempotent kube-down for AWS, and delete multiple security groups
There may be multiple security groups if we were using ELB, and
we have to delete them all apart from the default one, which EC2
prevents us from deleting.

Also use the same looping logic to clean up from partial up/downs.
2015-03-23 13:00:34 -07:00
Zach Loafman 407d1fec45 Both @satnam6502 and E2E tests confirm: This code is no longer needed
Deletion is wonderful. The only weird thing was where to put the
message about the proxy URLs. Satnam suggested kubectl clusterinfo,
which seemed like a good option to put at the end of cluster turn-up.
2015-03-19 22:23:11 -07:00
Justin Santa Barbara 6fec242f89 Format ephemeral disks with btrfs, put Docker storage on there
Docker's default storage has severe bugs with older (stable) kernels.

Use btrfs to bypass those issues.
2015-03-19 19:23:29 -04:00
Alex Robinson 2537665ecc Merge pull request #5381 from justinsb/aws_s3_in_us_east
Create S3 bucket in known region (default to us-east-1)
2015-03-18 09:34:59 -07:00
Justin Santa Barbara 8b535ff17a Fix copy-and-paste error in log message 2015-03-16 14:30:24 -07:00
Justin Santa Barbara 9361b76977 Configure separate IAM roles for master & minion, create roles automatically 2015-03-16 10:37:25 -07:00
Justin Santa Barbara 2413d25d29 Create S3 bucket in us-east-1, unless set by AWS_S3_REGION
Buckets are globally named, so we want a single bucket for all our regions.
2015-03-16 08:37:06 -07:00
Justin Santa Barbara e3e6c8377f Allow the S3 bucket name to be specified via AWS_S3_BUCKET 2015-03-13 16:44:26 -04:00
Alex Robinson 55ed70f723 Merge pull request #5382 from justinsb/aws_image_by_region
Choose AWS image by region
2015-03-12 10:31:00 -07:00
Alex Robinson 9344934637 Merge pull request #5377 from justinsb/aws_s3_not_world_readable
Don't make the s3 bucket world-readable (just the files)
2015-03-12 09:52:38 -07:00
Justin Santa Barbara 70b448b5d0 Choose AWS image by region
Also update to the latest LTS image (fewer security updates needed)
2015-03-12 12:05:45 -04:00
Justin Santa Barbara 0330cf616a Don't make the s3 bucket world-readable (just the files) 2015-03-12 11:52:56 -04:00
Justin Santa Barbara c43e8b192f Don't print a full-stop after the conf filename, it make copy&paste hard 2015-03-12 11:07:44 -04:00
Vishnu Kannan 663bdb91c9 Remove monitoring firewall setup methods. Heapster anf Grafana are now accessible via the API server proxy. 2015-03-11 23:20:45 +00:00
derekwaynecarr 20853410bd Unify azure and aws admission control 2015-03-11 11:06:00 -04:00
roberthbailey 83c3c53ff6 Merge pull request #5014 from justinsb/aws_cluster_push
Copy download-or-bust helper from gce to aws
2015-03-04 16:26:01 -08:00
Justin Santa Barbara 05d4aea3bf Copy download-or-bust helper from gce to aws
During a push, we were wget-ing to <filename>.1, .2, etc, which wasn't working
2015-03-04 16:30:11 -05:00
Justin Santa Barbara 12e785c3b8 Implement restart-apiserver for AWS, fix comment 2015-03-04 15:35:29 -05:00
Brian Grant 8fd315e324 Merge pull request #4988 from justinsb/sync_aws_with_gce
Use new config file & sync with GCE script
2015-03-03 11:41:48 -08:00
Brian Grant e1ad29e071 Merge pull request #4987 from justinsb/aws_ssh
Make public-ip detection more robust for AWS, enable SSH function
2015-03-03 11:41:05 -08:00
Justin Santa Barbara 6508ca78a5 Use new config file & sync with GCE script 2015-03-03 12:24:56 -05:00
Justin Santa Barbara 6b067b3c85 Make public-ip detection more robust for AWS, enable SSH function 2015-03-03 12:21:44 -05:00
Justin Santa Barbara 94a433157e Use internal ip for connecting to salt master, with clearer env var 2015-03-03 12:18:12 -05:00
Satnam Singh 19b927ea57 Name a cluster and use it to make forwarding rules for GCE 2015-02-23 17:04:33 -08:00
Justin Santa Barbara cf470f7da4 Fixes to get AWS tests to run
They don't pass (yet), but they at least run!
2015-02-11 15:58:33 -05:00
Justin Santa Barbara d8791acdba Pause between launching master and minions
Salt does not retry if master is not ready, apparently
2015-01-29 21:08:27 -05:00
Justin Santa Barbara 8058632f5f Move tag creation & retry logic to a function 2015-01-29 20:35:36 -05:00
Zach Loafman a305269e18 Deferred creation of SkyDNS, monitoring and logging objects
This implements phase 1 of the proposal in #3579, moving the creation
of the pods, RCs, and services to the master after the apiserver is
available.

This is such a wide commit because our existing initial config story
is special:

* Add kube-addons service and associated salt configuration:
** We configure /etc/kubernetes/addons to be a directory of objects
that are appropriately configured for the current cluster.
** "/etc/init.d/kube-addons start" slurps up everything in that dir.
(Most of the difficult is the business logic in salt around getting
that directory built at all.)
** We cheat and overlay cluster/addons into saltbase/salt/kube-addons
as config files for the kube-addons meta-service.
* Change .yaml.in files to salt templates
* Rename {setup,teardown}-{monitoring,logging} to
{setup,teardown}-{monitoring,logging}-firewall to properly reflect
their real purpose now (the purpose of these functions is now ONLY to
bring up the firewall rules, and possibly to relay the IP to the user).
* Rework GCE {setup,teardown}-{monitoring,logging}-firewall: Both
functions were improperly configuring global rules, yet used
lifecycles tied to the cluster. Use $NODE_INSTANCE_PREFIX with the
rule. The logging rule needed a $NETWORK specifier. The monitoring
rule tried gcloud describe first, but given the instancing, this feels
like a waste of time now.
* Plumb ENABLE_CLUSTER_MONITORING, ENABLE_CLUSTER_LOGGING,
ELASTICSEARCH_LOGGING_REPLICAS and DNS_REPLICAS down to the master,
since these are needed there now.

(Desperately want just a yaml or json file we can share between
providers that has all this crap. Maybe #3525 is an answer?)

Huge caveats: I've gone pretty firm testing on GCE, including
twiddling the env variables and making sure the objects I expect to
come up, come up. I've tested that it doesn't break GKE bringup
somehow. But I haven't had a chance to test the other providers.
2015-01-21 12:25:50 -08:00
Satnam Singh 295bd3768d Launch Elasticsearch and Kibana automatically 2015-01-09 13:41:46 -08:00
Tim Hockin 59164ca844 Make DNS an option for cluster turnup 2014-12-29 09:18:12 -08:00
Joe Beda d8efaab679 Merge pull request #3068 from brendandburns/dutchcoders-master
Fix up AWS support...
2014-12-19 15:14:26 -08:00
Brendan Burns 2369f0d50d Update aws. 2014-12-19 14:16:57 -08:00
Brendan Burns 824122f410 Merge pull request #2651 from justinsb/specify_zone_to_ec2
Specify AWS_DEFAULT_REGION to match configured region
2014-12-19 12:26:40 -08:00
Brendan Burns bd712b61e6 Merge branch 'master' of https://github.com/dutchcoders/kubernetes into dutchcoders-master
Conflicts:
	cluster/aws/config-default.sh
	cluster/saltbase/salt/kube-controller-manager/default
2014-12-19 11:45:46 -08:00
Brendan Burns 5b33534dcf Merge pull request #2652 from justinsb/default_vpc_has_no_tags
Don't assume every AWS VPC has tags
2014-12-08 15:53:47 -08:00
Justin Santa Barbara 0b05068f40 Default ENABLE_CLUSTER_MONITORING to false
Otherwise kube-down fails because the variable is not defined
2014-11-27 18:07:51 -08:00
Justin Santa Barbara a549468853 Specify AWS_DEFAULT_REGION to match configured region
The AWS CLI is configured with a default region, but it might not match
the configured Kubernetes region.  Specify the region to be sure.
2014-11-27 17:56:14 -08:00
Justin Santa Barbara f9a9c75697 Don't assume every AWS VPC has tags
The default VPC has no tags (by default), so we should use get()
2014-11-27 17:32:38 -08:00
Remco 1265db9686 fix(aws): fix with non us standard region 2014-11-26 16:18:57 +01:00
Remco 6d79fab0c2 enhancement(aws): added logging variable and more verbose output 2014-11-26 16:18:26 +01:00
Remco cbaa48bbc2 enhancement(aws): using iam role to pass aws credentials 2014-11-26 16:16:57 +01:00
Remco 2118c31bdb enhancement(aws): checks for existing kubernetes vpc first
without this check, kube-up creates a new vpc every time. Now it will
reuse the vpc tagged Kubernetes.
2014-11-26 12:17:14 +01:00
Remco 72bd5311bf fix(aws): error with other instances without tags
while starting the cluster, and having already other non-kubernetes
instances active without tags, an KeyError throwed.
2014-11-26 12:15:51 +01:00
Tim Hockin 47141f05c7 standard config opt names: logging 2014-11-25 17:41:53 -08:00
Tim Hockin 52ad94d766 standard config opt names: node monitoring 2014-11-25 17:39:28 -08:00
Tim Hockin fad806d1d1 standard config opt names: cluster monitoring 2014-11-25 17:38:30 -08:00
Joe Beda ee2f030623 Give the API server access to TLS certs.
Moved the cert generation to a separate salt state and put it in a more appropriate sharable location (`/srv/kubernetes/`).
2014-11-12 18:14:24 -08:00
James Davies 945339362f Strip extra '-' from md5sum output when creating S3 bucket
md5sum prints out the hash, followed by the filename. When piped in from
stdin, this equates to a '-' character.

cluster/aws/util.sh was incorrect including this '-' character as part
of the S3 bucket name, causing the script to fail on Linux machines with
the md5sum binary.

i.e. "s3://kubernetes-staging-0ac68d8c77915cc1069a9e2f5e1f1d2d -"

Fixed by using `awk` to return only the first column (up to the space)
2014-11-12 16:56:50 +10:00
Brendan Burns 48498cf9bb Fix docs somewhat, add monitoring hooks. 2014-11-10 17:17:25 -08:00
Brendan Burns 62bd634c07 Update the AWS config. 2014-11-10 13:33:43 -08:00
Andriy Yurchuk 1ff79fdeb0 AWS support draft
Fix calling function before declaration

Set Name tags on instances

Hide import-key-pair error

Fix instances names resolution

Implement kube-down for AWS provider

Add cluster validation routines. Make changes according to #1255

Implement post-deployment cluster validation

Set proper master name in userdata scripts

Fix kube-down path in hint

Add getting started for AWS
2014-11-10 13:32:52 -08:00