Commit Graph

268 Commits (0fa23716f04aa89bf0e5f28a7537d6620c92be79)

Author SHA1 Message Date
Alex Robinson f6f8cb07ae Transfer maintainer responsibilities for fluentd configs/images. 2015-08-26 21:48:51 +00:00
Satnam Singh 278a3911fd Fetch fluentd td agent securely 2015-08-19 13:34:03 -07:00
Kris Rousey ae6c64d9bb Moving everyone to unversioned client 2015-08-18 10:23:03 -07:00
Alex Robinson c887ad75b8 Stop collecting monit logs via fluentd. 2015-08-07 00:12:21 +00:00
Mike Danese 17defc7383 run gofmt on everything we touched 2015-08-05 17:52:56 -07:00
Mike Danese 8e33cbfa28 rewrite go imports 2015-08-05 17:30:03 -07:00
Alex Robinson cc02e50fc7 Collect more of the VM's log files (monit, salt) in fluentd-es. 2015-08-05 00:22:38 +00:00
Alex Robinson 69ab7026af Update fluentd-es to collect etcd logs and remove merge conflict cruft. 2015-07-30 22:21:01 +00:00
Satnam Singh 691c07763c Remove use of v1beta3 from ES logging 2015-07-06 18:47:45 -07:00
Brendan Burns 988aa6fdf6 Move things into a 'kube-system' namespace. 2015-07-06 15:08:23 -07:00
Zach Loafman a2250d1caf Merge pull request #10597 from a-robinson/fluentd
Run fluentd on the master to collect the core master logs
2015-07-04 21:12:47 -07:00
Alex Robinson 4bfbd5f553 Add limits to the fluentd buffer size. 2015-07-03 23:24:08 +00:00
Zach Loafman 1d16be675c Merge pull request #10653 from dchen1107/resource_management
Set resource limit for addon containers
2015-07-01 19:35:28 -07:00
Dawn Chen 6b61918b67 Set resource limit for both elasticsearch and kibana containers based on data collected by #10335 2015-07-01 16:01:09 -07:00
nikhiljindal 274792d7bb Stop exposing v1beta3 by default 2015-07-01 14:38:02 -07:00
Alex Robinson 8855371763 Configure fluentd to collect additional log files, including master log files.
This also merges the output blocks together because there's no compelling
reason to have a separate output block for each input.
2015-07-01 20:07:47 +00:00
Satnam Singh 4e58e001da Change port names to avoid validation failure 2015-06-30 10:56:20 -07:00
Satnam Singh cb9f7d17f5 Add readiness probes to Elasticsearch and Kibana logging 2015-06-22 12:40:15 -07:00
Tim St. Clair f5830edbbc s/gcloud preview docker/gcloud docker/ 2015-06-18 12:27:08 -07:00
Satnam Singh ce49c32d6a Add readiness probe to Elasticsearch and Kibana logging pods 2015-06-18 10:46:05 -07:00
Daniel Smith 40eb1599dd fix elasticsearch 2015-06-03 15:41:09 -07:00
Victor Marmol 7dde223c2a Merge pull request #8458 from satnam6502/elasticsearch
Give explicit version number to Elasticsearch logging RC
2015-05-18 16:27:08 -07:00
Satnam Singh 3337cea8bb Give explicit version number to Kibana RC 2015-05-18 16:04:13 -07:00
Victor Marmol e83a4e3fd8 Merge pull request #8450 from satnam6502/kibana
Add version label to Kibana controller and service
2015-05-18 15:56:27 -07:00
Satnam Singh 83cd97a2e7 Give explicit version number to Elasticsearch logging RC 2015-05-18 15:53:01 -07:00
Satnam Singh 4a0c6b3e0b Add version label to Kibana controller and service 2015-05-18 14:49:12 -07:00
Satnam Singh c84fd94b11 Update Elasticsearch cluster level service to add version label 2015-05-18 14:43:58 -07:00
Jeff Lowdermilk 553f9f822b Add ga-beacon analytics to gendocs scripts
hack/run-gendocs.sh puts ga-beacon analytics link into all md files,
hack/verify-gendocs.sh verifies presence of link.
2015-05-15 18:56:38 -07:00
Satnam Singh ca121ee98d Revert to using emptyDir for Elasticsearch data 2015-05-13 12:52:56 -07:00
Nikhil Jindal c2fecff3d5 Merge pull request #8067 from piosz/kubectl
Added label kubernetes.io/name to cluster services
2015-05-12 17:09:22 -07:00
Satnam Singh 7cff506c93 Make the Elasticsearch logging pods discover each other 2015-05-12 15:26:09 -07:00
Piotr Szczesniak 12e44ff8a9 Added label kubernetes.io/name to cluster services
The label specifies service name which is displayed in kubectl cluster-info
2015-05-11 19:35:53 +02:00
Satnam Singh fcbe2229df Kibana by itself
Upgrade to Kibana 4 for cluster logging
2015-05-08 14:55:52 -07:00
Eric Paris 6b3a6e6b98 Make copyright ownership statement generic
Instead of saying "Google Inc." (which is not always correct) say "The
Kubernetes Authors", which is generic.
2015-05-01 17:49:56 -04:00
Satnam Singh 95007d7217 Improve comment in Fluentd to ES logging config file 2015-04-30 13:42:30 -07:00
Satnam Singh e65f0aafd5 Trim cluster log tags to pod name and container name 2015-04-29 18:24:13 -07:00
Satnam Singh dc31c58443 Upgrade Elasticsearch to 1.5.2 for cluster logging 2015-04-28 16:20:03 -07:00
Alex Robinson ada9242155 Remove unused env variable from the fluentd dockerfiles and make their apt-get
arguments consistent.
2015-04-27 20:47:38 +00:00
Robert Bailey a9f8f17299 Fetch the kibana tar.gz over SSL. 2015-04-26 21:57:49 -07:00
Nikhil Jindal 84cb48be11 Merge pull request #7246 from satnam6502/es
Convert Elasticsearch logging to v1beta3 and de-salt
2015-04-24 09:21:41 -07:00
Satnam Singh c9b9e7651e Convert Elasticsearch logging to v1beta and de-salt 2015-04-23 13:06:15 -07:00
Satnam Singh 0abc34245a Update Kibana RC and service to v1beta3 2015-04-23 12:19:17 -07:00
Dawn Chen f9156c281a Merge pull request #7123 from satnam6502/logging
Propagate pod and container name for log files
2015-04-23 10:13:08 -07:00
Satnam Singh e4e830badc Update Kibana pod to speak to Elasticsearchusing v1beta3 2015-04-22 17:37:47 -07:00
Satnam Singh 2444c1f943 Propagate pod and container name for log files 2015-04-22 15:08:51 -07:00
Alex Robinson 059a8c92bd Move the logging-related directories to where I think they belong.
1. Move fluentd-gcp to be a core cluster addon, rather than a contrib.
2. Get rid of the synthetic logger under contrib, since the exact same
synthetic logger was also included in the logging-demo.
3. Move the logging-demo to examples, since it's effectively an example.

We should also consider adding on a GCP section to the logging-demo
example :)
2015-04-17 23:59:50 +00:00
Tim Hockin c2bacd588d Stop using dockerfile/* images
As per
http://blog.docker.com/2015/03/updates-available-to-popular-repos-update-your-images/
docker has stopped answering dockerfile/redis and dockerfile/nginx.  Fix all
users in our tree.  Sadly this means a lot of published examples are now broken.
2015-04-16 12:20:43 -07:00
Satnam Singh 4c0f06ec66 Convert es-image for gcr.io 2015-04-15 16:28:02 -04:00
Satnam Singh cfb8577324 Update Kibana to v1.2 which paramaterizes location of Elasticsearch 2015-04-03 14:55:00 -07:00
Jon Langemak 35038dc19a Added ES ENV variables so Kibana can be access in other ways 2015-04-02 21:58:10 -05:00
Jeff Lowdermilk 506887515d Rename google-containers to google_containers in image names 2015-04-01 09:16:48 -07:00
Satnam Singh 73a9ab2928 Rename logging image to use google_containers 2015-03-31 16:00:52 -07:00
Jeff Lowdermilk bec982774b Merge pull request #6038 from satnam6502/fluentd-es-image
Create a gcr.io image for Flunentd to Elasticsearch node level logging
2015-03-27 09:05:00 -07:00
Satnam Singh 19e621c951 Create a gcr.io image for Flunentd to Elasticsearch node level logging 2015-03-26 21:07:15 -07:00
Jeff Lowdermilk 2b71857816 Pull cluster addon images from gcr.io 2015-03-26 19:59:27 -07:00
Satnam Singh 985d5c1f95 Add a prefix to the POS file for Fluentd to Elasticsearch 2015-03-17 13:13:03 -07:00
Satnam Singh 3ce0ede78e Modify Fluentd to ES setup to pass flags 2015-03-10 10:11:26 -07:00
Satnam Singh 2486de1287 Change Fluentd config to read from head of Docker container logs 2015-03-09 23:25:52 -07:00
Satnam Singh 72183c064c Use proxy instead of load balancer 2015-03-02 13:44:41 -08:00
Tim Hockin e0e6741027 Stop using named ports in examples and scripts. 2015-03-01 21:30:45 -08:00
Satnam Singh eda8a676f4 Make a Kubernetes specific version of an Elasticsearch Docker container 2015-02-23 18:41:56 -08:00
Zach Loafman ea5d2f99ae Merge pull request #4622 from satnam6502/cluster-logging
Update cluster logging documentation to use kubectl for public IPs
2015-02-23 09:57:46 -08:00
Satnam Singh df2dfd6695 Add kubernetes.io/cluster-service label to cluster level logging services 2015-02-19 16:03:41 -08:00
Satnam Singh b2fcb14db1 Update cluster logging documentation to use kubectl for obtaining the public IPs 2015-02-19 14:05:16 -08:00
Zach Loafman ae27f29c56 Reinstate cluster logging URLs to the end of kube-up.sh
Adds labels to the services, waits for them to be created (which
should be instant, but just in case), query the forwarding rules like
as we did before.

Fixes #3893
2015-02-03 08:27:02 -08:00
Zach Loafman a305269e18 Deferred creation of SkyDNS, monitoring and logging objects
This implements phase 1 of the proposal in #3579, moving the creation
of the pods, RCs, and services to the master after the apiserver is
available.

This is such a wide commit because our existing initial config story
is special:

* Add kube-addons service and associated salt configuration:
** We configure /etc/kubernetes/addons to be a directory of objects
that are appropriately configured for the current cluster.
** "/etc/init.d/kube-addons start" slurps up everything in that dir.
(Most of the difficult is the business logic in salt around getting
that directory built at all.)
** We cheat and overlay cluster/addons into saltbase/salt/kube-addons
as config files for the kube-addons meta-service.
* Change .yaml.in files to salt templates
* Rename {setup,teardown}-{monitoring,logging} to
{setup,teardown}-{monitoring,logging}-firewall to properly reflect
their real purpose now (the purpose of these functions is now ONLY to
bring up the firewall rules, and possibly to relay the IP to the user).
* Rework GCE {setup,teardown}-{monitoring,logging}-firewall: Both
functions were improperly configuring global rules, yet used
lifecycles tied to the cluster. Use $NODE_INSTANCE_PREFIX with the
rule. The logging rule needed a $NETWORK specifier. The monitoring
rule tried gcloud describe first, but given the instancing, this feels
like a waste of time now.
* Plumb ENABLE_CLUSTER_MONITORING, ENABLE_CLUSTER_LOGGING,
ELASTICSEARCH_LOGGING_REPLICAS and DNS_REPLICAS down to the master,
since these are needed there now.

(Desperately want just a yaml or json file we can share between
providers that has all this crap. Maybe #3525 is an answer?)

Huge caveats: I've gone pretty firm testing on GCE, including
twiddling the env variables and making sure the objects I expect to
come up, come up. I've tested that it doesn't break GKE bringup
somehow. But I haven't had a chance to test the other providers.
2015-01-21 12:25:50 -08:00
Satnam Singh cd5a44e381 Remove curl check from Kibana image 2015-01-14 21:32:29 -08:00
Satnam Singh 295bd3768d Launch Elasticsearch and Kibana automatically 2015-01-09 13:41:46 -08:00