github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
k3s/examples/elasticsearch/README.md

164 lines
8.0 KiB
Markdown
Raw Normal View History

Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
# Elasticsearch for Kubernetes
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
Kubernetes makes it trivial for anyone to easily build and scale [Elasticsearch](http://www.elasticsearch.org/) clusters. Here, you'll find how to do so.
Current Elasticsearch version is `1.7.1`.
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
[A more robust example that follows Elasticsearch best-practices of separating nodes concern is also available](production_cluster/README.md).
apply changes
2015-07-17 02:01:02 +00:00
Fix broken warning image link in docs
2016-07-15 09:44:58 +00:00
<img src="http://kubernetes.io/kubernetes/img/warning.png" alt="WARNING" width="25" height="25"> Current pod descriptors use an `emptyDir` for storing data in each data node container. This is meant to be for the sake of simplicity and [should be adapted according to your storage needs](../../docs/design/persistent-storage.md).
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
## Docker image
examples/elasticsearch: fix awkward sentence
2016-08-14 19:18:24 +00:00
The [pre-built image](https://github.com/pires/docker-elasticsearch-kubernetes) used in this example will not be supported. Feel free to fork to fit your own needs, but keep in mind that you will need to change Kubernetes descriptors accordingly.
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
## Deploy
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
Let's kickstart our cluster with 1 instance of Elasticsearch.
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
```
kubectl create -f examples/elasticsearch/service-account.yaml
kubectl create -f examples/elasticsearch/es-svc.yaml
kubectl create -f examples/elasticsearch/es-rc.yaml
Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
```
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
Let's see if it worked:
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
```
$ kubectl get pods
Update Elasticsearch example to remove use of secrets
2015-08-12 23:52:50 +00:00
NAME READY STATUS RESTARTS AGE
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
es-kfymw 1/1 Running 0 7m
kube-dns-p3v1u 3/3 Running 0 19m
Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
```
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
```
$ kubectl logs es-kfymw
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
[2015-08-30 10:01:31,946][INFO ][node ] [Hammerhead] version[1.7.1], pid[7], build[b88f43f/2015-07-29T09:54:16Z]
[2015-08-30 10:01:31,946][INFO ][node ] [Hammerhead] initializing ...
[2015-08-30 10:01:32,110][INFO ][plugins ] [Hammerhead] loaded [cloud-kubernetes], sites []
[2015-08-30 10:01:32,153][INFO ][env ] [Hammerhead] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.4gb], net total_space [15.5gb], types [ext4]
[2015-08-30 10:01:37,188][INFO ][node ] [Hammerhead] initialized
[2015-08-30 10:01:37,189][INFO ][node ] [Hammerhead] starting ...
[2015-08-30 10:01:37,499][INFO ][transport ] [Hammerhead] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.244.48.2:9300]}
[2015-08-30 10:01:37,550][INFO ][discovery ] [Hammerhead] myesdb/n2-6uu_UT3W5XNrjyqBPiA
[2015-08-30 10:01:43,966][INFO ][cluster.service ] [Hammerhead] new_master [Hammerhead][n2-6uu_UT3W5XNrjyqBPiA][es-kfymw][inet[/10.244.48.2:9300]]{master=true}, reason: zen-disco-join (elected_as_master)
[2015-08-30 10:01:44,010][INFO ][http ] [Hammerhead] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.244.48.2:9200]}
[2015-08-30 10:01:44,011][INFO ][node ] [Hammerhead] started
[2015-08-30 10:01:44,042][INFO ][gateway ] [Hammerhead] recovered [0] indices into cluster_state
Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
```
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
So we have a 1-node Elasticsearch cluster ready to handle some work.
Use example syncer tags instead of hard-coded examples in doc
2015-07-20 22:46:20 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
## Scale
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
Scaling is as easy as:
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
```
kubectl scale --replicas=3 rc es
```
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
Did it work?
Update Elasticsearch example to remove use of secrets
2015-08-12 23:52:50 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
```
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
es-78e0s 1/1 Running 0 8m
es-kfymw 1/1 Running 0 17m
es-rjmer 1/1 Running 0 8m
kube-dns-p3v1u 3/3 Running 0 30m
Update Elasticsearch example to remove use of secrets
2015-08-12 23:52:50 +00:00
```
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
Let's take a look at logs:
Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
```
$ kubectl logs es-kfymw
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.DailyRollingFileAppender.
[2015-08-30 10:01:31,946][INFO ][node ] [Hammerhead] version[1.7.1], pid[7], build[b88f43f/2015-07-29T09:54:16Z]
[2015-08-30 10:01:31,946][INFO ][node ] [Hammerhead] initializing ...
[2015-08-30 10:01:32,110][INFO ][plugins ] [Hammerhead] loaded [cloud-kubernetes], sites []
[2015-08-30 10:01:32,153][INFO ][env ] [Hammerhead] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [14.4gb], net total_space [15.5gb], types [ext4]
[2015-08-30 10:01:37,188][INFO ][node ] [Hammerhead] initialized
[2015-08-30 10:01:37,189][INFO ][node ] [Hammerhead] starting ...
[2015-08-30 10:01:37,499][INFO ][transport ] [Hammerhead] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.244.48.2:9300]}
[2015-08-30 10:01:37,550][INFO ][discovery ] [Hammerhead] myesdb/n2-6uu_UT3W5XNrjyqBPiA
[2015-08-30 10:01:43,966][INFO ][cluster.service ] [Hammerhead] new_master [Hammerhead][n2-6uu_UT3W5XNrjyqBPiA][es-kfymw][inet[/10.244.48.2:9300]]{master=true}, reason: zen-disco-join (elected_as_master)
[2015-08-30 10:01:44,010][INFO ][http ] [Hammerhead] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.244.48.2:9200]}
[2015-08-30 10:01:44,011][INFO ][node ] [Hammerhead] started
[2015-08-30 10:01:44,042][INFO ][gateway ] [Hammerhead] recovered [0] indices into cluster_state
[2015-08-30 10:08:02,517][INFO ][cluster.service ] [Hammerhead] added {[Tenpin][2gv5MiwhRiOSsrTOF3DhuA][es-78e0s][inet[/10.244.54.4:9300]]{master=true},}, reason: zen-disco-receive(join from node[[Tenpin][2gv5MiwhRiOSsrTOF3DhuA][es-78e0s][inet[/10.244.54.4:9300]]{master=true}])
[2015-08-30 10:10:10,645][INFO ][cluster.service ] [Hammerhead] added {[Evilhawk][ziTq2PzYRJys43rNL2tbyg][es-rjmer][inet[/10.244.33.3:9300]]{master=true},}, reason: zen-disco-receive(join from node[[Evilhawk][ziTq2PzYRJys43rNL2tbyg][es-rjmer][inet[/10.244.33.3:9300]]{master=true}])
Update Elasticsearch example to remove use of secrets
2015-08-12 23:52:50 +00:00
```
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
So we have a 3-node Elasticsearch cluster ready to handle more work.
Update Elasticsearch example to remove use of secrets
2015-08-12 23:52:50 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
## Access the service
Update Elasticsearch example to remove use of secrets
2015-08-12 23:52:50 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
*Don't forget* that services in Kubernetes are only acessible from containers in the cluster. For different behavior you should [configure the creation of an external load-balancer](http://kubernetes.io/v1.0/docs/user-guide/services.html#type-loadbalancer). While it's supported within this example service descriptor, its usage is out of scope of this document, for now.
```
$ kubectl get service elasticsearch
NAME LABELS SELECTOR IP(S) PORT(S)
elasticsearch component=elasticsearch component=elasticsearch 10.100.108.94 9200/TCP
9300/TCP
Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
```
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
From any host on your cluster (that's running `kube-proxy`), run:
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
```
$ curl 10.100.108.94:9200
```
You should see something similar to the following:
```json
Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
{
"status" : 200,
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
"name" : "Hammerhead",
"cluster_name" : "myesdb",
Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
"version" : {
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
"number" : "1.7.1",
"build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
"build_timestamp" : "2015-07-29T09:54:16Z",
Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
```
apply changes
2015-07-17 02:01:02 +00:00
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
Or if you want to check cluster information:
apply changes
2015-07-17 02:01:02 +00:00
Elasticsearch Discovery with Kubernetes
2015-05-07 00:29:24 +00:00
```
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
curl 10.100.108.94:9200/_cluster/health?pretty
Add ga-beacon analytics to gendocs scripts hack/run-gendocs.sh puts ga-beacon analytics link into all md files, hack/verify-gendocs.sh verifies presence of link.
2015-05-14 22:12:45 +00:00
```
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
You should see something similar to the following:
```json
{
"cluster_name" : "myesdb",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0
}
```
Apply mungedocs changes
2015-07-14 00:13:09 +00:00
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
Add ga-beacon analytics to gendocs scripts hack/run-gendocs.sh puts ga-beacon analytics link into all md files, hack/verify-gendocs.sh verifies presence of link.
2015-05-14 22:12:45 +00:00
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/examples/elasticsearch/README.md?pixel)]()
Revamped Elasticsearch example that now uses an Alpine Linux container with JRE 8u51 and Elasticsearch 1.7.1. Replaced Go discovery mechanism for Elasticsearch discovery plug-in that supports Kubernetes.
2015-08-21 11:30:11 +00:00
<!-- END MUNGE: GENERATED_ANALYTICS -->