Commit Graph

41 Commits (fa76de79e5d1670b8e6add30f0159c833534a298)

Author SHA1 Message Date
Chao Xu 2b2f285777 run hack/update-generated-docs.sh 2015-12-21 10:41:30 -08:00
Matthew Farrellee 314e0c5512 Eliminate need for wget (use curl), use fewer intermediate files be quiet 2015-11-20 18:54:13 -05:00
Matthew Farrellee eff6169ac7 Change default to local build, require explicit "make push" to push 2015-11-20 16:58:59 -05:00
Matthew Farrellee fb4afda2bf Install numpy on all spark images 2015-11-20 16:05:47 -05:00
Matthew Farrellee 2d37029a39 Use upstream zeppelin 0.5.5 release 2015-11-20 14:22:56 -05:00
Matthew Farrellee 4e181c4c1f Add cleaning to clean target 2015-11-20 10:19:57 -05:00
Zach Loafman f8b83a65da Zeppelin: Compress the image by doing the build on a separate image
Pull the tar through to the image we need it on.

Addresses #17231
2015-11-17 14:51:16 -08:00
k8s-merge-robot c377b32b3f Merge pull request #17025 from zmerlynn/zeppelin
Auto commit by PR queue bot
2015-11-13 16:12:54 -08:00
Tim Hockin dab591b629 Run update-gendocs 2015-11-13 13:57:57 -08:00
Zach Loafman 51817850ba Zeppelin: Add Zeppelin image to Spark example
This adds a very basic Zeppelin image that works with the existing
Spark example. As can be seen from the documentation, it has a couple
of warts:

* It requires kubectl port-forward (which is unstable across long
periods of time, at least for me, on this app, bug incoming). See

* I needed to roll my own container (none of the existing containers
exactly matched needs, or even built anymore against modern Zeppelin
master, and the rest of the example is Spark 1.5).

The image itself is *huge*. One of the further refinements we need to
look at is how to possibly strip the Maven build for this container
down to just the interpreters we care about, because the deps here
are frankly ridiculous.

This might be a case where, if possible, we might want to open an
upstream request to build things dynamically, then use something like
probably the cut the image down considerably. (This might already be
possible, need to poke at whether you can late-bind interpreters
later.)
2015-11-13 12:02:11 -08:00
Zach Loafman de92fb698e Remove unneeded pieces 2015-11-05 14:58:18 -08:00
Zach Loafman 436839f196 Spark: Don't daemonize, don't pretend to tail logs we don't use
Along the way: Fixup port mismatch in worker container
2015-10-30 11:52:17 -07:00
Zach Loafman d24d2a21f9 Spark: Add liveness probes for master/worker
It turns out, the pods don't exit when the master or worker crash.

Along the way: remove redundant metadata.
2015-10-29 05:59:31 -07:00
Zach Loafman 413ec6c83a Spark: Update example to use kubectl proxy rather than load balancer 2015-10-29 05:59:31 -07:00
Zach Loafman 777d3a6ec0 Spark: Update to current example standards, add GCS connector
* Pod -> ReplicationController, which also forced me to hack around
hostname issue on the master. (Spark master sees the incoming slave
request to spark-master and assumes it's not meant for it, since it's
name is spark-master-controller-abcdef.)

* Remove service env dependencies (depend on DNS instead).

* JSON -> YAML.

* Add GCS connector.

* Make example do something actually useful: A familiar example to
anyone at Google, implement wordcount of all of Shakespeare's works.

* Fix a minor service connection issue in the gluster example.
2015-10-27 15:06:34 -07:00
Zach Loafman c0fba82da4 Spark: Bump images to 1.5.1, add Makefile 2015-10-24 12:44:12 -07:00
Jon Cope b6d3b2f64b Add Spark-GlusterFS example 2015-10-23 09:01:56 -05:00
Marcin Wielgus 1a1f751197 Add image version to examples/spark 2015-08-14 19:52:26 +02:00
Marcin Wielgus 41bc20d1e0 Fix spark-driver in examples/spark 2015-08-05 15:45:43 +02:00
Gurvinder Singh f64d89fd1d fixed documentations after running gendocs.sh 2015-07-29 22:21:10 +02:00
Gurvinder Singh 5599d83685 added standard copyright 2015-07-27 23:01:49 +02:00
Gurvinder Singh 5f48898498 added indent for sleep 2015-07-25 21:16:06 +02:00
Gurvinder Singh 2a6546cada fixed conflict with the current upstream master 2015-07-25 21:15:34 +02:00
Tim Hockin 995a7aef29 Collected markedown fixes around syntax. 2015-07-20 09:51:22 -07:00
Tim Hockin 33f1862830 Run gendocs 2015-07-17 15:35:43 -07:00
Daniel Smith f7873d2a1f apply changes 2015-07-17 11:04:46 -07:00
Tim Hockin 542e13d2d3 Better scary message 2015-07-17 09:28:49 -07:00
Tim Hockin 39b86908a1 Run gendocs 2015-07-14 17:28:47 -07:00
Tim Hockin 79f9cd7c83 Run gendocs 2015-07-14 15:28:59 -07:00
Mike Danese 3eff8fce41 automated link fixes 2015-07-14 10:51:10 -07:00
Daniel Smith 1263a30644 fix broken example links 2015-07-13 17:47:48 -07:00
Chao Xu f8047aa635 Update output of kubectl in examples 2015-07-10 12:48:58 -07:00
Gurvinder Singh a046fa478d modified spark example to use kubectl exec to interact with cluster and create spark driver pod 2015-07-10 11:19:55 +02:00
Marcin Wielgus bfc4ee789d Updated spark examples with docker images moved to gcr.io/google_containers 2015-07-07 19:36:32 +02:00
Kris Rousey 381093d585 Change spark example to use v1 2015-06-10 13:57:17 -07:00
RichieEscarez 7936334a78 Fixes #8701. Added inline links to "services" "pods" and "replication controllers" (using relative linking ../../folder/filename.md) 2015-06-04 09:24:50 -07:00
Daniel Smith 5d6b8d7de6 doc updates 2015-06-03 15:41:09 -07:00
Paul Morie 95806ea68c Make references to files in examples links 2015-05-24 03:22:26 -04:00
Jeff Lowdermilk 553f9f822b Add ga-beacon analytics to gendocs scripts
hack/run-gendocs.sh puts ga-beacon analytics link into all md files,
hack/verify-gendocs.sh verifies presence of link.
2015-05-15 18:56:38 -07:00
BC Broussard 709c00c63d Update examples/spark for v1beta3 2015-04-23 18:23:13 -07:00
Matthew Farrellee 31b923c987 add tl;dr version of Spark README.md
mention the spark cluster is standalone

add detailed master & worker instructions

add method to get master status

add links option for master status

add links option for worker status

add example use of cluster

add source location
2015-03-27 16:46:33 -04:00