Commit Graph

54 Commits (1b6e8f97d91a4a128753755b514829ed5ac040e8)

Author SHA1 Message Date
Adrian Ursu ca24ee02aa Update README.md 2016-05-16 14:32:33 +01:00
Adrian Ursu a7b2c5783c Fixed namespace name to spark-cluster 2016-05-16 14:06:07 +01:00
Klaus Ma 326e30772a Added namespace to Spark example. 2016-04-19 00:34:47 +08:00
k8s-merge-robot c9250bba2a Merge pull request #21899 from zmerlynn/update-spark
Auto commit by PR queue bot
2016-03-12 07:25:17 -08:00
David McMahon dae84f5306 Update the latestReleaseBranch to release-1.2 in the munger. 2016-03-08 18:59:28 -08:00
Zach Loafman 6e2e7bd3b6 Bump Spark / Zeppelin to 1.5.2 / 0.5.6 tags
From https://github.com/kubernetes/application-images/pull/5
2016-03-04 13:26:05 -08:00
Tim St. Clair 198c120036 Detect & fix backwards markdown links (of the form (foo)[bar]) 2016-02-26 13:24:21 -08:00
Alex Robinson c031697848 Merge pull request #21626 from jayunit100/examples-spark-fix
Fix broken spark tests to use yaml files, 1 worker, declare DNS depe…
2016-02-24 16:30:19 -08:00
jay vyas 782b268f02 [examples] [e2e] Fix spark example e2e tests: labels, forEach, json->yaml 2016-02-22 16:40:30 -05:00
k8s-merge-robot 60c6edaee7 Merge pull request #21634 from jayunit100/spark-README
Auto commit by PR queue bot
2016-02-20 23:59:59 -08:00
jay vyas 6cbcb5d6c7 Update spark README with container location and DNS req. 2016-02-20 17:33:30 -05:00
Matthew Farrellee 0336220bf1 update spark example yaml to use new images 2016-02-09 17:18:47 -05:00
Matthew Farrellee 0294bc2be7 move spark images to github.com/kubernetes/application-images/spark 2016-01-16 11:15:23 -05:00
Chao Xu 2b2f285777 run hack/update-generated-docs.sh 2015-12-21 10:41:30 -08:00
Matthew Farrellee 314e0c5512 Eliminate need for wget (use curl), use fewer intermediate files be quiet 2015-11-20 18:54:13 -05:00
Matthew Farrellee eff6169ac7 Change default to local build, require explicit "make push" to push 2015-11-20 16:58:59 -05:00
Matthew Farrellee fb4afda2bf Install numpy on all spark images 2015-11-20 16:05:47 -05:00
Matthew Farrellee 2d37029a39 Use upstream zeppelin 0.5.5 release 2015-11-20 14:22:56 -05:00
Matthew Farrellee 4e181c4c1f Add cleaning to clean target 2015-11-20 10:19:57 -05:00
Zach Loafman f8b83a65da Zeppelin: Compress the image by doing the build on a separate image
Pull the tar through to the image we need it on.

Addresses #17231
2015-11-17 14:51:16 -08:00
k8s-merge-robot c377b32b3f Merge pull request #17025 from zmerlynn/zeppelin
Auto commit by PR queue bot
2015-11-13 16:12:54 -08:00
Tim Hockin dab591b629 Run update-gendocs 2015-11-13 13:57:57 -08:00
Zach Loafman 51817850ba Zeppelin: Add Zeppelin image to Spark example
This adds a very basic Zeppelin image that works with the existing
Spark example. As can be seen from the documentation, it has a couple
of warts:

* It requires kubectl port-forward (which is unstable across long
periods of time, at least for me, on this app, bug incoming). See

* I needed to roll my own container (none of the existing containers
exactly matched needs, or even built anymore against modern Zeppelin
master, and the rest of the example is Spark 1.5).

The image itself is *huge*. One of the further refinements we need to
look at is how to possibly strip the Maven build for this container
down to just the interpreters we care about, because the deps here
are frankly ridiculous.

This might be a case where, if possible, we might want to open an
upstream request to build things dynamically, then use something like
probably the cut the image down considerably. (This might already be
possible, need to poke at whether you can late-bind interpreters
later.)
2015-11-13 12:02:11 -08:00
Zach Loafman de92fb698e Remove unneeded pieces 2015-11-05 14:58:18 -08:00
Zach Loafman 436839f196 Spark: Don't daemonize, don't pretend to tail logs we don't use
Along the way: Fixup port mismatch in worker container
2015-10-30 11:52:17 -07:00
Zach Loafman d24d2a21f9 Spark: Add liveness probes for master/worker
It turns out, the pods don't exit when the master or worker crash.

Along the way: remove redundant metadata.
2015-10-29 05:59:31 -07:00
Zach Loafman 413ec6c83a Spark: Update example to use kubectl proxy rather than load balancer 2015-10-29 05:59:31 -07:00
Zach Loafman 777d3a6ec0 Spark: Update to current example standards, add GCS connector
* Pod -> ReplicationController, which also forced me to hack around
hostname issue on the master. (Spark master sees the incoming slave
request to spark-master and assumes it's not meant for it, since it's
name is spark-master-controller-abcdef.)

* Remove service env dependencies (depend on DNS instead).

* JSON -> YAML.

* Add GCS connector.

* Make example do something actually useful: A familiar example to
anyone at Google, implement wordcount of all of Shakespeare's works.

* Fix a minor service connection issue in the gluster example.
2015-10-27 15:06:34 -07:00
Zach Loafman c0fba82da4 Spark: Bump images to 1.5.1, add Makefile 2015-10-24 12:44:12 -07:00
Jon Cope b6d3b2f64b Add Spark-GlusterFS example 2015-10-23 09:01:56 -05:00
Marcin Wielgus 1a1f751197 Add image version to examples/spark 2015-08-14 19:52:26 +02:00
Marcin Wielgus 41bc20d1e0 Fix spark-driver in examples/spark 2015-08-05 15:45:43 +02:00
Gurvinder Singh f64d89fd1d fixed documentations after running gendocs.sh 2015-07-29 22:21:10 +02:00
Gurvinder Singh 5599d83685 added standard copyright 2015-07-27 23:01:49 +02:00
Gurvinder Singh 5f48898498 added indent for sleep 2015-07-25 21:16:06 +02:00
Gurvinder Singh 2a6546cada fixed conflict with the current upstream master 2015-07-25 21:15:34 +02:00
Tim Hockin 995a7aef29 Collected markedown fixes around syntax. 2015-07-20 09:51:22 -07:00
Tim Hockin 33f1862830 Run gendocs 2015-07-17 15:35:43 -07:00
Daniel Smith f7873d2a1f apply changes 2015-07-17 11:04:46 -07:00
Tim Hockin 542e13d2d3 Better scary message 2015-07-17 09:28:49 -07:00
Tim Hockin 39b86908a1 Run gendocs 2015-07-14 17:28:47 -07:00
Tim Hockin 79f9cd7c83 Run gendocs 2015-07-14 15:28:59 -07:00
Mike Danese 3eff8fce41 automated link fixes 2015-07-14 10:51:10 -07:00
Daniel Smith 1263a30644 fix broken example links 2015-07-13 17:47:48 -07:00
Chao Xu f8047aa635 Update output of kubectl in examples 2015-07-10 12:48:58 -07:00
Gurvinder Singh a046fa478d modified spark example to use kubectl exec to interact with cluster and create spark driver pod 2015-07-10 11:19:55 +02:00
Marcin Wielgus bfc4ee789d Updated spark examples with docker images moved to gcr.io/google_containers 2015-07-07 19:36:32 +02:00
Kris Rousey 381093d585 Change spark example to use v1 2015-06-10 13:57:17 -07:00
RichieEscarez 7936334a78 Fixes #8701. Added inline links to "services" "pods" and "replication controllers" (using relative linking ../../folder/filename.md) 2015-06-04 09:24:50 -07:00
Daniel Smith 5d6b8d7de6 doc updates 2015-06-03 15:41:09 -07:00