github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
k3s/examples/spark/spark-worker-controller.yaml

24 lines
473 B
YAML
Raw Normal View History

Spark: Update to current example standards, add GCS connector * Pod -> ReplicationController, which also forced me to hack around hostname issue on the master. (Spark master sees the incoming slave request to spark-master and assumes it's not meant for it, since it's name is spark-master-controller-abcdef.) * Remove service env dependencies (depend on DNS instead). * JSON -> YAML. * Add GCS connector. * Make example do something actually useful: A familiar example to anyone at Google, implement wordcount of all of Shakespeare's works. * Fix a minor service connection issue in the gluster example.
2015-10-26 20:42:02 +00:00
kind: ReplicationController
apiVersion: v1
metadata:
name: spark-worker-controller
spec:
[examples] [e2e] Fix spark example e2e tests: labels, forEach, json->yaml
2016-02-20 22:31:13 +00:00
replicas: 2
Spark: Update to current example standards, add GCS connector * Pod -> ReplicationController, which also forced me to hack around hostname issue on the master. (Spark master sees the incoming slave request to spark-master and assumes it's not meant for it, since it's name is spark-master-controller-abcdef.) * Remove service env dependencies (depend on DNS instead). * JSON -> YAML. * Add GCS connector. * Make example do something actually useful: A familiar example to anyone at Google, implement wordcount of all of Shakespeare's works. * Fix a minor service connection issue in the gluster example.
2015-10-26 20:42:02 +00:00
selector:
component: spark-worker
template:
metadata:
labels:
component: spark-worker
spec:
containers:
- name: spark-worker
Switch to k8s.gcr.io vanity domain This is the 2nd attempt. The previous was reverted while we figured out the regional mirrors (oops). New plan: k8s.gcr.io is a read-only facade that auto-detects your source region (us, eu, or asia for now) and pulls from the closest. To publish an image, push k8s-staging.gcr.io and it will be synced to the regionals automatically (similar to today). For now the staging is an alias to gcr.io/google_containers (the legacy URL). When we move off of google-owned projects (working on it), then we just do a one-time sync, and change the google-internal config, and nobody outside should notice. We can, in parallel, change the auto-sync into a manual sync - send a PR to "promote" something from staging, and a bot activates it. Nice and visible, easy to keep track of.
2018-01-17 19:36:53 +00:00
image: k8s.gcr.io/spark:1.5.2_v1
update spark example yaml to use new images
2016-02-09 22:18:47 +00:00
command: ["/start-worker"]
Spark: Update to current example standards, add GCS connector * Pod -> ReplicationController, which also forced me to hack around hostname issue on the master. (Spark master sees the incoming slave request to spark-master and assumes it's not meant for it, since it's name is spark-master-controller-abcdef.) * Remove service env dependencies (depend on DNS instead). * JSON -> YAML. * Add GCS connector. * Make example do something actually useful: A familiar example to anyone at Google, implement wordcount of all of Shakespeare's works. * Fix a minor service connection issue in the gluster example.
2015-10-26 20:42:02 +00:00
ports:
Spark: Don't daemonize, don't pretend to tail logs we don't use Along the way: Fixup port mismatch in worker container
2015-10-30 18:37:29 +00:00
- containerPort: 8081
Spark: Update to current example standards, add GCS connector * Pod -> ReplicationController, which also forced me to hack around hostname issue on the master. (Spark master sees the incoming slave request to spark-master and assumes it's not meant for it, since it's name is spark-master-controller-abcdef.) * Remove service env dependencies (depend on DNS instead). * JSON -> YAML. * Add GCS connector. * Make example do something actually useful: A familiar example to anyone at Google, implement wordcount of all of Shakespeare's works. * Fix a minor service connection issue in the gluster example.
2015-10-26 20:42:02 +00:00
resources:
requests:
cpu: 100m
[examples] [e2e] Fix spark example e2e tests: labels, forEach, json->yaml
2016-02-20 22:31:13 +00:00