mirror of https://github.com/k3s-io/k3s
Merge pull request #15207 from elsonrodriguez/selenium-example-pr
Adds example for running Selenium on Kubernetes.pull/6/head
Dawn Chen
commit
e2659fc645
|
|
@ -0,0 +1,232 @@
|
|||
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
|
||||
|
||||
<!-- BEGIN STRIP_FOR_RELEASE -->
|
||||
|
||||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
|
||||
width="25" height="25">
|
||||
|
||||
<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>
|
||||
|
||||
If you are using a released version of Kubernetes, you should
|
||||
refer to the docs that go with that version.
|
||||
|
||||
<strong>
|
||||
The latest 1.0.x release of this document can be found
|
||||
[here](http://releases.k8s.io/release-1.0/examples/selenium/README.md).
|
||||
|
||||
Documentation for other releases can be found at
|
||||
[releases.k8s.io](http://releases.k8s.io).
|
||||
</strong>
|
||||
--
|
||||
|
||||
<!-- END STRIP_FOR_RELEASE -->
|
||||
|
||||
<!-- END MUNGE: UNVERSIONED_WARNING -->
|
||||
|
||||
## Selenium on Kubernetes
|
||||
|
||||
Selenium is a browser automation tool used primarily for testing web applications. However when Selenium is used in a CI pipeline to test applications, there is often contention around the use of Selenium resources. This example shows you how to deploy Selenium to Kubernetes in a scalable fashion.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
This example assumes you have a working Kubernetes cluster and a properly configured kubectl client. See the [Getting Started Guides](../../docs/getting-started-guides/) for details.
|
||||
|
||||
Google Container Engine is also a quick way to get Kubernetes up and running: https://cloud.google.com/container-engine/
|
||||
|
||||
Your cluster must have 4 CPU and 6 GB of RAM to complete the example up to the scaling portion.
|
||||
|
||||
### Deploy Selenium Grid Hub:
|
||||
|
||||
We will be using Selenium Grid Hub to make our Selenium install scalable via a master/worker model. The Selenium Hub is the master, and the Selenium Nodes are the workers(not to be confused with Kubernetes nodes). We only need one hub, but we're using a replication controller to ensure that the hub is always running:
|
||||
|
||||
```console
|
||||
kubectl create --filename=examples/selenium/selenium-hub-rc.yaml
|
||||
```
|
||||
|
||||
The Selenium Nodes will need to know how to get to the Hub, let's create a service for the nodes to connect to.
|
||||
|
||||
```console
|
||||
kubectl create --filename=examples/selenium/selenium-hub-svc.yaml
|
||||
```
|
||||
|
||||
### Verify Selenium Hub Deployment
|
||||
|
||||
Let's verify our deployment of Selenium hub by connecting to the web console.
|
||||
|
||||
#### Kubernetes Nodes Reachable
|
||||
|
||||
If your Kubernetes nodes are reachable from your network, you can verify the hub by hitting it on the nodeport. You can retrieve the nodeport by typing `kubectl describe svc selenium-hub`, however the snippet below automates that by using kubectl's template functionality:
|
||||
|
||||
```console
|
||||
export NODEPORT=`kubectl get svc --selector='app=selenium-hub' --output=template --template="{{ with index .items 0}}{{with index .spec.ports 0 }}{{.nodePort}}{{end}}{{end}}"`
|
||||
export NODE=`kubectl get nodes --output=template --template="{{with index .items 0 }}{{.metadata.name}}{{end}}"`
|
||||
|
||||
curl http://$NODE:$NODEPORT
|
||||
```
|
||||
|
||||
#### Kubernetes Nodes Unreachable
|
||||
|
||||
If you cannot reach your Kubernetes nodes from your network, you can proxy via kubectl.
|
||||
|
||||
```console
|
||||
export PODNAME=`kubectl get pods --selector="app=selenium-hub" --output=template --template="{{with index .items 0}}{{.metadata.name}}{{end}}"`
|
||||
kubectl port-forward --pod=$PODNAME 4444:4444
|
||||
```
|
||||
|
||||
In a seperate terminal, you can now check the status.
|
||||
|
||||
```console
|
||||
curl http://localhost:4444
|
||||
```
|
||||
|
||||
#### Using Google Container Engine
|
||||
|
||||
If you are using Google Container Engine, you can expose your hub via the internet. This is a bad idea for many reasons, but you can do it as follows:
|
||||
|
||||
```console
|
||||
kubectl expose rc selenium-hub --name=selenium-hub-external --labels="app=selenium-hub,external=true" --create-external-load-balancer=true
|
||||
```
|
||||
|
||||
Then wait a few minutes, eventually your new `selenium-hub-external` service will be assigned a load balanced IP from gcloud. Once `kubectl get svc selenium-hub-external` shows two IPs, run this snippet.
|
||||
|
||||
```console
|
||||
export INTERNET_IP=`kubectl get svc --selector="app=selenium-hub,external=true" --output=template --template="{{with index .items 0}}{{with index .status.loadBalancer.ingress 0}}{{.ip}}{{end}}{{end}}"`
|
||||
|
||||
curl http://$INTERNET_IP:4444/
|
||||
```
|
||||
|
||||
You should now be able to hit `$INTERNET_IP` via your web browser, and so can everyone else on the Internet!
|
||||
|
||||
### Deploy Firefox and Chrome Nodes:
|
||||
|
||||
Now that the Hub is up, we can deploy workers.
|
||||
|
||||
This will deploy 2 Chrome nodes.
|
||||
|
||||
```console
|
||||
kubectl create --file=examples/selenium/selenium-node-chrome-rc.yaml
|
||||
```
|
||||
|
||||
And 2 Firefox nodes to match.
|
||||
|
||||
```console
|
||||
kubectl create --file=examples/selenium/selenium-node-firefox-rc.yaml
|
||||
```
|
||||
|
||||
Once the pods start, you will see them show up in the Selenium Hub interface.
|
||||
|
||||
### Run a Selenium Job
|
||||
|
||||
Let's run a quick Selenium job to validate our setup.
|
||||
|
||||
#### Setup Python Environment
|
||||
|
||||
First, we need to start a python container that we can attach to.
|
||||
|
||||
```console
|
||||
kubectl run selenium-python --image=google/python-hello
|
||||
```
|
||||
|
||||
Next, we need to get inside this container.
|
||||
|
||||
```console
|
||||
export PODNAME=`kubectl get pods --selector="run=selenium-python" --output=template --template="{{with index .items 0}}{{.metadata.name}}{{end}}"`
|
||||
kubectl exec --stdin=true --tty=true $PODNAME bash
|
||||
```
|
||||
|
||||
Once inside, we need to install the Selenium library
|
||||
|
||||
```console
|
||||
pip install selenium
|
||||
```
|
||||
|
||||
#### Run Selenium Job with Python
|
||||
|
||||
We're all set up, start the python interpreter.
|
||||
|
||||
```console
|
||||
python
|
||||
```
|
||||
|
||||
And paste in the contents of selenium-test.py.
|
||||
|
||||
```python
|
||||
from selenium import webdriver
|
||||
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
|
||||
|
||||
def check_browser(browser):
|
||||
driver = webdriver.Remote(
|
||||
command_executor='http://selenium-hub:4444/wd/hub',
|
||||
desired_capabilities=getattr(DesiredCapabilities, browser)
|
||||
)
|
||||
driver.get("http://google.com")
|
||||
assert "google" in driver.page_source
|
||||
driver.close()
|
||||
print("Browser %s checks out!" % browser)
|
||||
|
||||
|
||||
check_browser("FIREFOX")
|
||||
check_browser("CHROME")
|
||||
```
|
||||
|
||||
You should get
|
||||
|
||||
```
|
||||
>>> check_browser("FIREFOX")
|
||||
Browser FIREFOX checks out!
|
||||
>>> check_browser("CHROME")
|
||||
Browser CHROME checks out!
|
||||
```
|
||||
|
||||
Congratulations, your Selenium Hub is up, with Firefox and Chrome nodes!
|
||||
|
||||
### Scale your Firefox and Chrome nodes.
|
||||
|
||||
If you need more Firefox or Chrome nodes, your hardware is the limit:
|
||||
|
||||
```console
|
||||
kubectl scale rc selenium-node-firefox --replicas=10
|
||||
kubectl scale rc selenium-node-chrome --replicas=10
|
||||
```
|
||||
|
||||
You now have 10 Firefox and 10 Chrome nodes, happy Seleniuming!
|
||||
|
||||
### Debugging
|
||||
|
||||
Sometimes it is neccessary to check on a hung test. Each pod is running VNC. To check on one of the browser nodes via VNC, it's reccomended that you proxy, since we don't want to expose a service for every pod, and the containers have a weak VNC password. Replace POD_NAME with the name of the pod you want to connect to.
|
||||
|
||||
```console
|
||||
kubectl port-forward --pod=POD_NAME 5900:5900
|
||||
```
|
||||
|
||||
Then connect to localhost:5900 with your VNC client using the password "secret"
|
||||
|
||||
Enjoy your scalable Selenium Grid!
|
||||
|
||||
Adapted from: https://github.com/SeleniumHQ/docker-selenium
|
||||
|
||||
### Teardown
|
||||
|
||||
To remove all created resources, run the following:
|
||||
|
||||
```console
|
||||
kubectl delete rc selenium-hub
|
||||
kubectl delete rc selenium-node-chrome
|
||||
kubectl delete rc selenium-node-firefox
|
||||
kubectl delete rc selenium-python
|
||||
kubectl delete svc selenium-hub
|
||||
kubectl delete svc selenium-hub-external
|
||||
```
|
||||
|
||||
|
||||
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
||||
[]()
|
||||
<!-- END MUNGE: GENERATED_ANALYTICS -->
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
apiVersion: v1
|
||||
kind: ReplicationController
|
||||
metadata:
|
||||
name: selenium-hub
|
||||
labels:
|
||||
app: selenium-hub
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
app: selenium-hub
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: selenium-hub
|
||||
spec:
|
||||
containers:
|
||||
- name: selenium-hub
|
||||
image: selenium/hub:2.47.1
|
||||
ports:
|
||||
- containerPort: 4444
|
||||
resources:
|
||||
limits:
|
||||
memory: "1000Mi"
|
||||
cpu: ".5"
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /grid/console
|
||||
port: 4444
|
||||
initialDelaySeconds: 30
|
||||
timeoutSeconds: 5
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /grid/console
|
||||
port: 4444
|
||||
initialDelaySeconds: 30
|
||||
timeoutSeconds: 5
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: selenium-hub
|
||||
labels:
|
||||
app: selenium-hub
|
||||
spec:
|
||||
ports:
|
||||
- port: 4444
|
||||
targetPort: 4444
|
||||
name: port0
|
||||
selector:
|
||||
app: selenium-hub
|
||||
type: NodePort
|
||||
sessionAffinity: None
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
apiVersion: v1
|
||||
kind: ReplicationController
|
||||
metadata:
|
||||
name: selenium-node-chrome
|
||||
labels:
|
||||
app: selenium-node-chrome
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
app: selenium-node-chrome
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: selenium-node-chrome
|
||||
spec:
|
||||
containers:
|
||||
- name: selenium-node-chrome
|
||||
image: selenium/node-chrome-debug:2.47.1
|
||||
ports:
|
||||
- containerPort: 5900
|
||||
env:
|
||||
- name: HUB_PORT_4444_TCP_ADDR
|
||||
value: "selenium-hub"
|
||||
- name: HUB_PORT_4444_TCP_PORT
|
||||
value: "4444"
|
||||
resources:
|
||||
limits:
|
||||
memory: "1000Mi"
|
||||
cpu: ".5"
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
apiVersion: v1
|
||||
kind: ReplicationController
|
||||
metadata:
|
||||
name: selenium-node-firefox
|
||||
labels:
|
||||
app: selenium-node-firefox
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
app: selenium-node-firefox
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: selenium-node-firefox
|
||||
spec:
|
||||
containers:
|
||||
- name: selenium-node-firefox
|
||||
image: selenium/node-firefox-debug:2.47.1
|
||||
ports:
|
||||
- containerPort: 5900
|
||||
env:
|
||||
- name: HUB_PORT_4444_TCP_ADDR
|
||||
value: "selenium-hub"
|
||||
- name: HUB_PORT_4444_TCP_PORT
|
||||
value: "4444"
|
||||
resources:
|
||||
limits:
|
||||
memory: "1000Mi"
|
||||
cpu: ".5"
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
#!/usr/bin/env python
|
||||
|
||||
# Copyright 2015 The Kubernetes Authors All rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from selenium import webdriver
|
||||
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
|
||||
|
||||
def check_browser(browser):
|
||||
driver = webdriver.Remote(
|
||||
command_executor='http://selenium-hub:4444/wd/hub',
|
||||
desired_capabilities=getattr(DesiredCapabilities, browser)
|
||||
)
|
||||
driver.get("http://google.com")
|
||||
assert "google" in driver.page_source
|
||||
driver.close()
|
||||
print("Browser %s checks out!" % browser)
|
||||
|
||||
|
||||
check_browser("FIREFOX")
|
||||
check_browser("CHROME")
|
||||
|
||||
Loading…
Reference in New Issue