Automatic merge from submit-queue
Delete the broken Celery+RabbitMQ example
The celery container used in the example is broken and does not come up
on most distros. The e2e test that was validating this example was not
detecting the fact the celery pod was crash looping.
I attempted to fix the celery container, but it proved to be tedious.
The proposed fix is to update the glibc version to >= 2.23. In this case
it requires updating the python docker image and the celery base image.
https://github.com/kubernetes/kubernetes/issues/31456
has more details.
I'm deleting the example instead of marking it as broken because a user
might overlook the broken warning and it should be trivial to revert
this PR if someone can fix the celery container.
Most of the contents of docs/ has moved to kubernetes.github.io.
Development of the docs and accompanying files has continued there, making
the copies in this repo stale. I've removed everything but the .md files
which remain to redirect old links. The .yaml config files in the docs
were used by some tests, these have been moved to test/fixtures/doc-yaml,
and can remain there to be used by tests or other purposes.
Before this change this example would result in:
$ kubectl create -f sysdig.yml
The DaemonSet "sysdig-agent" is invalid.
spec.template.metadata.labels: Invalid value: null: `selector` does not match template `labels`
Includes:
* A service for clients to use
* Readiness/liveness probes
* An extended graceful termination period
* Easy clean-up of all created resources
Automatic merge from submit-queue
Add encryption to EBS dynamic provisioner
Resolves https://github.com/kubernetes/kubernetes/issues/30792
Adds encryption to the EBS cloud provider and provisioner.
Follow up to #29006 (all commits but the one in this PR will drop out).
@kubernetes/sig-storage
```release-note
```
Automatic merge from submit-queue
change all PredicateFunc to use SelectionPredicate
What?
- This PR changes all PredicateFunc in registry to return SelectionPredicate instead of Matcher interface.
Why?
- We want to pass SelectionPredicate to storage layer. Matcher interface did not expose enough information for indexing.
Automatic merge from submit-queue
Add a CockroachDB PetSet example
The example starts a simple five-node cluster with otherwise
default settings (in particular, 3x replication).
cc @bprashanth
Automatic merge from submit-queue
Quobyte Volume plugin
@quofelix and myself developed a volume plugin for [Quobyte](http://www.quobyte.com) which is a software-defined storage solution. This PR allows Kubernetes users to mount a Quobyte Volume inside their containers over Kubernetes.
Here are some further informations about [Quobyte and Storage for containers](http://www.quobyte.com/containers)
Automatic merge from submit-queue
Fix resource requests in cassandra-daemonset.yaml
The current file fails validation because the resource request syntax is incorrect.
1. Use --client since -c is deprecated now
2. The command (./kubectl version --client | grep -o 'GitVersion:"[^"]*"')
now returns:
GitVersion:"v1.4.0-alpha.1.784+ed3a29bd6aeb98-dirty"
so parse out the version better using sed
Related to #23708
The README for persistent volume provisioning had information about the
hostpath provisioner for testing purposes, but lacked instructions on
how to enable it.
Automatic merge from submit-queue
Remove id field from rbd examples
Currently the RBD examples fail with the following error:
```
kubectl create -f examples/volumes/rbd/rbd-with-secret.json
error validating "examples/volumes/rbd/rbd-with-secret.json": error validating data: found invalid field id for v1.Pod; if you choose to ignore these errors, turn validation off with --validate=false
```
fixes#28831
HostPath volumes are not automatically provisioned unless we pass in `--enable-hostpath-provisioner` to the controller manager. This was not mentioned in the docs previously.
Automatic merge from submit-queue
Implement first version of indexing in cacher
This PR adds a first version of indexing in cacher.
It has a really significant impact on performance - __in empty 2000-node cluster, apiserver cpu usage drops by ~75%.__
Not for 1.3, but we need this soon after 1.3 is done.
@lavalamp @mqliang @davidopp @gmarek @kubernetes/sig-scalability
This allows us to start building real dependencies into Makefile.
Leave old hack/* scripts in place but advise to use 'make'. There are a few
rules that call things like 'go run' or 'build/*' that I left as-is for now.
id isn't a valid field in a Pod object so kubectl fails with:
error validating "glusterfs-pod.json": error validating data: found invalid field id for v1.Pod; if you choose to ignore these errors, turn validation off with --validate=false
Search and replace for references to moved examples
Reverted find and replace paths on auto gen docs
Reverting changes to changelog
Fix bugs in test-cmd.sh
Fixed path in examples README
ran update-all successfully
Updated verify-flags exceptions to include renamed files
Automatic merge from submit-queue
Fixed an error in lvm bash script
The space in this script is an obvious typo and caused an error
(trying to call volume_id as command).
Unified skydns templates using a simple underscore based template and
added transform sed scripts to transform into salt and sed yaml
templates
Moved all content out of cluster/addons/dns into build/kube-dns and
saltbase/salt/kube-dns
Automatic merge from submit-queue
fixing docker file, and bumping cassandra version
- had issue with current v9 build. `cassandra.yaml` was not editable by the cassandra user, chown and chmod is now correct
- rebuild jar against 3.5 cassandra version and bumped C* version
Automatic merge from submit-queue
Use pause image depending on the server's platform when testing
Removed all pause image constant strings, now the pause image is chosen by arch. Part of the effort of making e2e arch-agnostic.
The pause image name and version is also now only in two places, and it's documented to bump both
Also removed "amd64" constants in the code. Such constants should be replaced by `runtime.GOARCH` or by looking up the server platform
Fixes: #22876 and #15140
Makes it easier for: #25730
Related: #17981
This is for `v1.3`
@ixdy @thockin @vishh @kubernetes/sig-testing @andyzheng0831 @pensu
Automatic merge from submit-queue
Cassandra examples updating images to v9
- this is a version bump for the C* image
- I also increased the cpu to .5 because .1 is slow like warm death
Who can actually run the build to get the container to the examples repo?
Automatic merge from submit-queue
kube-apiserver options should be decoupled from impls
A few months ago we refactored options to keep it independent of the
implementations, so that it could be used in CLI tools to validate
config or to generate config, without pulling in the full dependency
tree of the master. This change restores that by separating
server_run_options.go back to its own package.
Also, options structs should never contain non-serializable types, which
storagebackend.Config was doing with runtime.Codec. Split the codec out.
Fix a typo on the name of the etcd2.go storage backend.
Finally, move DefaultStorageMediaType to server_run_options.
@nikhiljindal as per my comment in #24454, @liggitt because you and I
discussed this last time
A few months ago we refactored options to keep it independent of the
implementations, so that it could be used in CLI tools to validate
config or to generate config, without pulling in the full dependency
tree of the master. This change restores that by separating
server_run_options.go back to its own package.
Also, options structs should never contain non-serializable types, which
storagebackend.Config was doing with runtime.Codec. Split the codec out.
Fix a typo on the name of the etcd2.go storage backend.
Finally, move DefaultStorageMediaType to server_run_options.
Automatic merge from submit-queue
Newrelic daemonset
1. base64 on Mac does not support the **wrap** option. Easy to support on both Mac and Linux by using **tr** to remove the newline.
2. DaemonSet definition does not conform to latest schema:
> $ kubectl create -f ./newrelic-daemonset.yaml
error validating "./newrelic-daemonset.yaml": error validating data: found invalid field privileged for v1.PodSecurityContext; if you choose to ignore these errors, turn validation off with --validate=false
<!-- Reviewable:start -->
---
This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24564)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Fixed namespace name to spark-cluster
Just changed the namespace from **default** to **spark-cluster** in the spark example docs.
The guestbook-go example is broken because the latest tag of redis has
moved to redis 3.0 which speaks a new protocol. This means that the
slaves, which have fixed 2.0 versions, will error out on the protocol:
```
[7] 15 May 23:37:44.403 # Can't handle RDB format version 7
[7] 15 May 23:37:44.403 # Failed trying to load the MASTER synchronization DB from disk
[7] 15 May 23:37:45.333 * Connecting to MASTER redis-master:6379
[7] 15 May 23:37:45.427 * MASTER <-> SLAVE sync started
```
In this case the app simply never persists data.
cc @luebken @Gurpartap
The codec factory should support two distinct interfaces - negotiating
for a serializer with a client, vs reading or writing data to a storage
form (etcd, disk, etc). Make the EncodeForVersion and DecodeToVersion
methods only take Encoder and Decoder, and slight refactoring elsewhere.
In the storage factory, use a content type to control what serializer to
pick, and use the universal deserializer. This ensures that storage can
read JSON (which might be from older objects) while only writing
protobuf. Add exceptions for those resources that may not be able to
write to protobuf (specifically third party resources, but potentially
others in the future).
Automatic merge from submit-queue
Deleting duplicate code from federated-apiserver.Run()
This removes most of duplicate code from federated-apiserver.Run().
The code remaining is related to storage or authz and authn.
https://github.com/kubernetes/kubernetes/pull/24787 refactors the storage related code.
I am still figuring out authz and authn.
cc @jianhuiz
Automatic merge from submit-queue
Refactored SeedProvider and Updated Docker
This is a redo of the last PR that I munged 😄
- fixed maven build folder structure
- updated build to C* 3.4
- refactored Seed Provider - improved error handling, updated default SeedProvider code
- added start of unit tests. Not as comprehensive as I would like
- updated docker image to debian:jessie
- installed openjdk 8
- added some docker fu to make the image smaller
- updated docker to C* 3.4 and update yaml
- updated README content. Added a section about the docker, and the SeedProvider
Have not had a chance to test the docker on k8s, because I do not have a local docker repo.
NOTE: someone needs to push the docker image into the google repo. Not sure what the process is ... I will submit another PR request with changes to the yaml files.
Automatic merge from submit-queue
Move internal types of job from pkg/apis/extensions to pkg/apis/batch
This addressed the job part of #23216, this is still WIP. Will notify once finished. I'd like to have it in before starting working on ScheduledJob.
@lavalamp @erictune fyi
Automatic merge from submit-queue
Intial draft on SeedProvider docs
Alsa more documentation. We need to reference the config section in the example docs. There are multiple PRs open in those docs, so at this point I do not want to make a mess.
Let me know if there are docs standard template that will make this more pretty.
Automatic merge from submit-queue
update gb-frontend image. New image includes the change in PR # 23381.
Update to use the gcr.io/google-samples/gb-frontend:v4 image. New image includes the change in https://github.com/kubernetes/kubernetes/pull/23381.
Add tests to watch behavior in both protocols (http and websocket)
against all 3 media types. Adopt the
`application/vnd.kubernetes.protobuf;stream=watch` media type for the
content that comes back from a watch call so that it can be
distinguished from a Status result.
Automatic merge from submit-queue
Add mpio support for iscsi
This allows the iscsi volume to check if a iscsi device belongs to a mpio device
If it does belong to the device then we make sure we mount the mpio device instead of
the raw device.
The code is based on the current FibreChannel volume support for mpio
example
/dev/disk/by-path/iqn-example.com.2999 -> /dev/sde
Then we check
/sys/block/[dm-X]/slaves/xx
until we find the [dm-X] containing /dev/sde and mount it
Additional work that can be done in future
1. Add multiple portal support to iscsi
2. Move the FibreChannel volume provider to use the code that has been extracted
If it does belong to the device then we make sure we mount the mpio device instead of
the raw device.
Heuristics
Login into /dev/disk/by-path/iqn-example.com.2999 -> /dev/sde
Check if sde existsin in /sys/block/[dm-X]/slaves/xx
If it does mount /dev/[dm-x] which will look like /dev/mapper/mpiodevicename in mount
examples/iscsi has more details
Automatic merge from submit-queue
Flexvolume: Add support for multiple secrets
This PR adds support to pass multiple secrets for flexvolume plugins.
To allow multiple secrets, secrets are now passed as:
"kubernetes.io/secret/id-rsa":"value-2\r\n\r\n","kubernetes.io/secret/id-rsa.pub":"value-1\r\n"
Automatic merge from submit-queue
phase 2 of cassandra example overhaul
Here's the next iteration in overhauling this example, towards https://github.com/kubernetes/kubernetes/issues/20961. This removes the pod adoption part, but doesn't (yet) otherwise change any of the resources used.
It also includes some README cleanup, and removes some explicit specification of labels in the rc yaml.
This PR doesn't yet add any commentary on how we're using the seed provider (re: https://github.com/kubernetes/kubernetes/issues/20961#issuecomment-190405959 etc.). Maybe we should add that.
Also: LMK if this PR should include any changes to the links out to the docs.
cc @bgrant0607 @johndmulhausen
Automatic merge from submit-queue
Implement a streaming serializer for watch
Changeover watch to use streaming serialization. Properly version the
watch objects. Implement simple framing for JSON and Protobuf (but not
YAML).
@wojtek-t @lavalamp
Automatic merge from submit-queue
examples/guestbook/php-redis: don't reopen stdio fds
/etc/apache2/apache2.conf was configured to log in this way:
> ErrorLog /proc/self/fd/2
> CustomLog /proc/self/fd/1 combined
This causes apache to reopen the already-opened fds. It works fine when
the file descriptors are pipes or ttys but it fails when they are Unix
sockets because sockets cannot be opened with the open() syscall. The
issue happens when apache is connected to systemd-journald, like in the
rkt container run-time.
This patch uses "cat" to directly write to the stdio fds without
reopening them. apache2.conf now looks like:
> ErrorLog "|$/bin/cat 1>&2"
> CustomLog "|/bin/cat" combined
It works both with Docker and rkt (tested with and without
--interactive).
Symptoms:
> [ 2673.478868] apache2-foreground[4]: (6)No such device or address:
> AH00091: apache2: could not open error log file /proc/self/fd/2.
See also: https://github.com/coreos/rkt/issues/2300
-----
/cc @sjpotter @yifan-gu @jonboulle
/etc/apache2/apache2.conf was configured to log in this way:
> ErrorLog /proc/self/fd/2
> CustomLog /proc/self/fd/1 combined
This causes apache to reopen the already-opened fds. It works fine when
the file descriptors are pipes or ttys but it fails when they are Unix
sockets because sockets cannot be opened with the open() syscall. The
issue happens when apache is connected to systemd-journald, like in the
rkt container run-time.
This patch uses "cat" to directly write to the stdio fds without
reopening them. apache2.conf now looks like:
> ErrorLog "|$/bin/cat 1>&2"
> CustomLog "|/bin/cat" combined
It works both with Docker and rkt (tested with and without
--interactive).
Symptoms:
> [ 2673.478868] apache2-foreground[4]: (6)No such device or address:
> AH00091: apache2: could not open error log file /proc/self/fd/2.
See also: https://github.com/coreos/rkt/issues/2300
Documented manualSelector field.
Documented that you do not need to provide a selector
or unique labels with batch/v1 Job.
Updated all Job examples to apiVersion: batch/v1
Updated all Job examples to use generated selectors.
Added selector generation to Job's
strategy.Validate, right before validation.
Can't do in defaulting since UID is not known.
Added a validation to Job to ensure that the generated
labels and selector are correct when generation was requested.
This happens right after generation, but validation is in a better
place to return an error.
Adds "manualSelector" field to batch/v1 Job to control selector generation.
Adds same field to extensions/__internal. Conversion between those two
is automatic.
Adds "autoSelector" field to extensions/v1beta1 Job. Used for storing batch/v1 Jobs
- Default for v1 is to do generation.
- Default for v1beta1 is to not do it.
- In both cases, unset == false == do the default thing.
Release notes:
Added batch/v1 group, which contains just Job, and which is the next
version of extensions/v1beta1 Job.
The changes from the previous version are:
- Users no longer need to ensure labels on their pod template are unique to the enclosing
job (but may add labels as needed for categorization).
- In v1beta1, job.spec.selector was defaulted from pod labels, with the user responsible for uniqueness.
In v1, a unique label is generated and added to the pod template, and used as the selector (other
labels added by user stay on pod template, but need not be used by selector).
- a new field called "manualSelector" field exists to control whether the new behavior is used,
versus a more error-prone but more flexible "manual" (not generated) seletor. Most users
will not need to use this field and should leave it unset.
Users who are creating extensions.Job go objects and then posting them using the go client
will see a change in the default behavior. They need to either stop providing a selector (relying on
selector generation) or else specify "spec.manualSelector" until they are ready to do the former.
Update the Deployments' API types, defaulting code, conversions, helpers
and validation to use ReplicaSets instead of ReplicationControllers and
LabelSelector instead of map[string]string for selectors.
Also update the Deployment controller, registry, kubectl subcommands,
client listers package and e2e tests to use ReplicaSets and
LabelSelector for Deployments.
Adds a document on pod templates that can be shared
between various controller docs.
Move more philosophical content to later in the doc.
Add more task-oriented stuff earlier.
Put example config in the document, early on, so users have something concrete to relate the discussion of fields to.
Link to Job and DaemonSet docs.
Make format more like that of Job and DaemonSet docs.
Use jsonpath in examples, which is available in v1.1.
Added example files.
When job.spec.completions is nil, only
one task needs to succeed for the job to succeed,
and parallelism can be scaled freely during runtime.
Added tests.
Release Note:
This causes two minor changes to the API.
First, unset parallelism previously was defaulted to be
equal to completions. Now it always defaults to 1 if unset.
Second, having parallelism=N and completions unset would previously
be defaulted to 1 completion and N parallelism.
(this is not something we expect people to do, though)
Now, no defaulting occurs in that case, and the job's
behavior is different (any completion causes success).
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
change wording
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
change name of volume to be consistent
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
update node flag without =
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
make things a bit clearer, seperate More Info
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
refacter so we include -n example
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
keep uuids consistent in examples
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
detail example about how to set env vars
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
move demo video to more info
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
add references for how to create volume using docker cli
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
italics
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
fix italics
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
fix extra paren
Signed-off-by: Ryan Wallner <ryan.wallner@clusterhq.com>
run hack/update-generated-docs.sh
David McMahon
