Automatic merge from submit-queue
Coreos kube-up now with less cloud init
This update includes significant refactoring. It moves almost all of the
logic into bash scripts, modeled after the `gci` cluster scripts.
The reason to do this is:
1. Avoid duplicating the saltbase manifests by reusing gci's parsing logic (easier maintenance)
2. Take an incremental step towards sharing more code between gci/trusty/coreos, again for better maintenance
3. Pave the way for making future changes (e.g. improved rkt support, kubelet support) easier to share
The primary differences from the gci scripts are the following:
1. Use of the `/opt/kubernetes` directory over `/home/kubernetes`
2. Support for rkt as a runtime
3. No use of logrotate
4. No use of `/etc/default/`
5. No logic related to noexec mounts or gci-specific firewall-stuff
It will make sense to move 2 over to gci, as well as perhaps a few other small improvements. That will be a separate PR for ease of review.
Ref #29720, this is a part of that because it removes a copy of them.
Fixes#24165
cc @yifan-gu
Since this logic largely duplicates logic from the gci folder, it would be nice if someone closely familiar with that gave an OK or made sure I didn't fall into any gotchas related to that, so cc @andyzheng0831
Automatic merge from submit-queue (batch tested with PRs 38426, 38917, 38891, 38935)
Remove cluster/mesos from hack/verify-flags/exceptions.txt
`cluster/mesos` scripts was removed; so remove it from `hack/verify-flags/exceptions.txt`.
The diff was generated by `hack/verify-flags-underscore.py -e > hack/verify-flags/exceptions.txt`.
Automatic merge from submit-queue
conversion-gen: add --skip-unsafe flag
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
Automatic merge from submit-queue
[Federation] Make federation etcd PVC size configurable
This one implements one of the many TODO items pending in the previous set of kubefed PRs.
The design doc PR is at https://github.com/kubernetes/kubernetes/pull/34484
cc @kubernetes/sig-cluster-federation @madhusudancs
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```
[Federation] kubefed init now has a new flag, --etcd-pv-capacity, which can be used to configure the persistent volume capacity for etcd.
```
Automatic merge from submit-queue
kubedns: use initial resource listing as ready signal
Fix#35140.
Set up the ready signal after the first resource listing finished for both endpoints and services instead of listen on kubernetes service.
@bprashanth @bowei @thockin
**Release note**:
```
```
Automatic merge from submit-queue
add a configuration for kubelet to register as a node with taints
and deprecate --register-schedulable
ref #28687#29178
cc @dchen1107 @davidopp @roberthbailey
Automatic merge from submit-queue
Split inflight requests into read-only and mutating groups
cc @smarterclayton @lavalamp @caesarxuchao
```release-note
API server have two separate limits for read-only and mutating inflight requests.
```
Automatic merge from submit-queue
plumb in front proxy group header
Builds on https://github.com/kubernetes/kubernetes/pull/36662 and https://github.com/kubernetes/kubernetes/pull/36774, so only the last commit is unique.
This completes the plumbing for front proxy header information and makes it possible to add just the front proxy header authenticator.
WIP because I'm going to assess it in use downstream.
Automatic merge from submit-queue
kubectl top pod|node should handle when Heapster is somewhere else
OpenShift runs Heapster on HTTPS, which means `top node` and `top pod`
are broken because they hardcode 'http' as the scheme. Provide an
options struct allowing users to specify `--heapster-namespace`,
`--heapster-service`, `--heapster-scheme`, and `--heapster-port` to the
commands (leveraging the existing defaults).
@kubernetes/sig-metrics makes top a little more useful in other spots
Automatic merge from submit-queue (batch tested with PRs 35300, 36709, 37643, 37813, 37697)
Add generated informers
Add informer-gen and the informers it generates. We'll do follow-up PRs to convert everything currently using the hand-written informers to the generated ones.
TODO:
- [x] switch to `GroupVersionResource`
- [x] finish godoc
@deads2k @caesarxuchao @sttts @liggitt
Automatic merge from submit-queue
Add kubernetes-anywhere as a new e2e deployment option.
This change adds support for using `kubernetes-anywhere` as a deployment option for hack/e2e.go. This work is toward the larger goal of being able to run e2e tests against `kubeadm` clusters, which `kubernetes-anywhere` supports.
**Release note**:
```release-note
Add kubernetes-anywhere as a new e2e deployment option
```
The configuration in `getConfig()` comes mostly from the defaults in `kubernetes-anywhere`. In the future, we can add more plumbing to override them via CLI flags.
CC @mikedanese
Automatic merge from submit-queue
create service add create ExternalName service implementation
@kubernetes/kubectl create service add ExternalName support, refer #34731 for more detail.
```release-note
kubectl create service externalname
```
OpenShift runs Heapster on HTTPS, which means `top node` and `top pod`
are broken because they hardcode 'http' as the scheme. Provide an
options struct allowing users to specify `--heapster-namespace`,
`--heapster-service`, `--heapster-scheme`, and `--heapster-port` to the
commands (leveraging the existing defaults).
Automatic merge from submit-queue
move parts of the mega generic run struct out
This splits the main `ServerRunOptions` into composeable pieces that are bindable separately and adds easy paths for composing servers to run delegating authentication and authorization.
@sttts @ncdc alright, I think this is as far as I need to go to make the composing servers reasonable to write. I'll try leaving it here
Automatic merge from submit-queue
[Federation][join-flags] Add flags for cluster context and secret names while joining clusters to federation.
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make
`kubefed join` consistent with `kubefed unjoin`. This also means
`--cluster-context` is now made a flag and defaults to cluster name if
unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Fixes: Issue #35954
cc @kubernetes/sig-cluster-federation @quinton-hoole @irfanurrehman
- Adds command line flags --config-map, --config-map-ns.
- Fixes 36194 (https://github.com/kubernetes/kubernetes/issues/36194)
- Update kube-dns yamls
- Update bazel (hack/update-bazel.sh)
- Update known command line flags
- Temporarily reference new kube-dns image (this will be fixed with
a separate commit when the DNS image is created)
Vast majority of cluster contexts are not RFC 1123 subdomains. Since
cluster and secret names for the API objects are derived from the
cluster context name, there is no way for users to join clusters
with such context names to federation, unless they modify the context
name in their kubeconfigs itself. That's a lot of inconvenience and
entirely goes against the goal and beats the purpose of the `kubefed`
tool. So we are providing these flags to allow users to override these
values.
Also, since users register their clusters with federation, it is makes
sense in terms of user experience to make the cluster name a positional
argument because that feels more natural. Also, specifying cluster name
in the join command as a mandatory positional argument make `kubefed
join` consistent with `kubefed unjoin`. This also means `--cluster-
context` is now made a flag and defaults to cluster name if unspecified.
`--secret-name` also defaults to the cluster name if unspecified.
Automatic merge from submit-queue
Add a flag allowing contention profiling of the API server
Useful for performance debugging.
cc @smarterclayton @timothysc @lavalamp
```release-note
Add a flag allowing contention profiling of the API server
```
Automatic merge from submit-queue
Make GCI nodes mount non tmpfs, ext* & bind mounts using an external mounter
This PR downloads the stage1 & gci-mounter ACIs as part of cluster bring up instead of downloading them dynamically from gcr.io, which was the cause for #36206.
I have also optimized the containerized mounter to pre-load the mounter image once to avoid fetch latency while using it.
Original PR which got reverted: https://github.com/kubernetes/kubernetes/pull/35821
```release-note
GCI nodes use an external mounter script to mount NFS & GlusterFS storage volumes
```
@mtaufen Node e2e is not re-enabled in this PR.
cc @jingxu97
Automatic merge from submit-queue
specify custom ca file to verify the keystone server
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Sometimes the keystone server's certificate is self-signed, mainly used for internal development, testing and etc.
For this kind of ca, we need a way to verify the keystone server.
Otherwise, below error will occur.
> x509: certificate signed by unknown authority
This patch provide a way to pass in a ca file to verify the keystone server when starting `kube-apiserver`.
**Which issue this PR fixes** : fixes#22695, #24984
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
``` release-note
```
Provides an opt-in flag, --experimental-fail-swap-on (and corresponding
KubeletConfiguration value, ExperimentalFailSwapOn), which is false by default.
If we have a public & private zone with the same name (which is common
on AWS), this means we can still create records. Also tighten up some
of the logic to allow for zones with duplicate names.
Automatic merge from submit-queue
Node Conformance Test: Add system verification
For #30122 and #29081.
This PR introduces system verification test in node e2e and conformance test. It will run before the real test. Once the system verification fails, the test will just fail. The output of the system verification is like this:
```
I0909 23:33:20.622122 2717 validators.go:45] Validating os...
OS: Linux
I0909 23:33:20.623274 2717 validators.go:45] Validating kernel...
I0909 23:33:20.624037 2717 kernel_validator.go:79] Validating kernel version
KERNEL_VERSION: 3.16.0-4-amd64
I0909 23:33:20.624146 2717 kernel_validator.go:93] Validating kernel config
CONFIG_NAMESPACES: enabled
CONFIG_NET_NS: enabled
CONFIG_PID_NS: enabled
CONFIG_IPC_NS: enabled
CONFIG_UTS_NS: enabled
CONFIG_CGROUPS: enabled
CONFIG_CGROUP_CPUACCT: enabled
CONFIG_CGROUP_DEVICE: enabled
CONFIG_CGROUP_FREEZER: enabled
CONFIG_CGROUP_SCHED: enabled
CONFIG_CPUSETS: enabled
CONFIG_MEMCG: enabled
I0909 23:33:20.679328 2717 validators.go:45] Validating cgroups...
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
I0909 23:33:20.679454 2717 validators.go:45] Validating docker...
DOCKER_GRAPH_DRIVER: aufs
```
It verifies the system following a predefined `SysSpec`:
``` go
// DefaultSysSpec is the default SysSpec.
var DefaultSysSpec = SysSpec{
OS: "Linux",
KernelVersion: []string{`3\.[1-9][0-9].*`, `4\..*`}, // Requires 3.10+ or 4+
// TODO(random-liu): Add more config
KernelConfig: KernelConfig{
Required: []string{
"NAMESPACES", "NET_NS", "PID_NS", "IPC_NS", "UTS_NS",
"CGROUPS", "CGROUP_CPUACCT", "CGROUP_DEVICE", "CGROUP_FREEZER",
"CGROUP_SCHED", "CPUSETS", "MEMCG",
},
Forbidden: []string{},
},
Cgroups: []string{"cpu", "cpuacct", "cpuset", "devices", "freezer", "memory"},
RuntimeSpec: RuntimeSpec{
DockerSpec: &DockerSpec{
Version: []string{`1\.(9|\d{2,})\..*`}, // Requires 1.9+
GraphDriver: []string{"aufs", "overlay", "devicemapper"},
},
},
}
```
Currently, it only supports:
- Kernel validation: version validation and kernel configuration validation
- Cgroup validation: validating whether required cgroups subsystems are enabled.
- Runtime Validation: currently, only validates docker graph driver.
The validating framework is ready. The specific validation items could be added over time.
@dchen1107
/cc @kubernetes/sig-node
Automatic merge from submit-queue
New command: "kubeadm token generate"
As part of #33930, this PR adds a new top-level command to kubeadm to just generate a token for use with the init/join commands. Otherwise, users are left to either figure out how to generate a token on their own, or let `kubeadm init` generate a token, capture and parse the output, and then use that token for `kubeadm join`.
At this point, I was hoping for feedback on the CLI experience, and then I can add tests. I spoke with @mikedanese and he didn't like the original propose of `kubeadm util generate-token`, so here are the runners up:
```
$ kubeadm generate-token # <--- current implementation
$ kubeadm generate token # in case kubeadm might generate other things in the future?
$ kubeadm init --generate-token # possibly as a subcommand of an existing one
```
Currently, the output is simply the token on one line without any padding/formatting:
```
$ kubeadm generate-token
1087fd.722b60cdd39b1a5f
```
CC: @kubernetes/sig-cluster-lifecycle
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
``` release-note
New kubeadm command: generate-token
```
Automatic merge from submit-queue
make using service account credentials from controllers optional
Adds a flag to make the "SA per controller" behavior optional.
Automatic merge from submit-queue
[Kubelet] Use the custom mounter script for Nfs and Glusterfs only
This patch reduces the scope for the containerized mounter to NFS and GlusterFS on GCE + GCI clusters
This patch also enabled the containerized mounter on GCI nodes
Shepherding multiple PRs through the submit queue is painful. Hence I combined them into this PR. Please review each commit individually.
cc @jingxu97 @saad-ali
https://github.com/kubernetes/kubernetes/pull/35652 has also been reverted as part of this PR
Automatic merge from submit-queue
[Federation][init] Implement `kubefed init` command that performs federation control plane bootstrap.
Please review only the last commit here. This is based on PR #35593 which will be reviewed independently.
I am intentionally not including the unit tests in this PR to better distribute and parallelize reviews. This PR is already big.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
Automatic merge from submit-queue
[Federation][unjoin-00] Implement `kubefed unjoin` command.
Please review only the last commit here. This is based on PR #35493 which will be reviewed independently.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @nikhiljindal
Automatic merge from submit-queue
Move .gitattributes annotation to the root, so GitHub will respect them.
This should fix the merge conflicts by letting GitHub use the simpler line-by-line algorithm for this file. Having .gitattributes in a sub-directory would work for local merging, but would show conflicts on the web UI.
Automatic merge from submit-queue
[Federation][join-01] Implement `kubefed join` command.
Supersedes PR #35155.
Please review only the last commit here. This is based on PR #35492 which will be reviewed independently.
I will add a release note separately for this entire feature, so please don't worry too much about the release note here in the PR.
Design Doc: PR #34484
cc @kubernetes/sig-cluster-federation @quinton-hoole @mwielgus
Along the way: Fix ginkgo-e2e.sh. This change got dropped in the
original PR, but it was meant to allow a conformance-style
kubectl-auth test and still have a legit cloud provider.
We can put subdomains into hosted zones (for example,
foo.federation.example.com can be hosted in example.com)
By allowing sharing a common hosted zone, this means the user doesn't
have to do as much setup.
In order to be able to use new mounter library, this PR adds the
mounterPath flag to kubelet which passes the flag to the mount
interface. If flag is empty, mount uses default mount path.
Automatic merge from submit-queue
WIP: Remove the legacy networking mode
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
Removes the deprecated configure-cbr0 flag and networking mode to avoid having untested and maybe unstable code in kubelet, see: #33789
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes#30589fixes#31937
**Special notes for your reviewer**: There are a lot of deployments who rely on this networking mode. Not sure how we deal with that: force switch to kubenet or just delete the old deployment?
But please review the code changes first (the first commit)
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Removed the deprecated kubelet --configure-cbr0 flag, and with that the "classic" networking mode as well
```
PTAL @kubernetes/sig-network @kubernetes/sig-node @mikedanese
Automatic merge from submit-queue
Add flags for alternative API and discovery ports
**What this PR does / why we need it**:
We had many issues reported due to the fact that we are using port 443 by default, and we should allow users to override whatever defaults we pick. This doesn't touch on `localhost:8080` yet, which should just generally get rid of.
**Which issue this PR fixes**: fixes#34311#34307#33638
**Special notes for your reviewer**: cc @pires
**Release note**:
```release-note
Add `kubeadm` flags `--api-port` and `--discovery-port`, change default API port to 6443
```
Automatic merge from submit-queue
clientgen: allow to pass custom apiPath when generating client sets
This PR allow to pass the `--clientset-api-path` parameter to clientgen that allows to customize the default API path set in clients. This allows projects like OpenShift to re-use the client with different API path (`/oapi` in our case).
Automatic merge from submit-queue
add UpdateRuntimeConfig interface
Expose UpdateRuntimeConfig interface in RuntimeService for kubelet to pass a set of configurations to runtime. Currently it only takes PodCIDR.
The use case is for kubelet to pass configs to runtime. Kubelet holds some config/information which runtime does not have, such as PodCIDR. I expect some of kubelet configurations will gradually move to runtime, but I believe cases like PodCIDR, which dynamically assigned by k8s master, need to stay for a while.
Automatic merge from submit-queue
kubeadm implement preflight checks
Checks that user running kubeamd init and join is root and will only execute
command if user is root. Moved away from using kubectl error handling to
having kubeadm handle its own errors. This should allow kubeadm to have
more meaningful errors, exit codes, and logging for specific kubeadm use
cases.
fixes#33908
Add skip-preflight-checks to known flags.
Fix bug with preflight checks not returning system is-active as errors.
Fix error handling to use correct function.
Automatic merge from submit-queue
kube-dns: Add --dns-bind-address flag
Currently, `kube-dns` always binds to `0.0.0.0` when starting the SkyDNS server. Sometimes it's useful to have it bind to a different address for testing. So add a new `--dns-bind-address` flag that can be used to configure this (default is `0.0.0.0`).
Automatic merge from submit-queue
Add gcl cluster logging test
This PR changes default logging destination for tests to gcp and adds test for cluster logging using google cloud logging
Fix#20760
Automatic merge from submit-queue
log-dump.sh: Add a LOG_DUMP_USE_KUBECTL mode, refactor
**What this PR does / why we need it**: This refactors log-dump.sh so that it can optionally just use `"kubectl get node"` output, for dumping logs off a cluster that was brought up using something other than kube-up.sh (e.g. kops or kubeadm).
Note that for obvious reasons, if the cluster is fried, it'll probably fail to hit the apiserver and all bets are off, but for the 95% case where you're debugging tests, this is good. To cover the remaining 5%, you could actually squirrel away the relevant external IPs after the e2e.go:Up stage (because we better be able to hit the apiserver then) and fetch them from cache later.
**Special notes for your reviewer**: Sorry for so many changes. :)
This refactors log-dump.sh so that it can optionally just use "kubectl
get node" output, for dumping logs off a cluster that was brought up
using sometihng other than kube-up.sh (e.g. kops or kubeadm).
Note that for obvious reasons, if the cluster is fried, it'll probably
fail to hit the apiserver and all bets are off, but for the 95% case
where you're debugging tests, this is good. To cover the remaining 5%,
you could actually squirrel away the relevant external IPs after the
e2e.go:Up stage (because we better be able to hit the apiserver then)
and fetch them from cache later.
Automatic merge from submit-queue
Add cgroup-driver and cgroups-per-qos flags to kubelet
Add the flags needed to support pod-level cgroups to kubelet.
/cc @vishh @dchen1107 @dubstack
Automatic merge from submit-queue
add delete-namespace-on-failure flag
I have been doing this for a while.
Setting `--delete-namespace=false --clean-start=true` only works if you have only one e2e test running in a loop.
This PR lets someone to set `delete-namespace-on-failure=false` and run multiple tests in parallel and preserve the crime scene. It makes it easier to reproduce failures.
Let me know if this is worth it or there are some other tricks I am not aware.
Automatic merge from submit-queue
Add support for vpshere cloud provider in kubeup
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
vSphere cloud provider added in 1.3 was not configured when deploying via kubeup
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Add support for vSphere Cloud Provider when deploying via kubeup on vSphere.
```
When deploying on vSphere using kube up add configuration
for vSphere cloud provider.
Automatic merge from submit-queue
Add option to set a service nodeport
**Release note**:
```release-note
Add kubectl --node-port option for specifying the service nodeport
```
This patch adds the option to set a nodeport when creating a NodePort
service. In case of a port allocation error due to a specified port
being out of the valid range, the error now includes the valid
range. If a `--node-port` value is not specified, it defaults to zero, in
which case the allocator will default to its current behavior of
assigning an available port.
This patch also adds a new helper function in `cmd/util/helpers.go` to
retrieve `Int32` cobra flags.
**Example**
```
# create a nodeport service with an invalid port
$ kubectl create service nodeport mynodeport --tcp=8080:7777 --node-port=1
The Service "mynodeport" is invalid: spec.ports[0].nodePort: Invalid
value: 1: provided port is not in the valid range. Valid ports range
from 30000-32767
# create a nodeport service with a valid port
$ kubectl create service nodeport mynodeport --tcp=8080:7777 --node-port=30000
service "mynodeport" created
# create a nodeport service with a port already in use
$ kubectl create service nodeport mynodeport --tcp=8080:7777 --node-port=30000
The Service "mynodeport" is invalid: spec.ports[0].nodePort: Invalid value: 3000: provided port is already allocated
$ kubectl describe service mynodeport
Name: mynodeport
Namespace: default
Labels: app=mynodeport
Selector: app=mynodeport
Type: NodePort
IP: 172.30.81.254
Port: 8080-7777 8080/TCP
NodePort: 8080-7777 30000/TCP
Endpoints: <none>
Session Affinity: None
No events.
```
@fabianofranz
Automatic merge from submit-queue
CRI: Remove the mount name and port name.
Per discussion on https://github.com/kubernetes/kubernetes/issues/33873.
Currently the mount name is not being used and also involves some
incorrect usage (sometimes it's referencing a mount name, sometimes
it's referecing a volume name), so we decide to remove it from CRI.
The port name is also not used, so remove it as well.
Fix#33873Fix#33526
/cc @kubernetes/sig-node @kubernetes/sig-rktnetes
This adds a flag to override the version we're deploying with kops for
e2es. (The kops experience is disconnected from the actual source tree
we're running tests from, which is similar to GKE.)
Automatic merge from submit-queue
add anytoken authenticator
Adds `--insecure-allow-any-token` as a flag to the API server to create an authenticator that will accept any bearer token and transform it into a user by parsing it out as `username/group1,group2,...`.
This gives an easy way to identify as a user and check permissions:
```bash
ALLOW_ANY_TOKEN=true hack/local-up-cluster.sh
kubectl config set-cluster local-kube --server=https://localhost:6443 --insecure-skip-tls-verify=true
kubectl config set-credentials david --token=david/group1
kubectl config set-context local --cluster=local-kube --user=david
kubectl config use-context local
```
@kubernetes/sig-auth
Automatic merge from submit-queue
Fixing local cluster up
hack: fix local-cluster-up advertise_address default value
Fixes: 9fc1d61
Signed-off-by: André Martins <aanm90@gmail.com>
ping @dims
Automatic merge from submit-queue
Allow anonymous API server access, decorate authenticated users with system:authenticated group
When writing authorization policy, it is often necessary to allow certain actions to any authenticated user. For example, creating a service or configmap, and granting read access to all users
It is also frequently necessary to allow actions to any unauthenticated user. For example, fetching discovery APIs might be part of an authentication process, and therefore need to be able to be read without access to authentication credentials.
This PR:
* Adds an option to allow anonymous requests to the secured API port. If enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of `system:anonymous` and a group of `system:unauthenticated`. Note: this should only be used with an `--authorization-mode` other than `AlwaysAllow`
* Decorates user.Info returned from configured authenticators with the group `system:authenticated`.
This is related to defining a default set of roles and bindings for RBAC (https://github.com/kubernetes/features/issues/2). The bootstrap policy should allow all users (anonymous or authenticated) to request the discovery APIs.
```release-note
kube-apiserver learned the '--anonymous-auth' flag, which defaults to true. When enabled, requests to the secure port that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of 'system:anonymous' and a group of 'system:unauthenticated'.
Authenticated users are decorated with a 'system:authenticated' group.
NOTE: anonymous access is enabled by default. If you rely on authentication alone to authorize access, change to use an authorization mode other than AlwaysAllow, or or set '--anonymous-auth=false'.
```
c.f. https://github.com/kubernetes/kubernetes/issues/29177#issuecomment-244191596
Contination of #1111
I tried to keep this PR down to just a simple search-n-replace to keep
things simple. I may have gone too far in some spots but its easy to
roll those back if needed.
I avoided renaming `contrib/mesos/pkg/minion` because there's already
a `contrib/mesos/pkg/node` dir and fixing that will require a bit of work
due to a circular import chain that pops up. So I'm saving that for a
follow-on PR.
I rolled back some of this from a previous commit because it just got
to big/messy. Will follow up with additional PRs
Signed-off-by: Doug Davis <dug@us.ibm.com>
This splits off all the bash stuff into an interface, and plumbs
through a separate interface to bring up a cluster using "kops"
instead. Right now it assumes kops == AWS.
Automatic merge from submit-queue
Allow secure access to apiserver from Admission Controllers
* Allow options.InsecurePort to be set to 0 to switch off insecure access
* In NewSelfClient, Set the TLSClientConfig to the cert and key files
if InsecurePort is switched off
* Mint a bearer token that allows the client(s) created in NewSelfClient
to talk to the api server
* Add a new authenticator that checks for this specific bearer token
Fixes#13598
Automatic merge from submit-queue
Viper direct bindings to TestContext struct with hierarchichal suppor…
Part of #31453 to support hierarchichal parameters. This one does so for density, paves way for other tests as well.
Automatic merge from submit-queue
Enable hostpath provisioner for vagrant environment
This flag is required to run e2e tests for certain features (petset), and for manual tests and debugging.
related: https://github.com/kubernetes/kubernetes/issues/32119
The new flag, if specified, and if --container-runtime=docker, switches
kubelet to use the new CRI implementation for testing. This is hidden flag
since the feature is still under heavy development and the flag may be changed
in the near future.
Automatic merge from submit-queue
[kubelet] Fix oom-score-adj policy in kubelet
Fixes#32238
We have been having this regression since v1.3. It is critical for GKE/GCE deployments of k8s because docker daemon has a high likelihood of being OOM killed which will end up nuking all containers.
The reason for moving from mnt to pid is that docker daemon moves itself into a new mnt namespace with systemd based deployments.
Automatic merge from submit-queue
Pet Set Example for Cassandra
- updating cassandra to 3.7
- added pet set example
- adding pet set for Cassandra e2e tests
- changed service as we do not want a lb service, as we are running C*
- updated docs
cc @bgrant0607
cc @kubernetes/examples
We can probably close a couple of other open PR, since I did some other stuff.
It is required to run automated tests for certain features (petset),
and for manual tests and debugging.
Change-Id: I9203aab6d67c8ff0cc4574473e8d0af888fe1804
Automatic merge from submit-queue
Create a flag for route reconciliation and deprecate unused node-sync-…
cc @pwittrock to decide if it can go in 1.4. It's a zero risk cleanup, that will reduce confusion about cluster configuration.
Automatic merge from submit-queue
etcd: data rollback tool of v3 -> v2
ref: https://github.com/kubernetes/features/issues/44
ref #20504
What?
This provides a rollback tool for some users to rollback etcd data from v3 to v2.
Automatic merge from submit-queue
Add flag to set CNI bin dir, and use it on gci nodes
**What this PR does / why we need it**:
When using `kube-up` on GCE, following #31023 which moved the workers from debian to gci, CNI just isn't working. The root cause is basically as discussed in #28563: one flag (`--network-plugin-dir`) means two different things, and the `configure-helper` script uses it for the wrong purpose.
This PR adds a new flag `--cni-bin-dir`, then uses it to configure CNI as desired.
As discussed at #28563, I have also added a flag `--cni-conf-dir` so users can be explicit
**Which issue this PR fixes** : fixes#28563
**Special notes for your reviewer**:
I left the old flag largely alone for backwards-compatibility, with the exception that I stop setting the default when CNI is in use. The value of `"/usr/libexec/kubernetes/kubelet-plugins/net/exec/"` is unlikely to be what is wanted there.
**Release note**:
```release-note
Added new kubelet flags `--cni-bin-dir` and `--cni-conf-dir` to specify where CNI files are located.
Fixed CNI configuration on GCI platform when using CNI.
```
- updating java to 3.7
- added pet set example
- adding pet set for Cassandra e2e tests
- changed service as we do not want a lb service, as we are running C*
- updated docs
fixing headers and adding exception for run.sh
adding documentation, thank god for reflog
Did not mean to commit that as the README ... fixing
fixing problems in README
fixing more problems in README
more README tweaks
munge updates
updating examples_test for PetSet in Cassandra examples
updating petset to no use better security context
Automatic merge from submit-queue
Node E2E: Move the image and host related logic to be remote run specific.
This PR addresses #31597, and better fixes#31588.
With this PR:
1) All remote logic is moved into remote mode.
2) All gci related logic is only called when `HOSTS` and `IMAGES` are not specified.
3) Run node e2e against other node e2e images will work again - `make test-e2e-node REMOET=true IMAGES=e2e-node-containervm-v20160321-image`.
4) List images will work again - `make test-e2e-node REMOTE=true LIST_IMAGES=true`.
Mark 1.4, because https://github.com/kubernetes/kubernetes/pull/31588 is marked as 1.4. And this makes the node e2e work as is described in the [doc ](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/e2e-node-tests.md#run-tests-using-different-images).
@yujuhong @vishh
/cc @kubernetes/sig-node
Automatic merge from submit-queue
Separate federation build.sh into development and deployment scripts.
The idea behind this separation is that it provides a clear distinction
between the dev environment and the prod environment. The
deploy/deploy.sh script will be shipped to the users, but
develop/develop.sh will be purely for development purposes and won't
be part of a release distribution.
Purely for developer convenience, all the deployment functionality is
made available through the develop/develop.sh script.
This change also copies deploy/* files into the release distribution.
cc @kubernetes/sig-cluster-federation @colhom
```release-note
Federation can now be deployed using the `federation/deploy/deploy.sh` script. This script does not depend on any of the development environment shell library/scripts. This is an alternative to the current `federation-up.sh`/`federation-down.sh` scripts. Both the scripts are going to co-exist in this release, but the `federation-up.sh`/`federation-down.sh` scripts might be removed in a future release in favor of `federation/deploy/deploy.sh` script.
```
Automatic merge from submit-queue
Node Conformance Test: Refactor node e2e framework
For #30122, #30174.
Based on #30348.
**Please only review the last 3 commits.**
This PR is part of our roadmap to package node conformance test.
The 1st commit is from #30348, it removed unnecessary dependencies in the node e2e test framework, because we've statically linked these dependencies.
The PR refactored the node e2e framework. Moving different utilities into different packages under `pkg/`.
We need to do this because:
1) Files like e2e_remote.go and e2e_build.go should only be used by runner, but they were compiled into the test suite because they were placed in the same package. The worst thing is that it will introduce some never used flags in the test suite binary.
2) Make the directory structure more clear. Only test should be placed in `test/e2e_node`, other utilities should be placed in different packages in `pkg/`.
@dchen1107 @vishh
/cc @kubernetes/sig-node @kubernetes/sig-testing
Automatic merge from submit-queue
Add kubelet --network-plugin-mtu flag for MTU selection
* Add network-plugin-mtu option which lets us pass down a MTU to a network provider (currently processed by kubenet)
* Add a test, and thus make sysctl testable
MTU selection is difficult, and if there is a transport such as IPSEC in
use may be impossible. So we allow specification of the MTU with the
network-plugin-mtu flag, and we pass this down into the network
provider.
Currently implemented by kubenet.
Automatic merge from submit-queue
Kubelet: add --container-runtime-endpoint and --image-service-endpoint
Flag `--container-runtime-endpoint` (overrides `--container-runtime`) is introduced to identify the unix socket file of the remote runtime service. And flag `--image-service-endpoint` is introduced to identify the unix socket file of the image service.
This PR is part of #28789 Milestone 0.
CC @yujuhong @Random-Liu
Automatic merge from submit-queue
Implement TLS bootstrap for kubelet using `--experimental-bootstrap-kubeconfig` (2nd take)
Ref kubernetes/features#43 (comment)
cc @gtank @philips @mikedanese @aaronlevy @liggitt @deads2k @errordeveloper @justinsb
Continue on the older PR https://github.com/kubernetes/kubernetes/pull/30094 as there are too many comments on that one and it's not loadable now.
New flag --container-runtime-endpoint (overrides --container-runtime)
is introduced to kubelet which identifies the unix socket file of
the remote runtime service. And new flag --image-service-endpoint is
introduced to kubelet which identifies the unix socket file of the
image service.
Automatic merge from submit-queue
Expose flags for new NodeEviction logic in NodeController
Fix#28832
Last PR from the NodeController NodeEviction logic series.
cc @davidopp @lavalamp @mml
Automatic merge from submit-queue
Let load and density e2e tests use GC if it's on
I've run the 100 and 500 nodes tests and they both pass.
The test-infra half of the PR is https://github.com/kubernetes/test-infra/pull/369
cc @lavalamp
Automatic merge from submit-queue
A build and deploy script to manage cluster federation lifecycle and a Makefile wrapper to drive that process.
Also includes a sample config file to describe clusters.
The build script implements the following things:
1. Generates the required configs.
2. Builds the hyperkube binary and the corresponding docker image.
3. Pushes the image to a specified repository.
4. Pulls the federation installer docker images.
5. Builds the Kubernetes clusters described the config.json file.
6. Pushes the federation components to one of the Kubernetes clusters
built in the previous step.
7. Also turns down the federation components and the Kubernetes
clusters.
**NOTE**: Installer images are right now being pulled from my public repository of docker images. I am working on pushing them to our release repository.
```release-note
Cluster Federation components can now be built and deployed using the make command. Please see federation/README.md for details.
```
cc @kubernetes/sig-cluster-federation @colhom
Fixes: Issue #26655
Automatic merge from submit-queue
[kubelet] Introduce --protect-kernel-defaults flag to make the tunable behaviour configurable
Let's make the default behaviour of kernel tuning configurable. The default behaviour is kept modify as has been so far.
Automatic merge from submit-queue
[Kubelet] Rename `--config` to `--pod-manifest-path`. `--config` is deprecated.
This field holds the location of a manifest file or directory of manifest
files for pods the Kubelet is supposed to run. The name of the field
should reflect that purpose. I didn't change the flag name because that
API should remain stable.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29999)
<!-- Reviewable:end -->
Also provide a new --pod-manifest-path flag and deprecate the old
--config one.
This field holds the location of a manifest file or directory of manifest
files for pods the Kubelet is supposed to run. The name of the field
should reflect that purpose.
Kubernetes Submit Queue
