Remove the deprecated `scheduler.alpha.kubernetes.io/critical-pod` pod annotation and use
the `priorityClassName` first class attribute instead, setting all master components to
`system-cluster-critical`.
Add test files that exclude the field in question
under KubeletConfiguration -> evictionHard for non-Linux.
Add runtime abstraction for the test files in initconfiguration_tests.go
The current code logs an error and full blown backtrace if we fail to remove
the containers upon reset. This creates unneeded, huge and rather scary log
message. Fix that by leaving just the error message.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Currently ConfigFileAndDefaultsToInternalConfig and
FetchConfigFromFileOrCluster are used to default and load InitConfiguration
from file or cluster. These two APIs do a couple of completely separate things
depending on how they were invoked. In the case of
ConfigFileAndDefaultsToInternalConfig, an InitConfiguration could be either
defaulted with external override parameters, or loaded from file.
With FetchConfigFromFileOrCluster an InitConfiguration is either loaded from
file or from the config map in the cluster.
The two share both some functionality, but not enough code. They are also quite
difficult to use and sometimes even error prone.
To solve the issues, the following steps were taken:
- Introduce DefaultedInitConfiguration which returns defaulted version agnostic
InitConfiguration. The function takes InitConfiguration for overriding the
defaults.
- Introduce LoadInitConfigurationFromFile, which loads, converts, validates and
defaults an InitConfiguration from file.
- Introduce FetchInitConfigurationFromCluster that fetches InitConfiguration
from the config map.
- Reduce, when possible, the usage of ConfigFileAndDefaultsToInternalConfig by
replacing it with DefaultedInitConfiguration or LoadInitConfigurationFromFile
invocations.
- Replace all usages of FetchConfigFromFileOrCluster with calls to
LoadInitConfigurationFromFile or FetchInitConfigurationFromCluster.
- Delete FetchConfigFromFileOrCluster as it's no longer used.
- Rename ConfigFileAndDefaultsToInternalConfig to
LoadOrDefaultInitConfiguration in order to better describe what the function
is actually doing.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
systemd is the recommended driver as per the setup of running
the kubelet using systemd as the init system. Add a preflight
check that throws a warning if this isn't the case.
Currently JoinConfigFileAndDefaultsToInternalConfig is doing a couple of
different things depending on its parameters. It:
- loads a versioned JoinConfiguration from an YAML file.
- returns defaulted JoinConfiguration allowing for some overrides.
In order to make code more manageable, the following steps are taken:
- Introduce LoadJoinConfigurationFromFile, which loads a versioned
JoinConfiguration from an YAML file, defaults it (both dynamically and
statically), converts it to internal JoinConfiguration and validates it.
- Introduce DefaultedJoinConfiguration, which returns defaulted (both
dynamically and statically) and verified internal JoinConfiguration.
The possibility of overwriting defaults via versioned JoinConfiguration is
retained.
- Re-implement JoinConfigFileAndDefaultsToInternalConfig to use
LoadJoinConfigurationFromFile and DefaultedJoinConfiguration.
- Replace some calls to JoinConfigFileAndDefaultsToInternalConfig with calls to
either LoadJoinConfigurationFromFile or DefaultedJoinConfiguration where
appropriate.
- Rename JoinConfigFileAndDefaultsToInternalConfig to the more appropriate name
LoadOrDefaultJoinConfiguration.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
DetectUnsupportedVersion is somewhat uncomfortable, complex and inefficient
function to use. It takes an entire YAML document as bytes, splits it up to
byte slices of the different YAML sub-documents and group-version-kinds and
searches through those to detect an unsupported kubeadm config. If such config
is detected, the function returns an error, if it is not (i.e. the normal
function operation) everything done so far is discarded.
This could have been acceptable, if not the fact, that in all cases that this
function is called, the YAML document bytes are split up and an iteration on
GVK map is performed yet again. Hence, we don't need DetectUnsupportedVersion
in its current form as it's inefficient, complex and takes only YAML document
bytes.
This change replaces DetectUnsupportedVersion with ValidateSupportedVersion,
which takes a GroupVersion argument and checks if it is on the list of
unsupported config versions. In that case an error is returned.
ValidateSupportedVersion relies on the caller to read and split the YAML
document and then iterate on its GVK map checking if the particular
GroupVersion is supported or not.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
* Return `nil` instead of a pointer to an empty struct when possible,
before the pointer was introduced the empty struct was required.
* Explicitly accept only one argument maximum for `kubeadm join` as in
`kubeadm join <master>`.
* Accept no arguments for `kubeadm init`.
* Make phases leafs accept arguments, whereas intermediate phases
automatically gets set that they accept no arguments.
For historical reasons InitConfiguration is used almost everywhere in kubeadm
as a carrier of various configuration components such as ClusterConfiguration,
local API server endpoint, node registration settings, etc.
Since v1alpha2, InitConfiguration is meant to be used solely as a way to supply
the kubeadm init configuration from a config file. Its usage outside of this
context is caused by technical dept, it's clunky and requires hacks to fetch a
working InitConfiguration from the cluster (as it's not stored in the config
map in its entirety).
This change is a small step towards removing all unnecessary usages of
InitConfiguration. It reduces its usage by replacing it in some places with
some of the following:
- ClusterConfiguration only.
- APIEndpoint (as local API server endpoint).
- NodeRegistrationOptions only.
- Some combinations of the above types, or if single fields from them are used,
only those field.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Fix a couple of problems related to data used by the phases runners
of `init` and `join`.
1) make `newInitData()` and `newJoinData()` return pointers.
Methods of the data objects returned by these functions should
be able to modify fields in the data objects - e.g.
`func (d initData) Client()`. This allows us to store a state and
not execute the same logic multiple times - e.g. obtaining a client.
A side effect of this change is that the `new...` functions must return
pointers, so that casting the data object in a phase, from
`workflow.RunData` to a locally defined interface, works.
2) Make it possible to pass arguments from a parent command
to a sub-phase with regards to data initialization.
In output of "kubeadm init", there are two strings for meaning of
bootstrap token which are "bootstraptoken" and "bootstrap-token".
This changes `[bootstraptoken]` to `[bootstrap-token]`.
In order to allow for a smoother UX with CRIs different than Docker, we have to
make the --cri-socket command line flag optional when just one CRI is
installed.
This change does that by doing the following:
- Introduce a new runtime function (DetectCRISocket) that will attempt to
detect a CRI socket, or return an appropriate error.
- Default to using the above function if --cri-socket is not specified and
CRISocket in NodeRegistrationOptions is empty.
- Stop static defaulting to DefaultCRISocket. And rename it to
DefaultDockerCRISocket. Its use is now narrowed to "Docker or not"
distinguishment and tests.
- Introduce AddCRISocketFlag function that adds --cri-socket flag to a flagSet.
Use that in all commands, that support --cri-socket.
- Remove the deprecated --cri-socket-path flag from kubeadm config images pull
and deprecate --cri-socket in kubeadm upgrade apply.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
This avoids ending in a wrong cluster state by assuming that the
present certificates will work. It is specially important when we
are growing etcd from 1 member to 2, in which case in case of failure
upon joining etcd will be unavailable.
When the etcd cluster grows we need to explicitly wait for it to be
available. This ensures that we are not implicitly doing this in
following steps when they try to access the apiserver.
This is a minor cleanup which helps to make the code of kubeadm a bit
less error-prone by reducing the scope of local variables and
unexporting functions that are not meant to be used outside of their
respective modules.
e2e-node tests may use custom system specs for validating nodes to
conform the specs. The functionality is switched on when the tests
are run with this command:
make SYSTEM_SPEC_NAME=gke test-e2e-node
Currently the command fails with the error:
F1228 16:12:41.568836 34514 e2e_node_suite_test.go:106] Failed to load system spec: open /home/rojkov/go/src/k8s.io/kubernetes/k8s.io/kubernetes/cmd/kubeadm/app/util/system/specs/gke.yaml: no such file or directory
Move the spec file under `test/e2e_node/system/specs` and introduce a single
public constant referring the file to use instead of multiple private constants.
Selfhosting pivoting fails when using --store-certs-in-secrets
as controller-manager fails to start because of missing front-proxy CA
certificate:
unable to load client CA file: unable to load client CA file: open
/etc/kubernetes/pki/front-proxy-ca.crt: no such file or directory
Added required certificate to fix this.
This should fixkubernetes/kubeadm#1281
Modified command line options --authentication-kubeconfig and
--authorization-kubeconfig to point out to the correct location
of the controller-manager.conf
This should fix this controller-manager crash:
failed to get delegated authentication kubeconfig: failed to get
delegated authentication kubeconfig: stat
/etc/kubernetes/controller-manager.conf: no such file or directory
Related issue: kubernetes/kubeadm#1281
Used T.Run API for kubeadm tests in app/phases/selfhosting and
app/phases/update directories
This should improve testing output and make it more visible
which test is doing what.
Selfhosting pivoting fails when using --store-certs-in-secrets
as api-server fails to start because of missing etcd/ca and
apiserver-etcd-client certificates:
F1227 16:01:52.237352 1 storage_decorator.go:57] Unable to create storage backend:
config (&{ /registry [https://127.0.0.1:2379]
/etc/kubernetes/pki/apiserver-etcd-client.key
/etc/kubernetes/pki/apiserver-etcd-client.crt
/etc/kubernetes/pki/etcd/ca.crt true 0xc000884120 <nil> 5m0s 1m0s}),
err (open /etc/kubernetes/pki/apiserver-etcd-client.crt: no such file or directory)
Added required certificates to fix this.
Secret name for etc/ca certifcate has been converted to conform RFC-1123 subdomain
naming conventions to prevent this TLS secret creation failure:
unable to create secret: Secret "etcd/ca" is invalid: metadata.name:
Invalid value: "etcd/ca": a DNS-1123 subdomain must consist of lower
case alphanumeric characters, '-' or '.', and must start and end with an
alphanumeric character (e.g. 'example.com', regex used for validation is
'[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
Related issue: kubernetes/kubeadm#1281
Moved token related flags to constants.
Token related flags refactored into constants.
Moved token related flags to constants.
Used new Consts flags in api's validation
Build deps fix
Moved token related flags to constants.
Used new Consts flags in api's validation
Build deps fix
Lint fix
Lint BUILD
Fixed the following golint warnigns:
- package_validator.go:68:1: receiver name should not be an underscore, omit the name if it is unused
- package_validator.go:89:1: receiver name should be a reflection of its identity; don't use generic names such as "this" or "self"
- package_validator.go:96:1: receiver name should be a reflection of its identity; don't use generic names such as "this" or "self"
- package_validator.go:119:1: receiver name should be a reflection of its identity; don't use generic names such as "this" or "self"
When golint is run against kubeadm it reports severel warnings like
redundant if ...; err != nil check, just return error instead.
Fix the warnings by just returning error.
Replaced hardcoded "v0.12.0" strings with MinimumControlPlaneVersion and
MinimumKubeletVersion global variables.
This should help with a regular release version bumps.
Currently the function `cmd/kubeadm/app/util.validateStableVersion()`
doesn't validate remote versions in the special case when the client
version is empty. This makes the code more difficult to reason about,
because the function may successfully return a string which isn't a valid version.
Move handling the special case outside of the function to the place
where its meaning is more obvious.
Despite we were checking for the kubelet kubeconfig file to be present, the
kubelet first writes this file and then the certificates the kubeconfig file
refers to. This represents a race condition in kubeadm in which when we confirm
that the kubelet's kubeconfig file is present we continue creating a clientset
out of it. However, the clientset creation will ensure that the certificates the
kubeconfig file refers to exist on the filesystem.
To fix this problem, not only wait for the kubelet's kubeconfig file to be
present, but also ensure that we can create a clientset ouf of it on our polling
process, while we wait for the kubelet to have performed the TLS bootstrap.
If we let the hostpath with DirectoryOrCreate to create this directory
it defaults to 0755. A default install should use 0700 for better
security especially if the directory is not present.
Change-Id: Idc0266685895767b0d1c5710c8a4fb704805652f
'kubeadm reset' uses incorrect way of unmounting /var/lib/kubelet
directories. It queries /proc/mounts for /var/lib/kubelet mount point.
If /var/lib/kubelet directory is also mounted it makes 'kubelet reset'
to unmount it too, which is incorrect. It also makes it fail as it
can't unmount /var/lib/kubelet before unmounting mounts inside it.
Fixed by querying /var/lib/kubelet/ instead of /var/lib/kubelet.
This should exclude /var/lib/kubelet from the query results even if
it's mounted.
Fixes: kubernetes/kubeadm#1294
As localApiEndpoint is ignored by json.Unmarshaller this should be
localAPIEndpoint as defined in the json tag of the source code.
Signed-off-by: André Martins <aanm90@gmail.com>
'kubeadm init' silently ignores --node-name and --cri-socket
command line options if --config option is specified.
Implemented setting 'name' and 'criSocket' options from the command
line even if --config command line option is used.
kubeadm config migrate uses AnyConfigFileAndDefaultsToInternal, which can
unmarshal config from file only if InitConfiguration or JoinConfiguration are
present. Even with that in mind, it can only return a singlie config object,
with InitConfiguration taking precendence over JoinConfiguration. Thus, the
following cases were not handled properly, while they were perfectly valid for
kubeadm init/join:
- ClusterConfiguration only file caused kubeadm config migrate to exit with
error.
- Init + Join configurations in the same file caused Init + Cluster
configuration to be produced (ignoring JoinConfiguration). The same is valid
when the combo is Init + Cluster + Join configurations.
- Cluster + Join configuration ignores ClusterConfiguration and only
JoinConfiguration gets migrated.
To fix this, the following is done:
- Introduce MigrateOldConfigFromFile which migrates old config from a file,
while ensuring that all kubeadm originated input config kinds are taken care
of. Add comprehensive unit tests for this.
- Replace the use of AnyConfigFileAndDefaultsToInternal in
kubeadm config migrate with MigrateOldConfigFromFile.
- Remove the no longer used and error prone AnyConfigFileAndDefaultsToInternal.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
'kubeadm join' silently ignores --node-name and --cri-socket
command line options if --config option is specified.
In some cases it's much easier for users to override these parameters
from the command line instead of updating config, especially for
multi-node automatic deployments where only node name should be changed.
Implemented setting 'name' and 'criSocket' options from the command
line even if --config command line option is used.
Bump MinimumControlPlaneVersion and MinimumKubeletVersion to v1.12 and update
any related tests.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Added hostport and host/port parts to the error messages
produced by ParseHostPort API.
This should help users to better identify parsing issues.
Fixes: kubernetes/kubeadm#1159
- make the whole config example actually pass strict verification
(i.e. make the config work with --config)
- add print init-defaults/join defaults
- other small fixes
When 'kubeadm init ...' is used with an IPv6 kubeadm configuration,
kubeadm currently generates an etcd.yaml manifest that uses IP:port
combinatins where the IP is an IPv6 address, but it is not enclosed
in square brackets, e.g.:
- --advertise-client-urls=https://fd00:20::2:2379
For IPv6 advertise addresses, this should be of the form:
- --advertise-client-urls=https://[fd00:20::2]:2379
The lack of brackets around IPv6 addresses in cases like this is
causing failures to bring up IPv6-only clusters with Kubeadm as
described in kubernetes/kubeadm Issues #1212.
This format error is fixed by using net.JoinHostPort() to generate
URLs as shown above.
Fixes kubernetes/kubeadm Issue #1212
SataQiu