github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge pull request #54823 from mtaufen/structure-eviction-thresholds 

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Lift embedded structure out of eviction-related KubeletConfiguration fields

- Changes the following KubeletConfiguration fields from `string` to
`map[string]string`:
  - `EvictionHard`
  - `EvictionSoft`
  - `EvictionSoftGracePeriod`
  - `EvictionMinimumReclaim`
- Adds flag parsing shims to maintain Kubelet's public flags API, while
enabling structured input in the file API.
- Also removes `kubeletconfig.ConfigurationMap`, which was an ad-hoc flag
parsing shim living in the kubeletconfig API group, and replaces it
with the `MapStringString` shim introduced in this PR. Flag parsing
shims belong in a common place, not in the kubeletconfig API.
I manually audited these to ensure that this wouldn't cause errors
parsing the command line for syntax that would have previously been
error free (`kubeletconfig.ConfigurationMap` was unique in that it
allowed keys to be provided on the CLI without values. I believe this was
done in `flags.ConfigurationMap` to facilitate the `--node-labels` flag,
which rightfully accepts value-free keys, and that this shim was then
just copied to `kubeletconfig`). Fortunately, the affected fields
(`ExperimentalQOSReserved`, `SystemReserved`, and `KubeReserved`) expect
non-empty strings in the values of the map, and as a result passing the
empty string is already an error. Thus requiring keys shouldn't break
anyone's scripts.
- Updates code and tests accordingly.

Regarding eviction operators, directionality is already implicit in the
signal type (for a given signal, the decision to evict will be made when
crossing the threshold from either above or below, never both). There is
no need to expose an operator, such as `<`, in the API. By changing
`EvictionHard` and `EvictionSoft` to `map[string]string`, this PR
simplifies the experience of working with these fields via the
`KubeletConfiguration` type. Again, flags stay the same.

Other things:
- There is another flag parsing shim, `flags.ConfigurationMap`, from the
shared flag utility. The `NodeLabels` field still uses
`flags.ConfigurationMap`. This PR moves the allocation of the
`map[string]string` for the `NodeLabels` field from
`AddKubeletConfigFlags` to the defaulter for the external
`KubeletConfiguration` type. Flags are layered on top of an internal
object that has undergone conversion from a defaulted external object,
which means that previously the mere registration of flags would have
overwritten any previously-defined defaults for `NodeLabels` (fortunately
there were none).

Related: #53833 (lifting embedded structures out of string fields is part of getting this API to beta)

```release-note
The EvictionHard, EvictionSoft, EvictionSoftGracePeriod, EvictionMinimumReclaim, SystemReserved, and KubeReserved fields in the KubeletConfiguration object (kubeletconfig/v1alpha1) are now of type map[string]string, which facilitates writing JSON and YAML files.
```
pull/6/head
Kubernetes Submit Queue 2017-11-17 02:57:30 -08:00 committed by GitHub
parent b223955c06 617b49858f
commit 00fe2cfe6c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
30 changed files with 932 additions and 356 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
cmd/kubelet/app/BUILD
View File

@ -11,7 +11,6 @@ go_test(
srcs = ["server_test.go"],
importpath = "k8s.io/kubernetes/cmd/kubelet/app",
library = ":go_default_library",
deps = ["//pkg/kubelet/apis/kubeletconfig:go_default_library"],
)
go_library(

18
cmd/kubelet/app/options/options.go
View File

@ -139,7 +139,7 @@ type KubeletFlags struct {
// A set of ResourceName=Percentage (e.g. memory=50%) pairs that describe
// how pod resource requests are reserved at the QoS level.
// Currently only memory is supported. [default=none]"
ExperimentalQOSReserved kubeletconfig.ConfigurationMap
ExperimentalQOSReserved map[string]string
// Node Labels are the node labels to add when registering the node in the cluster
NodeLabels map[string]string
// volumePluginDir is the full path of the directory in which to search
@ -332,7 +332,7 @@ func (f *KubeletFlags) AddFlags(fs *pflag.FlagSet) {
fs.StringVar(&f.RemoteImageEndpoint, "image-service-endpoint", f.RemoteImageEndpoint, "[Experimental] The endpoint of remote image service. If not specified, it will be the same with container-runtime-endpoint by default. Currently unix socket is supported on Linux, and tcp is supported on windows. Examples:'unix:///var/run/dockershim.sock', 'tcp://localhost:3735'")
fs.BoolVar(&f.ExperimentalCheckNodeCapabilitiesBeforeMount, "experimental-check-node-capabilities-before-mount", f.ExperimentalCheckNodeCapabilitiesBeforeMount, "[Experimental] if set true, the kubelet will check the underlying node for required componenets (binaries, etc.) before performing the mount")
fs.BoolVar(&f.ExperimentalNodeAllocatableIgnoreEvictionThreshold, "experimental-allocatable-ignore-eviction", f.ExperimentalNodeAllocatableIgnoreEvictionThreshold, "When set to 'true', Hard Eviction Thresholds will be ignored while calculating Node Allocatable. See https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ for more details. [default=false]")
fs.Var(&f.ExperimentalQOSReserved, "experimental-qos-reserved", "A set of ResourceName=Percentage (e.g. memory=50%) pairs that describe how pod resource requests are reserved at the QoS level. Currently only memory is supported. [default=none]")
fs.Var(flag.NewMapStringString(&f.ExperimentalQOSReserved), "experimental-qos-reserved", "A set of ResourceName=Percentage (e.g. memory=50%) pairs that describe how pod resource requests are reserved at the QoS level. Currently only memory is supported. [default=none]")
bindableNodeLabels := flag.ConfigurationMap(f.NodeLabels)
fs.Var(&bindableNodeLabels, "node-labels", "<Warning: Alpha feature> Labels to add when registering the node in the cluster. Labels must be key=value pairs separated by ','.")
fs.StringVar(&f.VolumePluginDir, "volume-plugin-dir", f.VolumePluginDir, "<Warning: Alpha feature> The full path of the directory in which to search for additional third party volume plugins")
@ -430,7 +430,7 @@ func AddKubeletConfigFlags(fs *pflag.FlagSet, c *kubeletconfig.KubeletConfigurat
fs.Int32Var(&c.ImageGCHighThresholdPercent, "image-gc-high-threshold", c.ImageGCHighThresholdPercent, "The percent of disk usage after which image garbage collection is always run.")
fs.Int32Var(&c.ImageGCLowThresholdPercent, "image-gc-low-threshold", c.ImageGCLowThresholdPercent, "The percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to.")
fs.DurationVar(&c.VolumeStatsAggPeriod.Duration, "volume-stats-agg-period", c.VolumeStatsAggPeriod.Duration, "Specifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes. To disable volume calculations, set to 0.")
fs.Var(flag.MapStringBool(c.FeatureGates), "feature-gates", "A set of key=value pairs that describe feature gates for alpha/experimental features. "+
fs.Var(flag.NewMapStringBool(&c.FeatureGates), "feature-gates", "A set of key=value pairs that describe feature gates for alpha/experimental features. "+
"Options are:\n"+strings.Join(utilfeature.DefaultFeatureGate.KnownFeatures(), "\n"))
fs.StringVar(&c.KubeletCgroups, "kubelet-cgroups", c.KubeletCgroups, "Optional absolute name of cgroups to create and run the Kubelet in.")
fs.StringVar(&c.SystemCgroups, "system-cgroups", c.SystemCgroups, "Optional absolute name of cgroups in which to place all non-kernel processes that are not already inside a cgroup under `/`. Empty for no container. Rolling back the flag requires a reboot.")
@ -460,18 +460,18 @@ func AddKubeletConfigFlags(fs *pflag.FlagSet, c *kubeletconfig.KubeletConfigurat
fs.Int32Var(&c.KubeAPIBurst, "kube-api-burst", c.KubeAPIBurst, "Burst to use while talking with kubernetes apiserver")
fs.BoolVar(&c.SerializeImagePulls, "serialize-image-pulls", c.SerializeImagePulls, "Pull images one at a time. We recommend *not* changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details.")
fs.StringVar(&c.EvictionHard, "eviction-hard", c.EvictionHard, "A set of eviction thresholds (e.g. memory.available<1Gi) that if met would trigger a pod eviction.")
fs.StringVar(&c.EvictionSoft, "eviction-soft", c.EvictionSoft, "A set of eviction thresholds (e.g. memory.available<1.5Gi) that if met over a corresponding grace period would trigger a pod eviction.")
fs.StringVar(&c.EvictionSoftGracePeriod, "eviction-soft-grace-period", c.EvictionSoftGracePeriod, "A set of eviction grace periods (e.g. memory.available=1m30s) that correspond to how long a soft eviction threshold must hold before triggering a pod eviction.")
fs.Var(flag.NewLangleSeparatedMapStringString(&c.EvictionHard), "eviction-hard", "A set of eviction thresholds (e.g. memory.available<1Gi) that if met would trigger a pod eviction.")
fs.Var(flag.NewLangleSeparatedMapStringString(&c.EvictionSoft), "eviction-soft", "A set of eviction thresholds (e.g. memory.available<1.5Gi) that if met over a corresponding grace period would trigger a pod eviction.")
fs.Var(flag.NewMapStringString(&c.EvictionSoftGracePeriod), "eviction-soft-grace-period", "A set of eviction grace periods (e.g. memory.available=1m30s) that correspond to how long a soft eviction threshold must hold before triggering a pod eviction.")
fs.DurationVar(&c.EvictionPressureTransitionPeriod.Duration, "eviction-pressure-transition-period", c.EvictionPressureTransitionPeriod.Duration, "Duration for which the kubelet has to wait before transitioning out of an eviction pressure condition.")
fs.Int32Var(&c.EvictionMaxPodGracePeriod, "eviction-max-pod-grace-period", c.EvictionMaxPodGracePeriod, "Maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. If negative, defer to pod specified value.")
fs.StringVar(&c.EvictionMinimumReclaim, "eviction-minimum-reclaim", c.EvictionMinimumReclaim, "A set of minimum reclaims (e.g. imagefs.available=2Gi) that describes the minimum amount of resource the kubelet will reclaim when performing a pod eviction if that resource is under pressure.")
fs.Var(flag.NewMapStringString(&c.EvictionMinimumReclaim), "eviction-minimum-reclaim", "A set of minimum reclaims (e.g. imagefs.available=2Gi) that describes the minimum amount of resource the kubelet will reclaim when performing a pod eviction if that resource is under pressure.")
fs.Int32Var(&c.PodsPerCore, "pods-per-core", c.PodsPerCore, "Number of Pods per core that can run on this Kubelet. The total number of Pods on this Kubelet cannot exceed max-pods, so max-pods will be used if this calculation results in a larger number of Pods allowed on the Kubelet. A value of 0 disables this limit.")
fs.BoolVar(&c.ProtectKernelDefaults, "protect-kernel-defaults", c.ProtectKernelDefaults, "Default kubelet behaviour for kernel tuning. If set, kubelet errors if any of kernel tunables is different than kubelet defaults.")
// Node Allocatable Flags
fs.Var(&c.SystemReserved, "system-reserved", "A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]")
fs.Var(&c.KubeReserved, "kube-reserved", "A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for kubernetes system components. Currently cpu, memory and local ephemeral storage for root file system are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]")
fs.Var(flag.NewMapStringString(&c.SystemReserved), "system-reserved", "A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]")
fs.Var(flag.NewMapStringString(&c.KubeReserved), "kube-reserved", "A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=500Mi,ephemeral-storage=1Gi) pairs that describe resources reserved for kubernetes system components. Currently cpu, memory and local ephemeral storage for root file system are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]")
fs.StringSliceVar(&c.EnforceNodeAllocatable, "enforce-node-allocatable", c.EnforceNodeAllocatable, "A comma separated list of levels of node allocatable enforcement to be enforced by kubelet. Acceptible options are 'pods', 'system-reserved' & 'kube-reserved'. If the latter two options are specified, '--system-reserved-cgroup' & '--kube-reserved-cgroup' must also be set respectively. See https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ for more details.")
fs.StringVar(&c.SystemReservedCgroup, "system-reserved-cgroup", c.SystemReservedCgroup, "Absolute name of the top level cgroup that is used to manage non-kubernetes components for which compute resources were reserved via '--system-reserved' flag. Ex. '/system-reserved'. [default='']")
fs.StringVar(&c.KubeReservedCgroup, "kube-reserved-cgroup", c.KubeReservedCgroup, "Absolute name of the top level cgroup that is used to manage kubernetes components for which compute resources were reserved via '--kube-reserved' flag. Ex. '/kube-reserved'. [default='']")

6
cmd/kubelet/app/options/options_test.go
View File

@ -120,7 +120,12 @@ func asArgs(fn, defaultFn func(*pflag.FlagSet)) []string {
defaultFn(defaults)
var args []string
fs.VisitAll(func(flag *pflag.Flag) {
// if the flag implements utilflag.OmitEmpty and the value is Empty, then just omit it from the command line
if omit, ok := flag.Value.(utilflag.OmitEmpty); ok && omit.Empty() {
return
}
s := flag.Value.String()
// if the flag has the same value as the default, we can omit it without changing the meaning of the command line
var defaultValue string
if defaultFlag := defaults.Lookup(flag.Name); defaultFlag != nil {
defaultValue = defaultFlag.Value.String()
@ -128,6 +133,7 @@ func asArgs(fn, defaultFn func(*pflag.FlagSet)) []string {
return
}
}
// if the flag is a string slice, each element is specified with an independent flag invocation
if values, err := fs.GetStringSlice(flag.Name); err == nil {
for _, s := range values {
args = append(args, fmt.Sprintf("--%s=%s", flag.Name, s))

4
cmd/kubelet/app/server.go
View File

@ -439,7 +439,7 @@ func run(s *options.KubeletServer, kubeDeps *kubelet.Dependencies) (err error) {
var hardEvictionThresholds []evictionapi.Threshold
// If the user requested to ignore eviction thresholds, then do not set valid values for hardEvictionThresholds here.
if !s.ExperimentalNodeAllocatableIgnoreEvictionThreshold {
hardEvictionThresholds, err = eviction.ParseThresholdConfig([]string{}, s.EvictionHard, "", "", "")
hardEvictionThresholds, err = eviction.ParseThresholdConfig([]string{}, s.EvictionHard, nil, nil, nil)
if err != nil {
return err
}
@ -846,7 +846,7 @@ func CreateAndInitKubelet(kubeCfg *kubeletconfiginternal.KubeletConfiguration,
// parseResourceList parses the given configuration map into an API
// ResourceList or returns an error.
func parseResourceList(m kubeletconfiginternal.ConfigurationMap) (v1.ResourceList, error) {
func parseResourceList(m map[string]string) (v1.ResourceList, error) {
if len(m) == 0 {
return nil, nil
}

28
cmd/kubelet/app/server_test.go
View File

@ -18,46 +18,38 @@ package app
import (
"testing"
"k8s.io/kubernetes/pkg/kubelet/apis/kubeletconfig"
)
func TestValueOfAllocatableResources(t *testing.T) {
testCases := []struct {
kubeReserved string
systemReserved string
kubeReserved map[string]string
systemReserved map[string]string
errorExpected bool
name string
}{
{
kubeReserved: "cpu=200m,memory=-150G,ephemeral-storage=10Gi",
systemReserved: "cpu=200m,memory=15Ki",
kubeReserved: map[string]string{"cpu": "200m", "memory": "-150G", "ephemeral-storage": "10Gi"},
systemReserved: map[string]string{"cpu": "200m", "memory": "15Ki"},
errorExpected: true,
name: "negative quantity value",
},
{
kubeReserved: "cpu=200m,memory=150Gi,ephemeral-storage=10Gi",
systemReserved: "cpu=200m,memory=15Ky",
kubeReserved: map[string]string{"cpu": "200m", "memory": "150Gi", "ephemeral-storage": "10Gi"},
systemReserved: map[string]string{"cpu": "200m", "memory": "15Ky"},
errorExpected: true,
name: "invalid quantity unit",
},
{
kubeReserved: "cpu=200m,memory=15G,ephemeral-storage=10Gi",
systemReserved: "cpu=200m,memory=15Ki",
kubeReserved: map[string]string{"cpu": "200m", "memory": "15G", "ephemeral-storage": "10Gi"},
systemReserved: map[string]string{"cpu": "200m", "memory": "15Ki"},
errorExpected: false,
name: "Valid resource quantity",
},
}
for _, test := range testCases {
kubeReservedCM := make(kubeletconfig.ConfigurationMap)
systemReservedCM := make(kubeletconfig.ConfigurationMap)
kubeReservedCM.Set(test.kubeReserved)
systemReservedCM.Set(test.systemReserved)
_, err1 := parseResourceList(kubeReservedCM)
_, err2 := parseResourceList(systemReservedCM)
_, err1 := parseResourceList(test.kubeReserved)
_, err2 := parseResourceList(test.systemReserved)
if test.errorExpected {
if err1 == nil && err2 == nil {
t.Errorf("%s: error expected", test.name)

8
pkg/kubelet/apis/kubeletconfig/helpers_test.go
View File

@ -164,12 +164,12 @@ var (
"EnforceNodeAllocatable[*]",
"EventBurst",
"EventRecordQPS",
"EvictionHard",
"EvictionHard[*]",
"EvictionMaxPodGracePeriod",
"EvictionMinimumReclaim",
"EvictionMinimumReclaim[*]",
"EvictionPressureTransitionPeriod.Duration",
"EvictionSoft",
"EvictionSoftGracePeriod",
"EvictionSoft[*]",
"EvictionSoftGracePeriod[*]",
"FailSwapOn",
"FeatureGates[*]",
"FileCheckFrequency.Duration",

56
pkg/kubelet/apis/kubeletconfig/types.go
View File

@ -17,10 +17,6 @@ limitations under the License.
package kubeletconfig
import (
"fmt"
"sort"
"strings"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
@ -226,24 +222,26 @@ type KubeletConfiguration struct {
// run docker daemon with version < 1.9 or an Aufs storage backend.
// Issue #10959 has more details.
SerializeImagePulls bool
// Comma-delimited list of hard eviction expressions. For example, 'memory.available<300Mi'.
// Map of signal names to quantities that defines hard eviction thresholds. For example: {"memory.available": "300Mi"}.
// +optional
EvictionHard string
// Comma-delimited list of soft eviction expressions. For example, 'memory.available<300Mi'.
EvictionHard map[string]string
// Map of signal names to quantities that defines soft eviction thresholds. For example: {"memory.available": "300Mi"}.
// +optional
EvictionSoft string
// Comma-delimeted list of grace periods for each soft eviction signal. For example, 'memory.available=30s'.
EvictionSoft map[string]string
// Map of signal names to quantities that defines grace periods for each soft eviction signal. For example: {"memory.available": "30s"}.
// +optional
EvictionSoftGracePeriod string
EvictionSoftGracePeriod map[string]string
// Duration for which the kubelet has to wait before transitioning out of an eviction pressure condition.
// +optional
EvictionPressureTransitionPeriod metav1.Duration
// Maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met.
// +optional
EvictionMaxPodGracePeriod int32
// Comma-delimited list of minimum reclaims (e.g. imagefs.available=2Gi) that describes the minimum amount of resource the kubelet will reclaim when performing a pod eviction if that resource is under pressure.
// Map of signal names to quantities that defines minimum reclaims, which describe the minimum
// amount of a given resource the kubelet will reclaim when performing a pod eviction while
// that resource is under pressure. For example: {"imagefs.available": "2Gi"}
// +optional
EvictionMinimumReclaim string
EvictionMinimumReclaim map[string]string
// Maximum number of pods per core. Cannot exceed MaxPods
PodsPerCore int32
// enableControllerAttachDetach enables the Attach/Detach controller to
@ -275,12 +273,12 @@ type KubeletConfiguration struct {
// that describe resources reserved for non-kubernetes components.
// Currently only cpu and memory are supported. [default=none]
// See http://kubernetes.io/docs/user-guide/compute-resources for more detail.
SystemReserved ConfigurationMap
SystemReserved map[string]string
// A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs
// that describe resources reserved for kubernetes system components.
// Currently cpu, memory and local ephemeral storage for root file system are supported. [default=none]
// See http://kubernetes.io/docs/user-guide/compute-resources for more detail.
KubeReserved ConfigurationMap
KubeReserved map[string]string
// This flag helps kubelet identify absolute name of top level cgroup used to enforce `SystemReserved` compute resource reservation for OS system daemons.
// Refer to [Node Allocatable](https://git.k8s.io/community/contributors/design-proposals/node/node-allocatable.md) doc for more information.
SystemReservedCgroup string
@ -348,33 +346,3 @@ type KubeletAnonymousAuthentication struct {
// Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated.
Enabled bool
}
type ConfigurationMap map[string]string
func (m *ConfigurationMap) String() string {
pairs := []string{}
for k, v := range *m {
pairs = append(pairs, fmt.Sprintf("%s=%s", k, v))
}
sort.Strings(pairs)
return strings.Join(pairs, ",")
}
func (m *ConfigurationMap) Set(value string) error {
for _, s := range strings.Split(value, ",") {
if len(s) == 0 {
continue
}
arr := strings.SplitN(s, "=", 2)
if len(arr) == 2 {
(*m)[strings.TrimSpace(arr[0])] = strings.TrimSpace(arr[1])
} else {
(*m)[strings.TrimSpace(arr[0])] = ""
}
}
return nil
}
func (*ConfigurationMap) Type() string {
return "mapStringString"
}

17
pkg/kubelet/apis/kubeletconfig/v1alpha1/defaults.go
View File

@ -200,18 +200,16 @@ func SetDefaults_KubeletConfiguration(obj *KubeletConfiguration) {
obj.HairpinMode = PromiscuousBridge
}
if obj.EvictionHard == nil {
temp := "memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%,imagefs.available<15%"
obj.EvictionHard = &temp
obj.EvictionHard = map[string]string{
"memory.available": "100Mi",
"nodefs.available": "10%",
"nodefs.inodesFree": "5%",
"imagefs.available": "15%",
}
}
if obj.EvictionPressureTransitionPeriod == zeroDuration {
obj.EvictionPressureTransitionPeriod = metav1.Duration{Duration: 5 * time.Minute}
}
if obj.SystemReserved == nil {
obj.SystemReserved = make(map[string]string)
}
if obj.KubeReserved == nil {
obj.KubeReserved = make(map[string]string)
}
if obj.MakeIPTablesUtilChains == nil {
obj.MakeIPTablesUtilChains = boolVar(true)
}
@ -233,9 +231,6 @@ func SetDefaults_KubeletConfiguration(obj *KubeletConfiguration) {
if obj.EnforceNodeAllocatable == nil {
obj.EnforceNodeAllocatable = defaultNodeAllocatableEnforcement
}
if obj.FeatureGates == nil {
obj.FeatureGates = make(map[string]bool)
}
if obj.ManifestURLHeader == nil {
obj.ManifestURLHeader = make(map[string][]string)
}

22
pkg/kubelet/apis/kubeletconfig/v1alpha1/types.go
View File

@ -219,18 +219,24 @@ type KubeletConfiguration struct {
// run docker daemon with version < 1.9 or an Aufs storage backend.
// Issue #10959 has more details.
SerializeImagePulls *bool `json:"serializeImagePulls"`
// Comma-delimited list of hard eviction expressions. For example, 'memory.available<300Mi'.
EvictionHard *string `json:"evictionHard"`
// Comma-delimited list of soft eviction expressions. For example, 'memory.available<300Mi'.
EvictionSoft string `json:"evictionSoft"`
// Comma-delimeted list of grace periods for each soft eviction signal. For example, 'memory.available=30s'.
EvictionSoftGracePeriod string `json:"evictionSoftGracePeriod"`
// Map of signal names to quantities that defines hard eviction thresholds. For example: {"memory.available": "300Mi"}.
// +optional
EvictionHard map[string]string `json:"evictionHard"`
// Map of signal names to quantities that defines soft eviction thresholds. For example: {"memory.available": "300Mi"}.
// +optional
EvictionSoft map[string]string `json:"evictionSoft"`
// Map of signal names to quantities that defines grace periods for each soft eviction signal. For example: {"memory.available": "30s"}.
// +optional
EvictionSoftGracePeriod map[string]string `json:"evictionSoftGracePeriod"`
// Duration for which the kubelet has to wait before transitioning out of an eviction pressure condition.
EvictionPressureTransitionPeriod metav1.Duration `json:"evictionPressureTransitionPeriod"`
// Maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met.
EvictionMaxPodGracePeriod int32 `json:"evictionMaxPodGracePeriod"`
// Comma-delimited list of minimum reclaims (e.g. imagefs.available=2Gi) that describes the minimum amount of resource the kubelet will reclaim when performing a pod eviction if that resource is under pressure.
EvictionMinimumReclaim string `json:"evictionMinimumReclaim"`
// Map of signal names to quantities that defines minimum reclaims, which describe the minimum
// amount of a given resource the kubelet will reclaim when performing a pod eviction while
// that resource is under pressure. For example: {"imagefs.available": "2Gi"}
// +optional
EvictionMinimumReclaim map[string]string `json:"evictionMinimumReclaim"`
// Maximum number of pods per core. Cannot exceed MaxPods
PodsPerCore int32 `json:"podsPerCore"`
// enableControllerAttachDetach enables the Attach/Detach controller to

24
pkg/kubelet/apis/kubeletconfig/v1alpha1/zz_generated.conversion.go generated
View File

@ -230,14 +230,12 @@ func autoConvert_v1alpha1_KubeletConfiguration_To_kubeletconfig_KubeletConfigura
if err := v1.Convert_Pointer_bool_To_bool(&in.SerializeImagePulls, &out.SerializeImagePulls, s); err != nil {
return err
}
if err := v1.Convert_Pointer_string_To_string(&in.EvictionHard, &out.EvictionHard, s); err != nil {
return err
}
out.EvictionSoft = in.EvictionSoft
out.EvictionSoftGracePeriod = in.EvictionSoftGracePeriod
out.EvictionHard = *(*map[string]string)(unsafe.Pointer(&in.EvictionHard))
out.EvictionSoft = *(*map[string]string)(unsafe.Pointer(&in.EvictionSoft))
out.EvictionSoftGracePeriod = *(*map[string]string)(unsafe.Pointer(&in.EvictionSoftGracePeriod))
out.EvictionPressureTransitionPeriod = in.EvictionPressureTransitionPeriod
out.EvictionMaxPodGracePeriod = in.EvictionMaxPodGracePeriod
out.EvictionMinimumReclaim = in.EvictionMinimumReclaim
out.EvictionMinimumReclaim = *(*map[string]string)(unsafe.Pointer(&in.EvictionMinimumReclaim))
out.PodsPerCore = in.PodsPerCore
if err := v1.Convert_Pointer_bool_To_bool(&in.EnableControllerAttachDetach, &out.EnableControllerAttachDetach, s); err != nil {
return err
@ -254,8 +252,8 @@ func autoConvert_v1alpha1_KubeletConfiguration_To_kubeletconfig_KubeletConfigura
}
out.FeatureGates = *(*map[string]bool)(unsafe.Pointer(&in.FeatureGates))
out.FailSwapOn = in.FailSwapOn
out.SystemReserved = *(*kubeletconfig.ConfigurationMap)(unsafe.Pointer(&in.SystemReserved))
out.KubeReserved = *(*kubeletconfig.ConfigurationMap)(unsafe.Pointer(&in.KubeReserved))
out.SystemReserved = *(*map[string]string)(unsafe.Pointer(&in.SystemReserved))
out.KubeReserved = *(*map[string]string)(unsafe.Pointer(&in.KubeReserved))
out.SystemReservedCgroup = in.SystemReservedCgroup
out.KubeReservedCgroup = in.KubeReservedCgroup
out.EnforceNodeAllocatable = *(*[]string)(unsafe.Pointer(&in.EnforceNodeAllocatable))
@ -358,14 +356,12 @@ func autoConvert_kubeletconfig_KubeletConfiguration_To_v1alpha1_KubeletConfigura
if err := v1.Convert_bool_To_Pointer_bool(&in.SerializeImagePulls, &out.SerializeImagePulls, s); err != nil {
return err
}
if err := v1.Convert_string_To_Pointer_string(&in.EvictionHard, &out.EvictionHard, s); err != nil {
return err
}
out.EvictionSoft = in.EvictionSoft
out.EvictionSoftGracePeriod = in.EvictionSoftGracePeriod
out.EvictionHard = *(*map[string]string)(unsafe.Pointer(&in.EvictionHard))
out.EvictionSoft = *(*map[string]string)(unsafe.Pointer(&in.EvictionSoft))
out.EvictionSoftGracePeriod = *(*map[string]string)(unsafe.Pointer(&in.EvictionSoftGracePeriod))
out.EvictionPressureTransitionPeriod = in.EvictionPressureTransitionPeriod
out.EvictionMaxPodGracePeriod = in.EvictionMaxPodGracePeriod
out.EvictionMinimumReclaim = in.EvictionMinimumReclaim
out.EvictionMinimumReclaim = *(*map[string]string)(unsafe.Pointer(&in.EvictionMinimumReclaim))
out.PodsPerCore = in.PodsPerCore
if err := v1.Convert_bool_To_Pointer_bool(&in.EnableControllerAttachDetach, &out.EnableControllerAttachDetach, s); err != nil {
return err

29
pkg/kubelet/apis/kubeletconfig/v1alpha1/zz_generated.deepcopy.go generated
View File

@ -279,14 +279,33 @@ func (in *KubeletConfiguration) DeepCopyInto(out *KubeletConfiguration) {
}
if in.EvictionHard != nil {
in, out := &in.EvictionHard, &out.EvictionHard
if *in == nil {
*out = nil
} else {
*out = new(string)
**out = **in
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
if in.EvictionSoft != nil {
in, out := &in.EvictionSoft, &out.EvictionSoft
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
if in.EvictionSoftGracePeriod != nil {
in, out := &in.EvictionSoftGracePeriod, &out.EvictionSoftGracePeriod
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
out.EvictionPressureTransitionPeriod = in.EvictionPressureTransitionPeriod
if in.EvictionMinimumReclaim != nil {
in, out := &in.EvictionMinimumReclaim, &out.EvictionMinimumReclaim
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
if in.EnableControllerAttachDetach != nil {
in, out := &in.EnableControllerAttachDetach, &out.EnableControllerAttachDetach
if *in == nil {

32
pkg/kubelet/apis/kubeletconfig/zz_generated.deepcopy.go generated
View File

@ -133,7 +133,35 @@ func (in *KubeletConfiguration) DeepCopyInto(out *KubeletConfiguration) {
out.VolumeStatsAggPeriod = in.VolumeStatsAggPeriod
out.CPUManagerReconcilePeriod = in.CPUManagerReconcilePeriod
out.RuntimeRequestTimeout = in.RuntimeRequestTimeout
if in.EvictionHard != nil {
in, out := &in.EvictionHard, &out.EvictionHard
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
if in.EvictionSoft != nil {
in, out := &in.EvictionSoft, &out.EvictionSoft
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
if in.EvictionSoftGracePeriod != nil {
in, out := &in.EvictionSoftGracePeriod, &out.EvictionSoftGracePeriod
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
out.EvictionPressureTransitionPeriod = in.EvictionPressureTransitionPeriod
if in.EvictionMinimumReclaim != nil {
in, out := &in.EvictionMinimumReclaim, &out.EvictionMinimumReclaim
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
if in.FeatureGates != nil {
in, out := &in.FeatureGates, &out.FeatureGates
*out = make(map[string]bool, len(*in))
@ -143,14 +171,14 @@ func (in *KubeletConfiguration) DeepCopyInto(out *KubeletConfiguration) {
}
if in.SystemReserved != nil {
in, out := &in.SystemReserved, &out.SystemReserved
*out = make(ConfigurationMap, len(*in))
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}
}
if in.KubeReserved != nil {
in, out := &in.KubeReserved, &out.KubeReserved
*out = make(ConfigurationMap, len(*in))
*out = make(map[string]string, len(*in))
for key, val := range *in {
(*out)[key] = val
}

2
pkg/kubelet/cm/BUILD
View File

@ -32,7 +32,6 @@ go_library(
deps = [
"//pkg/features:go_default_library",
"//pkg/kubelet/apis/cri:go_default_library",
"//pkg/kubelet/apis/kubeletconfig:go_default_library",
"//pkg/kubelet/cadvisor:go_default_library",
"//pkg/kubelet/cm/cpumanager:go_default_library",
"//pkg/kubelet/container:go_default_library",
@ -95,7 +94,6 @@ go_test(
"//pkg/util/mount:go_default_library",
] + select({
"@io_bazel_rules_go//go/platform:linux_amd64": [
"//pkg/kubelet/apis/kubeletconfig:go_default_library",
"//pkg/kubelet/eviction/api:go_default_library",
"//vendor/github.com/stretchr/testify/assert:go_default_library",
"//vendor/github.com/stretchr/testify/require:go_default_library",

21
pkg/kubelet/cm/cgroup_manager_test.go
View File

@ -23,52 +23,49 @@ import (
"testing"
"k8s.io/api/core/v1"
"k8s.io/kubernetes/pkg/kubelet/apis/kubeletconfig"
)
func Test(t *testing.T) {
tests := []struct {
input string
input map[string]string
expected *map[v1.ResourceName]int64
}{
{
input: "memory",
input: map[string]string{"memory": ""},
expected: nil,
},
{
input: "memory=a",
input: map[string]string{"memory": "a"},
expected: nil,
},
{
input: "memory=a%",
input: map[string]string{"memory": "a%"},
expected: nil,
},
{
input: "memory=200%",
input: map[string]string{"memory": "200%"},
expected: nil,
},
{
input: "memory=0%",
input: map[string]string{"memory": "0%"},
expected: &map[v1.ResourceName]int64{
v1.ResourceMemory: 0,
},
},
{
input: "memory=100%",
input: map[string]string{"memory": "100%"},
expected: &map[v1.ResourceName]int64{
v1.ResourceMemory: 100,
},
},
{
// need to change this when CPU is added as a supported resource
input: "memory=100%,cpu=50%",
input: map[string]string{"memory": "100%", "cpu": "50%"},
expected: nil,
},
}
for _, test := range tests {
m := kubeletconfig.ConfigurationMap{}
m.Set(test.input)
actual, err := ParseQOSReserved(m)
actual, err := ParseQOSReserved(test.input)
if actual != nil && test.expected == nil {
t.Errorf("Unexpected success, input: %v, expected: %v, actual: %v, err: %v", test.input, test.expected, actual, err)
}

3
pkg/kubelet/cm/container_manager.go
View File

@ -23,7 +23,6 @@ import (
// TODO: Migrate kubelet to either use its own internal objects or client library.
"k8s.io/api/core/v1"
internalapi "k8s.io/kubernetes/pkg/kubelet/apis/cri"
"k8s.io/kubernetes/pkg/kubelet/apis/kubeletconfig"
kubecontainer "k8s.io/kubernetes/pkg/kubelet/container"
evictionapi "k8s.io/kubernetes/pkg/kubelet/eviction/api"
"k8s.io/kubernetes/pkg/kubelet/lifecycle"
@ -145,7 +144,7 @@ func parsePercentage(v string) (int64, error) {
}
// ParseQOSReserved parses the --qos-reserve-requests option
func ParseQOSReserved(m kubeletconfig.ConfigurationMap) (*map[v1.ResourceName]int64, error) {
func ParseQOSReserved(m map[string]string) (*map[v1.ResourceName]int64, error) {
reservations := make(map[v1.ResourceName]int64)
for k, v := range m {
switch v1.ResourceName(k) {

1
pkg/kubelet/eviction/BUILD
View File

@ -67,7 +67,6 @@ go_library(
"//vendor/k8s.io/apimachinery/pkg/api/resource:go_default_library",
"//vendor/k8s.io/apimachinery/pkg/apis/meta/v1:go_default_library",
"//vendor/k8s.io/apimachinery/pkg/util/clock:go_default_library",
"//vendor/k8s.io/apimachinery/pkg/util/sets:go_default_library",
"//vendor/k8s.io/apimachinery/pkg/util/wait:go_default_library",
"//vendor/k8s.io/apiserver/pkg/util/feature:go_default_library",
"//vendor/k8s.io/client-go/tools/record:go_default_library",

17
pkg/kubelet/eviction/api/types.go
View File

@ -50,6 +50,23 @@ const (
OpLessThan ThresholdOperator = "LessThan"
)
// OpForSignal maps Signals to ThresholdOperators.
// Today, the only supported operator is "LessThan". This may change in the future,
// for example if "consumed" (as opposed to "available") type signals are added.
// In both cases the directionality of the threshold is implicit to the signal type
// (for a given signal, the decision to evict will be made when crossing the threshold
// from either above or below, never both). There is thus no reason to expose the
// operator in the Kubelet's public API. Instead, we internally map signal types to operators.
var OpForSignal = map[Signal]ThresholdOperator{
SignalMemoryAvailable: OpLessThan,
SignalNodeFsAvailable: OpLessThan,
SignalNodeFsInodesFree: OpLessThan,
SignalImageFsAvailable: OpLessThan,
SignalImageFsInodesFree: OpLessThan,
SignalAllocatableMemoryAvailable: OpLessThan,
SignalAllocatableNodeFsAvailable: OpLessThan,
}
// ThresholdValue is a value holder that abstracts literal versus percentage based quantity
type ThresholdValue struct {
// The following fields are exclusive. Only the topmost non-zero field is used.

109
pkg/kubelet/eviction/helpers.go
View File

@ -26,7 +26,6 @@ import (
"github.com/golang/glog"
"k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
"k8s.io/apimachinery/pkg/util/sets"
utilfeature "k8s.io/apiserver/pkg/util/feature"
"k8s.io/kubernetes/pkg/features"
statsapi "k8s.io/kubernetes/pkg/kubelet/apis/stats/v1alpha1"
@ -101,7 +100,7 @@ func validSignal(signal evictionapi.Signal) bool {
}
// ParseThresholdConfig parses the flags for thresholds.
func ParseThresholdConfig(allocatableConfig []string, evictionHard, evictionSoft, evictionSoftGracePeriod, evictionMinimumReclaim string) ([]evictionapi.Threshold, error) {
func ParseThresholdConfig(allocatableConfig []string, evictionHard, evictionSoft, evictionSoftGracePeriod, evictionMinimumReclaim map[string]string) ([]evictionapi.Threshold, error) {
results := []evictionapi.Threshold{}
allocatableThresholds := getAllocatableThreshold(allocatableConfig)
results = append(results, allocatableThresholds...)
@ -145,60 +144,34 @@ func ParseThresholdConfig(allocatableConfig []string, evictionHard, evictionSoft
}
// parseThresholdStatements parses the input statements into a list of Threshold objects.
func parseThresholdStatements(expr string) ([]evictionapi.Threshold, error) {
if len(expr) == 0 {
func parseThresholdStatements(statements map[string]string) ([]evictionapi.Threshold, error) {
if len(statements) == 0 {
return nil, nil
}
results := []evictionapi.Threshold{}
statements := strings.Split(expr, ",")
signalsFound := sets.NewString()
for _, statement := range statements {
result, err := parseThresholdStatement(statement)
for signal, val := range statements {
result, err := parseThresholdStatement(evictionapi.Signal(signal), val)
if err != nil {
return nil, err
}
if signalsFound.Has(string(result.Signal)) {
return nil, fmt.Errorf("found duplicate eviction threshold for signal %v", result.Signal)
}
signalsFound.Insert(string(result.Signal))
results = append(results, result)
}
return results, nil
}
// parseThresholdStatement parses a threshold statement.
func parseThresholdStatement(statement string) (evictionapi.Threshold, error) {
tokens2Operator := map[string]evictionapi.ThresholdOperator{
"<": evictionapi.OpLessThan,
}
var (
operator evictionapi.ThresholdOperator
parts []string
)
for token := range tokens2Operator {
parts = strings.Split(statement, token)
// if we got a token, we know this was the operator...
if len(parts) > 1 {
operator = tokens2Operator[token]
break
}
}
if len(operator) == 0 || len(parts) != 2 {
return evictionapi.Threshold{}, fmt.Errorf("invalid eviction threshold syntax %v, expected <signal><operator><value>", statement)
}
signal := evictionapi.Signal(parts[0])
func parseThresholdStatement(signal evictionapi.Signal, val string) (evictionapi.Threshold, error) {
if !validSignal(signal) {
return evictionapi.Threshold{}, fmt.Errorf(unsupportedEvictionSignal, signal)
}
quantityValue := parts[1]
if strings.HasSuffix(quantityValue, "%") {
percentage, err := parsePercentage(quantityValue)
operator := evictionapi.OpForSignal[signal]
if strings.HasSuffix(val, "%") {
percentage, err := parsePercentage(val)
if err != nil {
return evictionapi.Threshold{}, err
}
if percentage <= 0 {
return evictionapi.Threshold{}, fmt.Errorf("eviction percentage threshold %v must be positive: %s", signal, quantityValue)
return evictionapi.Threshold{}, fmt.Errorf("eviction percentage threshold %v must be positive: %s", signal, val)
}
return evictionapi.Threshold{
Signal: signal,
@ -208,7 +181,7 @@ func parseThresholdStatement(statement string) (evictionapi.Threshold, error) {
},
}, nil
}
quantity, err := resource.ParseQuantity(quantityValue)
quantity, err := resource.ParseQuantity(val)
if err != nil {
return evictionapi.Threshold{}, err
}
@ -265,33 +238,22 @@ func parsePercentage(input string) (float32, error) {
}
// parseGracePeriods parses the grace period statements
func parseGracePeriods(expr string) (map[evictionapi.Signal]time.Duration, error) {
if len(expr) == 0 {
func parseGracePeriods(statements map[string]string) (map[evictionapi.Signal]time.Duration, error) {
if len(statements) == 0 {
return nil, nil
}
results := map[evictionapi.Signal]time.Duration{}
statements := strings.Split(expr, ",")
for _, statement := range statements {
parts := strings.Split(statement, "=")
if len(parts) != 2 {
return nil, fmt.Errorf("invalid eviction grace period syntax %v, expected <signal>=<duration>", statement)
}
signal := evictionapi.Signal(parts[0])
for signal, val := range statements {
signal := evictionapi.Signal(signal)
if !validSignal(signal) {
return nil, fmt.Errorf(unsupportedEvictionSignal, signal)
}
gracePeriod, err := time.ParseDuration(parts[1])
gracePeriod, err := time.ParseDuration(val)
if err != nil {
return nil, err
}
if gracePeriod < 0 {
return nil, fmt.Errorf("invalid eviction grace period specified: %v, must be a positive value", parts[1])
}
// check against duplicate statements
if _, found := results[signal]; found {
return nil, fmt.Errorf("duplicate eviction grace period specified for %v", signal)
return nil, fmt.Errorf("invalid eviction grace period specified: %v, must be a positive value", val)
}
results[signal] = gracePeriod
}
@ -299,51 +261,36 @@ func parseGracePeriods(expr string) (map[evictionapi.Signal]time.Duration, error
}
// parseMinimumReclaims parses the minimum reclaim statements
func parseMinimumReclaims(expr string) (map[evictionapi.Signal]evictionapi.ThresholdValue, error) {
if len(expr) == 0 {
func parseMinimumReclaims(statements map[string]string) (map[evictionapi.Signal]evictionapi.ThresholdValue, error) {
if len(statements) == 0 {
return nil, nil
}
results := map[evictionapi.Signal]evictionapi.ThresholdValue{}
statements := strings.Split(expr, ",")
for _, statement := range statements {
parts := strings.Split(statement, "=")
if len(parts) != 2 {
return nil, fmt.Errorf("invalid eviction minimum reclaim syntax: %v, expected <signal>=<value>", statement)
}
signal := evictionapi.Signal(parts[0])
for signal, val := range statements {
signal := evictionapi.Signal(signal)
if !validSignal(signal) {
return nil, fmt.Errorf(unsupportedEvictionSignal, signal)
}
quantityValue := parts[1]
if strings.HasSuffix(quantityValue, "%") {
percentage, err := parsePercentage(quantityValue)
if strings.HasSuffix(val, "%") {
percentage, err := parsePercentage(val)
if err != nil {
return nil, err
}
if percentage <= 0 {
return nil, fmt.Errorf("eviction percentage minimum reclaim %v must be positive: %s", signal, quantityValue)
}
// check against duplicate statements
if _, found := results[signal]; found {
return nil, fmt.Errorf("duplicate eviction minimum reclaim specified for %v", signal)
return nil, fmt.Errorf("eviction percentage minimum reclaim %v must be positive: %s", signal, val)
}
results[signal] = evictionapi.ThresholdValue{
Percentage: percentage,
}
continue
}
// check against duplicate statements
if _, found := results[signal]; found {
return nil, fmt.Errorf("duplicate eviction minimum reclaim specified for %v", signal)
}
quantity, err := resource.ParseQuantity(parts[1])
if quantity.Sign() < 0 {
return nil, fmt.Errorf("negative eviction minimum reclaim specified for %v", signal)
}
quantity, err := resource.ParseQuantity(val)
if err != nil {
return nil, err
}
if quantity.Sign() < 0 {
return nil, fmt.Errorf("negative eviction minimum reclaim specified for %v", signal)
}
results[signal] = evictionapi.ThresholdValue{
Quantity: &quantity,
}

152
pkg/kubelet/eviction/helpers_test.go
View File

@ -44,28 +44,28 @@ func TestParseThresholdConfig(t *testing.T) {
gracePeriod, _ := time.ParseDuration("30s")
testCases := map[string]struct {
allocatableConfig []string
evictionHard string
evictionSoft string
evictionSoftGracePeriod string
evictionMinReclaim string
evictionHard map[string]string
evictionSoft map[string]string
evictionSoftGracePeriod map[string]string
evictionMinReclaim map[string]string
expectErr bool
expectThresholds []evictionapi.Threshold
}{
"no values": {
allocatableConfig: []string{},
evictionHard: "",
evictionSoft: "",
evictionSoftGracePeriod: "",
evictionMinReclaim: "",
evictionHard: map[string]string{},
evictionSoft: map[string]string{},
evictionSoftGracePeriod: map[string]string{},
evictionMinReclaim: map[string]string{},
expectErr: false,
expectThresholds: []evictionapi.Threshold{},
},
"all flag values": {
"all memory eviction values": {
allocatableConfig: []string{cm.NodeAllocatableEnforcementKey},
evictionHard: "memory.available<150Mi",
evictionSoft: "memory.available<300Mi",
evictionSoftGracePeriod: "memory.available=30s",
evictionMinReclaim: "memory.available=0",
evictionHard: map[string]string{"memory.available": "150Mi"},
evictionSoft: map[string]string{"memory.available": "300Mi"},
evictionSoftGracePeriod: map[string]string{"memory.available": "30s"},
evictionMinReclaim: map[string]string{"memory.available": "0"},
expectErr: false,
expectThresholds: []evictionapi.Threshold{
{
@ -111,12 +111,12 @@ func TestParseThresholdConfig(t *testing.T) {
},
},
},
"all flag values in percentages": {
"all memory eviction values in percentages": {
allocatableConfig: []string{},
evictionHard: "memory.available<10%",
evictionSoft: "memory.available<30%",
evictionSoftGracePeriod: "memory.available=30s",
evictionMinReclaim: "memory.available=5%",
evictionHard: map[string]string{"memory.available": "10%"},
evictionSoft: map[string]string{"memory.available": "30%"},
evictionSoftGracePeriod: map[string]string{"memory.available": "30s"},
evictionMinReclaim: map[string]string{"memory.available": "5%"},
expectErr: false,
expectThresholds: []evictionapi.Threshold{
{
@ -142,12 +142,12 @@ func TestParseThresholdConfig(t *testing.T) {
},
},
},
"disk flag values": {
"disk eviction values": {
allocatableConfig: []string{},
evictionHard: "imagefs.available<150Mi,nodefs.available<100Mi",
evictionSoft: "imagefs.available<300Mi,nodefs.available<200Mi",
evictionSoftGracePeriod: "imagefs.available=30s,nodefs.available=30s",
evictionMinReclaim: "imagefs.available=2Gi,nodefs.available=1Gi",
evictionHard: map[string]string{"imagefs.available": "150Mi", "nodefs.available": "100Mi"},
evictionSoft: map[string]string{"imagefs.available": "300Mi", "nodefs.available": "200Mi"},
evictionSoftGracePeriod: map[string]string{"imagefs.available": "30s", "nodefs.available": "30s"},
evictionMinReclaim: map[string]string{"imagefs.available": "2Gi", "nodefs.available": "1Gi"},
expectErr: false,
expectThresholds: []evictionapi.Threshold{
{
@ -194,12 +194,12 @@ func TestParseThresholdConfig(t *testing.T) {
},
},
},
"disk flag values in percentages": {
"disk eviction values in percentages": {
allocatableConfig: []string{},
evictionHard: "imagefs.available<15%,nodefs.available<10.5%",
evictionSoft: "imagefs.available<30%,nodefs.available<20.5%",
evictionSoftGracePeriod: "imagefs.available=30s,nodefs.available=30s",
evictionMinReclaim: "imagefs.available=10%,nodefs.available=5%",
evictionHard: map[string]string{"imagefs.available": "15%", "nodefs.available": "10.5%"},
evictionSoft: map[string]string{"imagefs.available": "30%", "nodefs.available": "20.5%"},
evictionSoftGracePeriod: map[string]string{"imagefs.available": "30s", "nodefs.available": "30s"},
evictionMinReclaim: map[string]string{"imagefs.available": "10%", "nodefs.available": "5%"},
expectErr: false,
expectThresholds: []evictionapi.Threshold{
{
@ -246,12 +246,12 @@ func TestParseThresholdConfig(t *testing.T) {
},
},
},
"inode flag values": {
"inode eviction values": {
allocatableConfig: []string{},
evictionHard: "imagefs.inodesFree<150Mi,nodefs.inodesFree<100Mi",
evictionSoft: "imagefs.inodesFree<300Mi,nodefs.inodesFree<200Mi",
evictionSoftGracePeriod: "imagefs.inodesFree=30s,nodefs.inodesFree=30s",
evictionMinReclaim: "imagefs.inodesFree=2Gi,nodefs.inodesFree=1Gi",
evictionHard: map[string]string{"imagefs.inodesFree": "150Mi", "nodefs.inodesFree": "100Mi"},
evictionSoft: map[string]string{"imagefs.inodesFree": "300Mi", "nodefs.inodesFree": "200Mi"},
evictionSoftGracePeriod: map[string]string{"imagefs.inodesFree": "30s", "nodefs.inodesFree": "30s"},
evictionMinReclaim: map[string]string{"imagefs.inodesFree": "2Gi", "nodefs.inodesFree": "1Gi"},
expectErr: false,
expectThresholds: []evictionapi.Threshold{
{
@ -300,91 +300,73 @@ func TestParseThresholdConfig(t *testing.T) {
},
"invalid-signal": {
allocatableConfig: []string{},
evictionHard: "mem.available<150Mi",
evictionSoft: "",
evictionSoftGracePeriod: "",
evictionMinReclaim: "",
evictionHard: map[string]string{"mem.available": "150Mi"},
evictionSoft: map[string]string{},
evictionSoftGracePeriod: map[string]string{},
evictionMinReclaim: map[string]string{},
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},
"hard-signal-negative": {
allocatableConfig: []string{},
evictionHard: "memory.available<-150Mi",
evictionSoft: "",
evictionSoftGracePeriod: "",
evictionMinReclaim: "",
evictionHard: map[string]string{"memory.available": "-150Mi"},
evictionSoft: map[string]string{},
evictionSoftGracePeriod: map[string]string{},
evictionMinReclaim: map[string]string{},
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},
"hard-signal-negative-percentage": {
allocatableConfig: []string{},
evictionHard: "memory.available<-15%",
evictionSoft: "",
evictionSoftGracePeriod: "",
evictionMinReclaim: "",
evictionHard: map[string]string{"memory.available": "-15%"},
evictionSoft: map[string]string{},
evictionSoftGracePeriod: map[string]string{},
evictionMinReclaim: map[string]string{},
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},
"soft-signal-negative": {
allocatableConfig: []string{},
evictionHard: "",
evictionSoft: "memory.available<-150Mi",
evictionSoftGracePeriod: "",
evictionMinReclaim: "",
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},
"duplicate-signal": {
allocatableConfig: []string{},
evictionHard: "memory.available<150Mi,memory.available<100Mi",
evictionSoft: "",
evictionSoftGracePeriod: "",
evictionMinReclaim: "",
evictionHard: map[string]string{},
evictionSoft: map[string]string{"memory.available": "-150Mi"},
evictionSoftGracePeriod: map[string]string{},
evictionMinReclaim: map[string]string{},
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},
"valid-and-invalid-signal": {
allocatableConfig: []string{},
evictionHard: "memory.available<150Mi,invalid.foo<150Mi",
evictionSoft: "",
evictionSoftGracePeriod: "",
evictionMinReclaim: "",
evictionHard: map[string]string{"memory.available": "150Mi", "invalid.foo": "150Mi"},
evictionSoft: map[string]string{},
evictionSoftGracePeriod: map[string]string{},
evictionMinReclaim: map[string]string{},
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},
"soft-no-grace-period": {
allocatableConfig: []string{},
evictionHard: "",
evictionSoft: "memory.available<150Mi",
evictionSoftGracePeriod: "",
evictionMinReclaim: "",
evictionHard: map[string]string{},
evictionSoft: map[string]string{"memory.available": "150Mi"},
evictionSoftGracePeriod: map[string]string{},
evictionMinReclaim: map[string]string{},
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},
"soft-neg-grace-period": {
"soft-negative-grace-period": {
allocatableConfig: []string{},
evictionHard: "",
evictionSoft: "memory.available<150Mi",
evictionSoftGracePeriod: "memory.available=-30s",
evictionMinReclaim: "",
evictionHard: map[string]string{},
evictionSoft: map[string]string{"memory.available": "150Mi"},
evictionSoftGracePeriod: map[string]string{"memory.available": "-30s"},
evictionMinReclaim: map[string]string{},
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},
"neg-reclaim": {
"negative-reclaim": {
allocatableConfig: []string{},
evictionHard: "",
evictionSoft: "",
evictionSoftGracePeriod: "",
evictionMinReclaim: "memory.available=-300Mi",
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},
"duplicate-reclaim": {
allocatableConfig: []string{},
evictionHard: "",
evictionSoft: "",
evictionSoftGracePeriod: "",
evictionMinReclaim: "memory.available=-300Mi,memory.available=-100Mi",
evictionHard: map[string]string{},
evictionSoft: map[string]string{},
evictionSoftGracePeriod: map[string]string{},
evictionMinReclaim: map[string]string{"memory.available": "-300Mi"},
expectErr: true,
expectThresholds: []evictionapi.Threshold{},
},

11
staging/src/k8s.io/apiserver/pkg/util/flag/BUILD
View File

@ -10,16 +10,14 @@ go_test(
name = "go_default_test",
srcs = [
"colon_separated_multimap_string_string_test.go",
"langle_separated_map_string_string_test.go",
"map_string_bool_test.go",
"map_string_string_test.go",
"namedcertkey_flag_test.go",
],
importpath = "k8s.io/apiserver/pkg/util/flag",
library = ":go_default_library",
deps = [
"//vendor/github.com/spf13/pflag:go_default_library",
"//vendor/github.com/stretchr/testify/assert:go_default_library",
"//vendor/github.com/stretchr/testify/require:go_default_library",
],
deps = ["//vendor/github.com/spf13/pflag:go_default_library"],
)
go_library(
@ -28,8 +26,11 @@ go_library(
"colon_separated_multimap_string_string.go",
"configuration_map.go",
"flags.go",
"langle_separated_map_string_string.go",
"map_string_bool.go",
"map_string_string.go",
"namedcertkey_flag.go",
"omitempty.go",
"string_flag.go",
"tristate.go",
],

82
staging/src/k8s.io/apiserver/pkg/util/flag/langle_separated_map_string_string.go Normal file
View File

@ -0,0 +1,82 @@
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package flag
import (
"fmt"
"sort"
"strings"
)
// LangleSeparatedMapStringString can be set from the command line with the format `--flag "string<string"`.
// Multiple comma-separated key-value pairs in a single invocation are supported. For example: `--flag "a<foo,b<bar"`.
// Multiple flag invocations are supported. For example: `--flag "a<foo" --flag "b<foo"`.
type LangleSeparatedMapStringString struct {
Map *map[string]string
initialized bool // set to true after first Set call
}
// NewLangleSeparatedMapStringString takes a pointer to a map[string]string and returns the
// LangleSeparatedMapStringString flag parsing shim for that map
func NewLangleSeparatedMapStringString(m *map[string]string) *LangleSeparatedMapStringString {
return &LangleSeparatedMapStringString{Map: m}
}
// String implements github.com/spf13/pflag.Value
func (m *LangleSeparatedMapStringString) String() string {
pairs := []string{}
for k, v := range *m.Map {
pairs = append(pairs, fmt.Sprintf("%s<%s", k, v))
}
sort.Strings(pairs)
return strings.Join(pairs, ",")
}
// Set implements github.com/spf13/pflag.Value
func (m *LangleSeparatedMapStringString) Set(value string) error {
if m.Map == nil {
return fmt.Errorf("no target (nil pointer to map[string]string)")
}
if !m.initialized || *m.Map == nil {
// clear default values, or allocate if no existing map
*m.Map = make(map[string]string)
m.initialized = true
}
for _, s := range strings.Split(value, ",") {
if len(s) == 0 {
continue
}
arr := strings.SplitN(s, "<", 2)
if len(arr) != 2 {
return fmt.Errorf("malformed pair, expect string<string")
}
k := strings.TrimSpace(arr[0])
v := strings.TrimSpace(arr[1])
(*m.Map)[k] = v
}
return nil
}
// Type implements github.com/spf13/pflag.Value
func (*LangleSeparatedMapStringString) Type() string {
return "mapStringString"
}
// Empty implements OmitEmpty
func (m *LangleSeparatedMapStringString) Empty() bool {
return len(*m.Map) == 0
}

159
staging/src/k8s.io/apiserver/pkg/util/flag/langle_separated_map_string_string_test.go Normal file
View File

@ -0,0 +1,159 @@
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package flag
import (
"reflect"
"testing"
)
func TestStringLangleSeparatedMapStringString(t *testing.T) {
var nilMap map[string]string
cases := []struct {
desc string
m *LangleSeparatedMapStringString
expect string
}{
{"nil", NewLangleSeparatedMapStringString(&nilMap), ""},
{"empty", NewLangleSeparatedMapStringString(&map[string]string{}), ""},
{"one key", NewLangleSeparatedMapStringString(&map[string]string{"one": "foo"}), "one<foo"},
{"two keys", NewLangleSeparatedMapStringString(&map[string]string{"one": "foo", "two": "bar"}), "one<foo,two<bar"},
}
for _, c := range cases {
t.Run(c.desc, func(t *testing.T) {
str := c.m.String()
if c.expect != str {
t.Fatalf("expect %q but got %q", c.expect, str)
}
})
}
}
func TestSetLangleSeparatedMapStringString(t *testing.T) {
var nilMap map[string]string
cases := []struct {
desc string
vals []string
start *LangleSeparatedMapStringString
expect *LangleSeparatedMapStringString
err string
}{
// we initialize the map with a default key that should be cleared by Set
{"clears defaults", []string{""},
NewLangleSeparatedMapStringString(&map[string]string{"default": ""}),
&LangleSeparatedMapStringString{
initialized: true,
Map: &map[string]string{},
}, ""},
// make sure we still allocate for "initialized" maps where Map was initially set to a nil map
{"allocates map if currently nil", []string{""},
&LangleSeparatedMapStringString{initialized: true, Map: &nilMap},
&LangleSeparatedMapStringString{
initialized: true,
Map: &map[string]string{},
}, ""},
// for most cases, we just reuse nilMap, which should be allocated by Set, and is reset before each test case
{"empty", []string{""},
NewLangleSeparatedMapStringString(&nilMap),
&LangleSeparatedMapStringString{
initialized: true,
Map: &map[string]string{},
}, ""},
{"one key", []string{"one<foo"},
NewLangleSeparatedMapStringString(&nilMap),
&LangleSeparatedMapStringString{
initialized: true,
Map: &map[string]string{"one": "foo"},
}, ""},
{"two keys", []string{"one<foo,two<bar"},
NewLangleSeparatedMapStringString(&nilMap),
&LangleSeparatedMapStringString{
initialized: true,
Map: &map[string]string{"one": "foo", "two": "bar"},
}, ""},
{"two keys, multiple Set invocations", []string{"one<foo", "two<bar"},
NewLangleSeparatedMapStringString(&nilMap),
&LangleSeparatedMapStringString{
initialized: true,
Map: &map[string]string{"one": "foo", "two": "bar"},
}, ""},
{"two keys with space", []string{"one<foo, two<bar"},
NewLangleSeparatedMapStringString(&nilMap),
&LangleSeparatedMapStringString{
initialized: true,
Map: &map[string]string{"one": "foo", "two": "bar"},
}, ""},
{"empty key", []string{"<foo"},
NewLangleSeparatedMapStringString(&nilMap),
&LangleSeparatedMapStringString{
initialized: true,
Map: &map[string]string{"": "foo"},
}, ""},
{"missing value", []string{"one"},
NewLangleSeparatedMapStringString(&nilMap),
nil,
"malformed pair, expect string<string"},
{"no target", []string{"a:foo"},
NewLangleSeparatedMapStringString(nil),
nil,
"no target (nil pointer to map[string]string)"},
}
for _, c := range cases {
nilMap = nil
t.Run(c.desc, func(t *testing.T) {
var err error
for _, val := range c.vals {
err = c.start.Set(val)
if err != nil {
break
}
}
if c.err != "" {
if err == nil || err.Error() != c.err {
t.Fatalf("expect error %s but got %v", c.err, err)
}
return
} else if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !reflect.DeepEqual(c.expect, c.start) {
t.Fatalf("expect %#v but got %#v", c.expect, c.start)
}
})
}
}
func TestEmptyLangleSeparatedMapStringString(t *testing.T) {
var nilMap map[string]string
cases := []struct {
desc string
val *LangleSeparatedMapStringString
expect bool
}{
{"nil", NewLangleSeparatedMapStringString(&nilMap), true},
{"empty", NewLangleSeparatedMapStringString(&map[string]string{}), true},
{"populated", NewLangleSeparatedMapStringString(&map[string]string{"foo": ""}), false},
}
for _, c := range cases {
t.Run(c.desc, func(t *testing.T) {
result := c.val.Empty()
if result != c.expect {
t.Fatalf("expect %t but got %t", c.expect, result)
}
})
}
}

37
staging/src/k8s.io/apiserver/pkg/util/flag/map_string_bool.go
View File

@ -23,12 +23,24 @@ import (
"strings"
)
type MapStringBool map[string]bool
// MapStringBool can be set from the command line with the format `--flag "string=bool"`.
// Multiple comma-separated key-value pairs in a single invocation are supported. For example: `--flag "a=true,b=false"`.
// Multiple flag invocations are supported. For example: `--flag "a=true" --flag "b=false"`.
type MapStringBool struct {
Map *map[string]bool
initialized bool
}
// NewMapStringBool takes a pointer to a map[string]string and returns the
// MapStringBool flag parsing shim for that map
func NewMapStringBool(m *map[string]bool) *MapStringBool {
return &MapStringBool{Map: m}
}
// String implements github.com/spf13/pflag.Value
func (m MapStringBool) String() string {
func (m *MapStringBool) String() string {
pairs := []string{}
for k, v := range m {
for k, v := range *m.Map {
pairs = append(pairs, fmt.Sprintf("%s=%t", k, v))
}
sort.Strings(pairs)
@ -36,7 +48,15 @@ func (m MapStringBool) String() string {
}
// Set implements github.com/spf13/pflag.Value
func (m MapStringBool) Set(value string) error {
func (m *MapStringBool) Set(value string) error {
if m.Map == nil {
return fmt.Errorf("no target (nil pointer to map[string]bool)")
}
if !m.initialized || *m.Map == nil {
// clear default values, or allocate if no existing map
*m.Map = make(map[string]bool)
m.initialized = true
}
for _, s := range strings.Split(value, ",") {
if len(s) == 0 {
continue
@ -51,12 +71,17 @@ func (m MapStringBool) Set(value string) error {
if err != nil {
return fmt.Errorf("invalid value of %s: %s, err: %v", k, v, err)
}
m[k] = boolValue
(*m.Map)[k] = boolValue
}
return nil
}
// Type implements github.com/spf13/pflag.Value
func (MapStringBool) Type() string {
func (*MapStringBool) Type() string {
return "mapStringBool"
}
// Empty implements OmitEmpty
func (m *MapStringBool) Empty() bool {
return len(*m.Map) == 0
}

140
staging/src/k8s.io/apiserver/pkg/util/flag/map_string_bool_test.go
View File

@ -17,55 +17,147 @@ limitations under the License.
package flag
import (
"reflect"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestStringMapStringBool(t *testing.T) {
var nilMap map[string]bool
cases := []struct {
desc string
m MapStringBool
m *MapStringBool
expect string
}{
{"empty", MapStringBool{}, ""},
{"one key", MapStringBool{"one": true}, "one=true"},
{"two keys", MapStringBool{"one": true, "two": false}, "one=true,two=false"},
{"nil", NewMapStringBool(&nilMap), ""},
{"empty", NewMapStringBool(&map[string]bool{}), ""},
{"one key", NewMapStringBool(&map[string]bool{"one": true}), "one=true"},
{"two keys", NewMapStringBool(&map[string]bool{"one": true, "two": false}), "one=true,two=false"},
}
for _, c := range cases {
t.Run(c.desc, func(t *testing.T) {
str := c.m.String()
assert.Equal(t, c.expect, str)
if c.expect != str {
t.Fatalf("expect %q but got %q", c.expect, str)
}
})
}
}
func TestSetMapStringBool(t *testing.T) {
var nilMap map[string]bool
cases := []struct {
desc string
val string
expect MapStringBool
vals []string
start *MapStringBool
expect *MapStringBool
err string
}{
{"empty", "", MapStringBool{}, ""},
{"one key", "one=true", MapStringBool{"one": true}, ""},
{"two keys", "one=true,two=false", MapStringBool{"one": true, "two": false}, ""},
{"two keys with space", "one=true, two=false", MapStringBool{"one": true, "two": false}, ""},
{"empty key", "=true", MapStringBool{"": true}, ""},
{"missing value", "one", MapStringBool{}, "malformed pair, expect string=bool"},
{"non-boolean value", "one=foo", MapStringBool{}, `invalid value of one: foo, err: strconv.ParseBool: parsing "foo": invalid syntax`},
// we initialize the map with a default key that should be cleared by Set
{"clears defaults", []string{""},
NewMapStringBool(&map[string]bool{"default": true}),
&MapStringBool{
initialized: true,
Map: &map[string]bool{},
}, ""},
// make sure we still allocate for "initialized" maps where Map was initially set to a nil map
{"allocates map if currently nil", []string{""},
&MapStringBool{initialized: true, Map: &nilMap},
&MapStringBool{
initialized: true,
Map: &map[string]bool{},
}, ""},
// for most cases, we just reuse nilMap, which should be allocated by Set, and is reset before each test case
{"empty", []string{""},
NewMapStringBool(&nilMap),
&MapStringBool{
initialized: true,
Map: &map[string]bool{},
}, ""},
{"one key", []string{"one=true"},
NewMapStringBool(&nilMap),
&MapStringBool{
initialized: true,
Map: &map[string]bool{"one": true},
}, ""},
{"two keys", []string{"one=true,two=false"},
NewMapStringBool(&nilMap),
&MapStringBool{
initialized: true,
Map: &map[string]bool{"one": true, "two": false},
}, ""},
{"two keys, multiple Set invocations", []string{"one=true", "two=false"},
NewMapStringBool(&nilMap),
&MapStringBool{
initialized: true,
Map: &map[string]bool{"one": true, "two": false},
}, ""},
{"two keys with space", []string{"one=true, two=false"},
NewMapStringBool(&nilMap),
&MapStringBool{
initialized: true,
Map: &map[string]bool{"one": true, "two": false},
}, ""},
{"empty key", []string{"=true"},
NewMapStringBool(&nilMap),
&MapStringBool{
initialized: true,
Map: &map[string]bool{"": true},
}, ""},
{"missing value", []string{"one"},
NewMapStringBool(&nilMap),
nil,
"malformed pair, expect string=bool"},
{"non-boolean value", []string{"one=foo"},
NewMapStringBool(&nilMap),
nil,
`invalid value of one: foo, err: strconv.ParseBool: parsing "foo": invalid syntax`},
{"no target", []string{"one=true"},
NewMapStringBool(nil),
nil,
"no target (nil pointer to map[string]bool)"},
}
for _, c := range cases {
nilMap = nil
t.Run(c.desc, func(t *testing.T) {
m := MapStringBool{}
err := m.Set(c.val)
if c.err != "" {
require.EqualError(t, err, c.err)
} else {
require.NoError(t, err)
var err error
for _, val := range c.vals {
err = c.start.Set(val)
if err != nil {
break
}
}
if c.err != "" {
if err == nil || err.Error() != c.err {
t.Fatalf("expect error %s but got %v", c.err, err)
}
return
} else if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !reflect.DeepEqual(c.expect, c.start) {
t.Fatalf("expect %#v but got %#v", c.expect, c.start)
}
})
}
}
func TestEmptyMapStringBool(t *testing.T) {
var nilMap map[string]bool
cases := []struct {
desc string
val *MapStringBool
expect bool
}{
{"nil", NewMapStringBool(&nilMap), true},
{"empty", NewMapStringBool(&map[string]bool{}), true},
{"populated", NewMapStringBool(&map[string]bool{"foo": true}), false},
}
for _, c := range cases {
t.Run(c.desc, func(t *testing.T) {
result := c.val.Empty()
if result != c.expect {
t.Fatalf("expect %t but got %t", c.expect, result)
}
assert.Equal(t, c.expect, m)
})
}
}

82
staging/src/k8s.io/apiserver/pkg/util/flag/map_string_string.go Normal file
View File

@ -0,0 +1,82 @@
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package flag
import (
"fmt"
"sort"
"strings"
)
// MapStringString can be set from the command line with the format `--flag "string=string"`.
// Multiple comma-separated key-value pairs in a single invocation are supported. For example: `--flag "a=foo,b=bar"`.
// Multiple flag invocations are supported. For example: `--flag "a=foo" --flag "b=bar"`.
type MapStringString struct {
Map *map[string]string
initialized bool
}
// NewMapStringString takes a pointer to a map[string]string and returns the
// MapStringString flag parsing shim for that map
func NewMapStringString(m *map[string]string) *MapStringString {
return &MapStringString{Map: m}
}
// String implements github.com/spf13/pflag.Value
func (m *MapStringString) String() string {
pairs := []string{}
for k, v := range *m.Map {
pairs = append(pairs, fmt.Sprintf("%s=%s", k, v))
}
sort.Strings(pairs)
return strings.Join(pairs, ",")
}
// Set implements github.com/spf13/pflag.Value
func (m *MapStringString) Set(value string) error {
if m.Map == nil {
return fmt.Errorf("no target (nil pointer to map[string]string)")
}
if !m.initialized || *m.Map == nil {
// clear default values, or allocate if no existing map
*m.Map = make(map[string]string)
m.initialized = true
}
for _, s := range strings.Split(value, ",") {
if len(s) == 0 {
continue
}
arr := strings.SplitN(s, "=", 2)
if len(arr) != 2 {
return fmt.Errorf("malformed pair, expect string=string")
}
k := strings.TrimSpace(arr[0])
v := strings.TrimSpace(arr[1])
(*m.Map)[k] = v
}
return nil
}
// Type implements github.com/spf13/pflag.Value
func (*MapStringString) Type() string {
return "mapStringString"
}
// Empty implements OmitEmpty
func (m *MapStringString) Empty() bool {
return len(*m.Map) == 0
}

159
staging/src/k8s.io/apiserver/pkg/util/flag/map_string_string_test.go Normal file
View File

@ -0,0 +1,159 @@
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package flag
import (
"reflect"
"testing"
)
func TestStringMapStringString(t *testing.T) {
var nilMap map[string]string
cases := []struct {
desc string
m *MapStringString
expect string
}{
{"nil", NewMapStringString(&nilMap), ""},
{"empty", NewMapStringString(&map[string]string{}), ""},
{"one key", NewMapStringString(&map[string]string{"one": "foo"}), "one=foo"},
{"two keys", NewMapStringString(&map[string]string{"one": "foo", "two": "bar"}), "one=foo,two=bar"},
}
for _, c := range cases {
t.Run(c.desc, func(t *testing.T) {
str := c.m.String()
if c.expect != str {
t.Fatalf("expect %q but got %q", c.expect, str)
}
})
}
}
func TestSetMapStringString(t *testing.T) {
var nilMap map[string]string
cases := []struct {
desc string
vals []string
start *MapStringString
expect *MapStringString
err string
}{
// we initialize the map with a default key that should be cleared by Set
{"clears defaults", []string{""},
NewMapStringString(&map[string]string{"default": ""}),
&MapStringString{
initialized: true,
Map: &map[string]string{},
}, ""},
// make sure we still allocate for "initialized" maps where Map was initially set to a nil map
{"allocates map if currently nil", []string{""},
&MapStringString{initialized: true, Map: &nilMap},
&MapStringString{
initialized: true,
Map: &map[string]string{},
}, ""},
// for most cases, we just reuse nilMap, which should be allocated by Set, and is reset before each test case
{"empty", []string{""},
NewMapStringString(&nilMap),
&MapStringString{
initialized: true,
Map: &map[string]string{},
}, ""},
{"one key", []string{"one=foo"},
NewMapStringString(&nilMap),
&MapStringString{
initialized: true,
Map: &map[string]string{"one": "foo"},
}, ""},
{"two keys", []string{"one=foo,two=bar"},
NewMapStringString(&nilMap),
&MapStringString{
initialized: true,
Map: &map[string]string{"one": "foo", "two": "bar"},
}, ""},
{"two keys, multiple Set invocations", []string{"one=foo", "two=bar"},
NewMapStringString(&nilMap),
&MapStringString{
initialized: true,
Map: &map[string]string{"one": "foo", "two": "bar"},
}, ""},
{"two keys with space", []string{"one=foo, two=bar"},
NewMapStringString(&nilMap),
&MapStringString{
initialized: true,
Map: &map[string]string{"one": "foo", "two": "bar"},
}, ""},
{"empty key", []string{"=foo"},
NewMapStringString(&nilMap),
&MapStringString{
initialized: true,
Map: &map[string]string{"": "foo"},
}, ""},
{"missing value", []string{"one"},
NewMapStringString(&nilMap),
nil,
"malformed pair, expect string=string"},
{"no target", []string{"a:foo"},
NewMapStringString(nil),
nil,
"no target (nil pointer to map[string]string)"},
}
for _, c := range cases {
nilMap = nil
t.Run(c.desc, func(t *testing.T) {
var err error
for _, val := range c.vals {
err = c.start.Set(val)
if err != nil {
break
}
}
if c.err != "" {
if err == nil || err.Error() != c.err {
t.Fatalf("expect error %s but got %v", c.err, err)
}
return
} else if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !reflect.DeepEqual(c.expect, c.start) {
t.Fatalf("expect %#v but got %#v", c.expect, c.start)
}
})
}
}
func TestEmptyMapStringString(t *testing.T) {
var nilMap map[string]string
cases := []struct {
desc string
val *MapStringString
expect bool
}{
{"nil", NewMapStringString(&nilMap), true},
{"empty", NewMapStringString(&map[string]string{}), true},
{"populated", NewMapStringString(&map[string]string{"foo": ""}), false},
}
for _, c := range cases {
t.Run(c.desc, func(t *testing.T) {
result := c.val.Empty()
if result != c.expect {
t.Fatalf("expect %t but got %t", c.expect, result)
}
})
}
}

24
staging/src/k8s.io/apiserver/pkg/util/flag/omitempty.go Normal file
View File

@ -0,0 +1,24 @@
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package flag
// OmitEmpty is an interface for flags to report whether their underlying value
// is "empty." If a flag implements OmitEmpty and returns true for a call to Empty(),
// it is assumed that flag may be omitted from the command line.
type OmitEmpty interface {
Empty() bool
}

34
test/e2e_node/eviction_test.go
View File

@ -67,8 +67,8 @@ var _ = framework.KubeDescribe("InodeEviction [Slow] [Serial] [Disruptive] [Flak
if inodesFree <= inodesConsumed {
framework.Skipf("Too few inodes free on the host for the InodeEviction test to run")
}
initialConfig.EvictionHard = fmt.Sprintf("nodefs.inodesFree<%d", inodesFree-inodesConsumed)
initialConfig.EvictionMinimumReclaim = ""
initialConfig.EvictionHard = map[string]string{"nodefs.inodesFree": fmt.Sprintf("%d", inodesFree-inodesConsumed)}
initialConfig.EvictionMinimumReclaim = map[string]string{}
})
runEvictionTest(f, pressureTimeout, expectedNodeCondition, logInodeMetrics, []podEvictSpec{
{
@ -100,7 +100,9 @@ var _ = framework.KubeDescribe("MemoryAllocatableEviction [Slow] [Serial] [Disru
// The default hard eviction threshold is 250Mb, so Allocatable = Capacity - Reserved - 250Mb
// We want Allocatable = 50Mb, so set Reserved = Capacity - Allocatable - 250Mb = Capacity - 300Mb
kubeReserved.Sub(resource.MustParse("300Mi"))
initialConfig.KubeReserved = kubeletconfig.ConfigurationMap(map[string]string{string(v1.ResourceMemory): kubeReserved.String()})
initialConfig.KubeReserved = map[string]string{
string(v1.ResourceMemory): kubeReserved.String(),
}
initialConfig.EnforceNodeAllocatable = []string{cm.NodeAllocatableEnforcementKey}
initialConfig.CgroupsPerQOS = true
})
@ -129,13 +131,15 @@ var _ = framework.KubeDescribe("LocalStorageAllocatableEviction [Slow] [Serial]
diskConsumed := uint64(200000000) // At least 200 Mb for pods to consume
summary := eventuallyGetSummary()
availableBytes := *(summary.Node.Fs.AvailableBytes)
initialConfig.KubeReserved = kubeletconfig.ConfigurationMap(map[string]string{string(v1.ResourceEphemeralStorage): fmt.Sprintf("%d", availableBytes-diskConsumed)})
initialConfig.KubeReserved = map[string]string{
string(v1.ResourceEphemeralStorage): fmt.Sprintf("%d", availableBytes-diskConsumed),
}
initialConfig.EnforceNodeAllocatable = []string{cm.NodeAllocatableEnforcementKey}
initialConfig.CgroupsPerQOS = true
initialConfig.FeatureGates[string(features.LocalStorageCapacityIsolation)] = true
// set evictionHard to be very small, so that only the allocatable eviction threshold triggers
initialConfig.EvictionHard = "nodefs.available<1"
initialConfig.EvictionMinimumReclaim = ""
initialConfig.EvictionHard = map[string]string{"nodefs.available": "1"}
initialConfig.EvictionMinimumReclaim = map[string]string{}
framework.Logf("KubeReserved: %+v", initialConfig.KubeReserved)
})
runEvictionTest(f, pressureTimeout, expectedNodeCondition, logDiskMetrics, []podEvictSpec{
@ -162,8 +166,8 @@ var _ = framework.KubeDescribe("LocalStorageEviction [Slow] [Serial] [Disruptive
diskConsumed := uint64(100000000) // At least 100 Mb for pods to consume
summary := eventuallyGetSummary()
availableBytes := *(summary.Node.Fs.AvailableBytes)
initialConfig.EvictionHard = fmt.Sprintf("nodefs.available<%d", availableBytes-diskConsumed)
initialConfig.EvictionMinimumReclaim = ""
initialConfig.EvictionHard = map[string]string{"nodefs.available": fmt.Sprintf("%d", availableBytes-diskConsumed)}
initialConfig.EvictionMinimumReclaim = map[string]string{}
})
runEvictionTest(f, pressureTimeout, expectedNodeCondition, logDiskMetrics, []podEvictSpec{
{
@ -190,13 +194,13 @@ var _ = framework.KubeDescribe("LocalStorageSoftEviction [Slow] [Serial] [Disrup
diskConsumed := uint64(100000000) // At least 100 Mb for pods to consume
summary := eventuallyGetSummary()
availableBytes := *(summary.Node.Fs.AvailableBytes)
initialConfig.EvictionSoft = fmt.Sprintf("nodefs.available<%d", availableBytes-diskConsumed)
initialConfig.EvictionSoftGracePeriod = "nodefs.available=1m"
initialConfig.EvictionSoft = map[string]string{"nodefs.available": fmt.Sprintf("%d", availableBytes-diskConsumed)}
initialConfig.EvictionSoftGracePeriod = map[string]string{"nodefs.available": "1m"}
// Defer to the pod default grace period
initialConfig.EvictionMaxPodGracePeriod = 30
initialConfig.EvictionMinimumReclaim = ""
initialConfig.EvictionMinimumReclaim = map[string]string{}
// Ensure that pods are not evicted because of the eviction-hard threshold
initialConfig.EvictionHard = ""
initialConfig.EvictionHard = map[string]string{}
})
runEvictionTest(f, pressureTimeout, expectedNodeCondition, logDiskMetrics, []podEvictSpec{
{
@ -218,7 +222,7 @@ var _ = framework.KubeDescribe("LocalStorageCapacityIsolationEviction [Slow] [Se
Context(fmt.Sprintf(testContextFmt, "evictions due to pod local storage violations"), func() {
tempSetCurrentKubeletConfig(f, func(initialConfig *kubeletconfig.KubeletConfiguration) {
initialConfig.FeatureGates[string(features.LocalStorageCapacityIsolation)] = true
initialConfig.EvictionHard = ""
initialConfig.EvictionHard = map[string]string{}
})
sizeLimit := resource.MustParse("100Mi")
used := int64(200) // Consume 200 Mb
@ -269,8 +273,8 @@ var _ = framework.KubeDescribe("PriorityEvictionOrdering [Slow] [Serial] [Disrup
memoryConsumed := resource.MustParse("600Mi")
summary := eventuallyGetSummary()
availableBytes := *(summary.Node.Memory.AvailableBytes)
initialConfig.EvictionHard = fmt.Sprintf("memory.available<%d", availableBytes-uint64(memoryConsumed.Value()))
initialConfig.EvictionMinimumReclaim = ""
initialConfig.EvictionHard = map[string]string{"memory.available": fmt.Sprintf("%d", availableBytes-uint64(memoryConsumed.Value()))}
initialConfig.EvictionMinimumReclaim = map[string]string{}
})
specs := []podEvictSpec{
{

4
test/e2e_node/memory_eviction_test.go
View File

@ -37,8 +37,8 @@ import (
// https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/kubelet-eviction.md
var _ = framework.KubeDescribe("MemoryEviction [Slow] [Serial] [Disruptive]", func() {
const (
evictionHard = "memory.available<40%"
var (
evictionHard = map[string]string{"memory.available": "40%"}
)
f := framework.NewDefaultFramework("eviction-test")

6
test/e2e_node/node_container_manager_test.go
View File

@ -38,15 +38,15 @@ import (
func setDesiredConfiguration(initialConfig *kubeletconfig.KubeletConfiguration) {
initialConfig.EnforceNodeAllocatable = []string{"pods", "kube-reserved", "system-reserved"}
initialConfig.SystemReserved = kubeletconfig.ConfigurationMap{
initialConfig.SystemReserved = map[string]string{
string(v1.ResourceCPU): "100m",
string(v1.ResourceMemory): "100Mi",
}
initialConfig.KubeReserved = kubeletconfig.ConfigurationMap{
initialConfig.KubeReserved = map[string]string{
string(v1.ResourceCPU): "100m",
string(v1.ResourceMemory): "100Mi",
}
initialConfig.EvictionHard = "memory.available<100Mi"
initialConfig.EvictionHard = map[string]string{"memory.available": "100Mi"}
// Necessary for allocatable cgroup creation.
initialConfig.CgroupsPerQOS = true
initialConfig.KubeReservedCgroup = kubeReservedCgroup