github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

[systemd] Add value to LimitNOFILE due to performance problems 

When k3s is installed on an OS with default high ulimits, performance
issues can be observed. This was discovered on CoreOS where the default
value is 1073741816. Symptoms include very slow file operations such
as installing a Rook/Ceph cluster will take ~6 hours instead of ~10 minutes.

A google search for 'container LimitNOFILE' will show that most major
projects set this already, including the (unused) containerd systemd unit
found in this repository at /vendor/github.com/containerd/containerd/containerd.service

k3OS is not affected becuasse the default there is already 1048576.

See description in coreos/fedora-coreos-tracker#329
pull/1735/head
Adam Farden 2020-05-03 09:27:54 +02:00
parent 690a4ca7a4
commit b4335630b7
5 changed files with 15 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
contrib/ansible/roles/k3s/master/templates/k3s.service.j2
View File

@ -10,7 +10,9 @@ ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s server {{ extra_server_args | default("") }}
KillMode=process
Delegate=yes
LimitNOFILE=infinity
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity

4
contrib/ansible/roles/k3s/node/templates/k3s.service.j2
View File

@ -10,7 +10,9 @@ ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s agent --server https://{{ master_ip }}:6443 --token {{ hostvars[groups['master'][0]]['token'] }}
KillMode=process
Delegate=yes
LimitNOFILE=infinity
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity

4
install.sh
View File

@ -646,7 +646,9 @@ Type=${SYSTEMD_TYPE}
EnvironmentFile=${FILE_K3S_ENV}
KillMode=process
Delegate=yes
LimitNOFILE=infinity
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity

4
k3s.service
View File

@ -9,7 +9,9 @@ EnvironmentFile=/etc/systemd/system/k3s.service.env
ExecStart=/usr/local/bin/k3s server
KillMode=process
Delegate=yes
LimitNOFILE=infinity
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity

4
package/rpm/install.sh
View File

@ -670,7 +670,9 @@ Type=${SYSTEMD_TYPE}
EnvironmentFile=${FILE_K3S_ENV}
KillMode=process
Delegate=yes
LimitNOFILE=infinity
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity