github
/
k3s
mirror of https://github.com/k3s-io/k3s
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
79,025 Commits
31 Branches
881 Tags
633 MiB
Commit Graph

329 Commits (52f3380ef3a5d5de4d75ff92defd1c9af9d775bc)

Author SHA1 Message Date
Shyam Jeedigunta cc78a3f428 Passing correct master address to kubemark NPD & authenticating+authorizing it with apiserver 2017-01-18 18:23:23 +01:00
Kubernetes Submit Queue 6dfe5c49f6 Merge pull request #38865 from vwfs/ext4_no_lazy_init 
Automatic merge from submit-queue

Enable lazy initialization of ext3/ext4 filesystems

**What this PR does / why we need it**: It enables lazy inode table and journal initialization in ext3 and ext4.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #30752, fixes #30240

**Release note**:
```release-note
Enable lazy inode table and journal initialization for ext3 and ext4
```

**Special notes for your reviewer**:
This PR removes the extended options to mkfs.ext3/mkfs.ext4, so that the defaults (enabled) for lazy initialization are used.

These extended options come from a script that was historically located at */usr/share/google/safe_format_and_mount* and later ported to GO so this dependency to the script could be removed. After some search, I found the original script here: https://github.com/GoogleCloudPlatform/compute-image-packages/blob/legacy/google-startup-scripts/usr/share/google/safe_format_and_mount

Checking the history of this script, I found the commit [Disable lazy init of inode table and journal.](4d7346f7f5). This one introduces the extended flags with this description:
```
Now that discard with guaranteed zeroing is supported by PD,
initializing them is really fast and prevents perf from being affected
when the filesystem is first mounted.
```

The problem is, that this is not true for all cloud providers and all disk types, e.g. Azure and AWS. I only tested with magnetic disks on Azure and AWS, so maybe it's different for SSDs on these cloud providers. The result is that this performance optimization dramatically increases the time needed to format a disk in such cases.

When mkfs.ext4 is told to not lazily initialize the inode tables and the check for guaranteed zeroing on discard fails, it falls back to a very naive implementation that simply loops and writes zeroed buffers to the disk. Performance on this highly depends on free memory and also uses up all this free memory for write caching, reducing performance of everything else in the system. 

As of https://github.com/kubernetes/kubernetes/issues/30752, there is also something inside kubelet that somehow degrades performance of all this. It's however not exactly known what it is but I'd assume it has something to do with cgroups throttling IO or memory. 

I checked the kernel code for lazy inode table initialization. The nice thing is, that the kernel also does the guaranteed zeroing on discard check. If it is guaranteed, the kernel uses discard for the lazy initialization, which should finish in a just few seconds. If it is not guaranteed, it falls back to using *bio*s, which does not require the use of the write cache. The result is, that free memory is not required and not touched, thus performance is maxed and the system does not suffer.

As the original reason for disabling lazy init was a performance optimization and the kernel already does this optimization by default (and in a much better way), I'd suggest to completely remove these flags and rely on the kernel to do it in the best way.
 2017-01-18 09:09:52 -08:00
Shyam Jeedigunta 9b0d8b9747 Added RBAC for heapster in kubemark 2017-01-18 13:47:08 +01:00
Shyam Jeedigunta 491c26feca Fix RBAC role for kube-proxy in Kubemark 2017-01-17 11:39:00 +01:00
Jeff Grafton bc4b6ac397 Build release tarballs in bazel and add `make bazel-release` rule 2017-01-13 16:17:44 -08:00
Aleksandra Malinowska 043e809b8f update heapster version to 1.3.0-beta.0 2017-01-12 13:42:31 +01:00
Shyam Jeedigunta 312e2f85a6 run-gcloud-compute-with-retries in kubemark handles resource already exists case 2017-01-06 23:21:15 +01:00
Kubernetes Submit Queue 85ad3045be Merge pull request #39349 from shyamjvs/rbac-for-kubemark 
Automatic merge from submit-queue

Updated kubemark with RBAC for controllers, proxy and kubelet

Fixes issue #39244 

@kubernetes/sig-scalability-misc @wojtek-t @gmarek
 2017-01-06 13:42:54 -08:00
Shyam Jeedigunta ce8c207328 Updated kubemark with RBAC for controller-manager, kubecfg, kubelet and proxy 2017-01-06 08:54:54 +01:00
Kubernetes Submit Queue f4a8713088 Merge pull request #36229 from wojtek-t/bump_etcd_version 
Automatic merge from submit-queue (batch tested with PRs 36229, 39450)

Bump etcd to 3.0.14 and switch to v3 API in etcd.

Ref #20504

**Release note**:

```release-note
Switch default etcd version to 3.0.14.
Switch default storage backend flag in apiserver to `etcd3` mode.
```
 2017-01-04 17:36:06 -08:00
Shyam Jeedigunta ac30fb28bd Fixing 'systemd restart docker' command in kubemark master 2016-12-21 11:46:33 +01:00
Shyam Jeedigunta 7e12fd4bfd Added 'hollow'-node-problem-detector to hollow-nodes in kubemark 2016-12-20 12:04:24 +01:00
Wojciech Tyczynski 76f115a8ee Bump etcd to 3.0.14 2016-12-20 11:57:45 +01:00
Shyam Jeedigunta 9051462497 Migrated kubemark master to GCI from Debian. 2016-12-19 13:51:56 +01:00
Alexander Block 13a2bc8afb Enable lazy initialization of ext3/ext4 filesystems 2016-12-18 11:08:51 +01:00
Kubernetes Submit Queue 2e2661c403 Merge pull request #38803 from gmarek/kubemark-etcd-flags 
Automatic merge from submit-queue

Kubemark etcd flags

Fix for #38770.

cc @shyamjvs
 2016-12-15 05:38:38 -08:00
gmarek 40907930a3 Fix the kubemark fix 2016-12-15 10:26:39 +01:00
Shyam Jeedigunta 10b39d9dd7 Fixed kubemark to work when ETCD_X flags are set 2016-12-14 18:13:31 +01:00
aaronxu 3b20730620 spell check for test/* 2016-12-14 06:03:00 -08:00
Shyam Jeedigunta f7ce6a7d10 On kubemark master, kubelet now runs as a supervisord process and all master components as pods 2016-12-12 13:56:07 +01:00
Wojciech Tyczynski 9439453527 Increase single logfile size in kubemark 2016-12-12 11:18:20 +01:00
Shyam Jeedigunta 06ce9ae479 Moved start-kubemark-master.sh from test/kubemark/ to test/kubemark/resources/ 2016-12-07 18:15:24 +01:00
Wojciech Tyczynski aab227dd42 Collect controller-manager logs from kubemark 2016-12-06 16:30:52 +01:00
Hongchao Deng f9bc3ac419 kubemark: minor fix start-kubemark.sh 2016-11-22 18:00:23 -08:00
gmarek 4356cb633d Revert "Merge pull request #36065 from kubernetes/revert-35918-supervisor" 
This reverts commit a4ee3ed026, reversing
changes made to 8c90bc35e2.
 2016-11-22 09:46:48 +01:00
gmarek 15471488ba Update start-kubemark-master to correctly work in case of master-components restart 2016-11-21 15:40:58 +01:00
Wojciech Tyczynski 9ccddb9b7d Revert "Add log rotation to kubemark" 2016-11-21 14:55:52 +01:00
gmarek 3d9f002ebd Add commented local testing command in kubemark/run-e2e-tests.sh 2016-11-21 10:00:54 +01:00
gmarek 87f87206bc Pass GCLOUD_COMMON_ARGS do disk attach in Kubemark 2016-11-21 09:04:04 +01:00
Wojciech Tyczynski a96dd63367 Add log rotation to kubemark 2016-11-18 16:19:32 +01:00
gmarek 3fc177ff7c Allow storing events on a separate disk 2016-11-18 13:19:48 +01:00
gmarek 08d60999fc Allow passing custom list of admission plugins to kubemark 2016-11-17 12:32:41 +01:00
gmarek 7439a956ef Add ServiceAccounts to Kubemark 2016-11-15 16:03:48 +01:00
Wojciech Tyczynski 5ffda62588 Fix non-default etcd image handling in kubemark 2016-11-08 08:29:39 +01:00
Wojciech Tyczynski 314c4c2811 Revert "Make master components in kubemark monitored by supervisord" 2016-11-02 14:43:09 +01:00
gmarek 3e78f4feb8 Make master components in kubemark monitored by supervisord 2016-10-31 19:44:18 +01:00
Wojciech Tyczynski 42d8a13229 Fix conflict in kubemark 2016-10-31 08:34:33 +01:00
Kubernetes Submit Queue 569da52204 Merge pull request #35708 from wojtek-t/rename_test_etcd_version 
Automatic merge from submit-queue

Rename TEST_ETCD_VERSION to ETCD_VERSION

Ref #20504
 2016-10-29 05:46:51 -07:00
Wojciech Tyczynski 6a4a4bcf36 Fix wrong flag to etcd in kubemark 2016-10-28 15:54:15 +02:00
Wojciech Tyczynski 7ee7b55c5e Rename TEST_ETCD_VERSION to ETCD_VERSION 2016-10-28 13:56:59 +02:00
Wojciech Tyczynski 137d2398a8 Increase backend-quota-bytes in kubemark 2016-10-28 09:14:57 +02:00
Kubernetes Submit Queue cbe65701a2 Merge pull request #35104 from gmarek/root_disk 
Automatic merge from submit-queue

Allow setting master root disk size in kubemark
 2016-10-19 04:25:06 -07:00
gmarek 16d8e6400e Allow setting master root disk size in kubemark 2016-10-19 12:37:17 +02:00
gmarek cde44bc07c Mount disk in kubemark earlier 2016-10-19 12:11:10 +02:00
gmarek 6b7ddc56d3 Mount master-pd in kubemark 2016-10-18 11:25:17 +02:00
Wojciech Tyczynski 3215e8535a Extend logging for kubemark failures debugging 2016-09-30 14:29:29 +02:00
Wojciech Tyczynski 8abf3c1600 Make it possible to set etcd version in kubemark 2016-09-27 13:13:29 +02:00
Jeff Grafton 13a3313957 Bundle kubemark in test tarball instead of server tarball 2016-09-26 11:06:48 -07:00
gmarek c6ec8085f9 Extend kubemark cluster startup timeout to account for log route creation 2016-09-20 16:00:00 +02:00
Wojciech Tyczynski 086f7df09a Extend logging for debugging kubemark problems 2016-09-16 11:25:06 +02:00
Piotr Szczesniak 0f40f94dd9 Bumped Heapster to v1.2.0 2016-09-14 09:16:09 +02:00
Kubernetes Submit Queue 2076319020 Merge pull request #32362 from piosz/heap 
Automatic merge from submit-queue

Bumped Heapster to v1.2.0-beta.3

The new version contains Kubernetes deps in version `1.4.0-beta.0`
 2016-09-11 13:47:44 -07:00
Chao Xu 7e92025cfd stick to 2.2.1 etcd 2016-09-09 16:35:05 -07:00
Piotr Szczesniak 2d87deb043 Bumped Heapster to v1.2.0-beta.3 2016-09-09 11:41:48 +02:00
Daniel Smith 279219ca89 Use etcd 2.3.7 (container already pushed) 2016-09-09 09:45:53 +02:00
Wojciech Tyczynski 752f414905 Store logs from etcd in kubemark 2016-08-26 11:46:02 +02:00
Wojciech Tyczynski 52498e6050 Retry etcd starting in kubemark 2016-08-26 11:46:02 +02:00
Wojciech Tyczynski 56008de8d6 Fix heapster in kubemark 2016-08-22 15:38:02 +02:00
Kubernetes Submit Queue 6659d80973 Merge pull request #30882 from gmarek/address 
Automatic merge from submit-queue

Update address flag to insecure-bind-address in kubemark
 2016-08-18 07:53:03 -07:00
gmarek 97a376c4ea Update address flag to insecure-bind-address in kubemark 2016-08-18 15:57:20 +02:00
gmarek 7e5770b347 Add --verify-version-skew=false to kubemark test runner 2016-08-18 13:43:38 +02:00
Timothy St. Clair 730fc70107 Update etcd default ports for v3, and validate tests 2016-08-17 07:49:19 -05:00
Timothy St. Clair a0d177ca71 Update core etcd references to use 3.0.4 2016-08-12 13:39:07 -05:00
Wojciech Tyczynski dcd5c2f9e6 Mounted etcd directories in Kubemark 2016-08-11 11:37:30 +02:00
gmarek d0eb29cc67 Change the name of kubemark config file 2016-08-10 16:04:14 +02:00
gmarek 85f3bb271c Print location of kubemark kubeconfig at the end of start-kubemark 2016-08-10 13:50:19 +02:00
gmarek 8b17ec1447 fix kubemark 2016-08-10 12:42:47 +02:00
gmarek ea26433a9c Debug kubemark startup problems 2016-08-10 11:58:22 +02:00
Kubernetes Submit Queue 012eb941d6 Merge pull request #28575 from fejta/check-resources 
Automatic merge from submit-queue

Move kubemark, skew, upgrade, leak logic into hack/e2e.go

And move out of the `e2e_runner.sh`
Also make `start-kubemark.sh` automatically find `kubernetes-server-linux-amd64.tar.gz` instead of depending on a `RUN_FROM_DISTRO` env var

@gmarek can you review the kubemark-related changes?
@zmerlynn are you the right person to review upgrade changes?

@spxtr take a look at the overall `e2e.go` changes.

This is for https://github.com/kubernetes/kubernetes/issues/21843 and https://github.com/kubernetes/test-infra/pull/250

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/28575)
<!-- Reviewable:end -->
 2016-08-09 15:54:50 -07:00
Erick Fejta 7011d8826e Move kubemark, skew, upgrade, leak logic into hack/e2e.go 2016-08-09 14:04:19 -07:00
Timothy St. Clair 456c43c22d Update to shift etcd 2.2 references to use 3.0.3 2016-08-09 08:51:15 -05:00
Wojciech Tyczynski 2280737aa4 Set cache size in Kubemark 2016-08-09 13:34:13 +02:00
Hongchao Deng 74e6626967 Update etcd deprecated flag 
- "-addr" => "--advertise-client-urls"
- "-bind-addr" => "--listen-client-urls"
 2016-07-22 11:01:11 -07:00
Wojciech Tyczynski 28205d6a43 Fix Kubemark config after IP addresses machinations 2016-07-22 11:49:15 +02:00
Antoine Pelisse ab3e4494d3 Tentatively fix broken kubemark tests 
The multi-line bash command is ill-formed.
 2016-07-21 20:39:04 -07:00
Filip Grzadkowski a2c2f7d8f7 Add static IP address for kubemark master. 2016-07-21 21:03:18 +02:00
David McMahon ef0c9f0c5b Remove "All rights reserved" from all the headers. 2016-06-29 17:47:36 -07:00
k8s-merge-robot d1cc7f9e2c Merge pull request #27037 from wojtek-t/push_hollow_nodes_logs_to_kubelets 
Automatic merge from submit-queue

Mount hollow-node logs to parent node hostpath
 2016-06-27 22:40:52 -07:00
Piotr Szczesniak 8fff5319db Bumped Heapster to v1.1.0 2016-06-16 20:41:28 +02:00
Wojciech Tyczynski 770bd6b7a4 Mount hollow-node logs to parent node hostpath 2016-06-08 13:24:49 +02:00
gmarek 5109776485 Add retries for make in start-kubemark 2016-05-30 16:51:30 +02:00
gmarek cafbdba7d9 Change base64 -d flag to --decode flag to make it BSD compatible 2016-05-24 13:24:55 +02:00
Wojciech Tyczynski 6f43de4740 Don't dump everything in kubemarks 2016-05-23 11:53:40 +02:00
Mike Danese c2c030b6d8 fix kubemark 2016-05-20 15:42:40 -07:00
Wojciech Tyczynski fe470b664b Pipe content-type variable to hollow node 2016-05-11 14:57:40 +02:00
gmarek 4a5ec11ce6 Pipe variables from kubemark config to master binaries 2016-05-11 11:22:03 +02:00
gmarek 7981f479de Make Kubemark assign cluster CIDRs 2016-05-09 15:09:37 +02:00
gmarek b14809832b Add heapster to kubemark 2016-04-27 16:04:07 +02:00
gmarek 3627bb7be9 Add Services to Load test 2016-04-21 22:00:26 +02:00
Wojciech Tyczynski 0f60b33efc Don't remove kubemark binaries 2016-04-13 12:07:34 +02:00
gmarek 9120a60724 Kubemark uses kubectl from the repo 2016-03-31 09:13:42 -07:00
Joe Finney 9845639e14 Remove cluster/kube-env.sh. 2016-03-08 16:57:31 -08:00
Wojciech Tyczynski ca72f9908c Enable parallel namespace deletion in kubemarks and enormous cluster 2016-02-29 10:54:59 +01:00
gmarek 0cf1b922c4 Add more logging in case of Kubemark startup failure 2016-02-18 16:32:04 +01:00
Wojciech Tyczynski c9d6b5fc74 Unify running tests in Kubemark 2016-02-16 13:44:52 +01:00
k8s-merge-robot 62e574341c Merge pull request #20976 from wojtek-t/remove_annoying_logs 
Auto commit by PR queue bot
 2016-02-11 06:51:15 -08:00
Wojciech Tyczynski 519021b978 Timeout creating kubemark nodes 2016-02-11 12:19:37 +01:00
Wojciech Tyczynski 932302fa95 Remove annoying logs from Kubemark 2016-02-10 16:26:53 +01:00
gmarek de740236b8 Randomize password to kubemark master 2016-02-02 15:36:08 +01:00
gmarek 35b317c094 Kubemark scripts retry failed gcloud commands 2016-02-01 21:05:57 +01:00
Hongchao Deng 5c68e9171d clean up generated content in stop-kubemark.sh 2016-01-23 16:07:00 -08:00
gmarek 13773cbdf9 Remove cluster-name from kubemark setup 2015-12-16 16:10:52 +01:00
Brad Erickson 53172a5356 Minion->Node rename: NUM_NODES 2015-11-25 00:45:10 -08:00
Hongchao Deng 1fde840aa8 kubemark: check if kubectl is installed in start script 2015-11-18 09:28:05 -08:00
Joe Beda 993c1ee222 Update etcd to 2.2.1 2015-11-04 12:26:57 -08:00
gmarek ce3317cb9f Don't delete namespace after kubemark tests, as cluster will be torn down either way 2015-10-29 18:53:56 +01:00
gmarek 5a4e4d4d57 Add a HollowProxy to the HollowNode. 2015-10-27 18:23:29 +01:00
gmarek 3158741603 Run etcd for events on separate machine in Kubemark 2015-10-20 17:46:23 +02:00
gmarek 5184b40086 API server now must be started before controller-manager. Fix this in Kubemark startup script 2015-10-16 16:15:14 +02:00
gmarek 0574e0e21f Bash FTW - fix for run-e2e-tests.sh 2015-10-14 17:20:49 +02:00
gmarek 3a609c971d Fix to stop-kubemark script 2015-10-14 09:30:48 +02:00
gmarek 4625f50131 Small changes in Kubemark requested by @wojtek-t 2015-10-13 16:26:19 +02:00
gmarek b88eb80a59 Prefix hollow-cluster-master name with instance prefix 2015-10-12 20:15:59 +02:00
gmarek ea7f914394 Fix HollowNode template 2015-10-12 15:46:32 +02:00
gmarek 5d9c54edb1 Make Kubemark master components write logs to the /var/logs 2015-10-09 12:33:53 +02:00
gmarek 28d79d83c1 Create a separate etcd instance for events in Kubemark 2015-10-07 17:15:55 +02:00
gmarek 4dfd28086f Wait for all HollowNodes to become ready in start-kubemark.sh 2015-10-06 09:20:53 +02:00
gmarek 22574cfd6d Set E2E_MIN_STARTUP_PODS to 0 for Kubemark runs 2015-10-02 15:10:02 +02:00
gmarek 3137ecbf3b Create a firewall rule for kubemark master allowing connections on port 443 2015-10-02 14:36:31 +02:00
gmarek 1e7cdf95bd Add PROJECT to things added by SSL patch 2015-10-02 11:07:34 +02:00
gmarek bc76b1616f Add SSL to communication in Kubemark 2015-10-02 10:39:24 +02:00
gmarek e58aff001b Stop existing Kubemark cluster before creating new one in Jenkins 2015-10-02 10:32:57 +02:00
gmarek b7954068a3 Add missing PROJECT flag to Kubemark scripts 2015-10-02 09:42:02 +02:00
gmarek 9a74980da2 Make kubemark work on Jenkins 2015-10-01 15:26:39 +02:00
gmarek 7c6052187c Small fixes to kubemark scripts 2015-09-30 09:26:18 +02:00
Marek Grabowski 7fa09abecb Merge pull request #14776 from brendandburns/fixer2 
Fix some issues with the kubemark script
 2015-09-30 08:58:15 +02:00
Brendan Burns a26afdad30 Fix some issues with the kubemark script 2015-09-29 17:27:19 -07:00
gmarek 7cfb3479e0 Small fixes to Kubemark scripts 2015-09-29 10:24:46 +02:00
gmarek dd3bd73948 Initial Kubemark commit 2015-09-28 11:39:05 +02:00