c2216a62ad
Use pagination when retrieving etcd snapshot list
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2024-07-24 12:44:29 -07:00
c36db53e54
Add etcd s3 config secret implementation
...
* Move snapshot structs and functions into pkg/etcd/snapshot
* Move s3 client code and functions into pkg/etcd/s3
* Refactor pkg/etcd to track snapshot and s3 moves
* Add support for reading s3 client config from secret
* Add minio client cache, since S3 client configuration can now be
changed at runtime by modifying the secret, and don't want to have to
create a new minio client every time we read config.
* Add tests for pkg/etcd/s3
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2024-07-10 13:13:55 -07:00
aa4794b372
Replace 1-weight semaphore on snapshots with simple mutex
...
Fixes an issue where the semaphore wasn't permanently initialized
until a scheduled snapshot was taken, allowing multiple on-demand
snapshots to be taken until the first scheduled snapshot was triggered.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2024-06-19 09:47:58 -07:00
fe465cc832
Move etcd snapshot management CLI to request/response
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2024-04-09 15:21:26 -07:00
14f54d0b26
Transition from deprecated pointer library to ptr ( #9801 )
...
Signed-off-by: Derek Nola <derek.nola@suse.com>
2024-03-28 10:07:02 -07:00
edb0440017
Fix etcd snapshot reconcile for agentless nodes
...
Disable cleanup of orphaned snapshots and patching of node annotations if running agentless
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2024-03-27 16:44:36 -07:00
3576ed4327
Clean up snapshotDir create/exists logic
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2024-03-04 12:09:29 -08:00
6ec1926f88
Add check for etcd-snapshot-dir and fix panic in Walk
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2024-02-06 17:47:33 -08:00
b23e70d519
Don't apply s3 retention if S3 client failed to initialize
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-11-21 14:03:27 -08:00
d885162967
Add server token hash to CR and S3
...
This required pulling the token hash stuff out of the cluster package, into util.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
550ab36ab7
Switch to managing ETCDSnapshotFile resources
...
Reconcile snapshot CRs instead of ConfigMap; manage ConfigMap downstream from CR list
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
5cd4f69bfa
Move snapshot delete into local/s3 functions
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
7464007037
Store extra metadata and cluster ID for snapshots
...
Write the extra metadata both locally and to S3. These files are placed such that they will not be used by older versions of K3s that do not make use of them.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
80f909d0ca
Move s3 snapshot list functionality to s3.go
...
Also, don't list ONLY s3 snapshots if S3 is enabled.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
8d47645312
Consistently set snapshotFile timestamp
...
Attempt to use timestamp from creation or filename instead of file/object modification times
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
f1afe153a3
Tidy s3 upload functions
...
Consistently refer to object keys as such, simplify error handling.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
2b0e2e8ada
Elide old snapshot data when apiserver rejects configmap with ErrRequestEntityTooLarge
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
676b00aa0e
Move etcd snapshot code into separate file
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
Brad Davidson