6d3a92a658
Print key instead of file path in snapshot metadata log message
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-11-21 14:03:27 -08:00
a92c4a0f17
Don't request metadata when listing objects
...
While some implementations may support it, it appears that most don't,
and some may in fact return an error if it is requested.
We already stat the object to get the metadata anyway, so this was
unnecessary if harmless on most implementations.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-11-21 14:03:27 -08:00
2088218c5f
Fix issue with snapshot metadata configmap
...
Omit snapshot list configmap entries for snapshots without extra metadata; reduce log level of warnings about missing s3 metadata files.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-11-15 14:25:28 -08:00
49411e7084
Don't try to read token hash and cluster id during cluster-reset
...
These fields are only necessary when saving snapshots to S3, and will block restoration if attempted
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-27 15:06:29 -07:00
d885162967
Add server token hash to CR and S3
...
This required pulling the token hash stuff out of the cluster package, into util.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
550ab36ab7
Switch to managing ETCDSnapshotFile resources
...
Reconcile snapshot CRs instead of ConfigMap; manage ConfigMap downstream from CR list
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
5cd4f69bfa
Move snapshot delete into local/s3 functions
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
7464007037
Store extra metadata and cluster ID for snapshots
...
Write the extra metadata both locally and to S3. These files are placed such that they will not be used by older versions of K3s that do not make use of them.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
80f909d0ca
Move s3 snapshot list functionality to s3.go
...
Also, don't list ONLY s3 snapshots if S3 is enabled.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
8d47645312
Consistently set snapshotFile timestamp
...
Attempt to use timestamp from creation or filename instead of file/object modification times
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
f1afe153a3
Tidy s3 upload functions
...
Consistently refer to object keys as such, simplify error handling.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
2b0e2e8ada
Elide old snapshot data when apiserver rejects configmap with ErrRequestEntityTooLarge
...
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2023-10-12 15:04:45 -07:00
e83b1ba4aa
Fixed the etcd retention to delete orphaned snapshots based on the date ( #8177 )
...
* Fix retention using name instead of date
Signed-off-by: Vitor <vitor.savian@suse.com>
2023-08-14 18:48:59 -03:00
ca7aeed090
Etcd snapshots retention when node name changes ( #8099 )
...
Fixed the etcd retention to delete orphaned snapshots
Signed-off-by: Vitor <vitor.savian@suse.com>
2023-08-03 10:54:40 -03:00
06d81cb936
Replace deprecated ioutil package ( #6230 )
...
* Replace ioutil package
* check integration test null pointer
* Remove rotate retries
Signed-off-by: Derek Nola <derek.nola@suse.com>
2022-10-07 17:36:57 -07:00
ffe72eecc4
Address issues with etcd snapshots
...
* Increase the default snapshot timeout. The timeout is not currently
configurable from Rancher, and larger clusters are frequently seeing
uploads fail at 30 seconds.
* Enable compression for scheduled snapshots if enabled on the
command-line. The CLI flag was not being passed into the etcd config.
* Only set the S3 content-type to application/zip if the file is zipped.
* Don't run more than one snapshot at once, to prevent misconfigured
etcd snapshot cron schedules from stacking up.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2022-07-12 14:41:38 -07:00
9a849b1bb7
[master] changing package to k3s-io ( #4846 )
...
* changing package to k3s-io
Signed-off-by: Luther Monson <luther.monson@gmail.com>
Co-authored-by: Derek Nola <derek.nola@suse.com>
2022-03-02 15:47:27 -08:00
effcb15adb
Adds the ability to compress etcd snapshots ( #4866 )
2022-01-14 10:31:22 -07:00
ae4a1a144a
etcd snapshot functionality enhancements ( #4453 )
...
Signed-off-by: Chris Kim <oats87g@gmail.com>
2021-11-29 10:30:04 -08:00
0452f017c1
Add etcd s3 timeout ( #4207 )
2021-10-15 10:24:14 -07:00
f4cea90cb9
set transport to skip verify if se skip flag passed ( #4102 )
2021-09-28 10:13:50 -07:00
a1ec43e0b7
feat: add option to disable s3 over https
...
Signed-off-by: Devin Buhl <devin.kray@gmail.com>
2021-09-05 12:03:49 -04:00
b4eca61aeb
Prevent snapshot commands from creating empty snapshot directory ( #3783 )
...
Signed-off-by: dereknola <derek.nola@suse.com>
2021-08-09 09:04:18 -07:00
bba49ea447
Fix to allow prune to correctly cleanup custom named snapshots ( #3649 )
...
Signed-off-by: dereknola <derek.nola@suse.com>
2021-07-19 14:30:57 -07:00
f5fbb9a9a8
Export cli server flags and etcd restoration functions ( #3527 )
...
* Export cli server flags and etfd restoration functions
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
* export S3
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
2021-06-30 22:29:03 +02:00
e998cd110d
Add the ability to delete an etcd snapshot locally or from S3 ( #3277 )
...
* Add the ability to delete a given set of etcd snapshots from the CLI for locally stored and S3 store snapshots.
2021-05-07 16:10:04 -07:00
c5ad71ce0b
Collect and Store etcd Snapshots and Metadata ( #3239 )
...
* Add the ability to store local etcd snapshots and etcd snapshots stored in an S3 compatible object store in a ConfigMap.
2021-04-30 18:26:39 -07:00
4d1f9eda9d
Etcd Snapshot/Restore to/from S3 Compatible Backends ( #2902 )
...
* Add functionality for etcd snapshot/restore to and from S3 compatible backends.
* Update etcd restore functionality to extract and write certificates and configs from snapshot.
2021-03-03 11:14:12 -07:00
Brad Davidson