Commit Graph

48 Commits (permission)

Author SHA1 Message Date
Yifan Gao 0a9921fa79
fix(aliyundrive_open): resolve file duplication issues and improve path handling (#8358)
* fix(aliyundrive_open): resolve file duplication issues and improve path handling

1. Fix file duplication by implementing a new removeDuplicateFiles method that cleans up duplicate files after operations
2. Change Move operation to use "ignore" for check_name_mode instead of "refuse" to allow moves when destination has same filename
3. Set Copy operation to handle duplicates by removing them after successful copy
4. Improve path handling for all file operations (Move, Rename, Put, MakeDir) by properly maintaining the full path of objects
5. Implement GetRoot interface for proper root object initialization with correct path
6. Add proper path management in List operation to ensure objects have correct paths
7. Fix path handling in error cases and improve logging of failures

* refactor(aliyundrive_open): change error logging to warnings for duplicate file removal

Updated the Move, Rename, and Copy methods to log warnings instead of errors when duplicate file removal fails, as the primary operations have already completed successfully. This improves the clarity of logs without affecting the functionality.

* Update drivers/aliyundrive_open/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-04-19 14:22:12 +08:00
j2rong4cn ddffacf07b
perf: optimize IO read/write usage (#8243)
* perf: optimize IO read/write usage

* .

* Update drivers/139/driver.go

Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>

---------

Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>
2025-04-12 16:55:31 +08:00
KirCute_ECT 3b71500f23
feat(traffic): support limit task worker count & file stream rate (#7948)
* feat: set task workers num & client stream rate limit

* feat: server stream rate limit

* upgrade xhofe/tache

* .
2025-02-16 12:22:11 +08:00
j2rong4cn ab22cf8233
feat: add `Reference` interface to driver (#7805)
* feat: add `Reference` interface to driver

* feat(123_share): support reference 123pan
2025-01-18 23:26:58 +08:00
安稳 f58de9923a
refactor(aliyunopen,config): Modify default properties (#7476) 2024-11-08 22:07:35 +08:00
Mmx b95df1d745
perf: use io copy with buffer pool (#6389)
* feat: add io methods with buffer

* chore: move io.Copy calls to utils.CopyWithBuffer
2024-04-25 20:11:15 +08:00
NewbieOrange e37465e67e
feat(crypt): force stream upload for supported drivers (#6270) 2024-03-29 14:42:01 +08:00
xiaofei 6b8f35e7fa
feat(alipan): replace domain (#5751 close #5747) 2023-12-31 14:29:14 +08:00
Andy Hsu 296be88b5f fix: incorrect key of oidc username (close #5670) 2023-12-10 13:17:56 +08:00
BlueSkyXN 0fbb986ba9
fix(aliyundrive_open): mitigation measures for 15-minute limit (#5560 close #5547)
* fix(aliyundrive_open):Mitigation measures for AliOpen's 15-minute limit.

I conducted small-scale tests, which seem to have no significant negative impact. If the 15-minute issue still occurs, further measures will be needed. Methods like local proxy can be attempted.

* chore(aliyundrive_open): change cache of the link to 1 minute

---------

Co-authored-by: Andy Hsu <i@nn.ci>
2023-11-23 21:49:16 +08:00
Andy Hsu 769281bd40
feat: refactor offline download (#5408 close #4108)
* wip: refactor offline download (#5331)

* base tool

* working: aria2

* refactor: change type of percentage to float64

* wip: adapt aria2

* wip: use items in offline_download

* wip: use tool manager

* wip: adapt qBittorrent

* chore: fix typo

* Squashed commit of the following:

commit 4fc0a77565
Author: Andy Hsu <i@nn.ci>
Date:   Fri Oct 20 21:06:25 2023 +0800

    fix(baidu_netdisk): upload file > 4GB (close #5392)

commit aaffaee2b5
Author: gmugu <94156510@qq.com>
Date:   Thu Oct 19 19:17:53 2023 +0800

    perf(webdav): support request with cookies (#5391)

commit 8ef8023c20
Author: NewbieOrange <NewbieOrange@users.noreply.github.com>
Date:   Thu Oct 19 19:17:09 2023 +0800

    fix(aliyundrive_open): upload progress for normal upload (#5398)

commit cdfbe6dcf2
Author: foxxorcat <95907542+foxxorcat@users.noreply.github.com>
Date:   Wed Oct 18 16:27:07 2023 +0800

    fix: hash gcid empty file (#5394)

commit 94d028743a
Author: Andy Hsu <i@nn.ci>
Date:   Sat Oct 14 13:17:51 2023 +0800

    ci: remove `pr-welcome` label when close issue [skip ci]

commit 7f7335435c
Author: itsHenry <2671230065@qq.com>
Date:   Sat Oct 14 13:12:46 2023 +0800

    feat(cloudreve): support thumbnail (#5373 close #5348)

    * feat(cloudreve): support thumbnail

    * chore: remove unnecessary code

commit b9e192b29c
Author: foxxorcat <95907542+foxxorcat@users.noreply.github.com>
Date:   Thu Oct 12 20:57:12 2023 +0800

    fix(115): limit request rate (#5367 close #5275)

    * fix(115):limit request rate

    * chore(115): fix unit of `limit_rate`

    ---------

    Co-authored-by: Andy Hsu <i@nn.ci>

commit 69a98eaef6
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Wed Oct 11 22:01:55 2023 +0800

    fix(deps): update module github.com/aliyun/aliyun-oss-go-sdk to v2.2.9+incompatible (#5141)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 1ebc96a4e5
Author: Andy Hsu <i@nn.ci>
Date:   Tue Oct 10 18:32:00 2023 +0800

    fix(wopan): fatal error concurrent map writes (close #5352)

commit 66e2324cac
Author: Andy Hsu <i@nn.ci>
Date:   Tue Oct 10 18:23:11 2023 +0800

    chore(deps): upgrade dependencies

commit 7600dc28df
Author: Andy Hsu <i@nn.ci>
Date:   Tue Oct 10 18:13:58 2023 +0800

    fix(aliyundrive_open): change default api to raw server (close #5358)

commit 8ef89ad0a4
Author: foxxorcat <95907542+foxxorcat@users.noreply.github.com>
Date:   Tue Oct 10 18:08:27 2023 +0800

    fix(baidu_netdisk): hash and `error 2` (#5356)

    * fix(baidu):hash and error:2

    * fix:invalid memory address

commit 35d672217d
Author: jeffmingup <1960588251@qq.com>
Date:   Sun Oct 8 19:29:45 2023 +0800

    fix(onedrive_app): incorrect api on `_accessToken` (#5346)

commit 1a283bb272
Author: foxxorcat <95907542+foxxorcat@users.noreply.github.com>
Date:   Fri Oct 6 16:04:39 2023 +0800

    feat(google_drive): add `hash_info`, `ctime`, `thumbnail` (#5334)

commit a008f54f4d
Author: nkh0472 <67589323+nkh0472@users.noreply.github.com>
Date:   Thu Oct 5 13:10:51 2023 +0800

    docs: minor language improvements (#5329) [skip ci]

* fix: adapt update progress type

* Squashed commit of the following:

commit 65c5ec0c34
Author: itsHenry <2671230065@qq.com>
Date:   Sat Nov 4 13:35:09 2023 +0800

    feat(cloudreve): folder size count and switch (#5457 close #5395)

commit a6325967d0
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Mon Oct 30 15:11:20 2023 +0800

    fix(deps): update module github.com/charmbracelet/lipgloss to v0.9.1 (#5234)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 4dff49470a
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Mon Oct 30 15:10:36 2023 +0800

    fix(deps): update golang.org/x/exp digest to 7918f67 (#5366)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit cc86d6f3d1
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Sun Oct 29 14:45:55 2023 +0800

    fix(deps): update module golang.org/x/net to v0.17.0 [security] (#5370)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit c0f9c8ebaf
Author: Andy Hsu <i@nn.ci>
Date:   Thu Oct 26 19:21:09 2023 +0800

    feat: add ignore direct link params (close #5434)
2023-11-06 16:56:55 +08:00
NewbieOrange 8ef8023c20
fix(aliyundrive_open): upload progress for normal upload (#5398) 2023-10-19 19:17:09 +08:00
Andy Hsu 7600dc28df fix(aliyundrive_open): change default api to raw server (close #5358) 2023-10-10 18:13:58 +08:00
LaoShui 4024050dd0
chore: fix typo (#5316) 2023-10-01 16:58:00 +08:00
Sean 8e40465e86
fix(aliyundrive_open): date format on uploading (#5151)
(cherry picked from commit 88f815979ac91caa8bc425a2ff9a18bbd8a2e736)
2023-09-03 18:12:05 +08:00
Sean e7c0d94b44
fix: form upload when ticked `As A Task` (#5145) 2023-09-03 15:40:40 +08:00
Andy Hsu dd10c0c5d0 chore(aliyundrive_open): print resp content on refresh token (close #5129) 2023-08-31 18:43:25 +08:00
Sean a3748af772
feat: misc improvements about upload/copy/hash (#5045)
general: add createTime/updateTime support in webdav and some drivers
general: add hash support in some drivers
general: cross-storage rapid-upload support
general: enhance upload to avoid local temp file if possible
general: replace readseekcloser with File interface to speed upstream operations
feat(aliyun_open): same as above
feat(crypt): add hack for 139cloud

Close #4934 
Close #4819 

baidu_netdisk needs to improve the upload code to support rapid-upload
2023-08-27 21:14:23 +08:00
Andy Hsu 13e8d36e1a fix(aliyundrive_open): use `RawStdEncoding` for base64 2023-08-13 20:52:38 +08:00
foxxorcat df6b306fce
perf(drivers): fs operations and cache (#4965)
* perf(baidu_photo):multi-thread upload

* perf(baidu_netdisk):multi-thread upload and cache optimization

* fix:LimitWriter

* fix(weiyun):only one login is allowed

* feat(189pc):multi threaded upload

* feat(baidu_netdisk):multi threaded upload

* feat(baidu_photo):multi threaded upload

* feat(weiyun):multi threaded upload

* perf(aliyundriver_open):optimize upload code and optimize cache

* fix(weiyun):invalid directory ID

* fix(baidu_netdisk):modified time

* fix(baidu_netdisk,baidu_photo):upload slice error

* perf(baidu_netdisk):cancel unnecessary retries

* fix(limitWriter):must return a non-nil error if it returns n < len(p)

* fix(aliyundrive_open):Name and Filename only use one

* perf(mopan):multi-thread upload
2023-08-09 16:13:09 +08:00
Andy Hsu b91ed7a78a fix(aliyundrive_open): retry refresh token if sub not match 2023-08-08 22:08:05 +08:00
Andy Hsu 95386d777b feat(aliyundrive_open): record token exchange 2023-08-08 20:38:13 +08:00
Sean 7877184bee
feat(baidu_netdisk): add retry to most operations (close #4863 in #4939) 2023-08-07 13:44:28 +08:00
Andy Hsu a69d72aa20 feat(aliyundrive_open): support resource drive (close #4889) 2023-08-02 15:50:01 +08:00
Andy Hsu 1dc1dd1f07 feat(aliyundrive_open): support `livp` format file download (close #4890) 2023-08-01 21:50:25 +08:00
Andy Hsu 7e9cdd8b07 fix(aliyundrive_open): fail limit on concurrently call (#4851) 2023-07-28 15:55:39 +08:00
Andy Hsu 2e23ea68d4 fix(aliyundrive_open): increase limit interval (close #4851) 2023-07-27 18:26:11 +08:00
Andy Hsu 6ec9a8d4c7 fix(aliyundrive_open): the temp file is not delete (close #4777) 2023-07-16 15:01:22 +08:00
Andy Hsu 3f7882b467 feat(aliyundrive_open): rapid upload (close #4766) 2023-07-15 19:33:46 +08:00
Andy Hsu 4fabc27366 fix(aliyundrive_open): panic if driver not init 2023-07-05 13:51:46 +08:00
william 1c8fe3b24c
fix(aliyundrive_open): adaptive part size adjustment (#4609)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-06-23 14:25:30 +08:00
Andy Hsu 6cd8151cad fix(aliyundrive_open): change default `oauth_token_url` 2023-06-16 15:03:27 +08:00
Andy Hsu ca5a53fc24 fix(aliyundrive_open): `openFile/list` rate limit 2023-06-11 18:18:09 +08:00
Andy Hsu c77ed5fcb0 feat(aliyundrive_open): limit rate for `List` and `Link` (close #4290) 2023-05-02 22:06:03 +08:00
Andy Hsu 822be17fb9 feat(aliyundrive_open): add expiration for link (close #4061) 2023-05-02 16:12:40 +08:00
Noah Hsu b7ea73b3c2 fix(aliyundrive_open): can't refresh token if access_token is empty (#4255) 2023-04-28 00:01:47 +08:00
Andy Hsu 9fbc54314d chore(aliyundrive_open): change base url 2023-04-27 16:38:40 +08:00
NewbieOrange 4ec274e748
fix(aliyundrive_open): refresh upload url if expired (#3999 close #3823)
* fix(aliyundrive_open): refresh upload url for large files

* fix(aliyundrive_open): retry upload on url expiry

* fix(aliyundrive_open): ignore 409 error

* feat(aliyundrive): cleanup upload retry logic

* feat(util): add multireadable io utility

* feat(aliyundrive_open): make upload fully stream

* feat(aliyundrive_open): refresh upload url every 20 puts

* fix(aliyundrive_open): part info panic

* chore: change refresh upload url strategy

---------

Co-authored-by: Andy Hsu <i@nn.ci>
2023-04-01 14:54:29 +08:00
Wuxuan 0c5820a98f
docs(aliyundrive_open): revised the sentence that may cause ambiguity (#3989) 2023-03-29 20:26:21 +08:00
Andy Hsu aeb48b2ecc perf(aliyundrive_open): don't refresh token on init if token valid 2023-03-20 15:00:02 +08:00
Wuxuan 02d0aef611
feat(aliyundrive_open): add internal upload (aliyun ECS for Beijing area only) (#3775) 2023-03-09 20:48:30 +08:00
Andy Hsu 5596661ce8 feat(aliyundrive_open): optional delete file directly (close #3769) 2023-03-08 19:19:13 +08:00
Andy Hsu c915313ec9 feat: rename then delete if storage doesn't support overwrite upload (close #3643) 2023-03-05 15:36:12 +08:00
Andy Hsu 4dd753de52 fix(aliyundrive_open): missed `expire_sec` while get link (close #3610) 2023-02-25 17:54:36 +08:00
Andy Hsu 79df63d319 chore(aliyundrive): change alert info 2023-02-25 14:28:27 +08:00
Andy Hsu ec54831162 fix: only refresh token while do request (close #3591) 2023-02-24 20:31:12 +08:00
Andy Hsu 4be8524d80 feat: add alert for driver 2023-02-23 22:03:11 +08:00
Andy Hsu 28aee8c493
feat: add aliyundrive open driver (#3437)
close #3533 
close #3521 
close #3459 
close #3375 

* feat: add aliyundrive open driver

* feat: adapt alist api

* fix: trailing spaces

* feat(aliyundrive_open): video preview api
2023-02-23 20:45:57 +08:00