b95df1d745
perf: use io copy with buffer pool ( #6389 )
...
* feat: add io methods with buffer
* chore: move io.Copy calls to utils.CopyWithBuffer
2024-04-25 20:11:15 +08:00
769281bd40
feat: refactor offline download ( #5408 close #4108 )
...
* wip: refactor offline download (#5331 )
* base tool
* working: aria2
* refactor: change type of percentage to float64
* wip: adapt aria2
* wip: use items in offline_download
* wip: use tool manager
* wip: adapt qBittorrent
* chore: fix typo
* Squashed commit of the following:
commit 4fc0a77565close #5392 )
commit aaffaee2b5#5391 )
commit 8ef8023c20#5398 )
commit cdfbe6dcf2#5394 )
commit 94d028743a7f7335435c#5373 close #5348 )
* feat(cloudreve): support thumbnail
* chore: remove unnecessary code
commit b9e192b29c#5367 close #5275 )
* fix(115):limit request rate
* chore(115): fix unit of `limit_rate`
---------
Co-authored-by: Andy Hsu <i@nn.ci>
commit 69a98eaef6#5141 )
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
commit 1ebc96a4e5close #5352 )
commit 66e2324cac7600dc28dfclose #5358 )
commit 8ef89ad0a4#5356 )
* fix(baidu):hash and error:2
* fix:invalid memory address
commit 35d672217d#5346 )
commit 1a283bb272#5334 )
commit a008f54f4d#5329 ) [skip ci]
* fix: adapt update progress type
* Squashed commit of the following:
commit 65c5ec0c34#5457 close #5395 )
commit a6325967d0#5234 )
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
commit 4dff49470a#5366 )
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
commit cc86d6f3d1#5370 )
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
commit c0f9c8ebafclose #5434 )
2023-11-06 16:56:55 +08:00
7200344ace
feat: adapt hash feature for some drivers ( #5180 )
...
* feat(pikpak,thunder): adaptation gcid hash
* chore(weiyun): add note
* feat(baidu_netdisk): adaptation rapid
* feat(baidu_photo): adaptation hash
* feat(189pc): adaptation rapid
* feat(mopan):adaptation ctime
* feat(139):adaptation hash and ctime
---------
Co-authored-by: Andy Hsu <i@nn.ci>
2023-09-06 14:46:35 +08:00
a3748af772
feat: misc improvements about upload/copy/hash ( #5045 )
...
general: add createTime/updateTime support in webdav and some drivers
general: add hash support in some drivers
general: cross-storage rapid-upload support
general: enhance upload to avoid local temp file if possible
general: replace readseekcloser with File interface to speed upstream operations
feat(aliyun_open): same as above
feat(crypt): add hack for 139cloud
Close #4934
Close #4819
baidu_netdisk needs to improve the upload code to support rapid-upload
2023-08-27 21:14:23 +08:00
c1db3a36ad
feat: upload progress recovery ( #4987 )
...
* feat(189pc):upload progress recovery
* fix:some err
* feat(baidu_netdisk,baidu_photo):upload progress recovery
* feat(mopan):upload progress recovery
* feat(baidu_netdisk):custom upload api
2023-08-11 14:23:30 +08:00
df6b306fce
perf(drivers): fs operations and cache ( #4965 )
...
* perf(baidu_photo):multi-thread upload
* perf(baidu_netdisk):multi-thread upload and cache optimization
* fix:LimitWriter
* fix(weiyun):only one login is allowed
* feat(189pc):multi threaded upload
* feat(baidu_netdisk):multi threaded upload
* feat(baidu_photo):multi threaded upload
* feat(weiyun):multi threaded upload
* perf(aliyundriver_open):optimize upload code and optimize cache
* fix(weiyun):invalid directory ID
* fix(baidu_netdisk):modified time
* fix(baidu_netdisk,baidu_photo):upload slice error
* perf(baidu_netdisk):cancel unnecessary retries
* fix(limitWriter):must return a non-nil error if it returns n < len(p)
* fix(aliyundrive_open):Name and Filename only use one
* perf(mopan):multi-thread upload
2023-08-09 16:13:09 +08:00
7877184bee
feat(baidu_netdisk): add retry to most operations ( close #4863 in #4939 )
2023-08-07 13:44:28 +08:00
3c21a9a520
feat: Crypt driver, improve http/webdav handling ( #4884 )
...
this PR has several enhancements, fixes, and features:
- [x] Crypt: a transparent encryption driver. Anyone can easily, and safely store encrypted data on the remote storage provider. Consider your data is safely stored in the safe, and the storage provider can only see the safe, but not your data.
- [x] Optional: compatible with [Rclone Crypt](https://rclone.org/crypt/ ). More ways to manipulate the encrypted data.
- [x] directory and filename encryption
- [x] server-side encryption mode (server encrypts & decrypts all data, all data flows thru the server)
- [x] obfuscate sensitive information internally
- [x] introduced a server memory-cached multi-thread downloader.
- [x] Driver: **Quark** enabled this feature, faster load in any single thread scenario. e.g. media player directly playing from the link, now it's faster.
- [x] general improvement on HTTP/WebDAV stream processing & header handling & response handling
- [x] Driver: **Mega** driver support ranged http header
- [x] Driver: **Quark** fix bug of not closing HTTP request to Quark server while user end has closed connection to alist
## Crypt, a transparent Encrypt/Decrypt Driver. (Rclone Crypt compatible)
e.g.
Crypt mount path -> /vault
Crypt remote path -> /ali/encrypted
Aliyun mount paht -> /ali
when the user uploads a.jpg to /vault, the data will be encrypted and saved to /ali/encrypted/xxxxx. And when the user wants to access a.jpg, it's automatically decrypted, and the user can do anything with it.
Since it's Rclone Crypt compatible, users can download /ali/encrypted/xxxxx and decrypt it with rclone crypt tool. Or the user can mount this folder using rclone, then mount the decrypted folder in Linux...
NB. Some breaking changes is made to make it follow global standard, e.g. processing the HTTP header properly.
close #4679
close #4827
Co-authored-by: Sean He <866155+seanhe26@users.noreply.github.com>
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-02 14:40:36 +08:00
9f08353d31
feat(baidu_photo): optional delete album origin file ( close #4872 in #4875 )
2023-07-31 18:29:45 +08:00
06f46206db
fix(baidu_photo): album download ( close #4603 in #4871 )
...
Co-authored-by: Andy Hsu <i@nn.ci>
2023-07-31 16:27:16 +08:00
ef68f84787
fix(baidu_photo): legal album title check ( close #4479 in #4487 )
2023-05-27 17:07:57 +08:00
d41d868a8d
fix(baidu_photo): change folder name length limit ( close #4351 in #4353 )
...
Co-authored-by: Andy Hsu <i@nn.ci>
2023-05-09 20:44:57 +08:00
49f13b9b90
fix(baidu_photo): upload file has web prefix ( close #4233 in #4235 )
2023-04-24 19:13:33 +08:00
8b523fab8b
revert: add `Getter` interface back
2023-01-16 19:55:43 +08:00
a1ded3a339
refactor(baidu_photo): optimize code ( close #2911 pr #2924 )
2023-01-06 15:36:05 +08:00
105f22969c
feat: support cancel for some drivers ( close #2717 )
2022-12-21 15:03:09 +08:00
fb64f00640
refactor: obj name mapping and internal path processing ( #2733 )
...
* refactor:Prepare to remove the get interface
* feat:add obj Unwarp interface
* refactor:obj name mapping and program internal path processing
* chore: fix typo
* feat: unwrap get
* fix: no use op.Get to get parent id
* fix: set the path uniformly
Co-authored-by: Noah Hsu <i@nn.ci>
2022-12-17 19:49:05 +08:00
33bae52fa1
refactor: optimize driver initialization need to manually deserialize and assign values, and remove redundant driver registration parameters ( #2691 )
...
* refactor: optimize driver initialization need to manually deserialize and assign values, and remove redundant driver registration parameters
* fix typo
Co-authored-by: Noah Hsu <i@nn.ci>
2022-12-13 18:03:30 +08:00
83644dab85
fix: mapping filename in `GetName`
...
some missed filename mapping
2022-11-30 20:46:54 +08:00
af946ff13e
fix(baidu_photo): cannot download when proxy is opened
2022-09-23 01:15:12 +08:00
f22232de5d
chore: baidu_photo rename only duplicate folders
2022-09-15 09:25:20 +08:00
a90ef201c7
fix(189pc,baidu_photo,thunder): single link limit multithreading
2022-09-13 18:44:07 +08:00
53e08e75fe
fix(189pc,baidu_photo): source file not closed
2022-09-12 22:45:30 +08:00
6b5236f52e
feat: add baidu_photo driver
2022-09-12 17:10:02 +08:00
Mmx