48f50a2ceb
fix(search): BuildIndex concurrency error ( #7035 )
2024-08-22 00:44:55 +08:00
ca30849e24
feat: add support for halalcloud driver ( #6696 )
2024-07-07 13:20:34 +08:00
163af0515f
fix(pikpak): refresh_token contention ( #6501 close #6511 )
2024-05-27 21:31:59 +08:00
b95df1d745
perf: use io copy with buffer pool ( #6389 )
...
* feat: add io methods with buffer
* chore: move io.Copy calls to utils.CopyWithBuffer
2024-04-25 20:11:15 +08:00
d8e190406a
feat(189pc): add family transfer upload ( #6288 )
...
* feat(189pc): add family transfer upload
* fix(189):family transfer file delete
2024-04-02 16:51:02 +08:00
f1979a8bbc
feat(search): search with `meilisearch` ( #6060 )
...
* feat(search): search with meilisearch.
* feat(search): meilisearch supports auto update.
* chores: remove utils.Log.
* fix(search): the null pointer caused by deleting non-existing file/folder indexes.
---------
Co-authored-by: Andy Hsu <i@nn.ci>
2024-02-23 15:37:40 +08:00
4355dae491
fix: incorrect content-type of apk files ( close #5385 )
2023-11-06 18:20:25 +08:00
769281bd40
feat: refactor offline download ( #5408 close #4108 )
...
* wip: refactor offline download (#5331 )
* base tool
* working: aria2
* refactor: change type of percentage to float64
* wip: adapt aria2
* wip: use items in offline_download
* wip: use tool manager
* wip: adapt qBittorrent
* chore: fix typo
* Squashed commit of the following:
commit 4fc0a77565close #5392 )
commit aaffaee2b5#5391 )
commit 8ef8023c20#5398 )
commit cdfbe6dcf2#5394 )
commit 94d028743a7f7335435c#5373 close #5348 )
* feat(cloudreve): support thumbnail
* chore: remove unnecessary code
commit b9e192b29c#5367 close #5275 )
* fix(115):limit request rate
* chore(115): fix unit of `limit_rate`
---------
Co-authored-by: Andy Hsu <i@nn.ci>
commit 69a98eaef6#5141 )
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
commit 1ebc96a4e5close #5352 )
commit 66e2324cac7600dc28dfclose #5358 )
commit 8ef89ad0a4#5356 )
* fix(baidu):hash and error:2
* fix:invalid memory address
commit 35d672217d#5346 )
commit 1a283bb272#5334 )
commit a008f54f4d#5329 ) [skip ci]
* fix: adapt update progress type
* Squashed commit of the following:
commit 65c5ec0c34#5457 close #5395 )
commit a6325967d0#5234 )
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
commit 4dff49470a#5366 )
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
commit cc86d6f3d1#5370 )
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
commit c0f9c8ebafclose #5434 )
2023-11-06 16:56:55 +08:00
aaffaee2b5
perf(webdav): support request with cookies ( #5391 )
2023-10-19 19:17:53 +08:00
cdfbe6dcf2
fix: hash gcid empty file ( #5394 )
2023-10-18 16:27:07 +08:00
1a283bb272
feat(google_drive): add `hash_info`, `ctime`, `thumbnail` ( #5334 )
2023-10-06 16:04:39 +08:00
a64dd4885e
fix(139): fixed time zone ( close #5263 )
2023-09-22 16:54:16 +08:00
882112ed1c
feat: add `hash_info` field to `/fs/get` ( close #5259 )
2023-09-22 15:20:04 +08:00
7200344ace
feat: adapt hash feature for some drivers ( #5180 )
...
* feat(pikpak,thunder): adaptation gcid hash
* chore(weiyun): add note
* feat(baidu_netdisk): adaptation rapid
* feat(baidu_photo): adaptation hash
* feat(189pc): adaptation rapid
* feat(mopan):adaptation ctime
* feat(139):adaptation hash and ctime
---------
Co-authored-by: Andy Hsu <i@nn.ci>
2023-09-06 14:46:35 +08:00
b313ac4daa
fix(crypt): fix 139cloud hack ( #5178 )
...
(cherry picked from commit 18bf64af47e58cc69cdd2e598de9c19538a7bf78)
2023-09-06 14:12:01 +08:00
b604e21c69
feat(webdav): support http chunked request ( close #5161 in #5162 )
...
But we do not recommend not adding the content-length header when putting files
2023-09-05 13:03:29 +08:00
37dffd0fce
feat(crypt): customize `filename_encoding` ( #5148 )
...
close #5109
close #5080
2023-09-03 18:06:44 +08:00
6308f1c35d
fix: updateTime, createTime and HashInfo ( #5111 )
2023-08-29 13:31:24 +08:00
a3748af772
feat: misc improvements about upload/copy/hash ( #5045 )
...
general: add createTime/updateTime support in webdav and some drivers
general: add hash support in some drivers
general: cross-storage rapid-upload support
general: enhance upload to avoid local temp file if possible
general: replace readseekcloser with File interface to speed upstream operations
feat(aliyun_open): same as above
feat(crypt): add hack for 139cloud
Close #4934
Close #4819
baidu_netdisk needs to improve the upload code to support rapid-upload
2023-08-27 21:14:23 +08:00
1e3950c847
fix: copy tasks using multi-thread downloader can't be canceled ( #5028 )
...
#4981 related
2023-08-19 14:06:59 +08:00
5606c23768
perf(copy): use multi-thread downloader ( close #5000 )
2023-08-13 15:31:49 +08:00
c1db3a36ad
feat: upload progress recovery ( #4987 )
...
* feat(189pc):upload progress recovery
* fix:some err
* feat(baidu_netdisk,baidu_photo):upload progress recovery
* feat(mopan):upload progress recovery
* feat(baidu_netdisk):custom upload api
2023-08-11 14:23:30 +08:00
df6b306fce
perf(drivers): fs operations and cache ( #4965 )
...
* perf(baidu_photo):multi-thread upload
* perf(baidu_netdisk):multi-thread upload and cache optimization
* fix:LimitWriter
* fix(weiyun):only one login is allowed
* feat(189pc):multi threaded upload
* feat(baidu_netdisk):multi threaded upload
* feat(baidu_photo):multi threaded upload
* feat(weiyun):multi threaded upload
* perf(aliyundriver_open):optimize upload code and optimize cache
* fix(weiyun):invalid directory ID
* fix(baidu_netdisk):modified time
* fix(baidu_netdisk,baidu_photo):upload slice error
* perf(baidu_netdisk):cancel unnecessary retries
* fix(limitWriter):must return a non-nil error if it returns n < len(p)
* fix(aliyundrive_open):Name and Filename only use one
* perf(mopan):multi-thread upload
2023-08-09 16:13:09 +08:00
7877184bee
feat(baidu_netdisk): add retry to most operations ( close #4863 in #4939 )
2023-08-07 13:44:28 +08:00
15b7169df4
perf: multi-thread downloader, `Content-Disposition` ( #4921 )
...
general: enhance multi-thread downloader with cancelable context, immediately stop all stream processes when canceled;
feat(crypt): improve stream closing;
general: fix the bug of downloading files becomes previewing stream on modern browsers;
Co-authored-by: Sean He <866155+seanhe26@users.noreply.github.com>
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-04 15:29:54 +08:00
3c21a9a520
feat: Crypt driver, improve http/webdav handling ( #4884 )
...
this PR has several enhancements, fixes, and features:
- [x] Crypt: a transparent encryption driver. Anyone can easily, and safely store encrypted data on the remote storage provider. Consider your data is safely stored in the safe, and the storage provider can only see the safe, but not your data.
- [x] Optional: compatible with [Rclone Crypt](https://rclone.org/crypt/ ). More ways to manipulate the encrypted data.
- [x] directory and filename encryption
- [x] server-side encryption mode (server encrypts & decrypts all data, all data flows thru the server)
- [x] obfuscate sensitive information internally
- [x] introduced a server memory-cached multi-thread downloader.
- [x] Driver: **Quark** enabled this feature, faster load in any single thread scenario. e.g. media player directly playing from the link, now it's faster.
- [x] general improvement on HTTP/WebDAV stream processing & header handling & response handling
- [x] Driver: **Mega** driver support ranged http header
- [x] Driver: **Quark** fix bug of not closing HTTP request to Quark server while user end has closed connection to alist
## Crypt, a transparent Encrypt/Decrypt Driver. (Rclone Crypt compatible)
e.g.
Crypt mount path -> /vault
Crypt remote path -> /ali/encrypted
Aliyun mount paht -> /ali
when the user uploads a.jpg to /vault, the data will be encrypted and saved to /ali/encrypted/xxxxx. And when the user wants to access a.jpg, it's automatically decrypted, and the user can do anything with it.
Since it's Rclone Crypt compatible, users can download /ali/encrypted/xxxxx and decrypt it with rclone crypt tool. Or the user can mount this folder using rclone, then mount the decrypted folder in Linux...
NB. Some breaking changes is made to make it follow global standard, e.g. processing the HTTP header properly.
close #4679
close #4827
Co-authored-by: Sean He <866155+seanhe26@users.noreply.github.com>
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-02 14:40:36 +08:00
7e9cdd8b07
fix(aliyundrive_open): fail limit on concurrently call ( #4851 )
2023-07-28 15:55:39 +08:00
a4511c1963
refactor: change hash function
2023-07-15 16:29:44 +08:00
89832c296f
fix: judge can proxy with ext ( close #4688 )
2023-07-03 20:41:37 +08:00
cfee536b96
feat: add Dropbox driver ( #4639 close #4590 )
...
Co-authored-by: Andy Hsu <i@nn.ci>
2023-06-23 17:36:40 +08:00
f97f1d532e
fix(webdav): don't retry for put if body isn't seeker ( close #4149 close #4238 )
2023-05-11 18:57:35 +08:00
c77ed5fcb0
feat(aliyundrive_open): limit rate for `List` and `Link` ( close #4290 )
2023-05-02 22:06:03 +08:00
86a773674a
feat(task): print stack trace if panic
2023-04-11 15:16:57 +08:00
4ec274e748
fix(aliyundrive_open): refresh upload url if expired ( #3999 close #3823 )
...
* fix(aliyundrive_open): refresh upload url for large files
* fix(aliyundrive_open): retry upload on url expiry
* fix(aliyundrive_open): ignore 409 error
* feat(aliyundrive): cleanup upload retry logic
* feat(util): add multireadable io utility
* feat(aliyundrive_open): make upload fully stream
* feat(aliyundrive_open): refresh upload url every 20 puts
* fix(aliyundrive_open): part info panic
* chore: change refresh upload url strategy
---------
Co-authored-by: Andy Hsu <i@nn.ci>
2023-04-01 14:54:29 +08:00
3b07c72f88
fix(proxy): ignore `Referer` if got redirect ( close #3996 )
2023-03-31 20:29:55 +08:00
0eab31bdf5
fix(local): filename with whitespace issue ( #3928 )
...
* fix(local): filename whitespace problem
* fix(deps): remove deprecated package io/ioutil
---------
Co-authored-by: XZB <i@1248.ink>
2023-03-23 15:18:37 +08:00
8c0defce09
feat(task): add clear succeeded and retry ( #3856 close #3776 )
2023-03-16 15:56:27 +08:00
0d07d81802
feat(smb): support range header ( close #3192 )
2023-02-20 16:46:38 +08:00
805b1e4fa3
fix: different url encoding ( close #3423 )
2023-02-15 16:20:30 +08:00
46b2ed2507
fix(aliyundriver):x-device-id error code ( #3390 )
...
* fix(aliyundriver):x-drvice-id error code
* fix(aliyunpan):session signature error
* fix typo
---------
Co-authored-by: Andy Hsu <i@nn.ci>
2023-02-14 14:11:07 +08:00
22843ffc70
fix(fs): copy file if symlink failed ( #3368 )
2023-02-13 14:41:35 +08:00
b42ec3e810
fix: relative path judgment ( close #3130 )
2023-01-23 15:52:46 +08:00
6453ae0968
fix(search): empty parent where update ( close #2810 )
2023-01-16 17:33:24 +08:00
2f19d4a834
perf(lanzou): optimize the use of list cache ( #2956 )
...
* fix:local sort not cache
* perf(lanzou): Optimize the use of list cache
2023-01-08 21:31:35 +08:00
a1ded3a339
refactor(baidu_photo): optimize code ( close #2911 pr #2924 )
2023-01-06 15:36:05 +08:00
d756cf3e9f
fix(local): disable copying or moving to subfolders ( close #2760 )
2022-12-20 16:27:04 +08:00
146619134d
feat: customize proxy ignore headers ( close #2763 pr #2766 )
...
* clean referer when use proxy
* feat: customize proxy ignore headers
Co-authored-by: Noah Hsu <i@nn.ci>
2022-12-20 16:08:32 +08:00
62a06fa0f9
feat: optimize file operation interface ( #2757 )
...
* feat: optimize file operation interface
* chore: fix typo
Co-authored-by: Noah Hsu <i@nn.ci>
2022-12-20 15:02:40 +08:00
b50d486a63
fix: sub path check if subPath = `/`
2022-12-18 21:28:38 +08:00
6024e8d832
refactor: split the db package hook and cache to the op package ( #2747 )
...
* refactor:separate the setting method from the db package to the op package and add the cache
* refactor:separate the meta method from the db package to the op package
* fix:setting not load database data
* refactor:separate the user method from the db package to the op package
* refactor:remove user JoinPath error
* fix:op package user cache
* refactor:fs package list method
* fix:tile virtual paths (close #2743 )
* Revert "refactor:remove user JoinPath error"
This reverts commit 4e20daaf9e
2022-12-18 19:51:20 +08:00
Rammiah