Commit Graph

693 Commits (6fc6751463f907b29c2ff5e7ed5adae343616493)

Author SHA1 Message Date
foxxorcat c1db3a36ad
feat: upload progress recovery (#4987)
* feat(189pc):upload progress recovery

* fix:some err

* feat(baidu_netdisk,baidu_photo):upload progress recovery

* feat(mopan):upload progress recovery

* feat(baidu_netdisk):custom upload api
2023-08-11 14:23:30 +08:00
Andy Hsu c59dbb4f9e fix(local): files get deleted when copied to other storage (close #4983) 2023-08-10 16:42:09 +08:00
foxxorcat df6b306fce
perf(drivers): fs operations and cache (#4965)
* perf(baidu_photo):multi-thread upload

* perf(baidu_netdisk):multi-thread upload and cache optimization

* fix:LimitWriter

* fix(weiyun):only one login is allowed

* feat(189pc):multi threaded upload

* feat(baidu_netdisk):multi threaded upload

* feat(baidu_photo):multi threaded upload

* feat(weiyun):multi threaded upload

* perf(aliyundriver_open):optimize upload code and optimize cache

* fix(weiyun):invalid directory ID

* fix(baidu_netdisk):modified time

* fix(baidu_netdisk,baidu_photo):upload slice error

* perf(baidu_netdisk):cancel unnecessary retries

* fix(limitWriter):must return a non-nil error if it returns n < len(p)

* fix(aliyundrive_open):Name and Filename only use one

* perf(mopan):multi-thread upload
2023-08-09 16:13:09 +08:00
Andy Hsu b91ed7a78a fix(aliyundrive_open): retry refresh token if sub not match 2023-08-08 22:08:05 +08:00
Andy Hsu 95386d777b feat(aliyundrive_open): record token exchange 2023-08-08 20:38:13 +08:00
Sean 7877184bee
feat(baidu_netdisk): add retry to most operations (close #4863 in #4939) 2023-08-07 13:44:28 +08:00
Andy Hsu 6c1c7e5cc0 fix(wopan): missing `familyID` on mkdir (close #4927) 2023-08-04 22:26:56 +08:00
Sean 15b7169df4
perf: multi-thread downloader, `Content-Disposition` (#4921)
general: enhance multi-thread downloader with cancelable context, immediately stop all stream processes when canceled;
feat(crypt): improve stream closing;
general: fix the bug of downloading files becomes previewing stream on modern browsers;

Co-authored-by: Sean He <866155+seanhe26@users.noreply.github.com>
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-04 15:29:54 +08:00
Bnq Dzj e5ffd39cf2
feat: add 123Pan Share driver (close #4853 in #4898)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-03 15:01:43 +08:00
foxxorcat 49bde82426 perf(189pc): empty file upload and cache optimization (#4913)
- login captcha error
- cache optimization
- upload empty file
2023-08-03 14:08:40 +08:00
foxxorcat 3e285aaec4
feat: add `weiyun` support (close #4802 in #4883)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-02 21:39:59 +08:00
Andy Hsu a69d72aa20 feat(aliyundrive_open): support resource drive (close #4889) 2023-08-02 15:50:01 +08:00
Sean 3c21a9a520
feat: Crypt driver, improve http/webdav handling (#4884)
this PR has several enhancements, fixes, and features:
- [x] Crypt: a transparent encryption driver. Anyone can easily, and safely store encrypted data on the remote storage provider.  Consider your data is safely stored in the safe, and the storage provider can only see the safe, but not your data.
  - [x] Optional: compatible with [Rclone Crypt](https://rclone.org/crypt/). More ways to manipulate the encrypted data.
  - [x] directory and filename encryption
  - [x] server-side encryption mode (server encrypts & decrypts all data, all data flows thru the server)
- [x] obfuscate sensitive information internally
- [x] introduced a server memory-cached multi-thread downloader.
  - [x] Driver: **Quark** enabled this feature, faster load in any single thread scenario. e.g. media player directly playing from the link, now it's faster.
- [x] general improvement on HTTP/WebDAV stream processing & header handling & response handling
  - [x] Driver: **Mega** driver support ranged http header
  - [x] Driver: **Quark** fix bug of not closing HTTP request to Quark server while user end has closed connection to alist

## Crypt, a transparent Encrypt/Decrypt Driver. (Rclone Crypt compatible)

e.g.  
Crypt mount path ->  /vault 
Crypt remote path -> /ali/encrypted
Aliyun mount paht -> /ali

when the user uploads a.jpg to /vault, the data will be encrypted and saved to /ali/encrypted/xxxxx. And when the user wants to access a.jpg,  it's automatically decrypted, and the user can do anything with it.
Since it's Rclone Crypt compatible, users can download /ali/encrypted/xxxxx  and decrypt it with rclone crypt tool. Or the user can mount this folder using rclone, then mount the decrypted folder in Linux...

NB.  Some breaking changes is made to make it follow global standard, e.g. processing the HTTP header properly.

close #4679 
close #4827 

Co-authored-by: Sean He <866155+seanhe26@users.noreply.github.com>
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-02 14:40:36 +08:00
Andy Hsu 1dc1dd1f07 feat(aliyundrive_open): support `livp` format file download (close #4890) 2023-08-01 21:50:25 +08:00
foxxorcat c9ea9bce81
feat(lanzou): support login with account (close #4880 in #4885) 2023-08-01 19:44:57 +08:00
foxxorcat 9f08353d31
feat(baidu_photo): optional delete album origin file (close #4872 in #4875) 2023-07-31 18:29:45 +08:00
foxxorcat 06f46206db
fix(baidu_photo): album download (close #4603 in #4871)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-07-31 16:27:16 +08:00
Andy Hsu b12d92acc9 perf(baidu_netdisk): optimize memory allocate 2023-07-29 17:12:43 +08:00
Andy Hsu 7e9cdd8b07 fix(aliyundrive_open): fail limit on concurrently call (#4851) 2023-07-28 15:55:39 +08:00
meozky cee6bc6b5d
fix(terabox): slice out of range (close #4858 in #4860) 2023-07-28 15:52:20 +08:00
Andy Hsu cfd23c05b4 fix(139): upload empty file (close #4711) 2023-07-27 19:26:22 +08:00
Andy Hsu 2e23ea68d4 fix(aliyundrive_open): increase limit interval (close #4851) 2023-07-27 18:26:11 +08:00
Bnq Dzj 4afa822bec
fix(123): Use APP-side API (close #4834 in #4856) 2023-07-27 15:51:59 +08:00
Andy Hsu 4c2535cb22 fix(115): `user-agent` lost on upload (close #4831) 2023-07-23 15:18:33 +08:00
Andy Hsu d4ea8787c9 fix(123): upload file size that less than 16 MB (close #4816) 2023-07-21 14:35:18 +08:00
Rui Tang a4de04528a
fix(123): `auth-key` verification (close #4811 in #4814)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-07-21 14:33:45 +08:00
Andy Hsu 6ec9a8d4c7 fix(aliyundrive_open): the temp file is not delete (close #4777) 2023-07-16 15:01:22 +08:00
Andy Hsu 3f7882b467 feat(aliyundrive_open): rapid upload (close #4766) 2023-07-15 19:33:46 +08:00
Andy Hsu a4511c1963 refactor: change hash function 2023-07-15 16:29:44 +08:00
Andy Hsu 9d1f122717 fix(local): thumbnail rotated if exist orientation tag (close #4749) 2023-07-15 14:31:03 +08:00
Andy Hsu 5dd73d80d8 fix(123): remove stream upload method (close #4772) 2023-07-14 19:12:18 +08:00
Andy Hsu fce872bc1b feat(123): thumbnail support (#3953) 2023-07-14 14:43:40 +08:00
Andy Hsu df6c4c80c2 fix(123): update app-version (close #4758) 2023-07-14 14:17:29 +08:00
Andy Hsu d2ff040cf8 feat(s3): add `SessionToken` field (close #4761) 2023-07-13 15:58:19 +08:00
foxxorcat a31af209cc
fix(pikpak): hash calculation and fast upload judgment (#4745 fix #1081) 2023-07-11 22:19:21 +08:00
Andy Hsu 6887f14ec6 feat(pikpak): allow disable media link (close #4735) 2023-07-11 13:40:58 +08:00
Andy Hsu 61101a60f4 fix(s3): unable to copy empty folder (close #4620) 2023-07-10 14:55:19 +08:00
foxxorcat 3529023bf9
fix(mopan): size field type(close #4734 in #4736) 2023-07-10 14:25:27 +08:00
Andy Hsu fa66358b1e fix(sftp): read target obj of symlink file (close #4713) 2023-07-09 14:42:57 +08:00
Andy Hsu d17f7f7cad fix(123): judge status on get redirect_url (close #4718) 2023-07-07 19:55:37 +08:00
Andy Hsu 4fabc27366 fix(aliyundrive_open): panic if driver not init 2023-07-05 13:51:46 +08:00
Andy Hsu 5e8bfb017e fix(123): add `Referer` to request (close #4631) 2023-07-04 18:36:46 +08:00
foxxorcat f09bb88846
fix(thunder): upload issues (close #4663 in #4667) 2023-06-29 13:21:30 +08:00
foxxorcat c518f59528
feat: add `MoPan` driver (close #4325 in #4659)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-06-28 14:53:43 +08:00
wenmig 21b8e7f6e5
fix(aliyundrive_share): add limit rate and lift rate limit restrictions (#4587) 2023-06-26 14:49:21 +08:00
Andy Hsu 2ae9cd8634 fix(dropbox): failed get link in #4639
close cfee536b96 (commitcomment-119404554)
2023-06-25 17:07:31 +08:00
wenmig cfee536b96
feat: add Dropbox driver (#4639 close #4590)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-06-23 17:36:40 +08:00
william 1c8fe3b24c
fix(aliyundrive_open): adaptive part size adjustment (#4609)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-06-23 14:25:30 +08:00
Andy Hsu 84e23c397d fix(baidu_netdisk): rollback #3652 (close #4628) 2023-06-21 18:37:25 +08:00
Andy Hsu f7baec2e65 feat: add `WoPan` driver (close #4541) 2023-06-17 20:20:00 +08:00
wenmig 378bab32f1
chore(aliyundrive_share): increase the `limit` of the list api (#4588) 2023-06-17 20:10:34 +08:00
Andy Hsu 6cd8151cad fix(aliyundrive_open): change default `oauth_token_url` 2023-06-16 15:03:27 +08:00
Andy Hsu ca5a53fc24 fix(aliyundrive_open): `openFile/list` rate limit 2023-06-11 18:18:09 +08:00
Andy Hsu e23f00f349 fix(139): avoid panic due to `Authorization` for emptiness 2023-06-10 00:12:04 +08:00
Andy Hsu a66b0e0151 feat(139): auto extract account from `Authorization` 2023-06-09 23:41:41 +08:00
Andy Hsu 3bfa00d5d2 fix(189pc): add `REQID` header 2023-06-09 23:33:12 +08:00
Andy Hsu 6cbd2532cc fix(139): modify the authentication mode 2023-06-09 23:02:02 +08:00
hunshcn 47976af0d3
feat: set `ProxyFromEnvironment` for default http client (#4546) 2023-06-09 22:08:54 +08:00
Andy Hsu 4dca52be85 fix(s3): optional add filename to disposition (close #4538) 2023-06-06 22:47:27 +08:00
Andy Hsu 1e62666406 feat(baidu_netdisk): allow custom crack ua 2023-06-04 15:57:41 +08:00
Andy Hsu b124fdc092 perf(baidu): avoid refreshing the token on every startup 2023-06-02 18:31:42 +08:00
KAAAsS 8421c72c5c
fix(seafile): driver panic while downloading or uploading file (#4491)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-05-28 16:45:46 +08:00
KAAAsS a80e21997c
feat(cloudreve): auto remove trailing slash in address (#4492)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-05-28 16:18:09 +08:00
Andy Hsu 4369cbbac3 fix(alist_v3): missed `Content-Length` on upload (close #4457) 2023-05-27 20:23:36 +08:00
zqxiaojin 89f76d7899
feat: add `UC` driver (close #1127 in #4459)
Co-authored-by: lj98568 <lj98568@alibaba-inc.com>
Co-authored-by: Andy Hsu <i@nn.ci>
2023-05-27 19:36:14 +08:00
foxxorcat ef68f84787
fix(baidu_photo): legal album title check (close #4479 in #4487) 2023-05-27 17:07:57 +08:00
foxxorcat 2c1f70fbe9
fix(189pc): large file upload error (close #4417 in #4438) 2023-05-27 14:28:58 +08:00
Andy Hsu b2f5757f8d fix(copy): copy from driver that return `writer` (close #4291) 2023-05-26 21:57:43 +08:00
plzzzzg 6b97b4eb20
feat(s3): set content type from stream when uploading (#4460)
Co-authored-by: guopeilun <guopl@flatincbr.com>
2023-05-24 18:02:49 +08:00
Noah Hsu 571bcf07b0 fix(alias): add api prefix for proxy url (close #4392) 2023-05-19 00:12:57 +08:00
Noah Hsu 63de65be45 fix: increase timeout for http_client (close #4409) 2023-05-18 23:32:05 +08:00
XYUU a3446720a2
fix: make `TlsInsecureSkipVerify` enable for all request (#4386) 2023-05-14 17:05:47 +08:00
Andy Hsu 3c4c2ad4e0 feat(teambition): support s3 upload method (close #4365) 2023-05-13 23:06:25 +08:00
Andy Hsu 077a525961 fix(189): adapt new login method (close #4378) 2023-05-13 17:28:40 +08:00
william c19166be1c
feat(google_drive): support sa (close #3132 in #4360)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-05-12 14:47:50 +08:00
Andy Hsu daad61443c feat(local): support thumbnail cache (close #4216) 2023-05-11 19:57:24 +08:00
Andy Hsu 4b0c01158d fix: panic on nil pointer 2023-05-11 19:44:44 +08:00
Andy Hsu e15755fef0 fix(189): enable `TlsInsecureSkipVerify` (close #4355) 2023-05-11 18:48:31 +08:00
安稳 d41d868a8d
fix(baidu_photo): change folder name length limit (close #4351 in #4353)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-05-09 20:44:57 +08:00
Xi Wuuuuuuuuuuuu~~~~~~~~~~~~~~~ 9502f5acd7
fix(cloudreve): skip init login when using cookie (#4341) 2023-05-08 19:25:36 +08:00
Xi Wuuuuuuuuuuuu~~~~~~~~~~~~~~~ b03879403f
feat(cloudreve): support use cookie to login (close #4324 in #4339)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-05-08 15:19:51 +08:00
Andy Hsu ee4ac81677 fix(webdav): can't rename on infini-cloud (close #4333) 2023-05-08 14:21:12 +08:00
陈佳 ee6c31332d
feat(drivers): ipfs api (#4265)
Co-authored-by: Andy Hsu <i@nn.ci>
2023-05-05 17:42:22 +08:00
Andy Hsu c77ed5fcb0 feat(aliyundrive_open): limit rate for `List` and `Link` (close #4290) 2023-05-02 22:06:03 +08:00
Andy Hsu 822be17fb9 feat(aliyundrive_open): add expiration for link (close #4061) 2023-05-02 16:12:40 +08:00
Noah Hsu 4bf46268da feat(alias): support thumbnail (close #4256) 2023-04-28 00:17:15 +08:00
Noah Hsu b7ea73b3c2 fix(aliyundrive_open): can't refresh token if access_token is empty (#4255) 2023-04-28 00:01:47 +08:00
Andy Hsu 9fbc54314d chore(aliyundrive_open): change base url 2023-04-27 16:38:40 +08:00
Andy Hsu cf8ab29a17 feat: optional allow be mounted (close #4218) 2023-04-27 16:33:01 +08:00
Andy Hsu 9d55ad3af6 fix(123): get download url (close #4244) 2023-04-26 15:06:24 +08:00
Andy Hsu 36cd504783 fix(alist_v3): missed meta_password update
fix: adb0739dfe (commitcomment-110328033)
2023-04-24 20:56:46 +08:00
foxxorcat 49f13b9b90
fix(baidu_photo): upload file has web prefix (close #4233 in #4235) 2023-04-24 19:13:33 +08:00
Andy Hsu adb0739dfe feat!(alist_v3): support username & password login (close #4226)
Breaking changes:
- rename access_token to token
- rename old password to meta_password
2023-04-23 17:48:26 +08:00
Andy Hsu 8711f2a1c5 feat(quark): shard request file (close #4175) 2023-04-17 15:33:38 +08:00
Andy Hsu 7f35aab071 revert(quark): remove preset range header 2023-04-17 14:39:21 +08:00
Andy Hsu ecd167d2f9 feat(quark): add preset range header (close #4166) 2023-04-16 19:26:03 +08:00
Andy Hsu 5cba10446e fix(123): adapt new upload method (close #4141) 2023-04-14 15:48:39 +08:00
Andy Hsu c5f6a90f54 fix(quark): download file size limit (close #4140) 2023-04-14 13:47:05 +08:00
Andy Hsu fdcad9c154 fix(123): incorrect endpoint (close #4046) 2023-04-12 23:04:12 +08:00
Andy Hsu 570b8be022 fix(onedrive): error check in `upBig` 2023-04-11 22:52:42 +08:00