For non-Mingw32 build, set FD_CLOEXEC to file descriptors which live
beyond function scope. For Mingw32 build, just pass false to
bInheritHandles parameter of CreateProcessW.
WinTLSSession buffers received decrypted data into its own buffer. If
read is requested, it copies the data from its buffer. But if
requested buffer size is less than decrypted buffer, some of the data
is left in the buffer. Previously, we had no facility to check the
existence of this pending data. If this data is the last requested
data from remote server, we may end up waiting for read event even if
we have already data in our buffer, which may cause hang. This commit
fixes this issue by introducing function to return the buffered length
in TLSSession. SocketCore also provides the same function, which
delegates to TLSSession object.
Set the maximum file size to enable mmap (see --enable-mmap
option). The file size is determined by the sum of all files contained
in one download. For example, if a download contains 5 files, then
file size is the total size of those files. If file size is strictly
greater than the size specified in this option, mmap will be disabled.
We eliminated 1 second delay between completion of name resolution for
tracker request and time when it is actually issued. We also elimited
1 second delay after last torrent download exists and it is told to
DHTInteractionCommand.
system.listMethods just returns the available RPC method names. It
can be executed without secret token, because it just returns the
method names, and does not alter anything.
See GH-437
Previously, --retry-wait was not used when retrying after 404
(--max-file-not-found). The retry was done without delay. Also the
maximum retry count (--max-retries) was ignored when retrying with
404.
This commit makes --retry-wait apply for retrying with 404 as well.
It also combines --max-retries and --max-file-not-found. When
retrying with 404, it is counted toward retry count.
Previously, with --bt-detach-seed-only, when user issued pause and
unpause command to seeding torrent, and it was started again, and it
immediately got seeding status, but aria2 did not start new download
waiting in the queue. This commit fixes this bug.
See GH-490
This option sets flag which allows hook command invocation after hash
check (see -V option) in BitTorrent download. By default, when hash
check succeeds, the command given by --on-bt-download-complete is
executed. To disable this action, give false to this option.
Set the maximum socket receive buffer in bytes. Specifing 0 will
disable this option. This value will be set to socket file descriptor
using SO_RCVBUF socket option with setsockopt() call.
See GH-487 about the usecase of this option
With --deferred-input=true, aria2 only reads input file to fill active
download slots, while keeping input file open. Meanwhile,
--save-session saves all download info inside memory, but this does
not take into account of unread item in input file. This will lead to
lose input data in saved session file. Also current BufferedFile
implementation used to read/write input/output file take a lock on
Windows. This effectively prevents session serializer from writing
session data to the same file which is still kept open because of
--deferred-input. See GH-493
_wstat family does not work with symbolic links. They always returns
size 0. To workaround this, we now use GetFileSize to get file size.
To do that, we need to first obtain file HANDLE, which is a bit
tedious because of the messy interface of CreateFileW, but we have no
way to get around it.
Previously we only scanned pool socket to check they are timed out
when we pooled another socket. This means that pooled socket is not
closed long time (stays in CLOSE-WAIT state) if we don't pool any more
socket. In this commit, we now check pooled socket periodically (30
seconds) to avoid the sockets hanging in CLOSE-WAIT state long time.
See GH-477
This is desirable so that we don't have to wake up disk if control
file is not changed, and it is not have to be written again. We use
the same method (SHA1 hash) to check the content is the same. The
limitation is the hash is stored in memory, so we have to write the
first time in each session.
See GH-382
See http://standards.freedesktop.org/basedir-spec/latest/ for further
details. This implementation decides the default based on whether a
file exists at the legacy location, if it doesn't, it picks the
XDG-conforming location instead.
Set checksum for SSH host public key. Use same syntax with --checksum
option. TYPE is hash type. The supported hash type is sha-1 or
md5. DIGEST is hex digest. For example:
sha-1=b030503d4de4539dc7885e6f0f5e256704edf4c3. This option can be
used to validate server's public key when SFTP is used. If this
option is not set, which is default, no validation takes place.
This commits implements basic downloading feature for sftp URI. This
includes segmented downloads, and connection (sftp session) pooling.
Download via HTTP proxy has not been implemented yet.
We use libssh2 to implement sftp feature. SSHSession is closely
designed to TLSSession, but it differs in several aspects. In this
code, we only implements read part of sftp, since aria2 won't offer
uploading feature. Adding SSH/sftp to SocketCore is a bit bloat. But
by doing this we can reuse DownloadCommand, without mostly no
modification. We can just create SSHDownloadCommand by inheriting it,
just like existing ftp.
We have to use dedicated DiskWriter instead of
(*entryItr_)->getDiskWriter(). This is because
SingleFileAllocationIterator cannot reopen file if file is closed by
OpenedFileCounter.
Use one of the following to provide random bytes:
- Windows CryptGenRandom
- Linux getrandom (syscall interface to urandom, without nasty corner
cases such as file descriptor exhaustion or re-linked /dev/urandom)
- std::device_random (C++ random device, which usually will be urandom)
This also equalizes util::getRandom and SimpleRandomizer (the former
will now use the latter) instead of having essentially two different
PRNG interfaces with potentially different quality.
Closes GH-320
This option requires BitTorrent message payload encryption with
arc4. This is a shorthand of --bt-requre-crypto
--bt-min-crypto-level=arc4. If true is given, deny legacy BitTorrent
handshake and only use Obfuscation handshake and always encrypt
message payload. This option defaults to false.
This option excludes seed only downloads when counting concurrent
active downloads (-j option). This means that if -j3 is given and
this option is turned on and 3 downloads are active and one of those
enters seed mode, then it is excluded from active download count (thus
it becomes 2), and the next download waiting in queue gets started.
But be aware that seeding item is still recognized as active download
in RPC method.
Generally, bitfield is not available for download whose total length
is unknown. We create bitfield when download is completed (usually
connection EOF) so that we can use it to show additional info in RPC
aria2.tellStatus response. Specifically, bitfield is now shown. And
completedLength under files key (or completedLength in
aria2.getFiles() response) is correctly shown.
This commit fixes the bug that aria2.tellStopped RPC method returns
total length and completedLength as 0 when file size is unknown in
advance and turns out > 2GB in the end. This commit and addresses the
performance degradation in this case.
This commit fixes following compiler warning with mingw compiler:
In file included from common.h:63:0,
from util_security.h:42,
from util_security.cc:36:
/usr/share/mingw-w64/include/winsock2.h:15:2: warning: #warning
Please include winsock2.h before windows.h [-Wcpp]
#warning Please include winsock2.h before windows.h
The algorithms the `CryptProv` on Windows supports does not currently
include SHA-224, so there is a "dark spot" in this implementation. Also
on Win XP < SP3, most of the SHA-2 family is not actually supported.
All other implementation provide support for MD5, SHA-1 and all of the
SHA-2 family, hence drop the incomplete WinMessageDigest implementation
in favor of any other supported implementation (at least the internal
implementation is always available at compile-time).
This option pauses downloads created as a result of metadata
download. There are 3 types of metadata downloads in aria2: (1)
downloading .torrent file. (2) downloading torrent metadata using
magnet link. (3) downloading metalink file. These metadata downloads
will generate downloads using their metadata. This option pauses these
subsequent downloads.
Add and use usedCompilerAndPlatform(). This adds compiler information to
INFO logs and the --version output, and may be helpful when trying to
diagnose/reproduce user-reported problems.
Also make INFO logs include usedLibs() output.
Closes#235
DefaultPieceStorage may be referenced by one of DHT task (e.g.,
DHTPeerLookupTask), after RequestGroup was deleted, and even after
RequestGroupMan was deleted. DefaultPieceStorage has a reference to
MultiDiskAdaptor which calls RequestGroupMan object on destruction.
So when DHT task is destroyed, DefaultPieceStorage is destroyed, which
in turn destroys MultiDiskAdaptor. DHT task is destroyed after
RequestGroupMan was destroyed, MultiDiskAdaptor will use now freed
RequestGroupMan object, this is use-after-free.
When the directory is changed via aria2.changeOption RPC method, we
directly change first FileEntry's path using FileEntry::setPath(). If
there is no PREF_OUT option is given, basically file name is unknown,
so we just set empty string and let the next run determine the correct
file name and new directory is applied there. But previous code does
not reset length property of FileEntry, so the unexpected code path is
taken when unpaused and its path expects path is not empty string.
This commit fixes this issue by setting length to 0 using
FileEntry::setLength().
This is a slight optimization not to cause useless disk access. This
only applies to saving session automatically (see
--save-session-interval). aria2.saveSession and serialization at the
end of the session are always performed as before.
When serialization, we first check that whether there is any change
since the last serialization. To do this, we first calculate hash
value of serialized content without writing into file. Then compare
this value to the value of last serialization. If they do not match,
perform serialization.
This is a slight optimization not to cause useless disk access. This
only applies to saving session automatically (see
--save-session-interval). aria2.saveSession and serialization at the
end of the session are always performed as before.
In my quest to explore the code and understand it, why not clean it up
in the progress. Most formatting provided by clang-format. ;)
Aside from formatting, also extracted some method and unnested some
control structs.