Historically, aria2 did not send HAVE message to the peer which the
piece is coming from, thinking it is obvious that the peer knows we
have the piece. But it is not obvious if one piece is download from
more than 1 peers (e.g., end game mode). So it is better to send HAVE
to all peers connected.
On HAVE message reception and PIECE message transmission, update peer
bitfield and piece stat only if current peer bitfield indicates it
does not have the piece.
Before, only spent uris where sanitized not to be contained within
remaining uris. Change this so that each uri in the
union(remaining,spent) get saved once at most.
The order of the uris will won't be changed, with remaining uris going
first followed by spent uris.
Also avoid copying the uri std::strings around during dupe checking,
usually resulting in better performance regarding CPU and space.
IOFile provides the same interface of old BufferedFile. It provides
extension points as pure virtual functions. Those functions are
implemented by 2 concrete subclasses: BufferedFile and GZipFile.
WebSocketSessionMan now implements DownloadEventListener and is added
to Notifier. It becomes member variable of DownloadEngine. The event
constant for download event is defined in aria2.h so that we can add
event callback API later.
Android NDK R8e does not provide ftruncate64, but bionic has the
assembler code to access kernel function. We borrowed those
ftruncate64.S files from android source code repository. It turns out
that x86 asm.h in NDK R8e is also broken, so latest asm.h was also
borrowed.
By default, libaria2 feature is disabled. Use --enable-libaria2
to enable it. libaria2 static build is also disabled by default.
Use --enable-static to enable it.
sessionConfigSetKeepRunning() was removed and SessionConfig has
keepRunning member instead. Signal handlers are now all prepared in
MultiUrlRequestGroup object. Setting SessionConfig.useSignalHandler to
false will avoid signal handler setup.
If we got IPv4 lookup response, we don't wait for IPv6 lookup
response. This is because DNS server may drop AAAA query and we have
to wait for the long time before timeout. We don't do the inverse,
because, based on todays deployment of DNS server, almost all of them
can respond A query just fine.
Currently, aria2 checks configured addresses at the startup. But there
are chances that interfaces are not setup at that moment. For example,
if aria2 is used as daemon, it may start before network interfaces
up. To workaround this, we check addresses again if both addresses are
not configured at the startup.
This change makes --save-session save only options specified for
download, more specifically, options in command-line, -i file and via
RPC. The other options from conf file and default values are not
saved. This will drastically decrease the size of session file.
The currently used URIs are inserted back into remaining URI list in
FileEntry::putBackRequest(), which overlaps to some of the URIs in
spentUris_. If we save spent URIs, each time save is performed, the
number of URIs are increased due to this overlap. This change fixes
this bug.
LibUV event will use the best available polling method on a system, kind
of like aria2 does already with the different *EventPoll
implementations.
However, libuv may support different/newer polling mechanisms; for
example on Windows it will use IO Completion Ports which are superior to
select() ;)
Make it clear that TLS_WANT_READ must be returned if underlying
SSL/TLS lib does not indicate I/O direction. This is because
SocketCore sets wantRead_ = true when TLS_WANT_READ is returned and
otherwise sets wantWrite_ = true. We only want to set wantWrite_ =
true iff we have some pending data to send.
This is because on some platforms (gnutls on cygwin for example),
library always fails for this function and getting ERROR every time
aria2c invoked is too hard.
The IPv6 asynchronous name resolver is enabled if the host has at
least one interface with IPv6 address configured (the loopback address
will not be counted), which is roughly the same behaviour of the
standard getaddrinfo(3). To disable IPv6 asynchronous name resolver,
use --disable-ipv6.
Now dedicated ConnectCommand handles connection establishment. It
checks whether connection is established or not. It also handles
backup connection. The next Command creation is abstracted using
ControlChain struct template.
Issue A record query only when non-loopback IPv4 address is
configured. Likewise, issue AAA record query only when non-loopback
and non-linklocak IPv6 address is configured.
Since the change b782a56b, we use endOffsetOverride_ as the return
value of getEndByte(). But aria2 does not send Range header field when
range starts 0 (this is because some server returns error if it
received Range: 0-), and the HttpRequest::isRangeSatisfied() checks
the equality of getEndByte() and the end byte in response header and
fails. The fix is send Range header if getEndByte() is set.
MINGW does not have vasprintf(), so we use _vscprintf() and
vsnprintf(). We want to reuse buffer, so for non-MINGW we use
vsnprintf() with retrying doubling buffer size if output is truncated.
With -c option, aria2 can continue download after the existing file
position. If it is not completed, then after completion aria2 runs
checksum checking if available. But if existing file has already been
completed, then CreateRequestCommand exits without issuing checksum
checking. And aria2 treats it download error because it needs checksum
verification but it has not been done. This change fixes this by
properly checking download state and issue checksum checking before
CreateRequestCommand.
Saved sessions may very large, as in hundreds and even thousands of
megabyte when dealing with large queues.
Add support to save and reload sessions to gzipped files, when libz is
available.
The session serializer will output gzipped contents when the file ends
with .gz, while the input file reader (UriListParser) will accept
whatever is thrown at it.
mingw-w64 does not actually have sys/signal.h, while OSX currently has a
broken signal.h
Better check the presence of both and use sys/signal.h if present, else
signal.h
There is an initial vector of SharedHandle<RequestGroup> to seed the
DownloadEngine. This vector was however kept alive via main.cc ->
MultiUrlRequestInfo up until the program exits, hence effetively leaking
all initial RequestGroups and associated object instances.
Hence, as a matter of dirty-workaround, drop the contents of that initial
vector as soon as it is not required any longer.
Previously, unless HTTP pipelining is enabled, end byte in that
message is always 0. With this change, it shows correct end byte sent
to the HTTP server.
Peer list is now divided into 2: unusedPeers_ and usedPeers_.
Duplicate check is done using std::set by comparing pair of IP address
and port. For this, only IP address and port given to the Peer
constructor are used. In other words, TCP port received from extended
message is not used for this purpose.
In InitiatorMSEHandshakeCommand, when aborting connection, we must
return peer to the PeerStorage. But it is not done if sequence_ is
INITIATOR_SEND_KEY. This causes stale Peer objects whose usedBy()
returns true eventually occupies peer list and aria2 cannot make any
connections.
--save-session-interval option saves error/unfinished downloads to a
file specified by --save-session option every SEC seconds. If 0 is
given, file will be saved only when aria2 exits.
On Mingw32 build, if aria2 opens file with GENERIC_WRITE access right,
some programs cannot open the file aria2 is seeding. To avoid this
situation, re-open files with read-only enabled when seeding is about
to begin.
This option sets GID manually. aria2 identifies each download by the
ID called GID. The GID must be hex string of 16 characters, thus
[0-9a-zA-Z] are allowed and leading zeros must not be stripped. The
GID all 0 is reserved and must not be used. The GID must be unique,
otherwise error is reported and the download is not added. This
option is useful when restoring the sessions saved using
--save-session option. If this option is not used, new GID is
generated by aria2.
This change replaces the current 64 bit sequential GID with 64 bits
random bytes GID in an attempt to support persistent GID. Internally,
the GID is stored as uint64_t. For human representation and RPC
interface, GID is represented as 16 bytes hex string. For console
readout, 16 bytes are too long, so it is abbreviated to first 6 bytes.
When querying GID in RPC calls, user can speicfy the prefix of GID as
long as the prefix is shared by more than 1 GID entries.
If more than 1 simultaneous downloads are going on, use more compact
format in readout. Currently, at most 5 download stats are displayed.
util::abbrevSize() is rewritten to support "Gi" unit and provides more
compact abbreviation.
Not realeasing these resources does not hurt the correctness of the
program, but memory leak detection tool, such as valgrind, reports it
as potential memory leak. It is better to distinguish it from the real
ones.
The old implementation starts to find faster host when the number of
missing segment becomes 1. Because of --min-split-size option, before
the number of missing segment becomes 1, the number of connection
becomes 1 and it can be slow. In this case, we have to wait until the
last segment is reached. The new implementation starts to find faster
host when the remaining length is less than --min-split-size * 2, to
mitigate the problem stated above.
If we receive small data (e.g., 1 or 2 bytes), cache entry becomes a
headache. To mitigate this problem, we allocate cache buffer at least
4KiB and append the data to the contagious cache data.
This option enables disk cache. If SIZE is 0, the disk cache is
disabled. This feature caches the downloaded data in memory, which
grows to at most SIZE bytes. The cache storage is created for aria2
instance and shared by all downloads. The one advantage of the disk
cache is reduce the disk seek time because the data is written in
larger unit and it is reordered by the offset of the file. If the
underlying file is heavily fragmented it is not the case.
I tried CreateFile but the subsequent ReadFile fails with Access
Denied if sparse file is read on NTFS. I mostly reverted previous
changes and use _wsopen with read/write share enabled instead of
CreateFile.
This change also includes --enable-mmap support for MinGW32
build. Memory mapped file may be useful for 64-bits OS and lots of
RAM. Currently, FlushViewOfFile is not called during the download, so
it is slightly vulnerable against sudden power loss. I found lots of
read when resuming download due to page fault. So for now it is useful
for the initial download. I recommend not to use
--file-allocation=prealloc with --enable-mmap for MinGW32, because it
triggers page faults even in the initial download. Anyway, the option
is experimental.
--force-save option saves download with --save-session option even if
the download is completed or removed. This may be useful to save
BitTorrent seeding which is recognized as completed state. The
default value is false.
Subtract msgHdrLen_ from writtenLength to get the uploaded data size.
Without this correction, the seeder assumes it has uploaded more data
than it actually has.
The old implementation calculates download/upload statistics for a
RequestGroup by summing up all PeerStat objects. For global
statistics, those are summed together. This clearly incurs runtime
penalty and we introduced some kind of caching which updates
statistics every 250ms but it did not work right.
This change removes all these aggregation code, and instead makes
RequestGroup and RequestGroupMan objects hold NetStat object and
download/upload bytes are directly calculated by thier own NetStat.
This is far more simplar than the old way and less runtime penalty and
brings more accuracy.
If true is given, which is default, save the uploaded torrent or
metalink metadata in the directory specified by --dir option. The
filename consists of SHA1-hash hex string of metadata plus
extension. For torrent, the extension is '.torrent'. For metalink, it
is '.meta4'. If false is given to this option, the downloads added by
aria2.addTorrent or aria2.addMetalink will not be saved by
--save-session option.
To enable RPC over SSL/TLS, specify server certificate and private key
using --rpc-certificate and --rpc-private-key options and enable
--rpc-secure option. After the encryption is enabled, use https and
wss scheme to access RPC server.
In this change, we defined HTTP header fields we are interested in.
We only store those headers in HttpHeader object. Accessing HTTP
headers in HttpHeader object is now done through enum values.
DownloadHandlerConstants was simplified. MIME type handling in Accept
header was also reworked. DownloadContext's metalinkServerContacted_
is replaced with acceptMetalink_ and its boolean value is reverted.
RequestGroup and HttpRequest now do not hold vector of accepting
types. HttpRequest has the flag acceptMetalink_ which will be set by
the same value of DownloadContext::accpetMetalink_ and if it is true,
Metalink MIME types are added to Accept header field.
Socket::getPeerInfo() may fail if its TCP connection has already
disconnected. In this case, we log this error. The success or failure
of pooling connection should not affect the later execution of the
program.