The old implementation calculates download/upload statistics for a
RequestGroup by summing up all PeerStat objects. For global
statistics, those are summed together. This clearly incurs runtime
penalty and we introduced some kind of caching which updates
statistics every 250ms but it did not work right.
This change removes all these aggregation code, and instead makes
RequestGroup and RequestGroupMan objects hold NetStat object and
download/upload bytes are directly calculated by thier own NetStat.
This is far more simplar than the old way and less runtime penalty and
brings more accuracy.
DownloadHandlerConstants was simplified. MIME type handling in Accept
header was also reworked. DownloadContext's metalinkServerContacted_
is replaced with acceptMetalink_ and its boolean value is reverted.
RequestGroup and HttpRequest now do not hold vector of accepting
types. HttpRequest has the flag acceptMetalink_ which will be set by
the same value of DownloadContext::accpetMetalink_ and if it is true,
Metalink MIME types are added to Accept header field.
The only exception is zero-length file. If server tells file is
zero-length and --checksum option is given, aria2 now correctly checks
its checksum. There is one known issue: If downloaded file is
zero-length file and .aria2 file exists, it will not be deleted on
successful verification, because .aria2 file is not loaded.
The delay is caused because some Commands are only called in certain
interval(called refreshInterval, default, 1000ms). In aria2 download
stops when all Commands associated to it are stopped. Since some
Commands are called in each 1000ms by default, as mentioned before, we
have to wait for them. To fix this issue, we call
DownloadEngine::setRefreshInterval(0) when pausing/stopping downloads.
DownloadEngine::setRefreshInterval(0) makes refreshInterval 0 in one
shot.
When all segments are ignored, now DownloadFailureException is thrown.
And stop the download immediately. As described earlier, we call
DownloadEngine::setRefreshInterval(0) in catch block of
DownloadFailureException to eliminate delay.
Removed SharedHandle::isNull(). Instead we added operator* and
operator unspecified_bool_type. Removed use of WeakHandle and
replaced with raw pointer.
Fixed the bug that a file gets overwritten if -V is given and no
hash is provided. Fixed the bug that --dry-run leads download
error. Added RequestGroup::createCheckIntegrityEntry() which
correctly creates CheckIntegrityEntry objects and open files based
on -V option and the existence of control file.
* src/AbstractCommand.cc
* src/AbstractCommand.h
* src/ChecksumCheckIntegrityEntry.cc
* src/DownloadContext.cc
* src/DownloadContext.h
* src/FtpNegotiationCommand.cc
* src/HttpResponseCommand.cc
* src/PieceHashCheckIntegrityEntry.cc
* src/RequestGroup.cc
* src/RequestGroup.h
* src/RequestGroupEntry.cc
* src/RequestGroupEntry.h
Count the number of command used in HTTP(S)/FTP and the number of
connections in HTTP(S)/FTP independently. The former is used to
determin whether additional command is needed. The latter is used
to report user to how many connections are used in a download.
* src/AbstractCommand.cc
* src/AbstractCommand.h
* src/CreateRequestCommand.cc
* src/RequestGroup.cc
* src/RequestGroup.h
Hold reference to PieceStorage when paused in order to provide
progress information via XML-RPC.
* src/RequestGroup.cc
* src/RequestGroup.h
* src/RequestGroupMan.cc
Added aria2.pause and aria2.unpause XML-RPC method. aria2.pause
pauses the download denoted by gid. gid is of type string. The
status of paused download becomes "paused" and the download is
placed on the first position of waiting queue. As long as the
status is "paused", the download is not started. To change status
to "waiting", use aria2.unpause method. This method returns GID of
paused download. aria2.unpause changes the status of the download
denoted by gid from "paused" to "waiting". This makes the download
eligible to restart. gid is of type string. This method returns
GID of unpaused download.
* doc/aria2c.1.txt
* src/DownloadContext.cc
* src/FileEntry.cc
* src/OptionHandlerFactory.cc
* src/RequestGroup.cc
* src/RequestGroup.h
* src/RequestGroupMan.cc
* src/XmlRpcMethodFactory.cc
* src/XmlRpcMethodImpl.cc
* src/XmlRpcMethodImpl.h
* src/prefs.cc
* src/prefs.h
* src/usage_text.h
Added --always-resume and --max-resume-failure-tries option. If
--always-resume=false is given, when all given URIs does not
support resume or aria2 encounters N URIs which does not support
resume
(N is the value specified using --max-resume-failure-tries
option), aria2 download file from scratch. The default behavior
is --always-resume=true, which means if all URIs do not support
resume, download fails. I think this is OK because user normally
don't like to see that partially downloaded file is
overwritten(this is particularly true if file size is big). This
option is useful when aria2 is used as download backend and
graceful falling back to overwritten behavior is preferable.
Added exit status value 8, which means download failed because
server did not support resume.
* src/AbstractCommand.cc
* src/DefaultPieceStorage.cc
* src/DownloadCommand.cc
* src/DownloadResultCode.h
* src/FileEntry.h
* src/FtpNegotiationCommand.cc
* src/HttpResponse.cc
* src/HttpResponseCommand.cc
* src/OptionHandlerFactory.cc
* src/RequestGroup.cc
* src/RequestGroup.h
* src/SegmentMan.cc
* src/SegmentMan.h
* src/prefs.cc
* src/prefs.h
* src/usage_text.h
* test/DefaultPieceStorageTest.cc
* test/SegmentManTest.cc
Create CreateRequestCommand to the number of -s/-C when URIs are
added by XML-RPC.
* src/RequestGroup.cc
* src/RequestGroup.h
* src/XmlRpcMethodImpl.cc
Only execute RequestGroupMan::fillRequestGroupFromReserver() when
queue maintenance is requested by RequestGroup to avoid to call
the function unnecessarily.
* src/FillRequestGroupCommand.cc
* src/RequestGroup.cc
* src/RequestGroup.h
* src/RequestGroupMan.cc
* src/RequestGroupMan.h
Removed RequestGroup::initSegmentMan(). Guaranteed that either
both _pieceStorage and _segmentMan are initialized or they are
not.
* src/RequestGroup.cc
* src/RequestGroup.h
* test/BtDependencyTest.cc
Added following 2 keys, followedBy and belongsTo, to the response
of tellStatus.
followedBy: List of GIDs which are generated by the consequence of
this download. For example, when aria2 downloaded Metalink file,
it generates downloads described in it(see *--follow-metalink*
option). This value is useful to track these auto generated
downloads. If there is no such downloads, this key will not be
included in the response.
belongsTo: GID of a parent download. Some downloads are a part of
another download. For example, if a file in Metalink has
BitTorrent resource, the download of .torrent is a part of that
file. If this download has no parent, this key will not be
included in the response.
* src/BtPostDownloadHandler.cc
* src/DownloadResult.h
* src/Metalink2RequestGroup.cc
* src/MetalinkPostDownloadHandler.cc
* src/RequestGroup.cc
* src/RequestGroup.h
* src/UTMetadataPostDownloadHandler.cc
* src/XmlRpcMethodImpl.cc
* src/XmlRpcMethodImpl.h
Don't save control file when aria2 exists while checking piece
hash(behavior of -V option). If control file doesn't exist when
aria2 launched, the completed length in saved control file will be
0 byte and this confuses user. disableSaveControlFile() is called
in RequestGroup::processCheckIntegrityEntry().
enableSaveControlFile() will be called after hash checking is
done. See CheckIntegrityCommand.
* src/CheckIntegrityCommand.cc
* src/RequestGroup.cc
* src/RequestGroup.h
* src/RequestGroupMan.cc
Rewritten PeerStat handling. In the previous implementation,
faster command tries to find slower command. In this new
implementation, slower command tries to find faster command.
PeerStat is now created in each HTTP/FTP request to get download
rate correctly. If the download range is small, the download rate
tends to small and this will occur at the later stage of download.
* src/AbstractCommand.cc
* src/DownloadCommand.cc
* src/FileEntry.cc
* src/FileEntry.h
* src/Request.cc
* src/Request.h
* src/RequestGroup.h
* src/RequestGroupMan.cc
* src/SegmentMan.cc
* src/SegmentMan.h
* test/SegmentManTest.cc
Removed _acceptFeatures and its accessor functions from
RequestGroup since they are not used practically.
* src/HttpRequestCommand.cc
* src/RequestGroup.cc
* src/RequestGroup.h
Removed member _maxTries from RequestGroup class to avoid
duplicate option value. Those classes who refer this variable
should use option value of PREF_MAX_TRIES directly.
* src/AbstractCommand.cc
* src/RequestGroup.cc
* src/RequestGroup.h
* src/TrackerWatcherCommand.cc
Tatsuhiro Tsujikawa