Applied AdaptiveURISelector patch from Aurelien Lefebvre. This
patch adds AdaptiveURISelector, which selects one of the bests
mirrors for first and reserved connections. For supplementary
ones, it returns mirrors which has not been tested yet, and if
each of them already tested, returns mirrors which has to be
tested again. Otherwise, it doesn't return anymore mirrors.
* src/AdaptiveURISelector.cc
* src/AdaptiveURISelector.h
* src/FtpNegotiationCommand.cc
* src/HttpResponseCommand.cc
* src/Makefile.am
* src/Makefile.in
* src/OptionHandlerFactory.cc
* src/RequestGroup.cc
* src/RequestGroup.h
* src/RequestGroupMan.cc
* src/ServerStat.cc
* src/ServerStat.h
* src/ServerStatMan.cc
* src/SpeedCalc.cc
* src/URISelector.h
* src/prefs.cc
* src/prefs.h
* test/RequestGroupManTest.cc
* test/ServerStatManTest.cc
* test/ServerStatTest.cc
Cache last calculated average download/upload speed.
PeerStat::getAvgDownloadSpeed(), PeerStat::getAvgUploadSpeed()
return
cached value.
Now SpeedCalc::changeSw() is called from
SpeedCalc::calculateSpeed()
* src/PeerStat.h
* src/SpeedCalc.cc
* src/SpeedCalc.h
Reduce the number of calls to gettimeofday to lower CPU load.
* src/TimeA2.h, src/TimeA2.cc (differenceInMillis): New
function.
* src/SpeedCalc.h, src/SpeedCalc.cc (calculateSpeed): New
function.
* src/Peer.h (calculateUploadSpeed): New function.
(calculateDownloadSpeed): New function.
* src/DefaultPeerStorage.cc
(CalculateStat): Added _now variable.
In operator(), call Peer::calculateDownloadSpeed(const struct
timeval&)
and Peer::calculateUploadSpeed(const struct timeval&)
Drop connection if no request or piece message is exchanged in
a certain interval.
* src/DefaultBtInteractive.h
(btRuntime): New variable.
(inactiveCheckPoint): New variable.
(checkActiveInteraction): New function.
* src/DefaultBtInteractive.cc (receiveMessages): Reset timer
when
request or piece message is received.
(checkActiveInteraction): New function.
(doInteractionProcessing): Call checkActiveInteraction.
Fixed the bug that causes remote Metalink and Torrent files are
not
processed.
* src/MultiUrlRequestInfo.cc (createNextRequestInfo): Fixed the
bug.
To add timeout to async name resolution:
* src/AbstractCommand.h
(nameResolveFinished): New function. This is not elegant way. It needs
to be more refined.
(nameResolverCheck): New variable.
* src/AbstractCommand.cc
(AbstractCommand): Added nameResolverCheck.
(execute): Added the check whether the name resolution has finished.
(setNameResolverCheck): Set nameResolverCheck to true.
(disableNameResolverCheck): Set nameResolverCheck to false.
(nameResolverFinished): New function.
* src/FtpInitiateConnectionCommand.h
(nameResolverFinished): New function.
* src/HttpInitiateConnectionCommand.h
(nameResolverFinished): New function.
To add the support for a non-compact response from a tracker:
* src/PeerListProcessor.h: New class.
* src/DefaultPeerListProcessor.h: New class.
* src/DefaultPeerListProcessor.cc: New class.
* src/CompactPeerListProcessor.h: New class.
* src/CompactPeerListProcessor.cc: New class.
* src/DelegatingPeerListProcessor.h: New class.
* src/DelegatingPeerListProcessor.cc: New class.
* src/TorrentMan.cc
(addPeer): New function(overload).
(addPeer): Delete unused peers only when new peer is added.
* src/TorrentMan.h
(PeerListProcessor.h): Included.
(Peers): Removed.
(addPeer): New function(overload).
* src/TrackerUpdateCommand.cc
(netinet/in.h): Removed.
(DelegatingPeerListProcessor.h): Included.
(execute): Updated to use DelegatingPeerListProcessor.
To fix the memory leak in TorrentMan::peers:
* src/PeerAbstractCommand.cc
(onAbort): Added peer->resetStatus().
* src/Peer.h
(resetStatus): Made public.
To improve the precision of the speed calculation:
* src/SpeedCalc.h
(nextInterval): New variable.
* src/SpeedCalc.cc
(reset): Added nextInterval.
(isIntervalOver): Use nextInterval instead of CHANGE_INTERVAL_SEC.
(changeSw): Set nextInterval to 15 seconds relative to the current
instant time.
* src/main.cc
(showVersion): Updated.
* src/AbstractCommand.cc
(execute): Check whether the download has finished before
checking
socket status.
Return true if peerStat->getStatus() == REQUEST_IDLE.
Do not exit even if no segment is available.
* src/prefs.h
(PREF_STARTUP_IDLE_TIME): New definition.
* src/PeerInteractionCommand.cc
(executeInternal): Removed max speed limit. Because it performs
bad.
(receiveMessages): Added max speed limit. This was better than
above,
but still a little bit to be desired. Sometimes the download
speed
became much faster than I specified.
* src/SpeedCalc.h
(start): New variable.
(accumulatedLength): New variable.
(getAvgSpeed): New function.
* src/SpeedCalc.cc
(reset): Added start, accumulatedLength.
(update): Added calculation of an average speed.
(getAvgSpeed): New function.
* src/DownloadCommand.h
(sw): Removed.
* src/main.cc
(main): Added PREF_STARTUP_IDLE_TIME.
* src/PeerStat.h
(STATUS): Added REQUEST_IDLE.
(getMaxSpeed): Renamed as getMaxDownloadSpeed().
(getAvgDownloadSpeed): New function
(requestIdle): New function.
* src/SegmentMan.h
(SegmentEntryHandle): New type definition.
(SegmentEntries): Now holds SegmentEntryHandle.
(findSlowerSegmentEntry): New funtion.
* src/SegmentMan.cc
(save): Updated according to the changes in SegmentEntries.
(read): Updated according to the changes in SegmentEntries.
(FindSegmentEntryByIndex): Updated according to the changes in
SegmentEntries.
(FindSegmentEntryByCuid): Updated according to the changes in
SegmentEntries.
(checkoutSegment): Updated according to the changes in
SegmentEntries.
(onNullBitfield): Updated according to the changes in
SegmentEntries.
Renamed uitr as itr.
(findSlowerSegmentEntry): New function.
(getSegment): Updated according to the changes in
SegmentEntries.
Added the feature that cancels the segment with slow server and
fast
one takes it over.
(cancelSegment): Updated according to the changes in
SegmentEntries.
(getDownloadLength): Updated according to the changes in
SegmentEntries.
(init): Assigned 0 to bitfield after deleting it.
* src/DownloadCommand.cc
(STARTUP_IDLE_TIME): Removed.
(executeInternal): Use PREF_STARTUP_IDLE_TIME.
To rewrite segment download mechanism for HTTP/FTP download.
Use BitfieldMan to manage segment download.
* src/HttpResponseCommand.h
(executeInternal): Pass the reference of segment.
* src/AbstractCommand.cc
(prepareForRetry): Call segmentMan->cancelSegment here.
(onAbort): Call segmentMan->cancelSegment here.
* src/HttpDownloadCommand.cc
(prepareForNextSegment): New function.
* src/DownloadEngineFactory.cc
(newConsoleEngine): Removed splitter.
(newTorrentConsoleEngine): Removed splitter.
* src/Request.h
(segment): Renamed from seg.
* src/FtpInitiateConnectionCommand.h
(executeInternal): Pass the reference of segment.
* src/AbstractCommand.h
(executeInternal): Pass the reference of segment.
* src/pref.h
(PREF_SEGMENT_SIZE): New definition.
* src/HttpProxyRequestCommand.h
(executeInternal): Pass the reference of segment.
* src/HttpResponseCommand.cc
(checkResponse): Allowed status 206 when a request range starts
0.
(handleDefaultEncoding): Rewritten the code related to Segment.
(handleOtherEncoding): Rewritten the code related to Segment.
* src/SegmentMan.h
(SegmentEntry): New class.
(SegmentEntries): New type definition.
(bitfield): New variable.
(usedSegmentEntries): New variable.
(onNullBitfield): New function.
(checkoutSegment): New function.
(segments): Removed.
(splitter): Removed.
(unregisterId): Removed.
(getSegment): New function(overload)
(getDownloadedSize): Removed.
(cancelSegment): New function.
(completeSegment): New function.
(initBitfield): New function.
(hasSegment): New function.
(getDownloadLength): New function.
* src/BitfieldMan.h
(getStartIndex): New function.
(getEndIndex): New function.
(getMissingUnusedIndex): New function(overload).
(getSparseMissingUnusedIndex): New function.
* src/BitfieldMan.cc
(getMissingIndexRandomly): Handle the last byte of bitfield
properly.
(getMissingUnusedIndex): New function(overload).
(Range): New class.
(getStartIndex): New function.
(getEndIndex): New function.
(getSparseMissingUnusedIndex): New function.
(isBitSetInternal): Return false if the given index is less than
0.
* src/HttpInitiateConnectionCommand.h
(executeInternal): Pass the reference of segment.
* src/FtpNegotiateCommand.h
(executeInternal): Pass the reference of segment.
* src/FtpNegotiateCommand.cc
(recvSize): Initialize bitfield here.
* src/FtpTunnelResponseCommand.h
(executeInternal): Pass the reference of segment.
* src/HttpConnection.cc
(createRequest): Rewritten range header processing.
* src/DownloadCommand.h
(executeInternal): Pass the reference of segment.
(prepareForRetry): Removed.
(prepareForNextSegment): Added an argument segment. Made it a
virtual
function.
* src/main.cc
(main): Set the initial value of PREF_SEGMENT_SIZE to 1MB.
* src/SegmentMan.cc
(SegmentMan): Added bitfield. Removed splitter.
(~SegmentMan): Added bitfield. Removed splitter.
(unregisterId): Removed.
(getSegment): Rewritten.
(updateSegment): Rewritten.
(save): Rewritten.
(read): Rewritten.
(finished): Rewritten.
(getDownloadedSize): Removed.
(initBitfield): New function.
(FindSegmentEntryByIndex): New function object.
(FindSegmentEntryByCuid): New function object.
(checkoutSegment): New function.
(onNullBitfield): New function.
(getSegment): New function(overload).
(CancelSegment): New function object.
(cancelSegment): New function.
(completeSegment): New function.
(hasSegment): New function.
(getDownloadLength): New function.
* src/FtpInitiateConnectionCommand.cc
(executeInternal): Load .aria2 file after hostname resolution
finishes.
* src/Segment.h: Rewritten.
* src/Segment.cc (operator<<): New function.
* src/HttpDownloadCommand.h
(prepareForNextSegment): New function.
* src/Request.cc
(resetUrl): Made segment null.
* src/DownloadEngine.cc
(~DownloadEngine): Call cleanQueue before deleting segmentMan.
* src/HttpProxyRequestCommand.h
(executeInternal): Pass the reference of segment.
* src/DownloadCommand.cc
(executeInternal): Rewritten the code related to Segment.
(prepareForRetry): Removed.
(prepareForNextSegment): Rewritten.
* src/FtpTunnelResponseCommand.h
(executeInternal): Pass the reference of segment.
To add HTTP 1.1 persistent connection support(experimental)
* src/HttpRequestCommand.cc
(executeInternal): Disable keep alive if it is disabled by
configuration.
* src/Request.h
(keepAlive): New variable.
(isKeepAlive): New function.
(setKeepAlive): New function.
* src/pref.h
(PREF_HTTP_KEEP_ALIVE): New definition.
* src/HttpResponseCommand.cc
(executeInternal): Check the remote server supports keep alive.
* src/HttpConnection.cc
(createRequest): Send "Connection: close" only if keep alive is
disabled.
* src/main.cc
(main):
Set the initial value(false) of PREF_KEEP_ALIVE to false.
To add max download speed limit:
* src/pref.h
(PREF_MAX_SPEED_LIMIT): New definition.
* src/PeerInteractionCommand.cc
(executeInternal): Added max download speed limit. Not tested
yet.
* src/SegmentMan.h
(PeerStats): New type definition.
(peerStats): New variable.
(registerPeerStat): New function.
(FindPeerStat): New function object.
(getPeerStat): New function.
(calculateDownloadSpeed): New function.
* src/SpeedCalc.h: New class.
* src/SpeedCalc.cc: New class.
* src/main.cc
(main):
Set the initial value of PREF_MAX_SPEED_LIMIT to 0(which means
the
download speed is not restricted).
* src/PeerStat.h: New class.
* src/SegmentMan.cc
(registerPeerStat): New function.
(calculateDownloadSpeed): New function.
* src/DownloadCommand.cc
(STARTUP_IDLE_TIME): New definition.
(DownloadCommand): Register peerStat to segmentMan. Call
peerStat->
downloadStart.
(~DownloadCommand): Call peerStat->downloadStop.
(executeInternal): Added download speed limitter. Rewritten
lowest
speed limitter.
* src/HttpConnection.cc
(receiveResponse): Fixed: eohIndex[headerBuf] ->
headerBuf[eohIndex].
* src/AbstractCommand.cc
(resolveHostname): Throw DlAbortEx if a name resolution failes.
Added hostname to the error message.
* src/ConsoleDownloadEngine.cc
(calculateStatistics): Initialize psize with dlSize.
* src/PieceMessage.cc
(receivedAction): Do not call peerInteraction->abortPiece here.
(onGotWrongPiece): Call peerInteraction->abortPiece here.
* src/BitfieldMan.h
(clearAllUseBit): New function.
(setAllUseBit): New function.
* src/BitfieldMan.cc
(clearAllBit): Do not clear useBitfield here.
(clearAllUseBit): New function.
(setAllUseBit): New function.
* src/Piece.cc
(clearAllBlock): Call bitfield->clearAllUseBit().
Tatsuhiro Tsujikawa