To rewrite segment download mechanism for HTTP/FTP download.
Use BitfieldMan to manage segment download.
* src/HttpResponseCommand.h
(executeInternal): Pass the reference of segment.
* src/AbstractCommand.cc
(prepareForRetry): Call segmentMan->cancelSegment here.
(onAbort): Call segmentMan->cancelSegment here.
* src/HttpDownloadCommand.cc
(prepareForNextSegment): New function.
* src/DownloadEngineFactory.cc
(newConsoleEngine): Removed splitter.
(newTorrentConsoleEngine): Removed splitter.
* src/Request.h
(segment): Renamed from seg.
* src/FtpInitiateConnectionCommand.h
(executeInternal): Pass the reference of segment.
* src/AbstractCommand.h
(executeInternal): Pass the reference of segment.
* src/pref.h
(PREF_SEGMENT_SIZE): New definition.
* src/HttpProxyRequestCommand.h
(executeInternal): Pass the reference of segment.
* src/HttpResponseCommand.cc
(checkResponse): Allowed status 206 when a request range starts
0.
(handleDefaultEncoding): Rewritten the code related to Segment.
(handleOtherEncoding): Rewritten the code related to Segment.
* src/SegmentMan.h
(SegmentEntry): New class.
(SegmentEntries): New type definition.
(bitfield): New variable.
(usedSegmentEntries): New variable.
(onNullBitfield): New function.
(checkoutSegment): New function.
(segments): Removed.
(splitter): Removed.
(unregisterId): Removed.
(getSegment): New function(overload)
(getDownloadedSize): Removed.
(cancelSegment): New function.
(completeSegment): New function.
(initBitfield): New function.
(hasSegment): New function.
(getDownloadLength): New function.
* src/BitfieldMan.h
(getStartIndex): New function.
(getEndIndex): New function.
(getMissingUnusedIndex): New function(overload).
(getSparseMissingUnusedIndex): New function.
* src/BitfieldMan.cc
(getMissingIndexRandomly): Handle the last byte of bitfield
properly.
(getMissingUnusedIndex): New function(overload).
(Range): New class.
(getStartIndex): New function.
(getEndIndex): New function.
(getSparseMissingUnusedIndex): New function.
(isBitSetInternal): Return false if the given index is less than
0.
* src/HttpInitiateConnectionCommand.h
(executeInternal): Pass the reference of segment.
* src/FtpNegotiateCommand.h
(executeInternal): Pass the reference of segment.
* src/FtpNegotiateCommand.cc
(recvSize): Initialize bitfield here.
* src/FtpTunnelResponseCommand.h
(executeInternal): Pass the reference of segment.
* src/HttpConnection.cc
(createRequest): Rewritten range header processing.
* src/DownloadCommand.h
(executeInternal): Pass the reference of segment.
(prepareForRetry): Removed.
(prepareForNextSegment): Added an argument segment. Made it a
virtual
function.
* src/main.cc
(main): Set the initial value of PREF_SEGMENT_SIZE to 1MB.
* src/SegmentMan.cc
(SegmentMan): Added bitfield. Removed splitter.
(~SegmentMan): Added bitfield. Removed splitter.
(unregisterId): Removed.
(getSegment): Rewritten.
(updateSegment): Rewritten.
(save): Rewritten.
(read): Rewritten.
(finished): Rewritten.
(getDownloadedSize): Removed.
(initBitfield): New function.
(FindSegmentEntryByIndex): New function object.
(FindSegmentEntryByCuid): New function object.
(checkoutSegment): New function.
(onNullBitfield): New function.
(getSegment): New function(overload).
(CancelSegment): New function object.
(cancelSegment): New function.
(completeSegment): New function.
(hasSegment): New function.
(getDownloadLength): New function.
* src/FtpInitiateConnectionCommand.cc
(executeInternal): Load .aria2 file after hostname resolution
finishes.
* src/Segment.h: Rewritten.
* src/Segment.cc (operator<<): New function.
* src/HttpDownloadCommand.h
(prepareForNextSegment): New function.
* src/Request.cc
(resetUrl): Made segment null.
* src/DownloadEngine.cc
(~DownloadEngine): Call cleanQueue before deleting segmentMan.
* src/HttpProxyRequestCommand.h
(executeInternal): Pass the reference of segment.
* src/DownloadCommand.cc
(executeInternal): Rewritten the code related to Segment.
(prepareForRetry): Removed.
(prepareForNextSegment): Rewritten.
* src/FtpTunnelResponseCommand.h
(executeInternal): Pass the reference of segment.
To add HTTP 1.1 persistent connection support(experimental)
* src/HttpRequestCommand.cc
(executeInternal): Disable keep alive if it is disabled by
configuration.
* src/Request.h
(keepAlive): New variable.
(isKeepAlive): New function.
(setKeepAlive): New function.
* src/pref.h
(PREF_HTTP_KEEP_ALIVE): New definition.
* src/HttpResponseCommand.cc
(executeInternal): Check the remote server supports keep alive.
* src/HttpConnection.cc
(createRequest): Send "Connection: close" only if keep alive is
disabled.
* src/main.cc
(main):
Set the initial value(false) of PREF_KEEP_ALIVE to false.
To add max download speed limit:
* src/pref.h
(PREF_MAX_SPEED_LIMIT): New definition.
* src/PeerInteractionCommand.cc
(executeInternal): Added max download speed limit. Not tested
yet.
* src/SegmentMan.h
(PeerStats): New type definition.
(peerStats): New variable.
(registerPeerStat): New function.
(FindPeerStat): New function object.
(getPeerStat): New function.
(calculateDownloadSpeed): New function.
* src/SpeedCalc.h: New class.
* src/SpeedCalc.cc: New class.
* src/main.cc
(main):
Set the initial value of PREF_MAX_SPEED_LIMIT to 0(which means
the
download speed is not restricted).
* src/PeerStat.h: New class.
* src/SegmentMan.cc
(registerPeerStat): New function.
(calculateDownloadSpeed): New function.
* src/DownloadCommand.cc
(STARTUP_IDLE_TIME): New definition.
(DownloadCommand): Register peerStat to segmentMan. Call
peerStat->
downloadStart.
(~DownloadCommand): Call peerStat->downloadStop.
(executeInternal): Added download speed limitter. Rewritten
lowest
speed limitter.
* src/HttpConnection.cc
(receiveResponse): Fixed: eohIndex[headerBuf] ->
headerBuf[eohIndex].
* src/AbstractCommand.cc
(resolveHostname): Throw DlAbortEx if a name resolution failes.
Added hostname to the error message.
* src/ConsoleDownloadEngine.cc
(calculateStatistics): Initialize psize with dlSize.
* src/PieceMessage.cc
(receivedAction): Do not call peerInteraction->abortPiece here.
(onGotWrongPiece): Call peerInteraction->abortPiece here.
* src/BitfieldMan.h
(clearAllUseBit): New function.
(setAllUseBit): New function.
* src/BitfieldMan.cc
(clearAllBit): Do not clear useBitfield here.
(clearAllUseBit): New function.
(setAllUseBit): New function.
* src/Piece.cc
(clearAllBlock): Call bitfield->clearAllUseBit().
* src/TorrentMan.h:
(advertisePiece): Updated doc.
(getAdvertisedPieceIndexes): Updated doc.
(removeAdvertisedPiece); New function.
* src/TorrentMan.cc
(FindElapsedHave): New function object.
(removeAdvertisedPiece): New function.
* src/HaveEraseCommand.h: New class.
* src/HaveEraseCommand.cc: New class.
* src/FeatureConfig.h: New class.
* src/FeatureConfig.cc: New class.
* src/Request.h
(defaultPorts): Removed.
* src/Request.cc
(FeatureConfig.h): Included.
(Request): Removed the statements related to defaultPorts.
(parseUrl): Removed metalinkEnabled. Use FeatureConfig instead.
A default port number is now retrieved from FeatureConfig.
* src/main.cc
(HaveEraseCommand.h): Included.
(showVersion): Added the output of feature list.
(main): Added HaveEraseCommand to command queue in BitTorrent
downloading.
* src/PeerInteractionCommand.h
(chokeCheckPoint): Commented out.
(periodicExecPoint): New variable.
* src/PeerInteractionCommand.cc
(executeInternal): Following methods are now called in at least
every
0.5 seconds to reduce CPU usage:
detectMessageFlooding(), peerInteraction->checkRequestSlot(),
checkHave(), sendKeepAlive().
(checkLongTimePeerChoking): Commented out.
* src/BitfieldMan.h
(getNthBitIndex): Changed the method signature.
(getMissingIndexRandomly): Changed the method signature.
* src/BitfieldMan.cc
(getNthBitIndex): Rewritten
(getMissingIndexRandomly): Rewritten.
(hasMissingPiece): Rewritten.
(getMissingIndex): Refactored.
(getMissingUnusedIndex); Refactored.
(getMissingIndex): Refactored.
To add the support for Metalink3.0 backward compatible links:
* src/Request.h
(SAFE_CHARS): Added '#'.
(METALINK_MARK): New definition.
* src/Request.cc
(parseUrl): Added the support for Metalink3.0 backward
compatible
links.
etc
* src/PeerInteraction.cc
(MsgPushBack): Removed.
(sendMessages): Use STL copy and back_inserter.
(rejectAllPieceMessageInQueue): Use STL copy and back_inserter.
(rejectPieceMessageInQueue): Use STL copy and back_inserter.
* src/TorrentMan.h
(MAX_PEER_LIST_SIZE): Changed to 100 from 250.
To add Metalink support(http/ftp only):
* src/AbstractCommand.h
(tryReserved): New function.
* src/AbstractCommand.cc
(execute): Call tryReserved().
(tryReserved): New function.
* src/Request.h
(Requests): New type definition.
* src/SegmentMan.h
(reserved): New variable.
* src/Util.h
(fileChecksum): New function.
(toUpper): New function.
(toLower): New function.
* src/Util.cc
(messageDigest.h): Included.
(trim): Trim \r\n\t.
(fileChecksum): New function.
(toUpper): New function.
(toLower): New function.
* src/main.cc
(normalDownload): New function.
(main): Added 2 command-line options: metalink-file,
metalink-connection. Their usage has not been written yet.
* src/MetalinkProcessor.h: New class.
* src/Xml2MetalinkProcessor.h: New class.
* src/Xml2MetalinkProcessor.cc: New class.
* src/MetalinkEntry.h: New class.
* src/MetalinkEntry.cc: New class.
* src/MetalinkResource.h: New class.
* src/MetalinkResource.cc: New class.
To add md5 message digest checking:
* src/messageDigest.h: Rewritten.
* src/MultiDiskWriter.cc: Updated according to the changes in
messageDigest.h.
* src/ShaVisitor.cc: Updated according to the changes in
messageDigest.h.
* src/Util.cc: Updated according to the changes in
messageDigest.h.
* src/AbstractDiskWriter.cc: Updated according to the changes in
messageDigest.h.
To fix a bug that causes segfault when the payload length in
peer
message is less than 0:
* src/PeerConnection.cc:
(receiveMessage): Fixed the bug.
* src/PeerMessageUtil.cc
(checkLength): Throw an exception if length is less than or
equals to
0.
To add new interfaces to Base64 encoding/decoding:
* src/Base64.h
(part_encode): Changed the method signature.
(encode): New function(overload).
(decode): New function(overload).
* src/Base64.cc
(part_encode): Rewritten.
(encode): Rewritten.
(encode): New function(overload).
To prevent a peer to download same piece if there is an error in
checksum:
* src/PieceMessage.cc
(receivedAction): Call peerInteraction->abortPiece().
* TorrentDownloadEngine.cc: Prints "Download complete" message
instead of downloaded size and progress(%) after download
completes.
* PeerInteractionCommand.cc: After download completes, sends
unchoke
message to the peer if it is interested in what localhost has
downloaded.
* TorrentMan.cc: In single-file mode, copy temporary file to the
final
destination instead of just renaming it.
* TorrentMan.cc: Added deleteTempFile().
* PeerAbstractCommand.cc: do not stop execution after download
completes. This makes localhost a seeder.
* Util.{h,cc}: Added fileCopy().
* PeerListenCommand.cc: do not stop execution after download
completes.
This makes localhost a seeder.
* main.cc: Do not call TorrentMan::fixFilename() in
torrentHandler.
Added TorrentMan::deleteTempFile() to torrentHandler.
Initialized the variable dir as ".".
* TorrentMan.h: Changed DEFAULT_ANNOUNCE_INTERVAL to 120
seconds.
Deleted renameSingleFile().
Added copySingleFile(), deleteTempFile().
* DownloadEngine.h: Added virtual function afterEachIteration().
* TorrentDownloadEngine.cc: Move a call to
TorrentMan::fixFilename()
in onEndOfRun() to afterEachIteration().
In onEndOfRun(), changed if condition to check whether
filenameFixed is
true.
* Util.cc: Implemented fileCopy() using rangedFileCopy().
In rangedFileCopy(), added try-catch block to properly close
file
descriptors.
* TorrentDownloadEngine.cc: Added a member variable
filenameFixed.
Added afterEachIteration(), isFilenameFixed().
* Peer.cc: Changed choking strategy.
* PreAllocationDiskWriter.cc: Drop O_DIRECT flag.
* TrackerInitCommand.cc: Send completed event only once.
* DownloadEngine.cc: Added a call to afterEachIteration().
* TrackerUpdateCommand.cc: Do not stop execution after download
completes.
* TorrentMan.h: Defined MAX_PEER_UPDATE as 15. aria2 attempts to
connect the peers at most MAX_PEER_UPDATE when a peer list is
received from a tracker.
* TrackerUpdateCommand.cc: Implemented above mentioned behavior.
Decreased the number of failure peers to delete to 0(just
comment out
the line).
* Release 0.3.1
* Added SocketCore. Socket is now handle class for SocketCore.
* Fixed bug in ChunkedEncoding: expanding buffer size is wrong
* Fixed bug in DownloadCommand: In Chunked Encoding, it wrongly
adds to Segment.ds buff length from the socket.
Tatsuhiro Tsujikawa