To rewrite segment download mechanism for HTTP/FTP download.
Use BitfieldMan to manage segment download.
* src/HttpResponseCommand.h
(executeInternal): Pass the reference of segment.
* src/AbstractCommand.cc
(prepareForRetry): Call segmentMan->cancelSegment here.
(onAbort): Call segmentMan->cancelSegment here.
* src/HttpDownloadCommand.cc
(prepareForNextSegment): New function.
* src/DownloadEngineFactory.cc
(newConsoleEngine): Removed splitter.
(newTorrentConsoleEngine): Removed splitter.
* src/Request.h
(segment): Renamed from seg.
* src/FtpInitiateConnectionCommand.h
(executeInternal): Pass the reference of segment.
* src/AbstractCommand.h
(executeInternal): Pass the reference of segment.
* src/pref.h
(PREF_SEGMENT_SIZE): New definition.
* src/HttpProxyRequestCommand.h
(executeInternal): Pass the reference of segment.
* src/HttpResponseCommand.cc
(checkResponse): Allowed status 206 when a request range starts
0.
(handleDefaultEncoding): Rewritten the code related to Segment.
(handleOtherEncoding): Rewritten the code related to Segment.
* src/SegmentMan.h
(SegmentEntry): New class.
(SegmentEntries): New type definition.
(bitfield): New variable.
(usedSegmentEntries): New variable.
(onNullBitfield): New function.
(checkoutSegment): New function.
(segments): Removed.
(splitter): Removed.
(unregisterId): Removed.
(getSegment): New function(overload)
(getDownloadedSize): Removed.
(cancelSegment): New function.
(completeSegment): New function.
(initBitfield): New function.
(hasSegment): New function.
(getDownloadLength): New function.
* src/BitfieldMan.h
(getStartIndex): New function.
(getEndIndex): New function.
(getMissingUnusedIndex): New function(overload).
(getSparseMissingUnusedIndex): New function.
* src/BitfieldMan.cc
(getMissingIndexRandomly): Handle the last byte of bitfield
properly.
(getMissingUnusedIndex): New function(overload).
(Range): New class.
(getStartIndex): New function.
(getEndIndex): New function.
(getSparseMissingUnusedIndex): New function.
(isBitSetInternal): Return false if the given index is less than
0.
* src/HttpInitiateConnectionCommand.h
(executeInternal): Pass the reference of segment.
* src/FtpNegotiateCommand.h
(executeInternal): Pass the reference of segment.
* src/FtpNegotiateCommand.cc
(recvSize): Initialize bitfield here.
* src/FtpTunnelResponseCommand.h
(executeInternal): Pass the reference of segment.
* src/HttpConnection.cc
(createRequest): Rewritten range header processing.
* src/DownloadCommand.h
(executeInternal): Pass the reference of segment.
(prepareForRetry): Removed.
(prepareForNextSegment): Added an argument segment. Made it a
virtual
function.
* src/main.cc
(main): Set the initial value of PREF_SEGMENT_SIZE to 1MB.
* src/SegmentMan.cc
(SegmentMan): Added bitfield. Removed splitter.
(~SegmentMan): Added bitfield. Removed splitter.
(unregisterId): Removed.
(getSegment): Rewritten.
(updateSegment): Rewritten.
(save): Rewritten.
(read): Rewritten.
(finished): Rewritten.
(getDownloadedSize): Removed.
(initBitfield): New function.
(FindSegmentEntryByIndex): New function object.
(FindSegmentEntryByCuid): New function object.
(checkoutSegment): New function.
(onNullBitfield): New function.
(getSegment): New function(overload).
(CancelSegment): New function object.
(cancelSegment): New function.
(completeSegment): New function.
(hasSegment): New function.
(getDownloadLength): New function.
* src/FtpInitiateConnectionCommand.cc
(executeInternal): Load .aria2 file after hostname resolution
finishes.
* src/Segment.h: Rewritten.
* src/Segment.cc (operator<<): New function.
* src/HttpDownloadCommand.h
(prepareForNextSegment): New function.
* src/Request.cc
(resetUrl): Made segment null.
* src/DownloadEngine.cc
(~DownloadEngine): Call cleanQueue before deleting segmentMan.
* src/HttpProxyRequestCommand.h
(executeInternal): Pass the reference of segment.
* src/DownloadCommand.cc
(executeInternal): Rewritten the code related to Segment.
(prepareForRetry): Removed.
(prepareForNextSegment): Rewritten.
* src/FtpTunnelResponseCommand.h
(executeInternal): Pass the reference of segment.
To add HTTP 1.1 persistent connection support(experimental)
* src/HttpRequestCommand.cc
(executeInternal): Disable keep alive if it is disabled by
configuration.
* src/Request.h
(keepAlive): New variable.
(isKeepAlive): New function.
(setKeepAlive): New function.
* src/pref.h
(PREF_HTTP_KEEP_ALIVE): New definition.
* src/HttpResponseCommand.cc
(executeInternal): Check the remote server supports keep alive.
* src/HttpConnection.cc
(createRequest): Send "Connection: close" only if keep alive is
disabled.
* src/main.cc
(main):
Set the initial value(false) of PREF_KEEP_ALIVE to false.
To add max download speed limit:
* src/pref.h
(PREF_MAX_SPEED_LIMIT): New definition.
* src/PeerInteractionCommand.cc
(executeInternal): Added max download speed limit. Not tested
yet.
* src/SegmentMan.h
(PeerStats): New type definition.
(peerStats): New variable.
(registerPeerStat): New function.
(FindPeerStat): New function object.
(getPeerStat): New function.
(calculateDownloadSpeed): New function.
* src/SpeedCalc.h: New class.
* src/SpeedCalc.cc: New class.
* src/main.cc
(main):
Set the initial value of PREF_MAX_SPEED_LIMIT to 0(which means
the
download speed is not restricted).
* src/PeerStat.h: New class.
* src/SegmentMan.cc
(registerPeerStat): New function.
(calculateDownloadSpeed): New function.
* src/DownloadCommand.cc
(STARTUP_IDLE_TIME): New definition.
(DownloadCommand): Register peerStat to segmentMan. Call
peerStat->
downloadStart.
(~DownloadCommand): Call peerStat->downloadStop.
(executeInternal): Added download speed limitter. Rewritten
lowest
speed limitter.
* src/HttpConnection.cc
(receiveResponse): Fixed: eohIndex[headerBuf] ->
headerBuf[eohIndex].
* src/AbstractCommand.cc
(resolveHostname): Throw DlAbortEx if a name resolution failes.
Added hostname to the error message.
* src/ConsoleDownloadEngine.cc
(calculateStatistics): Initialize psize with dlSize.
* src/PieceMessage.cc
(receivedAction): Do not call peerInteraction->abortPiece here.
(onGotWrongPiece): Call peerInteraction->abortPiece here.
* src/BitfieldMan.h
(clearAllUseBit): New function.
(setAllUseBit): New function.
* src/BitfieldMan.cc
(clearAllBit): Do not clear useBitfield here.
(clearAllUseBit): New function.
(setAllUseBit): New function.
* src/Piece.cc
(clearAllBlock): Call bitfield->clearAllUseBit().
* src/Option.h
(getAsBool): New function.
* src/Option.cc
(prefs.h): Included.
(defined): 0-length value is now recognized as undefined.
(getAsInt): Rewritten.
(getAsLLInt): Rewritten.
(getAsBool): New function.
* src/FeatureConfig.h: Rewritten.
* src/FeatureConfig.cc: Rewritten.
* src/prefs.h
(PREF_STDOUT_LOG): New definition.
(PREF_LOG): New definition.
(PREF_DIR): New definition.
(PREF_OUT): New definition.
(PREF_SPLIT): New definition.
(PREF_DAEMON): New definition.
(PREF_REFERER): New definition.
(PREF_TORRENT_FILE): New definition.
(PREF_LISTEN_PORT): New definition.
(PREF_METALINK_FILE): New definition.
(PREF_METALINK_VERSION): New definition.
(PREF_METALINK_LANGUAGE): New definition.
(PREF_METALINK_OS): New definition.
(PREF_METALINK_SERVERS): New definition.
* src/main.cc
(main): Following command-line parameters are now put into
Option
class: stdoutLog, logfile, dir, ufilename, split, daemonMode,
referer, torrentFile, metalinkFile, listenPort, metalinkVersion,
metalinkLanguage, metalinkOs, metalinkServers
To fix the bug that aria2 can not handle http response header
properly.
* src/HttpHeader.cc
(put): Made name lowercased.
(defined): Made name lowercased.
(getFirst): Made name lowercased.
(get): Made name lowercased.
(getFirstAsInt): Rewritten.
(getFirstAsLLInt): Rewritten.
* src/TorrentMan.h:
(advertisePiece): Updated doc.
(getAdvertisedPieceIndexes): Updated doc.
(removeAdvertisedPiece); New function.
* src/TorrentMan.cc
(FindElapsedHave): New function object.
(removeAdvertisedPiece): New function.
* src/HaveEraseCommand.h: New class.
* src/HaveEraseCommand.cc: New class.
* src/FeatureConfig.h: New class.
* src/FeatureConfig.cc: New class.
* src/Request.h
(defaultPorts): Removed.
* src/Request.cc
(FeatureConfig.h): Included.
(Request): Removed the statements related to defaultPorts.
(parseUrl): Removed metalinkEnabled. Use FeatureConfig instead.
A default port number is now retrieved from FeatureConfig.
* src/main.cc
(HaveEraseCommand.h): Included.
(showVersion): Added the output of feature list.
(main): Added HaveEraseCommand to command queue in BitTorrent
downloading.
* src/PeerInteractionCommand.h
(chokeCheckPoint): Commented out.
(periodicExecPoint): New variable.
* src/PeerInteractionCommand.cc
(executeInternal): Following methods are now called in at least
every
0.5 seconds to reduce CPU usage:
detectMessageFlooding(), peerInteraction->checkRequestSlot(),
checkHave(), sendKeepAlive().
(checkLongTimePeerChoking): Commented out.
* src/BitfieldMan.h
(getNthBitIndex): Changed the method signature.
(getMissingIndexRandomly): Changed the method signature.
* src/BitfieldMan.cc
(getNthBitIndex): Rewritten
(getMissingIndexRandomly): Rewritten.
(hasMissingPiece): Rewritten.
(getMissingIndex): Refactored.
(getMissingUnusedIndex); Refactored.
(getMissingIndex): Refactored.
To add the support for Metalink3.0 backward compatible links:
* src/Request.h
(SAFE_CHARS): Added '#'.
(METALINK_MARK): New definition.
* src/Request.cc
(parseUrl): Added the support for Metalink3.0 backward
compatible
links.
etc
* src/PeerInteraction.cc
(MsgPushBack): Removed.
(sendMessages): Use STL copy and back_inserter.
(rejectAllPieceMessageInQueue): Use STL copy and back_inserter.
(rejectPieceMessageInQueue): Use STL copy and back_inserter.
* src/TorrentMan.h
(MAX_PEER_LIST_SIZE): Changed to 100 from 250.
* src/TorrentMan.cc (setupInternal1): Fixed peerId generation
bug.
* src/SimpleLogger.h (writeHeader): New function.
* src/SimpleLogger.cc (writeHeader): New function.
(writeLog): Fixed the bug that causes segfaults if exception
message
contains an unescaped "%" character.
* src/TrackerWatcherCommand.cc (execute): Added a short sleep
when a tracker request fails.
* src/Request.cc (parseUrl): Query string is now handled
properly.
* configure.in: Added gnutls support. Added several CPP macros.
* m4/openssl.m4: Added.
* src/SocketCore.{h,cc}: Drop const quarifier from writeData(),
readData(), peekData(). Added gnutls support.
* src/HttpDownloadCommand.cc: Removed SleepCommand.h
* src/TrackerWatcherCommand.{h,cc}: Added. This command creates
TrackerInitCommand periodicaly(TorrentMan::minInterval).
* src/TorrentMan.cc: Remove downloadedSize == 0 check from
save().
Instead, added a check for whether setup method has executed
successfully.
* src/TorrentMan.h: Added member vaiable setupComplete. Updated
DEFAULT_ANNOUNCE_INTERVAL and DEFAULT_ANNOUNCE_MIN_INTERVAL to
300.
* src/Makefile.am: Updated.
* src/messageDigest.h: Added. This is a macro calculating SHA1
digest
using whether OpenSSL or gcrypt, depending on the result of
configure
script.
* src/ShaVisitor.{h,cc}: Removed direct dependency on OpenSSL by
using
messageDigest.h.
* src/TorrentAutoSaveCommand.h: Removed unused variable cuid.
* src/PeerListenCommand.cc: Added log about port binded
successfully.
Fixed memory leak.
* src/main.cc: Added gnutls support. Replaced LIB_SSL with
ENABLE_BITTORRENT where they are not related to OpenSSL but
BitTorrent.
Removed instantiation of TrackerInitCommand. Instead,
TrackerWatcherCommand is instantiated and pushed to the command
queue.
* src/InitiateConnectionCommandFactory.cc: Replaced HAVE_LIBSSL
with
ENABLE_SSL.
* src/Request.cc: Replaced HAVE_LIBSSL with ENABLE_SSL.
* src/RequestSlotMan.cc:
(deleteCompletedRequestSlot)
If a piece is already acquired by another command, delete the
request
slots for the piece.
* src/TrackerUpdateCommand.cc:
(execute)
Changed log level of MSG_TRACKER_WARNING_MESSAGE from info to
warn.
Added a check whether peer list is null.
Fixed the bug that causes sending completed event to the tracker
several times.
* src/TrackerInitCommand.cc:
(execute)
Fixed the bug that causes sending completed event to the tracker
several times.
* src/AbstractDiskWriter.{h,cc}: Removed direct dependency on
OpenSSL
by using messageDigest.h.
2006-03-26 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com>
* PeerConnection.cc: Replaced log message "keep-alive" with
"keep alive".
* PeerInteractionCommand.{h,cc}: Close connection if peer is
choking
localhost long time.
* TorrentMan.cc: When adding new peer with duplicate = true, if
the
number of peer list is equal to or grater than MAX_PEER_LIST,
delete
at most 100 failure entry from the list. If with duplicate =
false,
MAX_PEER_LIST is not checked.
* PeerListenCommand.cc: Fixed the argument order of log message.
2006-03-25 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com>
* Logger.h: Moved enum LEVEL from SimpleLogger.h to here.
Added warn().
* SimpleLogger.h: Moved enum LEVEL to Logger.h.
Implemented warn().
Defined 2 macros(WRITE_LOG, WRITE_LOG_EX) to avoid duplicated
code.
* ChunkedEncoding.cc: fixed memory leak.
* PeerInteractionCommand.cc: casted timeout value to long long
int.
* SleepCommand.cc: casted timeout value to long long int.
* Data.cc: fixed memory leak.
* Data.cc: fixed toLLInt().
* BitfieldMan.cc: fixed memory leak.
* TorrentMan.cc: initialized storeDir to ".".
* TorrentMan.cc: fixed memory leak.
* TorrentMan.cc: corrected file paths of splitted files.
* PeerAbstractCommand.cc: casted timeout to long long int.
* main.cc: added delete(req) and delete(te->diskWriter).
* RequestSlot.cc: casted timeout value to long long int.
* Request.cc: fixed memory leak.
* PendingMessage.cc: make HAVE messages sent only when peer does
not
have the piece.
* Peer.{h,cc}: added hasPiece(int index).
* Added SocketCore. Socket is now handle class for SocketCore.
* Fixed bug in ChunkedEncoding: expanding buffer size is wrong
* Fixed bug in DownloadCommand: In Chunked Encoding, it wrongly
adds to Segment.ds buff length from the socket.
Tatsuhiro Tsujikawa