Added direct I/O support. The current implementation uses O_DIRECT,
which is not posix standard and is tested on linux 2.6.21.
Currently only file allocation uses direct I/O.
* src/SingleFileAllocationIterator.{h, cc}
* test/SingleFileAllocationIteratorTest.cc
* src/MultiFileAllocationIterator.{h, cc}
* test/MultiFileAllocationIteratorTest.cc
* src/BinaryStream.h
* src/DiskWriter.h
* src/AbstractDiskWriter.{h, cc}
* src/ByteArrayDiskWriter.h
* src/DiskAdaptor.h
* src/AbstractSingleDiskAdaptor.{h, cc}
* src/MultiDiskAdaptor.{h, cc}
* src/FileAllocationEntry.cc
* src/Util.{h, cc}
* src/OptionHandlerFactory.cc
* src/prefs.h
* src/version_usage.cc
* src/option_processing.cc
Moved FileAllocationMan::markCurrentFileAllocationEntryDone() to
handleException.
* src/MultiFileAllocationIterator.cc
Added EINTR handling
* src/SocketCore.cc
Implemented BitTorrent/http/ftp integrated download.
I've rewritten lots of files and now some headers have forward
class declarations to reduce compile time.
The implementation is extremely alpha stage, I recommend to use this
for testing purpose only.
To add the ability to resume downloading a partially downloaded
file
which is downloaded from the beginning:
* src/FileAllocator.h: Made abstract class. New
DefaultFileAllocator
takes this role.
* src/main.cc: Added -c option.
* src/BitfieldMan.h, src/BitfieldMan.cc (setBitRange): New
function.
* src/DiskWriter.h (openExistingFile): Added totalLength
argument.
* src/prefs.h (PREF_CONTINUE): New definition.
* src/SegmentMan.h, src/SegmentMan.cc (markPieceDone): New
function.
* src/DefaultDiskWriter.cc
(createNewDiskWriter): Add GlowFileAllocator to the new object.
* src/AbstractDiskWriter.h
(glowFileAllocator): New variable.
* src/AbstractDiskWriter.cc
(openExistingFile): Now preallocate file space from the end of
the
existing file if totalLength argument is specified and grater
than 0.
* src/UrlRequestInfo.cc: If -c option is specified and the file
to download exists in local, continue the download of the file.
--allow-overwrite=true is assumed in this context.
* src/DefaultFileAllocator.h, src/DefaultFileAllocator.cc: New
class.
* src/GlowFileAllocator.h, src/GlowFileAllocator.cc: New class.
Throw exception if --check-integrity=true is specified but chunk
checksums are not provided:
* src/UrlRequestInfo.cc
* src/Xml2MetalinkProcessor.h
(xpathExists): New function.
* src/Xml2MetalinkProcessor.cc
(xpathExists): New function.
Not to send HEAD request if filename and size are available in
Metalink
file:
* src/UrlRequestInfo.h
(filename): New variable.
(totalLength): New variable.
(setTotalLength): New function.
(setFilename): New function.
* src/MetalinkRequestInfo.cc
(execute): Set filename and size to UrlRequestInfo.
* src/MetalinkEntry.cc
(MetalinkEntry): Initialize size with 0.
* src/UrlRequestInfo.cc
(execute): Set filename and size to SegmentMan.
Not to download rest of the files after selected files are
downloaded
in BitTorrent:
* src/PieceStorage.h
(allDownloadFinished): New function.
* src/DefaultBtAnnounce.cc
(isCompleteAnnounceReady): Use allDownloadFinished instead of
downloadFinished.
(getAnnounceUrl): Use allDownloadFinished instead of
downloadFinished.
* src/DefaultPieceStorage.cc
(completePiece): Use allDownloadFinished instead of
downloadFinished.
Commented out the call to finishSelectiveDownloadingMode().
(downloadFinished): Call isFilteredAllBitSet() instead of
isAllBitSet().
(allDownloadFinished): New function.
* src/DefaultBtInteractive.cc
(addBitfieldMessageToQueue): Call allDownloadFinished() instead
of
downloadFinished().
(checkHave): Call allDownloadFinished() instead of
downloadFinished().
* src/TorrentDownloadEngine.cc
(onEndOfRun): Call allDownloadFinished() instead of
downloadFinished().
* src/BitfieldMan.h
(isFilteredAllBitSet): New function.
* src/ShareRatioSeedCriteria.h
(PieceStorage.h): New include.
(pieceStorage): New variable.
(evaluate):
btContext->getTotalLength() ->
pieceStorage->getCompletedLength()
* src/BitfieldMan.cc
(isFilteredAllBitSet): New function.
(isAllBitSet): Filter is not took into account.
Rename --force-truncate as --allow-overwrite:
* src/TorrentRequestInfo.cc
(execute): PREF_FORCE_TRUNCATE -> PREF_ALLOW_OVERWRITE
* src/main.cc
(showUsage): --force-truncate -> --allow-overwrite
* src/message.h
(EX_FILE_ALREADY_EXISTS): --force-truncate -> --allow-overwrite
* src/prefs.h
(PREF_FORCE_TRUNCATE): Removed.
(PREF_ALLOW_OVERWRITE): New definition.
* src/SegmentMan.cc
(shouldCancelDownloadForSafety): --force-truncate ->
--allow-overwrite
* src/MetalinkRequestInfo.cc
(execute): Queueing message are now logged in info level.
* src/common.h
(LONG_LONG_MAX): Removed.
(LONG_LONG_MIN): Removed.
* src/HttpResponseCommand.cc
(handleDefaultEncoding): LONG_LONG_MAX -> INT64_MAX
* src/FtpNegotiateCommand.cc
(recvSize): LONG_LONG_MAX -> INT64_MAX
* src/main.cc
(showUsage): Added --check-integiry and
--realtime-chunk-checksum
command-line option.
(main): Added --check-integiry and --realtime-chunk-checksum
command-line option.
--force-truncate -> --allow-overwrite
Set initial value of --check-integrity option to false.
Don't show usage when error occurs while persing command-line
options.
Removed deprecated --upload-limit option.
To add chunk checksum validation:
* src/MetalinkEntry.h
(MetalinkChunkChecksum.h): New include.
(chunkChecksum): New variable.
* src/Request.h
(method): New variable.
(setMethod): New function.
(getMethod): New function.
(METHOD_GET): New static constant.
(METHOD_HEAD): New static constant.
* src/Xml2MetalinkProcessor.h
(getPieceHash): New function.
* src/PieceStorage.h
(markAllPiecesDone): New function.
(checkIntegrity): New function.
* src/FileAllocator.h
(NullFileAllocationMonitor.h): New include.
(FileAllocator): Initialize fileAllocationMonitor with new
NullFileAllocationMonitor().
* src/MultiDiskAdaptor.h
(messageDigest.h): Remove include.
(ctx): Removed.
(hashUpdate): Added ctx.
(MultiDiskAdaptor): Removed ctx.
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/UrlRequestInfo.h
(HeadResult): New class.
(digestAlgo): New variable.
(chunkChecksumLength): New variable.
(chunkChecksums): New variable.
(getHeadResult): New function.
(UrlRequestInfo): Added digestAlgo, chunkChecksumLength.
(setDigestAlgo): New function.
(setChunkChecksumLength): New function.
(setChunkChecksums): New function.
* src/DefaultPieceStorage.cc
(DiskAdaptorWriter.h): New include.
(ChunkChecksumValidator.h): New include.
(markAllPiecesDone): New function.
(checkIntegrity): New function.
* src/DefaultBtContext.h
(getPieceHashes): New function.
* src/TorrentRequestInfo.cc
(execute): Try to validate chunk checksum if file already exists
and
.aria2 file doesn't there and user allows aria2 to overwrite it.
* src/messageDigest.h
(~MessageDigestContext): Added digestFree().
* src/MetalinkRequestInfo.cc
(execute): Set digestAlgo, chunkChecksum, chunkChecksums to
reqInfo.
* src/DiskAdaptor.h
(messageDigest.h): New include.
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/DownloadCommand.h
(PeerStat.h): New include.
(maxDownloadSpeedLimit): New variable.
(startupIdleTime): New variable.
(lowestDownloadSpeedLimit): New variable.
(peerStat): New variable.
(setMaxDownloadSpeedLimit): New function.
(setStartupIdleTime): New function.
(setLowestDownloadSPeedLimit): New function.
* src/BtContext.h
(getPieceHashes): New function.
* src/main.cc
(main): Set PREF_REALTIME_CHUNK_CHECKSUM and
PREF_CHECK_INTEGRITY
option to true for testing purpose.
* src/BtPieceMessage.cc
(checkPieceHash): Use messageDigest
* src/DownloadEngine.cc
(SetDescriptor): Removed.
(AccumulateActiveCommand): Removed.
(waitData): Rewritten.
(updateFdSet): Rewritten.
* src/MultiDiskAdaptor.cc
(hashUpdate): Added ctx.
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/BitfieldMan.h
(isBitRangeSet): New function.
(unsetBitRange): New function.
* src/ByteArrayDiskWriter.h
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/ConsoleDownloadEngine.cc
(calculateStatistics): If nspeed < 0 then set nspeed to 0.
* src/DiskWriter.h
(messageDigest.h): New include.
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/ChunkChecksumValidator.h: New class.
* src/DiskAdaptorWriter.h: New class.
* src/prefs.h
(PREF_REALTIME_CHUNK_CHECKSUM): New definition.
(PREF_CHECK_INTEGRITY): New definition.
* src/HttpResponseCommand.cc
(handleDefaultEncoding): Added method "HEAD" handling.
Removed the call to
e->segmentMan->shouldCancelDownloadForSafety().
(handleOtherEncoding):
Added the call to
e->segmentMan->shouldCancelDownloadForSafety().
(createHttpDownloadCommand): Set maxDownloadSpeedLimit,
startupIdleTime, lowestDownloadSpeedLimit to command.
* src/SegmentMan.h
(getSegmentEntryByIndex): New function.
(getSegmentEntryByCuid): New function.
(getSegmentEntryIteratorByCuid): New function.
(diskWriter): DiskWriter -> DiskWriterHandle
(pieceHashes): New variable.
(chunkHashLength): New variable.
(digestAlgo): New variable.
(FindPeerStat): Removed.
(getPeerStat): Rewritten.
(markAllPiecesDone): New function.
(checkIntegrity): New function.
(tryChunkChecksumValidation): New function.
(isChunkChecksumValidationReady): New function.
* src/BitfieldMan.cc
(BitfieldMan): Initialized bitfieldLength, blocks to 0.
(BitfieldMan): Initialized blockLength, totalLength,
bitfieldLength,
blocks to 0.
(isBitRangeSet): New function.
(unsetBitRange): New function.
* src/FtpNegotiateCommand.cc
(executeInternal): Set maxDownloadSpeedLimit,
startupIdleTime, lowestDownloadSpeedLimit to command.
(recvSize): Added method "HEAD" handling.
Removed the call to
e->segmentMan->shouldCancelDownloadForSafety().
* src/AbstractSingleDiskAdaptor.cc
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/AbstractSingleDiskAdaptor.h
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/Util.h
(indexRange): New function.
* src/MetalinkEntry.cc
(MetalinkEntry): Initialized chunkChecksum to 0.
* src/ShaVisitor.cc
(~ShaVisitor): Removed the call to ctx.digestFree().
* src/SegmentMan.cc
(ChunkChecksumValidator.h): New include.
(SegmentMan): Initialized chunkHashLength to 0. Initialized
digestAlgo
to DIGEST_ALGO_SHA1.
(~SegmentMan): Removed diskWriter.
(FindSegmentEntryByIndex): Removed.
(FindSegmentEntryByCuid): Removed.
(checkoutSegment): Rewritten.
(findSlowerSegmentEntry): Rewritten.
(getSegment): Rewritten.
(updateSegment): Rewritten.
(completeSegment): Rewritten.
(markAllPiecesDone): New function.
(checkIntegrity): New function.
(isChunkChecksumValidationReady): New function.
(tryChunkChecksumValidation): New function.
* src/Xml2MetalinkProcessor.cc
(getEntry): Get size and set it to entry.
Get chunk checksum and set it to entry.
(getPieceHash): New function.
* src/Util.cc
(sha1Sum): Removed ctx.digestFree()
(fileChecksum): Removed ctx.digestFree()
(indexRange): New function.
* src/Request.cc
(METHOD_GET): New variable.
(METHOD_HEAD): New variable.
(Request): Added method.
* src/UrlRequestInfo.cc
(FatalException.h): New include.
(message.h): New include.
(operator<<): Added operator<< for class HeadResult.
(getHeadResult): New function.
(execute): Get filename and size in separate download engine.
* src/ChunkChecksumValidator.cc: New class.
* src/DownloadCommand.cc:
(DownloadCommand): Added peerStat.
(executeInternal): Use maxDownloadSpeedLimit member instead of
getting
the value from Option.
The buffer size is now 16KB.
Use peerStat member instead of getting it from SegmentMan.
Use startupIdleTime member instead of gettingit from Option.
Added chunk checksum validation.
* src/AbstractDiskWriter.cc
(AbstractDiskWriter): Removed ctx.
(~AbstractDiskWriter): Removed ctx.digestFree()
(writeDataInternal): Returns the return value of write.
(readDataInternal): Returns the return value of read.
(sha1Sum): Renamed as messageDigest
(messageDigest): New function.
* src/AbstractDiwkWriter.h
(messageDigest.h): Removed include.
(ctx): Removed.
(sha1Sum): Renamed as messageDigest
(messageDigest): New function.
* src/DefaultPieceStorage.h
(markAllPiecesDone): New function.
(checkIntegrity): New function.
* src/NullFileAllocationMonitor.h: New class.
To improve the conditional compilation:
* src/MultiDiskWriter.h:
Replaced ENABLE_SHA1DIGEST with ENABLE_MESSAGE_DIGEST.
* src/MultiDiskWriter.cc:
Replaced ENABLE_SHA1DIGEST with ENABLE_MESSAGE_DIGEST.
* src/Util.h:
Replaced ENABLE_SHA1DIGEST with ENABLE_MESSAGE_DIGEST.
Added ENABLE_BITTORRENT around computeFastSet().
* src/Util.cc:
Replaced ENABLE_SHA1DIGEST with ENABLE_MESSAGE_DIGEST.
Added ENABLE_BITTORRENT around computeFastSet().
* src/messageDigest.h:
Replaced ENABLE_BITTORRENT with ENABLE_SSL.
* src/ShaVisitor.h:
Replaced ENABLE_SHA1DIGEST with ENABLE_MESSAGE_DIGEST.
* src/ShaVisitor.cc:
Replaced ENABLE_SHA1DIGEST with ENABLE_MESSAGE_DIGEST.
* src/main.cc:
Added ENABLE_BITTORRENT around includes and blocks related to
BitTorrent.
* src/AbstractDiskWriter.h:
Replaced ENABLE_SHA1DIGEST with ENABLE_MESSAGE_DIGEST.
* src/AbstractDiskWriter.cc:
Replaced ENABLE_SHA1DIGEST with ENABLE_MESSAGE_DIGEST.
To add command-line options for Metalink:
* src/main.cc:
Added metalink-version, metalink-language, metalink-os,
follow-metalink.
To use EXIT_SUCCESS and EXIT_FAILURE with exit():
* src/main.cc: Use these definition.
To add the ability to download multi torrent into respective
files
directly:
* src/DiskWriter.h (openFile): New function.
(seek): Removed.
* src/MultiDiskWriter.h: New class.
* src/MultiDiskWriter.cc: New class.
* src/AbstractDiskWriter.h (seek): Changed its scope from public
to
protected.
(openFile): New function.
* src/AbstractDiskWriter.cc (openFile): New function.
* src/prefs.h (V_FALSE): New definition.
(PREF_DIRECT_FILE_MAPPING): New definition.
* src/TorrentMan.h (setupDiskWriter): New function.
(setAllMultiFileRequestedState): New function.
(onDownloadComplete): New function.
* src/TorrentMan.cc : Included MultiDiskWriter.h
(setupDiskWriter): New function.
(getFilePath): Updated.
(getTempFilePath): Updated.
(getSegmentFilePath): Updated.
(fixFilename): Updated.
(deleteTempFile): Updated.
(setAllMultiFileRequestedState): New function.
(setFileEntriesToDownload): Use setAllMultiFileRequestedState().
(finishPartialDownloadingMode): Reset requested flags.
(onDownloadComplete): New function.
* src/main.cc: Added --direct-file-mapping option.
Use TorretMan::setupDiskWriter().
* src/TorrentDownloadEngine.cc (afterEachIteration): Use
TorrentMan::
onDownloadComplete().
To fix ETA bug:
* src/Util.h (difftvsec): New function.
* src/Util.cc (difftvsec): New function.
* src/TorrentConsoleDownloadEngine.cc (calculateSpeed): Use int
for the
type of "elapsed" instead of long long int.
(calculateStatistics): Use Util::difftvsec instead of
Util::difftv.
The updates of statistics takes place every 1 seconds.
* src/TorrentConsoleDownloadEngine.h (lastElapsed): Changed its
type.
(calculateSpeed): Changed its argument signature.
* src/PeerMessage.cc (toString): Fixed message.
* configure.in: Added gnutls support. Added several CPP macros.
* m4/openssl.m4: Added.
* src/SocketCore.{h,cc}: Drop const quarifier from writeData(),
readData(), peekData(). Added gnutls support.
* src/HttpDownloadCommand.cc: Removed SleepCommand.h
* src/TrackerWatcherCommand.{h,cc}: Added. This command creates
TrackerInitCommand periodicaly(TorrentMan::minInterval).
* src/TorrentMan.cc: Remove downloadedSize == 0 check from
save().
Instead, added a check for whether setup method has executed
successfully.
* src/TorrentMan.h: Added member vaiable setupComplete. Updated
DEFAULT_ANNOUNCE_INTERVAL and DEFAULT_ANNOUNCE_MIN_INTERVAL to
300.
* src/Makefile.am: Updated.
* src/messageDigest.h: Added. This is a macro calculating SHA1
digest
using whether OpenSSL or gcrypt, depending on the result of
configure
script.
* src/ShaVisitor.{h,cc}: Removed direct dependency on OpenSSL by
using
messageDigest.h.
* src/TorrentAutoSaveCommand.h: Removed unused variable cuid.
* src/PeerListenCommand.cc: Added log about port binded
successfully.
Fixed memory leak.
* src/main.cc: Added gnutls support. Replaced LIB_SSL with
ENABLE_BITTORRENT where they are not related to OpenSSL but
BitTorrent.
Removed instantiation of TrackerInitCommand. Instead,
TrackerWatcherCommand is instantiated and pushed to the command
queue.
* src/InitiateConnectionCommandFactory.cc: Replaced HAVE_LIBSSL
with
ENABLE_SSL.
* src/Request.cc: Replaced HAVE_LIBSSL with ENABLE_SSL.
* src/RequestSlotMan.cc:
(deleteCompletedRequestSlot)
If a piece is already acquired by another command, delete the
request
slots for the piece.
* src/TrackerUpdateCommand.cc:
(execute)
Changed log level of MSG_TRACKER_WARNING_MESSAGE from info to
warn.
Added a check whether peer list is null.
Fixed the bug that causes sending completed event to the tracker
several times.
* src/TrackerInitCommand.cc:
(execute)
Fixed the bug that causes sending completed event to the tracker
several times.
* src/AbstractDiskWriter.{h,cc}: Removed direct dependency on
OpenSSL
by using messageDigest.h.
2006-03-26 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com>
* PeerConnection.cc: Replaced log message "keep-alive" with
"keep alive".
* PeerInteractionCommand.{h,cc}: Close connection if peer is
choking
localhost long time.
* TorrentMan.cc: When adding new peer with duplicate = true, if
the
number of peer list is equal to or grater than MAX_PEER_LIST,
delete
at most 100 failure entry from the list. If with duplicate =
false,
MAX_PEER_LIST is not checked.
* PeerListenCommand.cc: Fixed the argument order of log message.
2006-03-25 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com>
* Logger.h: Moved enum LEVEL from SimpleLogger.h to here.
Added warn().
* SimpleLogger.h: Moved enum LEVEL to Logger.h.
Implemented warn().
Defined 2 macros(WRITE_LOG, WRITE_LOG_EX) to avoid duplicated
code.
TorrentMan::unadvertisePiece in Destructor.
* PeerInteractionCommand.cc: make have message sent immediately
if the size of pending message queue is zero.
* TorrentMan.cc: set the maximum size of peer list to 250.
* TorrentMan.h: changed the container type of Peers and
UsedPieces
to deque.
* Util.cc: fixed rangedFileCopy.
* AbstractDiskWriter.{h,cc}: moved digest context initialization
to Constructor. Also, moved digest cleanup to Destructor.
* MetaFileUtil.cc: fixed memory leak
Tatsuhiro Tsujikawa