Using off_t, at least, in DiskAdaptor layer is problematic because
torrent can contain under 2GiB files but total sum of those files may
exceed 2GiB limit, which makes off_t overflow in 32 bit system without
large file support. So we use int64_t in API. We'll check the file
length before download so that it does not exceed max off_t.
Removed MultiDiskAdaptor::mkdir() because directory structure is
created at AbstractDiskWriter::createFile().
* src/MultiDiskAdaptor.cc
* src/MultiDiskAdaptor.h
Added .cc file for classes/structs that only provided by header
file. Defined non-POD classes' ctor, dtor in .cc file. Moved
implementation code in header file to .cc file for major
classes/strucsts.
Fixed the bug that read-only file with wrong file size will
cause error because it cannot be truncated to the correct size.
Now if file size is different than the expected one, re-open
file in writable mode.
* src/AbstractSingleDiskAdaptor.cc
* src/AbstractSingleDiskAdaptor.h
* src/BtCheckIntegrityEntry.cc
* src/DiskAdaptor.h
* src/MultiDiskAdaptor.h
* src/RequestGroup.cc
Added --index-out option to specify each file path for torrent.
Here index shown in --show-files option is used to specify which
file path should be altered. For example, to change the file
path with index=2, use --index-out=2=aria2.tar.bz2. You can use
this option multiple times: --index-out=1=aria2.tar.bz2
--index-out=2=aria2-opt.tar.bz2. The short hand form -O is also
available. This option can be specified in -i list.
* src/DefaultBtContext.cc
* src/DefaultBtContext.h
* src/MultiDiskAdaptor.cc
* src/MultiDiskAdaptor.h
* src/MultiFileAllocationIterator.cc
* src/OptionHandlerFactory.cc
* src/OptionHandlerImpl.h
* src/Util.cc
* src/Util.h
* src/download_helper.cc
* src/prefs.cc
* src/prefs.h
* src/usage_text.h
* test/DefaultBtContextTest.cc
* test/MultiDiskAdaptorTest.cc
* test/MultiFileAllocationIteratorTest.cc
* test/UtilTest.cc
In BitTorrent, Open files in read-only mode for hash check.
After that, re-open file in read/write mode only when the
download is not finished.
* src/AbstractDiskWriter.cc
* src/AbstractDiskWriter.h
* src/AbstractSingleDiskAdaptor.cc
* src/AbstractSingleDiskAdaptor.h
* src/BtCheckIntegrityEntry.cc
* src/DiskAdaptor.h
* src/DiskWriter.h
* src/MultiDiskAdaptor.cc
* src/MultiDiskAdaptor.h
* src/RequestGroup.cc
Fixed the bug that the DiskWriter of the first FileEntry whose
`needsFileAllocation' property is false is not created
even if it shares a piece with next FileEntry which `requested'
property is true.
Fixed the bug that zero-length file is not created if pre file
allocation is not done.
* src/MultiDiskAdaptor.cc
* src/MultiDiskAdaptor.h
* test/MultiDiskAdaptorTest.cc
Implemented the ability to get timestamp from remote HTTP server
and
apply it to local file. To enable this feature, --remote-time
option
is added. No usage text has been written yet.
If several servers returns difference timestamp, then aria2 uses
latest
one.
* src/CopyDiskAdaptor.cc
* src/CopyDiskAdaptor.h
* src/DirectDiskAdaptor.cc
* src/DirectDiskAdaptor.h
* src/DiskAdaptor.h
* src/File.cc
* src/File.h
* src/HttpHeader.cc
* src/HttpHeader.h
* src/HttpResponse.cc
* src/HttpResponse.h
* src/HttpResponseCommand.cc
* src/HttpResponseCommand.h
* src/MultiDiskAdaptor.cc
* src/MultiDiskAdaptor.h
* src/OptionHandlerFactory.cc
* src/RequestGroup.cc
* src/RequestGroup.h
* src/RequestGroupMan.cc
* src/option_processing.cc
* src/prefs.cc
* src/prefs.h
* test/CopyDiskAdaptorTest.cc
* test/FileTest.cc
* test/Makefile.am
* test/Makefile.in
* test/MultiDiskAdaptorTest.cc
* test/TestUtil.cc
Fixed the bug that DiskWriterEntry is not created when its
FileEntry.isRequested() is false and it doesn't share a piece
with
other FileEntries that are requested. This bug causes
segmentation fault
in the end.
Fixed the bug that exception is thrown when
MultiDiskAdaptor::size() is
called if the number of file entries are greater than max open
files.
* src/AbstractSingleDiskAdaptor.cc
* src/AbstractSingleDiskAdaptor.h
* src/DiskAdaptor.h
* src/MultiDiskAdaptor.cc
* src/MultiDiskAdaptor.h
* test/MultiDiskAdaptorTest.cc
Limited the number files opened in each BitTorrent download. The
default
value is 100. The option to change this value will be added.
* src/MultiDiskAdaptor.cc
* src/MultiDiskAdaptor.h
* src/MultiFileAllocationIterator.cc
Removed repeated call of getTopDirPath(). Instead, call it once
and
cache the result.
* src/MultiDiskAdaptor.cc
* src/MultiDiskAdaptor.h
(mkdir, openFile, initAndOpenFile, openExistingFile)
Added direct I/O support. The current implementation uses O_DIRECT,
which is not posix standard and is tested on linux 2.6.21.
Currently only file allocation uses direct I/O.
* src/SingleFileAllocationIterator.{h, cc}
* test/SingleFileAllocationIteratorTest.cc
* src/MultiFileAllocationIterator.{h, cc}
* test/MultiFileAllocationIteratorTest.cc
* src/BinaryStream.h
* src/DiskWriter.h
* src/AbstractDiskWriter.{h, cc}
* src/ByteArrayDiskWriter.h
* src/DiskAdaptor.h
* src/AbstractSingleDiskAdaptor.{h, cc}
* src/MultiDiskAdaptor.{h, cc}
* src/FileAllocationEntry.cc
* src/Util.{h, cc}
* src/OptionHandlerFactory.cc
* src/prefs.h
* src/version_usage.cc
* src/option_processing.cc
Moved FileAllocationMan::markCurrentFileAllocationEntryDone() to
handleException.
* src/MultiFileAllocationIterator.cc
Added EINTR handling
* src/SocketCore.cc
Preallocate non-requested file which is adjacent forward to
requested
file('requested' files means the files given in --select-file
option)
if they share a same piece.
This fixes long pause in the file system which doesn't support
sparse
files like FAT32 while downloading.
* src/MultiFileAllocationIterator.{h, cc}
* test/MultiFileAllocationIteratorTest.cc
* src/FileEntry.{h, cc}
Removed unused _option.
* src/MultiDiskAdaptor.h
* test/MultiDiskAdaptorTest.cc
* src/DefaultPieceStorage.cc
Set the default value of --seed-ratio to 1.0.
If 0.0 is given, then seeding continues regardless of share
ratio.
* src/version_usage.cc
* src/option_processing.cc
* src/BtSetup.cc
* doc/aria2c.1.txt
* doc/aria2c.1
Fixed: Selective download is not working in BitTorrent
* src/RequestGroup.cc
Introduced Sequence class. Use this instead of
Util::unfoldRange()
* src/PieceStorage.h
* test/MockPieceStorage.h
* src/UnknownLengthPieceStorage.h
* src/DefaultPieceStorage.{h, cc}
* src/Metalink2RequestGroup.cc
* src/RequestGroup.cc
* src/Sequence.h
* test/SequenceTest.cc
* src/IntSequence.h
* src/message.h
* src/Util.{h, cc}
* test/UtilTest.cc
Added new function 'parse' to catch exception thrown by
subclass's
parseArg
* src/OptionHandler.h
* src/OptionParser.cc
* src/NameMatchOptionHandler.h
* src/OptionHandlerImpl.h
* test/OptionHandlerTest.cc
Added IntegerRangeOptionHandler. Used for --listen-port and
--select-file. Now --listen-port accepts range of port.
* src/OptionHandlerFactory.cc
* src/version_usage.cc
* src/OptionHandlerImpl.h
* src/option_processing.cc
* src/BtSetup.cc
* src/PeerListenCommand.{h, cc}
* doc/aria2c.1.txt
* doc/aria2c.1
Implemented operator< for Exception class to provide easy way to
print
exception stack trace.
* src/Exception.{h, cc}
* src/main.cc
* src/option_processing.cc
Implemented BitTorrent/http/ftp integrated download.
I've rewritten lots of files and now some headers have forward
class declarations to reduce compile time.
The implementation is extremely alpha stage, I recommend to use this
for testing purpose only.
To add chunk checksum validation:
* src/MetalinkEntry.h
(MetalinkChunkChecksum.h): New include.
(chunkChecksum): New variable.
* src/Request.h
(method): New variable.
(setMethod): New function.
(getMethod): New function.
(METHOD_GET): New static constant.
(METHOD_HEAD): New static constant.
* src/Xml2MetalinkProcessor.h
(getPieceHash): New function.
* src/PieceStorage.h
(markAllPiecesDone): New function.
(checkIntegrity): New function.
* src/FileAllocator.h
(NullFileAllocationMonitor.h): New include.
(FileAllocator): Initialize fileAllocationMonitor with new
NullFileAllocationMonitor().
* src/MultiDiskAdaptor.h
(messageDigest.h): Remove include.
(ctx): Removed.
(hashUpdate): Added ctx.
(MultiDiskAdaptor): Removed ctx.
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/UrlRequestInfo.h
(HeadResult): New class.
(digestAlgo): New variable.
(chunkChecksumLength): New variable.
(chunkChecksums): New variable.
(getHeadResult): New function.
(UrlRequestInfo): Added digestAlgo, chunkChecksumLength.
(setDigestAlgo): New function.
(setChunkChecksumLength): New function.
(setChunkChecksums): New function.
* src/DefaultPieceStorage.cc
(DiskAdaptorWriter.h): New include.
(ChunkChecksumValidator.h): New include.
(markAllPiecesDone): New function.
(checkIntegrity): New function.
* src/DefaultBtContext.h
(getPieceHashes): New function.
* src/TorrentRequestInfo.cc
(execute): Try to validate chunk checksum if file already exists
and
.aria2 file doesn't there and user allows aria2 to overwrite it.
* src/messageDigest.h
(~MessageDigestContext): Added digestFree().
* src/MetalinkRequestInfo.cc
(execute): Set digestAlgo, chunkChecksum, chunkChecksums to
reqInfo.
* src/DiskAdaptor.h
(messageDigest.h): New include.
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/DownloadCommand.h
(PeerStat.h): New include.
(maxDownloadSpeedLimit): New variable.
(startupIdleTime): New variable.
(lowestDownloadSpeedLimit): New variable.
(peerStat): New variable.
(setMaxDownloadSpeedLimit): New function.
(setStartupIdleTime): New function.
(setLowestDownloadSPeedLimit): New function.
* src/BtContext.h
(getPieceHashes): New function.
* src/main.cc
(main): Set PREF_REALTIME_CHUNK_CHECKSUM and
PREF_CHECK_INTEGRITY
option to true for testing purpose.
* src/BtPieceMessage.cc
(checkPieceHash): Use messageDigest
* src/DownloadEngine.cc
(SetDescriptor): Removed.
(AccumulateActiveCommand): Removed.
(waitData): Rewritten.
(updateFdSet): Rewritten.
* src/MultiDiskAdaptor.cc
(hashUpdate): Added ctx.
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/BitfieldMan.h
(isBitRangeSet): New function.
(unsetBitRange): New function.
* src/ByteArrayDiskWriter.h
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/ConsoleDownloadEngine.cc
(calculateStatistics): If nspeed < 0 then set nspeed to 0.
* src/DiskWriter.h
(messageDigest.h): New include.
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/ChunkChecksumValidator.h: New class.
* src/DiskAdaptorWriter.h: New class.
* src/prefs.h
(PREF_REALTIME_CHUNK_CHECKSUM): New definition.
(PREF_CHECK_INTEGRITY): New definition.
* src/HttpResponseCommand.cc
(handleDefaultEncoding): Added method "HEAD" handling.
Removed the call to
e->segmentMan->shouldCancelDownloadForSafety().
(handleOtherEncoding):
Added the call to
e->segmentMan->shouldCancelDownloadForSafety().
(createHttpDownloadCommand): Set maxDownloadSpeedLimit,
startupIdleTime, lowestDownloadSpeedLimit to command.
* src/SegmentMan.h
(getSegmentEntryByIndex): New function.
(getSegmentEntryByCuid): New function.
(getSegmentEntryIteratorByCuid): New function.
(diskWriter): DiskWriter -> DiskWriterHandle
(pieceHashes): New variable.
(chunkHashLength): New variable.
(digestAlgo): New variable.
(FindPeerStat): Removed.
(getPeerStat): Rewritten.
(markAllPiecesDone): New function.
(checkIntegrity): New function.
(tryChunkChecksumValidation): New function.
(isChunkChecksumValidationReady): New function.
* src/BitfieldMan.cc
(BitfieldMan): Initialized bitfieldLength, blocks to 0.
(BitfieldMan): Initialized blockLength, totalLength,
bitfieldLength,
blocks to 0.
(isBitRangeSet): New function.
(unsetBitRange): New function.
* src/FtpNegotiateCommand.cc
(executeInternal): Set maxDownloadSpeedLimit,
startupIdleTime, lowestDownloadSpeedLimit to command.
(recvSize): Added method "HEAD" handling.
Removed the call to
e->segmentMan->shouldCancelDownloadForSafety().
* src/AbstractSingleDiskAdaptor.cc
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/AbstractSingleDiskAdaptor.h
(sha1Sum): Renamed as messageDigest.
(messageDigest): New function.
* src/Util.h
(indexRange): New function.
* src/MetalinkEntry.cc
(MetalinkEntry): Initialized chunkChecksum to 0.
* src/ShaVisitor.cc
(~ShaVisitor): Removed the call to ctx.digestFree().
* src/SegmentMan.cc
(ChunkChecksumValidator.h): New include.
(SegmentMan): Initialized chunkHashLength to 0. Initialized
digestAlgo
to DIGEST_ALGO_SHA1.
(~SegmentMan): Removed diskWriter.
(FindSegmentEntryByIndex): Removed.
(FindSegmentEntryByCuid): Removed.
(checkoutSegment): Rewritten.
(findSlowerSegmentEntry): Rewritten.
(getSegment): Rewritten.
(updateSegment): Rewritten.
(completeSegment): Rewritten.
(markAllPiecesDone): New function.
(checkIntegrity): New function.
(isChunkChecksumValidationReady): New function.
(tryChunkChecksumValidation): New function.
* src/Xml2MetalinkProcessor.cc
(getEntry): Get size and set it to entry.
Get chunk checksum and set it to entry.
(getPieceHash): New function.
* src/Util.cc
(sha1Sum): Removed ctx.digestFree()
(fileChecksum): Removed ctx.digestFree()
(indexRange): New function.
* src/Request.cc
(METHOD_GET): New variable.
(METHOD_HEAD): New variable.
(Request): Added method.
* src/UrlRequestInfo.cc
(FatalException.h): New include.
(message.h): New include.
(operator<<): Added operator<< for class HeadResult.
(getHeadResult): New function.
(execute): Get filename and size in separate download engine.
* src/ChunkChecksumValidator.cc: New class.
* src/DownloadCommand.cc:
(DownloadCommand): Added peerStat.
(executeInternal): Use maxDownloadSpeedLimit member instead of
getting
the value from Option.
The buffer size is now 16KB.
Use peerStat member instead of getting it from SegmentMan.
Use startupIdleTime member instead of gettingit from Option.
Added chunk checksum validation.
* src/AbstractDiskWriter.cc
(AbstractDiskWriter): Removed ctx.
(~AbstractDiskWriter): Removed ctx.digestFree()
(writeDataInternal): Returns the return value of write.
(readDataInternal): Returns the return value of read.
(sha1Sum): Renamed as messageDigest
(messageDigest): New function.
* src/AbstractDiwkWriter.h
(messageDigest.h): Removed include.
(ctx): Removed.
(sha1Sum): Renamed as messageDigest
(messageDigest): New function.
* src/DefaultPieceStorage.h
(markAllPiecesDone): New function.
(checkIntegrity): New function.
* src/NullFileAllocationMonitor.h: New class.
Tatsuhiro Tsujikawa