mirror of https://github.com/aria2/aria2
![]() To add chunk checksum validation: * src/MetalinkEntry.h (MetalinkChunkChecksum.h): New include. (chunkChecksum): New variable. * src/Request.h (method): New variable. (setMethod): New function. (getMethod): New function. (METHOD_GET): New static constant. (METHOD_HEAD): New static constant. * src/Xml2MetalinkProcessor.h (getPieceHash): New function. * src/PieceStorage.h (markAllPiecesDone): New function. (checkIntegrity): New function. * src/FileAllocator.h (NullFileAllocationMonitor.h): New include. (FileAllocator): Initialize fileAllocationMonitor with new NullFileAllocationMonitor(). * src/MultiDiskAdaptor.h (messageDigest.h): Remove include. (ctx): Removed. (hashUpdate): Added ctx. (MultiDiskAdaptor): Removed ctx. (sha1Sum): Renamed as messageDigest. (messageDigest): New function. * src/UrlRequestInfo.h (HeadResult): New class. (digestAlgo): New variable. (chunkChecksumLength): New variable. (chunkChecksums): New variable. (getHeadResult): New function. (UrlRequestInfo): Added digestAlgo, chunkChecksumLength. (setDigestAlgo): New function. (setChunkChecksumLength): New function. (setChunkChecksums): New function. * src/DefaultPieceStorage.cc (DiskAdaptorWriter.h): New include. (ChunkChecksumValidator.h): New include. (markAllPiecesDone): New function. (checkIntegrity): New function. * src/DefaultBtContext.h (getPieceHashes): New function. * src/TorrentRequestInfo.cc (execute): Try to validate chunk checksum if file already exists and .aria2 file doesn't there and user allows aria2 to overwrite it. * src/messageDigest.h (~MessageDigestContext): Added digestFree(). * src/MetalinkRequestInfo.cc (execute): Set digestAlgo, chunkChecksum, chunkChecksums to reqInfo. * src/DiskAdaptor.h (messageDigest.h): New include. (sha1Sum): Renamed as messageDigest. (messageDigest): New function. * src/DownloadCommand.h (PeerStat.h): New include. (maxDownloadSpeedLimit): New variable. (startupIdleTime): New variable. (lowestDownloadSpeedLimit): New variable. (peerStat): New variable. (setMaxDownloadSpeedLimit): New function. (setStartupIdleTime): New function. (setLowestDownloadSPeedLimit): New function. * src/BtContext.h (getPieceHashes): New function. * src/main.cc (main): Set PREF_REALTIME_CHUNK_CHECKSUM and PREF_CHECK_INTEGRITY option to true for testing purpose. * src/BtPieceMessage.cc (checkPieceHash): Use messageDigest * src/DownloadEngine.cc (SetDescriptor): Removed. (AccumulateActiveCommand): Removed. (waitData): Rewritten. (updateFdSet): Rewritten. * src/MultiDiskAdaptor.cc (hashUpdate): Added ctx. (sha1Sum): Renamed as messageDigest. (messageDigest): New function. * src/BitfieldMan.h (isBitRangeSet): New function. (unsetBitRange): New function. * src/ByteArrayDiskWriter.h (sha1Sum): Renamed as messageDigest. (messageDigest): New function. * src/ConsoleDownloadEngine.cc (calculateStatistics): If nspeed < 0 then set nspeed to 0. * src/DiskWriter.h (messageDigest.h): New include. (sha1Sum): Renamed as messageDigest. (messageDigest): New function. * src/ChunkChecksumValidator.h: New class. * src/DiskAdaptorWriter.h: New class. * src/prefs.h (PREF_REALTIME_CHUNK_CHECKSUM): New definition. (PREF_CHECK_INTEGRITY): New definition. * src/HttpResponseCommand.cc (handleDefaultEncoding): Added method "HEAD" handling. Removed the call to e->segmentMan->shouldCancelDownloadForSafety(). (handleOtherEncoding): Added the call to e->segmentMan->shouldCancelDownloadForSafety(). (createHttpDownloadCommand): Set maxDownloadSpeedLimit, startupIdleTime, lowestDownloadSpeedLimit to command. * src/SegmentMan.h (getSegmentEntryByIndex): New function. (getSegmentEntryByCuid): New function. (getSegmentEntryIteratorByCuid): New function. (diskWriter): DiskWriter -> DiskWriterHandle (pieceHashes): New variable. (chunkHashLength): New variable. (digestAlgo): New variable. (FindPeerStat): Removed. (getPeerStat): Rewritten. (markAllPiecesDone): New function. (checkIntegrity): New function. (tryChunkChecksumValidation): New function. (isChunkChecksumValidationReady): New function. * src/BitfieldMan.cc (BitfieldMan): Initialized bitfieldLength, blocks to 0. (BitfieldMan): Initialized blockLength, totalLength, bitfieldLength, blocks to 0. (isBitRangeSet): New function. (unsetBitRange): New function. * src/FtpNegotiateCommand.cc (executeInternal): Set maxDownloadSpeedLimit, startupIdleTime, lowestDownloadSpeedLimit to command. (recvSize): Added method "HEAD" handling. Removed the call to e->segmentMan->shouldCancelDownloadForSafety(). * src/AbstractSingleDiskAdaptor.cc (sha1Sum): Renamed as messageDigest. (messageDigest): New function. * src/AbstractSingleDiskAdaptor.h (sha1Sum): Renamed as messageDigest. (messageDigest): New function. * src/Util.h (indexRange): New function. * src/MetalinkEntry.cc (MetalinkEntry): Initialized chunkChecksum to 0. * src/ShaVisitor.cc (~ShaVisitor): Removed the call to ctx.digestFree(). * src/SegmentMan.cc (ChunkChecksumValidator.h): New include. (SegmentMan): Initialized chunkHashLength to 0. Initialized digestAlgo to DIGEST_ALGO_SHA1. (~SegmentMan): Removed diskWriter. (FindSegmentEntryByIndex): Removed. (FindSegmentEntryByCuid): Removed. (checkoutSegment): Rewritten. (findSlowerSegmentEntry): Rewritten. (getSegment): Rewritten. (updateSegment): Rewritten. (completeSegment): Rewritten. (markAllPiecesDone): New function. (checkIntegrity): New function. (isChunkChecksumValidationReady): New function. (tryChunkChecksumValidation): New function. * src/Xml2MetalinkProcessor.cc (getEntry): Get size and set it to entry. Get chunk checksum and set it to entry. (getPieceHash): New function. * src/Util.cc (sha1Sum): Removed ctx.digestFree() (fileChecksum): Removed ctx.digestFree() (indexRange): New function. * src/Request.cc (METHOD_GET): New variable. (METHOD_HEAD): New variable. (Request): Added method. * src/UrlRequestInfo.cc (FatalException.h): New include. (message.h): New include. (operator<<): Added operator<< for class HeadResult. (getHeadResult): New function. (execute): Get filename and size in separate download engine. * src/ChunkChecksumValidator.cc: New class. * src/DownloadCommand.cc: (DownloadCommand): Added peerStat. (executeInternal): Use maxDownloadSpeedLimit member instead of getting the value from Option. The buffer size is now 16KB. Use peerStat member instead of getting it from SegmentMan. Use startupIdleTime member instead of gettingit from Option. Added chunk checksum validation. * src/AbstractDiskWriter.cc (AbstractDiskWriter): Removed ctx. (~AbstractDiskWriter): Removed ctx.digestFree() (writeDataInternal): Returns the return value of write. (readDataInternal): Returns the return value of read. (sha1Sum): Renamed as messageDigest (messageDigest): New function. * src/AbstractDiwkWriter.h (messageDigest.h): Removed include. (ctx): Removed. (sha1Sum): Renamed as messageDigest (messageDigest): New function. * src/DefaultPieceStorage.h (markAllPiecesDone): New function. (checkIntegrity): New function. * src/NullFileAllocationMonitor.h: New class. |
||
---|---|---|
doc | ||
intl | ||
lib | ||
m4 | ||
po | ||
src | ||
test | ||
ABOUT-NLS | ||
AUTHORS | ||
COPYING | ||
ChangeLog | ||
INSTALL | ||
LICENSE.OpenSSL | ||
Makefile.am | ||
Makefile.in | ||
NEWS | ||
README | ||
TODO | ||
aclocal.m4 | ||
config.guess | ||
config.h.in | ||
config.rpath | ||
config.sub | ||
configure | ||
configure.ac | ||
depcomp | ||
install-sh | ||
missing | ||
mkinstalldirs | ||
stamp-h | ||
stamp-h.in |
README
aria2 - The high speed download utility 1. Disclaimer ------------- This program comes with no warranty. You must use this program at your own risk. 2. About aria2 -------------- aria2 has segmented downloading engine in its core. By segmented downloading, it can download files very much faster than ordinary browsers. aria2 is in very early development stage. Currently it has following features: * HTTP/HTTPS GET support * HTTP Proxy support * HTTP BASIC authentication support * HTTP Proxy authentication support * FTP support(active, passive mode) * FTP through HTTP proxy(GET command or tunneling) * Segmented download * Cookie support(currently aria2 ignores "expires") * It can run as a daemon process. * BitTorrent protocol support with fast extension. * Selective download in multi-file torrent * Metalink version 3.0 support(HTTP/FTP/BitTorrent). * Limiting download/upload speed. 3. How to build --------------- $ ./configure $ make The executable is aria2c in src directory. 4. Dependency ------------- In order to enable HTTPS support, you need GNU TLS or OpenSSL. In order to enable BitTorrent support, you need GNU TLS+libgcrypt or OpenSSL. In order to enable Metalink support, you need libxml2. Optionally GNU TLS+ libgcrypt or OpenSSL are required for checksum checking support(MD5, SHA1). GNU TLS has precedence over OpenSSL if both libraries are installed. If you prefer OpenSSL, run configure with "--without-gnutls". You can disable BitTorrent, Metalink support by providing --disable-bittorrent, --disable-metalink respectively to configure script. In order to enable async DNS support, you need c-ares or ares. c-ares: http://daniel.haxx.se/projects/c-ares/ ares: ftp://athena-dist.mit.edu/pub/ATHENA/ares 5. BitTorrrent -------------- The filename of the downloaded file is determined as follows: single-file mode: If "name" key is present in .torrent file, filename is the value of "name" key. Otherwise, filename is the basename of .torrent file appended by ".file". For example, .torrent file is "test.torrrent", then filename is "test.torrent.file". The directory to store the downloaded file can be specified by -d option. multi-file mode: The complete directory/file structure mentioned in .torrent file is created. The directory to store the top directory of downloaded files can be specified by -d option. In the default behavior, before download starts, complete directory structure is created if needed. Then aria2 opens all files mentioned in .torrent file, directly writes to and reads from these files. NOTE: Even in selective download, all files are opened. If "--direct-file-mapping" option set to be false, aria2 creates temporary file in the store directory. The length of this file is the sum of length of the files in .torrent file, so at least 2 times more disk space than the file size itself is required. Writing and reading is done against this file. After download completes, aria2 creates complete directory structure if needed, and copies whole file or a part of it to the destination. Note: * -o option is used to change the filename of downloaded .torrent file. * This version only supports compact peers list format. * The ports aria2c uses are 6881-6999. * The maximum number of peers is 55. * After selective download completes, aria2 is going to download rest of the files. 6. Metalink ----------- The current implementation supports HTTP/FTP/BitTorrent. BitTorrent always has precedence over HTTP/FTP. The other P2P protocols are ignored. For checksum checking, both MD5 and SHA1 are supported. If both values are provided, then aria2 uses SHA1. If checksum checking is failed, aria2 doesn't retry the download and just exits with non-zero return code. The supported user preferences are version, language and os.