Implemented BitTorrent/http/ftp integrated download.
I've rewritten lots of files and now some headers have forward
class declarations to reduce compile time.
The implementation is extremely alpha stage, I recommend to use this
for testing purpose only.
Merged Ross's win32 patch(manually)
In the course of merging, following files are added.
* src/strptime.{h,c}
* src/libgen.{h,c}
* src/inet_aton.{h,c}
* src/Platform.{h,cc}
* src/localtime_r.{h,c}
* src/getaddrinfo.{h,c}
I've gethered network related things, and put them to following
file:
* src/a2netcompat.h
Also io related things are put to following file:
* src/a2io.h
Changed %lld to %s because mingw32 doesn't recognize %lld.
* src/message.h
(MSG_ALLOCATION_COMPLETED)
(EX_TOO_LARGE_FILE)
(EX_SIZE_MISMATCH)
(EX_FILE_OFFSET_OUT_OF_RANGE)
(EX_INVALID_CHUNK_CHECKSUM)
(EX_INVALID_RANGE_HEADER)
* src/FileAllocationCommand.cc
* src/HttpResponse.cc
* src/RequestGroup.cc
* src/MultiDiskAdaptor.cc
* src/OptionHandlerImpl.h
* src/HttpResponseCommand.cc
* src/FtpNegotiateCommand.cc
* src/IteratableChecksumValidator.cc
* src/SegmentMan.cc
* src/ChunkChecksumValidator.cc
Added Randomizer::getRandomNumber(long int)
* src/Randomizer.h (getRandomNumber)
* src/SimpleRandomizer.h (getRandomNumber)
* src/BitfieldMan.cc (getMissingIndexRandomly): Use this new
function.
* src/Util.cc (randomAlpha): Use this new function.
Made -S option work with metalink file and provided selective
download
to metalink.
* src/MetalinkEntry.h, src/MetalinkEntry.cc
(filename): Removed.
(file): New variable.
(size): Removed.
(operator=): Updated.
(getPath): New function.
(getLength): New function.
(toFileEntry): New function.
* src/TorrentRequestInfo.h, src/TorrentRequestInfo.cc
(execute): Use toStream.
(showFileEntry): Removed.
* src/MetalinkRequestInfo.h
(targetFiles): New variable.
(setTargetFiles): New variable.
* src/MetalinkRequestInfo.cc
(execute): Added the ability to print file information included
in
a metalink file.
Added selective download mode to metalink.
* src/main.cc
(showUsage): Updated to denote that -S and --select-file options
are
applicable to metalink.
* src/FileEntry.h
(operator=): New function.
(getBasename): New function.
(getDirname): New function.
* src/Util.h, src/Util.cc
(toStream): New function.
* src/Xml2MetalinkProcessor.cc: Updated.
Made aria2 work with metalink with directory structure.
* src/File.h, src/File.cc
(getBasename): New function.
(getDirname): New function.
* src/RequestGroup.h, src/RequestGroup.cc
(_topDir): New variable.
(setTopDir): New function.
(initSegmentMan): A directory structure is added to
_segmentMan->dir.
Rewrote HTTP header parsing with stringstream.
* src/HttpConnection.h, src/HttpConnection.cc
(HttpRequestEntry): New class.
(headerBuf): Removed.
(headerBufLength): Removed.
(outstandingHttpRequests): Now its element type is
HttpRequestEntryHandle.
(findEndOfHeader): Removed.
(receiveResponse): Rewritten.
Updated doc for -j option to notice that it should be used with
-i
option.
* src/main.cc (showUsage)
Removed unused classes.
* src/RequestInfo.h
(FileInfo): Removed.
(checksum): Removed.
(fileInfo): Removed.
(setChecksum): Removed.
(getChecksum): Removed.
(getFileInfo): Removed.
Use ISO units.
* src/ConsoleDownloadEngine.cc
* src/TorrentConsoleDownloadEngine.cc
* src/Util.cc (abbrevSize)
To the ability to read options from a config file:
* src/main.cc: Command-line parameter validation is delegated to
OptionHandler class.
* src/OptionHandlerFactory.h, src/OptionHandlerFactory.cc: New
class.
* src/Option.h, src/Option.cc (clear): New function.
* src/OptionParser.h, src/OptionParser.cc: New class.
* src/OptionHandler.h: New class.
* src/NameMatchOptionHandler.h: New class.
* src/OptionHandlerImpl.h: New classes.
* src/prefs.h: '_' -> '-'
(FTP_PASV_ENABLED): Renamed to FTP_PASV.
(FTP_PASV): New definition.
* src/Util.h, src/Util.cc (getRealSize): New function.
To disable netrc support if .netrc file does not have correct
permissions:
* src/File.h, src/File.cc (mode): New function.
To prevent confidential information to be logged:
* src/HttpConnection.h, src/HttpConnection.cc
(eraseConfidentialInfo): New function.
(sendRequest): Call eraseConfidentialInfo().
(sendProxyRequest): Call eraseConfidentialInfo().
* src/main.cc: Validate permissions of .netrc file.
To add --user-agent command-line option:
* src/main.cc: Added new command line option: --user-agent
* src/prefs.h (PREF_USER_AGENT): New definition.
* src/HttpRequestCommand.cc (executeInternal): Set user-agent
option
parameter to HttpRequest object.
Marged the patches from Dan Fandrich.
To handle Segment as SegmentHandle:
* src/AbstractCommand.cc (execute): Rewritten.
* src/SegmentMan.h: Segment -> SegmentHandle
Introducded HttpResponse class, HttpRequest class to improve
code
extensiveness and make it clear:
* src/HttpDownloadCommand.cc: transfer encoders are now managed
by
HttpResponse class.
* src/HttpRequest.h, src/HttpRequest.cc: New class.
* src/HttpResponse.h, src/HttpResponse.cc: New class.
* src/HttpConnection.cc: Contruction of http request were moved
to
HttpRequest class.
* src/HttpResponseCommand.h, src/HttpResponseCommand.cc:
Refactored.
* src/HttpRequestCommand.cc (executeInternal): Rewritten.
* src/HttpAuthConfig.h: New class.
* src/Range.h: New class.
To make FtpTunnel{Request, Response}Command and
HttpProxy{Request, Response}Command derived from
AbstractProxy{Request, Response}Command:
* src/FtpTunnelResponseCommand.h,
src/FtpTunnelResponseCommand.cc:
Derived from AbstractProxyRequestCommand class.
* src/FtpTunnelRequestCommand.h, src/FtpTunnelRequestCommand.cc:
Derived from AbstractProxyResponseCommand class.
* src/HttpProxyRequestCommand.h, src/HttpProxyRequestCommand.cc:
Derived from AbstractProxyRequestCommand class.
* src/HttpProxyResponseCommand.h,
src/HttpProxyResponseCommand.cc:
Derived from AbstractProxyResponseCommand class.
* src/AbstractProxyRequestCommand.h,
src/AbstractProxyRequestCommand.cc
: New class.
* src/AbstractProxyResponseCommand.h,
src/AbstractProxyResponseCommand.cc: New class.
To add netrc support:
* src/Netrc.h, src/Netrc.cc: New class.
* src/Util.h, src/Util.cc (split): New function.
* src/HttpHeader.cc (getRange): Fixed so that it inspects
"Content-Range" header instead of "Range" header.
* src/HttpHeader.h
(getStatus): Removed.
(setStatus): Removed.
* src/Segment.h
(getPositionToWrite): New function.
* src/HttpHeader.h
(Range.h): New include.
(status): New variable.
(HttpHeader): Initialized status with 0.
(getStatus): New function.
(setStatus): New function.
(getRange): New function.
(HttpHeaderHandle): New function.
* src/HttpHeader.cc
(getRange): New function.
* src/Request.h
(RequestWeakHandle): New definition.
* src/HttpConnection.h
(HttpConnectionHandle): New type definition.
* src/HttpConnection.cc
(receiveResponse): Set HTTP status to headers.
* src/main.cc
(showUsage): Fixed typo.
* src/Segment.h
(SegmentHandle): New type definition.
* src/BitfieldMan.h
(getMissingUnusedLength): New function.
* src/BitfieldMan.cc
(getMissingUnusedLength): New function.
* src/SharedHandle.h: New class.
To wrap Socket, Command, PeerMessage and Peer with SharedHandle:
* src/HttpResponseCommand.h
(HttpResponseCommand): Wrapped Socket.
* src/SocketCore.h
(operator==): New function.
(operator!=): New function.
(operator<): New function.
(getSockfd): New function.
(isOpen): New function.
(writeData): New function.
* src/SocketCore.cc
(operator==): New function.
(operator!=): New function.
(operator<): New function.
* src/AbstractCommand.h
(socket): Changed its type to SocketHandle.
(setReadCheckSocket): Replaced Socket with SocketHandle.
(setWriteCheckSocket): Replaced Socket with SocketHandle.
(disableReadCheckSocket): New function.
(disableWriteCheckSocket): New function.
(readCheckTarget): Changed its type to SocketHandle.
(writeCheckTarget): Changed its type to SocketHandle.
(AbstractCommand): Replaced Socket with SocketHandle.
* src/AbstractCommand.cc
(AbstractCommand): Replaced Socket with SocketHandle.
(~AbstractCommand): Removed the deallocation for Socket object.
(disableReadCheckSocket): New function.
(setReadCheckSocket): Replaced Socket with SocketHandle.
(disableWriteCheckSocket): New function.
(setWriteCheckSocket): Replaced Socket with SocketHandle.
* src/HttpDownloadCommand.cc
(DownloadCommand): Replaced Socket with SocketHandle.
* src/PeerAbstractCommand.h
(socket): Changed its type to SocketHandle.
(peer): Changed its type to PeerHandle.
(setReadCheckSocket): Replaced Socket with SocketHandle.
(setWriteCheckSocket): Replaced Socket with SocketHandle.
(disableReadCheckSocket): New function.
(disableWriteCheckSocket): New function.
(readCheckTarget): Changed its type to SocketHandle.
(writeCheckTarget): Changed its type to SocketHandle.
(PeerAbstractCommand): Replaced Socket with SocketHandle.
Replaced Peer with PeerHandle.
* src/HttpRequestCommand.cc
(HttpRequestCommand): Replaced Socket with SocketHandle.
Use disableReadCheckSocket.
* src/PeerInitiateConnectionCommand.h
(PeerInitiateConnectionCommand): Replaced Peer with PeerHandle.
* src/PeerChokeCommand.cc
(UploadFaster::operator()): Replaced Peer with PeerHandle.
(DownloadFaster::operator()): Replaced Peer with PeerHandle.
(execute): Use PeerHandle.
* src/PeerConnection.h
(HandshakeMessage.h): Removed include of HandshakeMessage.h.
(socket): Changed its type to SocketHandle.
(PeerConnection): Replaced Socket with SocketHandle.
* src/PeerConnection.cc
(PeerConnection): Replaced Socket with SocketHandle.
* src/PeerInteractionCommand.h
(PeerInteractionCommand): Replaced socket with SocketHandle.
Replaced Peer with PeerHandle.
* src/PeerInteractionCommand.cc
(PeerInteractionCommand): Replaced Socket with SocketHandle.
Replaced Peer with PeerHandle.
(executeInternal): Use disableWriteCheckSocket.
Use HandshakeMessageHandle.
(receiveMessages): Use PeerMessageHandle.
(prepareForNextPeer): Use PeerHandle.
* src/HttpProxyRequestCommand.h
(HttpProxyRequestCommand): Replaced Socket with SocketHandle.
* src/HttpResponseCommand.cc
(HttpResponseCommand): Replaced Socket with SocketHandle.
* src/TorrentMan.cc
(nullPeer): Added external reference.
(~TorrentMan): Removed the deallocation of the elements of
peers.
(addPeer): Rewritten.
(isPeerAvailable): Use nullPeer.
(deleteOldpeers): Replaced with deleteErrorPeer.
(deleteErrorPeer): New function.
(getPeer): Use PeerHandle and nullPeer.
(hasMissingPiece): Replaced Peer with PeerHandle.
(getMissingPieceIndex): Replaced Peer with PeerHandle.
(getMissingFastPieceIndex): Replaced Peer with PeerHandle.
(getMissingFastPiece): Replaced Peer with PeerHandle.
(getMissingPiece): Replaced Peer with PeerHandle.
* src/FtpNegotiateCommand.cc
(FtpNegotiationCommand): Replaced Peer with PeerHandle.
(~FtpNegotiationCommand): Removed the deallocation of Sockets.
(recvGreeting): Use disableWriteCheckSocket.
(recvPasv): Removed the allocation of Socket.
Use disableReadCheckSocket.
(sendRestPasv): Use disableWriteCheckSocket.
(recvRetr): Changed assertion.
* src/PeerInteraction.h
(SharedHandle.h): Included SharedHandle.h.
(PeerMessageHandle): New type definition.
(HandshakeMessageHandle): New type definition.
(MessageQueue): Changed. Now its element is of type
PeerMessageHandle.
(peer): Changed its type to PeerHandle.
(createHandshakeMessage): Replaced HandshakeMessage with
HandshakeMessageHandle.
(createPeerMessage): Replaced PeerMessageHandle with
PeerMessage.
(PeerInteraction): Replaced Peer with PeerHandle.
Replaced Socket with SocketHandle.
(addMessage): Replaced PeerMessage with PeerMessageHandle.
(receiveMessage): Replaced PeerMessage with PeerMessageHandle.
(receiveHandshake): Replaced HandshakeMessage with
HandshakeMessageHandle.
* src/PeerInteraction.cc
(PeerInteraction): Replaced Peer with PeerHandle.
Replaced Socket with SocketHandle.
(~PeerInteraction): Removed the deallocation of the elements of
messageQueue.
(MsgPushBack::operator()): Replaced PeerMessage with
PeerMessageHandle.
(isSendingMessageInProgress): Replaced PeerMessage with
PeerMessageHandle.
(sendMessages): Use PeerMessageHandle. Removed try-catch block.
(addMessage): Replaced PeerMessage with PeerMessageHandle.
(rejectAllPieceMessageInQueue): Use PeerMessageHandle.
(rejectPieceMessageInQueue): Use PeerMessageHandle.
(abortPiece): Use PeerMessageHandle.
(receiveHandshake): Replaced HandshakeMessage with
HandshakeMessageHandle. Removed try-catch block.
(createHandshakeMessage): Replaced HandshakeMessage with
HandshakeMessageHandle.
(receiveMessage): Replaced PeerMessage with PeerMessageHandle.
Removed try-catch block.
(createPeerMessage): Replaced PeerMessage with
PeerMessageHandle.
* src/HttpProxyResponseCommand.cc
(HttpProxyRequestCommand): Replaced Socket with SocketHandle.
* src/FtpTunnelResponseCommand.h
(FtpTunnelResponseCommand): Replaced Socket with SocketHandle.
* src/HttpConnection.cc
(HttpConnection): Replaced Socket with SocketHandle.
* src/PeerAbstractCommand.cc
(PeerAbstractCommand): Replaced Socket with SocketHandle.
(~PeerAbstractCommand): Removed the deallocation of socket.
Use disableReadCheckSocket, disableWriteCheckSocket.
(disableReadCheckSocket): New function.
(setReadCheckSocket): Replaced Socket with SocketHandle.
(disableWriteCheckSocket): New function.
(setWriteCheckSocket): Replaced Socket with SocketHandle.
* src/InitiateConnectionCommandFactory.h: Corrected indentation.
* src/FtpTunnelRequestCommand.cc
(FtpTunnelRequestCommand): Replaced Socket with SocketHandle.
(~FtpTunnelRequestCommand): Corrected indentation.
* src/DownloadCommand.h
(DownloadCommand): Replaced Socket with SocketHandle.
* src/PeerListenCommand.cc
(PeerListenCommand): Removed the initialization of socket.
(~PeerListenCommand): Removed the deallocation of socket.
(bindPort): Use SocketHandle.
(execute): Use SocketHandle and PeerHandle.
* src/FtpDowndloadCommand.cc
(FtpDownloadCommand): Replaced Socket with SocketHandle.
(~FtpDownloadCommand): Removed the deallocation of ctrlSocket.
* src/main.cc
(main): Corrected indentation.
* src/HttpInitiateConnectionCommand.cc
(HttpInitiateConnectionCommand): Replaced Socket with
SocketHandle.
(executeInternal): Removed the allocation of socket.
* src/HttpRequestCommand.h
(HttpRequestCommand): Replaced Socket with SocketHandle.
* src/FtpNegotiationCommand.h
(dataSocket): Changed its type to SocketHandle.
(serverSocket): Changed its type to SocketHandle.
(FtpNegotiationCommand): Replaced Socket with SocketHandle.
* src/TorrentMan.h
(MAX_PEER_UPDATE): Removed.
(MAX_PEERS): New definition.
(Peers): The element is now of type PeerHandle.
(addPeer): Replaced Peer with PeerHandle. Removed 'duplicate'
argument.
(getPeer): Replaced Peer with PeerHandle.
(deleteOldErrorPeers): Removed.
(deleteErrorPeer): New function.
(hasMissingPiece): Replaced Peer with PeerHandle.
(getMissingPieceIndex): Replaced Peer with PeerHandle.
(getMissingPiece): Replaced Peer with PeerHandle.
(getMissingFastPieceIndex): Replaced Peer with PeerHandle.
(getMissingFastPiece): Replaced Peer with PeerHandle.
(addActivePeer): Replaced Peer with PeerHandle.
(deleteActivePeer): Replaced Peer with PeerHandle.
Added a check for the return value of find.
* src/FtpTunnelResponseCommand.cc
(FtpTunnelResponseCommand): Replaced Socket with SocketHandle.
* src/FtpInitiateConnectionCommand.cc
(executeInternal): Removed the allocation of socket.
* src/DownloadEngine.h
(Sockets): An element is now of type SocketHandle.
(SockCmdMap): A key is of type SocketHandle, a value is of type
int.
(CommandUuids): New type definition.
(rsockets): Changed its type to SockCmdMap.
(wsockets): Changed its type to SockCmdMap.
(addSocket): Rewritten.
(deleteSocket): Rewritten.
(addSocketForReadCheck): Rewritten.
(deleteSocketForReadCheck): Rewritten.
(addSocketForWriteCheck): Rewritten.
(deleteSocketForWriteCheck): Rewritten.
(PairFind): New template class.
* src/HttpDownloadCommand.h
(HttpDownloadCommand): Replaced Socket with SocketHandle.
* src/FtpConnection.cc
(FtpConnection): Replaced Socket with SocketHandle.
(sendPort): Removed the allocation of serverSocket. Removed
try-catch
block.
* src/InitiateConnectionCommandFactory.cc
(DlAbortEx.h): Included DlAbortEx.h.
(createInitiateConnectionCommand): Throw exception if the
protocol of
requested URI is not supported.
* src/Peer.cc
(nullPeer): Changed its type to PeerHandle.
(operator==): New function.
(operator!=): New function.
* src/Peer.h
(SharedHandle.h): Included SharedHandle.h.
(operator==): New function.
(operator!=): New function.
(Peer): Added the default constructor.
Use resetStatus() to initialize member variables.
(nullPeer): Removed.
* src/TrackerUpdateCommand.cc
(execute): Brushed up using SharedHandle. Replaced
MAX_PEER_UPDATE
with MIN_PEERS.
* src/PeerListenCommand.h
(socket): Changed its type to SocketHandle.
* src/Command.h
(CommandUuid): New type definition.
(uuid): New variable.
(uuidGen): New variable.
(Command): Added the initialization of uuid.
(getUuid): New function.
* src/Socket.h
(Socket): Removed.
(SocketHandle): New type definition.
* src/DownloadEngine.h
(FindCommand): New function object.
(run): The portion of socket check was rewritten.
(SetDescriptor): New function object.
(AccumulateActiveCommandUuid): New function object.
(waitData): Rewritten.
(addSocket): Rewritten.
(deleteSocket): Rewritten.
(addSocketForReadCheck): Rewritten.
(addSocketForWriteCheck): Rewritten.
(deleteSocketForReadCheck): Rewritten.
(deleteSocketForWriteCheck): Rewritten.
* src/HttpProxyResponseCommand.h
(HttpProxyResponseCommand): Replaced Socket with SocketHandle.
* src/HttpConnection.h
(socket): Changed its type to SocketHandle.
(HttpConnection): Replaced Socket with SocketHandle.
* src/PeerInitiateConnectionCommand.cc
(PeerInitiateConnectionCommand): Replaced Peer with PeerHandle.
(executeInternal): Removed the allocation of socket.
(prepareForNextPeer): Use PeerHandle.
* src/PeerMessage.h
(peer): Changed its type to PeerHandle.
(getPeer): Replaced Peer with PeerHandle.
(setPeer): Replaced Peer with PeerHandle.
* src/DownloadCommand.cc
(DownloadCommand): Replaced Socket with SocketHandle.
* src/FtpConnection.h
(socket): Changed its type to SocketHandle.
(FtpConnection): Replaced Socket with SocketHandle.
(sendPort); Replaced Socket with SocketHandle.
* src/FtpDowndloadCommand.h
(ctrlSocket): Changed its type to SocketHandle.
(FtpDownloadCommand): Replaced Socket with SocketHandle.
* src/HttpProxyRequestCommand.cc
(HttpProxyRequestCommand): Replaced Socket with SocketHandle.
* src/FtpTunnelRequestCommand.h
(FtpTunnelRequestCommand): Replaced Socket with SocketHandle.
etc
* src/PeerChokeCommand.h
(setAllPeerChoked): Removed.
(setAllPeerResetDelta): Removed.
* src/PeerChokeCommand.cc
(setAllPeerChoked): Removed.
(ChokePeer): New function object.
(setAllPeerResetDelta): Removed.
(ResetDelta): New function object.
(orderByDownloadRate): Fixed a bug: use DowloadFaster, not
UploadFaster
(execute): Show download speed when the local node is a seeder.
setAllPeerChoked and setAllPeerResetDelta were rewritten
using STL.
* src/TrackerWatcherCommand.h
(MIN_PEERS): Removed.
* src/TorrentMan.cc
(getPeer): Replaced MAX_PEER_UPDATE with MIN_PEERS.
* CookieBox.cc: rewrited criteriaFind() using Util::startsWith()
and
Util::endsWith().
* SocketCore.cc: struct addrinfo is now zero-initialized.
* common.h: added #include directive of limit.h.
* DownloadEngine.cc: added #include directive of sys/time.h and
algorithm.
* Exception.h: added #include directive of stdio.h.
* AbstractCommand.h: added #include directive of sys/time.h.
* DownloadCommand.h: added #include directive of sys/time.h.
* *.h: added #include directive of common.h to all base classes.
subclasses' one was removed.
Tatsuhiro Tsujikawa