We introduced SocketRecvBuffer which buffers received bytes. Since
HTTP response header and response body are divided with \r\n, we have
to buffer up several bytes to find this delimiter. We use
SocketRecvBuffer to hold these bytes and only consumes header and
passes SocketRecvBuffer, which may contain head of response body, to
next Command. Since FTPConnection doesn't use SocketCore::peekData(),
we left it as is.
Added .cc file for classes/structs that only provided by header
file. Defined non-POD classes' ctor, dtor in .cc file. Moved
implementation code in header file to .cc file for major
classes/strucsts.
Removed SharedHandle::isNull(). Instead we added operator* and
operator unspecified_bool_type. Removed use of WeakHandle and
replaced with raw pointer.
Moved connectedHostname, connectedAddr and connectedPort to
Request object. Mark cached IP address bad on timeout to allow
aria2 to renew IP address cache.
* src/AbstractCommand.cc
* src/AbstractProxyRequestCommand.cc
* src/AbstractProxyRequestCommand.h
* src/FtpInitiateConnectionCommand.cc
* src/FtpNegotiationCommand.cc
* src/FtpNegotiationCommand.h
* src/HttpInitiateConnectionCommand.cc
* src/HttpRequestCommand.cc
* src/HttpRequestCommand.h
* src/InitiateConnectionCommand.cc
* src/InitiateConnectionCommand.h
* src/Request.cc
* src/Request.h
Send each CWD component of FTP URI in separate CWD command as
described in RFC1738.
* src/FtpConnection.cc
* src/FtpConnection.h
* src/FtpInitiateConnectionCommand.cc
* src/FtpNegotiationCommand.cc
* src/FtpNegotiationCommand.h
* test/FtpConnectionTest.cc
Fixed the bug that connection pooling does not take into account
proxy. This means that when connection A via proxy X is pooled,
it will be wrongly reused in the download using proxy Y.
* src/DownloadEngine.cc
* src/DownloadEngine.h
* src/FtpFinishDownloadCommand.cc
* src/FtpInitiateConnectionCommand.cc
* src/FtpNegotiationCommand.cc
* src/HttpDownloadCommand.cc
* src/HttpInitiateConnectionCommand.cc
* src/HttpResponseCommand.cc
* src/HttpSkipResponseCommand.cc
Fixed the bug that FTP download may fail when control connection
is reused. This happens because FTP server can offer different
root directory for different account. If pooled connections has
different root directory, then download will fail.
* src/DownloadEngine.cc
* src/DownloadEngine.h
* src/FtpConnection.cc
* src/FtpConnection.h
* src/FtpFinishDownloadCommand.cc
* src/FtpInitiateConnectionCommand.cc
* src/FtpNegotiationCommand.cc
Changed the default value of --proxy-method option from 'tunnel'
to 'get'. Use 'tunnel' for HTTPS regardless of --proxy-method
option.
* src/AbstractCommand.cc
* src/AbstractCommand.h
* src/FtpInitiateConnectionCommand.cc
* src/HttpDownloadCommand.cc
* src/HttpInitiateConnectionCommand.cc
* src/HttpInitiateConnectionCommand.h
* src/OptionHandlerFactory.cc
Added the ability to pool proxy connection.
The conneciton in FTP with proxy-method=GET is not pooled.
Proxy-Connection header will not be sent when sending CONNECT
method.
* src/DownloadEngine.cc
* src/DownloadEngine.h
* src/FtpFinishDownloadCommand.cc
* src/FtpInitiateConnectionCommand.cc
* src/FtpNegotiationCommand.cc
* src/HttpDownloadCommand.cc
* src/HttpInitiateConnectionCommand.cc
* src/HttpRequest.cc
* src/HttpRequest.h
* src/HttpResponse.cc
* src/HttpSkipResponseCommand.cc
* test/HttpRequestTest.cc
* test/HttpResponseTest.cc
Issue PWD command first and get working directory and use it as
a prefix
for CWD command.
* src/DownloadEngine.cc
* src/DownloadEngine.h
* src/FtpConnection.cc
* src/FtpConnection.h
* src/FtpFinishDownloadCommand.cc
* src/FtpInitiateConnectionCommand.cc
* src/FtpNegotiationCommand.cc
* src/FtpNegotiationCommand.h
* test/FtpConnectionTest.cc
Added the ability to reuse connection in FTP and it is enabled
by
default. It can be disabled by --ftp-reuse-connection=false
option.
* src/Command.cc
* src/DownloadCommand.cc
* src/FtpDownloadCommand.cc
* src/FtpDownloadCommand.h
* src/FtpFinishDownloadCommand.cc
* src/FtpFinishDownloadCommand.h
* src/FtpInitiateConnectionCommand.cc
* src/FtpNegotiationCommand.cc
* src/FtpNegotiationCommand.h
* src/HelpItemFactory.cc
* src/Makefile.am
* src/Makefile.in
* src/OptionHandlerFactory.cc
* src/option_processing.cc
* src/prefs.h
* src/usage_text.h
Rewritten Exception class. Throw exception object, not its pointer and
catch by reference, so that remove problematic delete operator for
catched exception.
* src/Exception.cc
* src/Exception.h
* test/ExceptionTest.cc
* src/*: All files throwing/catching exception.
* test/*: All files throwing/catching exception.
Eliminates the time lag between sequential downloads and
commands in
the same RequestGroup.
In old implementation, aria2 occasionally waits 1 seconds before
executing next command or RequestGroup.
This is really a waste of time, and new implementation
eliminates
this unnecessary time lag.
* src/AbstractCommand.cc
* src/AutoSaveCommand.cc
* src/CheckIntegrityCommand.cc
* src/Command.{cc, h}
* src/DownloadEngine.{cc, h}
* src/DownloadEngineFactory.cc
* src/FileAllocationCommand.cc
* src/FileAllocationDispatcherCommand.cc
* src/FillRequestGroupCommand.cc
* src/FtpInitiateConnectionCommand.cc
* src/HaveEraseCommand.cc
* src/HttpInitiateConnectionCommand.cc
* src/HttpResponseCommand.cc
* src/RealtimeCommand.cc
* src/RequestGroup.cc
* src/RequestGroupMan.cc
* src/StreamFileAllocationEntry.cc
* src/TimeBasedCommand.{cc, h}
* src/TimedHaltCommand.cc
Rewritten SharedHandle. Now copy constructor taking raw pointer
has
keyword explicit and SharedHandle's default constructor
initializes
its internal obj to null, old implementation initializes it
using
obj's default constructor.
To assign null, write SharedHandle<T> x(...); x.reset();
TODO: test/SharedHandleTest.cc needs more tests.
* src/SharedHandle.h
Implemented BitTorrent/http/ftp integrated download.
I've rewritten lots of files and now some headers have forward
class declarations to reduce compile time.
The implementation is extremely alpha stage, I recommend to use this
for testing purpose only.
To handle Segment as SegmentHandle:
* src/AbstractCommand.cc (execute): Rewritten.
* src/SegmentMan.h: Segment -> SegmentHandle
Introducded HttpResponse class, HttpRequest class to improve
code
extensiveness and make it clear:
* src/HttpDownloadCommand.cc: transfer encoders are now managed
by
HttpResponse class.
* src/HttpRequest.h, src/HttpRequest.cc: New class.
* src/HttpResponse.h, src/HttpResponse.cc: New class.
* src/HttpConnection.cc: Contruction of http request were moved
to
HttpRequest class.
* src/HttpResponseCommand.h, src/HttpResponseCommand.cc:
Refactored.
* src/HttpRequestCommand.cc (executeInternal): Rewritten.
* src/HttpAuthConfig.h: New class.
* src/Range.h: New class.
To make FtpTunnel{Request, Response}Command and
HttpProxy{Request, Response}Command derived from
AbstractProxy{Request, Response}Command:
* src/FtpTunnelResponseCommand.h,
src/FtpTunnelResponseCommand.cc:
Derived from AbstractProxyRequestCommand class.
* src/FtpTunnelRequestCommand.h, src/FtpTunnelRequestCommand.cc:
Derived from AbstractProxyResponseCommand class.
* src/HttpProxyRequestCommand.h, src/HttpProxyRequestCommand.cc:
Derived from AbstractProxyRequestCommand class.
* src/HttpProxyResponseCommand.h,
src/HttpProxyResponseCommand.cc:
Derived from AbstractProxyResponseCommand class.
* src/AbstractProxyRequestCommand.h,
src/AbstractProxyRequestCommand.cc
: New class.
* src/AbstractProxyResponseCommand.h,
src/AbstractProxyResponseCommand.cc: New class.
To add netrc support:
* src/Netrc.h, src/Netrc.cc: New class.
* src/Util.h, src/Util.cc (split): New function.
* src/HttpHeader.cc (getRange): Fixed so that it inspects
"Content-Range" header instead of "Range" header.
* src/HttpHeader.h
(getStatus): Removed.
(setStatus): Removed.
* src/Segment.h
(getPositionToWrite): New function.
To rewrite segment download mechanism for HTTP/FTP download.
Use BitfieldMan to manage segment download.
* src/HttpResponseCommand.h
(executeInternal): Pass the reference of segment.
* src/AbstractCommand.cc
(prepareForRetry): Call segmentMan->cancelSegment here.
(onAbort): Call segmentMan->cancelSegment here.
* src/HttpDownloadCommand.cc
(prepareForNextSegment): New function.
* src/DownloadEngineFactory.cc
(newConsoleEngine): Removed splitter.
(newTorrentConsoleEngine): Removed splitter.
* src/Request.h
(segment): Renamed from seg.
* src/FtpInitiateConnectionCommand.h
(executeInternal): Pass the reference of segment.
* src/AbstractCommand.h
(executeInternal): Pass the reference of segment.
* src/pref.h
(PREF_SEGMENT_SIZE): New definition.
* src/HttpProxyRequestCommand.h
(executeInternal): Pass the reference of segment.
* src/HttpResponseCommand.cc
(checkResponse): Allowed status 206 when a request range starts
0.
(handleDefaultEncoding): Rewritten the code related to Segment.
(handleOtherEncoding): Rewritten the code related to Segment.
* src/SegmentMan.h
(SegmentEntry): New class.
(SegmentEntries): New type definition.
(bitfield): New variable.
(usedSegmentEntries): New variable.
(onNullBitfield): New function.
(checkoutSegment): New function.
(segments): Removed.
(splitter): Removed.
(unregisterId): Removed.
(getSegment): New function(overload)
(getDownloadedSize): Removed.
(cancelSegment): New function.
(completeSegment): New function.
(initBitfield): New function.
(hasSegment): New function.
(getDownloadLength): New function.
* src/BitfieldMan.h
(getStartIndex): New function.
(getEndIndex): New function.
(getMissingUnusedIndex): New function(overload).
(getSparseMissingUnusedIndex): New function.
* src/BitfieldMan.cc
(getMissingIndexRandomly): Handle the last byte of bitfield
properly.
(getMissingUnusedIndex): New function(overload).
(Range): New class.
(getStartIndex): New function.
(getEndIndex): New function.
(getSparseMissingUnusedIndex): New function.
(isBitSetInternal): Return false if the given index is less than
0.
* src/HttpInitiateConnectionCommand.h
(executeInternal): Pass the reference of segment.
* src/FtpNegotiateCommand.h
(executeInternal): Pass the reference of segment.
* src/FtpNegotiateCommand.cc
(recvSize): Initialize bitfield here.
* src/FtpTunnelResponseCommand.h
(executeInternal): Pass the reference of segment.
* src/HttpConnection.cc
(createRequest): Rewritten range header processing.
* src/DownloadCommand.h
(executeInternal): Pass the reference of segment.
(prepareForRetry): Removed.
(prepareForNextSegment): Added an argument segment. Made it a
virtual
function.
* src/main.cc
(main): Set the initial value of PREF_SEGMENT_SIZE to 1MB.
* src/SegmentMan.cc
(SegmentMan): Added bitfield. Removed splitter.
(~SegmentMan): Added bitfield. Removed splitter.
(unregisterId): Removed.
(getSegment): Rewritten.
(updateSegment): Rewritten.
(save): Rewritten.
(read): Rewritten.
(finished): Rewritten.
(getDownloadedSize): Removed.
(initBitfield): New function.
(FindSegmentEntryByIndex): New function object.
(FindSegmentEntryByCuid): New function object.
(checkoutSegment): New function.
(onNullBitfield): New function.
(getSegment): New function(overload).
(CancelSegment): New function object.
(cancelSegment): New function.
(completeSegment): New function.
(hasSegment): New function.
(getDownloadLength): New function.
* src/FtpInitiateConnectionCommand.cc
(executeInternal): Load .aria2 file after hostname resolution
finishes.
* src/Segment.h: Rewritten.
* src/Segment.cc (operator<<): New function.
* src/HttpDownloadCommand.h
(prepareForNextSegment): New function.
* src/Request.cc
(resetUrl): Made segment null.
* src/DownloadEngine.cc
(~DownloadEngine): Call cleanQueue before deleting segmentMan.
* src/HttpProxyRequestCommand.h
(executeInternal): Pass the reference of segment.
* src/DownloadCommand.cc
(executeInternal): Rewritten the code related to Segment.
(prepareForRetry): Removed.
(prepareForNextSegment): Rewritten.
* src/FtpTunnelResponseCommand.h
(executeInternal): Pass the reference of segment.
To add HTTP 1.1 persistent connection support(experimental)
* src/HttpRequestCommand.cc
(executeInternal): Disable keep alive if it is disabled by
configuration.
* src/Request.h
(keepAlive): New variable.
(isKeepAlive): New function.
(setKeepAlive): New function.
* src/pref.h
(PREF_HTTP_KEEP_ALIVE): New definition.
* src/HttpResponseCommand.cc
(executeInternal): Check the remote server supports keep alive.
* src/HttpConnection.cc
(createRequest): Send "Connection: close" only if keep alive is
disabled.
* src/main.cc
(main):
Set the initial value(false) of PREF_KEEP_ALIVE to false.
To add max download speed limit:
* src/pref.h
(PREF_MAX_SPEED_LIMIT): New definition.
* src/PeerInteractionCommand.cc
(executeInternal): Added max download speed limit. Not tested
yet.
* src/SegmentMan.h
(PeerStats): New type definition.
(peerStats): New variable.
(registerPeerStat): New function.
(FindPeerStat): New function object.
(getPeerStat): New function.
(calculateDownloadSpeed): New function.
* src/SpeedCalc.h: New class.
* src/SpeedCalc.cc: New class.
* src/main.cc
(main):
Set the initial value of PREF_MAX_SPEED_LIMIT to 0(which means
the
download speed is not restricted).
* src/PeerStat.h: New class.
* src/SegmentMan.cc
(registerPeerStat): New function.
(calculateDownloadSpeed): New function.
* src/DownloadCommand.cc
(STARTUP_IDLE_TIME): New definition.
(DownloadCommand): Register peerStat to segmentMan. Call
peerStat->
downloadStart.
(~DownloadCommand): Call peerStat->downloadStop.
(executeInternal): Added download speed limitter. Rewritten
lowest
speed limitter.
* src/HttpConnection.cc
(receiveResponse): Fixed: eohIndex[headerBuf] ->
headerBuf[eohIndex].
* src/AbstractCommand.cc
(resolveHostname): Throw DlAbortEx if a name resolution failes.
Added hostname to the error message.
* src/ConsoleDownloadEngine.cc
(calculateStatistics): Initialize psize with dlSize.
* src/PieceMessage.cc
(receivedAction): Do not call peerInteraction->abortPiece here.
(onGotWrongPiece): Call peerInteraction->abortPiece here.
* src/BitfieldMan.h
(clearAllUseBit): New function.
(setAllUseBit): New function.
* src/BitfieldMan.cc
(clearAllBit): Do not clear useBitfield here.
(clearAllUseBit): New function.
(setAllUseBit): New function.
* src/Piece.cc
(clearAllBlock): Call bitfield->clearAllUseBit().
To make filename URL-decoded:
* src/HttpResponseCommand.h: Updated doc.
* src/HttpResponseCommand.cc
(determinFilename): Made filename URL-decoded.
* src/FtpInitiateConnectionCommand.cc
(executeInternal): Made filename URL-decoded.
* src/Util.h (urldecode): New function.
* src/Util.cc (urldecode): New function.
To fix compilation problem on gcc4.1.1(patch#1542283 by
tizianomueller)
* src/NameResolver.h: Added the prototype declaration of
callback().
To support c-ares library:
* src/AbstractCommand.cc: Replaced HAVE_LIBARES with
ENABLE_ASYNC_DNS
* src/FeatureConfig.cc: Replaced HAVE_LIBARES with
ENABLE_ASYNC_DNS
* src/FtpInitiateConnectionCommand.h:
Replaced HAVE_LIBARES with ENABLE_ASYNC_DNS
* src/AbstractCommand.h: Replaced HAVE_LIBARES with
ENABLE_ASYNC_DNS
* src/HttpInitiateConnectionCommand.h:
Replaced HAVE_LIBARES with ENABLE_ASYNC_DNS
* src/NameResolver.cc: Replaced HAVE_LIBARES with
ENABLE_ASYNC_DNS
* src/HttpInitiateConnectionCommand.cc:
Replaced HAVE_LIBARES with ENABLE_ASYNC_DNS
* src/FtpInitiateConnectionCommand.cc:
Replaced HAVE_LIBARES with ENABLE_ASYNC_DNS
* src/DownloadEngine.h:
Replaced HAVE_LIBARES with ENABLE_ASYNC_DNS
* src/DownloadEngine.cc:
Replaced HAVE_LIBARES with ENABLE_ASYNC_DNS
To add asynchronous DNS support(libares):
* src/AbstractCommand.h
(setNameResolverCheck): New function.
(disableNameResolverCheck): New function
(resolveHostname): New function.
* src/AbstractCommand.cc
(setNameResolverCheck): New function.
(disableNameResolverCheck): New function
(resolveHostname): New function.
* src/FtpInitiateConnectionCommand.h
(nameResolver): New function.
* src/FtpInitiateConnectionCommand.cc
(Util.h): Included.
(FtpInitiateConnectionCommand): Call disableReadCheckSocket,
disableWriteCheckSocket.
(~FtpInitiateConnectionCommand): Call disableNameResolverCheck.
(executeInternal): Added async DNS support.
* src/HttpInitiateConnectionCommand.h
(nameResolver): New function.
* src/HttpInitiateConnectionCommand.cc
(DlRetryEx.h): Included.
(HttpInitiateConnectionCommand): Call disableReadCheckSocket,
disableWriteCheckSocket.
(~HttpInitiateConnectionCommand): Call disableNameResolverCheck.
(executeInternal): Added async DNS support.
* src/NameResolver.h: New class. Note: #include <ares.h> is
surrounded
by extern "C" declaration.
* src/NameResolver.cc: New class.
* src/Util.h:
(isNumberAndDotsNotation): New function
* src/Util.cc
(isNumberAndDotsNotation): New function.
* src/DownloadEngine.h
(NameResolver.h): Included.
(NameResolverEntry): New class.
(NameResolverEntries): New type definition.
(addNameResolverCheck): New function.
(deleteNameResolverCheck): New function.
* src/DownloadEngine.cc
(run): Initialize cp with 0.
(SetDescriptor::operator()): Added for NameResolverEntry.
(AccumulateActiveUuid::operator()): Added for NameResolverEntry.
(waitData): Check nameResolver entries.
(updateFdSet): Update fds in nameResolver entries.
(addNameResolverCheck): New function.
(deleteNameResolverCheck): new function.
* src/PeerInteractionCommand.cc
(executeInternal): Fixed wrong socket handling.
* src/main.cc
(main): Fixed the initial value of PREF_LOWEST_SPEED_LIMIT
option to 0.
* src/Util.cc
(fileChecksum): Removed the call to ctx.digestReset().
Tatsuhiro Tsujikawa