We use libssh2 to implement sftp feature. SSHSession is closely
designed to TLSSession, but it differs in several aspects. In this
code, we only implements read part of sftp, since aria2 won't offer
uploading feature. Adding SSH/sftp to SocketCore is a bit bloat. But
by doing this we can reuse DownloadCommand, without mostly no
modification. We can just create SSHDownloadCommand by inheriting it,
just like existing ftp.
Use one of the following to provide random bytes:
- Windows CryptGenRandom
- Linux getrandom (syscall interface to urandom, without nasty corner
cases such as file descriptor exhaustion or re-linked /dev/urandom)
- std::device_random (C++ random device, which usually will be urandom)
This also equalizes util::getRandom and SimpleRandomizer (the former
will now use the latter) instead of having essentially two different
PRNG interfaces with potentially different quality.
Closes GH-320
The algorithms the `CryptProv` on Windows supports does not currently
include SHA-224, so there is a "dark spot" in this implementation. Also
on Win XP < SP3, most of the SHA-2 family is not actually supported.
All other implementation provide support for MD5, SHA-1 and all of the
SHA-2 family, hence drop the incomplete WinMessageDigest implementation
in favor of any other supported implementation (at least the internal
implementation is always available at compile-time).
Add and use usedCompilerAndPlatform(). This adds compiler information to
INFO logs and the --version output, and may be helpful when trying to
diagnose/reproduce user-reported problems.
Also make INFO logs include usedLibs() output.
Closes#235
Now mingw-w64 C++ compiler defines __USE_MINGW_ANSI_STDIO to 1, we
have to follow it as well. We hope that mingw version of stdio
function behaves like Linux ones. We have not tested them thoroughly
yet. pthread for windows defines its own HAVE_STRUCT_TIMESPEC macro.
To avoid warning, we rename our version of it as
HAVE_A2_STRUCT_TIMESPEC.
Since we use libtool, -static-libgcc and -static-libstdc++ are
stripped by libtool and not used. We just use -all-static libtool flag
for this purpose.
To create windows dll, it turns out that we need -no-undefined flag.
Now you can build bittorrent support without without external
libraries, meaning you can skip libnettle, libgmp, libgcrypt, GnuTLS and
OpenSSL on OSX (for now).
Reusing a bignum (well, unsigned very-long) implementation I had lying
around for years and just cleaned up a bit and brought to C++11 land.
It might not be the most performant implementation, but it shoud be fast
enough for our purposes and will go a long way of removing gcrypt,
nettle, gmp, openssl dependencies when using AppleTLS and WinTLS
(upcoming).
Since Android ndk r9, __set_errno is deprecated. It is now defined as
inline function in errno.h. The syscall assembly calls __set_errno,
but since libc.so does not export it, the link fails. To workaround
this, replace all occurrences of __set_errno with a2_set_errno and
define it as normal C function.
Supported since automake-1.11. There is no point in having the very
verbose compile stuff running about, which cannot even silenced properly
with `make -s` by default. Otherwise, `make V=1` or
`--disable-silent-rules` are your friends
AC_USE_SYSTEM_EXTENSIONS will cause AC_PROG_CC, which is overridden by
automake-1.14, which will then init (part) of automake, in particular
am_aux_dir expansion, which in turn relies on ac_aux-dir, which is not
initialized at this point, and thus: certain doom (or fun, depending on
your POV and mood :p)
Hence call AC_USE_SYSTEM_EXTENSIONS only after AM_INIT_AUTOMAKE. This,
of course, caused a lot of related macro shuffling.
Tested against automake-1.10 (OSX Lion/XCode version) and automake-1.14
(homebrew version)
Android NDK R8e does not provide ftruncate64, but bionic has the
assembler code to access kernel function. We borrowed those
ftruncate64.S files from android source code repository. It turns out
that x86 asm.h in NDK R8e is also broken, so latest asm.h was also
borrowed.
By default, libaria2 feature is disabled. Use --enable-libaria2
to enable it. libaria2 static build is also disabled by default.
Use --enable-static to enable it.
Both tcmalloc_minimal and jemalloc outperform the native malloc
implemention on Windows (MSVCRT) in terms of committed memory
consumption (~-30%) and performance (e.g. far less page faults, ~-60%),
depending, of course, on the actual workload.
The longer the download queue, the bigger the impact ;)
On *nix the picture is a little different... tcmalloc usually still
outperforms the native malloc implementation, but not that significantly
than on Windows. jemalloc however is only marginally better than recent
native Linux implementations, while it is already used by some BSD as the
native allocator.
tcmalloc is part of gperftools and very mature and tested by now. It
doesn't work on OSX in the default configuration, however.
http://code.google.com/p/gperftools/
jemalloc is the default allocator at least on FreeBSD and NetBSD and
used in Firefox.
http://www.canonware.com/jemalloc/index.html
LibUV event will use the best available polling method on a system, kind
of like aria2 does already with the different *EventPoll
implementations.
However, libuv may support different/newer polling mechanisms; for
example on Windows it will use IO Completion Ports which are superior to
select() ;)
Most other software uses --disable/--without for features it does build
or at least check by default.
Change aria2 configure options so that:
* --enable-*: do not build by default, unless --enable specified
* --disable-*: check and build by default, unless --disable specified
* --with-*: do not use by default, unless --with specified
* --without-*: check and use by default, unless without specified
Saved sessions may very large, as in hundreds and even thousands of
megabyte when dealing with large queues.
Add support to save and reload sessions to gzipped files, when libz is
available.
The session serializer will output gzipped contents when the file ends
with .gz, while the input file reader (UriListParser) will accept
whatever is thrown at it.
mingw-w64 does not actually have sys/signal.h, while OSX currently has a
broken signal.h
Better check the presence of both and use sys/signal.h if present, else
signal.h
I tried CreateFile but the subsequent ReadFile fails with Access
Denied if sparse file is read on NTFS. I mostly reverted previous
changes and use _wsopen with read/write share enabled instead of
CreateFile.
This change also includes --enable-mmap support for MinGW32
build. Memory mapped file may be useful for 64-bits OS and lots of
RAM. Currently, FlushViewOfFile is not called during the download, so
it is slightly vulnerable against sudden power loss. I found lots of
read when resuming download due to page fault. So for now it is useful
for the initial download. I recommend not to use
--file-allocation=prealloc with --enable-mmap for MinGW32, because it
triggers page faults even in the initial download. Anyway, the option
is experimental.
Don't use AC_FUNC_MMAP becaue it fails on some platforms (e.g.,
OpenWRT) which have mmap and it works in the way we use in aria2.
Instead use mmap in AC_CHECK_FUNCS list.
Use system-wide certificates for SSL. For GnuTLS it requires the
latest version, 3.0.20. OpenSSL had it for longer. This means that if
SSL library is properly configured to locate system-wide certificates
store, the user don't have to use --ca-certificate option. Also
packagers don't have to use --with-ca-bundle configure option.
Patch from Cristian Morales Vega
Check sockaddr_in.sin_len and sockaddr_in6.sin6_len are available and
assign values to them properly. This change fixes unit test error and
most error related to getnameinfo() on netbsd.
Tatsuhiro Tsujikawa