Android NDK R8e does not provide ftruncate64, but bionic has the
assembler code to access kernel function. We borrowed those
ftruncate64.S files from android source code repository. It turns out
that x86 asm.h in NDK R8e is also broken, so latest asm.h was also
borrowed.
I tried CreateFile but the subsequent ReadFile fails with Access
Denied if sparse file is read on NTFS. I mostly reverted previous
changes and use _wsopen with read/write share enabled instead of
CreateFile.
This change also includes --enable-mmap support for MinGW32
build. Memory mapped file may be useful for 64-bits OS and lots of
RAM. Currently, FlushViewOfFile is not called during the download, so
it is slightly vulnerable against sudden power loss. I found lots of
read when resuming download due to page fault. So for now it is useful
for the initial download. I recommend not to use
--file-allocation=prealloc with --enable-mmap for MinGW32, because it
triggers page faults even in the initial download. Anyway, the option
is experimental.
Using off_t, at least, in DiskAdaptor layer is problematic because
torrent can contain under 2GiB files but total sum of those files may
exceed 2GiB limit, which makes off_t overflow in 32 bit system without
large file support. So we use int64_t in API. We'll check the file
length before download so that it does not exceed max off_t.
~AbstractDiskWriter calles closeFile(), but suppresses exception.
MultiDiskAdaptor::closeFile() logs error if child
DiskWriter::closeFile() throws exception. This exception is not
rethrown. If at least one exception is caught,
MultiDiskAdaptor::closeFile() throws new DlAbortEx.
RequestGroupMan::closeFile() just logs exception and suppress each
exception. Generally, don't call closeFile() in destructor. If you
need to call it, it must suppress the exception.
We replaced most of std::cerr with global::cerr. windows.h is now
included from common.h. Before including it, we define WINVER. We
renamed some variable name because some macros in windows.h collide
with them.
We use SetFilePointerEx and SetEndOfFile to allocate extents. This
only works with NTFS. To enable this feature, --file-allocation=falloc
must be given.
When allocating disk space, for Linux system with fallocate()
system call, first check file system supports fallocate. This
just run fallocate with small chunk and see it succeeds or fails.
If it succeeds, use fallocate() to allocate entire file otherwise
fall back to traditional slower method: writing zeros. This
behavior is enabled in --file-allocation=prealloc, so this is
enabled by default for most modern Linux.
* configure.ac
* src/AbstractDiskWriter.cc
* src/AbstractDiskWriter.h
* src/AbstractSingleDiskAdaptor.cc
* src/AdaptiveFileAllocationIterator.cc
* src/AdaptiveFileAllocationIterator.h
* src/DefaultPieceStorage.cc
* src/DiskAdaptor.cc
* src/DiskAdaptor.h
* src/FallocFileAllocationIterator.cc
* src/Makefile.am
* src/MultiFileAllocationIterator.cc
* src/OptionHandlerFactory.cc
* test/FallocFileAllocationIteratorTest.cc
* test/Makefile.am
Now the constructor of AbstractDiskWriter takes filename as an
argument and filename argument is removed from openFile(),
initAndOpenFile(), openExistingFile() interface. storeDir
member and its accessor functions are removed from DiskAdaptor
because it is not used anymore. size() member function of
DefaultDiskWriter, DirectDiskAdaptor and MultiDiskAdaptor now
can be called without opening file.
* src/AbstractDiskWriter.cc
* src/AbstractDiskWriter.h
* src/AbstractSingleDiskAdaptor.cc
* src/AbstractSingleDiskAdaptor.h
* src/ByteArrayDiskWriter.cc
* src/ByteArrayDiskWriter.h
* src/ByteArrayDiskWriterFactory.cc
* src/ByteArrayDiskWriterFactory.h
* src/DefaultDiskWriter.cc
* src/DefaultDiskWriter.h
* src/DefaultDiskWriterFactory.cc
* src/DefaultDiskWriterFactory.h
* src/DefaultPieceStorage.cc
* src/DirectDiskAdaptor.cc
* src/DirectDiskAdaptor.h
* src/DiskAdaptor.h
* src/DiskWriter.h
* src/DiskWriterFactory.h
* src/MessageDigestHelper.cc
* src/MultiDiskAdaptor.cc
* src/RequestGroup.cc
* src/UnknownLengthPieceStorage.cc
* src/Util.cc
* test/DefaultDiskWriterTest.cc
* test/DirectDiskAdaptorTest.cc
* test/FallocFileAllocationIteratorTest.cc
* test/MessageDigestHelperTest.cc
* test/MetalinkProcessorTest.cc
* test/MultiDiskAdaptorTest.cc
* test/MultiFileAllocationIteratorTest.cc
* test/SingleFileAllocationIteratorTest.cc
* test/UtilTest.cc
Added 'falloc' parameter for --file-allocation option. 'falloc'
allocation mode uses posix_fallocate() system call to allocate
file on disk. If you are using newer file systems such as ext4
(with extents support), btrfs or xfs, 'falloc' is your best
choice. It allocates large(few GiB) files almost instantly.
Don't use 'falloc' with legacy file systems such as ext3 because
it takes almost same time as 'prealloc' and it blocks aria2
entirely until allocation finishes. 'falloc' may not be
available if your system doesn't have posix_fallocate() system
call.
* configure.ac
* src/AbstractDiskWriter.cc
* src/AbstractDiskWriter.h
* src/AbstractSingleDiskAdaptor.cc
* src/BinaryStream.h
* src/BtCheckIntegrityEntry.cc
* src/ByteArrayDiskWriter.h
* src/CheckIntegrityEntry.cc
* src/CheckIntegrityEntry.h
* src/DefaultPieceStorage.cc
* src/DiskAdaptor.cc
* src/DiskAdaptor.h
* src/DiskWriter.h
* src/FallocFileAllocationIterator.cc
* src/FallocFileAllocationIterator.h
* src/FileAllocationEntry.cc
* src/FileAllocationEntry.h
* src/Makefile.am
* src/MultiFileAllocationIterator.cc
* src/MultiFileAllocationIterator.h
* src/OptionHandlerFactory.cc
* src/RequestGroup.cc
* src/StreamCheckIntegrityEntry.cc
* src/prefs.cc
* src/prefs.h
* src/usage_text.h
* test/FallocFileAllocationIteratorTest.cc
* test/Makefile.am
In BitTorrent, Open files in read-only mode for hash check.
After that, re-open file in read/write mode only when the
download is not finished.
* src/AbstractDiskWriter.cc
* src/AbstractDiskWriter.h
* src/AbstractSingleDiskAdaptor.cc
* src/AbstractSingleDiskAdaptor.h
* src/BtCheckIntegrityEntry.cc
* src/DiskAdaptor.h
* src/DiskWriter.h
* src/MultiDiskAdaptor.cc
* src/MultiDiskAdaptor.h
* src/RequestGroup.cc
Use seek() and SetEndOfFile() for mingw32 build instead of
ftruncate(),
because mingw32's ftruncate() cannot handle more than 2GB-size
file.
* src/AbstractDiskWriter.cc
Use File::exists() instead of File::isFile() to allow
non-regular file
such as block spacial device.
* src/AbstractDiskWriter.cc
* src/MultiDiskWriter.cc
Introduced a2_struct_stat. It is defined as `struct _stati64' if
__MINGW32__ is defined, because under MinGW32, _stati64 is used
and its
second argument is of type `struct _stati64'. Otherwise it is
defined as
`struct stat'.
* src/AbstractDiskWriter.cc
* src/File.cc
* src/File.h
* src/a2io.h
Nils Maier