github
/
aria2
mirror of https://github.com/aria2/aria2
1
0
Fork 0
Code Issues Releases Wiki Activity

2006-09-21 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com> 

* src/AbstractCommand.cc
	(execute): Check whether the download has finished before 
checking
	socket status.
	Return true if peerStat->getStatus() == REQUEST_IDLE.
	Do not exit even if no segment is available.

	* src/prefs.h
	(PREF_STARTUP_IDLE_TIME): New definition.

	* src/PeerInteractionCommand.cc
	(executeInternal): Removed max speed limit. Because it performs 
bad.
	(receiveMessages): Added max speed limit. This was better than 
above,
	but still a little bit to be desired. Sometimes the download 
speed
	became much faster than I specified.

	* src/SpeedCalc.h
	(start): New variable.
	(accumulatedLength): New variable.
	(getAvgSpeed): New function.
	* src/SpeedCalc.cc
	(reset): Added start, accumulatedLength.
	(update): Added calculation of an average speed.
	(getAvgSpeed): New function.

	* src/DownloadCommand.h
	(sw): Removed.

	* src/main.cc
	(main): Added PREF_STARTUP_IDLE_TIME.

	* src/PeerStat.h
	(STATUS): Added REQUEST_IDLE.
	(getMaxSpeed): Renamed as getMaxDownloadSpeed().
	(getAvgDownloadSpeed): New function
	(requestIdle): New function.

	* src/SegmentMan.h
	(SegmentEntryHandle): New type definition.
	(SegmentEntries): Now holds SegmentEntryHandle.
	(findSlowerSegmentEntry): New funtion.
	* src/SegmentMan.cc
	(save): Updated according to the changes in SegmentEntries.
	(read): Updated according to the changes in SegmentEntries.
	(FindSegmentEntryByIndex): Updated according to the changes in
	SegmentEntries.
	(FindSegmentEntryByCuid): Updated according to the changes in
	SegmentEntries.
	(checkoutSegment): Updated according to the changes in 
SegmentEntries.
	(onNullBitfield): Updated according to the changes in 
SegmentEntries.
	Renamed uitr as itr.
	(findSlowerSegmentEntry): New function.
	(getSegment): Updated according to the changes in 
SegmentEntries.
	Added the feature that cancels the segment with slow server and 
fast
	one takes it over.
	(cancelSegment): Updated according to the changes in 
SegmentEntries.
	(getDownloadLength): Updated according to the changes in
	SegmentEntries.
	(init): Assigned 0 to bitfield after deleting it.
	
	* src/DownloadCommand.cc
	(STARTUP_IDLE_TIME): Removed.
	(executeInternal): Use PREF_STARTUP_IDLE_TIME.
pull/1/head
Tatsuhiro Tsujikawa 2006-09-21 06:56:54 +00:00
parent 2fb9b5be97
commit 65e0ffe6ca
14 changed files with 198 additions and 46 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

65
ChangeLog
View File

@ -1,3 +1,68 @@
2006-09-21 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com>
* src/AbstractCommand.cc
(execute): Check whether the download has finished before checking
socket status.
Return true if peerStat->getStatus() == REQUEST_IDLE.
Do not exit even if no segment is available.
* src/prefs.h
(PREF_STARTUP_IDLE_TIME): New definition.
* src/PeerInteractionCommand.cc
(executeInternal): Removed max speed limit. Because it performs bad.
(receiveMessages): Added max speed limit. This was better than above,
but still a little bit to be desired. Sometimes the download speed
became much faster than I specified.
* src/SpeedCalc.h
(start): New variable.
(accumulatedLength): New variable.
(getAvgSpeed): New function.
* src/SpeedCalc.cc
(reset): Added start, accumulatedLength.
(update): Added calculation of an average speed.
(getAvgSpeed): New function.
* src/DownloadCommand.h
(sw): Removed.
* src/main.cc
(main): Added PREF_STARTUP_IDLE_TIME.
* src/PeerStat.h
(STATUS): Added REQUEST_IDLE.
(getMaxSpeed): Renamed as getMaxDownloadSpeed().
(getAvgDownloadSpeed): New function
(requestIdle): New function.
* src/SegmentMan.h
(SegmentEntryHandle): New type definition.
(SegmentEntries): Now holds SegmentEntryHandle.
(findSlowerSegmentEntry): New funtion.
* src/SegmentMan.cc
(save): Updated according to the changes in SegmentEntries.
(read): Updated according to the changes in SegmentEntries.
(FindSegmentEntryByIndex): Updated according to the changes in
SegmentEntries.
(FindSegmentEntryByCuid): Updated according to the changes in
SegmentEntries.
(checkoutSegment): Updated according to the changes in SegmentEntries.
(onNullBitfield): Updated according to the changes in SegmentEntries.
Renamed uitr as itr.
(findSlowerSegmentEntry): New function.
(getSegment): Updated according to the changes in SegmentEntries.
Added the feature that cancels the segment with slow server and fast
one takes it over.
(cancelSegment): Updated according to the changes in SegmentEntries.
(getDownloadLength): Updated according to the changes in
SegmentEntries.
(init): Assigned 0 to bitfield after deleting it.
* src/DownloadCommand.cc
(STARTUP_IDLE_TIME): Removed.
(executeInternal): Use PREF_STARTUP_IDLE_TIME.
2006-09-19 Tatsuhiro Tsujikawa <tujikawa at rednoah dot com>
To rewrite segment download mechanism for HTTP/FTP download.

2
TODO
View File

@ -16,8 +16,10 @@
* Save URLs and command-line arguments to .aria2 file.
* Add multi-file metalink support.
* Add a control port for GUI frontend
* Add a version header to .aria2 file to check the compatibiliy.
0.8.0
* Add a statement for the permission to link with OpenSSL.
* Add upload speed limit command-line option(not tested for torrent yet).

2
po/Makefile.in
View File

@ -9,7 +9,7 @@
# General Public License and is *not* in the public domain.
PACKAGE = aria2c
VERSION = 0.7.3
VERSION = 0.8.0
SHELL = /bin/sh

21
src/AbstractCommand.cc
View File

@ -44,23 +44,34 @@ AbstractCommand::~AbstractCommand() {
bool AbstractCommand::execute() {
try {
if(e->segmentMan->finished()) {
logger->debug("CUID#%d - finished.", cuid);
return true;
}
PeerStatHandle peerStat = e->segmentMan->getPeerStat(cuid);
if(peerStat.get()) {
if(peerStat->getStatus() == PeerStat::REQUEST_IDLE) {
logger->info("CUID#%d - Request idle.", cuid);
onAbort(0);
req->resetUrl();
tryReserved();
return true;
}
}
if(checkSocketIsReadable && readCheckTarget->isReadable(0) ||
checkSocketIsWritable && writeCheckTarget->isWritable(0) ||
!checkSocketIsReadable && !checkSocketIsWritable) {
checkPoint.reset();
if(e->segmentMan->finished()) {
logger->debug("CUID#%d - finished.", cuid);
return true;
}
Segment segment;
if(e->segmentMan->downloadStarted) {
if(!e->segmentMan->getSegment(segment, cuid)) {
logger->info(MSG_NO_SEGMENT_AVAILABLE, cuid);
return true;
return prepareForRetry(1);
}
}
return executeInternal(segment);
} else {
if(checkPoint.elapsed(timeout)) {
throw new DlRetryEx(EX_TIME_OUT);
}

5
src/DownloadCommand.cc
View File

@ -29,8 +29,6 @@
#include "prefs.h"
#include <sys/time.h>
#define STARTUP_IDLE_TIME 10
DownloadCommand::DownloadCommand(int cuid, Request* req, DownloadEngine* e,
const SocketHandle& s):
AbstractCommand(cuid, req, e, s), lastSize(0) {
@ -78,7 +76,7 @@ bool DownloadCommand::executeInternal(Segment& segment) {
peerStat->updateDownloadLength(bufSize);
}
// calculate downloading speed
if(/*sw.elapsed(1) >= 1 && */peerStat->getDownloadStartTime().elapsed(STARTUP_IDLE_TIME)) {
if(peerStat->getDownloadStartTime().elapsed(e->option->getAsInt(PREF_STARTUP_IDLE_TIME))) {
int lowestLimit = e->option->getAsInt(PREF_LOWEST_SPEED_LIMIT);
int nowSpeed = peerStat->calculateDownloadSpeed();
if(lowestLimit > 0 && nowSpeed <= lowestLimit) {
@ -87,7 +85,6 @@ bool DownloadCommand::executeInternal(Segment& segment) {
nowSpeed,
lowestLimit);
}
//sw.reset();
}
if(e->segmentMan->totalSize != 0 && bufSize == 0) {
throw new DlRetryEx(EX_GOT_EOF);

1
src/DownloadCommand.h
View File

@ -30,7 +30,6 @@ using namespace std;
class DownloadCommand : public AbstractCommand {
private:
Time sw;
long long int lastSize;
protected:
bool executeInternal(Segment& segment);

14
src/PeerInteractionCommand.cc
View File

@ -127,11 +127,8 @@ bool PeerInteractionCommand::executeInternal() {
}
receiveMessages();
int maxSpeedLimit = e->option->getAsInt(PREF_MAX_SPEED_LIMIT);
if(maxSpeedLimit == 0 ||
maxSpeedLimit > 0 && maxSpeedLimit <= e->getDownloadSpeed()) {
peerInteraction->addRequests();
}
peerInteraction->addRequests();
peerInteraction->sendMessages(e->getUploadSpeed());
break;
}
@ -194,6 +191,13 @@ void PeerInteractionCommand::decideChoking() {
void PeerInteractionCommand::receiveMessages() {
for(int i = 0; i < 50; i++) {
int maxSpeedLimit = e->option->getAsInt(PREF_MAX_SPEED_LIMIT);
if(maxSpeedLimit > 0 && maxSpeedLimit < e->getDownloadSpeed()) {
disableReadCheckSocket();
setNoCheck(true);
break;
}
PeerMessageHandle message = peerInteraction->receiveMessage();
if(message.get() == NULL) {
return;

13
src/PeerStat.h
View File

@ -30,7 +30,8 @@ class PeerStat {
public:
enum STATUS {
IDLE,
ACTIVE
ACTIVE,
REQUEST_IDLE,
};
private:
int cuid;
@ -54,10 +55,14 @@ public:
downloadSpeed.update(bytes);
}
int getMaxSpeed() const {
int getMaxDownloadSpeed() const {
return downloadSpeed.getMaxSpeed();
}
int getAvgDownloadSpeed() const {
return downloadSpeed.getAvgSpeed();
}
void reset() {
downloadSpeed.reset();
downloadStartTime.reset();
@ -72,6 +77,10 @@ public:
status = IDLE;
}
void requestIdle() {
status = REQUEST_IDLE;
}
const Time& getDownloadStartTime() const {
return downloadStartTime;
}

97
src/SegmentMan.cc
View File

@ -115,7 +115,7 @@ void SegmentMan::save() const {
}
for(SegmentEntries::const_iterator itr = usedSegmentEntries.begin();
itr != usedSegmentEntries.end(); itr++) {
if(fwrite(&itr->segment, sizeof(Segment), 1, segFile) < 1) {
if(fwrite(&(*itr)->segment, sizeof(Segment), 1, segFile) < 1) {
throw string("writeError");
}
}
@ -170,7 +170,7 @@ void SegmentMan::read(FILE* file) {
if(fread(&seg, sizeof(Segment), 1, file) < 1) {
throw string("readError");
}
usedSegmentEntries.push_back(SegmentEntry(0, seg));
usedSegmentEntries.push_back(SegmentEntryHandle(new SegmentEntry(0, seg)));
}
}
@ -209,6 +209,7 @@ void SegmentMan::init() {
//segments.clear();
usedSegmentEntries.clear();
delete bitfield;
bitfield = 0;
peerStats.clear();
diskWriter->closeFile();
@ -224,8 +225,8 @@ private:
public:
FindSegmentEntryByIndex(int index):index(index) {}
bool operator()(const SegmentEntry& entry) {
return entry.segment.index == index;
bool operator()(const SegmentEntryHandle& entry) {
return entry->segment.index == index;
}
};
@ -235,8 +236,8 @@ private:
public:
FindSegmentEntryByCuid(int cuid):cuid(cuid) {}
bool operator()(const SegmentEntry& entry) {
return entry.cuid == cuid;
bool operator()(const SegmentEntryHandle& entry) {
return entry->cuid == cuid;
}
};
@ -251,11 +252,12 @@ Segment SegmentMan::checkoutSegment(int cuid, int index) {
if(itr == usedSegmentEntries.end()) {
segment = Segment(index, bitfield->getBlockLength(index),
bitfield->getBlockLength());
SegmentEntry entry(cuid, segment);
SegmentEntryHandle entry =
SegmentEntryHandle(new SegmentEntry(cuid, segment));
usedSegmentEntries.push_back(entry);
} else {
(*itr).cuid = cuid;
segment = (*itr).segment;
(*itr)->cuid = cuid;
segment = (*itr)->segment;
}
logger->debug("index=%d, length=%d, segmentLength=%d, writtenLength=%d",
@ -267,35 +269,76 @@ Segment SegmentMan::checkoutSegment(int cuid, int index) {
bool SegmentMan::onNullBitfield(Segment& segment, int cuid) {
if(usedSegmentEntries.size() == 0) {
segment = Segment(0, 0, 0);
usedSegmentEntries.push_back(SegmentEntry(cuid, segment));
usedSegmentEntries.push_back(SegmentEntryHandle(new SegmentEntry(cuid, segment)));
return true;
} else {
SegmentEntries::iterator uitr = find_if(usedSegmentEntries.begin(),
usedSegmentEntries.end(),
FindSegmentEntryByCuid(cuid));
if(uitr == usedSegmentEntries.end()) {
SegmentEntries::iterator itr = find_if(usedSegmentEntries.begin(),
usedSegmentEntries.end(),
FindSegmentEntryByCuid(cuid));
if(itr == usedSegmentEntries.end()) {
return false;
} else {
segment = uitr->segment;
segment = (*itr)->segment;
return true;
}
}
}
SegmentEntryHandle SegmentMan::findSlowerSegmentEntry(const PeerStatHandle& peerStat) const {
int speed = (int)(peerStat->getAvgDownloadSpeed()*0.8);
SegmentEntryHandle slowSegmentEntry(0);
for(SegmentEntries::const_iterator itr = usedSegmentEntries.begin();
itr != usedSegmentEntries.end(); itr++) {
const SegmentEntryHandle& segmentEntry = *itr;
if(segmentEntry->cuid == 0) {
continue;
}
PeerStatHandle p = getPeerStat(segmentEntry->cuid);
if(!p.get() || p->getCuid() == peerStat->getCuid()) {
continue;
}
int pSpeed = p->calculateDownloadSpeed();
if(p->getStatus() == PeerStat::ACTIVE &&
p->getDownloadStartTime().elapsed(option->getAsInt(PREF_STARTUP_IDLE_TIME)) &&
pSpeed < speed) {
speed = pSpeed;
slowSegmentEntry = segmentEntry;
}
}
return slowSegmentEntry;
}
bool SegmentMan::getSegment(Segment& segment, int cuid) {
if(!bitfield) {
return onNullBitfield(segment, cuid);
}
SegmentEntries::iterator uitr = find_if(usedSegmentEntries.begin(),
usedSegmentEntries.end(),
FindSegmentEntryByCuid(cuid));
if(uitr != usedSegmentEntries.end()) {
segment = uitr->segment;
SegmentEntries::iterator itr = find_if(usedSegmentEntries.begin(),
usedSegmentEntries.end(),
FindSegmentEntryByCuid(cuid));
if(itr != usedSegmentEntries.end()) {
segment = (*itr)->segment;
return true;
}
int index = bitfield->getSparseMissingUnusedIndex();
if(index == -1) {
return false;
PeerStatHandle myPeerStat = getPeerStat(cuid);
if(!myPeerStat.get()) {
return false;
}
SegmentEntryHandle slowSegmentEntry = findSlowerSegmentEntry(myPeerStat);
if(slowSegmentEntry.get()) {
logger->info("CUID#%d cancels segment index=%d. CUID#%d handles it instead.",
slowSegmentEntry->cuid,
slowSegmentEntry->segment.index,
cuid);
PeerStatHandle slowPeerStat = getPeerStat(slowSegmentEntry->cuid);
slowPeerStat->requestIdle();
cancelSegment(slowSegmentEntry->cuid);
segment = checkoutSegment(cuid, slowSegmentEntry->segment.index);
return true;
} else {
return false;
}
} else {
segment = checkoutSegment(cuid, index);
return true;
@ -327,7 +370,7 @@ bool SegmentMan::updateSegment(int cuid, const Segment& segment) {
if(itr == usedSegmentEntries.end()) {
return false;
} else {
(*itr).segment = segment;
(*itr)->segment = segment;
return true;
}
}
@ -340,10 +383,10 @@ public:
CancelSegment(int cuid, BitfieldMan* bitfield):cuid(cuid),
bitfield(bitfield) {}
void operator()(SegmentEntry& entry) {
if(entry.cuid == cuid) {
bitfield->unsetUseBit(entry.segment.index);
entry.cuid = 0;
void operator()(SegmentEntryHandle& entry) {
if(entry->cuid == cuid) {
bitfield->unsetUseBit(entry->segment.index);
entry->cuid = 0;
}
}
};
@ -394,7 +437,7 @@ long long int SegmentMan::getDownloadLength() const {
}
for(SegmentEntries::const_iterator itr = usedSegmentEntries.begin();
itr != usedSegmentEntries.end(); itr++) {
dlLength += itr->segment.writtenLength;
dlLength += (*itr)->segment.writtenLength;
}
return dlLength;
}

4
src/SegmentMan.h
View File

@ -45,7 +45,8 @@ public:
~SegmentEntry() {}
};
typedef deque<SegmentEntry> SegmentEntries;
typedef SharedHandle<SegmentEntry> SegmentEntryHandle;
typedef deque<SegmentEntryHandle> SegmentEntries;
typedef deque<PeerStatHandle> PeerStats;
/**
@ -62,6 +63,7 @@ private:
FILE* openSegFile(const string& segFilename, const string& mode) const;
bool onNullBitfield(Segment& segment, int cuid);
Segment checkoutSegment(int cuid, int index);
SegmentEntryHandle findSlowerSegmentEntry(const PeerStatHandle& peerStat) const;
public:
/**
* The total number of bytes to download.

13
src/SpeedCalc.cc
View File

@ -39,6 +39,8 @@ void SpeedCalc::reset() {
sw = 0;
maxSpeed = 0;
prevSpeed = 0;
start.reset();
accumulatedLength = 0;
}
int SpeedCalc::calculateSpeed() {
@ -65,6 +67,7 @@ public:
};
void SpeedCalc::update(int bytes) {
accumulatedLength += bytes;
for_each(&lengthArray[0], &lengthArray[2], Plus(bytes));
if(isIntervalOver()) {
changeSw();
@ -80,3 +83,13 @@ void SpeedCalc::changeSw() {
cpArray[sw].reset();
sw ^= 0x01;
}
int SpeedCalc::getAvgSpeed() const {
int milliElapsed = start.differenceInMillis();
if(milliElapsed) {
int speed = accumulatedLength*1000/milliElapsed;
return speed;
} else {
return 0;
}
}

4
src/SpeedCalc.h
View File

@ -32,6 +32,8 @@ private:
Time cpArray[2];
int maxSpeed;
int prevSpeed;
Time start;
long long int accumulatedLength;
bool isIntervalOver() const;
void changeSw();
@ -51,6 +53,8 @@ public:
return maxSpeed;
}
int getAvgSpeed() const;
void update(int bytes);
void reset();

1
src/main.cc
View File

@ -305,6 +305,7 @@ int main(int argc, char* argv[]) {
op->put(PREF_UPLOAD_LIMIT, "0");
op->put(PREF_LOWEST_SPEED_LIMIT, "0");
op->put(PREF_MAX_SPEED_LIMIT, "0");
op->put(PREF_STARTUP_IDLE_TIME, "10");
while(1) {
int optIndex = 0;
int lopt;

2
src/prefs.h
View File

@ -63,6 +63,8 @@
#define PREF_SEGMENT_SIZE "segment_size"
// value: 1*digit
#define PREF_MAX_SPEED_LIMIT "max_speed_limit"
// value: 1*digit
#define PREF_STARTUP_IDLE_TIME "startup_idle_time"
/**
* FTP related preferences