normalize increment of ban-count or time (count increased in BanManager now, some dual increments fixed in the test-cases);
introduced new action-tag `<bancount>`, that is always incremented by each ban (starting by 1), opposite to tag `<bantime>` which can be prolonged retarded (up to 10 seconds)
Many times faster and fewer CPU-hungry because of parsing with `maxlines=1`, so without line buffering (scrolling of the buffer-window).
Combination of tags `<F-MLFID>` and `<F-NOFAIL>` can be used now to process multi-line logs using single-line expressions:
- tag `<F-MLFID>`: used to identify resp. store failure info for groups of log-lines with the same identifier (e. g. combined failure-info for the same conn-id by `<F-MLFID>(?:conn-id)</F-MLFID>`, see sshd.conf for example)
- tag `<F-NOFAIL>`: used as mark for no-failure (helper to accumulate common failure-info);
filter.d/sshd.conf: [sshd], [sshd-ddos], [sshd-aggressive] optimized with pre-filtering using new option `prefregex` and new multi-line handling.
- better ban/unban handling within actions (e.g. used dict instead of list)
- don't copy bans resp. its list on some operations;
- added new unbantime handling to relieve unBanList (prevent permanent searching for tickets to unban)
- prefer failure-ID as identifier of the ticket to its IP (most of the time the same, but it can be something else e.g. user name in some complex jails, as introduced in 0.10)
- better recognition for usage of textToIp, expected or raw value should be used;
- separated failure id vs. host (if found use `fid` instead of `host` resp. `ip`);
- additional optional groups may be used in tags replacement by executing actions;
- new options for "fail2ban-testcases" introduced: "-g" or "--no-gamin" and "-m" or "--memory-db", both are true also if "-f" or "--fast" specified,
for example: `fail2ban-testcases -ngm` will runs faster (because no network, no gamin, memory database)
but will use the same default sleep intervals as in production (in comparison to -nf);
- seekToTime rewritten, accuracy increased by seekToTime, extended for all FileFilter (PyInotify, Gamin also), test cases extended etc.
- common performance optimized and code reviewed;
-- cache dnsToIp, ipToName to prevent long wait during retrieving of ip/name for wrong dns or lazy dns-system;
-- instead of simple "sleep" used conditional wait "wait_for", that internal increases sleep interval up to sleeptime;
-- ticket / banmanager / failmanager modules are performance optimized;
-- api of filter (log files), jail, etc. rewritten and extended for performance purposes;
- performance of test cases optimized:
-- added option "--fast" to decrease wait intervals, avoid passive waiting, and skip few very slow test cases;
- code review after partially cherry pick of branch 'ban-time-incr' (see gh-716)
-- ticket module prepared to easy merge with newest version of 'ban-time-incr', now additionally holds banTime, banCount and json-data;
-- executeCmd partially moved from action to new module utils, etc.
-- python 2.6 compatibility;
- testExecuteTimeoutWithNastyChildren: test case repaired - wait for pid file inside bash, kill tree in any case (gh-1155);
- testSocket: test case repaired - wait for server thread starts a socket (listener)
Conflicts resolved:
ChangeLog
fail2ban/server/filter.py
fail2ban/server/jail.py
fail2ban/tests/actionstestcase.py
Test cases fixed:
testBanActionsAInfo - fail ticket with current time (otherwise ticket will be ignored - ban time too old)
testFail2BanExceptHook - use local sys.__excepthook__ to check was really executed and prevent write error in stderr.
"bantimeextra.enabled" in jail.conf allows to use database for searching of previously banned ip's to increase a default ban time using special formula,
by default, each next ban it will be original banTime * 1, 2, 4, 8, 16, 32...
see "jail.conf" for some other options of "bantimeextra";
additional we can configure a little randomization of ban time, to prevent "clever" botnets calculate exact time IP can be unbanned.
WARNING: by first start the server upgrades sqlite database (table "bans" will recreated with another schema);
* master: (51 commits)
ENH: Use real (resolving) example.com instead of test.example.com
DOC: Slight tune ups to ChangeLog -- we must release!
Changelog entries for the latest merges
BF: add bash-completion to MANIFEST
DOC: ChangeLog for default action type change
ENH: consolidate where blocktype is defined for iptables rules
BF: default type to unreachable
ENH: separate out regex and escape a .
ENH: logs/sshd -- have ":" after [daemon] (other uses are uncommon)
ENH: logs/sshd -- use example.com as the resolved hostname in sample log lines
ENH: filter.d/sshd.conf -- allow for trailing "via IP" in logs
DOC: Drop sudo from bash-completion
DOC: Added bash-completion script
ENH: add blocktype to all relevant actions. Also default the rejection to a ICMP reject rather than a drop
ENH: Removed unused log line
ENH: logrotate file
BF: missed MANIFEST include
BF: missed MANIFEST include
BF: missed MANIFEST include
ENH: some form of logrotate based on what distros are doing
...
Conflicts:
ChangeLog
MANIFEST
client/actionreader.py
config/jail.conf
fail2ban/server/datedetector.py
fail2ban/tests/datedetectortestcase.py
sebres