Previously only the last line of the match was being saved, not all
lines involved in matching.
Log lines are now broken into 3 part tuple, with the line pre-datetime,
the datetime, and post-datetime. Allows reformation of full line, but
also use of the line without the datetime present.
Attempting to use the term "tupleLine(s)" where possible, to avoid
confusion with normal read lines.
May also wish to consider that regexs could be made to capture more
lines of interest if some form of unique reference is available. This
may allow more lines of interest to be captured, which may not be picked
up by the traditional "grep <ip>" approach i.e. ones which do not have
the ip address in.
This also simplified the fail2ban-regex statistics for missed lines.
Also resolved bug with missed lines time extracted for debuggex having
some lines present which were captured in a multiline regex.
Also resolved independent issue with ignored line check including the
datetime, which raised assertion error in the rare case the datetime
matched the ignore regex, and the rest of line only matched a failregex
* commit '0.8.10-31-g1ab0f0f': (24 commits)
BF/ENH: Incorrect authentication data doesn't need tailier so that's optional. Also gained log entry for Unrouteable address
ENH: readibility thanks to Yaroslav
DOC: Changelog for fail2ban-regex RF
DOC: Changelog for asterisk hardening
ENH: fail2ban-regex -- add specification of loglevels to enable
RF: reworked -regex cmdline tool to use optparse, some unification and enhancement of outputs
ENH: 'heavydebug' level == 5 for even more debugging in tricky cases
ENH: asterisk -- use \S instead of [^:] + prefix failregex with ^\[
BF: missed a space
BF: [SSL-out] is optional in assp
ENH: regex hardening on assp
ENH: anchor a bit mor. Use \d and \w where possible. Escape a literal .
TST: attempts at injection with username=rhost=1.2.3.4 have no user= logged in dovecot-1.2.15
ENH: proftpd chan accept usernames with spaces
ENH: injection of fail data into USER field
ENH: dovecot regexs rewritten and extra failures
ENH: proftp regex hardening and log messages
ENH/BF: exim improvements with sample
BF: fix to proxy port in 3proxy example
ENH: sample log + more specific regex
...
Conflicts: -- it was a messy merge/resolution.
ChangeLog
bin/fail2ban-regex
fail2ban-testcases
fail2ban/server/filter.py
This issue was caused by the fact that every "line" is processed for each
regex, meaning each line was duplicated for every regex. This caused
duplicate fail matches and the buffer filling too quickly and possibly
missing failures.
Hank Leininger