Origin: https://github.com/lenrico/fail2ban
Squashing was done via rebase -i 1524b076d6
to eliminate massive assp sample log file originally added
fixed test date thx to steven
tight control of the filter for ASSP
as yaroslav wishes
as daniel desires
changed from DateASSPlike class to DateStrptime
fixed little things
added new date format support for ASSP SMTP Proxy
* 'master' of git://github.com/fail2ban/fail2ban:
BF: log error only if there were missed config files that couldn't be read
DOC: missing cinfo tags are ok. Log error for self referencing definitions
DOC: s/defination/definition/g learn to spell
ENH: remove stats of config files and use results of SafeConfigParserWithIncludes.read to facilitate meaningful error messages
DOC: ChangeLog for recursive tag substition
ENH: allow recursive tag substitution in action files.
is None is generally faster than == and from looking at those places
should be adequate.
Also while at those files removed unneded duplicate author listing +
expanded copyright/authors with myself where applicable
* 'Support_for_mysql_log_example' of https://github.com/arto-p/fail2ban:
Added testcase for MySQL date format to testcases/datedetectortestcase.py and example of MySQL log file.
Added support for MySQL logfiles
Conflicts:
testcases/datedetectortestcase.py -- conflictde with other added test cases
* 001-fail2ban-server-socket-close-on-exec-no-leak.diff
Add code that marks server and client sockets with FD_CLOEXEC flags.
Avoid leaking file descriptors to processes spawned when handling
fail2ban actions (ex: iptables).
Unix sockets managed by fail2ban-server don't need to be passed to any
child process. Fail2ban already uses the FD_CLOEXEC flags in the filter
code.
This patch also avoids giving iptables access to fail2ban UNIX socket in
a SELinux environment (A sane SELinux policy should trigger an audit
event because "iptables" will be given read/write access to the fail2ban
control socket).
Some random references related to this bug:
http://sourceforge.net/tracker/?func=detail&atid=689044&aid=2086568&group_id=121032http://www.redhat.com/archives/fedora-selinux-list/2009-June/msg00124.htmlhttp://forums.fedoraforum.org/showthread.php?t=234230
* 002-fail2ban-filters-close-on-exec-typo-fix.diff
There is a typo in the fail2ban server/filter.py source code. The
FD_CLOEXEC is correctly set but additional *random* flags are also set.
It has no side-effect as long as the fd doesn't match a valid flag :)
"fcntl.fcntl(fd, fcntl.F_SETFD, fd | fcntl.FD_CLOEXEC)" <== the 3rd
parameter should be flags, not a file descriptor.
* 003-fail2ban-gamin-socket-close-on-exec-no-leak.diff
Add code that marks the Gamin monitor file descriptor with FD_CLOEXEC
flags. Avoid leaking file descriptors to processes spawned when handling
fail2ban actions (ex: iptables).
---
File descriptors in action process before patches:
dr-x------ 2 root root 0 .
dr-xr-xr-x 8 root root 0 ..
lr-x------ 1 root root 64 0 -> /dev/null <== OK
l-wx------ 1 root root 64 1 -> /tmp/test.log <== used by test action
lrwx------ 1 root root 64 2 -> /dev/null <== OK
lrwx------ 1 root root 64 3 -> socket:[116361] <== NOK (fail2ban.sock leak)
lr-x------ 1 root root 64 4 -> /proc/20090/fd <== used by test action
l-wx------ 1 root root 64 5 -> /var/log/fail2ban.log <== OK
lrwx------ 1 root root 64 6 -> socket:[115608] <== NOK (gamin sock leak)
File descriptors in action process after patches:
dr-x------ 2 root root 0 .
dr-xr-xr-x 8 root root 0 ..
lr-x------ 1 root root 64 0 -> /dev/null <== OK
l-wx------ 1 root root 64 1 -> /tmp/test.log <== used by test action
lrwx------ 1 root root 64 2 -> /dev/null <== OK
lr-x------ 1 root root 64 3 -> /proc/18284/fd <== used by test action
l-wx------ 1 root root 64 5 -> /var/log/fail2ban.log <== OK
Now it should be evaluated only if logging mode is debug or "lower", and also
should happen not only on initial detections but for every added failure.
It also reports counts per each IP now
* 'master' of https://github.com/sebres/fail2ban:
invalid date recognition, irregular because of sorting template list, now via setRegex
invalid date recognition, irregular because of sorting template list (sometimes not reproducible by fail2ban-regex, cause will be not sorted)
* 'datedetector-testcase' of https://github.com/kwirk/fail2ban:
Correct datedetector testcases for TAI64N format
Additional date/time formats for datedetector testcases
Also made a note that some workarounds for certain date/time formats
may cause week/year day to be incorrect. This does not affect
functionality of fail2ban so no urgency to fix.
date in following log line (from nginx) will be wrong detected:
2012/10/11 02:37:17 [error] 18434#0: *947 user "test" was not found in "/www/...", client: 192.168.1.5, ...
sometimes it is [correct] - 2012/10/11 (%Y/%m/%d) = 1349919861.71
sometimes it is [invalid] - 12/10/11 (%d/%m/%y) = 1349915838.06
and older as now - 1800 seconds (therefore will be not found)
solution: regular expression fixed, cause date in log used always after non symbol (\W) character, so r"\d{2}/\d{2}/\d{2}" will be r"(?<!\w)\d{2}/\d{2}/\d{2}".
Before, it would first do stable sort followed with explicit reverse.
Now reverse is given as an argument to sort, and it results in actually
preserving the order in case of e.g. no sorting needed
Otherwise it might lead to spurious characters dumped into the
terminal at few places, unless casted upon every use in the logs. To
prevent those issues in the source, store IP as a basic string if it
is a string at all
* all of the *Filters had too much of common logic in their *LogPath
methods, which is now handled by FileFilter and derived classes only
add custom actions in corresponding _(add|del)LogPath methods
pyinotify:
* upon CREATE event:
- unknown files should not be handled at all
- "watcher" for the monitored files should be recreated.
Lead to adding _(add|del)FileWatcher helper methods
* callback now obtains full event to judge what to do
although %m-%d-%Y is ambioius with %d-%m-%Y it comes after so it should not
be too dangerous (i.e. in upcoming days having smth like 02-01 should work as before matching
first one first) and proper fix to select between the two should follow some time
soon
following commits were squashed from feature branch use_dns
commit 068c105eb5
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 22:19:04 2012 -0500
Prevent warning when IP is read from log
commit 635ed36a8c
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 22:17:08 2012 -0500
Removed logDebug
commit 24656d2812
Merge: 7957fbec429f5c
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 21:13:11 2012 -0500
Merge branch 'enh/use_dns' of github:leeclemens/fail2ban into enh/use_dns
Conflicts:
testcases/filtertestcase.py
commit 7957fbe821
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 21:09:58 2012 -0500
filtertestcase fixes from yarikoptic
commit 6ce9d04640
Author: Yaroslav Halchenko <debian@onerussian.com>
Date: Tue Jan 10 19:26:05 2012 -0500
RF: for consistency use_dns -> usedns
I guess it was might fault of inconsistency suggesting that name.
Other options/commands do not have _ in the names, so let it be
consistent with the rest for now
commit cfb2c75b49
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 19:18:41 2012 -0500
Updated DNSUtilsTests to test use_dns and added positive test to testTextToIp
commit f6186eff14
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 19:02:04 2012 -0500
Changed wording of 'DNS Reverse lookup used' message
commit 82c62d29dc
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 18:53:17 2012 -0500
Removed extraneous "n"
commit dc0ae21932
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 23:07:59 2012 -0500
ENH: use_dns - removed debugging statements
commit 594e25818c
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 22:53:39 2012 -0500
Added use_dns protocol to set and get per jail during runtime
commit 48ff80ffac
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 22:41:18 2012 -0500
Completed use_dns for initial startup - with debugging statements
commit 0bdab4c2d7
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 20:05:35 2012 -0500
ENH: Added use_dns option
commit 6d6b734ea5
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 20:01:34 2012 -0500
ENH: Added use_dns option
commit 11ad2b6125
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 19:17:30 2012 -0500
Added useDns flag to testcase
commit b48fa9b6af
Author: Lee Clemens <java@leeclemens.net>
Date: Sun Jan 8 15:13:27 2012 -0500
Added use_dns option in jail.conf
commit c429f5c91a
Merge: 4b18afb0021906
Author: leeclemens <java@leeclemens.net>
Date: Tue Jan 10 16:32:22 2012 -0800
Merge pull request #3 from yarikoptic/enh/use_dns
let's be consistent ;-)
commit 0021906358
Author: Yaroslav Halchenko <debian@onerussian.com>
Date: Tue Jan 10 19:26:05 2012 -0500
RF: for consistency use_dns -> usedns
I guess it was might fault of inconsistency suggesting that name.
Other options/commands do not have _ in the names, so let it be
consistent with the rest for now
commit 4b18afb28a
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 19:18:41 2012 -0500
Updated DNSUtilsTests to test use_dns and added positive test to testTextToIp
commit 4fae37e46f
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 19:02:04 2012 -0500
Changed wording of 'DNS Reverse lookup used' message
commit e94806ce48
Author: Lee Clemens <java@leeclemens.net>
Date: Tue Jan 10 18:53:17 2012 -0500
Removed extraneous "n"
commit 4d30c52907
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 23:07:59 2012 -0500
ENH: use_dns - removed debugging statements
commit 76696d452a
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 22:53:39 2012 -0500
Added use_dns protocol to set and get per jail during runtime
commit 0631618087
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 22:41:18 2012 -0500
Completed use_dns for initial startup - with debugging statements
commit d23d495547
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 20:05:35 2012 -0500
ENH: Added use_dns option
commit 9538553bc5
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 20:01:34 2012 -0500
ENH: Added use_dns option
commit ae1e857e53
Author: Lee Clemens <java@leeclemens.net>
Date: Mon Jan 9 19:17:30 2012 -0500
Added useDns flag to testcase
commit ace43eb941
Author: Lee Clemens <java@leeclemens.net>
Date: Sun Jan 8 15:13:27 2012 -0500
Added use_dns option in jail.conf
should now be
- easier to comprehend
- more scalable happens new backends come
- more informative with all the info and debug messages
NB not sure why __setBackend was instance's attribute, now there is a
method _setBackend
Yaroslav Halchenko