Commit Graph

306 Commits (d583637c506439ecb738b595ed232cefabef67a9)

Author SHA1 Message Date
Daniel Black caf284d518 DOC: ChangeLog deconflict 2013-10-02 09:11:15 +10:00
Steven Hiscocks a8f2448349 ENH: Allow SE Linux epoch date detection 2013-09-30 20:58:24 +01:00
Daniel Black 6224d761ab BF: simplify 2013-09-29 15:49:45 +10:00
Daniel Black 4d1c060e21 ENH: dont execute empty commands 2013-09-29 13:14:52 +10:00
Daniel Black 4b5ecbccd1 ENH: debuggex URLs with fail2ban-regex 2013-09-22 13:20:17 +10:00
Yaroslav Halchenko f1adf75b59 ENH: basic testing for iso8601 code which had no explicit tests + spit out ValueError for incorrect type of input and ParseError otherwise 2013-09-12 23:12:18 -04:00
Daniel Black b589533d69 Merge branch 'master' into kwirk-merge
Conflicts:
	ChangeLog
	testcases/files/logs/dropbear
2013-08-25 21:21:14 +10:00
Daniel Black ced271b908 ENH: date for apache-2.4 - adds milliseconds 2013-08-25 21:11:47 +10:00
Yaroslav Halchenko c84a2e595a ENH(BF): put 'standard' template after more detailed ones with day of week and year
otherwise years present in the freshly contributed by Dan apache regexes do not match
although should have.  I had also to adjust failing now vsftpd test
2013-08-25 17:52:12 +10:00
Yaroslav Halchenko c0456fd835 BF: Fixing a name for MySQL date pattern which misplaced Year (should not effect functionality) Closes #312 2013-08-08 09:47:47 -04:00
Steven Hiscocks 1e270078b4 TST: Warn if date templates overlap in default detectors 2013-07-27 20:21:05 +01:00
Yaroslav Halchenko 3b52eca608 ENH+TST: Ticket -- drop unused/bogus get|setFile + enh __str__ + basic testing 2013-07-22 12:09:33 -04:00
Steven Hiscocks bf05f2ac95 Merge branch 'filter-failregex-return'
Conflicts:
	server/filter.py
2013-07-16 21:17:18 +01:00
Yaroslav Halchenko 0a02cfe9e8 ENH: <HOST> must end with alphanumeric \w (not a dot or a dash etc)
Otherwise <HOST> regexp might swallow period in the sentence right after the address.
I have decided to enforce alphanumeric instead of switching to non-greedy +? ... because
I think it is closer to what we actually want here
2013-07-16 15:03:06 -04:00
Yaroslav Halchenko 148cbd8d2a ENH: heavier debugging -- log split date/log line even for no match. Log matching regex upon match 2013-07-16 14:16:23 -04:00
Steven Hiscocks 1a2b6442a0 ENH+BF+TST: Filter now returns reference to failregex and ignoreregex
This avoids duplication of code across fail2ban-regex and samples test
cases. This also now more neatly resolves the issue of double counting
date templates matches in fail2ban-regex.
In addition, the samples test cases now also print a warning message
that not all regexs have samples for them, with future plan to change
this to an assertion.
2013-07-15 22:22:13 +01:00
Steven Hiscocks 40f67c64b8 TST: Test sample logs' entries are matched by filter regexs 2013-07-13 23:03:01 +01:00
Yaroslav Halchenko e282d6b1c7 ENH: Remove unused any longer _ctime helper 2013-07-03 00:09:39 -04:00
Yaroslav Halchenko 5df6796e69 ENH: DNS resolution -- catch parent exception
IMHO there is no good reason to capture only gaierror.

on my network it was consistent to error out with

======================================================================
ERROR: testIgnoreIPNOK (testcases.filtertestcase.IgnoreIP)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/deb/gits/fail2ban/testcases/filtertestcase.py", line 166, in testIgnoreIPNOK
    self.assertFalse(self.filter.inIgnoreIPList(ip))
  File "/home/yoh/deb/gits/fail2ban/server/filter.py", line 277, in inIgnoreIPList
    ips = DNSUtils.dnsToIp(i)
  File "/home/yoh/deb/gits/fail2ban/server/filter.py", line 625, in dnsToIp
    return socket.gethostbyname_ex(dns)[2]
error: [Errno 11] Resource temporarily unavailable

with this commit tests would pass normally as they should
2013-07-02 23:51:09 -04:00
Yaroslav Halchenko 8d25e83f5d BF: race condition -- file should not be read unless it is not empty
Previous code would store md5sum of an empty line as the one
identifying the monitored file.  That file then was read and possibly
failures were found.  Upon next "container.open()", md5 digest of now
present first line was compared against previous digest of an empty
line, which was different, thus file was assumed to be rotated and all
the log lines were read once again.

The History
-----------
In rare cases various tests failed somewhat consistently.  Below you
can find one case in test_move_file where such failure really made no
sense -- we should have not had 4 failures by that point.

Fail2ban 0.8.10.dev test suite. Python 2.4.6 (#2, Sep 25 2009, 22:22:06) [GCC 4.3.4]. Please wait...
I: Skipping gamin backend testing. Got exception 'No module named gamin'
I: Skipping pyinotify backend testing. Got exception 'No module named pyinotify'
D: fn=/home/yoh/test/monitorfailures_FilterPoll9nUKoCfail2ban-0 hashes=d41d8cd98f00b204e9800998ecf8427e/d41d8cd98f00b204e9800998ecf8427e inos=5398862/5398862 pos=0 rotate=False
D: Adding a ticket for ('193.168.0.128', 1124013599.0, ['Aug 14 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 193.168.0.128\n'])
D: Adding a ticket for ('193.168.0.128', 1124013599.0, ['Aug 14 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 193.168.0.128\n'])
D: Closed <closed file '/home/yoh/test/monitorfailures_FilterPoll9nUKoCfail2ban-0', mode 'r' at 0x7f472b2efdc8> with pos 1231
D: fn=/home/yoh/test/monitorfailures_FilterPoll9nUKoCfail2ban-0 hashes=d41d8cd98f00b204e9800998ecf8427e/aeb4e73e6922a746d027eb365ece2149 inos=5398862/5398862 pos=1231 rotate=True
D: Adding a ticket for ('193.168.0.128', 1124013599.0, ['Aug 14 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 193.168.0.128\n'])
D: Adding a ticket for ('193.168.0.128', 1124013599.0, ['Aug 14 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 193.168.0.128\n'])
D: Closed <closed file '/home/yoh/test/monitorfailures_FilterPoll9nUKoCfail2ban-0', mode 'r' at 0x7f472b2ef558> with pos 1231
F
======================================================================
FAIL: test_move_file (testcases.filtertestcase.MonitorFailures<FilterPoll>(/home/yoh/test/monitorfailures_FilterPoll9nUKoCfail2ban))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/deb/gits/fail2ban/testcases/filtertestcase.py", line 451, in test_move_file
    "Queue must be empty but it is not: %s."
AssertionError: Queue must be empty but it is not: server.ticket.FailTicket: ip=193.168.0.128 time=1124013599.0 #attempts=4.

----------------------------------------------------------------------

N.B.1 I preserved here and in the code corresponding additional debug
print statements, which are commented out by default. sensible md5
digest was generated by using hexdigest() instead of current a bit
faster digest().  Running tests with all the debug output simply
breaks the race loose and failure doesn't trigger.

N.B.2 d41d8cd98f00b204e9800998ecf8427e is an md5sum of an empty
string, and aeb4e73e6922a746d027eb365ece2149 of the first line in that
file.
2013-07-02 23:37:23 -04:00
Yaroslav Halchenko f340e5c0f5 ENH(+possibly BF): for FilterPoll rely not only on mtime but also ino and size to assess if file was modified
mtime alone is a poor measure here as many of our tests shown -- on older Pythons and some file systems
mtime might be reported only with precision up to a second.  If file gets rotated fast, or there are new
modifications within the same second, fail2ban might miss them with Polling backend if judging only by
mtime.  With this modification we will track also inode and size which are all indicative of a file
change.
2013-07-02 20:43:51 -04:00
Yaroslav Halchenko 2ffc143597 ENH: more of heavydebug'ing for FilterPoll 2013-07-02 19:49:41 -04:00
Yaroslav Halchenko 591590860a BF: setSleepTime -- would barf since value is not str (wasn't used/tested) 2013-07-02 17:11:24 -04:00
Yaroslav Halchenko 97f9cfc0b0 ENH: 'heavydebug' level == 5 for even more debugging in tricky cases
I mocked logging library directly -- seems to be Ok.
2013-06-13 23:19:28 -04:00
Yaroslav Halchenko 21474884e0 ENH: now we know that logging handlers closing was still buggy in 2.6.2 2013-05-12 22:55:02 -04:00
Yaroslav Halchenko 571ff33fde ENH: issue a warning if jail name is longer than 19 symbols (Close #222) 2013-05-12 22:19:50 -04:00
Yaroslav Halchenko 14f92d9144 Merge pull request #225 from yarikoptic/master
Additional check that pyinotify is functional on this platform to enable it
2013-05-12 17:24:17 -07:00
Steven Hiscocks 8af3ffb332 BF: Fix for filterpoll incorrectly checking for jailless state 2013-05-10 16:54:53 +01:00
Yaroslav Halchenko 8161038987 ENH: strengthen detection of working pyinotify
Even though import might work -- pyinotify might be dysfunctional.
Check by creating/deleting a dummy WatchManager upon import
2013-05-10 11:40:12 -04:00
Yaroslav Halchenko 90d6a4a6cd ENH: consistent operation of formatExceptionInfo + unittest for it 2013-05-09 22:46:59 -04:00
Yaroslav Halchenko 6fef85ff2d ENH: strip CR and LF while analyzing the lines (processLine) (Close #202)
This should allow to resolve issues with logs written in MS-DOS fashion,
e.g. with daemontools

See https://github.com/fail2ban/fail2ban/issues/202\#issuecomment-17393613
2013-05-08 12:07:29 -04:00
Daniel Black 495f2dd877 DOC: purge of svn tags 2013-05-03 16:03:38 +10:00
Yaroslav Halchenko 89adcd7ff7 Merge branch PR #193 ASSP SMTP Proxy support (with some manual squashing)
Origin: https://github.com/lenrico/fail2ban

Squashing was done via rebase -i 1524b076d6
to eliminate massive assp sample log file originally added

  fixed test date thx to steven
  tight control of the filter for ASSP
  as yaroslav wishes
  as daniel desires
  changed from DateASSPlike class to DateStrptime
  fixed little things
  added new date format support for ASSP SMTP Proxy
2013-05-03 00:57:49 -04:00
Enrico Labedzki e27385e873 as yaroslav wishes 2013-05-03 00:56:53 -04:00
Enrico Labedzki 9185c070eb changed from DateASSPlike class to DateStrptime 2013-05-03 00:56:52 -04:00
Enrico Labedzki ba8f012637 fixed little things 2013-05-03 00:56:52 -04:00
Enrico Labedzki 24a8d07c20 added new date format support for ASSP SMTP Proxy 2013-05-03 00:56:46 -04:00
Yaroslav Halchenko 7c409dd24f Merge branch 'master' of git://github.com/fail2ban/fail2ban
* 'master' of git://github.com/fail2ban/fail2ban:
  BF: log error only if there were missed config files that couldn't be read
  DOC: missing cinfo tags are ok. Log error for self referencing definitions
  DOC: s/defination/definition/g learn to spell
  ENH: remove stats of config files and use results of SafeConfigParserWithIncludes.read to facilitate meaningful error messages
  DOC: ChangeLog for recursive tag substition
  ENH: allow recursive tag substitution in action files.
2013-05-02 23:28:18 -04:00
Yaroslav Halchenko 8e63d4c6da ENH: "is None" instead of "== None" + tune ups in headers
is None is generally faster than == and from looking at those places
should be adequate.

Also while at those files removed unneded duplicate author listing +
expanded copyright/authors with myself where applicable
2013-05-02 23:25:43 -04:00
Daniel Black d7862266d6 DOC: missing cinfo tags are ok. Log error for self referencing definitions 2013-04-30 08:14:50 +10:00
Daniel Black d28f3fa285 DOC: s/defination/definition/g learn to spell 2013-04-30 08:07:21 +10:00
Yaroslav Halchenko f21566049c BF: pyinotify backend should also handle IN_MOVED_TO events 2013-04-29 13:54:14 -04:00
Daniel Black 1d9abd1b39 ENH: allow recursive tag substitution in action files. 2013-04-29 12:37:16 +10:00
Steven Hiscocks 3d6791fe3e ENH: Minor change to action for consistency of execStart/Stop 2013-04-14 15:57:37 +01:00
Steven Hiscocks 28e9acf86a TST: no cover additions to server, primarily daemon creation 2013-04-14 15:55:18 +01:00
Yaroslav Halchenko ffe48741e3 DOC: thanks @kwirk for spotting the typos in exception message 2013-04-13 22:20:57 -04:00
Yaroslav Halchenko 301460f451 Merge remote-tracking branch 'pr/167/head': FD_CLOEXEC bug fixes (filters) + support (actions). Avoid sockets descriptors leak.
* pr/167/head:
  FD_CLOEXEC support
2013-04-11 15:05:56 -04:00
Yaroslav Halchenko 3e6be243bf Merge branch 'Support_for_mysql_log_example' of https://github.com/arto-p/fail2ban
* 'Support_for_mysql_log_example' of https://github.com/arto-p/fail2ban:
  Added testcase for MySQL date format to testcases/datedetectortestcase.py and example of MySQL log file.
  Added support for MySQL logfiles

Conflicts:
	testcases/datedetectortestcase.py -- conflictde with other added test cases
2013-04-09 17:55:14 -04:00
Nicolas Collignon 39667ff6f7 FD_CLOEXEC support
* 001-fail2ban-server-socket-close-on-exec-no-leak.diff

Add code that marks server and client sockets with FD_CLOEXEC flags.
Avoid leaking file descriptors to processes spawned when handling
fail2ban actions (ex: iptables).

Unix sockets managed by fail2ban-server don't need to be passed to any
child process. Fail2ban already uses the FD_CLOEXEC flags in the filter
code.

This patch also avoids giving iptables access to fail2ban UNIX socket in
a SELinux environment (A sane SELinux policy should trigger an audit
event because "iptables" will be given read/write access to the fail2ban
control socket).

Some random references related to this bug:
 http://sourceforge.net/tracker/?func=detail&atid=689044&aid=2086568&group_id=121032
 http://www.redhat.com/archives/fedora-selinux-list/2009-June/msg00124.html
 http://forums.fedoraforum.org/showthread.php?t=234230

 * 002-fail2ban-filters-close-on-exec-typo-fix.diff

There is a typo in the fail2ban server/filter.py source code. The
FD_CLOEXEC is correctly set but additional *random* flags are also set.
It has no side-effect as long as the fd doesn't match a valid flag :)
"fcntl.fcntl(fd, fcntl.F_SETFD, fd | fcntl.FD_CLOEXEC)" <== the 3rd
parameter should be flags, not a file descriptor.

 * 003-fail2ban-gamin-socket-close-on-exec-no-leak.diff

Add code that marks the Gamin monitor file descriptor with FD_CLOEXEC
flags. Avoid leaking file descriptors to processes spawned when handling
fail2ban actions (ex: iptables).

---

File descriptors in action process before patches:
dr-x------ 2 root root  0 .
dr-xr-xr-x 8 root root  0 ..
lr-x------ 1 root root 64 0 -> /dev/null        <== OK
l-wx------ 1 root root 64 1 -> /tmp/test.log    <== used by test action
lrwx------ 1 root root 64 2 -> /dev/null        <== OK
lrwx------ 1 root root 64 3 -> socket:[116361]  <== NOK (fail2ban.sock leak)
lr-x------ 1 root root 64 4 -> /proc/20090/fd   <== used by test action
l-wx------ 1 root root 64 5 -> /var/log/fail2ban.log <== OK
lrwx------ 1 root root 64 6 -> socket:[115608]  <== NOK (gamin sock leak)

File descriptors in action process after patches:
dr-x------ 2 root root  0 .
dr-xr-xr-x 8 root root  0 ..
lr-x------ 1 root root 64 0 -> /dev/null        <== OK
l-wx------ 1 root root 64 1 -> /tmp/test.log    <== used by test action
lrwx------ 1 root root 64 2 -> /dev/null        <== OK
lr-x------ 1 root root 64 3 -> /proc/18284/fd   <== used by test action
l-wx------ 1 root root 64 5 -> /var/log/fail2ban.log <== OK
2013-04-02 19:11:59 +02:00
Yaroslav Halchenko 33a31e096a RF+TST: bring inBanList back from private to protected and enabled its rudimentary unittests 2013-03-29 15:33:08 -04:00