Merge commit '0.8.11.pre1-29-gccd2657' into debian

* commit '0.8.11.pre1-29-gccd2657': (363 commits)
  DOC: minor typos in ChangeLog
  DOC: adding DEV Notes for for non-greedy matchin within sshd.conf
  BF: disallow exploiting of non-greedy .* in previous fix by providing too long rhost -- do not impose length limits for user-provided input
  BF: fixing injection for OpenSSH 6.3 -- making .* before <HOST> non-greedy
  Changelog for prior changes (gen_buildbots)
  ENH: condense asterisk regexs for speed
  BF: missed action in nginx-http-auth
  ENH: add filter.d/nginx-http-auth. Partially forfills #405
  ENH: regenerated config/filter.d/apache-badbots.conf
  NF: gen_badbots script to (re)generate/update config/filter.d/apache-badbots.conf
  DOC: keeping Changelog release-phrases uniform, simplified intro, unified
  DOC: Untabifying and reindenting a bit ChangeLog
  DOC: few more links for DEVELOP
  BF: fix dovecot filter for newer failure message. Closes Debian bug #709324
  BF: fix to filter.d/wuftp to support pam authentication - Debian bug #665925
  Add Fedora git repo of fail2ban package to DEVELOP
  firewalld-0.3.8 release that support --remove-rules out so documenting this.
  BF: remove duplication definition secion in webmin-auth
  DOC: alter release notes a bit more and versions in README.md
  BF/DOC: fix hopefully final MANIFEST and release instructions
  ...

ChangeLog

@@ -4,9 +4,172 @@
                        |_| \__,_|_|_/___|_.__/\__,_|_||_|
 
 ================================================================================
-Fail2Ban (version 0.8.10)                                             2013/06/12
+Fail2Ban (version 0.8.11.pre1)                                        2013/10/30
 ================================================================================
 
+ver. 0.8.11 (2013/11/XXX) - loves-unittests-and-tight-DoS-free-filter-regexes
+-----------
+
+In light of CVE-2013-2178 that triggered our last release we have put
+a significant effort into tightening all of the regexs of our filters
+to avoid another similar vulnerability. All filters have been updated
+and some to catch more login/authentication failures and to support
+for newer application versions. There are test cases for most log
+cases of failures now.
+
+As usual, if you have other examples that demonstrate that a filter is
+insufficient, or if we have inadvertently introduced a regression,
+please provide us with example log lines on the github issue tracker
+http://github.com/fail2ban/fail2ban/issues and NOT on a random blog in
+some obscure corner of the Internet.
+
+- IMPORTANT incompatible changes:
+  Filter name changes:
+   * 'lighttpd-fastcgi' filter has been renamed to 'suhosin'
+   * 'sasl' has been renamed to 'postfix-sasl'
+   * 'exim' spam catching failregexes was split out into 'exim-spam'
+  These changes will require changing jail.{conf,local} if any of
+  those filters were used.
+
+- Fixes:
+  Daniel Black & Marcel Dopita
+   * filter.d/apache-auth -- fixed and apache auth samples provide. Closes gh-286
+  Yaroslav Halchenko
+   * filter.d/common.conf -- make colon after [daemon] optional. Closes gh-267
+   * filter.d/apache-common.conf -- support apache 2.4 more detailed error
+     log format.  Closes gh-268
+   * Backends changes detection and parsing. Close gh-223 and gh-103:
+     - Polling backend: detect changes in the files not only based on
+       mtime, but also on the size and inode.  It should allow for
+       better detection of changes and log rotations on busy servers,
+       older python 2.4, and file systems with precision of mtime only
+       up to a second (e.g. ext3).
+     - All backends, possible race condition: do not read from a file
+       initially reported empty.  Originally could have lead to
+       accounting for detected log lines multiple times.
+     - Do not crash if executing a command in fail2ban-client interactive
+       mode has failed (e.g. due to incorrect syntax). Closes gh-353
+  Daniel Black & Мернов Георгий
+   * filter.d/dovecot.conf -- Fix when no TLS enabled - line doesn't end in ,
+  Daniel Black & Georgiy Mernov & ftoppi & Мернов Георгий
+   * filter.d/exim.conf -- regex hardening and extra failure examples in
+     sample logs
+   * filter.d/named-refused.conf - BIND 9.9.3 regex changes
+  Daniel Black & Sebastian Arcus
+   * filter.d/asterisk -- more regexes
+  Daniel Black
+   * action.d/hostsdeny -- NOTE: new dependancy 'ed'. Switched to use 'ed' across
+     all platforms to ensure permissions are the same before and after a ban.
+     Closes gh-266. hostsdeny supports daemon_list now too.
+   * action.d/bsd-ipfw - action option unsed. Change blocktype to port unreach
+     instead of deny for consistancy.
+   * filter.d/dovecot - added to support different dovecot failure
+     "..disallowed plaintext auth". Closes Debian bug #709324
+   * filter.d/roundcube-auth - timezone offset can be positive or negative
+   * action.d/bsd-ipfw - action option unsed. Fixed to blocktype for
+     consistency. default to port unreach instead of deny
+   * filter.d/dropbear - fix regexs to match standard dropbear and the patched
+     http://www.unchartedbackwaters.co.uk/files/dropbear/dropbear-0.52.patch
+     and add PAM is it in dropbear-2013.60 source code.
+   * filter.d/{asterisk,assp,dovecot,proftpd}.conf -- regex hardening
+     and extra failure examples in sample logs
+   * filter.d/apache-auth - added expressions for mod_authz, mod_auth and
+     mod_auth_digest failures.
+   * filter.d/recidive -- support f2b syslog target and anchor regex at start
+   * filter.d/mysqld-auth.conf - mysql can use syslog
+   * filter.d/sshd - regex enhancements to support openssh-6.3. Closes Debian
+     bug #722970. Thanks Colin Watson for the regex analysis.
+   * filter.d/wuftpd - regex enhancements to support pam and wuftpd. Closes
+     Debian bug #665925
+  Rolf Fokkens
+   * action.d/dshield.conf and complain.conf -- reorder mailx arguments.
+     https://bugzilla.redhat.com/show_bug.cgi?id=998020
+  John Doe (ache)
+   * action.d/bsd-ipfw.conf - invert actionstop logic to make exist status 0.
+     Closes gh-343.
+  JP Espinosa (Reviewed by O.Poplawski)
+   * files/redhat-initd - rewritten to use stock init.d functions thus
+     avoiding problems with getpid.  Also $network and iptables moved
+     to Should- rc init fields
+  Rick Mellor
+   * filter.d/vsftp - fix capture with tty=ftp
+
+- New Features:
+  Edgar Hoch
+   * action.d/firewall-cmd-direct-new.conf - action for firewalld
+     from https://bugzilla.redhat.com/show_bug.cgi?id=979622
+     NOTE: requires firewalld-0.3.8+
+  Andy Fragen and Daniel Black
+   * filter.d/osx-ipfw.conf - ipfw action for OSX based on random rule
+     numbers.
+  Anonymous:
+   * action.d/osx-afctl - an action based on afctl for osx
+  Daniel Black & ykimon
+   * filter.d/3proxy.conf -- filter added
+   * fail2ban-regex - now generates http://www.debuggex.com urls for debugging
+     regular expressions with the -D parameter.
+  Daniel Black
+   * filter.d/exim-spam.conf -- a splitout of exim's spam regexes
+     with additions for greater control over filtering spam.
+   * add date expression for apache-2.4 - milliseconds
+   * filter.d/nginx-http-auth -- filter added for http basic authentication
+     failures in nginx. Partially fulfills gh-405.
+  Christophe Carles & Daniel Black
+   * filter.d/perdition.conf -- filter added
+  Mark McKinstry
+   * action.d/apf.conf - add action for Advanced Policy Firewall (apf)
+  Amir Caspi and kjohnsonecl
+   * filter.d/uwimap-auth - filter for uwimap-auth IMAP/POP server
+  Steven Hiscocks and Daniel Black
+   * filter.d/selinux-{common,ssh} -- add SELinux date and ssh filter
+
+- Enhancements:
+  François Boulogne and Frédéric
+   * filter.d/lighttpd - auth regexs for lighttpd-1.4.31
+  Daniel Black
+   * reorder parsing of jail.conf, jail.d/*.conf, jail.local, jail.d/*.local
+     and likewise for fail2ban.{conf|local|d/*.conf|d/*.local}. Closes gh-392
+   * jail.conf now has asterisk jail - no need for asterisk-tcp and
+     asterisk-udp. Users should replace existing jails with asterisk to
+     reduce duplicate parsing of the asterisk log file.
+   * filter.d/{suhosin,pam-generic,gssftpd,sogo-auth,webmin}- regex anchor at
+     start
+   * filter.d/vsftpd - anchored regex at start. disable old pam format regex
+   * filter.d/pam-generic - added syslog prefix. Disabled support for
+     linux-pam before version 0.99.2.0 (2005)
+   * filter.d/postfix-sasl - renamed from sasl, anchor at start and base on
+     syslog
+   * filter.d/qmail - rewrote regex to anchor at start. Added regex for
+     another "in the wild" patch to rblsmtp.
+  Yaroslav Halchenko
+   * fail2ban-regex -- refactored to provide more details (missing and
+     ignored lines, control over logging, etc) while maintaining look&feel
+   * fail2ban-client -- log to standard error. Closes gh-264
+   * Fail to configure if not a single log file was found for an
+     enabled jail. Closes gh-63
+   * <HOST> is now enforced to end with an alphanumeric
+   * filter.d/roundcube-auth.conf -- anchored version
+   * date matching - for standard asctime formats prefer more detailed
+     first (thus use year if available)
+   * files/gen_badbots was added and filter.d/apache-badbots.conf was
+     regenerated to get updated (although now still an old) list of
+     "bad" bots
+  Alexander Dietrich
+   * action.d/sendmail-common.conf -- added common sendmail settings file
+     and made the sender display name configurable
+  Steven Hiscocks
+   * filter.d/dovecot - Addition of session, time values and possible blank
+     user
+  Zurd and Daniel Black
+   * filter/named-refused - added refused on zone transfer
+   * filter.d/{courier{login,smtp},proftpd,sieve,wuftpd,xinetd} - General
+     regex impovements
+  Zurd
+   * filter.d/postfix - add filter for VRFY failures. Closes gh-322.
+  Orion Poplawski
+   * fail2ban.d/ and jail.d/ directories are added to etc/fail2ban to facilitate
+     their use
+
 ver. 0.8.10 (2013/06/12) - wanna-be-secure
 -----------
 

DEVELOP

@@ -1,6 +1,6 @@
-                         __      _ _ ___ _               
+                         __      _ _ ___ _
-                        / _|__ _(_) |_  ) |__  __ _ _ _  
+                        / _|__ _(_) |_  ) |__  __ _ _ _
-                       |  _/ _` | | |/ /| '_ \/ _` | ' \ 
+                       |  _/ _` | | |/ /| '_ \/ _` | ' \
                        |_| \__,_|_|_/___|_.__/\__,_|_||_|
 
 ================================================================================
@@ -26,7 +26,7 @@ Pull Requests
 
 When submitting pull requests on GitHub we ask you to:
 * Clearly describe the problem you're solving;
-* Don't introduce regressions that will make it hard for systems adminstrators 
+* Don't introduce regressions that will make it hard for systems administrators
   to update;
 * If adding a major feature rebase your changes on master and get to a single commit;
 * Include test cases (see below);
@@ -37,12 +37,383 @@ When submitting pull requests on GitHub we ask you to:
 Filters
 =======
 
-* Include sample logs with 1.2.3.4 used for IP addresses and 
+Filters are tricky. They need to:
-  example.com/example.org used for DNS names
+* work with a variety of the versions of the software that generates the logs;
-* Ensure ./fail2ban-regex testcases/files/logs/{samplelog} config/filter.d/{filter}.conf
+* work with the range of logging configuration options available in the
-  has matches for EVERY regex
+  software;
-* Ensure regexs end with a $ and are restrictive as possible. E.g. not .* if
+* work with multiple operating systems;
- [0-9]+ is sufficient
+* not make assumptions about the log format in excess of the software
+  (e.g. do not assume a username doesn't contain spaces and use \S+ unless
+  you've checked the source code);
+* account for how future versions of the software will log messages
+  (e.g. guess what would happen to the log message if different authentication
+  types are added);
+* not be susceptible to DoS vulnerabilities (see Filter Security below); and
+* match intended log lines only.
+
+Please follow the steps from Filter Test Cases to Developing Filter Regular
+Expressions and submit a GitHub pull request (PR) afterwards. If you get stuck,
+you can push your unfinished changes and still submit a PR -- describe
+what you have done, what is the hurdle, and we'll attempt to help (PR
+will be automagically updated with future commits you would push to
+complete it).
+
+Filter test cases
+-----------------
+
+Purpose:
+
+Start by finding the log messages that the application generates related to
+some form of authentication failure. If you are adding to an existing filter
+think about whether the log messages are of a similar importance and purpose
+to the existing filter. If you were a user of Fail2Ban, and did a package
+update of Fail2Ban that started matching new log messages, would anything
+unexpected happen?  Would the bantime/findtime for the jail be appropriate for
+the new log messages?  If it doesn't, perhaps it needs to be in a separate
+filter definition, for example like exim filter aims at authentication failures
+and exim-spam at log messages related to spam.
+
+Even if it is a new filter you may consider separating the log messages into
+different filters based on purpose.
+
+Cause:
+
+Are some of the log lines a result of the same action? For example, is a PAM
+failure log message, followed by an application specific failure message the
+result of the same user/script action?  If you add regular expressions for
+both you would end up with two failures for a single action.
+Therefore, select the most appropriate log message and document the other log
+message) with a test case not to match it and a description as to why you chose
+one over another.
+
+With the selected log lines consider what action has caused those log
+messages and whether they could have been generated by accident? Could
+the log message be occurring due to the first step towards the application
+asking for authentication? Could the log messages occur often? If some of
+these are true make a note of this in the jail.conf example that you provide.
+
+Samples:
+
+It is important to include log file samples so any future change in the regular
+expression will still work with the log lines you have identified.
+
+The sample log messages are provided in a file under testcases/files/logs/
+named identically as the corresponding filter (but without .conf extension).
+Each log line should be preceded by a line with failJSON metadata (so the logs
+lines are tested in the test suite) directly above the log line. If there is
+any specific information about the log message, such as version or an
+application configuration option that is needed for the message to occur,
+include this in a comment (line beginning with #) above the failJSON metadata.
+
+Log samples should include only one, definitely not more than 3, examples of
+log messages of the same form. If log messages are different in different
+versions of the application log messages that show this are encouraged.
+
+Also attempt to inject an IP into the application (e.g. by specifying
+it as a username) so that Fail2Ban possibly detects the IP
+from user input rather than the true origin. See the Filter Security section
+and the top example in testcases/files/logs/apache-auth as to how to do this.
+One you have discovered that this is possible, correct the regex so it doesn't
+match and provide this as a test case with "match": false (see failJSON below).
+
+If the mechanism to create the log message isn't obvious provide a
+configuration and/or sample scripts testcases/files/config/{filtername} and
+reference these in the comments above the log line.
+
+FailJSON metadata:
+
+A failJSON metadata is a comment immediately above the log message. It will
+look like:
+
+# failJSON: { "time": "2013-06-10T10:10:59", "match": true , "host": "93.184.216.119" }
+
+Time should match the time of the log message. It is in a specific format of
+Year-Month-Day'T'Hour:minute:Second.  If your log message does not include a
+year, like the example below, the year should be listed as 2005, if before Sun
+Aug 14 10am UTC, and 2004 if afterwards.  Here is an example failJSON
+line preceding a sample log line:
+
+# failJSON: { "time": "2005-03-24T15:25:51", "match": true , "host": "198.51.100.87" }
+Mar 24 15:25:51 buffalo1 dropbear[4092]: bad password attempt for 'root' from 198.51.100.87:5543
+
+The "host" in failJSON should contain the IP or domain that should be blocked.
+
+For long lines that you do not want to be matched (e.g. from log injection
+attacks) and any log lines to be excluded (see "Cause" section above), set
+"match": false in the failJSON and describe the reason in the comment above.
+
+After developing regexes, the following command will test all failJSON metadata
+against the log lines in all sample log files
+
+./fail2ban-testcases testSampleRegex
+
+Developing Filter Regular Expressions
+-------------------------------------
+
+Date/Time:
+
+At the moment, Fail2Ban depends on log lines to have time stamps.  That is why
+before starting to develop failregex, check if your log line format known to
+Fail2Ban.  Copy the time component from the log line and append an IP address to
+test with following command:
+
+./fail2ban-regex "2013-09-19 02:46:12 1.2.3.4" "<HOST>"
+
+Output of such command should contain something like:
+
+Date template hits:
+|- [# of hits] date format
+|  [1] Year-Month-Day Hour:Minute:Second
+
+Ensure that the template description matches time/date elements in your log line
+time stamp.  If there is no matched format then date template needs to be added
+to server/datedetector.py.  Ensure that a new template is added in the order
+that more specific matches occur first and that there is no confusion between a
+Day and a Month.
+
+Filter file:
+
+The filter is specified in a config/filter.d/{filtername}.conf file. Filter file
+can have sections INCLUDES (optional) and Definition as follows:
+
+[INCLUDES]
+
+before = common.conf
+
+after = filtername.local
+
+[Definition]
+
+failregex = ....
+
+ignoreregex = ....
+
+This is also documented in the man page jail.conf (section 5). Other definitions
+can be added to make failregex's more readable and maintainable to be used
+through string Interpolations (see http://docs.python.org/2.7/library/configparser.html)
+
+
+General rules:
+
+Use "before" if you need to include a common set of rules, like syslog or if
+there is a common set of regexes for multiple filters.
+
+Use "after" if you wish to allow the user to overwrite a set of customisations
+of the current filter. This file doesn't need to exist.
+
+Try to avoid using ignoreregex mainly for performance reasons. The case when you
+would use it is if in trying to avoid using it, you end up with an unreadable
+failregex.
+
+Syslog:
+
+If your application logs to syslog you can take advantage of log line prefix
+definitions present in common.conf.  So as a base use:
+
+[INCLUDES]
+
+before = common.conf
+
+[Definition]
+
+_daemon = app
+
+failregex = ^%(__prefix_line)s
+
+In this example common.conf defines __prefix_line which also contains the
+_daemon name (in syslog terms the service) you have just specified. _daemon
+can also be a regex.
+
+For example, to capture following line _daemon should be set to "dovecot"
+
+Dec 12 11:19:11 dunnart dovecot: pop3-login: Aborted login (tried to use disabled plaintext auth): rip=190.210.136.21, lip=113.212.99.193
+
+and then ^%(__prefix_line)s would match "Dec 12 11:19:11 dunnart dovecot:
+". Note it matches the trailing space(s) as well.
+
+Substitutions (AKA string interpolations):
+
+We have used string interpolations in above examples.  They are useful for
+making the regexes more readable, reuse generic patterns in multiple failregex
+lines, and also to refer definition of regex parts to specific filters or even
+to the user.  General principle is that value of a _name variable replaces
+occurrences of %(_name)s within the same section or anywhere in the config file
+if defined in [DEFAULT] section.
+
+Regular Expressions:
+
+Regular expressions (failregex, ignoreregex) assume that the date/time has been
+removed from the log line (this is just how fail2ban works internally ATM).
+
+If the format is like '<date...> error 1.2.3.4 is evil' then you need to match
+the < at the start so regex should be similar to '^<> <HOST> is evil$' using
+<HOST> where the IP/domain name appears in the log line.
+
+The following general rules apply to regular expressions:
+
+* ensure regexes start with a ^ and are as restrictive as possible. E.g. do not
+  use .* if \d+ is sufficient;
+* use functionality of Python regexes defined in the standard Python re library
+  http://docs.python.org/2/library/re.html;
+* make regular expressions readable (as much as possible). E.g.
+  (?:...) represents a non-capturing regex but (...) is more readable, thus
+  preferred.
+
+If you have only a basic knowledge of regular repressions we advise to read
+http://docs.python.org/2/library/re.html first.  It doesn't take long and would
+remind you e.g. which characters you need to escape and which you don't.
+
+Developing/testing a regex:
+
+You can develop a regex in a file or using command line depending on your
+preference. You can also use samples you have already created in the test cases
+or test them one at a time.
+
+The general tool for testing Fail2Ban regexes is fail2ban-regex. To see how to
+use it run:
+
+./fail2ban-regex --help
+
+Take note of  -l heavydebug  / -l debug  and -v as they might be very useful.
+
+TIP: Take a look at the source code of the application you are developing
+     failregex for. You may see optional or extra log messages, or parts there
+     of, that need to form part of your regex.  It may also reveal how some
+     parts are constrained and different formats depending on configuration or
+     less common usages.
+
+TIP: For looking through source code - http://sourcecodebrowser.com/ . It has
+     call graphs and can browse different versions.
+
+TIP: Some applications log spaces at the end. If you are not sure add \s*$ as
+     the end part of the regex.
+
+If your regex is not matching, http://www.debuggex.com/?flavor=python can help
+to tune it:
+
+* use regex from the ./fail2ban-regex output (to ensure all substitutions are
+done) and replace <HOST> with (?&.ipv4). Make sure that regex type set to
+Python;
+* for the test data put your log output with the time removed;
+- when you have fixed the regex put it back into your filter file.
+
+Please spread the good word about debuggex - Serge Toarca is kindly continuing
+its free availability to Open Source developers.
+
+Finishing up:
+
+If you've added a new filter, add a new entry in config/jail.conf. The theory
+here is that a user will create a jail.local with [filtername]\nenable=true to
+enable your jail.
+
+So more specifically in the [filter] section in jail.conf:
+* ensure that you have "enabled = false" (users will enable as needed);
+* use "filter =" set to your filter name;
+* use a typical action to disable ports associated with the application;
+* set "logpath" to the usual location of application log file;
+* if the default findtime or bantime isn't appropriate to the filter, specify
+  more appropriate choices (possibly with a brief comment line).
+
+Submit github pull request (See "Pull Requests" above) for
+github.com/fail2ban/fail2ban containing your great work.
+
+Filter Security
+---------------
+
+Poor filter regular expressions are susceptible to DoS attacks.
+
+When a remote user has the ability to introduce text that would match filter's
+failregex, while matching inserted text to the <HOST> part, they have the
+ability to deny any host they choose.
+
+So the <HOST> part must be anchored on text generated by the application, and
+not the user, to a extent sufficient to prevent user inserting the entire text
+matching this or any other failregex.
+
+Ideally filter regex should anchor at the beginning and at the end of log line.
+However as more applications log at the beginning than the end, anchoring the
+beginning is more important. If the log file used by the application is shared
+with other applications, like system logs, ensure the other application that use
+that log file do not log user generated text at the beginning of the line, or,
+if they do, ensure the regexes of the filter are sufficient to mitigate the risk
+of insertion.
+
+
+Examples of poor filters
+------------------------
+
+1. Too restrictive
+
+We find a log message:
+
+    Apr-07-13 07:08:36 Invalid command fial2ban from 1.2.3.4
+
+We make a failregex
+
+    ^Invalid command \S+ from <HOST>
+
+Now think evil. The user does the command 'blah from 1.2.3.44'
+
+The program diligently logs:
+
+    Apr-07-13 07:08:36 Invalid command blah from 1.2.3.44 from 1.2.3.4
+
+And fail2ban matches 1.2.3.44 as the IP that it ban. A DoS attack was successful.
+
+The fix here is that the command can be anything so .* is appropriate.
+
+    ^Invalid command .* from <HOST>
+
+Here the .* will match until the end of the string. Then realise it has more to
+match, i.e. "from <HOST>" and go back until it find this. Then it will ban
+1.2.3.4 correctly. Since the <HOST> is always at the end, end the regex with a $.
+
+    ^Invalid command .* from <HOST>$
+
+Note if we'd just had the expression:
+
+    ^Invalid command \S+ from <HOST>$
+
+Then provided the user put a space in their command they would have never been
+banned.
+
+2. Filter regex can match other user injected data
+
+From the Apache vulnerability CVE-2013-2178
+( original ref: https://vndh.net/note:fail2ban-089-denial-service ).
+
+An example bad regex for Apache:
+
+    failregex = [[]client <HOST>[]] user .* not found
+
+Since the user can do a get request on:
+
+    GET /[client%20192.168.0.1]%20user%20root%20not%20found HTTP/1.0
+Host: remote.site
+
+Now the log line will be:
+
+    [Sat Jun 01 02:17:42 2013] [error] [client 192.168.33.1] File does not exist: /srv/http/site/[client 192.168.0.1] user root not found
+
+As this log line doesn't match other expressions hence it matches the above
+regex and blocks 192.168.33.1 as a denial of service from the HTTP requester.
+
+3. Application generates two identical log messages with different meanings
+
+If the application generates the following two messages under different
+circumstances:
+
+    client <IP>: authentication failed
+    client <USER>: authentication failed
+
+
+Then it's obvious that a regex of "^client <HOST>: authentication
+failed$" will still cause problems if the user can trigger the second
+log message with a <USER> of 123.1.1.1.
+
+Here there's nothing to do except request/change the application so it logs
+messages differently.
+
 
 Code Testing
 ============
@@ -66,7 +437,7 @@ coverage run fail2ban-testcases
 coverage html
 
 Then look at htmlcov/index.html and see how much coverage your test cases
-exert over the codebase. Full coverage is a good thing however it may not be
+exert over the code base. Full coverage is a good thing however it may not be
 complete. Try to ensure tests cover as many independent paths through the
 code.
 
@@ -136,6 +507,14 @@ Use the following tags in your commit messages:
 
 Multiple tags could be joined with +, e.g. "BF+TST:".
 
+Use the text "closes #333"/"resolves #333 "/"fixes #333" where 333 represents
+an issue that is closed. Other text and details in link below.
+See: https://help.github.com/articles/closing-issues-via-commit-messages
+
+If merge resulted in conflicts, clarify what changes were done to
+corresponding files in the 'Conflicts:' section of the merge commit
+message.  See e.g. https://github.com/fail2ban/fail2ban/commit/f5a8a8ac
+
 Adding Actions
 --------------
 
@@ -149,7 +528,7 @@ Design
 Fail2Ban was initially developed with Python 2.3 (IIRC). It should
 still be compatible with Python 2.4 and such compatibility assurance
 makes code ... old-fashioned in many places (RF-Note).  In 0.7 the
-design went through major refactoring into client/server,
+design went through major re-factoring into client/server,
 a-thread-per-jail design which made it a bit difficult to follow.
 Below you can find a sketchy description of the main components of the
 system to orient yourself better.
@@ -260,7 +639,7 @@ one way or another provide
 		except FailManagerEmpty:
 			self.failManager.cleanup(MyTime.time())
 
-thus channeling "ban tickets" from their failManager to the
+thus channelling "ban tickets" from their failManager to the
 corresponding jail.
 
 action.py
@@ -290,30 +669,45 @@ Releasing
   * https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&component=fail2ban&classification=Red%20Hat&classification=Fedora
   * http://www.freebsd.org/cgi/query-pr-summary.cgi?text=fail2ban
 
-# Provide a release sample to distributors
+# Make sure the tests pass
 
-  * Debian: Yaroslav Halchenko <debian@onerussian.com>
+  ./fail2ban-testcases-all
-            http://packages.qa.debian.org/f/fail2ban.html
-  * FreeBSD: Christoph Theis theis@gmx.at>, Nick Hilliard <nick@foobar.org>
-            http://svnweb.freebsd.org/ports/head/security/py-fail2ban/Makefile?view=markup
-  * Fedora: Axel Thimm <Axel.Thimm@atrpms.net>
-            https://apps.fedoraproject.org/packages/fail2ban
-  * Gentoo: netmon@gentoo.org
-            http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/net-analyzer/fail2ban/metadata.xml?view=markup
-  * openSUSE: Stephan Kulow <coolo@suse.com>
-            https://build.opensuse.org/package/users?package=fail2ban&project=openSUSE%3AFactory
-  * Mac Ports: @Malbrouck on github (gh-49)
-            https://trac.macports.org/browser/trunk/dports/security/fail2ban/Portfile
 
-# Wait for feedback from distributors
+# Ensure the version is correct
 
-# Ensure the version is correct in ./common/version.py
+  in:
+  * ./common/version.py
+  * top of ChangeLog
+  * README.md
+
+# Ensure the MANIFEST is complete
+
+Run:
+
+  python setup.py sdist
+
+Look for errors like:
+ 'testcases/files/logs/mysqld.log' not a regular file -- skipping
+
+Which indicates that testcases/files/logs/mysqld.log has been moved or is a directory
+
+  tar -C /tmp -jxf dist/fail2ban-0.8.11.tar.bz2
+
+# clean up current direcory
+
+  diff -rul --exclude \*.pyc . /tmp/fail2ban-0.8.11/
+
+  # Only differences should be files that you don't want distributed.
+
+# Ensure the tests work from the tarball
+
+  cd /tmp/fail2ban-0.8.11/ && ./fail2ban-testcases-all
 
 # Add/finalize the corresponding entry in the ChangeLog
 
   To generate a list of committers use e.g.
 
-  git shortlog -sn 0.8.8.. | sed -e 's,^[ 0-9\t]*,,g' | tr '\n' '\|' | sed -e 's:|:, :g'
+  git shortlog -sn 0.8.10.. | sed -e 's,^[ 0-9\t]*,,g' | tr '\n' '\|' | sed -e 's:|:, :g'
 
   Ensure the top of the ChangeLog has the right version and current date.
 
@@ -322,23 +716,59 @@ Releasing
 # Update man pages
 
   (cd man ; ./generate-man )
-  git commit -m 'update man pages for release' man/*
+  git commit -m 'DOC/ENH: update man pages for release' man/*
 
-# Make sure the tests pass
+# Prepare source and rpm binary distributions
 
-  ./fail2ban-testcases-all
-
-# Prepare/upload source and rpm binary distributions
-
-  python setup.py check
   python setup.py sdist
   python setup.py bdist_rpm
   python setup.py upload
 
-# Run the following and update the wiki with output:
+# Provide a release sample to distributors
 
+  * Debian: Yaroslav Halchenko <debian@onerussian.com>
+            http://packages.qa.debian.org/f/fail2ban.html
+  * FreeBSD: Christoph Theis theis@gmx.at>, Nick Hilliard <nick@foobar.org>
+            http://svnweb.freebsd.org/ports/head/security/py-fail2ban/Makefile?view=markup
+            http://www.freebsd.org/cgi/query-pr-summary.cgi?text=fail2ban
+  * Fedora: Axel Thimm <Axel.Thimm@atrpms.net>
+            https://apps.fedoraproject.org/packages/fail2ban
+            http://pkgs.fedoraproject.org/cgit/fail2ban.git
+            https://admin.fedoraproject.org/pkgdb/acls/bugs/fail2ban
+  * Gentoo: netmon@gentoo.org
+            http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/net-analyzer/fail2ban/metadata.xml?view=markup
+            https://bugs.gentoo.org/buglist.cgi?quicksearch=fail2ban
+  * openSUSE: Stephan Kulow <coolo@suse.com>
+            https://build.opensuse.org/package/show/openSUSE:Factory/fail2ban
+  * Mac Ports: @Malbrouck on github (gh-49)
+            https://trac.macports.org/browser/trunk/dports/security/fail2ban/Portfile
+  * Mageia:
+            https://bugs.mageia.org/buglist.cgi?quicksearch=fail2ban
+  An potentially to the fail2ban-users directory.
+
+# Wait for feedback from distributors
+
+# Prepare a release notice https://github.com/fail2ban/fail2ban/releases/new
+
+  Upload the source/binaries from the dist directory and tag the release using the URL
+
+# Upload source/binaries to sourceforge http://sourceforge.net/projects/fail2ban/
+
+# Run the following and update the wiki with output:
   python -c 'import common.protocol; common.protocol.printWiki()'
 
+  page: http://www.fail2ban.org/wiki/index.php/Commands
+
+* Update:
+  http://www.fail2ban.org/wiki/index.php/Downloads
+  http://www.fail2ban.org/wiki/index.php/ChangeLog
+  http://www.fail2ban.org/wiki/index.php/Requirements (Check requirement)
+  http://www.fail2ban.org/wiki/index.php/Main_Page (Add to News)
+  http://www.fail2ban.org/wiki/index.php/Features
+
+* See if any filters are upgraded:
+  http://www.fail2ban.org/wiki/index.php/Special:AllPages
+
 # Email users and development list of release
 
 # notify distributors
@@ -348,10 +778,17 @@ Post Release
 
 Add the following to the top of the ChangeLog
 
-ver. 0.8.11 (2013/XX/XXX) - wanna-be-stable
+ver. 0.8.12 (2013/XX/XXX) - wanna-be-released
-- Fixes
+-----------
-- New Features
+
-- Enhancements
+- Fixes:
+
+- New Features:
+
+- Enhancements:
+
+Alter the git shortlog command in the previous section to refer to the just
+released version.
 
 and adjust common/version.py to carry .dev suffix to signal
 a version under development.

MANIFEST

@@ -5,11 +5,14 @@ TODO
 THANKS
 COPYING
 DEVELOP
-doc/run-rootless.txt
 fail2ban-client
 fail2ban-server
 fail2ban-testcases
 fail2ban-regex
+fail2ban-testcases-all
+setup.py
+setup.cfg
+kill-server
 client/configreader.py
 client/configparserinc.py
 client/jailreader.py
@@ -43,6 +46,8 @@ server/banmanager.py
 server/datetemplate.py
 server/mytime.py
 server/failregex.py
+testcases/actionstestcase.py
+testcases/dummyjail.py
 testcases/files/testcase-usedns.log
 testcases/files/logs/bsd/syslog-plain.txt
 testcases/files/logs/bsd/syslog-v.txt
@@ -52,21 +57,61 @@ testcases/files/logs/assp
 testcases/files/logs/asterisk
 testcases/files/logs/dovecot
 testcases/files/logs/exim
-testcases/files/logs/lighttpd
+testcases/files/logs/suhosin
-testcases/files/logs/mysqld.log
+testcases/files/logs/mysqld-auth
 testcases/files/logs/named-refused
 testcases/files/logs/pam-generic
 testcases/files/logs/postfix
 testcases/files/logs/proftpd
 testcases/files/logs/pure-ftpd
 testcases/files/logs/roundcube-auth
-testcases/files/logs/sasl
+testcases/files/logs/postfix-sasl
 testcases/files/logs/sogo-auth
 testcases/files/logs/sshd
 testcases/files/logs/sshd-ddos
 testcases/files/logs/vsftpd
 testcases/files/logs/webmin-auth
-testcases/files/logs/wu-ftpd
+testcases/files/logs/wuftpd
+testcases/files/logs/3proxy
+testcases/files/logs/apache-auth
+testcases/files/logs/apache-badbots
+testcases/files/logs/apache-nohome
+testcases/files/logs/apache-noscript
+testcases/files/logs/courierlogin
+testcases/files/logs/couriersmtp
+testcases/files/logs/cyrus-imap
+testcases/files/logs/dropbear
+testcases/files/logs/exim-spam
+testcases/files/logs/gssftpd
+testcases/files/logs/lighttpd-auth
+testcases/files/logs/mysqld-auth
+testcases/files/logs/perdition
+testcases/files/logs/php-url-fopen
+testcases/files/logs/qmail
+testcases/files/logs/recidive
+testcases/files/logs/sieve
+testcases/files/logs/selinux-ssh
+testcases/files/logs/suhosin
+testcases/files/logs/uwimap-auth
+testcases/files/logs/wuftpd
+testcases/files/logs/xinetd-fail
+testcases/files/config/apache-auth/digest/.htaccess
+testcases/files/config/apache-auth/digest/.htpasswd
+testcases/files/config/apache-auth/digest_time/.htaccess
+testcases/files/config/apache-auth/digest_time/.htpasswd
+testcases/files/config/apache-auth/basic/authz_owner/.htaccess
+testcases/files/config/apache-auth/basic/authz_owner/cant_get_me.html
+testcases/files/config/apache-auth/basic/authz_owner/.htpasswd
+testcases/files/config/apache-auth/basic/file/.htaccess
+testcases/files/config/apache-auth/basic/file/.htpasswd
+testcases/files/config/apache-auth/digest.py
+testcases/files/config/apache-auth/digest_wrongrelm/.htaccess
+testcases/files/config/apache-auth/digest_wrongrelm/.htpasswd
+testcases/files/config/apache-auth/digest_anon/.htaccess
+testcases/files/config/apache-auth/digest_anon/.htpasswd
+testcases/files/config/apache-auth/README
+testcases/files/config/apache-auth/noentry/.htaccess
+testcases/samplestestcase.py
 testcases/banmanagertestcase.py
 testcases/failmanagertestcase.py
 testcases/clientreadertestcase.py
@@ -82,8 +127,6 @@ testcases/files/testcase03.log
 testcases/files/testcase04.log
 testcases/misctestcase.py
 testcases/utils.py
-setup.py
-setup.cfg
 common/__init__.py
 common/exceptions.py
 common/helpers.py
@@ -101,7 +144,7 @@ config/filter.d/couriersmtp.conf
 config/filter.d/cyrus-imap.conf
 config/filter.d/exim.conf
 config/filter.d/gssftpd.conf
-config/filter.d/lighttpd-fastcgi.conf
+config/filter.d/suhosin.conf
 config/filter.d/named-refused.conf
 config/filter.d/postfix.conf
 config/filter.d/proftpd.conf
@@ -109,7 +152,7 @@ config/filter.d/pure-ftpd.conf
 config/filter.d/qmail.conf
 config/filter.d/pam-generic.conf
 config/filter.d/php-url-fopen.conf
-config/filter.d/sasl.conf
+config/filter.d/postfix-sasl.conf
 config/filter.d/sieve.conf
 config/filter.d/sshd.conf
 config/filter.d/sshd-ddos.conf
@@ -124,10 +167,24 @@ config/filter.d/lighttpd-auth.conf
 config/filter.d/recidive.conf
 config/filter.d/roundcube-auth.conf
 config/filter.d/assp.conf
-config/filter.d/mysqld-auth.conf
 config/filter.d/sogo-auth.conf
+config/filter.d/mysqld-auth.conf
+config/filter.d/selinux-common.conf
+config/filter.d/selinux-ssh.conf
+config/filter.d/3proxy.conf
+config/filter.d/apache-common.conf
+config/filter.d/exim-common.conf
+config/filter.d/exim-spam.conf
+config/filter.d/perdition.conf
+config/filter.d/uwimap-auth.conf
+config/action.d/apf.conf
+config/action.d/osx-afctl.conf
+config/action.d/osx-ipfw.conf
+config/action.d/sendmail-common.conf
 config/action.d/bsd-ipfw.conf
 config/action.d/dummy.conf
+config/action.d/firewall-cmd-direct-new.conf
+config/action.d/iptables-ipset-proto6-allports.conf
 config/action.d/iptables-blocktype.conf
 config/action.d/iptables-ipset-proto4.conf
 config/action.d/iptables-ipset-proto6.conf
@@ -155,6 +212,7 @@ config/action.d/sendmail-whois.conf
 config/action.d/sendmail-whois-lines.conf
 config/action.d/shorewall.conf
 config/fail2ban.conf
+doc/run-rootless.txt
 man/fail2ban-client.1
 man/fail2ban.1
 man/jail.conf.5
@@ -176,9 +234,8 @@ files/cacti/fail2ban_stats.sh
 files/cacti/cacti_host_template_fail2ban.xml
 files/cacti/README
 files/nagios/check_fail2ban
-files/nagios/f2ban.txt
+files/nagios/README
 files/bash-completion
 files/fail2ban-tmpfiles.conf
 files/fail2ban.service
 files/ipmasq-ZZZzzz_fail2ban.rul
-files/nagios/README

README.Solaris

@@ -69,27 +69,10 @@ FAIL2BAN CONFIGURATION
 
 OPT: Create /etc/fail2ban/fail2ban.local containing:
 
-# Fail2Ban main configuration file
+# Fail2Ban configuration file for logging fail2ban on Solaris
-#
-# Comments: use '#' for comment lines and ';' (following a space) for inline comments
-#
-# Changes:  in most of the cases you should not modify this
-#           file, but provide customizations in fail2ban.local file, e.g.:
-#
-# [Definition]
-# loglevel = 4
 #
 [Definition]
 
-# Option:  logtarget
-# Notes.:  Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
-#          Only one log target can be specified.
-#          If you change logtarget from the default value and you are
-#          using logrotate -- also adjust or disable rotation in the
-#          corresponding configuration file
-#          (e.g. /etc/logrotate.d/fail2ban on Debian systems)
-# Values:  STDOUT STDERR SYSLOG file  Default:  /var/log/fail2ban.log
-#
 logtarget = /var/adm/fail2ban.log
 
 
@@ -99,13 +82,13 @@ REQ: Create /etc/fail2ban/jail.local containing:
 
 enabled     = true
 filter      = sshd
-action      = hostsdeny
+action      = hostsdeny[daemon_list=sshd]
               sendmail-whois[name=SSH, dest=you@example.com]
 ignoreregex = for myuser from
 logpath     = /var/adm/auth.log
 
 Set the sendmail dest address to something useful or drop the line to stop it spamming you.
-Set 'myuser' to your username to avoid banning yourself or drop it.
+Set 'myuser' to your username to avoid banning yourself or remove the line.
 
 START (OR RESTART) FAIL2BAN
 
@@ -128,7 +111,7 @@ GOTCHAS AND FIXMES
     svcadm enable fail2ban
 
 * If svcs -xv says that fail2ban failed to start or svcs says it's in maintenance mode
-  chcek /var/svc/log/network-fail2ban:default.log for clues. 
+  check /var/svc/log/network-fail2ban:default.log for clues. 
   Check permissions on /var/adm, /var/adm/auth.log /var/adm/fail2ban.log and /var/run/fail2ban
   You may need to:
 
@@ -136,6 +119,4 @@ GOTCHAS AND FIXMES
 
 * Fail2ban adds lines like these to /etc/hosts.deny:
 
-    ALL: 1.2.3.4
+    sshd: 1.2.3.4
-
-  wouldn't it be better to just block sshd?

README.md

@@ -2,7 +2,7 @@
                         / _|__ _(_) |_  ) |__  __ _ _ _  
                        |  _/ _` | | |/ /| '_ \/ _` | ' \ 
                        |_| \__,_|_|_/___|_.__/\__,_|_||_|
-                       v0.8.10                  2013/06/12
+                       v0.8.11-pre1            2013/10/30
 
 ## Fail2Ban: ban hosts that cause multiple authentication errors
 
@@ -30,8 +30,8 @@ Optional:
 
 To install, just do:
 
-    tar xvfj fail2ban-0.8.10.tar.bz2
+    tar xvfj fail2ban-0.8.11.tar.bz2
-    cd fail2ban-0.8.10
+    cd fail2ban-0.8.11
     python setup.py install
 
 This will install Fail2Ban into /usr/share/fail2ban. The executable scripts are

THANKS

@@ -1,44 +1,65 @@
-Fail2Ban is an open source project with many contributions from its
+Fail2Ban is an open source project which was conceived and originally
-users community. Below is an alphabetically sorted partial list of the
+developed by Cyril Jaquier until 2010.  Since then Fail2Ban grew into
-contributors to the project.  If you have been left off, please let us
+a community-driven project with many contributions from its users.
-know (preferably send a pull request on github with the "fix") and you
+Below is an alphabetically sorted partial list of the contributors to
-will be added
+the project.  If you have been left off, please let us know
+(preferably send a pull request on github with the "fix") and you will
+be added
 
 Adrien Clerc
+ache
+Amir Caspi
 Andrey G. Grozin
+Andy Fragen
 Arturo 'Buanzo' Busleiman
 Axel Thimm
+Beau Raines
 Bill Heaton
 Carlos Alberto Lopez Perez
 Christian Rauch
+Christophe Carles
 Christoph Haas
 Christos Psonis
+Cyril Jaquier
 Daniel B. Cid
 Daniel Black
 David Nutter
 Eric Gerbier
 Enrico Labedzki
+ftoppi
+François Boulogne
+Frédéric
+Georgiy Mernov
 Guillaume Delvit
 Hanno 'Rince' Wagner
 Iain Lea
 Jonathan Kamens
 Jonathan Underwood
 Joël Bertrand
+JP Espinosa
 Justin Shore
 Kévin Drapel
+kjohnsonecl
 kojiro
+Manuel Arostegui Ramirez
+Marcel Dopita
 Mark Edgington
+Mark McKinstry
 Markus Hoffmann
 Marvin Rouge
 mEDI
+Мернов Георгий
 Michael C. Haller
 Michael Hanselmann
-NickMunger
+Nick Munger
 Patrick Börjesson
 Raphaël Marichez
+RealRancor
 René Berber
 Robert Edeker
+Rolf Fokkens
 Russell Odom
+Sebastian Arcus
 Sireyessire
 silviogarbes
 Stephen Gildea
@@ -48,5 +69,7 @@ Tyler
 Vaclav Misek
 Vincent Deffontaines
 Yaroslav Halchenko
+ykimon
 Yehuda Katz
 zugeschmiert
+Zurd

client/configreader.py

@@ -54,16 +54,19 @@ class ConfigReader(SafeConfigParserWithIncludes):
 							  % self._basedir)
 		basename = os.path.join(self._basedir, filename)
 		logSys.debug("Reading configs for %s under %s "  % (basename, self._basedir))
-		config_files = [ basename + ".conf",
+		config_files = [ basename + ".conf" ]
-						 basename + ".local" ]
-
-		# choose only existing ones
-		config_files = filter(os.path.exists, config_files)
 
 		# possible further customizations under a .conf.d directory
 		config_dir = basename + '.d'
 		config_files += sorted(glob.glob('%s/*.conf' % config_dir))
 
+		config_files.append(basename + ".local")
+	
+		config_files += sorted(glob.glob('%s/*.local' % config_dir))
+
+		# choose only existing ones
+		config_files = filter(os.path.exists, config_files)
+
 		if len(config_files):
 			# at least one config exists and accessible
 			logSys.debug("Reading config files: " + ', '.join(config_files))

client/fail2banreader.py

@@ -39,7 +39,7 @@ class Fail2banReader(ConfigReader):
 		ConfigReader.read(self, "fail2ban")
 	
 	def getEarlyOptions(self):
-		opts = [["string", "socket", "/tmp/fail2ban.sock"],
+		opts = [["string", "socket", "/var/run/fail2ban/fail2ban.sock"],
 				["string", "pidfile", "/var/run/fail2ban/fail2ban.pid"]]
 		return ConfigReader.getOptions(self, "Definition", opts)
 	

client/jailreader.py

@@ -24,7 +24,7 @@ __author__ = "Cyril Jaquier"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
-import logging, re, glob
+import logging, re, glob, os.path
 
 from configreader import ConfigReader
 from filterreader import FilterReader
@@ -55,7 +55,23 @@ class JailReader(ConfigReader):
 	
 	def isEnabled(self):
 		return self.__force_enable or self.__opts["enabled"]
-	
+
+	@staticmethod
+	def _glob(path):
+		"""Given a path for glob return list of files to be passed to server.
+
+		Dangling symlinks are warned about and not returned
+		"""
+		pathList = []
+		for p in glob.glob(path):
+			if not os.path.exists(p):
+				logSys.warning("File %s doesn't even exist, thus cannot be monitored" % p)
+			elif not os.path.lexists(p):
+				logSys.warning("File %s is a dangling link, thus cannot be monitored" % p)
+			else:
+				pathList.append(p)
+		return pathList
+
 	def getOptions(self):
 		opts = [["bool", "enabled", "false"],
 				["string", "logpath", "/var/log/messages"],
@@ -103,16 +119,30 @@ class JailReader(ConfigReader):
 				logSys.warn("No actions were defined for %s" % self.__name)
 		return True
 	
-	def convert(self):
+	def convert(self, allow_no_files=False):
+		"""Convert read before __opts to the commands stream
+
+		Parameters
+		----------
+		allow_missing : bool
+		  Either to allow log files to be missing entirely.  Primarily is
+		  used for testing
+		 """
+
 		stream = []
 		for opt in self.__opts:
 			if opt == "logpath":
+				found_files = 0
 				for path in self.__opts[opt].split("\n"):
-					pathList = glob.glob(path)
+					pathList = JailReader._glob(path)
 					if len(pathList) == 0:
-						logSys.error("No file found for " + path)
+						logSys.error("No file(s) found for glob %s" % path)
 					for p in pathList:
+						found_files += 1
 						stream.append(["set", self.__name, "addlogpath", p])
+				if not (found_files or allow_no_files):
+					raise ValueError(
+						"Have not found any log file for %s jail" % self.__name)
 			elif opt == "backend":
 				backend = self.__opts[opt]
 			elif opt == "maxretry":

client/jailsreader.py

@@ -18,7 +18,7 @@
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
 # Author: Cyril Jaquier
-# 
+#
 
 __author__ = "Cyril Jaquier"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
@@ -32,7 +32,7 @@ from jailreader import JailReader
 logSys = logging.getLogger("fail2ban.client.config")
 
 class JailsReader(ConfigReader):
-	
+
 	def __init__(self, force_enable=False, **kwargs):
 		"""
 		Parameters
@@ -44,17 +44,25 @@ class JailsReader(ConfigReader):
 		ConfigReader.__init__(self, **kwargs)
 		self.__jails = list()
 		self.__force_enable = force_enable
-	
+
 	def read(self):
 		return ConfigReader.read(self, "jail")
-	
+
-	def getOptions(self, section = None):
+	def getOptions(self, section=None):
+		"""Reads configuration for jail(s) and adds enabled jails to __jails
+		"""
 		opts = []
 		self.__opts = ConfigReader.getOptions(self, "Definition", opts)
 
-		if section:
+		if section is None:
-			# Get the options of a specific jail.
+			sections = self.sections()
-			jail = JailReader(section, basedir=self.getBaseDir(), force_enable=self.__force_enable)
+		else:
+			sections = [ section ]
+
+		# Get the options of all jails.
+		for sec in sections:
+			jail = JailReader(sec, basedir=self.getBaseDir(),
+							  force_enable=self.__force_enable)
 			jail.read()
 			ret = jail.getOptions()
 			if ret:
@@ -62,34 +70,30 @@ class JailsReader(ConfigReader):
 					# We only add enabled jails
 					self.__jails.append(jail)
 			else:
-				logSys.error("Errors in jail '%s'. Skipping..." % section)
+				logSys.error("Errors in jail %r. Skipping..." % sec)
 				return False
-		else:
-			# Get the options of all jails.
-			for sec in self.sections():
-				jail = JailReader(sec, basedir=self.getBaseDir(), force_enable=self.__force_enable)
-				jail.read()
-				ret = jail.getOptions()
-				if ret:
-					if jail.isEnabled():
-						# We only add enabled jails
-						self.__jails.append(jail)
-				else:
-					logSys.error("Errors in jail '" + sec + "'. Skipping...")
-					return False
 		return True
-	
+
-	def convert(self):
+	def convert(self, allow_no_files=False):
+		"""Convert read before __opts and jails to the commands stream
+
+		Parameters
+		----------
+		allow_missing : bool
+		  Either to allow log files to be missing entirely.  Primarily is
+		  used for testing
+		"""
+
 		stream = list()
 		for opt in self.__opts:
 			if opt == "":
 				stream.append([])
 		# Convert jails
 		for jail in self.__jails:
-			stream.extend(jail.convert())
+			stream.extend(jail.convert(allow_no_files=allow_no_files))
 		# Start jails
 		for jail in self.__jails:
 			stream.append(["start", jail.getName()])
-		
+
 		return stream
-		
+

common/__init__.py

@@ -23,3 +23,8 @@
 __author__ = "Cyril Jaquier"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
+
+import logging
+
+# Custom debug level
+logging.HEAVYDEBUG = 5

common/helpers.py

@@ -17,24 +17,12 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-__author__ = "Cyril Jaquier, Arturo 'Buanzo' Busleiman"
+__author__ = "Cyril Jaquier, Arturo 'Buanzo' Busleiman, Yaroslav Halchenko"
-__copyright__ = "Copyright (c) 2009 Cyril Jaquier"
 __license__ = "GPL"
 
 
 def formatExceptionInfo():
-	""" Author: Arturo 'Buanzo' Busleiman """
+	""" Consistently format exception information """
 	import sys
 	cla, exc = sys.exc_info()[:2]
-	excName = cla.__name__
+	return (cla.__name__, str(exc))
-	try:
-		excArgs = exc.__dict__["args"]
-		# Assure that we always return a string, without unneeded
-		# 'decorations' with python <= 2.5 where args would be a tuple
-		if isinstance(excArgs, tuple) and len(excArgs) == 1:
-			excArgs = excArgs[0]
-		excArgs = str(excArgs)
-	except KeyError:
-		# And always provide a string output
-		excArgs = str(exc)
-	return (excName, excArgs)

common/version.py

@@ -24,4 +24,4 @@ __author__ = "Cyril Jaquier, Yaroslav Halchenko"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2011-2013 Yaroslav Halchenko"
 __license__ = "GPL"
 
-version = "0.8.10"
+version = "0.8.11.pre1"

config/action.d/apf.conf

@@ -0,0 +1,43 @@
+# Fail2Ban configuration file
+#
+# Author: Mark McKinstry
+#
+[Definition]
+
+# Option:  actionstart
+# Notes.:  command executed once at the start of Fail2Ban.
+# Values:  CMD
+#
+actionstart = 
+
+# Option:  actionstop
+# Notes.:  command executed once at the end of Fail2Ban
+# Values:  CMD
+#
+actionstop = 
+
+# Option:  actioncheck
+# Notes.:  command executed once before each actionban command
+# Values:  CMD
+#
+actioncheck = 
+
+# Option:  actionban
+# Notes.:  command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    <ip>  IP address
+#          <failures>  number of failures
+#          <time>  unix timestamp of the ban time
+# Values:  CMD
+#
+actionban = apf --deny <ip> "banned by Fail2Ban <name>"
+
+# Option:  actionunban
+# Notes.:  command executed when unbanning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    <ip>  IP address
+#          <failures>  number of failures
+#          <time>  unix timestamp of the ban time
+# Values:  CMD
+#
+actionunban = apf --remove <ip>

config/action.d/bsd-ipfw.conf

@@ -14,14 +14,14 @@
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = ipfw show | fgrep -q 'table(<table>)' || ( ipfw show | awk 'BEGIN { b = 1 } { if ($1 <= b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num deny <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
+actionstart = ipfw show | fgrep -q 'table(<table>)' || ( ipfw show | awk 'BEGIN { b = 1 } { if ($1 <= b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num <blocktype> <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
 
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop =  [ -f <startstatefile> ] && ( read num < "<startstatefile>" <br> ipfw -q delete $num <br> rm "<startstatefile>" )
+actionstop =  [ ! -f <startstatefile> ] || ( read num < "<startstatefile>" <br> ipfw -q delete $num <br> rm "<startstatefile>" )
 
 
 # Option:  actioncheck
@@ -68,15 +68,16 @@ port =
 # Values:  STRING
 startstatefile = /var/run/fail2ban/ipfw-started-table_<table>
 
-# Option:  action
-# Notes:   This is the action to take for automaticly created rules. See the 
-#          ACTION defination at the top of man ipfw for allowed values.
-#          "deny" and "unreach port" are probably the useful.
-# Values:  STRING
-action = deny
-
 # Option: block
 # Notes:  This is how much to block.
 #         Can be "ip", "tcp", "udp" or various other options.
 # Values: STRING
 block = ip
+
+# Option:  blocktype
+# Notes.:  How to block the traffic. Use a action from man 5 ipfw
+#          Common values: deny, unreach port, reset
+#          ACTION defination at the top of man ipfw for allowed values.
+# Values:  STRING
+#
+blocktype = unreach port

config/action.d/complain.conf

@@ -58,7 +58,7 @@ actioncheck =
 actionban = ADDRESSES=`whois <ip> | perl -e 'while (<STDIN>) { next if /^changed|@(ripe|apnic)\.net/io; $m += (/abuse|trouble:|report|spam|security/io?3:0); if (/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)/io) { while (s/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)//io) { if ($m) { $a{lc($1)}=$m } else { $b{lc($1)}=$m } } $m=0 } else { $m && --$m } } if (%%a) {print join(",",keys(%%a))} else {print join(",",keys(%%b))}'`
 	    IP=<ip>
             if [ ! -z "$ADDRESSES" ]; then
-                (printf %%b "<message>\n"; date '+Note: Local timezone is %%z (%%Z)'; grep '<ip>' <logpath>) | <mailcmd> "Abuse from <ip>" $ADDRESSES <mailargs>
+                (printf %%b "<message>\n"; date '+Note: Local timezone is %%z (%%Z)'; grep '<ip>' <logpath>) | <mailcmd> "Abuse from <ip>" <mailargs> $ADDRESSES
             fi
 
 # Option:  actionunban
@@ -78,7 +78,7 @@ logpath = /dev/null
 
 # Option:  mailcmd
 # Notes.:  Your system mail command. Is passed 2 args: subject and recipient
-# Values:  CMD  Default: mail -s
+# Values:  CMD
 #
 mailcmd = mail -s
 
@@ -89,7 +89,7 @@ mailcmd = mail -s
 #          Appear to come from a different address - the '--' indicates
 #          arguments to be passed to Sendmail:
 #              -- -f me@example.com
-# Values:  [ STRING ]  Default: (empty)
+# Values:  [ STRING ]
 #
 mailargs =
 

config/action.d/dshield.conf

@@ -39,7 +39,7 @@ actionstart =
 # Values:  CMD
 #
 actionstop = if [ -f <tmpfile>.buffer ]; then
-                 cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ `date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'` Fail2Ban" <dest> <mailargs>
+                 cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ `date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'` Fail2Ban" <mailargs> <dest>
                  date +%%s > <tmpfile>.lastsent
              fi
              rm -f <tmpfile>.buffer <tmpfile>.first
@@ -80,7 +80,7 @@ actionban = TZONE=`date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'`
             LASTREPORT=$(($NOW - `cat <tmpfile>.lastsent`))
             LINES=$( wc -l <tmpfile>.buffer | awk '{ print $1 }' )
             if [ $LINES -ge <lines> && $LASTREPORT -gt <minreportinterval> ] || [ $LOGAGE -gt <maxbufferage> ]; then
-                cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ $TZONE Fail2Ban" <dest> <mailargs>
+                cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ $TZONE Fail2Ban" <mailargs> <dest>
                 rm -f <tmpfile>.buffer <tmpfile>.first
                 echo $NOW > <tmpfile>.lastsent
             fi
@@ -95,7 +95,7 @@ actionunban = if [ -f <tmpfile>.first ]; then
                   NOW=`date +%%s`
                   LOGAGE=$(($NOW - `cat <tmpfile>.first`))
                   if [ $LOGAGE -gt <maxbufferage> ]; then
-                      cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ `date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'` Fail2Ban" <dest> <mailargs>
+                      cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ `date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'` Fail2Ban" <mailargs> <dest>
                       rm -f <tmpfile>.buffer <tmpfile>.first
                       echo $NOW > <tmpfile>.lastsent
                   fi
@@ -106,7 +106,7 @@ actionunban = if [ -f <tmpfile>.first ]; then
 # Option:  port
 # Notes.:  The target port for the attack (numerical). MUST be provided in the
 #          jail config, as it cannot be detected here.
-# Values:  [ NUM ]  Default: ???
+# Values:  [ NUM ]
 #
 port = ???
 
@@ -114,7 +114,7 @@ port = ???
 # Notes.:  Your DShield user ID. Should be provided either in the jail config or
 #          in a .local file.
 #          Register at https://secure.dshield.org/register.html
-# Values:  [ NUM ]  Default: 0
+# Values:  [ NUM ]
 #
 userid = 0
 
@@ -137,7 +137,7 @@ protocol = tcp
 # Notes.:  How many lines to buffer before making a report. Regardless of this,
 #          reports are sent a minimum of <minreportinterval> apart, or if the
 #          buffer contains an event over <maxbufferage> old, or on shutdown
-# Values:  [ NUM ]  Default: 50
+# Values:  [ NUM ]
 #
 lines = 50
 
@@ -145,7 +145,7 @@ lines = 50
 # Notes.:  Minimum period (in seconds) that must elapse before we submit another
 #          batch of reports. DShield request a minimum of 1 hour (3600 secs)
 #          between reports.
-# Values:  [ NUM ]  Default: 3600
+# Values:  [ NUM ]
 #
 minreportinterval = 3600
 
@@ -154,27 +154,27 @@ minreportinterval = 3600
 #          submit the batch, even if we haven't reached <lines> yet. Note that
 #          this is only checked on each ban/unban, and that we always send
 #          anything in the buffer on shutdown. Must be greater than
-# Values:  [ NUM ]  Default: 21600 (6 hours)
+# Values:  [ NUM ]
 #
 maxbufferage = 21600
 
 # Option:  srcport
 # Notes.:  The source port of the attack. You're unlikely to have this info, so
 #          you can leave the default
-# Values:  [ NUM ]  Default: ???
+# Values:  [ NUM ]
 #
 srcport = ???
 
 # Option:  tcpflags
 # Notes.:  TCP flags on attack. You're unlikely to have this info, so you can
 #          leave empty
-# Values:  [ STRING ]  Default: (empty)
+# Values:  [ STRING ]
 #
 tcpflags =
 
 # Option:  mailcmd
 # Notes.:  Your system mail command. Is passed 2 args: subject and recipient
-# Values:  CMD  Default: mail -s
+# Values:  CMD
 #
 mailcmd = mail -s
 
@@ -186,19 +186,19 @@ mailcmd = mail -s
 #          the one configured at DShield - the '--' indicates arguments to be
 #          passed to Sendmail):
 #              -- -f me@example.com
-# Values:  [ STRING ]  Default: (empty)
+# Values:  [ STRING ]
 #
 mailargs =
 
 # Option:  dest
 # Notes.:  Destination e-mail address for reports
-# Values:  [ STRING ]  Default: reports@dshield.org
+# Values:  [ STRING ]
 #
 dest = reports@dshield.org
 
 # Option:  tmpfile
 # Notes.:  Base name of temporary files used for buffering
-# Values:  [ STRING ]  Default: /var/run/fail2ban/tmp-dshield
+# Values:  [ STRING ]
 #
 tmpfile = /var/run/fail2ban/tmp-dshield
 

config/action.d/dummy.conf

@@ -10,14 +10,14 @@
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = touch /tmp/fail2ban.dummy
+actionstart = touch /var/run/fail2ban/fail2ban.dummy
-              printf %%b "<init>\n" >> /tmp/fail2ban.dummy
+              printf %%b "<init>\n" >> /var/run/fail2ban/fail2ban.dummy
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = rm -f /tmp/fail2ban.dummy
+actionstop = rm -f /var/run/fail2ban/fail2ban.dummy
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -31,7 +31,7 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "+<ip>\n" >> /tmp/fail2ban.dummy
+actionban = printf %%b "+<ip>\n" >> /var/run/fail2ban/fail2ban.dummy
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -39,7 +39,7 @@ actionban = printf %%b "+<ip>\n" >> /tmp/fail2ban.dummy
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = printf %%b "-<ip>\n" >> /tmp/fail2ban.dummy
+actionunban = printf %%b "-<ip>\n" >> /var/run/fail2ban/fail2ban.dummy
 
 [Init]
 

config/action.d/firewall-cmd-direct-new.conf

@@ -0,0 +1,52 @@
+# Fail2Ban configuration file
+#
+# Author: Edgar Hoch
+# Copied from iptables-new.conf and modified for use with firewalld by Edgar Hoch.
+#  It uses "firewall-cmd" instead of "iptables".
+#
+# Because of the --remove-rules in stop this action requires firewalld-0.3.8+
+
+[INCLUDES]
+
+before = iptables-blocktype.conf
+
+[Definition]
+
+actionstart = firewall-cmd --direct --add-chain ipv4 filter fail2ban-<name>
+              firewall-cmd --direct --add-rule ipv4 filter fail2ban-<name> 1000 -j RETURN
+              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -m state --state NEW -p <protocol> --dport <port> -j fail2ban-<name>
+
+actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -m state --state NEW -p <protocol> --dport <port> -j fail2ban-<name>
+             firewall-cmd --direct --remove-rules ipv4 filter fail2ban-<name>
+             firewall-cmd --direct --remove-chain ipv4 filter fail2ban-<name>
+
+actioncheck = firewall-cmd --direct --get-chains ipv4 filter | grep -q 'fail2ban-<name>[ \t]'
+
+actionban = firewall-cmd --direct --add-rule ipv4 filter fail2ban-<name> 0 -s <ip> -j <blocktype>
+
+actionunban = firewall-cmd --direct --remove-rule ipv4 filter fail2ban-<name> 0 -s <ip> -j <blocktype>
+
+[Init]
+
+# Default name of the chain
+#
+name = default
+
+# Option:  port
+# Notes.:  specifies port to monitor
+# Values:  [ NUM | STRING ]
+#
+port = ssh
+
+# Option:  protocol
+# Notes.:  internally used by config reader for interpolations.
+# Values:  [ tcp | udp | icmp | all ]
+#
+protocol = tcp
+
+# Option:  chain
+# Notes    specifies the iptables chain to which the fail2ban rules should be
+#          added
+# Values:  [ STRING ]
+#
+chain = INPUT_direct

config/action.d/hostsdeny.conf

@@ -1,6 +1,7 @@
 # Fail2Ban configuration file
 #
 # Author: Cyril Jaquier
+# Edited for cross platform by: James Stout, Yaroslav Halchenko and Daniel Black
 #
 #
 
@@ -31,7 +32,7 @@ actioncheck =
 # Values:  CMD
 #
 actionban = IP=<ip> &&
-            printf %%b "ALL: $IP\n" >> <file>
+            printf %%b "<daemon_list>: $IP\n" >> <file>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -39,7 +40,7 @@ actionban = IP=<ip> &&
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = IP=<ip> && sed  /ALL:\ $IP/d <file> > <file>.new && mv <file>.new <file>
+actionunban = echo "/^<daemon_list>: <ip>$/<br>d<br>w<br>q" | ed <file>
 
 [Init]
 
@@ -48,3 +49,9 @@ actionunban = IP=<ip> && sed  /ALL:\ $IP/d <file> > <file>.new && mv <file>.new
 # Values:  STR  Default:  /etc/hosts.deny
 #
 file = /etc/hosts.deny
+
+# Option:  daemon_list
+# Notes:   The list of services that this action will deny. See the man page
+#          for hosts.deny/hosts_access. Default is all services.
+# Values:  STR  Default: ALL
+daemon_list = ALL

config/action.d/iptables-ipset-proto4.conf

@@ -11,12 +11,11 @@
 # IPset was a feature introduced in the linux kernel 2.6.39 and 3.0.0 kernels.
 #
 # If you are running on an older kernel you make need to patch in external
-# modules.
+# modules. Debian squeeze can do this with:
+#   apt-get install xtables-addons-source 
+#   module-assistant auto-install xtables-addons
 #
-# On Debian machines this can be done with:
+# Debian wheezy and above uses protocol 6
-#
-# apt-get install ipset xtables-addons-source 
-# module-assistant auto-install xtables-addons
 
 [INCLUDES]
 

config/action.d/iptables-ipset-proto6-allports.conf

@@ -0,0 +1,64 @@
+# Fail2Ban configuration file
+#
+# Author: Daniel Black
+#
+# This is for ipset protocol 6 (and hopefully later) (ipset v6.14).
+# Use ipset -V to see the protocol and version. Version 4 should use
+# iptables-ipset-proto4.conf.
+#
+# This requires the program ipset which is normally in package called ipset.
+#
+# IPset was a feature introduced in the linux kernel 2.6.39 and 3.0.0 kernels.
+#
+# If you are running on an older kernel you make need to patch in external
+# modules which probably won't be protocol version 6.
+
+[INCLUDES]
+
+before = iptables-blocktype.conf
+
+
+[Definition]
+
+# Option:  actionstart
+# Notes.:  command executed once at the start of Fail2Ban.
+# Values:  CMD
+#
+actionstart = ipset create fail2ban-<name> hash:ip timeout <bantime>
+              iptables -I INPUT -m set --match-set fail2ban-<name> src -j <blocktype>
+
+# Option:  actionstop
+# Notes.:  command executed once at the end of Fail2Ban
+# Values:  CMD
+#
+actionstop = iptables -D INPUT -m set --match-set fail2ban-<name> src -j <blocktype>
+             ipset flush fail2ban-<name>
+             ipset destroy fail2ban-<name>
+
+# Option:  actionban
+# Notes.:  command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionban = ipset add fail2ban-<name> <ip> timeout <bantime> -exist
+
+# Option:  actionunban
+# Notes.:  command executed when unbanning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionunban = ipset del fail2ban-<name> <ip> -exist
+
+[Init]
+
+# Default name of the ipset
+#
+name = default
+
+# Option: bantime
+# Notes:  specifies the bantime in seconds (handled internally rather than by fail2ban)
+# Values:  [ NUM ]  Default: 600
+
+bantime = 600

config/action.d/iptables-ipset-proto6.conf

@@ -12,11 +12,6 @@
 #
 # If you are running on an older kernel you make need to patch in external
 # modules.
-#
-# On Debian machines this can be done with:
-#
-# apt-get install ipset xtables-addons-source 
-# module-assistant auto-install xtables-addons
 
 [INCLUDES]
 
@@ -30,13 +25,13 @@ before = iptables-blocktype.conf
 # Values:  CMD
 #
 actionstart = ipset create fail2ban-<name> hash:ip timeout <bantime>
-              iptables -I INPUT -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j DROP
+              iptables -I INPUT -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j <blocktype>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = iptables -D INPUT -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j DROP
+actionstop = iptables -D INPUT -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j <blocktype>
              ipset flush fail2ban-<name>
              ipset destroy fail2ban-<name>
 

config/action.d/mail-buffered.conf

@@ -14,7 +14,7 @@ actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Output will be buffered until <lines> lines are available.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started" <dest>
+              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -25,13 +25,13 @@ actionstop = if [ -f <tmpfile> ]; then
                  These hosts have been banned by Fail2Ban.\n
                  `cat <tmpfile>`
                  Regards,\n
-                 Fail2Ban"|mail -s "[Fail2Ban] <name>: Summary" <dest>
+                 Fail2Ban"|mail -s "[Fail2Ban] <name>: Summary from `uname -n`" <dest>
                  rm <tmpfile>
              fi
              printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped" <dest>
+             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command

config/action.d/mail-whois-lines.conf

@@ -13,7 +13,7 @@
 actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started" <dest>
+              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -22,7 +22,7 @@ actionstart = printf %%b "Hi,\n
 actionstop = printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped" <dest>
+             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -44,7 +44,7 @@ actionban = printf %%b "Hi,\n
             Lines containing IP:<ip> in <logpath>\n
             `grep '\<<ip>\>' <logpath>`\n\n
             Regards,\n
-            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip>" <dest>
+            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from  `uname -n`" <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the

config/action.d/mail-whois.conf

@@ -13,7 +13,7 @@
 actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started" <dest>
+              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -22,7 +22,7 @@ actionstart = printf %%b "Hi,\n
 actionstop = printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped" <dest>
+             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -42,7 +42,7 @@ actionban = printf %%b "Hi,\n
             Here are more information about <ip>:\n
             `whois <ip>`\n
             Regards,\n
-            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip>" <dest>
+            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the

config/action.d/mail.conf

@@ -13,7 +13,7 @@
 actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started" <dest>
+              Fail2Ban"|mail -s "[Fail2Ban] <name>: started  on `uname -n`" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -22,7 +22,7 @@ actionstart = printf %%b "Hi,\n
 actionstop = printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped" <dest>
+             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -40,7 +40,7 @@ actionban = printf %%b "Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
             <failures> attempts against <name>.\n
             Regards,\n
-            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip>" <dest>
+            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the

config/action.d/osx-afctl.conf

@@ -0,0 +1,16 @@
+# Fail2Ban configuration file for using afctl on Mac OS X Server 10.5
+#
+# Anonymous author
+# http://www.fail2ban.org/wiki/index.php?title=HOWTO_Mac_OS_X_Server_(10.5)&diff=prev&oldid=4081
+#
+# Ref: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man8/afctl.8.html
+
+[Definition]
+actionstart = 
+actionstop = 
+actioncheck = 
+actionban = /usr/libexec/afctl -a <ip> -t <bantime>
+actionunban = /usr/libexec/afctl -r <ip>
+
+[Init]
+bantime = 2880

config/action.d/osx-ipfw.conf

@@ -0,0 +1,87 @@
+# Fail2Ban configuration file
+#
+# Author: Nick Munger
+# Modified by: Andy Fragen and Daniel Black
+#
+# Mod for OS X, using random rulenum as OSX ipfw doesn't include tables
+#
+
+[Definition]
+
+# Option:  actionstart
+# Notes.:  command executed once at the start of Fail2Ban.
+# Values:  CMD
+#
+actionstart = 
+
+
+# Option:  actionstop
+# Notes.:  command executed once at the end of Fail2Ban
+# Values:  CMD
+#
+actionstop = 
+
+
+# Option:  actioncheck
+# Notes.:  command executed once before each actionban command
+# Values:  CMD
+#
+actioncheck = 
+
+
+# Option:  actionban
+# Notes.:  command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    <ip>  IP address
+# Values:  CMD
+#
+actionban = ipfw add <rulenum> set <setnum> <blocktype> log <block> from <ip> to <dst> <port>
+
+
+# Option:  actionunban
+# Notes.:  command executed when unbanning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    <ip>  IP address
+# Values:  CMD
+#
+actionunban = ipfw delete `ipfw -S list | grep -i 'set <setnum> <blocktype> log <block> from <ip> to <dst>' | awk '{print $1;}'`
+
+[Init]
+
+# Option:  port
+# Notes.:  specifies port to block. Can be blank however may require block="ip"
+# Values:  [ NUM | STRING ]
+#
+port = ssh
+
+# Option:  dst
+# Notes.:  the local IP address of the network interface
+# Values:  IP, any, me or anything support by ipfw as a dst
+#
+dst = me
+
+# Option: block
+# Notes:  This is how much to block.
+#         Can be "ip", "tcp", "udp" or various other options.
+# Values: STRING
+block = tcp
+
+# Option:  blocktype
+# Notes.:  How to block the traffic. Use a action from man 8 ipfw
+#          Common values: deny, unreach port, reset
+# Values:  STRING
+#
+blocktype = unreach port
+
+# Option:  set number
+# Notes.:  The ipset number this is added to.
+# Values:  0-31
+setnum = 10
+
+# Option:  number for ipfw rule
+# Notes:   This is meant to be automaticly generated and not overwritten
+# Values:  Random value between 10000 and 12000
+rulenum="`echo $((RANDOM%%2000+10000))`"
+
+# Duplicate prevention mechanism
+#rulenum = "`a=$((RANDOM%%2000+10000)); while ipfw show | grep -q ^$a\ ; do a=$((RANDOM%%2000+10000)); done; echo $a`"

config/action.d/pf.conf

@@ -56,7 +56,7 @@ actionunban = /sbin/pfctl -t <tablename> -T delete <ip>/32
 [Init]
 # Option:  tablename
 # Notes.:  The pf table name.
-# Values:  [ STRING ]  Default: fail2ban
+# Values:  [ STRING ]
 #
 tablename = fail2ban
 

config/action.d/sendmail-buffered.conf

@@ -4,14 +4,18 @@
 #
 #
 
+[INCLUDES]
+
+before = sendmail-common.conf
+
 [Definition]
 
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
+actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
-              From: Fail2Ban <<sender>>
+              From: <sendername> <<sender>>
               To: <dest>\n
               Hi,\n
               The jail <name> has been started successfully.\n
@@ -24,8 +28,8 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
 # Values:  CMD
 #
 actionstop = if [ -f <tmpfile> ]; then
-                 printf %%b "Subject: [Fail2Ban] <name>: summary
+                 printf %%b "Subject: [Fail2Ban] <name>: summary from `uname -n`
-                 From: Fail2Ban <<sender>>
+                 From: <sendername> <<sender>>
                  To: <dest>\n
                  Hi,\n
                  These hosts have been banned by Fail2Ban.\n
@@ -34,7 +38,7 @@ actionstop = if [ -f <tmpfile> ]; then
                  Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
                  rm <tmpfile>
              fi
-             printf %%b "Subject: [Fail2Ban] <name>: stopped
+             printf %%b "Subject: [Fail2Ban] <name>: stopped  on `uname -n`
              From: Fail2Ban <<sender>>
              To: <dest>\n
              Hi,\n
@@ -57,8 +61,8 @@ actioncheck =
 actionban = printf %%b "`date`: <ip> (<failures> failures)\n" >> <tmpfile>
             LINE=$( wc -l <tmpfile> | awk '{ print $1 }' )
             if [ $LINE -ge <lines> ]; then
-                printf %%b "Subject: [Fail2Ban] <name>: summary
+                printf %%b "Subject: [Fail2Ban] <name>: summary from `uname -n`
-                From: Fail2Ban <<sender>>
+                From: <sendername> <<sender>>
                 To: <dest>\n
                 Hi,\n
                 These hosts have been banned by Fail2Ban.\n
@@ -82,14 +86,6 @@ actionunban =
 #
 name = default
 
-# Destination/Addressee of the mail
-#
-dest = root
-
-# Sender of the mail
-#
-sender = fail2ban
-
 # Default number of lines that are buffered
 #
 lines = 5

config/action.d/sendmail-common.conf

@@ -0,0 +1,23 @@
+# Fail2Ban configuration file
+#
+# Common settings for sendmail actions
+#
+# Users can override the defaults in sendmail-common.local
+
+[INCLUDES]
+
+after = sendmail-common.local
+
+[Init]
+
+# Recipient mail address
+#
+dest = root
+
+# Sender mail address
+#
+sender = fail2ban
+
+# Sender display name
+#
+sendername = Fail2Ban

config/action.d/sendmail-whois-lines.conf

@@ -4,15 +4,19 @@
 #
 #
 
+[INCLUDES]
+
+before = sendmail-common.conf
+
 [Definition]
 
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
+actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
               Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
-              From: Fail2Ban <<sender>>
+              From: <sendername> <<sender>>
               To: <dest>\n
               Hi,\n
               The jail <name> has been started successfully.\n
@@ -23,9 +27,9 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped
+actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on `uname -n`
              Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
-             From: Fail2Ban <<sender>>
+             From: <sendername> <<sender>>
              To: <dest>\n
              Hi,\n
              The jail <name> has been stopped.\n
@@ -44,9 +48,9 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
+actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
             Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
-            From: Fail2Ban <<sender>>
+            From: <sendername> <<sender>>
             To: <dest>\n
             Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
@@ -72,14 +76,6 @@ actionunban =
 #
 name = default
 
-# Destination/Addressee of the mail
-#
-dest = root
-
-# Sender of the mail
-#
-sender = fail2ban
-
 # Path to the log files which contain relevant lines for the abuser IP
 #
 logpath = /dev/null

config/action.d/sendmail-whois.conf

@@ -4,15 +4,19 @@
 #
 #
 
+[INCLUDES]
+
+before = sendmail-common.conf
+
 [Definition]
 
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
+actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
               Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
-              From: Fail2Ban <<sender>>
+              From: <sendername> <<sender>>
               To: <dest>\n
               Hi,\n
               The jail <name> has been started successfully.\n
@@ -23,9 +27,9 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped
+actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on `uname -n`
              Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
-             From: Fail2Ban <<sender>>
+             From: <sendername> <<sender>>
              To: <dest>\n
              Hi,\n
              The jail <name> has been stopped.\n
@@ -44,9 +48,9 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
+actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
             Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
-            From: Fail2Ban <<sender>>
+            From: <sendername> <<sender>>
             To: <dest>\n
             Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
@@ -70,11 +74,3 @@ actionunban =
 #
 name = default
 
-# Destination/Addressee of the mail
-#
-dest = root
-
-# Sender of the mail
-#
-sender = fail2ban
-

config/action.d/sendmail.conf

@@ -4,15 +4,19 @@
 #
 #
 
+[INCLUDES]
+
+before = sendmail-common.conf
+
 [Definition]
 
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
+actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
               Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
-              From: Fail2Ban <<sender>>
+              From: <sendername> <<sender>>
               To: <dest>\n
               Hi,\n
               The jail <name> has been started successfully.\n
@@ -23,9 +27,9 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped
+actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on `uname -n`
              Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
-             From: Fail2Ban <<sender>>
+             From: <sendername> <<sender>>
              To: <dest>\n
              Hi,\n
              The jail <name> has been stopped.\n
@@ -44,9 +48,9 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
+actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
             Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
-            From: Fail2Ban <<sender>>
+            From: <sendername> <<sender>>
             To: <dest>\n
             Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
@@ -68,11 +72,3 @@ actionunban =
 #
 name = default
 
-# Destination/Addressee of the mail
-#
-dest = root
-
-# Sender of the mail
-#
-sender = fail2ban
-

config/fail2ban.conf

@@ -11,24 +11,24 @@
 
 [Definition]
 
-# Option:  loglevel
+# Option: loglevel
-# Notes.:  Set the log level output.
+# Notes.: Set the log level output.
-#          1 = ERROR
+#         1 = ERROR
-#          2 = WARN
+#         2 = WARN
-#          3 = INFO
+#         3 = INFO
-#          4 = DEBUG
+#         4 = DEBUG
-# Values:  NUM  Default:  3
+# Values: [ NUM ]  Default: 1
 #
 loglevel = 3
 
-# Option:  logtarget
+# Option: logtarget
-# Notes.:  Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
+# Notes.: Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
-#          Only one log target can be specified.
+#         Only one log target can be specified.
-#          If you change logtarget from the default value and you are
+#         If you change logtarget from the default value and you are
-#          using logrotate -- also adjust or disable rotation in the
+#         using logrotate -- also adjust or disable rotation in the
-#          corresponding configuration file
+#         corresponding configuration file
-#          (e.g. /etc/logrotate.d/fail2ban on Debian systems)
+#         (e.g. /etc/logrotate.d/fail2ban on Debian systems)
-# Values:  STDOUT STDERR SYSLOG file  Default:  /var/log/fail2ban.log
+# Values: [ STDOUT | STDERR | SYSLOG | FILE ]  Default: STDERR
 #
 logtarget = /var/log/fail2ban.log
 
@@ -36,14 +36,14 @@ logtarget = /var/log/fail2ban.log
 # Notes.: Set the socket file. This is used to communicate with the daemon. Do
 #         not remove this file when Fail2ban runs. It will not be possible to
 #         communicate with the server afterwards.
-# Values: FILE  Default:  /var/run/fail2ban/fail2ban.sock
+# Values: [ FILE ]  Default: /var/run/fail2ban/fail2ban.sock
 #
 socket = /var/run/fail2ban/fail2ban.sock
 
 # Option: pidfile
 # Notes.: Set the PID file. This is used to store the process ID of the
 #         fail2ban server.
-# Values: FILE  Default:  /var/run/fail2ban/fail2ban.pid
+# Values: [ FILE ]  Default: /var/run/fail2ban/fail2ban.pid
 #
 pidfile = /var/run/fail2ban/fail2ban.pid
 

config/filter.d/3proxy.conf

@@ -0,0 +1,18 @@
+# Fail2Ban filter for 3proxy
+#
+#
+
+[Definition]
+
+
+failregex = ^\s[+-]\d{4} \S+ \d{3}0[1-9] \S+ <HOST>:\d+ [\d.]+:\d+ \d+ \d+ \d+\s
+
+ignoreregex = 
+
+# DEV Notes:
+# http://www.3proxy.ru/howtoe.asp#ERRORS indicates that 01-09 are
+# all authentication problems (%E field)
+# Log format is: "L%d-%m-%Y %H:%M:%S %z %N.%p %E %U %C:%c %R:%r %O %I %h %T"
+#
+# Requested by ykimon in https://github.com/fail2ban/fail2ban/issues/246
+# Author: Daniel Black

config/filter.d/apache-auth.conf

@@ -1,28 +1,54 @@
-# Fail2Ban configuration file
+# Fail2Ban apache-auth filter
-#
-# Author: Cyril Jaquier
-#
 #
 
 [INCLUDES]
 
 # Read common prefixes. If any customizations available -- read them from
-# common.local
+# apache-common.local
 before = apache-common.conf
 
 [Definition]
 
-# Option:  failregex
-# Notes.:  regex to match the password failure messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
-failregex = ^%(_apache_error_client)s user .* (authentication failure|not found|password mismatch)\s*$
 
-# Option:  ignoreregex
+failregex = ^%(_apache_error_client)s (AH01797: )?client denied by server configuration: (uri )?\S*\s*$
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
+            ^%(_apache_error_client)s (AH01617: )?user .* authentication failure for "\S*": Password Mismatch$
-# Values:  TEXT
+            ^%(_apache_error_client)s (AH01618: )?user .* not found(: )?\S*\s*$
-#
+            ^%(_apache_error_client)s (AH01614: )?client used wrong authentication scheme: \S*\s*$
+            ^%(_apache_error_client)s (AH\d+: )?Authorization of user \S+ to access \S* failed, reason: .*$
+            ^%(_apache_error_client)s (AH0179[24]: )?(Digest: )?user .*: password mismatch: \S*\s*$
+            ^%(_apache_error_client)s (AH0179[01]: |Digest: )user `.*' in realm `.+' (not found|denied by provider): \S*\s*$
+            ^%(_apache_error_client)s (AH01631: )?user .*: authorization failure for "\S*":\s*$
+            ^%(_apache_error_client)s (AH01775: )?(Digest: )?invalid nonce .* received - length is not \S+\s*$
+            ^%(_apache_error_client)s (AH01788: )?(Digest: )?realm mismatch - got `.*' but expected `.+'\s*$
+            ^%(_apache_error_client)s (AH01789: )?(Digest: )?unknown algorithm `.*' received: \S*\s*$
+            ^%(_apache_error_client)s (AH01793: )?invalid qop `.*' received: \S*\s*$
+            ^%(_apache_error_client)s (AH01777: )?(Digest: )?invalid nonce .* received - user attempted time travel\s*$
+
 ignoreregex = 
+
+# DEV Notes:
+#
+# This filter matches the authorization failures of Apache. It takes the log messages
+# from the modules in aaa that return HTTP_UNAUTHORIZED, HTTP_METHOD_NOT_ALLOWED or
+# HTTP_FORBIDDEN and not AUTH_GENERAL_ERROR or HTTP_INTERNAL_SERVER_ERROR.
+#
+# An unauthorized response 401 is the first step for a browser to instigate authentication
+# however apache doesn't log this as an error. Only subsequent errors are logged in the 
+# error log.
+#
+# Source:
+#
+# By searching the code in http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/aaa/*
+# for ap_log_rerror(APLOG_MARK, APLOG_ERR and examining resulting return code should get
+# all of these expressions. Lots of submodules like mod_authz_* return back to mod_authz_core
+# to return the actual failure.
+#
+# See also: http://wiki.apache.org/httpd/ListOfErrors
+# Expressions that don't have tests and aren't common.
+# more be added with  https://issues.apache.org/bugzilla/show_bug.cgi?id=55284 
+#     ^%(_apache_error_client)s (AH01778: )?user .*: nonce expired \([\d.]+ seconds old - max lifetime [\d.]+\) - sending new nonce\s*$
+#     ^%(_apache_error_client)s (AH01779: )?user .*: one-time-nonce mismatch - sending new nonce\s*$
+#     ^%(_apache_error_client)s (AH02486: )?realm mismatch - got `.*' but no realm specified\s*$
+#
+# Author: Cyril Jaquier
+# Major edits by Daniel Black

config/filter.d/apache-badbots.conf

@@ -1,27 +1,21 @@
 # Fail2Ban configuration file
 #
-# List of bad bots fetched from http://www.user-agents.org
+# Regexp to catch known spambots and software alike. Please verify
-# Generated on Sun Feb 11 01:09:15 EST 2007 by ./badbots.sh
+# that it is your intent to block IPs which were driven by
-#
+# above mentioned bots.
-# Author: Yaroslav Halchenko
+
-#
-#
 
 [Definition]
 
 badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider
-badbots = atSpider/1\.0|autoemailspider|China Local Browse 2\.6|ContentSmartz|DataCha0s/2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Franklin Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Industry Program 1\.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsCrawl\.com/1\.0 +http\://letscrawl\.com/|Lincoln State Web Browser|LWP\:\:Simple/5\.803|Mac Finder 1\.0\.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control - 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missigua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|Mozilla/2\.0 \(compatible; NEWT ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/4\.0 \(compatible; Advanced Email Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0 \(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Version\: xxxx Type\:xx\)|MVAClient|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google\.com|sogou spider|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|WebVulnCrawl\.blogspot\.com/1\.0 libwww-perl/5\.803|Wells Search II|WEP Search 00
+badbots = Atomic_Email_Hunter/4\.0|atSpider/1\.0|autoemailspider|bwh3_user_agent|China Local Browse 2\.6|ContactBot/0\.2|ContentSmartz|DataCha0s/2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailSpider|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Franklin Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Guestbook Auto Submitter|Industry Program 1\.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsCrawl\.com/1\.0 +http\://letscrawl\.com/|Lincoln State Web Browser|LMQueueBot/0\.2|LWP\:\:Simple/5\.803|Mac Finder 1\.0\.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control - 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missigua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|MVAClient|Mozilla/2\.0 \(compatible; NEWT ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/3\.0 \(compatible; scan4mail \(advanced version\) http\://www\.peterspages\.net/?scan4mail\)|Mozilla/4\.0 \(compatible; Advanced Email Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0 \(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Version\: xxxx Type\:xx\)|NameOfAgent \(CMS Spider\)|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google\.com|ShablastBot 1\.0|snap\.com beta crawler v0|Snapbot/1\.0|Snapbot/1\.0 \(Snap Shots, +http\://www\.snap\.com\)|sogou develop spider|Sogou Orion spider/3\.0\(+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sogou spider|Sogou web spider/3\.0\(+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|VadixBot|WebVulnCrawl\.unknown/1\.0 libwww-perl/5\.803|Wells Search II|WEP Search 00
 
-# Option:  failregex
-# Notes.:  Regexp to catch known spambots and software alike. Please verify
-#          that it is your intent to block IPs which were driven by
-#          above mentioned bots.
-# Values:  TEXT
-#
 failregex = ^<HOST> -.*"(GET|POST).*HTTP.*"(?:%(badbots)s|%(badbotscustom)s)"$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex =
+
+# DEV Notes:
+# List of bad bots fetched from http://www.user-agents.org
+# Generated on Thu Nov  7 14:23:35 PST 2013 by files/gen_badbots.
+#
+# Author: Yaroslav Halchenko

config/filter.d/apache-common.conf

@@ -1,17 +1,20 @@
 # Generic configuration items (to be used as interpolations) in other
-# apache filters
+# apache filters.
-#
-# Author: Yaroslav Halchenko
-#
-#
 
 [INCLUDES]
 
 # Load customizations if any available
 after = apache-common.local
 
-
 [DEFAULT]
 
+_apache_error_client = \[[^]]*\] \[(error|\S+:\S+)\]( \[pid \d+:\S+ \d+\])? \[client <HOST>(:\d{1,5})?\]
+
 # Common prefix for [error] apache messages which also would include <HOST>
-_apache_error_client = \[[^]]+\] \[error\] \[client <HOST>\]
+# Depending on the version it could be
+# 2.2: [Sat Jun 01 11:23:08 2013] [error] [client 1.2.3.4]
+# 2.4: [Thu Jun 27 11:55:44.569531 2013] [core:info] [pid 4101:tid 2992634688] [client 1.2.3.4:46652]
+#
+# Reference: https://github.com/fail2ban/fail2ban/issues/268
+#
+# Author: Yaroslav Halchenko

config/filter.d/apache-nohome.conf

@@ -1,28 +1,20 @@
-# Fail2Ban configuration file
+# Fail2Ban filter to web requests for home directories on Apache servers
-#
-# Author: Yaroslav O. Halchenko <debian@onerussian.com>
-#
 #
+# Regex to match failures to find a home directory on a server, which
+# became popular last days. Most often attacker just uses IP instead of
+# domain name -- so expect to see them in generic error.log if you have
+# per-domain log files.
 
 [INCLUDES]
 
-# Read common prefixes. If any customizations available -- read them from
+# overwrite with apache-common.local if _apache_error_client is incorrect.
-# common.local
 before = apache-common.conf
 
 [Definition]
 
-# Option:  failregex
-# Notes.:  regex to match failures to find a home directory on a server, which
-#          became popular last days. Most often attacker just uses IP instead of
-#          domain name -- so expect to see them in generic error.log if you have
-#          per-domain log files.
-# Values:  TEXT
-#
-failregex = ^%(_apache_error_client)s File does not exist: .*/~.*
 
-# Option:  ignoreregex
+failregex = ^%(_apache_error_client)s (AH00128: )?File does not exist: .*/~.*
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
+
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Yaroslav O. Halchenko <debian@onerussian.com>

config/filter.d/apache-noscript.conf

@@ -1,29 +1,18 @@
-# Fail2Ban configuration file
+# Fail2Ban filter to block web requests for scripts (on non scripted websites)
-#
-# Author: Cyril Jaquier
 #
 #
 
 [INCLUDES]
 
-# Read common prefixes. If any customizations available -- read them from
+# overwrite with apache-common.local if _apache_error_client is incorrect.
-# common.local
 before = apache-common.conf
 
 [Definition]
 
-# Option:  failregex
-# Notes.:  regex to match the password failure messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
 failregex = ^%(_apache_error_client)s (File does not exist|script not found or unable to stat): /\S*(\.php|\.asp|\.exe|\.pl)\s*$
             ^%(_apache_error_client)s script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+
+# Author: Cyril Jaquier

config/filter.d/apache-overflows.conf

@@ -1,25 +1,15 @@
-# Fail2Ban configuration file
+# Fail2Ban filter to block web requests on a long or suspicious nature
-#
-# Author: Tim Connors
-#
 #
 
 [INCLUDES]
 
-# Read common prefixes. If any customizations available -- read them from
+# overwrite with apache-common.local if _apache_error_client is incorrect.
-# common.local
 before = apache-common.conf
 
 [Definition]
 
-# Option:  failregex
-# Notes.:  Regexp to catch Apache overflow attempts.
-# Values:  TEXT
-#
 failregex = ^%(_apache_error_client)s (Invalid (method|URI) in request|request failed: URI too long|erroneous characters after protocol string)
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex =
+
+# Author: Tim Connors

config/filter.d/assp.conf

@@ -1,33 +1,24 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for Anti-Spam SMTP Proxy Server also known as ASSP
-# for Anti-Spam SMTP Proxy Server also known as ASSP
+# 
 #    Honmepage:   http://www.magicvillage.de/~Fritz_Borgstedt/assp/0003D91C-8000001C/
 #    ProjektSite: http://sourceforge.net/projects/assp/?source=directory
 #
-# Author: Enrico Labedzki (enrico.labedzki@deiwos.de)
 #
 
 [Definition] 
 
-# Option:  failregex
+__assp_actions = (?:dropping|refusing)
-# Notes.:  regex to match the SMTP failure messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+failregex = ^(:? \[SSL-out\])? <HOST> max sender authentication errors \(\d{,3}\) exceeded -- %(__assp_actions)s connection - after reply: \d{3} \d{1}\.\d{1}.\d{1} Error: authentication failed: \w+;$
-#          be used for standard IP/hostname matching and is only an alias for
+			^(?: \[SSL-out\])? <HOST> SSL negotiation with client failed: SSL accept attempt failed with unknown error.*:unknown protocol;$
-#          (?:::f{4,6}:)?(?P<host>\S+)
+			^ Blocking <HOST> - too much AUTH errors \(\d{,3}\);$
-# Values:  TEXT
+
+ignoreregex = 
+
+# DEV Notes:
 #
 # Examples: Apr-27-13 02:33:09 Blocking 217.194.197.97 - too much AUTH errors (41);
 #           Dec-29-12 17:10:31 [SSL-out] 200.247.87.82 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol;
 #           Dec-30-12 04:01:47 [SSL-out] 81.82.232.66 max sender authentication errors (5) exceeded 
-__assp_actions = (dropping|refusing)
-
-failregex = <HOST> max sender authentication errors \(\d{,3}\) exceeded -- %(__assp_actions)s connection - after reply: \d{3} \d{1}\.\d{1}.\d{1} Error: authentication failed: [a-zA-Z0-9]+;$
-			<HOST> SSL negotiation with client failed: SSL accept attempt failed with unknown error.*:unknown protocol;$
-			Blocking <HOST> - too much AUTH errors \(\d{,3}\);$
-
-
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
 #
-ignoreregex = 
+# Author: Enrico Labedzki (enrico.labedzki@deiwos.de)
-

config/filter.d/asterisk.conf

@@ -1,43 +1,23 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for asterisk authentication failures
 #
-# Author: Xavier Devlamynck
-#
-#
-
-
-[INCLUDES]
-
-# Read common prefixes. If any customizations available -- read them from
-# common.local
-before = common.conf
 
 [Definition]
 
-# Option:  failregex
+__pid_re = (?:\[\d+\])
-# Notes.:  regex to match the password failures messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+# All Asterisk log messages begin like this:
-#          be used for standard IP/hostname matching and is only an alias for
+log_prefix= \[\]\s*(?:NOTICE|SECURITY)%(__pid_re)s:?(?:\[\S+\d*\])? \S+:\d*
-#          (?:::f{4,6}:)?(?P<host>\S+)
+
-# Values:  TEXT
+failregex = ^%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - (Wrong password|No matching peer found|Username/auth name mismatch|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$
-#
+            ^%(log_prefix)s Call from '[^']*' \(<HOST>:\d+\) to extension '\d+' rejected because extension not found in context 'default'\.$
-failregex = NOTICE%(__pid_re)s [^:]+: Registration from '[^']*' failed for '<HOST>(:[0-9]+)?' - Wrong password$
+            ^%(log_prefix)s Host <HOST> failed to authenticate as '[^']*'$
-            NOTICE%(__pid_re)s [^:]+: Registration from '[^']*' failed for '<HOST>(:[0-9]+)?' - No matching peer found$
+            ^%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$
-            NOTICE%(__pid_re)s [^:]+: Registration from '[^']*' failed for '<HOST>(:[0-9]+)?' - Username/auth name mismatch$
+            ^%(log_prefix)s Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$
-            NOTICE%(__pid_re)s [^:]+: Registration from '[^']*' failed for '<HOST>(:[0-9]+)?' - Device does not match ACL$
+            ^%(log_prefix)s Failed to authenticate (user|device) [^@]+@<HOST>\S*$
-            NOTICE%(__pid_re)s [^:]+: Registration from '[^']*' failed for '<HOST>(:[0-9]+)?' - Peer is not supposed to register$
+            ^%(log_prefix)s (?:handle_request_subscribe: )?Sending fake auth rejection for (device|user) \d*<sip:[^@]+@<HOST>>;tag=\w+\S*$
-            NOTICE%(__pid_re)s [^:]+: Registration from '[^']*' failed for '<HOST>(:[0-9]+)?' - ACL error \(permit/deny\)$
+            ^%(log_prefix)s SecurityEvent="(FailedACL|InvalidAccountID|ChallengeResponseFailed|InvalidPassword)",EventTV="[\d-]+",Severity="[\w]+",Service="[\w]+",EventVersion="\d+",AccountID="\d+",SessionID="0x[\da-f]+",LocalAddress="IPV[46]/(UD|TC)P/[\da-fA-F:.]+/\d+",RemoteAddress="IPV[46]/(UD|TC)P/<HOST>/\d+"(,Challenge="\w+",ReceivedChallenge="\w+")?(,ReceivedHash="[\da-f]+")?$
-            NOTICE%(__pid_re)s [^:]+: Registration from '[^']*' failed for '<HOST>(:[0-9]+)?' - Not a local domain$
-            NOTICE%(__pid_re)s\[[^:]+\] [^:]+: Call from '[^']*' \(<HOST>:[0-9]+\) to extension '[0-9]+' rejected because extension not found in context 'default'.$
-            NOTICE%(__pid_re)s [^:]+: Host <HOST> failed to authenticate as '[^']*'$
-            NOTICE%(__pid_re)s [^:]+: No registration for peer '[^']*' \(from <HOST>\)$
-            NOTICE%(__pid_re)s [^:]+: Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$
-            NOTICE%(__pid_re)s [^:]+: Failed to authenticate user [^@]+@<HOST>\S*$
-            SECURITY%(__pid_re)s [^:]+: SecurityEvent="InvalidAccountID",EventTV="[0-9-]+",Severity="[a-zA-Z]+",Service="[a-zA-Z]+",EventVersion="[0-9]+",AccountID="[0-9]+",SessionID="0x[0-9a-f]+",LocalAddress="IPV[46]/(UD|TC)P/[0-9a-fA-F:.]+/[0-9]+",RemoteAddress="IPV[46]/(UD|TC)P/<HOST>/[0-9]+"$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex =
 
 
+# Author: Xavier Devlamynck

config/filter.d/common.conf

@@ -1,9 +1,6 @@
 # Generic configuration items (to be used as interpolations) in other
 # filters  or actions configurations
 #
-# Author: Yaroslav Halchenko
-#
-#
 
 [INCLUDES]
 
@@ -33,7 +30,7 @@ __daemon_extra_re = (?:\[ID \d+ \S+\])
 
 # Combinations of daemon name and PID
 # EXAMPLES: sshd[31607], pop(pam_unix)[4920]
-__daemon_combs_re = (?:%(__pid_re)s?:\s+%(__daemon_re)s|%(__daemon_re)s%(__pid_re)s?:)
+__daemon_combs_re = (?:%(__pid_re)s?:\s+%(__daemon_re)s|%(__daemon_re)s%(__pid_re)s?:?)
 
 # Some messages have a kernel prefix with a timestamp
 # EXAMPLES: kernel: [769570.846956]
@@ -41,12 +38,14 @@ __kernel_prefix = kernel: \[\d+\.\d+\]
 
 __hostname = \S+
 
+# A MD5 hex
+# EXAMPLES: 07:06:27:55:b0:e3:0c:3c:5a:28:2d:7c:7e:4c:77:5f
+__md5hex = (?:[\da-f]{2}:){15}[\da-f]{2}
 
 # bsdverbose is where syslogd is started with -v or -vv and results in <4.3> or
 # <auth.info> appearing before the host as per testcases/files/logs/bsd/*.
 __bsd_syslog_verbose = (<[^.]+\.[^.]+>)
 
-#
 # Common line prefixes (beginnings) which could be used in filters
 #
 #      [bsdverbose]? [hostname] [vserver tag] daemon_id spaces
@@ -54,3 +53,4 @@ __bsd_syslog_verbose = (<[^.]+\.[^.]+>)
 # This can be optional (for instance if we match named native log files)
 __prefix_line = \s*%(__bsd_syslog_verbose)s?\s*(?:%(__hostname)s )?(?:%(__kernel_prefix)s )?(?:@vserver_\S+ )?%(__daemon_combs_re)s?\s%(__daemon_extra_re)s?\s*
 
+# Author: Yaroslav Halchenko

config/filter.d/courierlogin.conf

@@ -1,23 +1,19 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for courier authentication failures
-#
-# Author: Christoph Haas
-# Modified by: Cyril Jaquier
-#
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
 [Definition]
 
-# Option:  failregex
+_daemon = (?:courier)?(?:imapd?|pop3d?)(?:login)?(?:-ssl)?
-# Notes.:  regex to match the password failures messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+failregex = ^%(__prefix_line)sLOGIN FAILED, user=.*, ip=\[<HOST>\]$
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
-failregex = LOGIN FAILED, .*, ip=\[<HOST>\]$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Christoph Haas
+# Modified by: Cyril Jaquier

config/filter.d/couriersmtp.conf

@@ -1,22 +1,19 @@
-# Fail2Ban configuration file
+# Fail2Ban filter to block relay attempts though a Courier smtp server
-#
-# Author: Cyril Jaquier
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
 [Definition]
 
-# Option:  failregex
+_daemon = courieresmtpd
-# Notes.:  regex to match the password failures messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+failregex = ^%(__prefix_line)serror,relay=<HOST>,.*: 550 User unknown\.$
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
-failregex = error,relay=<HOST>,.*550 User unknown
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Cyril Jaquier

config/filter.d/cyrus-imap.conf

@@ -1,25 +1,20 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for authentication failures on Cyrus imap server
-#
-# Author: Jan Wagner <waja@cyconet.org>
 #
 #
+#
+
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
 
 [Definition]
 
-# Option:  failregex
+_daemon = (?:cyrus/)?(?:imapd?|pop3d?)
-# Notes.:  regex to match the password failures messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+failregex = ^%(__prefix_line)sbadlogin: \S+ ?\[<HOST>\] \S+ .*?\[?SASL\(-13\): authentication failure: .*\]?$
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
-failregex = : badlogin: .*\[<HOST>\] plaintext .*SASL\(-13\): authentication failure: checkpass failed$
-	    : badlogin: .*\[<HOST>\] LOGIN \[SASL\(-13\): authentication failure: checkpass failed\]$
-	    : badlogin: .*\[<HOST>\] (?:CRAM-MD5|NTLM) \[SASL\(-13\): authentication failure: incorrect (?:digest|NTLM) response\]$
-	    : badlogin: .*\[<HOST>\] DIGEST-MD5 \[SASL\(-13\): authentication failure: client response doesn't match what we generated\]$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Jan Wagner <waja@cyconet.org>

config/filter.d/dovecot.conf

@@ -1,23 +1,23 @@
-# Fail2Ban configuration file for dovcot
+# Fail2Ban filter Dovecot authentication and pop3/imap server
-#
-# Author: Martin Waschbuesch
-#
 #
 
+[INCLUDES]
+
+before = common.conf
+
 [Definition]
 
-# Option:  failregex
+_daemon = (auth|dovecot(-auth)?|auth-worker)
-# Notes.:  regex to match the password failures messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+failregex = ^%(__prefix_line)s(pam_unix(\(dovecot:auth\))?:)?\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* rhost=<HOST>(\s+user=\S*)?\s*$
-#          be used for standard IP/hostname matching and is only an alias for
+            ^%(__prefix_line)s(pop3|imap)-login: (Info: )?(Aborted login|Disconnected)(: Inactivity)? \(((no auth attempts|auth failed, \d+ attempts)( in \d+ secs)?|tried to use (disabled|disallowed) \S+ auth)\):( user=<\S*>,)?( method=\S+,)? rip=<HOST>, lip=(\d{1,3}\.){3}\d{1,3}(, session=<\w+>)?(, TLS( handshaking)?(: Disconnected)?)?\s*$
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
+            ^%(__prefix_line)s(Info|dovecot: auth\(default\)): pam\(\S+,<HOST>\): pam_authenticate\(\) failed: (User not known to the underlying authentication module: \d+ Time\(s\)|Authentication failure \(password mismatch\?\))\s*$
-# Values:  TEXT
-#
-failregex = .*(?:pop3-login|imap-login):.*(?:Authentication failure|Aborted login \(auth failed|Aborted login \(tried to use disabled|Disconnected \(auth failed).*\s+rip=(?P<host>\S*),.*
-            pam.*dovecot.*(?:authentication failure).*\s+rhost=<HOST>(?:\s+user=.*)?\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# DEV Notes:
+# * the first regex is essentially a copy of pam-generic.conf
+# * Probably doesn't do dovecot sql/ldap backends properly
+#
+# Author: Martin Waschbuesch
+#         Daniel Black (rewrote with begin and end anchors)

config/filter.d/dropbear.conf

@@ -1,8 +1,15 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for dropbear
 #
-# Author: Francis Russell
+# NOTE: The regex below is ONLY intended to work with a patched
-#         Zak B. Elep
+# version of Dropbear as described here:
+# http://www.unchartedbackwaters.co.uk/pyblosxom/static/patches
+#            ^%(__prefix_line)sexit before auth from <HOST>.*\s*$
 #
+# The standard Dropbear output doesn't provide enough information to
+# ban all types of attack.  The Dropbear patch adds IP address
+# information to the 'exit before auth' message which is always
+# produced for any form of non-successful login. It is that message
+# which this file matches.
 #
 # More information: http://bugs.debian.org/546913
 
@@ -12,40 +19,23 @@
 # common.local
 before = common.conf
 
-
 [Definition]
 
 _daemon = dropbear
 
-# Option:  failregex
+failregex = ^%(__prefix_line)s[Ll]ogin attempt for nonexistent user ('.*' )?from <HOST>:.*$
-# Notes.:  regex to match the password failures messages in the logfile. The
+            ^%(__prefix_line)s[Bb]ad (PAM )?password attempt for .+ from <HOST>.*$
-#          host must be matched by a group named "host". The tag "<HOST>" can
+            ^%(__prefix_line)s[Ee]xit before auth \(user '.+', \d+ fails\): Max auth tries reached - user '.+' from <HOST>:\d+\s*$
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>\S+)
-# Values:  TEXT
 
-# These match the unmodified dropbear messages. It isn't possible to
-# match the source of the 'exit before auth' messages from dropbear.
-#
-failregex = ^%(__prefix_line)slogin attempt for nonexistent user ('.*' )?from <HOST>:.*\s*$
-            ^%(__prefix_line)sbad password attempt for .+ from <HOST>:.*\s*$
-
-# The only line we need to match with the modified dropbear.
-
-# NOTE: The failregex below is ONLY intended to work with a patched
-# version of Dropbear as described here:
-# http://www.unchartedbackwaters.co.uk/pyblosxom/static/patches
-#
-# The standard Dropbear output doesn't provide enough information to
-# ban all types of attack.  The Dropbear patch adds IP address
-# information to the 'exit before auth' message which is always
-# produced for any form of non-successful login. It is that message
-# which this file matches.
-
-# failregex = ^%(__prefix_line)sexit before auth from <HOST>.*\s*$
-
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# DEV Notes:
+#
+# The first two regexs here match the unmodified dropbear messages. It isn't
+# possible to match the source of the 'exit before auth' messages from dropbear
+# as they don't include the "from <HOST>" bit.
+#
+# The second last failregex line we need to match with the modified dropbear.
+#
+# Author: Francis Russell
+#         Zak B. Elep

config/filter.d/exim-common.conf

@@ -0,0 +1,18 @@
+# Fail2Ban filter file for common exim expressions
+#
+# This is to be used by other exim filters
+
+[INCLUDES]
+
+# Load customizations if any available
+after = exim-common.local
+
+[Definition]
+
+host_info = H=([\w.-]+ )?(\(\S+\) )?\[<HOST>\](:\d+)? (I=\[\S+\]:\d+ )?(U=\S+ )?(P=e?smtp )?
+pid = ( \[\d+\])?
+
+# DEV Notes:
+# From exim source code: ./src/receive.c:add_host_info_for_log
+#
+# Author:  Daniel Black

config/filter.d/exim-spam.conf

@@ -0,0 +1,22 @@
+# Fail2Ban filter for exim the spam rejection messages
+#
+
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# exim-common.local
+before = exim-common.conf
+
+[Definition]
+
+failregex =  ^%(pid)s \S+ F=(<>|\S+@\S+) %(host_info)srejected by local_scan\(\): .{0,256}$
+             ^%(pid)s %(host_info)sF=(<>|[^@]+@\S+) rejected RCPT [^@]+@\S+: .*dnsbl.*\s*$
+             ^%(pid)s \S+ %(host_info)sF=(<>|[^@]+@\S+) rejected after DATA: This message contains a virus \(\S+\)\.\s*$
+
+ignoreregex = 
+
+# DEV Notes:
+# The %(host_info) defination contains a <HOST> match
+#
+# Author: Cyril Jaquier
+#         Daniel Black (rewrote with strong regexs)

config/filter.d/exim.conf

@@ -1,23 +1,28 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for exim
-#
-# Author: Cyril Jaquier
 #
+# This includes the rejection messages of exim. For spam and filter
+# related bans use the exim-spam.conf
 #
 
+
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# exim-common.local
+before = exim-common.conf
+
 [Definition]
 
-# Option:  failregex
+failregex = ^%(pid)s %(host_info)ssender verify fail for <\S+>: (?:Unknown user|Unrouteable address|all relevant MX records point to non-existent hosts)\s*$
-# Notes.:  regex to match the password failures messages in the logfile. The
+             ^%(pid)s (plain|login) authenticator failed for (\S+ )?\(\S+\) \[<HOST>\]: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
-#          host must be matched by a group named "host". The tag "<HOST>" can
+             ^%(pid)s %(host_info)sF=(<>|[^@]+@\S+) rejected RCPT [^@]+@\S+: (relay not permitted|Sender verify failed|Unknown user)\s*$
-#          be used for standard IP/hostname matching and is only an alias for
+             ^%(pid)s SMTP protocol synchronization error \(.*\): rejected (connection from|"\S+") %(host_info)s(next )?input=".*"\s*$
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
+             ^%(pid)s SMTP call from \S+ \[<HOST>\](:\d+)? (I=\[\S+\]:\d+ )?dropped: too many nonmail commands \(last was "\S+"\)\s*$
-# Values:  TEXT
-#
-failregex = \[<HOST>\] .*(?:rejected by local_scan|Unrouteable address)
-             login authenticator failed for .* \[<HOST>\]: 535 Incorrect authentication data \(set_id=.*\)\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# DEV Notes:
+# The %(host_info) defination contains a <HOST> match
+#
+# Author: Cyril Jaquier
+#         Daniel Black (rewrote with strong regexs)

config/filter.d/gssftpd.conf

@@ -1,19 +1,18 @@
-# Fail2Ban configuration file for wuftpd
+# Fail2Ban filter file for gssftp
-#
-# Author: Kevin Zembower (copied from wsftpd.conf)
 #
+# Note: gssftp is part of the krb5-appl-servers in Fedora
 #
+[INCLUDES]
+
+before = common.conf
 
 [Definition]
 
-# Option: failregex
+_daemon = ftpd
-# Notes.: regex to match the password failures messages in the logfile.
+
-# Values: TEXT
+failregex = ^%(__prefix_line)srepeated login failures from <HOST> \(\S+\)$
-#
-failregex = ftpd(?:\[\d+\])?:\s+repeated login failures from <HOST> \(\S+\)$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Kevin Zembower
+# Edited: Daniel Black - syslog based daemon

config/filter.d/lighttpd-auth.conf

@@ -1,18 +1,10 @@
-# Fail2Ban configuration file
+# Fail2Ban filter to match wrong passwords as notified by lighttpd's auth Module
-#
-# Author: Francois Boulogne <fboulogne@april.org>
 #
 
 [Definition]
 
-# Option:  failregex
+failregex = ^: \(http_auth\.c\.\d+\) (password doesn\'t match .* username: .*|digest: auth failed for .*: wrong password|get_password failed), IP: <HOST>\s*$
-# Notes.:  regex to match wrong passwords as notified by lighttpd's auth Module
-# Values:  TEXT
-#
-failregex = .*http_auth.*(password doesn\'t match|wrong password).*IP: <HOST>\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Francois Boulogne <fboulogne@april.org>

config/filter.d/lighttpd-fastcgi.conf

@@ -1,18 +0,0 @@
-# Fail2Ban configuration file
-#
-# Author: Arturo 'Buanzo' Busleiman <buanzo@buanzo.com.ar>
-#
-
-[Definition]
-
-# Option:  failregex
-# Notes.:  regex to match ALERTS as notified by lighttpd's FastCGI Module
-# Values:  TEXT
-#
-failregex = .*ALERT\ -\ .*attacker\ \'<HOST>\'
-
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
-ignoreregex = 

config/filter.d/mysqld-auth.conf

@@ -1,8 +1,11 @@
-# Fail2Ban configuration file for unsuccesfull MySQL authentication attempts
+# Fail2Ban filter for unsuccesfull MySQL authentication attempts
 #
-# Authors: Artur Penttinen
-#          Yaroslav O. Halchenko
 #
+# To log wrong MySQL access attempts add to /etc/my.cnf in [mysqld]:
+# log-error=/var/log/mysqld.log
+# log-warning = 2
+#
+# If using mysql syslog [mysql_safe] has syslog in /etc/my.cnf
 
 [INCLUDES]
 
@@ -10,22 +13,20 @@
 # common.local
 before = common.conf
 
-
 [Definition]
 
-#_daemon = mysqld
+_daemon = mysqld
 
-# Option:  failregex
+failregex = ^%(__prefix_line)s(\d{6} \s?\d{1,2}:\d{2}:\d{2} )?\[Warning\] Access denied for user '\w+'@'<HOST>' (to database '[^']*'|\(using password: (YES|NO)\))*\s*$
-# Notes.:  regex to match the password failures messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-# 130322 11:26:54 [Warning] Access denied for user 'root'@'127.0.0.1' (using password: YES)
-failregex = Access denied for user '\w+'@'<HOST>' (to database '[^']*'|\(using password: (YES|NO)\))*\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# DEV Notes:
+#
+# Technically __prefix_line can equate to an empty string hence it can support
+# syslog and non-syslog at once.
+# Example:
+# 130322 11:26:54 [Warning] Access denied for user 'root'@'127.0.0.1' (using password: YES)
+#
+# Authors: Artur Penttinen
+#          Yaroslav O. Halchenko

config/filter.d/named-refused.conf

@@ -1,34 +1,48 @@
-# Fail2Ban configuration file for named (bind9). Trying to generalize the
+# Fail2Ban filter file for named (bind9).
-#          structure which is general to capture general patterns in log
-#          lines to cover different configurations/distributions
 #
-# Author: Yaroslav Halchenko
+
+# This filter blocks attacks against named (bind9) however it requires special
+# configuration on bind.
 #
+# By default, logging is off with bind9 installation.
 #
+# You will need something like this in your named.conf to provide proper logging.
+#
+# logging {
+#     channel security_file {
+#         file "/var/log/named/security.log" versions 3 size 30m;
+#         severity dynamic;
+#         print-time yes;
+#     };
+#     category security {
+#         security_file;
+#     };
+# };
 
 [Definition]
 
-# 
 # Daemon name
 _daemon=named
 
-#
 # Shortcuts for easier comprehension of the failregex
+
 __pid_re=(?:\[\d+\])
 __daemon_re=\(?%(_daemon)s(?:\(\S+\))?\)?:?
 __daemon_combs_re=(?:%(__pid_re)s?:\s+%(__daemon_re)s|%(__daemon_re)s%(__pid_re)s?:)
+
 #       hostname       daemon_id         spaces
 # this can be optional (for instance if we match named native log files)
 __line_prefix=(?:\s\S+ %(__daemon_combs_re)s\s+)?
 
-# Option: failregex
+failregex = ^%(__line_prefix)s(\.\d+)?( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: (view (internal|external): )?query(?: \(cache\))? '.*' denied\s*$
-# Notes.: regex to match the password failures messages in the logfile.
+            ^%(__line_prefix)s(\.\d+)?( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: zone transfer '\S+/AXFR/\w+' denied\s*$
-# Values: TEXT
+            ^%(__line_prefix)s(\.\d+)?( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: bad zone transfer request: '\S+/IN': non-authoritative zone \(NOTAUTH\)\s*$
-#
-failregex = %(__line_prefix)sclient <HOST>#\S+: (view (internal|external): )?query(?: \(cache\))? '.*' denied\s*$
 
-# Option:  ignoreregex
+# DEV Notes:
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
+# Trying to generalize the
-# Values:  TEXT
+#          structure which is general to capture general patterns in log
+#          lines to cover different configurations/distributions
+#          
+# (\.\d+)? is a really ugly catch of the microseconds not captured in the date detector
 #
-ignoreregex = 
+# Author: Yaroslav Halchenko

config/filter.d/nginx-http-auth.conf

@@ -0,0 +1,15 @@
+# fail2ban filter configuration for nginx
+
+
+[Definition]
+
+
+failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (password mismatch|was not found in ".*"), client: <HOST>, server: \S+, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"
+
+ignoreregex = 
+
+# DEV NOTES:
+# Based on samples in https://github.com/fail2ban/fail2ban/pull/43/files
+# Extensive search of all nginx auth failures not done yet.
+# 
+# Author: Daniel Black

config/filter.d/pam-generic.conf

@@ -1,30 +1,29 @@
 # Fail2Ban configuration file for generic PAM authentication errors
 #
-# Author: Yaroslav Halchenko
+
-#
+[INCLUDES]
-#
+
+before = common.conf
 
 [Definition]
 
-# if you want to catch only login erros from specific daemons, use smth like
+# if you want to catch only login errors from specific daemons, use something like
 #_ttys_re=(?:ssh|pure-ftpd|ftp)
-# To catch all failed logins
+#
+# Default: catch all failed logins
 _ttys_re=\S*
 
-#
-# Shortcuts for easier comprehension of the failregex
-__pid_re=(?:\[\d+\])
 __pam_re=\(?pam_unix(?:\(\S+\))?\)?:?
-__pam_combs_re=(?:%(__pid_re)s?:\s+%(__pam_re)s|%(__pam_re)s%(__pid_re)s?:)
+_daemon = \S+
 
-# Option: failregex
+failregex = ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=%(_ttys_re)s ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$
-# Notes.: regex to match the password failures messages in the logfile.
-# Values: TEXT
-#
-failregex = \s\S+ \S+%(__pam_combs_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=%(_ttys_re)s ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# DEV Notes:
+#
+# for linux-pam before 0.99.2.0 (late 2005) (removed before 0.8.11 release)
+# _daemon = \S*\(?pam_unix\)?
+# failregex = ^%(__prefix_line)sauthentication failure; logname=\S* uid=\S* euid=\S* tty=%(_ttys_re)s ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$
+#
+# Author: Yaroslav Halchenko

config/filter.d/perdition.conf

@@ -0,0 +1,18 @@
+# Fail2Ban filter for perdition
+#
+#
+
+[INCLUDES]
+
+before = common.conf
+
+[Definition]
+
+_daemon=perdition.\S+
+
+failregex = ^%(__prefix_line)sAuth: <HOST>:\d+->(\d{1,3}\.){3}\d{1,3}:\d+ client-secure=\S+ authorisation_id=NONE authentication_id=".+" server="\S+" protocol=\S+ server-secure=\S+ status="failed: (local authentication failure|Re-Authentication Failure)"$
+            ^%(__prefix_line)sFatal Error reading authentication information from client <HOST>:\d+->(\d{1,3}\.){3}\d{1,3}:\d+: Exiting child$
+
+ignoreregex =
+
+# Author: Christophe Carles and Daniel Black

config/filter.d/php-url-fopen.conf

@@ -1,23 +1,20 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for URLs with a URL as a script parameters
+# which can be an indication of a fopen url php injection
+#
+# Example of web requests in Apache access log:
+# 66.185.212.172 - - [26/Mar/2009:08:44:20 -0500] "GET /index.php?n=http://eatmyfood.hostinginfive.com/pizza.htm? HTTP/1.1" 200 114 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
+
+[Definition]
+
+failregex = ^<HOST> -.*"(GET|POST).*\?.*\=http\:\/\/.* HTTP\/.*$
+
+ignoreregex = 
+
+# DEV Notes:
 #
-# Author: Arturo 'Buanzo' Busleiman <buanzo@buanzo.com.ar>
 # Version 2
 # fixes the failregex so REFERERS that contain =http:// don't get blocked
 # (mentioned by "fasuto" (no real email provided... blog comment) in this entry:
 # http://blogs.buanzo.com.ar/2009/04/fail2ban-filter-for-php-injection-attacks.html#comment-1489
 #
-
+# Author: Arturo 'Buanzo' Busleiman <buanzo@buanzo.com.ar>
-[Definition]
-
-# Option:  failregex
-# Notes.:  regex to match this kind of request:
-#
-# 66.185.212.172 - - [26/Mar/2009:08:44:20 -0500] "GET /index.php?n=http://eatmyfood.hostinginfive.com/pizza.htm? HTTP/1.1" 200 114 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
-#
-failregex = ^<HOST> -.*"(GET|POST).*\?.*\=http\:\/\/.* HTTP\/.*$
-
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
-ignoreregex = 

config/filter.d/postfix-sasl.conf

@@ -0,0 +1,14 @@
+# Fail2Ban filter for postfix authentication failures
+#
+
+[INCLUDES]
+
+before = common.conf
+
+[Definition]
+
+_daemon = postfix/smtpd
+
+failregex = ^%(__prefix_line)swarning: [-._\w]+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(: [ A-Za-z0-9+/]*={0,2})?\s*$
+
+# Author: Yaroslav Halchenko

config/filter.d/postfix.conf

@@ -1,23 +1,21 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for selected Postfix SMTP rejections
-#
-# Author: Cyril Jaquier
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
 [Definition]
 
-# Option:  failregex
+_daemon = postfix/smtpd
-# Notes.:  regex to match the password failures messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 554 5\.7\.1 .*$
-#          be used for standard IP/hostname matching and is only an alias for
+            ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 450 4\.7\.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo= *$
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
+            ^%(__prefix_line)sNOQUEUE: reject: VRFY from \S+\[<HOST>\]: 550 5\.1\.1 .*$
-# Values:  TEXT
-#
-failregex = reject: RCPT from (.*)\[<HOST>\]: 554
-            reject: RCPT from (.*)\[<HOST>\]: 450 4\.7\.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo= *$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Cyril Jaquier

config/filter.d/proftpd.conf

@@ -1,25 +1,22 @@
-# Fail2Ban configuration file
+# Fail2Ban fitler for the Proftpd FTP daemon
-#
-# Author: Yaroslav Halchenko
-#
 #
 
+[INCLUDES]
+
+before = common.conf
+
 [Definition]
 
-# Option: failregex
+_daemon = proftpd
-# Notes.: regex to match the password failures messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+__suffix_failed_login = (User not authorized for login|No such user found|Incorrect password|Password expired|Account disabled|Invalid shell: '\S+'|User in \S+|Limit (access|configuration) denies login|Not a UserAlias|maximum login length exceeded).?
-#          be used for standard IP/hostname matching and is only an alias for
+
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
+failregex = ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ USER .*: no such user found from \S+ \[\S+\] to \S+:\S+ *$
-# Values: TEXT
+            ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ USER .* \(Login failed\): %(__suffix_failed_login)s\s*$
-#
+            ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ SECURITY VIOLATION: .* login attempted\. *$
-failregex = \(\S+\[<HOST>\]\)[: -]+ USER \S+: no such user found from \S+ \[\S+\] to \S+:\S+ *$
+            ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ Maximum login attempts \(\d+\) exceeded *$
-            \(\S+\[<HOST>\]\)[: -]+ USER \S+ \(Login failed\): .*$
-            \(\S+\[<HOST>\]\)[: -]+ SECURITY VIOLATION: \S+ login attempted\. *$
-            \(\S+\[<HOST>\]\)[: -]+ Maximum login attempts \(\d+\) exceeded *$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Yaroslav Halchenko
+#         Daniel Black - hardening of regex

config/filter.d/pure-ftpd.conf

@@ -1,28 +1,19 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for pureftp
-#
-# Author: Cyril Jaquier
-# Modified: Yaroslav Halchenko for pure-ftpd
 #
 #
+#
+[INCLUDES]
+
+before = common.conf
 
 [Definition]
 
 # Error message specified in multiple languages
 __errmsg = (?:Authentication failed for user|Erreur d'authentification pour l'utilisateur)
 
-#
+failregex = ^%(__prefix_line)s\(.+?@<HOST>\) \[WARNING\] %(__errmsg)s \[.+\]\s*$
-# Option: failregex
-# Notes.: regex to match the password failures messages in the logfile. The
-#         host must be matched by a group named "host". The tag "<HOST>" can
-#         be used for standard IP/hostname matching and is only an alias for
-#         (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values: TEXT
-#
-failregex = pure-ftpd(?:\[\d+\])?: \(.+?@<HOST>\) \[WARNING\] %(__errmsg)s \[.+\]\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
 
+# Author: Cyril Jaquier
+# Modified: Yaroslav Halchenko for pure-ftpd

config/filter.d/qmail.conf

@@ -1,22 +1,31 @@
-# Fail2Ban configuration file
+# Fail2Ban filters for qmail RBL patches/fake proxies
 #
-# Author: Cyril Jaquier
+# the default djb RBL implementation doesn't log any rejections 
+# so is useless with this filter.
 #
+# One patch is here:
 #
+# http://www.tjsi.com/rblsmtpd/faq/ patch to rblsmtpd
+
+[INCLUDES]
+
+before = common.conf
 
 [Definition]
 
-# Option:  failregex
+_daemon = (?:qmail|rblsmtpd)
-# Notes.:  regex to match the password failures messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
-failregex = (?:[\d,.]+[\d,.] rblsmtpd: |421 badiprbl: ip )<HOST>
 
-# Option:  ignoreregex
+failregex = ^%(__prefix_line)s\d+\.\d+ rblsmtpd: <HOST> pid \d+ \S+ 4\d\d \S+\s*$
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
+            ^%(__prefix_line)s\d+\.\d+ qmail-smtpd: 4\d\d badiprbl: ip <HOST> rbl: \S+\s*$
-# Values:  TEXT
+            ^%(__prefix_line)s\S+ blocked <HOST> \S+ -\s*$
+
+ignoreregex =
+
+# DEV Notes:
 #
-ignoreregex = 
+# These seem to be for two or 3 different patches to qmail or rblsmtpd
+# so you'll probably only ever see one of these regex's that match.
+#
+# ref: https://github.com/fail2ban/fail2ban/pull/386
+#
+# Author: Daniel Black

config/filter.d/recidive.conf

@@ -1,9 +1,8 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for repeat bans
 #
-# Author: Tom Hendrikx, modifications by Amir Caspi 
-# 
 # This filter monitors the fail2ban log file, and enables you to add long 
 # time bans for ip addresses that get banned by fail2ban multiple times.
+#
 # Reasons to use this: block very persistent attackers for a longer time, 
 # stop receiving email notifications about the same attacker over and 
 # over again.
@@ -13,26 +12,21 @@
 # drawbacks, namely in that it works only with iptables, or if you use a 
 # different blocking mechanism for this jail versus others (e.g. hostsdeny 
 # for most jails, and shorewall for this one).
-#
+
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
 
 [Definition]
 
+_daemon = fail2ban\.actions
+
 # The name of the jail that this filter is used for. In jail.conf, name the 
 # jail using this filter 'recidive', or change this line!
 _jailname = recidive
 
-# Option:  failregex
+failregex = ^(%(__prefix_line)s|,\d{3} fail2ban.actions:\s+)WARNING\s+\[(?!%(_jailname)s\])(?:.*)\]\s+Ban\s+<HOST>\s*$
-# Notes.:  regex to match the password failures messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>\S+)
-# Values:  TEXT
-#
-failregex = fail2ban.actions:\s+WARNING\s+\[(?:.*)\]\s+Ban\s+<HOST>
 
-# Option:  ignoreregex
+# Author: Tom Hendrikx, modifications by Amir Caspi 
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
-# Ignore our own bans, to keep our counts exact.
-ignoreregex = fail2ban.actions:\s+WARNING\s+\[%(_jailname)s\]\s+Ban\s+<HOST>

config/filter.d/roundcube-auth.conf

@@ -1,22 +1,16 @@
 # Fail2Ban configuration file for roundcube web server
 #
-# Author: Teodor Micu & Yaroslav Halchenko & terence namusonge
 #
 #
 
+[INCLUDES]
+
+before = common.conf
+
 [Definition]
 
-# Option:  failregex
+failregex = ^\s*(\[(\s[+-][0-9]{4})?\])?(%(__hostname)s roundcube: IMAP Error)?: (FAILED login|Login failed) for .*? from <HOST>(\. AUTHENTICATE .*)?\s*$
-# Notes.:  regex to match the password failure messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
-failregex = (FAILED login|Login failed) for .* from <HOST>\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Teodor Micu & Yaroslav Halchenko & terence namusonge

config/filter.d/sasl.conf

@@ -1,22 +0,0 @@
-# Fail2Ban configuration file
-#
-# Author: Yaroslav Halchenko
-#
-#
-
-[Definition]
-
-# Option: failregex
-# Notes.: regex to match the password failures messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values: TEXT
-#
-failregex = (?i): warning: [-._\w]+\[<HOST>\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(: [ A-Za-z0-9+/]*={0,2})?\s*$
-
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
-ignoreregex = 

config/filter.d/selinux-common.conf

@@ -0,0 +1,21 @@
+# Fail2Ban configuration file for generic SELinux audit messages
+#
+# This file is not intended to be used directly, and should be included into a
+# filter file which would define following variables. See selinux-ssh.conf as
+# and example.
+#
+# _type
+# _uid
+# _auid 
+# _subj
+# _msg
+#
+# Also one of these variables must include <HOST>.
+
+[Definition]
+
+failregex = ^type=%(_type)s msg=audit\(:\d+\): (user )?pid=\d+ uid=%(_uid)s auid=%(_auid)s ses=\d+ subj=%(_subj)s msg='%(_msg)s'$
+
+ignoreregex =
+
+# Author: Daniel Black

config/filter.d/selinux-ssh.conf

@@ -0,0 +1,25 @@
+# Fail2Ban configuration file for SELinux ssh authentication errors
+#
+
+[INCLUDES]
+
+after = selinux-common.conf
+
+[Definition]
+
+_type = USER_(ERR|AUTH)
+_uid  = 0
+_auid = \d+
+_subj = (?:unconfined_u|system_u):system_r:sshd_t:s0-s0:c0\.c1023
+
+_exe  =/usr/sbin/sshd
+_terminal = ssh
+
+_msg = op=\S+ acct=(?P<_quote_acct>"?)\S+(?P=_quote_acct) exe="%(_exe)s" hostname=(\?|(\d+\.){3}\d+) addr=<HOST> terminal=%(_terminal)s res=failed
+
+# DEV Notes:
+#
+# Note: USER_LOGIN is ignored as this is the duplicate messsage
+# ssh logs after 3 USER_AUTH failures.
+# 
+# Author: Daniel Black

config/filter.d/sieve.conf

@@ -1,21 +1,18 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for sieve authentication failures
-#
-# Author: Jan Wagner <waja@cyconet.org>
-#
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
 [Definition]
 
-# Option: failregex
+_deamon = (?:cyrus/)?(?:tim)?sieved?
-# Notes.: regex to match the password failures messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+failregex = ^%(__prefix_line)sbadlogin: \S+ ?\[<HOST>\] \S+ authentication failure$
-#          be used for standard IP/hostname matching.
-# Values: TEXT
-#
-failregex = : badlogin: .*\[<HOST>\] (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failure$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Jan Wagner <waja@cyconet.org>

config/filter.d/sogo-auth.conf

@@ -1,20 +1,17 @@
-# /etc/fail2ban/filter.d/sogo-auth.conf
+# Fail2ban filter for SOGo authentcation
-#
-# Fail2Ban configuration file
-# By Arnd Brandes
-# SOGo
 #
+# Log file usually in /var/log/sogo/sogo.log
 
 [Definition]
-# Option: failregex
-# Filter Ban in /var/log/sogo/sogo.log
-# Note: the error log may contain multiple hosts, whereas the first one 
-# is the client and all others are poxys. We match the first one, only
 
-failregex = Login from '<HOST>' for user '.*' might not have worked( - password policy: \d*  grace: -?\d*  expire: -?\d*  bound: -?\d*)?\s*$
+failregex = ^ sogod \[\d+\]: SOGoRootPage Login from '<HOST>' for user '.*' might not have worked( - password policy: \d*  grace: -?\d*  expire: -?\d*  bound: -?\d*)?\s*$
 
-# Option: ignoreregex
-# Notes.: regex to ignore. If this regex matches, the line is ignored.
-# Values: TEXT
-#
 ignoreregex = 
+
+# 
+# DEV Notes:
+#
+# The error log may contain multiple hosts, whereas the first one 
+# is the client and all others are poxys. We match the first one, only
+#
+# Author: Arnd Brandes

config/filter.d/sshd-ddos.conf

@@ -1,6 +1,4 @@
-# Fail2Ban configuration file
+# Fail2Ban ssh filter for at attempted exploit
-#
-# Author: Yaroslav Halchenko
 #
 # The regex here also relates to a exploit:
 #
@@ -20,17 +18,8 @@ before = common.conf
 
 _daemon = sshd
 
-# Option:  failregex
-# Notes.:  regex to match the password failures messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
 failregex = ^%(__prefix_line)sDid not receive identification string from <HOST>\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Yaroslav Halchenko

config/filter.d/sshd.conf

@@ -1,7 +1,4 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for openssh
-#
-# Author: Cyril Jaquier
-#
 #
 
 [INCLUDES]
@@ -15,16 +12,9 @@ before = common.conf
 
 _daemon = sshd
 
-# Option:  failregex
-# Notes.:  regex to match the password failures messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
 failregex = ^%(__prefix_line)s(?:error: PAM: )?[aA]uthentication (?:failure|error) for .* from <HOST>( via \S+)?\s*$
             ^%(__prefix_line)s(?:error: PAM: )?User not known to the underlying authentication module for .* from <HOST>\s*$
-            ^%(__prefix_line)sFailed \S+ for .* from <HOST>(?: port \d*)?(?: ssh\d*)?\s*$
+            ^%(__prefix_line)sFailed \S+ for .*? from <HOST>(?: port \d*)?(?: ssh\d*)?(: (ruser .*|(\S+ ID \S+ \(serial \d+\) CA )?\S+ %(__md5hex)s(, client user ".*", client host ".*")?))?\s*$
             ^%(__prefix_line)sROOT LOGIN REFUSED.* FROM <HOST>\s*$
             ^%(__prefix_line)s[iI](?:llegal|nvalid) user .* from <HOST>\s*$
             ^%(__prefix_line)sUser .+ from <HOST> not allowed because not listed in AllowUsers\s*$
@@ -34,8 +24,13 @@ failregex = ^%(__prefix_line)s(?:error: PAM: )?[aA]uthentication (?:failure|erro
             ^%(__prefix_line)sUser .+ from <HOST> not allowed because a group is listed in DenyGroups\s*$
             ^%(__prefix_line)sUser .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# DEV Notes:
+#
+#   "Failed \S+ for .*? from <HOST>..." failregex uses non-greedy catch-all because
+#   it is coming before use of <HOST> which is not hard-anchored at the end as well,
+#   and later catch-all's could contain user-provided input, which need to be greedily
+#   matched away first.
+#
+# Author: Cyril Jaquier, Yaroslav Halchenko, Petr Voralek, Daniel Black

config/filter.d/suhosin.conf

@@ -0,0 +1,28 @@
+# Fail2Ban filter for suhosian PHP hardening
+#
+# This occurs with lighttpd or directly from the plugin
+#
+
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+
+[Definition]
+
+_daemon = (?:lighttpd|suhosin)
+
+
+_lighttpd_prefix = (?:\(mod_fastcgi\.c\.\d+\) FastCGI-stderr:\s)
+
+failregex = ^%(__prefix_line)s%(_lighttpd_prefix)s?ALERT - .* \(attacker '<HOST>', file '.*'(?:, line \d+)?\)$
+
+ignoreregex = 
+
+# DEV Notes:
+#
+# https://github.com/stefanesser/suhosin/blob/1fba865ab73cc98a3109f88d85eb82c1bfc29b37/log.c#L161
+#
+# Author: Arturo 'Buanzo' Busleiman <buanzo@buanzo.com.ar>

config/filter.d/uwimap-auth.conf

@@ -0,0 +1,17 @@
+# Fail2Ban filter for uwimap
+#
+
+[INCLUDES]
+
+before = common.conf
+
+[Definition]
+
+_daemon = (?:ipop3d|imapd)
+
+failregex = ^%(__prefix_line)sLogin (?:failed|excessive login failures|disabled|SYSTEM BREAK-IN ATTEMPT) user=\S* auth=\S* host=.*\[<HOST>\]\s*$ 
+            ^%(__prefix_line)sFailed .* override of user=.* host=.*\[<HOST>\]\s*$
+
+ignoreregex = 
+
+# Author: Amir Caspi

config/filter.d/vsftpd.conf

@@ -1,23 +1,18 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for vsftp
-#
-# Author: Cyril Jaquier
-#
 #
 
+[INCLUDES]
+
+before = common.conf
+
 [Definition]
 
-# Option: failregex
+__pam_re=\(?pam_unix(?:\(\S+\))?\)?:?
-# Notes.: regex to match the password failures messages in the logfile. The
+_daemon =  vsftpd
-#          host must be matched by a group named "host". The tag "<HOST>" can
+
-#          be used for standard IP/hostname matching and is only an alias for
+failregex = ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=(ftp)? ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
+            ^ \[pid \d+\] \[.+\] FAIL LOGIN: Client "<HOST>"\s*$
-# Values: TEXT
-#
-failregex = vsftpd(?:\(pam_unix\))?(?:\[\d+\])?:.* authentication failure; .* rhost=<HOST>(?:\s+user=\S*)?\s*$
-            \[.+\] FAIL LOGIN: Client "<HOST>"\s*$
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Cyril Jaquier

config/filter.d/webmin-auth.conf

@@ -1,27 +1,22 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for webmin
-#
-# Author: Cyril Jaquier
-# Rule by : Delvit Guillaume
-#
 #
 
+[INCLUDES]
+
+before = common.conf
+
 [Definition]
 
-# patern :      webmin[15673]: Non-existent login as toto from 86.0.6.217
+_daemon = webmin
+
+failregex = ^%(__prefix_line)sNon-existent login as .+ from <HOST>\s*$
+            ^%(__prefix_line)sInvalid login as .+ from <HOST>\s*$
+
+ignoreregex = 
+
+# DEV Notes:
+#
+# pattern :     webmin[15673]: Non-existent login as toto from 86.0.6.217
 #               webmin[29544]: Invalid login as root from 86.0.6.217
 #
-# Option:  failregex
+# Rule Author: Delvit Guillaume
-# Notes.:  regex to match the password failure messages in the logfile. The
-#          host must be matched by a group named "host". The tag "<HOST>" can
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
-#
-failregex = webmin.* Non-existent login as .+ from <HOST>\s*$
-            webmin.* Invalid login as .+ from <HOST>\s*$
-
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
-ignoreregex = 

config/filter.d/wuftpd.conf

@@ -1,20 +1,22 @@
 # Fail2Ban configuration file for wuftpd
 #
-# Author: Yaroslav Halchenko
-#
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
 [Definition]
 
-# Option: failregex
+_daemon = wu-ftpd
-# Notes.: regex to match the password failures messages in the logfile.
+__pam_re=\(?pam_unix(?:\(wu-ftpd:auth\))?\)?:?
-# Values: TEXT
+
-#
+failregex = ^%(__prefix_line)sfailed login from \S+ \[<HOST>\]\s*$
-failregex = wu-ftpd(?:\[\d+\])?:\s+\(pam_unix\)\s+authentication failure.* rhost=<HOST>$
+            ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=(ftp)? ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$
-            wu-ftpd(?:\[\d+\])?: *failed login from .*\[<HOST>\] *$
+
 
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
 ignoreregex = 
+
+# Author: Yaroslav Halchenko

config/filter.d/xinetd-fail.conf

@@ -1,29 +1,27 @@
-# Fail2Ban configuration file
+# Fail2Ban filter for xinetd failures
 #
-# Author: Guido Bozzetto
+# Cfr.: /var/log/(daemon\.|sys)log
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
 [Definition]
 
-# Option:  failregex
+_daemon = xinetd
-# Notes.:  regex to match the password failures messages in the logfile. The
+
-#          host must be matched by a group named "host". The tag "<HOST>" can
+failregex = ^%(__prefix_line)sFAIL: \S+ address from=<HOST>$
-#          be used for standard IP/hostname matching and is only an alias for
+            ^%(__prefix_line)sFAIL: \S+ libwrap from=<HOST>$
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
+
-# Values:  TEXT
+ignoreregex = 
+
+# DEV Notes:
 #
-# Cfr.: /var/log/(daemon\.|sys)log
 # libwrap => tcp wrappers: hosts.(allow|deny)
 # address => xinetd: deny_from|only_from
-# load => xinetd: max_load (temporary problem)
 #
-
+# Author: Guido Bozzetto
-failregex = xinetd(?:\[\d{1,5}\])?: FAIL: \S+ address from=<HOST>$
-            xinetd(?:\[\d{1,5}\])?: FAIL: \S+ libwrap from=<HOST>$
-
-# Option:  ignoreregex
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
-# Values:  TEXT
-#
-ignoreregex = 

config/jail.conf

@@ -1,16 +1,25 @@
-# Fail2Ban jail specifications file
+# Fail2Ban jail base specification file
 #
-# Comments: use '#' for comment lines and ';' (following a space) for inline comments
+# HOW TO ACTIVATE JAILS:
 #
-# Changes:  in most of the cases you should not modify this
+# YOU SHOULD NOT MODIFY THIS FILE.
-#           file, but provide customizations in jail.local file, e.g.:
+#
+# It will probably be overwitten or improved in a distribution update.
+#
+# Provide customizations in a jail.local file or a jail.d/customisation.local.
+# For example to change the default bantime for all jails and to enable the
+# ssh-iptables jail the following (uncommented) would appear in the .local file.
+# See man 5 jail.conf for details.
 #
 # [DEFAULT]
 # bantime = 3600
 #
 # [ssh-iptables]
 # enabled = true
-#
+
+
+
+# Comments: use '#' for comment lines and ';' (following a space) for inline comments
 
 # The DEFAULT allows a global definition of the options. They can be overridden
 # in each jail afterwards.
@@ -56,7 +65,7 @@ backend = auto
 usedns = warn
 
 
-# This jail corresponds to the standard configuration in Fail2ban 0.6.
+# This jail corresponds to the standard configuration in Fail2ban.
 # The mail-whois action send a notification e-mail with a whois request
 # in the body.
 
@@ -65,7 +74,7 @@ usedns = warn
 enabled  = false
 filter   = sshd
 action   = iptables[name=SSH, port=ssh, protocol=tcp]
-           sendmail-whois[name=SSH, dest=you@example.com, sender=fail2ban@example.com]
+           sendmail-whois[name=SSH, dest=you@example.com, sender=fail2ban@example.com, sendername="Fail2Ban"]
 logpath  = /var/log/sshd.log
 maxretry = 5
 
@@ -78,47 +87,50 @@ action   = iptables[name=ProFTPD, port=ftp, protocol=tcp]
 logpath  = /var/log/proftpd/proftpd.log
 maxretry = 6
 
-# This jail forces the backend to "polling".
 
+# This jail forces the backend to "polling".
 [sasl-iptables]
 
 enabled  = false
-filter   = sasl
+filter   = postfix-sasl
 backend  = polling
 action   = iptables[name=sasl, port=smtp, protocol=tcp]
            sendmail-whois[name=sasl, dest=you@example.com]
 logpath  = /var/log/mail.log
 
+
 # ASSP SMTP Proxy Jail
 [assp]
-enabled  = false
+
-filter   = assp
+enabled = false
-action = iptables-multiport[name=assp,port="25,465,587"]
+filter  = assp
-logpath  = /root/path/to/assp/logs/maillog.txt
+action  = iptables-multiport[name=assp,port="25,465,587"]
+logpath = /root/path/to/assp/logs/maillog.txt
+
 
 # Here we use TCP-Wrappers instead of Netfilter/Iptables. "ignoreregex" is
 # used to avoid banning the user "myuser".
-
 [ssh-tcpwrapper]
 
 enabled     = false
 filter      = sshd
-action      = hostsdeny
+action      = hostsdeny[daemon_list=sshd]
               sendmail-whois[name=SSH, dest=you@example.com]
 ignoreregex = for myuser from
 logpath     = /var/log/sshd.log
 
+
 # Here we use blackhole routes for not requiring any additional kernel support
 # to store large volumes of banned IPs
-
 [ssh-route]
 
-enabled = false
+enabled  = false
-filter = sshd
+filter   = sshd
-action = route
+action   = route
-logpath = /var/log/sshd.log
+logpath  = /var/log/sshd.log
 maxretry = 5
 
+
 # Here we use a combination of Netfilter/Iptables and IPsets
 # for storing large volumes of banned IPs
 #
@@ -132,13 +144,16 @@ action   = iptables-ipset-proto4[name=SSH, port=ssh, protocol=tcp]
 logpath  = /var/log/sshd.log
 maxretry = 5
 
+
 [ssh-iptables-ipset6]
+
 enabled  = false
 filter   = sshd
 action   = iptables-ipset-proto6[name=SSH, port=ssh, protocol=tcp, bantime=600]
 logpath  = /var/log/sshd.log
 maxretry = 5
 
+
 # bsd-ipfw is ipfw used by BSD. It uses ipfw tables.
 # table number must be unique.
 # 
@@ -146,15 +161,16 @@ maxretry = 5
 # for the table doesn't ready exist.
 #
 [ssh-bsd-ipfw]
+
 enabled  = false
 filter   = sshd
 action   = bsd-ipfw[port=ssh,table=1]
 logpath  = /var/log/auth.log
 maxretry = 5
 
+
 # This jail demonstrates the use of wildcards in "logpath".
 # Moreover, it is possible to give other files on a new line.
-
 [apache-tcpwrapper]
 
 enabled  = false
@@ -164,9 +180,17 @@ logpath  = /var/log/apache*/*error.log
            /home/www/myhomepage/error.log
 maxretry = 6
 
+
+[nginx-http-auth]
+
+enabled = false
+filter  = nginx-http-auth
+action  = iptables-multiport[name=nginx-http-auth,port="80,443"]
+logpath = /var/log/nginx/error.log
+
+
 # The hosts.deny path can be defined with the "file" argument if it is
 # not in /etc.
-
 [postfix-tcpwrapper]
 
 enabled  = false
@@ -176,9 +200,9 @@ action   = hostsdeny[file=/not/a/standard/path/hosts.deny]
 logpath  = /var/log/postfix.log
 bantime  = 300
 
+
 # Do not ban anybody. Just report information about the remote host.
 # A notification is sent at most every 600 seconds (bantime).
-
 [vsftpd-notification]
 
 enabled  = false
@@ -188,8 +212,8 @@ logpath  = /var/log/vsftpd.log
 maxretry = 5
 bantime  = 1800
 
-# Same as above but with banning the IP address.
 
+# Same as above but with banning the IP address.
 [vsftpd-iptables]
 
 enabled  = false
@@ -200,9 +224,9 @@ logpath  = /var/log/vsftpd.log
 maxretry = 5
 bantime  = 1800
 
+
 # Ban hosts which agent identifies spammer robots crawling the web
 # for email addresses. The mail outputs are buffered.
-
 [apache-badbots]
 
 enabled  = false
@@ -213,8 +237,8 @@ logpath  = /var/www/*/logs/access_log
 bantime  = 172800
 maxretry = 1
 
-# Use shorewall instead of iptables.
 
+# Use shorewall instead of iptables.
 [apache-shorewall]
 
 enabled  = false
@@ -223,72 +247,63 @@ action   = shorewall
            sendmail[name=Postfix, dest=you@example.com]
 logpath  = /var/log/apache2/error_log
 
-# Monitor roundcube server
 
+# Monitor roundcube server
 [roundcube-iptables]
 
 enabled  = false
 filter   = roundcube-auth
-action   = iptables[name=RoundCube, port="http,https"]
+action   = iptables-multiport[name=RoundCube, port="http,https"]
 logpath  = /var/log/roundcube/userlogins
 
 
 # Monitor SOGo groupware server
-
 [sogo-iptables]
 
 enabled  = false
 filter   = sogo-auth
 # without proxy this would be:
 # port    = 20000
-action   = iptables[name=SOGo, port="http,https"]
+action   = iptables-multiport[name=SOGo, port="http,https"]
 logpath  = /var/log/sogo/sogo.log
 
+
 # Ban attackers that try to use PHP's URL-fopen() functionality
 # through GET/POST variables. - Experimental, with more than a year
 # of usage in production environments.
-
 [php-url-fopen]
 
-enabled = false
+enabled  = false
-action  = iptables[name=php-url-open, port="http,https"]
+action   = iptables-multiport[name=php-url-open, port="http,https"]
-filter  = php-url-fopen
+filter   = php-url-fopen
-logpath = /var/www/*/logs/access_log
+logpath  = /var/www/*/logs/access_log
 maxretry = 1
 
-# A simple PHP-fastcgi jail which works with lighttpd.
-# If you run a lighttpd server, then you probably will
-# find these kinds of messages in your error_log:
-# ALERT – tried to register forbidden variable ‘GLOBALS’
-# through GET variables (attacker '1.2.3.4', file '/var/www/default/htdocs/index.php')
-# This jail would block the IP 1.2.3.4.
 
-[lighttpd-fastcgi]
+[suhosin]
 
-enabled = false
+enabled  = false
-filter  = lighttpd-fastcgi
+filter   = suhosin
-action  = iptables[name=lighttpd-fastcgi, port="http,https"]
+action   = iptables-multiport[name=suhosin, port="http,https"]
 # adapt the following two items as needed
-logpath = /var/log/lighttpd/error.log
+logpath  = /var/log/lighttpd/error.log
 maxretry = 2
 
-# Same as above for mod_auth
-# It catches wrong authentications
 
 [lighttpd-auth]
 
-enabled = false
+enabled  = false
-filter  = lighttpd-auth
+filter   = lighttpd-auth
-action  = iptables[name=lighttpd-auth, port="http,https"]
+action   = iptables-multiport[name=lighttpd-auth, port="http,https"]
 # adapt the following two items as needed
-logpath = /var/log/lighttpd/error.log
+logpath  = /var/log/lighttpd/error.log
 maxretry = 2
 
+
 # This jail uses ipfw, the standard firewall on FreeBSD. The "ignoreip"
 # option is overridden in this jail. Moreover, the action "mail-whois" defines
 # the variable "name" which contains a comma using "". The characters '' are
 # valid too.
-
 [ssh-ipfw]
 
 enabled  = false
@@ -298,22 +313,6 @@ action   = ipfw[localhost=192.168.0.1]
 logpath  = /var/log/auth.log
 ignoreip = 168.192.0.1
 
-# These jails block attacks against named (bind9). By default, logging is off
-# with bind9 installation. You will need something like this:
-#
-# logging {
-#     channel security_file {
-#         file "/var/log/named/security.log" versions 3 size 30m;
-#         severity dynamic;
-#         print-time yes;
-#     };
-#     category security {
-#         security_file;
-#     };
-# };
-#
-# in your named.conf to provide proper logging.
-# This jail blocks UDP traffic for DNS requests.
 
 # !!! WARNING !!!
 #   Since UDP is connection-less protocol, spoofing of IP and imitation
@@ -323,6 +322,8 @@ ignoreip = 168.192.0.1
 #    http://nion.modprobe.de/blog/archives/690-fail2ban-+-dns-fail.html
 #   Please DO NOT USE this jail unless you know what you are doing.
 #
+# IMPORTANT: see filter.d/named-refused for instructions to enable logging
+# This jail blocks UDP traffic for DNS requests.
 # [named-refused-udp]
 #
 # enabled  = false
@@ -332,8 +333,8 @@ ignoreip = 168.192.0.1
 # logpath  = /var/log/named/security.log
 # ignoreip = 168.192.0.1
 
+# IMPORTANT: see filter.d/named-refused for instructions to enable logging
 # This jail blocks TCP traffic for DNS requests.
-
 [named-refused-tcp]
 
 enabled  = false
@@ -343,8 +344,19 @@ action   = iptables-multiport[name=Named, port="domain,953", protocol=tcp]
 logpath  = /var/log/named/security.log
 ignoreip = 168.192.0.1
 
-# Multiple jails, 1 per protocol, are necessary ATM:
+
-# see https://github.com/fail2ban/fail2ban/issues/37
+[asterisk]
+
+enabled  = false
+filter   = asterisk
+action   = iptables-multiport[name=asterisk-tcp, port="5060,5061", protocol=tcp]
+           iptables-multiport[name=asterisk-udp, port="5060,5061", protocol=udp]
+           sendmail-whois[name=Asterisk, dest=you@example.com, sender=fail2ban@example.com]
+logpath  = /var/log/asterisk/messages
+maxretry = 10
+
+#  Historical support (before https://github.com/fail2ban/fail2ban/issues/37 was fixed )
+#  use [asterisk] for new jails
 [asterisk-tcp]
 
 enabled  = false
@@ -354,6 +366,9 @@ action   = iptables-multiport[name=asterisk-tcp, port="5060,5061", protocol=tcp]
 logpath  = /var/log/asterisk/messages
 maxretry = 10
 
+
+#  Historical support (before https://github.com/fail2ban/fail2ban/issues/37 was fixed )
+#  use [asterisk] for new jails
 [asterisk-udp]
 
 enabled  = false
@@ -363,9 +378,7 @@ action   = iptables-multiport[name=asterisk-udp, port="5060,5061", protocol=udp]
 logpath  = /var/log/asterisk/messages
 maxretry = 10
 
-# To log wrong MySQL access attempts add to /etc/my.cnf:
+
-# log-error=/var/log/mysqld.log
-# log-warning = 2
 [mysqld-iptables]
 
 enabled  = false
@@ -376,6 +389,15 @@ logpath  = /var/log/mysqld.log
 maxretry = 5
 
 
+[mysqld-syslog-iptables]
+
+enabled  = false
+filter   = mysqld-auth
+action   = iptables[name=mysql, port=3306, protocol=tcp]
+logpath  = /var/log/daemon.log
+maxretry = 5
+
+
 # Jail for more extended banning of persistent abusers
 # !!! WARNING !!!
 #   Make sure that your loglevel specified in fail2ban.conf/.local
@@ -392,13 +414,113 @@ bantime  = 604800  ; 1 week
 findtime = 86400   ; 1 day
 maxretry = 5
 
+
 # PF is a BSD based firewall
 [ssh-pf]
 
-enabled=false
+enabled  = false
-filter = sshd
+filter   = sshd
-action = pf
+action   = pf
 logpath  = /var/log/sshd.log
-maxretry=5
+maxretry = 5
 
 
+[3proxy]
+
+enabled = false
+filter  = 3proxy
+action  = iptables[name=3proxy, port=3128, protocol=tcp]
+logpath = /var/log/3proxy.log
+
+
+[exim]
+
+enabled = false
+filter  = exim
+action  = iptables-multiport[name=exim,port="25,465,587"]
+logpath = /var/log/exim/mainlog
+
+
+[exim-spam]
+
+enabled = false
+filter  = exim-spam
+action  = iptables-multiport[name=exim-spam,port="25,465,587"]
+logpath = /var/log/exim/mainlog
+
+
+[perdition]
+
+enabled = false
+filter  = perdition
+action  = iptables-multiport[name=perdition,port="110,143,993,995"]
+logpath = /var/log/maillog
+
+
+[uwimap-auth]
+
+enabled = false
+filter  = uwimap-auth
+action  = iptables-multiport[name=uwimap-auth,port="110,143,993,995"]
+logpath = /var/log/maillog
+
+
+[osx-ssh-ipfw]
+
+enabled  = false
+filter   = sshd
+action   = osx-ipfw
+logpath  = /var/log/secure.log
+maxretry = 5
+
+
+[ssh-apf]
+
+enabled = false
+filter  = sshd
+action  = apf[name=SSH]
+logpath = /var/log/secure
+maxretry = 5
+
+
+[osx-ssh-afctl]
+
+enabled  = false
+filter   = sshd
+action   = osx-afctl[bantime=600]
+logpath  = /var/log/secure.log
+maxretry = 5
+
+
+[webmin-auth]
+
+enabled = false
+filter  = webmin-auth
+action  = iptables-multiport[name=webmin,port="10000"]
+logpath = /var/log/auth.log
+
+
+# dovecot defaults to logging to the mail syslog facility
+# but can be set by syslog_facility in the dovecot configuration.
+[dovecot]
+
+enabled = false
+filter  = dovecot
+action  = iptables-multiport[name=dovecot, port="pop3,pop3s,imap,imaps,submission,smtps,sieve", protocol=tcp]
+logpath = /var/log/mail.log
+
+
+[dovecot-auth]
+
+enabled = false
+filter  = dovecot
+action  = iptables-multiport[name=dovecot-auth, port="pop3,pop3s,imap,imaps,submission,smtps,sieve", protocol=tcp]
+logpath = /var/log/secure
+
+
+[selinux-ssh]
+enabled = false
+filter  = selinux-ssh
+action  = iptables[name=SELINUX-SSH, port=ssh, protocol=tcp]
+logpath  = /var/log/audit/audit.log
+maxretry = 5

fail2ban-client

@@ -155,8 +155,9 @@ class Fail2banClient:
 					if showRet:
 						print beautifier.beautify(ret[1])
 				else:
-					logSys.debug("NOK: " + `ret[1].args`)
+					logSys.error("NOK: " + `ret[1].args`)
-					print beautifier.beautifyError(ret[1])
+					if showRet:
+						print beautifier.beautifyError(ret[1])
 					return False
 			except socket.error:
 				if showRet:
@@ -229,7 +230,7 @@ class Fail2banClient:
 		elif len(cmd) == 2 and cmd[0] == "reload":
 			if self.__ping():
 				jail = cmd[1]
-				ret = self.__readJailConfig(jail)
+				ret = self.__readConfig(jail)
 				# Do not continue if configuration is not 100% valid
 				if not ret:
 					return False
@@ -336,13 +337,13 @@ class Fail2banClient:
 			logSys.setLevel(logging.INFO)
 		else:
 			logSys.setLevel(logging.DEBUG)
-		# Add the default logging handler
+		# Add the default logging handler to dump to stderr
-		stdout = logging.StreamHandler(sys.stdout)
+		logout = logging.StreamHandler(sys.stderr)
 		# set a format which is simpler for console use
 		formatter = logging.Formatter('%(levelname)-6s %(message)s')
 		# tell the handler to use this format
-		stdout.setFormatter(formatter)
+		logout.setFormatter(formatter)
-		logSys.addHandler(stdout)
+		logSys.addHandler(logout)
 
 		# Set the configuration path
 		self.__configurator.setBaseDir(self.__conf["conf"])
@@ -383,7 +384,10 @@ class Fail2banClient:
 						if cmd == "help":
 							self.dispUsage()
 						elif not cmd == "":
-							self.__processCommand(shlex.split(cmd))
+							try:
+								self.__processCommand(shlex.split(cmd))
+							except Exception, e:
+								logSys.error(e)
 			except (EOFError, KeyboardInterrupt):
 				print
 				return True
@@ -394,19 +398,18 @@ class Fail2banClient:
 				return False
 			return self.__processCommand(args)
 
-	def __readConfig(self):
+	def __readConfig(self, jail=None):
 		# Read the configuration
-		self.__configurator.readAll()
+		# TODO: get away from stew of return codes and exception
-		ret = self.__configurator.getOptions()
+		# handling -- handle via exceptions
-		self.__configurator.convertToProtocol()
+		try:
-		self.__stream = self.__configurator.getConfigStream()
+			self.__configurator.readAll()
-		return ret
+			ret = self.__configurator.getOptions(jail)
-
+			self.__configurator.convertToProtocol()
-	def __readJailConfig(self, jail):
+			self.__stream = self.__configurator.getConfigStream()
-		self.__configurator.readAll()
+		except Exception, e:
-		ret = self.__configurator.getOptions(jail)
+			logSys.error("Failed during configuration: %s" % e)
-		self.__configurator.convertToProtocol()
+			ret = False
-		self.__stream = self.__configurator.getConfigStream()
 		return ret
 
 	#@staticmethod

fail2ban-regex

@@ -17,12 +17,19 @@
 # You should have received a copy of the GNU General Public License
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+"""
+Fail2Ban  reads log file that contains password failure report
+and bans the corresponding IP addresses using firewall rules.
+
+This tools can test regular expressions for "fail2ban".
+
+"""
 
 __author__ = "Cyril Jaquier, Yaroslav Halchenko"
-__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2012 Yaroslav Halchenko"
+__copyright__ = "Copyright (c) 2004-2008 Cyril Jaquier, 2012-2013 Yaroslav Halchenko"
 __license__ = "GPL"
 
-import getopt, sys, time, logging, os
+import getopt, sys, time, logging, os, urllib
 
 # Inserts our own modules path first in the list
 # fix for bug #343821
@@ -32,231 +39,283 @@ except ImportError, e:
 	sys.path.insert(1, "/usr/share/fail2ban")
 	from common.version import version
 
+from optparse import OptionParser, Option
+
 from client.configparserinc import SafeConfigParserWithIncludes
 from ConfigParser import NoOptionError, NoSectionError, MissingSectionHeaderError
 from server.filter import Filter
 from server.failregex import RegexException
 
+from testcases.utils import FormatterWithTraceBack
 # Gets the instance of the logger.
-logSys = logging.getLogger("fail2ban.regex")
+logSys = logging.getLogger("fail2ban")
 
-class RegexStat:
+def debuggexURL(sample, regex):
+	q = urllib.urlencode({ 're': regex.replace('<HOST>', '(?&.ipv4)'),
+							'str': sample,
+							'flavor': 'python' })
+	return 'http://www.debuggex.com/?' + q
+
+def shortstr(s, l=53):
+	"""Return shortened string
+	"""
+	if len(s) > l:
+		return s[:l-3] + '...'
+	return s
+
+def pprint_list(l, header=None):
+	if not len(l):
+		return
+	if header:
+		s = "|- %s\n" % header
+	else:
+		s = ''
+	print s + "|  " + "\n|  ".join(l) + '\n`-'
+
+
+def get_opt_parser():
+	# use module docstring for help output
+	p = OptionParser(
+				usage="%s [OPTIONS] <LOG> <REGEX> [IGNOREREGEX]\n" % sys.argv[0] + __doc__
+				+ """
+LOG:
+    string                  a string representing a log line
+    filename                path to a log file (/var/log/auth.log)
+
+REGEX:
+    string                  a string representing a 'failregex'
+    filename                path to a filter file (filter.d/sshd.conf)
+
+IGNOREREGEX:
+    string                  a string representing an 'ignoreregex'
+    filename                path to a filter file (filter.d/sshd.conf)
+
+Copyright (c) 2004-2008 Cyril Jaquier, 2008- Fail2Ban Contributors
+Copyright of modifications held by their respective authors.
+Licensed under the GNU General Public License v2 (GPL).
+
+Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
+Many contributions by Yaroslav O. Halchenko and Steven Hiscocks.
+
+Report bugs to https://github.com/fail2ban/fail2ban/issues
+""",
+				version="%prog " + version)
+
+	p.add_options([
+		Option('-l', "--log-level", type="choice",
+			   dest="log_level",
+			   choices=('heavydebug', 'debug', 'info', 'warning', 'error', 'fatal'),
+			   default=None,
+			   help="Log level for the Fail2Ban logger to use"),
+		Option("-v", "--verbose", action='store_true',
+			   help="Be verbose in output"),
+		Option("-D", "--debuggex", action='store_true',
+			   help="Produce debuggex.com urls for debugging there"),
+		Option("--print-all-missed", action='store_true',
+			   help="Either to print all missed lines"),
+		Option("--print-all-ignored", action='store_true',
+			   help="Either to print all ignored lines"),
+		Option("-t", "--log-traceback", action='store_true',
+			   help="Enrich log-messages with compressed tracebacks"),
+		Option("--full-traceback", action='store_true',
+			   help="Either to make the tracebacks full, not compressed (as by default)"),
+		])
+
+	return p
+
+
+class RegexStat(object):
 
 	def __init__(self, failregex):
-		self.__stats = 0
+		self._stats = 0
-		self.__failregex = failregex
+		self._failregex = failregex
-		self.__ipList = list()
+		self._ipList = list()
 
 	def __str__(self):
 		return "%s(%r) %d failed: %s" \
-		  % (self.__class__, self.__failregex, self.__stats, self.__ipList)
+		  % (self.__class__, self._failregex, self._stats, self._ipList)
 
 	def inc(self):
-		self.__stats += 1
+		self._stats += 1
 
 	def getStats(self):
-		return self.__stats
+		return self._stats
 
 	def getFailRegex(self):
-		return self.__failregex
+		return self._failregex
 
 	def appendIP(self, value):
-		self.__ipList.extend(value)
+		self._ipList.append(value)
 
 	def getIPList(self):
-		return self.__ipList
+		return self._ipList
 
-class Fail2banRegex:
 
-	test = None
+class LineStats(object):
+	"""Just a convenience container for stats
+	"""
+	def __init__(self):
+		self.tested = self.matched = 0
+		self.missed_lines = []
+		self.missed_lines_timeextracted = []
+		self.ignored_lines = []
+		self.ignored_lines_timeextracted = []
+
+	def __str__(self):
+		return "%(tested)d lines, %(ignored)d ignored, %(matched)d matched, %(missed)d missed" % self
+
+	@property
+	def ignored(self):
+		return len(self.ignored_lines)
+
+	@property
+	def missed(self):
+		return self.tested - (self.ignored + self.matched)
+
+	# just for convenient str
+	def __getitem__(self, key):
+		return getattr(self, key)
+
+
+class Fail2banRegex(object):
 
 	CONFIG_DEFAULTS = {'configpath' : "/etc/fail2ban/"}
 
-	def __init__(self):
+	def __init__(self, opts):
-		self.__filter = Filter(None)
+		self._verbose = opts.verbose
-		self.__ignoreregex = list()
+		self._debuggex = opts.debuggex
-		self.__failregex = list()
+		self._print_all_missed = opts.print_all_missed
-		self.__verbose = False
+		self._print_all_ignored = opts.print_all_ignored
-		# Setup logging
-		logging.getLogger("fail2ban").handlers = []
-		self.__hdlr = logging.StreamHandler(Fail2banRegex.test)
-		# set a format which is simpler for console use
-		formatter = logging.Formatter("%(message)s")
-		# tell the handler to use this format
-		self.__hdlr.setFormatter(formatter)
-		self.__logging_level = self.__verbose and logging.DEBUG or logging.WARN
-		logging.getLogger("fail2ban").addHandler(self.__hdlr)
-		logging.getLogger("fail2ban").setLevel(logging.ERROR)
 
-	#@staticmethod
+		self._filter = Filter(None)
-	def dispVersion():
+		self._ignoreregex = list()
-		print "Fail2Ban v" + version
+		self._failregex = list()
-		print
+		self._line_stats = LineStats()
-		print "Copyright (c) 2004-2008 Cyril Jaquier"
-		print "Copyright of modifications held by their respective authors."
-		print "Licensed under the GNU General Public License v2 (GPL)."
-		print
-		print "Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>."
-		print "Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>."
-	dispVersion = staticmethod(dispVersion)
 
-	#@staticmethod
-	def dispUsage():
-		print "Usage: "+sys.argv[0]+" [OPTIONS] <LOG> <REGEX> [IGNOREREGEX]"
-		print
-		print "Fail2Ban v" + version + " reads log file that contains password failure report"
-		print "and bans the corresponding IP addresses using firewall rules."
-		print
-		print "This tools can test regular expressions for \"fail2ban\"."
-		print
-		print "Options:"
-		print "    -h, --help              display this help message"
-		print "    -V, --version           print the version"
-		print "    -v, --verbose           verbose output"
-		print
-		print "Log:"
-		print "    string                  a string representing a log line"
-		print "    filename                path to a log file (/var/log/auth.log)"
-		print
-		print "Regex:"
-		print "    string                  a string representing a 'failregex'"
-		print "    filename                path to a filter file (filter.d/sshd.conf)"
-		print
-		print "IgnoreRegex:"
-		print "    string                  a string representing an 'ignoreregex'"
-		print "    filename                path to a filter file (filter.d/sshd.conf)"
-		print
-		print "Report bugs to https://github.com/fail2ban/fail2ban/issues"
-	dispUsage = staticmethod(dispUsage)
 
-	def getCmdLineOptions(self, optList):
+	def readRegex(self, value, regextype):
-		""" Gets the command line options
+		assert(regextype in ('fail', 'ignore'))
-		"""
+		regex = regextype + 'regex'
-		for opt in optList:
-			if opt[0] in ["-h", "--help"]:
-				self.dispUsage()
-				sys.exit(0)
-			elif opt[0] in ["-V", "--version"]:
-				self.dispVersion()
-				sys.exit(0)
-			elif opt[0] in ["-v", "--verbose"]:
-				self.__verbose = True
-
-	#@staticmethod
-	def logIsFile(value):
-		return os.path.isfile(value)
-	logIsFile = staticmethod(logIsFile)
-
-	def readIgnoreRegex(self, value):
 		if os.path.isfile(value):
 			reader = SafeConfigParserWithIncludes(defaults=self.CONFIG_DEFAULTS)
 			try:
 				reader.read(value)
-				print "Use ignoreregex file : " + value
+				print "Use %11s file : %s" % (regex, value)
-				self.__ignoreregex = [RegexStat(m)
+				# TODO: reuse functionality in client
-									for m in reader.get("Definition", "ignoreregex").split('\n')]
+				regex_values = [
+					RegexStat(m)
+					for m in reader.get("Definition", regex).split('\n')
+					if m != ""]
 			except NoSectionError:
-				print "No [Definition] section in " + value
+				print "No [Definition] section in %s" % value
-				print
 				return False
 			except NoOptionError:
-				print "No failregex option in " + value
+				print "No %s option in %s" % (regex, value)
-				print
 				return False
 			except MissingSectionHeaderError:
-				print "No section headers in " + value
+				print "No section headers in %s" % value
-				print
 				return False
 		else:
-			if len(value) > 53:
+			print "Use %11s line : %s" % (regex, shortstr(value))
-				stripReg = value[0:50] + "..."
+			regex_values = [RegexStat(value)]
-			else:
-				stripReg = value
-			print "Use ignoreregex line : " + stripReg
-			self.__ignoreregex = [RegexStat(value)]
-		return True
 
-	def readRegex(self, value):
+		setattr(self, "_" + regex, regex_values)
-		if os.path.isfile(value):
+		for regex in regex_values:
-			reader = SafeConfigParserWithIncludes(defaults=self.CONFIG_DEFAULTS)
+			getattr(
-			try:
+				self._filter,
-				reader.read(value)
+				'add%sRegex' % regextype.title())(regex.getFailRegex())
-				print "Use regex file : " + value
-				self.__failregex = [RegexStat(m)
-									for m in reader.get("Definition", "failregex").split('\n')]
-			except NoSectionError:
-				print "No [Definition] section in " + value
-				print
-				return False
-			except NoOptionError:
-				print "No failregex option in " + value
-				print
-				return False
-			except MissingSectionHeaderError:
-				print "No section headers in " + value
-				print
-				return False
-		else:
-			if len(value) > 53:
-				stripReg = value[0:50] + "..."
-			else:
-				stripReg = value
-			print "Use regex line : " + stripReg
-			self.__failregex = [RegexStat(value)]
 		return True
 
 	def testIgnoreRegex(self, line):
 		found = False
-		for regex in self.__ignoreregex:
+		try:
-			logging.getLogger("fail2ban").setLevel(self.__logging_level)
+			ret = self._filter.ignoreLine(line)
-			try:
+			if ret is not None:
-				self.__filter.addIgnoreRegex(regex.getFailRegex())
+				found = True
-				try:
+				regex = self._ignoreregex[ret].inc()
-					ret = self.__filter.ignoreLine(line)
+		except RegexException, e:
-					if ret:
+			print e
-						regex.inc()
+			return False
-				except RegexException, e:
+		return found
-					print e
-					return False
-			finally:
-				self.__filter.delIgnoreRegex(0)
-				logging.getLogger("fail2ban").setLevel(self.__logging_level)
 
 	def testRegex(self, line):
-		found = False
+		try:
-		for regex in self.__ignoreregex:
+			line, ret = self._filter.processLine(line, checkAllRegex=True)
-			self.__filter.addIgnoreRegex(regex.getFailRegex())
+			for match in ret:
-		for regex in self.__failregex:
+				# Append True/False flag depending if line was matched by
-			logging.getLogger("fail2ban").setLevel(logging.DEBUG)
+				# more than one regex
-			try:
+				match.append(len(ret)>1)
-				self.__filter.addFailRegex(regex.getFailRegex())
+				regex = self._failregex[match[0]]
-				try:
+				regex.inc()
-					ret = self.__filter.processLine(line)
+				regex.appendIP(match)
-					if not len(ret) == 0:
+		except RegexException, e:
-						if found == True:
+			print e
-							ret[0].append(True)
+			return False
-						else:
+		except IndexError:
-							found = True
+			print "Sorry, but no <HOST> found in regex"
-							ret[0].append(False)
+			return False
-						regex.inc()
+		return line, ret
-						regex.appendIP(ret)
+
-				except RegexException, e:
+
-					print e
+	def process(self, test_lines):
-					return False
+
-				except IndexError:
+		for line_no, line in enumerate(test_lines):
-					print "Sorry, but no <host> found in regex"
+			if line.startswith('#') or not line.strip():
-					return False
+				# skip comment and empty lines
-			finally:
+				continue
-				self.__filter.delFailRegex(0)
+			is_ignored = fail2banRegex.testIgnoreRegex(line)
-				logging.getLogger("fail2ban").setLevel(logging.CRITICAL)
+			line_datetimestripped, ret = fail2banRegex.testRegex(line)
-		for regex in self.__ignoreregex:
+
-					self.__filter.delIgnoreRegex(0)
+			if is_ignored:
+				self._line_stats.ignored_lines.append(line)
+				self._line_stats.ignored_lines_timeextracted.append(line_datetimestripped)
+
+			if len(ret) > 0:
+				assert(not is_ignored)
+				self._line_stats.matched += 1
+			else:
+				if not is_ignored:
+					self._line_stats.missed_lines.append(line)
+					self._line_stats.missed_lines_timeextracted.append(line_datetimestripped)
+			self._line_stats.tested += 1
+
+			if line_no % 10 == 0:
+				self._filter.dateDetector.sortTemplate()
+
+
+
+	def printLines(self, ltype):
+		lstats = self._line_stats
+		assert(len(lstats.missed_lines) == lstats.tested - (lstats.matched + lstats.ignored))
+		l = lstats[ltype + '_lines']
+		if len(l):
+			header = "%s line(s):" % (ltype.capitalize(),)
+			if self._debuggex:
+				if ltype == 'missed':
+					regexlist = self._failregex
+				else:
+					regexlist = self._ignoreregex
+				l = lstats[ltype + '_lines_timeextracted']
+				lines = len(l)*len(regexlist)
+				if lines < 20 or getattr(self, '_print_all_' + ltype):
+					ans = [[]]
+					for arg in [l, regexlist]:
+					    ans = [ x + [y] for x in ans for y in arg ]
+					b = map(lambda a: a[0] +  ' | ' + a[1].getFailRegex() + ' |  ' + debuggexURL(a[0], a[1].getFailRegex()), ans)
+					pprint_list([x.rstrip() for x in b], header)
+				else:
+					print "%s: too many to print.  Use --print-all-%s " \
+						  "to print all %d lines" % (header, ltype, lines)
+			elif len(l) < 20 or getattr(self, '_print_all_' + ltype):
+				pprint_list([x.rstrip() for x in l], header)
+			else:
+				print "%s: too many to print.  Use --print-all-%s " \
+					  "to print all %d lines" % (header, ltype, len(l))
 
 	def printStats(self):
 		print
 		print "Results"
 		print "======="
-		print
 
 		def print_failregexes(title, failregexes):
 			# Print title
@@ -264,106 +323,110 @@ class Fail2banRegex:
 			for cnt, failregex in enumerate(failregexes):
 				match = failregex.getStats()
 				total += match
-				if (match or self.__verbose):
+				if (match or self._verbose):
-					out.append("|  %d) [%d] %s" % (cnt+1, match, failregex.getFailRegex()))
+					out.append("%2d) [%d] %s" % (cnt+1, match, failregex.getFailRegex()))
-			print "%s: %d total" % (title, total)
+
-			if len(out):
+				if self._verbose and len(failregex.getIPList()):
-				print "|- #) [# of hits] regular expression"
+					for ip in failregex.getIPList():
-				print '\n'.join(out)
+						timeTuple = time.localtime(ip[2])
-				print '`-'
+						timeString = time.strftime("%a %b %d %H:%M:%S %Y", timeTuple)
-			print
+						out.append(
+							"    %s  %s%s" % (
+								ip[1],
+								timeString,
+								ip[3] and " (multiple regex matched)" or ""))
+
+			print "\n%s: %d total" % (title, total)
+			pprint_list(out, " #) [# of hits] regular expression")
 			return total
 
 		# Print title
-		total = print_failregexes("Failregex", self.__failregex)
+		total = print_failregexes("Failregex", self._failregex)
-		_ = print_failregexes("Ignoreregex", self.__ignoreregex)
+		_ = print_failregexes("Ignoreregex", self._ignoreregex)
 
-		print "Summary"
-		print "======="
-		print
 
-		if total == 0:
+		print "\nDate template hits:"
-			print "Sorry, no match"
+		out = []
-			print
+		for template in self._filter.dateDetector.getTemplates():
-			print "Look at the above section 'Running tests' which could contain important"
+			if self._verbose or template.getHits():
-			print "information."
+				out.append("[%d] %s" % (template.getHits(), template.getName()))
-			return False
+		pprint_list(out, "[# of hits] date format")
-		else:
-			# Print stats
-			print "Addresses found:"
-			for cnt, failregex in enumerate(self.__failregex):
-				if self.__verbose or len(failregex.getIPList()):
-					print "[%d]" % (cnt+1)
-					for ip in failregex.getIPList():
-						timeTuple = time.localtime(ip[1])
-						timeString = time.strftime("%a %b %d %H:%M:%S %Y", timeTuple)
-						print "    %s (%s)%s" % (
-							ip[0], timeString, ip[2] and " (already matched)" or "")
-			print
 
-			print "Date template hits:"
+		print "\nLines: %s" % self._line_stats
-			for template in self.__filter.dateDetector.getTemplates():
-				if self.__verbose or template.getHits():
-					print `template.getHits()` + " hit(s): " + template.getName()
-			print
 
-			print "Success, the total number of match is " + str(total)
+		self.printLines('ignored')
-			print
+		self.printLines('missed')
-			print "However, look at the above section 'Running tests' which could contain important"
+
-			print "information."
+		return True
-			return True
 
 
 if __name__ == "__main__":
-	fail2banRegex = Fail2banRegex()
+
-	# Reads the command line options.
+	parser = get_opt_parser()
-	try:
+	(opts, args) = parser.parse_args()
-		cmdOpts = 'hVcv'
+
-		cmdLongOpts = ['help', 'version', 'verbose']
+	fail2banRegex = Fail2banRegex(opts)
-		optList, args = getopt.getopt(sys.argv[1:], cmdOpts, cmdLongOpts)
-	except getopt.GetoptError:
-		fail2banRegex.dispUsage()
-		sys.exit(-1)
-	# Process command line
-	fail2banRegex.getCmdLineOptions(optList)
 
 	# We need 2 or 3 parameters
 	if not len(args) in (2, 3):
-		fail2banRegex.dispUsage()
+		sys.stderr.write("ERROR: provide both <LOG> and <REGEX>.\n\n")
+		parser.print_help()
 		sys.exit(-1)
+
+	# TODO: taken from -testcases -- move common functionality somewhere
+	if opts.log_level is not None: # pragma: no cover
+		# so we had explicit settings
+		logSys.setLevel(getattr(logging, opts.log_level.upper()))
+	else: # pragma: no cover
+		# suppress the logging but it would leave unittests' progress dots
+		# ticking, unless like with '-l fatal' which would be silent
+		# unless error occurs
+		logSys.setLevel(getattr(logging, 'FATAL'))
+
+	# Add the default logging handler
+	stdout = logging.StreamHandler(sys.stdout)
+
+	fmt = 'D: %(message)s'
+
+	if opts.log_traceback:
+		Formatter = FormatterWithTraceBack
+		fmt = (opts.full_traceback and ' %(tb)s' or ' %(tbc)s') + fmt
 	else:
-		print
+		Formatter = logging.Formatter
-		print "Running tests"
-		print "============="
-		print
 
-		cmd_log, cmd_regex = args[:2]
+	# Custom log format for the verbose tests runs
+	if opts.verbose > 1: # pragma: no cover
+		stdout.setFormatter(Formatter(' %(asctime)-15s %(thread)s' + fmt))
+	else: # pragma: no cover
+		# just prefix with the space
+		stdout.setFormatter(Formatter(fmt))
+	logSys.addHandler(stdout)
 
-		if len(args) == 3:
+	print
-			fail2banRegex.readIgnoreRegex(args[2]) or sys.exit(-1)
+	print "Running tests"
+	print "============="
+	print
 
-		fail2banRegex.readRegex(cmd_regex) or sys.exit(-1)
+	cmd_log, cmd_regex = args[:2]
 
-		if fail2banRegex.logIsFile(cmd_log):
+	if len(args) == 3:
-			try:
+		fail2banRegex.readRegex(args[2], 'ignore') or sys.exit(-1)
-				hdlr = open(cmd_log)
-				print "Use log file   : " + cmd_log
-				print
-				for line in hdlr:
-					fail2banRegex.testIgnoreRegex(line)
-					fail2banRegex.testRegex(line)
-			except IOError, e:
-				print e
-				print
-				sys.exit(-1)
-		else:
-			if len(sys.argv[1]) > 53:
-				stripLog = cmd_log[0:50] + "..."
-			else:
-				stripLog = cmd_log
-			print "Use single line: " + stripLog
-			print
-			fail2banRegex.testIgnoreRegex(cmd_log)
-			fail2banRegex.testRegex(cmd_log)
 
-		fail2banRegex.printStats() or sys.exit(-1)
+	fail2banRegex.readRegex(cmd_regex, 'fail') or sys.exit(-1)
+
+	if os.path.isfile(cmd_log):
+		try:
+			hdlr = open(cmd_log)
+			print "Use         log file : %s" % cmd_log
+			test_lines = hdlr # Iterable
+		except IOError, e:
+			print e
+			sys.exit(-1)
+	else:
+		print "Use      single line : %s" % shortstr(cmd_log)
+		test_lines = [ cmd_log ]
+	print
+
+	fail2banRegex.process(test_lines)
+
+	fail2banRegex.printStats() or sys.exit(-1)

fail2ban-testcases

@@ -27,6 +27,14 @@ __license__ = "GPL"
 
 import unittest, logging, sys, time, os
 
+if sys.version_info >= (2, 6):
+	import json
+else: # pragma: no cover
+	try:
+		import simplejson as json
+	except ImportError:
+		json = None
+
 from common.version import version
 from testcases import banmanagertestcase
 from testcases import clientreadertestcase
@@ -37,8 +45,11 @@ from testcases import datedetectortestcase
 from testcases import actiontestcase
 from testcases import sockettestcase
 from testcases import misctestcase
+if json:
+	from testcases import samplestestcase
 
 from testcases.utils import FormatterWithTraceBack
+from testcases import actionstestcase
 from server.mytime import MyTime
 
 from optparse import OptionParser, Option
@@ -52,7 +63,7 @@ def get_opt_parser():
 	p.add_options([
 		Option('-l', "--log-level", type="choice",
 			   dest="log_level",
-			   choices=('debug', 'info', 'warn', 'error', 'fatal'),
+			   choices=('heavydebug', 'debug', 'info', 'warning', 'error', 'fatal'),
 			   default=None,
 			   help="Log level for the logger to use during running tests"),
 		Option('-n', "--no-network", action="store_true",
@@ -76,9 +87,10 @@ parser = get_opt_parser()
 logSys = logging.getLogger("fail2ban")
 
 # Numerical level of verbosity corresponding to a log "level"
-verbosity = {'debug': 3,
+verbosity = {'heavydebug': 4,
+			 'debug': 3,
 			 'info': 2,
-			 'warn': 1,
+			 'warning': 1,
 			 'error': 1,
 			 'fatal': 0,
 			 None: 1}[opts.log_level]
@@ -97,7 +109,7 @@ stdout = logging.StreamHandler(sys.stdout)
 
 fmt = ' %(message)s'
 
-if opts.log_traceback:
+if opts.log_traceback or opts.full_traceback:
 	Formatter = FormatterWithTraceBack
 	fmt = (opts.full_traceback and ' %(tb)s' or ' %(tbc)s') + fmt
 else:
@@ -142,6 +154,7 @@ else: # pragma: no cover
 tests.addTest(unittest.makeSuite(servertestcase.Transmitter))
 tests.addTest(unittest.makeSuite(servertestcase.JailTests))
 tests.addTest(unittest.makeSuite(actiontestcase.ExecuteAction))
+tests.addTest(unittest.makeSuite(actionstestcase.ExecuteActions))
 # FailManager
 tests.addTest(unittest.makeSuite(failmanagertestcase.AddFailure))
 # BanManager
@@ -156,6 +169,7 @@ tests.addTest(unittest.makeSuite(sockettestcase.Socket))
 tests.addTest(unittest.makeSuite(misctestcase.HelpersTest))
 tests.addTest(unittest.makeSuite(misctestcase.SetupTest))
 tests.addTest(unittest.makeSuite(misctestcase.TestsUtilsTest))
+tests.addTest(unittest.makeSuite(misctestcase.CustomDateFormatsTest))
 
 # Filter
 if not opts.no_network:
@@ -169,6 +183,11 @@ tests.addTest(unittest.makeSuite(filtertestcase.JailTests))
 
 # DateDetector
 tests.addTest(unittest.makeSuite(datedetectortestcase.DateDetectorTest))
+if json:
+	# Filter Regex tests with sample logs
+	tests.addTest(unittest.makeSuite(samplestestcase.FilterSamplesRegex))
+else: # pragma: no cover
+	print "I: Skipping filter samples testing. No simplejson/json module"
 
 #
 # Extensive use-tests of different available filters backends

files/gen_badbots

@@ -0,0 +1,75 @@
+#!/bin/bash
+#-------------------------- =+- Shell script -+= --------------------------
+#
+#  Yaroslav Halchenko                                      CS@UNM, CS@NJIT
+#  web:     http://www.onerussian.com                      & PSYCH@RUTGERS
+#  e-mail:  yoh@onerussian.com                              ICQ#: 60653192
+#
+# DESCRIPTION (NOTES):
+#
+# Script to fetch list of agent strings from http://www.user-agents.org
+# which are known to be from mailicious bots, and create apache-badbots.conf
+# filter for fail2ban
+#
+# COPYRIGHT: Yaroslav Halchenko 2007-2013
+#
+# LICENSE:
+#
+#  This program is free software; you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation; either version 2 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program; if not, write to the 
+#  Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston,
+#  MA 02110-1301, USA.
+#
+# On Debian system see /usr/share/common-licenses/GPL for the full license.
+#
+#-----------------\____________________________________/------------------
+
+url=http://www.user-agents.org/index.shtml
+badbots=$(
+for f in "" "?g_m" "?moz" "?n_s" "?t_z"; do
+	wget -q -O- $url$f;
+done \
+| grep -h -B4 '<td class="smallcell" nowrap>S&nbsp;</td>'\
+| sed -e 's/&nbsp;//g' \
+| awk '/^--/{getline; gsub("  ",""); print $0}' \
+| sed -e 's/\([.\:|()]\)/\\\1/g' \
+| uniq \
+| tr '\n' '|' \
+| sed -e 's/|$//g'
+)
+
+echo $badbots >| /tmp/badbots.tmp
+
+cat >| config/filter.d/apache-badbots.conf <<EOF
+# Fail2Ban configuration file
+#
+# Regexp to catch known spambots and software alike. Please verify
+# that it is your intent to block IPs which were driven by
+# above mentioned bots.
+
+
+[Definition]
+
+badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider
+badbots = $badbots
+
+failregex = ^<HOST> -.*"(GET|POST).*HTTP.*"(?:%(badbots)s|%(badbotscustom)s)"$
+
+ignoreregex =
+
+# DEV Notes:
+# List of bad bots fetched from http://www.user-agents.org
+# Generated on `date` by $0.
+#
+# Author: Yaroslav Halchenko
+EOF

files/nagios/README

@@ -35,7 +35,7 @@ HELP:
 /etc/init.d/fail2ban stop
 
 2.) delete the socket if available
-rm /tmp/fail2ban.sock
+rm /var/run/fail2ban/fail2ban.sock
 
 3.) start the Service 
 /etc/init.d/fail2ban start

files/redhat-initd

@@ -1,38 +1,42 @@
 #!/bin/bash
 #
 # chkconfig: 345 92 08
-# description: Fail2ban daemon
+# processname: fail2ban-server
-#              http://fail2ban.sourceforge.net/wiki/index.php/Main_Page
+# config: /etc/fail2ban/fail2ban.conf
-# process name: fail2ban-server
+# pidfile: /var/run/fail2ban/fail2ban.pid
-#
+# description: fail2ban is a daemon to ban hosts that cause multiple authentication errors
-#
-# Author: Tyler Owen
 #
+### BEGIN INIT INFO
+# Provides: fail2ban
+# Required-Start: $local_fs $remote_fs
+# Required-Stop: $local_fs $remote_fs
+# Should-Start: $time $network $syslog iptables firehol shorewall ferm
+# Should-Stop: $network $syslog iptables firehol shorewall ferm
+# Default-Start: 3 4 5
+# Default-Stop: 0 1 6
+# Short-Description: Start/Stop fail2ban
+# Description: Start/Stop fail2ban, a daemon to ban hosts that cause multiple authentication errors
+### END INIT INFO
 
 # Source function library.
-. /etc/init.d/functions
+. /etc/rc.d/init.d/functions
 
 # Check that the config file exists
 [ -f /etc/fail2ban/fail2ban.conf ] || exit 0
 
 FAIL2BAN="/usr/bin/fail2ban-client"
-
+prog=fail2ban-server
+lockfile=${LOCKFILE-/var/lock/subsys/fail2ban}
+socket=${SOCKET-/var/run/fail2ban/fail2ban.sock}
+pidfile=${PIDFILE-/var/run/fail2ban/fail2ban.pid}
 RETVAL=0
 
-getpid() {
-    pid=`ps -eo pid,comm | grep fail2ban- | awk '{ print $1 }'`
-}
-
 start() {
     echo -n $"Starting fail2ban: "
-    getpid
+    ${FAIL2BAN} -x start > /dev/null
-    if [ -z "$pid" ]; then
+    RETVAL=$?
-	rm -rf /var/run/fail2ban/fail2ban.sock # in case of unclean shutdown
+    if [ $RETVAL = 0 ]; then
-        $FAIL2BAN start > /dev/null
+        touch ${lockfile}
-        RETVAL=$?
-    fi
-    if [ $RETVAL -eq 0 ]; then
-        touch /var/lock/subsys/fail2ban
         echo_success
     else
         echo_failure
@@ -43,51 +47,50 @@ start() {
 
 stop() {
     echo -n $"Stopping fail2ban: "
-    getpid
+    ${FAIL2BAN} stop > /dev/null
     RETVAL=$?
-    if [ -n "$pid" ]; then
+    if [ $RETVAL = 0 ]; then
-        $FAIL2BAN stop > /dev/null
+        rm -f ${lockfile} ${pidfile}
-    sleep 1
-    getpid
-    if [ -z "$pid" ]; then
-        rm -f /var/lock/subsys/fail2ban
         echo_success
     else
         echo_failure
     fi
-    else
+    echo
-        echo_failure
+    return $RETVAL
-    fi
+}
+
+reload() {
+    echo "Reloading fail2ban: "
+    ${FAIL2BAN} reload
+    RETVAL=$?
     echo
     return $RETVAL
 }
 
 # See how we were called.
 case "$1" in
-  start)
+    start)
+        status -p ${pidfile} ${prog} >/dev/null 2>&1 && exit 0
         start
         ;;
-  stop)
+    stop)
         stop
         ;;
-  status)
+    reload)
-        getpid
+        reload
-        if [ -n "$pid" ]; then
-                echo "Fail2ban (pid $pid) is running..."
-                $FAIL2BAN status
-        else
-                RETVAL=1
-                echo "Fail2ban is stopped"
-        fi
         ;;
-  restart)
+    restart)
         stop
         start
         ;;
-  *)
+    status)
-        echo $"Usage: $0 {start|stop|status|restart}"
+        status -p ${pidfile} ${prog}
-        exit 1
+        RETVAL=$?
+        [ $RETVAL = 0 ] && ${FAIL2BAN} status
         ;;
+    *)
+        echo $"Usage: fail2ban {start|stop|restart|reload|status}"
+        RETVAL=2
 esac
 
 exit $RETVAL

man/fail2ban-client.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.41.2.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.40.12.
-.TH FAIL2BAN-CLIENT "1" "June 2013" "fail2ban-client v0.8.10" "User Commands"
+.TH FAIL2BAN-CLIENT "1" "October 2013" "fail2ban-client v0.8.11.pre1" "User Commands"
 .SH NAME
 fail2ban-client \- configure and control the server
 .SH SYNOPSIS
 .B fail2ban-client
 [\fIOPTIONS\fR] \fI<COMMAND>\fR
 .SH DESCRIPTION
-Fail2Ban v0.8.10 reads log file that contains password failure report
+Fail2Ban v0.8.11.pre1 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .SH OPTIONS
 .TP
@@ -274,3 +274,4 @@ Licensed under the GNU General Public License v2 (GPL).
 .SH "SEE ALSO"
 .br 
 fail2ban-server(1)
+jail.conf(5)

man/fail2ban-client.h2m

@@ -10,3 +10,4 @@ fail2ban-client \- configure and control the server
 [see also]
 .br 
 fail2ban-server(1)
+jail.conf(5)

man/fail2ban-regex.1

@@ -1,53 +1,72 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.41.2.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.40.12.
-.TH FAIL2BAN-REGEX "1" "June 2013" "fail2ban-regex v0.8.10" "User Commands"
+.TH FAIL2BAN-REGEX "1" "October 2013" "fail2ban-regex 0.8.11.pre1" "User Commands"
 .SH NAME
 fail2ban-regex \- test Fail2ban "failregex" option
 .SH SYNOPSIS
 .B fail2ban-regex
 [\fIOPTIONS\fR] \fI<LOG> <REGEX> \fR[\fIIGNOREREGEX\fR]
 .SH DESCRIPTION
-Fail2Ban v0.8.10 reads log file that contains password failure report
+Fail2Ban  reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .PP
 This tools can test regular expressions for "fail2ban".
-.SH OPTIONS
+.SS "LOG:"
 .TP
-\fB\-h\fR, \fB\-\-help\fR
+string
-display this help message
-.TP
-\fB\-V\fR, \fB\-\-version\fR
-print the version
-.TP
-\fB\-v\fR, \fB\-\-verbose\fR
-verbose output
-.SH LOG
-.TP
-\fBstring\fR
 a string representing a log line
 .TP
-\fBfilename\fR
+filename
-path to a log file (\fI/var/log/auth.log\fP)
+path to a log file (/var/log/auth.log)
-.SH REGEX
+.SS "REGEX:"
 .TP
-\fBstring\fR
+string
 a string representing a 'failregex'
 .TP
-\fBfilename\fR
+filename
 path to a filter file (filter.d/sshd.conf)
-.SS "IgnoreRegex:"
+.SS "IGNOREREGEX:"
 .TP
-\fBstring\fR
+string
 a string representing an 'ignoreregex'
 .TP
-\fBfilename\fR
+filename
 path to a filter file (filter.d/sshd.conf)
+.SH OPTIONS
+.TP
+\fB\-\-version\fR
+show program's version number and exit
+.TP
+\fB\-h\fR, \fB\-\-help\fR
+show this help message and exit
+.TP
+\fB\-l\fR LOG_LEVEL, \fB\-\-log\-level\fR=\fILOG_LEVEL\fR
+Log level for the Fail2Ban logger to use
+.TP
+\fB\-v\fR, \fB\-\-verbose\fR
+Be verbose in output
+.TP
+\fB\-D\fR, \fB\-\-debuggex\fR
+Produce debuggex.com urls for debugging there
+.TP
+\fB\-\-print\-all\-missed\fR
+Either to print all missed lines
+.TP
+\fB\-\-print\-all\-ignored\fR
+Either to print all ignored lines
+.TP
+\fB\-t\fR, \fB\-\-log\-traceback\fR
+Enrich log\-messages with compressed tracebacks
+.TP
+\fB\-\-full\-traceback\fR
+Either to make the tracebacks full, not compressed (as
+by default)
 .SH AUTHOR
 Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
-Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>.
+Many contributions by Yaroslav O. Halchenko and Steven Hiscocks.
 .SH "REPORTING BUGS"
 Report bugs to https://github.com/fail2ban/fail2ban/issues
 .SH COPYRIGHT
-Copyright \(co 2004\-2008 Cyril Jaquier
+Copyright \(co 2004\-2008 Cyril Jaquier, 2008\- Fail2Ban Contributors
 .br
 Copyright of modifications held by their respective authors.
 Licensed under the GNU General Public License v2 (GPL).

man/fail2ban-server.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.41.2.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.40.12.
-.TH FAIL2BAN-SERVER "1" "June 2013" "fail2ban-server v0.8.10" "User Commands"
+.TH FAIL2BAN-SERVER "1" "October 2013" "fail2ban-server v0.8.11.pre1" "User Commands"
 .SH NAME
 fail2ban-server \- start the server
 .SH SYNOPSIS
 .B fail2ban-server
 [\fIOPTIONS\fR]
 .SH DESCRIPTION
-Fail2Ban v0.8.10 reads log file that contains password failure report
+Fail2Ban v0.8.11.pre1 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .PP
 Only use this command for debugging purpose. Start the server with

man/jail.conf.5

@@ -1,7 +1,10 @@
-.TH JAIL.CONF "5" "March 2013" "Fail2Ban" "Fail2Ban Configuration"
+.TH JAIL.CONF "10" "October 2013" "Fail2Ban" "Fail2Ban Configuration"
 .SH NAME
 jail.conf \- configuration for the fail2ban server
 .SH SYNOPSIS
+
+.I fail2ban.conf fail2ban.d/*.conf fail2ban.d/*.local
+
 .I jail.conf / jail.local
 
 .I action.d/*.conf action.d/*.local
@@ -45,6 +48,18 @@ identical entries, parsed alphabetically, e.g.
 .RE
 .RE
 
+The order \fIjail\fR configuration is parsed is:
+
+jail.conf ,
+jail.d/*.conf (in alphabetical order), 
+jail.local, followed by
+jail.d/*.local (in alphabetical order).
+
+Likewise for fail2ban configuration.
+
+Comments: use '#' for comment lines and ';' (following a space) for inline comments
+
+
 .SH DEFAULT
 The following options are applicable to all jails. Their meaning is described in the default \fIjail.conf\fR file.
 .TP

server/action.py

@@ -359,6 +359,10 @@ class Action:
 	#@staticmethod
 	def executeCmd(realCmd):
 		logSys.debug(realCmd)
+		if not realCmd:
+			logSys.debug("Nothing to do")
+			return True
+		
 		_cmd_lock.acquire()
 		try: # Try wrapped within another try needed for python version < 2.5
 			try:

server/datedetector.py

@@ -46,13 +46,13 @@ class DateDetector:
 	def addDefaultTemplate(self):
 		self.__lock.acquire()
 		try:
-			# standard
+			# asctime with subsecond
 			template = DateStrptime()
-			template.setName("MONTH Day Hour:Minute:Second")
+			template.setName("WEEKDAY MONTH Day Hour:Minute:Second[.subsecond] Year")
-			template.setRegex("\S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}")
+			template.setRegex("\S{3} \S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}\.\d+ \d{4}")
-			template.setPattern("%b %d %H:%M:%S")
+			template.setPattern("%a %b %d %H:%M:%S.%f %Y")
 			self._appendTemplate(template)
-			# asctime
+			# asctime without no subsecond
 			template = DateStrptime()
 			template.setName("WEEKDAY MONTH Day Hour:Minute:Second Year")
 			template.setRegex("\S{3} \S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2} \d{4}")
@@ -64,6 +64,12 @@ class DateDetector:
 			template.setRegex("\S{3} \S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%a %b %d %H:%M:%S")
 			self._appendTemplate(template)
+			# standard - most loose from above 3 so by default follows after
+			template = DateStrptime()
+			template.setName("MONTH Day Hour:Minute:Second")
+			template.setRegex("\S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}")
+			template.setPattern("%b %d %H:%M:%S")
+			self._appendTemplate(template)
 			# simple date
 			template = DateStrptime()
 			template.setName("Year/Month/Day Hour:Minute:Second")
@@ -151,7 +157,7 @@ class DateDetector:
 			self._appendTemplate(template)
 			# MySQL: 130322 11:46:11
 			template = DateStrptime()
-			template.setName("MonthDayYear Hour:Minute:Second")
+			template.setName("YearMonthDay Hour:Minute:Second")
 			template.setRegex("^\d{2}\d{2}\d{2} +\d{1,2}:\d{2}:\d{2}")
 			template.setPattern("%y%m%d %H:%M:%S")
 			self._appendTemplate(template)
@@ -174,6 +180,7 @@ class DateDetector:
 				match = template.matchDate(line)
 				if not match is None:
 					logSys.debug("Matched time template %s" % template.getName())
+					template.incHits()
 					return match
 			return None
 		finally:

server/datetemplate.py

@@ -59,11 +59,15 @@ class DateTemplate:
 	
 	def getHits(self):
 		return self.__hits
+
+	def incHits(self):
+		self.__hits += 1
+
+	def resetHits(self):
+		self.__hits = 0
 	
 	def matchDate(self, line):
 		dateMatch = self.__cRegex.search(line)
-		if not dateMatch is None:
-			self.__hits += 1
 		return dateMatch
 	
 	def getDate(self, line):
@@ -74,8 +78,7 @@ class DateEpoch(DateTemplate):
 	
 	def __init__(self):
 		DateTemplate.__init__(self)
-		# We already know the format for TAI64N
+		self.setRegex("(?:^|(?P<selinux>(?<=audit\()))\d{10}(?:\.\d{3,6})?(?(selinux)(?=:\d+\)))")
-		self.setRegex("^\d{10}(\.\d{6})?")
 	
 	def getDate(self, line):
 		date = None

server/failregex.py

@@ -41,7 +41,7 @@ class Regex:
 		self._matchCache = None
 		# Perform shortcuts expansions.
 		# Replace "<HOST>" with default regular expression for host.
-		regex = regex.replace("<HOST>", "(?:::f{4,6}:)?(?P<host>[\w\-.^_]+)")
+		regex = regex.replace("<HOST>", "(?:::f{4,6}:)?(?P<host>[\w\-.^_]*\w)")
 		if regex.lstrip() == '':
 			raise RegexException("Cannot add empty regex")
 		try:
@@ -50,7 +50,8 @@ class Regex:
 		except sre_constants.error:
 			raise RegexException("Unable to compile regular expression '%s'" %
 								 regex)
-	
+	def __str__(self):
+		return "%s(%r)" % (self.__class__.__name__, self._regex)
 	##
 	# Gets the regular expression.
 	#

server/filter.py

@@ -21,6 +21,8 @@ __author__ = "Cyril Jaquier and Fail2Ban Contributors"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2011-2013 Yaroslav Halchenko"
 __license__ = "GPL"
 
+import sys
+
 from failmanager import FailManagerEmpty
 from failmanager import FailManager
 from ticket import FailTicket
@@ -282,7 +284,7 @@ class Filter(JailThread):
 		return False
 
 
-	def processLine(self, line):
+	def processLine(self, line, returnRawHost=False, checkAllRegex=False):
 		"""Split the time portion from log msg and return findFailures on them
 		"""
 		try:
@@ -291,6 +293,8 @@ class Filter(JailThread):
 		except UnicodeDecodeError:
 			l = line
 		l = l.rstrip('\r\n')
+
+		logSys.log(7, "Working on line %r", l)
 		timeMatch = self.dateDetector.matchTime(l)
 		if timeMatch:
 			# Lets split into time part and log part of the line
@@ -302,14 +306,15 @@ class Filter(JailThread):
 		else:
 			timeLine = l
 			logLine = l
-		return self.findFailure(timeLine, logLine)
+		return logLine, self.findFailure(timeLine, logLine, returnRawHost, checkAllRegex)
 
 	def processLineAndAdd(self, line):
 		"""Processes the line for failures and populates failManager
 		"""
-		for element in self.processLine(line):
+		for element in self.processLine(line)[1]:
-			ip = element[0]
+			failregex = element[0]
-			unixTime = element[1]
+			ip = element[1]
+			unixTime = element[2]
 			logSys.debug("Processing line with time:%s and ip:%s"
 						 % (unixTime, ip))
 			if unixTime < MyTime.time() - self.getFindTime():
@@ -320,6 +325,7 @@ class Filter(JailThread):
 				logSys.debug("Ignore %s" % ip)
 				continue
 			logSys.debug("Found %s" % ip)
+			## print "D: Adding a ticket for %s" % ((ip, unixTime, [line]),)
 			self.failManager.addFailure(FailTicket(ip, unixTime, [line]))
 
 	##
@@ -330,11 +336,11 @@ class Filter(JailThread):
 	# @return: a boolean
 
 	def ignoreLine(self, line):
-		for ignoreRegex in self.__ignoreRegex:
+		for ignoreRegexIndex, ignoreRegex in enumerate(self.__ignoreRegex):
 			ignoreRegex.search(line)
 			if ignoreRegex.hasMatched():
-				return True
+				return ignoreRegexIndex
-		return False
+		return None
 
 	##
 	# Finds the failure in a line given split into time and log parts.
@@ -343,18 +349,22 @@ class Filter(JailThread):
 	# to find the logging time.
 	# @return a dict with IP and timestamp.
 
-	def findFailure(self, timeLine, logLine):
+	def findFailure(self, timeLine, logLine,
+			returnRawHost=False, checkAllRegex=False):
+		logSys.log(5, "Date: %r, message: %r", timeLine, logLine)
 		failList = list()
 		# Checks if we must ignore this line.
-		if self.ignoreLine(logLine):
+		if self.ignoreLine(logLine) is not None:
 			# The ignoreregex matched. Return.
+			logSys.log(7, "Matched ignoreregex and was ignored")
 			return failList
+		date = self.dateDetector.getUnixTime(timeLine)
 		# Iterates over all the regular expressions.
-		for failRegex in self.__failRegex:
+		for failRegexIndex, failRegex in enumerate(self.__failRegex):
 			failRegex.search(logLine)
 			if failRegex.hasMatched():
 				# The failregex matched.
-				date = self.dateDetector.getUnixTime(timeLine)
+				logSys.log(7, "Matched %s", failRegex)
 				if date is None:
 					logSys.debug("Found a match for %r but no valid date/time "
 								 "found for %r. Please file a detailed issue on"
@@ -364,12 +374,17 @@ class Filter(JailThread):
 				else:
 					try:
 						host = failRegex.getHost()
-						ipMatch = DNSUtils.textToIp(host, self.__useDns)
+						if returnRawHost:
-						if ipMatch:
+							failList.append([failRegexIndex, host, date])
-							for ip in ipMatch:
+							if not checkAllRegex:
-								failList.append([ip, date])
+								break
-							# We matched a regex, it is enough to stop.
+						else:
-							break
+							ipMatch = DNSUtils.textToIp(host, self.__useDns)
+							if ipMatch:
+								for ip in ipMatch:
+									failList.append([failRegexIndex, ip, date])
+								if not checkAllRegex:
+									break
 					except RegexException, e: # pragma: no cover - unsure if reachable
 						logSys.error(e)
 		return failList
@@ -473,7 +488,7 @@ class FileFilter(Filter):
 			return False
 		# Try to open log file.
 		try:
-			container.open()
+			has_content = container.open()
 		# see http://python.org/dev/peps/pep-3151/
 		except IOError, e:
 			logSys.error("Unable to open %s" % filename)
@@ -488,7 +503,12 @@ class FileFilter(Filter):
 			logSys.exception(e)
 			return False
 
-		while True:
+		# yoh: has_content is just a bool, so do not expect it to
+		# change -- loop is exited upon break, and is not entered at
+		# all if upon container opening that one was empty.  If we
+		# start reading tested to be empty container -- race condition
+		# might occur leading at least to tests failures.
+		while has_content:
 			line = container.readline()
 			if (line == "") or not self._isActive():
 				# The jail reached the bottom or has been stopped
@@ -551,10 +571,20 @@ class FileContainer:
 		fd = self.__handler.fileno()
 		flags = fcntl.fcntl(fd, fcntl.F_GETFD)
 		fcntl.fcntl(fd, fcntl.F_SETFD, flags | fcntl.FD_CLOEXEC)
+		# Stat the file before even attempting to read it
+		stats = os.fstat(self.__handler.fileno())
+		if not stats.st_size:
+			# yoh: so it is still an empty file -- nothing should be
+			#      read from it yet
+			# print "D: no content -- return"
+			return False
 		firstLine = self.__handler.readline()
 		# Computes the MD5 of the first line.
 		myHash = md5sum(firstLine).digest()
-		stats = os.fstat(self.__handler.fileno())
+		## print "D: fn=%s hashes=%s/%s inos=%s/%s pos=%s rotate=%s" % (
+		## 	self.__filename, self.__hash, myHash, stats.st_ino, self.__ino, self.__pos,
+		## 	self.__hash != myHash or self.__ino != stats.st_ino)
+		## sys.stdout.flush()
 		# Compare hash and inode
 		if self.__hash != myHash or self.__ino != stats.st_ino:
 			logSys.debug("Log rotation detected for %s" % self.__filename)
@@ -563,6 +593,7 @@ class FileContainer:
 			self.__pos = 0
 		# Sets the file pointer to the last position.
 		self.__handler.seek(self.__pos)
+		return True
 
 	def readline(self):
 		if self.__handler is None:
@@ -576,6 +607,8 @@ class FileContainer:
 			# Closes the file.
 			self.__handler.close()
 			self.__handler = None
+		## print "D: Closed %s with pos %d" % (handler, self.__pos)
+		## sys.stdout.flush()
 
 
 
@@ -598,9 +631,9 @@ class DNSUtils:
 		"""
 		try:
 			return socket.gethostbyname_ex(dns)[2]
-		except socket.gaierror:
+		except socket.error, e:
-			logSys.warn("Unable to find a corresponding IP address for %s"
+			logSys.warn("Unable to find a corresponding IP address for %s: %s"
-						% dns)
+						% (dns, e))
 			return list()
 	dnsToIp = staticmethod(dnsToIp)