MRG: merge from master

ChangeLog

@@ -45,6 +45,8 @@ ver. 0.8.11 (2013/XX/XXX) - loves-unittests
 -----------
 
 - Fixes:
+  Daniel Black & Marcel Dopita
+   * filter.d/apache-auth -- fixed and apache auth samples provide. closes #286
   Yaroslav Halchenko
    * filter.d/common.conf -- make colon after [daemon] optional. Closes gh-267
    * filter.d/apache-common.conf -- support apache 2.4 more detailed error
@@ -61,24 +63,42 @@ ver. 0.8.11 (2013/XX/XXX) - loves-unittests
   Daniel Black & Мернов Георгий
    * filter.d/dovecot.conf -- Fix when no TLS enabled - line doesn't end in ,
   Daniel Black
-  * action.d/hostsdeny -- NOTE: new dependancy 'ed'. Switched to use 'ed' across
+   * action.d/hostsdeny -- NOTE: new dependancy 'ed'. Switched to use 'ed' across
-    all platforms to ensure permissions are the same before and after a ban -
+     all platforms to ensure permissions are the same before and after a ban -
-		closes gh-266. hostsdeny supports daemon_list now too.
+     closes gh-266. hostsdeny supports daemon_list now too.
+  * filter.d/roundcube-auth - timezone offset can be positive or negative
+  Rolf Fokkens
+   * action.d/dshield.conf and complain.conf -- reorder mailx arguments.
+     https://bugzilla.redhat.com/show_bug.cgi?id=998020
+  John Doe (ache)
+   * action.d/bsd-ipfw.conf - invert actionstop logic to make exist status 0.
+     closes gh-343.
+  JP Espinosa (Reviewed by O.Poplawski)
+   * files/redhat-initd - rewritten to use stock init.d functions thus
+     avoiding problems with getpid.  Also $network and iptables moved
+     to Should- rc init fields
 - New Features:
   Daniel Black & ykimon
    * filter.d/3proxy.conf -- filter added
   Daniel Black
    * filter.d/exim-spam.conf -- a splitout of exim's spam regexes
      with additions for greater control over filtering spam.
+   * add date expression for apache-2.4 - milliseconds
   Christophe Carles & Daniel Black
    * filter.d/perdition.conf -- filter added
+
 - Enhancements:
+  François Boulogne and Frédéric
+   * filter.d/lighttpd - auth regexs for lighttpd-1.4.31
   Daniel Black
    * filter.d/{asterisk,assp,dovecot,proftpd}.conf -- regex hardening
      and extra failure examples in sample logs
+	 * filter.d/apache-auth - added expressions for mod_authz, mod_auth and
+     mod_auth_digest failures.
   Daniel Black & Georgiy Mernov & ftoppi & Мернов Георгий
    * filter.d/exim.conf -- regex hardening and extra failure examples in
      sample logs
+   * filter.d/named-refused.conf - BIND 9.9.3 regex changes
   Daniel Black & Sebastian Arcus
    * filter.d/asterisk -- more regexes
   Yaroslav Halchenko
@@ -89,12 +109,25 @@ ver. 0.8.11 (2013/XX/XXX) - loves-unittests
      enabled jail. Closes gh-63
    * <HOST> is now enforced to end with an alphanumeric
    * filter.d/roundcube-auth.conf -- anchored version
+   * date matching - for standard asctime formats prefer more detailed
+     first (thus use year if available)
   Alexander Dietrich
    * action.d/sendmail-common.conf -- added common sendmail settings file
      and made the sender display name configurable
   Steven Hiscocks
    * filter.d/dovecot - Addition of session, time values and possible blank
      user
+  Zurd and Daniel Black
+   * filter/named-refused - added refused on zone transfer
+   * filter.d/{courier{login,smtp},proftpd,sieve,wuftpd,xinetd} - General
+     regex impovements
+   * IMPORTANT: 'lighttpd-fastcgi' filter has been renamed to 'suhosin', which
+     will require changing in jail.{conf,local} if using this filter.
+  Zurd
+   * filter.d/postfix - add filter for VRFY failures. closes gh-322.
+  Orion Poplawski
+   * fail2ban.d/ and jail.d/ directories are added to etc/fail2ban to facilitate
+     their use
 
 ver. 0.8.10 (2013/06/12) - wanna-be-secure
 -----------

MANIFEST

@@ -103,7 +103,7 @@ config/filter.d/couriersmtp.conf
 config/filter.d/cyrus-imap.conf
 config/filter.d/exim.conf
 config/filter.d/gssftpd.conf
-config/filter.d/lighttpd-fastcgi.conf
+config/filter.d/suhosin.conf
 config/filter.d/named-refused.conf
 config/filter.d/postfix.conf
 config/filter.d/proftpd.conf

THANKS

@@ -5,6 +5,7 @@ know (preferably send a pull request on github with the "fix") and you
 will be added
 
 Adrien Clerc
+ache
 Andrey G. Grozin
 Arturo 'Buanzo' Busleiman
 Axel Thimm
@@ -20,6 +21,8 @@ David Nutter
 Eric Gerbier
 Enrico Labedzki
 ftoppi
+François Boulogne
+Frédéric
 Georgiy Mernov
 Guillaume Delvit
 Hanno 'Rince' Wagner
@@ -27,9 +30,11 @@ Iain Lea
 Jonathan Kamens
 Jonathan Underwood
 Joël Bertrand
+JP Espinosa
 Justin Shore
 Kévin Drapel
 kojiro
+Marcel Dopita
 Mark Edgington
 Markus Hoffmann
 Marvin Rouge
@@ -42,6 +47,7 @@ Patrick Börjesson
 Raphaël Marichez
 René Berber
 Robert Edeker
+Rolf Fokkens
 Russell Odom
 Sebastian Arcus
 Sireyessire
@@ -56,3 +62,4 @@ Yaroslav Halchenko
 ykimon
 Yehuda Katz
 zugeschmiert
+Zurd

bin/fail2ban-testcases

@@ -112,8 +112,8 @@ logSys.addHandler(stdout)
 # Let know the version
 #
 if not opts.log_level or opts.log_level != 'fatal': # pragma: no cover
-	print "Fail2ban %s test suite. Python %s. Please wait..." \
+	print("Fail2ban %s test suite. Python %s. Please wait..." \
-	    % (version, str(sys.version).replace('\n', ''))
+	    % (version, str(sys.version).replace('\n', '')))
 
 tests = gatherTests(regexps, opts.no_network)
 #

config/action.d/bsd-ipfw.conf

@@ -21,7 +21,7 @@ actionstart = ipfw show | fgrep -q 'table(<table>)' || ( ipfw show | awk 'BEGIN
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop =  [ -f <startstatefile> ] && ( read num < "<startstatefile>" <br> ipfw -q delete $num <br> rm "<startstatefile>" )
+actionstop =  [ ! -f <startstatefile> ] || ( read num < "<startstatefile>" <br> ipfw -q delete $num <br> rm "<startstatefile>" )
 
 
 # Option:  actioncheck

config/action.d/complain.conf

@@ -58,7 +58,7 @@ actioncheck =
 actionban = ADDRESSES=`whois <ip> | perl -e 'while (<STDIN>) { next if /^changed|@(ripe|apnic)\.net/io; $m += (/abuse|trouble:|report|spam|security/io?3:0); if (/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)/io) { while (s/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)//io) { if ($m) { $a{lc($1)}=$m } else { $b{lc($1)}=$m } } $m=0 } else { $m && --$m } } if (%%a) {print join(",",keys(%%a))} else {print join(",",keys(%%b))}'`
 	    IP=<ip>
             if [ ! -z "$ADDRESSES" ]; then
-                (printf %%b "<message>\n"; date '+Note: Local timezone is %%z (%%Z)'; grep '<ip>' <logpath>) | <mailcmd> "Abuse from <ip>" $ADDRESSES <mailargs>
+                (printf %%b "<message>\n"; date '+Note: Local timezone is %%z (%%Z)'; grep '<ip>' <logpath>) | <mailcmd> "Abuse from <ip>" <mailargs> $ADDRESSES
             fi
 
 # Option:  actionunban

config/action.d/dshield.conf

@@ -39,7 +39,7 @@ actionstart =
 # Values:  CMD
 #
 actionstop = if [ -f <tmpfile>.buffer ]; then
-                 cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ `date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'` Fail2Ban" <dest> <mailargs>
+                 cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ `date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'` Fail2Ban" <mailargs> <dest>
                  date +%%s > <tmpfile>.lastsent
              fi
              rm -f <tmpfile>.buffer <tmpfile>.first
@@ -80,7 +80,7 @@ actionban = TZONE=`date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'`
             LASTREPORT=$(($NOW - `cat <tmpfile>.lastsent`))
             LINES=$( wc -l <tmpfile>.buffer | awk '{ print $1 }' )
             if [ $LINES -ge <lines> && $LASTREPORT -gt <minreportinterval> ] || [ $LOGAGE -gt <maxbufferage> ]; then
-                cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ $TZONE Fail2Ban" <dest> <mailargs>
+                cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ $TZONE Fail2Ban" <mailargs> <dest>
                 rm -f <tmpfile>.buffer <tmpfile>.first
                 echo $NOW > <tmpfile>.lastsent
             fi
@@ -95,7 +95,7 @@ actionunban = if [ -f <tmpfile>.first ]; then
                   NOW=`date +%%s`
                   LOGAGE=$(($NOW - `cat <tmpfile>.first`))
                   if [ $LOGAGE -gt <maxbufferage> ]; then
-                      cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ `date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'` Fail2Ban" <dest> <mailargs>
+                      cat <tmpfile>.buffer | <mailcmd> "FORMAT DSHIELD USERID <userid> TZ `date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'` Fail2Ban" <mailargs> <dest>
                       rm -f <tmpfile>.buffer <tmpfile>.first
                       echo $NOW > <tmpfile>.lastsent
                   fi

config/action.d/dummy.conf

@@ -10,14 +10,14 @@
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = touch /tmp/fail2ban.dummy
+actionstart = touch /var/run/fail2ban/fail2ban.dummy
-              printf %%b "<init>\n" >> /tmp/fail2ban.dummy
+              printf %%b "<init>\n" >> /var/run/fail2ban/fail2ban.dummy
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = rm -f /tmp/fail2ban.dummy
+actionstop = rm -f /var/run/fail2ban/fail2ban.dummy
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -31,7 +31,7 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "+<ip>\n" >> /tmp/fail2ban.dummy
+actionban = printf %%b "+<ip>\n" >> /var/run/fail2ban/fail2ban.dummy
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -39,7 +39,7 @@ actionban = printf %%b "+<ip>\n" >> /tmp/fail2ban.dummy
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = printf %%b "-<ip>\n" >> /tmp/fail2ban.dummy
+actionunban = printf %%b "-<ip>\n" >> /var/run/fail2ban/fail2ban.dummy
 
 [Init]
 

config/filter.d/apache-auth.conf

@@ -12,14 +12,42 @@ before = apache-common.conf
 
 [Definition]
 
-# Option:  failregex
+# This filter matches the authorization failures of Apache. It takes the log messages
-# Notes.:  regex to match the password failure messages in the logfile. The
+# from the modules in aaa that return HTTP_UNAUTHORIZED, HTTP_METHOD_NOT_ALLOWED or
-#          host must be matched by a group named "host". The tag "<HOST>" can
+# HTTP_FORBIDDEN and not AUTH_GENERAL_ERROR or HTTP_INTERNAL_SERVER_ERROR.
-#          be used for standard IP/hostname matching and is only an alias for
-#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
-# Values:  TEXT
 #
-failregex = ^%(_apache_error_client)s user .* (authentication failure|not found|password mismatch)\s*$
+# An unauthorized response 401 is the first step for a browser to instigate authentication
+# however apache doesn't log this as an error. Only subsequent errors are logged in the 
+# error log.
+#
+# Source:
+#
+# By searching the code in http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/aaa/*
+# for ap_log_rerror(APLOG_MARK, APLOG_ERR and examining resulting return code should get
+# all of these expressions. Lots of submodules like mod_authz_* return back to mod_authz_core
+# to return the actual failure.
+#
+# See also: http://wiki.apache.org/httpd/ListOfErrors
+# Expressions that don't have tests and aren't common.
+# more be added with  https://issues.apache.org/bugzilla/show_bug.cgi?id=55284 
+#     ^%(_apache_error_client)s (AH01778: )?user .*: nonce expired \([\d.]+ seconds old - max lifetime [\d.]+\) - sending new nonce\s*$
+#     ^%(_apache_error_client)s (AH01779: )?user .*: one-time-nonce mismatch - sending new nonce\s*$
+#     ^%(_apache_error_client)s (AH02486: )?realm mismatch - got `.*' but no realm specified\s*$
+#
+failregex = ^%(_apache_error_client)s (AH01797: )?client denied by server configuration: (uri )?\S*\s*$
+            ^%(_apache_error_client)s (AH01617: )?user .* authentication failure for "\S*": Password Mismatch$
+            ^%(_apache_error_client)s (AH01618: )?user .* not found(: )?\S*\s*$
+            ^%(_apache_error_client)s (AH01614: )?client used wrong authentication scheme: \S*\s*$
+            ^%(_apache_error_client)s (AH\d+: )?Authorization of user \S+ to access \S* failed, reason: .*$
+            ^%(_apache_error_client)s (AH0179[24]: )?(Digest: )?user .*: password mismatch: \S*\s*$
+            ^%(_apache_error_client)s (AH0179[01]: |Digest: )user `.*' in realm `.+' (not found|denied by provider): \S*\s*$
+            ^%(_apache_error_client)s (AH01631: )?user .*: authorization failure for "\S*":\s*$
+            ^%(_apache_error_client)s (AH01775: )?(Digest: )?invalid nonce .* received - length is not \S+\s*$
+            ^%(_apache_error_client)s (AH01788: )?(Digest: )?realm mismatch - got `.*' but expected `.+'\s*$
+            ^%(_apache_error_client)s (AH01789: )?(Digest: )?unknown algorithm `.*' received: \S*\s*$
+            ^%(_apache_error_client)s (AH01793: )?invalid qop `.*' received: \S*\s*$
+            ^%(_apache_error_client)s (AH01777: )?(Digest: )?invalid nonce .* received - user attempted time travel\s*$
+
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/apache-common.conf

@@ -18,4 +18,4 @@ after = apache-common.local
 # 2.2: [Sat Jun 01 11:23:08 2013] [error] [client 1.2.3.4]
 # 2.4: [Thu Jun 27 11:55:44.569531 2013] [core:info] [pid 4101:tid 2992634688] [client 1.2.3.4:46652]
 # Reference: https://github.com/fail2ban/fail2ban/issues/268
-_apache_error_client = \[[^]]*\] \[(error|core:\S+)\]( \[pid \d+:\S+ \d+\])? \[client <HOST>(:\d{1,5})?\]( \S+:)?
+_apache_error_client = \[[^]]*\] \[(error|\S+:\S+)\]( \[pid \d+:\S+ \d+\])? \[client <HOST>(:\d{1,5})?\]

config/filter.d/apache-nohome.conf

@@ -19,7 +19,7 @@ before = apache-common.conf
 #          per-domain log files.
 # Values:  TEXT
 #
-failregex = ^%(_apache_error_client)s File does not exist: .*/~.*
+failregex = ^%(_apache_error_client)s (AH00128: )?File does not exist: .*/~.*
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/asterisk.conf

@@ -30,7 +30,7 @@ failregex = ^%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?'
             ^%(log_prefix)s Host <HOST> failed to authenticate as '[^']*'$
             ^%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$
             ^%(log_prefix)s Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$
-            ^%(log_prefix)s Failed to authenticate user [^@]+@<HOST>\S*$
+            ^%(log_prefix)s Failed to authenticate (user|device) [^@]+@<HOST>\S*$
             ^%(log_prefix)s (?:handle_request_subscribe: )?Sending fake auth rejection for (device|user) \d*<sip:[^@]+@<HOST>>;tag=\w+\S*$
             ^%(log_prefix)s SecurityEvent="(FailedACL|InvalidAccountID|ChallengeResponseFailed|InvalidPassword)",EventTV="[\d-]+",Severity="[\w]+",Service="[\w]+",EventVersion="\d+",AccountID="\d+",SessionID="0x[\da-f]+",LocalAddress="IPV[46]/(UD|TC)P/[\da-fA-F:.]+/\d+",RemoteAddress="IPV[46]/(UD|TC)P/<HOST>/\d+"(,Challenge="\w+",ReceivedChallenge="\w+")?(,ReceivedHash="[\da-f]+")?$
 

config/filter.d/courier-auth.conf

@@ -5,8 +5,17 @@
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+
 [Definition]
 
+_daemon = (?:courier)?(?:imapd?|pop3d?)(?:login)?(?:-ssl)?
+
 # Option:  failregex
 # Notes.:  regex to match the password failures messages in the logfile. The
 #          host must be matched by a group named "host". The tag "<HOST>" can
@@ -14,7 +23,7 @@
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = LOGIN FAILED, .*, ip=\[<HOST>\]$
+failregex = ^%(__prefix_line)sLOGIN FAILED, user=.*, ip=\[<HOST>\]$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/courier-smtp.conf

@@ -4,8 +4,17 @@
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+
 [Definition]
 
+_daemon = courieresmtpd
+
 # Option:  failregex
 # Notes.:  regex to match the password failures messages in the logfile. The
 #          host must be matched by a group named "host". The tag "<HOST>" can
@@ -13,7 +22,7 @@
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = error,relay=<HOST>,.*550 User unknown
+failregex = ^%(__prefix_line)serror,relay=<HOST>,.*: 550 User unknown\.$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/cyrus-imap.conf

@@ -4,8 +4,17 @@
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+
 [Definition]
 
+_daemon = (?:cyrus/)?(?:imapd?|pop3d?)
+
 # Option:  failregex
 # Notes.:  regex to match the password failures messages in the logfile. The
 #          host must be matched by a group named "host". The tag "<HOST>" can
@@ -13,10 +22,7 @@
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = : badlogin: .*\[<HOST>\] plaintext .*SASL\(-13\): authentication failure: checkpass failed$
+failregex = ^%(__prefix_line)sbadlogin: \S+ ?\[<HOST>\] \S+ .*?\[?SASL\(-13\): authentication failure: .*\]?$
-	    : badlogin: .*\[<HOST>\] LOGIN \[SASL\(-13\): authentication failure: checkpass failed\]$
-	    : badlogin: .*\[<HOST>\] (?:CRAM-MD5|NTLM) \[SASL\(-13\): authentication failure: incorrect (?:digest|NTLM) response\]$
-	    : badlogin: .*\[<HOST>\] DIGEST-MD5 \[SASL\(-13\): authentication failure: client response doesn't match what we generated\]$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/dropbear.conf

@@ -27,8 +27,9 @@ _daemon = dropbear
 # These match the unmodified dropbear messages. It isn't possible to
 # match the source of the 'exit before auth' messages from dropbear.
 #
-failregex = ^%(__prefix_line)slogin attempt for nonexistent user ('.*' )?from <HOST>:.*\s*$
+failregex = ^%(__prefix_line)s(L|l)ogin attempt for nonexistent user ('.*' )?from <HOST>:.*\s*$
-            ^%(__prefix_line)sbad password attempt for .+ from <HOST>:.*\s*$
+            ^%(__prefix_line)s(B|b)ad password attempt for .+ from <HOST>:.*\s*$
+            ^%(__prefix_line)sExit before auth \(user '.+', \d+ fails\): Max auth tries reached - user '.+' from <HOST>:\d+\s*$
 
 # The only line we need to match with the modified dropbear.
 

config/filter.d/exim.conf

@@ -20,7 +20,7 @@ before = exim-common.conf
 #          Note the %(host_info) defination contains a <HOST> match
 
 failregex = ^%(pid)s %(host_info)ssender verify fail for <\S+>: (?:Unknown user|Unrouteable address|all relevant MX records point to non-existent hosts)\s*$
-             ^%(pid)s login authenticator failed for (\S+ )?\(\S+\) \[<HOST>\]: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
+             ^%(pid)s (plain|login) authenticator failed for (\S+ )?\(\S+\) \[<HOST>\]: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
              ^%(pid)s %(host_info)sF=(<>|[^@]+@\S+) rejected RCPT [^@]+@\S+: (relay not permitted|Sender verify failed|Unknown user)\s*$
              ^%(pid)s SMTP protocol synchronization error \(.*\): rejected (connection from|"\S+") %(host_info)s(next )?input=".*"\s*$
              ^%(pid)s SMTP call from \S+ \[<HOST>\](:\d+)? (I=\[\S+\]:\d+ )?dropped: too many nonmail commands \(last was "\S+"\)\s*$

config/filter.d/lighttpd-auth.conf

@@ -9,7 +9,7 @@
 # Notes.:  regex to match wrong passwords as notified by lighttpd's auth Module
 # Values:  TEXT
 #
-failregex = .*http_auth.*(password doesn\'t match|wrong password).*IP: <HOST>\s*$
+failregex = ^: \(http_auth\.c\.\d+\) (password doesn\'t match .* username: .*|digest: auth failed for .*: wrong password|get_password failed), IP: <HOST>\s*$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/named-refused.conf

@@ -21,14 +21,11 @@ __daemon_combs_re=(?:%(__pid_re)s?:\s+%(__daemon_re)s|%(__daemon_re)s%(__pid_re)
 # this can be optional (for instance if we match named native log files)
 __line_prefix=(?:\s\S+ %(__daemon_combs_re)s\s+)?
 
-# Option: failregex
-# Notes.: regex to match the password failures messages in the logfile.
-# Values: TEXT
-#
-failregex = %(__line_prefix)sclient <HOST>#\S+: (view (internal|external): )?query(?: \(cache\))? '.*' denied\s*$
 
-# Option:  ignoreregex
+# note - (\.\d+)? is a really ugly catch of the microseconds not captured in
-# Notes.:  regex to ignore. If this regex matches, the line is ignored.
+# in the date detector
-# Values:  TEXT
 #
-ignoreregex = 
+failregex = ^%(__line_prefix)s(\.\d+)?( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: (view (internal|external): )?query(?: \(cache\))? '.*' denied\s*$
+            ^%(__line_prefix)s(\.\d+)?( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: zone transfer '\S+/AXFR/\w+' denied\s*$
+            ^%(__line_prefix)s(\.\d+)?( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: bad zone transfer request: '\S+/IN': non-authoritative zone \(NOTAUTH\)\s*$
+

config/filter.d/postfix.conf

@@ -4,8 +4,17 @@
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+
 [Definition]
 
+_daemon = postfix/smtpd
+
 # Option:  failregex
 # Notes.:  regex to match the password failures messages in the logfile. The
 #          host must be matched by a group named "host". The tag "<HOST>" can
@@ -13,8 +22,9 @@
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = reject: RCPT from (.*)\[<HOST>\]: 554
+failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 554 5\.7\.1 .*$
-            reject: RCPT from (.*)\[<HOST>\]: 450 4\.7\.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo= *$
+            ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 450 4\.7\.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo= *$
+            ^%(__prefix_line)sNOQUEUE: reject: VRFY from \S+\[<HOST>\]: 550 5\.1\.1 .*$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/proftpd.conf

@@ -13,6 +13,8 @@ before = common.conf
 
 [Definition]
 
+_deamon = proftpd
+
 # Option: failregex
 # Notes.: regex to match the password failures messages in the logfile. The
 #          host must be matched by a group named "host". The tag "<HOST>" can
@@ -20,10 +22,12 @@ before = common.conf
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values: TEXT
 #
-failregex = ^ %(__hostname)s %(__daemon_re)s%(__pid_re)s %(__hostname)s \(\S+\[<HOST>\]\)[: -]+ USER .*: no such user found from \S+ \[\S+\] to \S+:\S+ *$
+
-            ^ %(__hostname)s %(__daemon_re)s%(__pid_re)s %(__hostname)s \(\S+\[<HOST>\]\)[: -]+ USER .* \(Login failed\): .*$
+__suffix_failed_login = (User not authorized for login|No such user found|Incorrect password|Password expired|Account disabled|Invalid shell: '\S+'|User in \S+|Limit (access|configuration) denies login|Not a UserAlias|maximum login length exceeded).?
-            ^ %(__hostname)s %(__daemon_re)s%(__pid_re)s %(__hostname)s \(\S+\[<HOST>\]\)[: -]+ SECURITY VIOLATION: .* login attempted\. *$
+failregex = ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ USER .*: no such user found from \S+ \[\S+\] to \S+:\S+ *$
-            ^ %(__hostname)s %(__daemon_re)s%(__pid_re)s %(__hostname)s \(\S+\[<HOST>\]\)[: -]+ Maximum login attempts \(\d+\) exceeded *$
+            ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ USER .* \(Login failed\): %(__suffix_failed_login)s\s*$
+            ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ SECURITY VIOLATION: .* login attempted\. *$
+            ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ Maximum login attempts \(\d+\) exceeded *$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/roundcube-auth.conf

@@ -17,7 +17,7 @@ before = common.conf
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = ^\s*(\[(\s\+[0-9]{4})?\])?(%(__hostname)s roundcube: IMAP Error)?: (FAILED login|Login failed) for .*? from <HOST>(\. AUTHENTICATE .*)?\s*$
+failregex = ^\s*(\[(\s[+-][0-9]{4})?\])?(%(__hostname)s roundcube: IMAP Error)?: (FAILED login|Login failed) for .*? from <HOST>(\. AUTHENTICATE .*)?\s*$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/sieve.conf

@@ -4,15 +4,24 @@
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+
 [Definition]
 
+_deamon = (?:cyrus/)?(?:tim)?sieved?
+
 # Option: failregex
 # Notes.: regex to match the password failures messages in the logfile. The
 #          host must be matched by a group named "host". The tag "<HOST>" can
 #          be used for standard IP/hostname matching.
 # Values: TEXT
 #
-failregex = : badlogin: .*\[<HOST>\] (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failure$
+failregex = ^%(__prefix_line)sbadlogin: \S+ ?\[<HOST>\] \S+ authentication failure$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/suhosin.conf

@@ -9,7 +9,8 @@
 # Notes.:  regex to match ALERTS as notified by lighttpd's FastCGI Module
 # Values:  TEXT
 #
-failregex = .*ALERT\ -\ .*attacker\ \'<HOST>\'
+# https://github.com/stefanesser/suhosin/blob/1fba865ab73cc98a3109f88d85eb82c1bfc29b37/log.c#L161
+failregex = ALERT - .* \(attacker '<HOST>', file '.*'(?:, line \d+)?\)$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/wuftpd.conf

@@ -4,14 +4,21 @@
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
 [Definition]
 
+_daemon = wu-ftpd
+
 # Option: failregex
 # Notes.: regex to match the password failures messages in the logfile.
 # Values: TEXT
 #
-failregex = wu-ftpd(?:\[\d+\])?:\s+\(pam_unix\)\s+authentication failure.* rhost=<HOST>$
+failregex = ^%(__prefix_line)sfailed login from \S+ \[<HOST>\]\s*$
-            wu-ftpd(?:\[\d+\])?: *failed login from .*\[<HOST>\] *$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/xinetd-fail.conf

@@ -4,8 +4,17 @@
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+
 [Definition]
 
+_daemon = xinetd
+
 # Option:  failregex
 # Notes.:  regex to match the password failures messages in the logfile. The
 #          host must be matched by a group named "host". The tag "<HOST>" can
@@ -19,8 +28,8 @@
 # load => xinetd: max_load (temporary problem)
 #
 
-failregex = xinetd(?:\[\d{1,5}\])?: FAIL: \S+ address from=<HOST>$
+failregex = ^%(__prefix_line)sFAIL: \S+ address from=<HOST>$
-            xinetd(?:\[\d{1,5}\])?: FAIL: \S+ libwrap from=<HOST>$
+            ^%(__prefix_line)sFAIL: \S+ libwrap from=<HOST>$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/jail.conf

@@ -185,13 +185,6 @@ maxretry  = 2
 
 # .. custom jails
 
-# ASSP SMTP Proxy Jail
-[assp]
-enabled  = false
-filter   = assp
-action = iptables-multiport[name=assp,port="25,465,587"]
-logpath  = /root/path/to/assp/logs/maillog.txt
-
 # Here we use TCP-Wrappers instead of Netfilter/Iptables. "ignoreregex" is
 # used to avoid banning the user "myuser".
 
@@ -239,7 +232,6 @@ filter   = sshd
 action   = ipfw[localhost=192.168.0.1]
            sendmail-whois[name="SSH,IPFW", dest=you@example.com]
 logpath  = /var/log/auth.log
-ignoreip = 168.192.0.1
 
 # bsd-ipfw is ipfw used by BSD. It uses ipfw tables.
 # table number must be unique.
@@ -248,10 +240,19 @@ ignoreip = 168.192.0.1
 # for the table doesn't ready exist.
 #
 [ssh-bsd-ipfw]
+
 filter   = sshd
 action   = bsd-ipfw[port=ssh,table=1]
 logpath  = /var/log/auth.log
 
+# PF is a BSD based firewall
+[ssh-pf]
+
+filter  = sshd
+action  = pf
+logpath = /var/log/sshd.log
+maxretry= 5
+
 #
 # HTTP servers
 #
@@ -293,19 +294,13 @@ maxretry = 2
 port    = http,https
 logpath = /var/www/*/logs/access_log
 
-# A simple PHP-fastcgi jail which works with lighttpd.
+[suhosin]
-# If you run a lighttpd server, then you probably will
-# find these kinds of messages in your error_log:
-#   ALERT – tried to register forbidden variable ‘GLOBALS’
-#   through GET variables (attacker '1.2.3.4', file '/var/www/default/htdocs/index.php')
-
-[lighttpd-fastcgi]
-
-port    = http,https
-logpath = /var/log/lighttpd/error.log
 
+filter  = suhosin
 # Same as above for mod_auth
 # It catches wrong authentifications
+logpath = /var/log/lighttpd/error.log
+
 
 [lighttpd-auth]
 
@@ -333,6 +328,11 @@ action   = hostsdeny
 logpath  = /var/log/apache*/*error.log
 maxretry = 6
 
+[3proxy]
+
+filter  = 3proxy
+port    = 3128
+logpath = /var/log/3proxy.log
 
 #
 # FTP servers
@@ -381,6 +381,12 @@ maxretry = 6
 # Mail servers
 #
 
+# ASSP SMTP Proxy Jail
+[assp]
+
+port     = smtp,ssmtp,submission
+logpath  = /root/path/to/assp/logs/maillog.txt
+
 [courier-smtp]
 
 port     = smtp,ssmtp,submission
@@ -402,6 +408,17 @@ action   = hostsdeny[file=/not/a/standard/path/hosts.deny]
 logpath  = /var/log/postfix.log
 bantime  = 300
 
+[exim]
+
+port   = smtp,ssmtp,submission
+logpath = /var/log/exim/mainlog
+
+[exim-spam]
+
+port   = smtp,ssmtp,submission
+logpath = /var/log/exim/mainlog
+
+
 #
 # Mail servers authenticators: might be used for smtp,ftp,imap servers, so
 # all relevant ports get banned
@@ -426,6 +443,11 @@ logpath  = /var/log/mail.log
 port    = smtp,ssmtp,submission,imap2,imap3,imaps,pop3,pop3s
 logpath = /var/log/mail.log
 
+[perdition]
+
+port   = imap2,imap3,imaps,pop3,pop3s
+logpath = /var/log/maillog
+
 #
 # DNS servers
 #
@@ -461,16 +483,13 @@ logpath = /var/log/mail.log
 # port     = domain,953
 # protocol = udp
 # logpath  = /var/log/named/security.log
-# ignoreip = 168.192.0.1
 
 # This jail blocks TCP traffic for DNS requests.
 
 [named-refused]
 
-filter   = named-refused
 port     = domain,953
 logpath  = /var/log/named/security.log
-ignoreip = 168.192.0.1
 
 #
 # Miscelaneous
@@ -512,38 +531,3 @@ action   = iptables-allports[name=recidive]
 bantime  = 604800  ; 1 week
 findtime = 86400   ; 1 day
 maxretry = 5
-
-# PF is a BSD based firewall
-[ssh-pf]
-
-enabled=false
-filter = sshd
-action = pf
-logpath  = /var/log/sshd.log
-maxretry=5
-
-[3proxy]
-
-enabled = false
-filter  = 3proxy
-action  = iptables[name=3proxy, port=3128, protocol=tcp]
-logpath = /var/log/3proxy.log
-
-[exim]
-enabled = false
-filter = exim
-action = iptables-multiport[name=exim,port="25,465,587"]
-logpath = /var/log/exim/mainlog
-
-[exim-spam]
-enabled = false
-filter = exim-spam
-action = iptables-multiport[name=exim-spam,port="25,465,587"]
-logpath = /var/log/exim/mainlog
-
-[perdition]
-enabled = false
-filter = perdition
-action = iptables-multiport[name=perdition,port="110,143,993,995"]
-logpath = /var/log/maillog
-

fail2ban/client/fail2banreader.py

@@ -39,7 +39,7 @@ class Fail2banReader(ConfigReader):
 		ConfigReader.read(self, "fail2ban")
 	
 	def getEarlyOptions(self):
-		opts = [["string", "socket", "/tmp/fail2ban.sock"],
+		opts = [["string", "socket", "/var/run/fail2ban/fail2ban.sock"],
 				["string", "pidfile", "/var/run/fail2ban/fail2ban.pid"]]
 		return ConfigReader.getOptions(self, "Definition", opts)
 	

fail2ban/tests/config/apache-auth/README

@@ -0,0 +1,13 @@
+
+Apache Auth.
+
+This directory contains the configuration file of Apache's Web Server to
+simulate authentication files.
+
+These assumed that /var/www/html is the web root and AllowOverides is "All".
+
+The subdirectories here are copied to the /var/www/html directory.
+
+Commands executed are in testcases/files/log/apache-auth with their
+corresponding failure mechanism.
+

fail2ban/tests/config/apache-auth/basic/authz_owner/.htaccess

@@ -0,0 +1,5 @@
+AuthType basic
+AuthName "private area"
+AuthBasicProvider file
+AuthUserFile /var/www/html/basic/authz_owner/.htpasswd
+Require file-owner

fail2ban/tests/config/apache-auth/basic/authz_owner/.htpasswd

@@ -0,0 +1 @@
+username:$apr1$1f5oQUl4$21lLXSN7xQOPtNsj5s4Nk/

fail2ban/tests/config/apache-auth/basic/file/.htaccess

@@ -0,0 +1,5 @@
+AuthType basic
+AuthName "private area"
+AuthBasicProvider file
+AuthUserFile /var/www/html/basic/file/.htpasswd
+Require valid-user

fail2ban/tests/config/apache-auth/basic/file/.htpasswd

@@ -0,0 +1 @@
+username:$apr1$uUMsOjCQ$.BzXClI/B/vZKddgIAJCR.

fail2ban/tests/config/apache-auth/digest.py

@@ -0,0 +1,159 @@
+#!/bin/env python
+import requests
+import md5
+
+
+def auth(v):
+
+    ha1 = md5.new(username + ':' + realm + ':' + password).hexdigest()
+    ha2 = md5.new("GET:" + url).hexdigest()
+    
+    #response = md5.new(ha1 + ':' + v['nonce'][1:-1] + ':' + v['nc'] + ':' + v['cnonce'][1:-1]
+    #                  + ':' + v['qop'][1:-1] + ':' + ha2).hexdigest()
+    
+    nonce = v['nonce'][1:-1]
+    nc=v.get('nc') or ''
+    cnonce = v.get('cnonce') or ''
+    opaque = v.get('opaque') or ''
+    qop = v['qop'][1:-1]
+    algorithm = v['algorithm']
+    response = md5.new(ha1 + ':' + nonce + ':' + nc + ':' + cnonce + ':' + qop + ':' + ha2).hexdigest()
+    
+    p = requests.Request('GET', host + url).prepare()
+    #p.headers['Authentication-Info'] = response 
+    p.headers['Authorization'] = """
+        Digest username="%s",
+        algorithm="%s",
+        realm="%s",
+        uri="%s",
+        nonce="%s",
+        cnonce="",
+        nc="",
+        qop=%s,
+        response="%s"
+    """ % ( username, algorithm, realm, url, nonce, qop, response )
+#        opaque="%s",
+    print p.method, p.url, p.headers
+    s =  requests.Session()
+    return s.send(p)
+
+def preauth():
+    r = requests.get(host + url)
+    print r
+    r.headers['www-authenticate'].split(', ')
+    return dict([ a.split('=',1) for a in r.headers['www-authenticate'].split(', ') ])
+
+
+url='/digest/'
+host = 'http://localhost:801'
+
+v = preauth()
+
+username="username"
+password = "password"
+print v
+
+realm = 'so far away'
+r = auth(v)
+
+realm = v['Digest realm'][1:-1]
+
+# [Sun Jul 28 21:27:56.549667 2013] [auth_digest:error] [pid 24835:tid 139895297222400] [client 127.0.0.1:57052] AH01788: realm mismatch - got `so far away' but expected `digest private area'
+
+
+algorithm = v['algorithm']
+v['algorithm'] = 'super funky chicken'
+r = auth(v)
+
+# [Sun Jul 28 21:41:20 2013] [error] [client 127.0.0.1] Digest: unknown algorithm `super funky chicken' received: /digest/
+
+print r.status_code,r.headers, r.text
+v['algorithm'] = algorithm
+
+
+r = auth(v)
+print r.status_code,r.headers, r.text
+
+nonce = v['nonce']
+v['nonce']=v['nonce'][5:-5]
+
+r = auth(v)
+print r.status_code,r.headers, r.text
+
+# [Sun Jul 28 21:05:31.178340 2013] [auth_digest:error] [pid 24224:tid 139895539455744] [client 127.0.0.1:56906] AH01793: invalid qop `auth' received: /digest/qop_none/
+
+
+v['nonce']=nonce[0:11] + 'ZZZ' + nonce[14:]
+
+r = auth(v)
+print r.status_code,r.headers, r.text
+
+#[Sun Jul 28 21:18:11.769228 2013] [auth_digest:error] [pid 24752:tid 139895505884928] [client 127.0.0.1:56964] AH01776: invalid nonce b9YAiJDiBAZZZ1b1abe02d20063ea3b16b544ea1b0d981c1bafe received - hash is not d42d824dee7aaf50c3ba0a7c6290bd453e3dd35b
+
+
+url='/digest_time/'
+v=preauth()
+
+import time
+time.sleep(1)
+
+r = auth(v)
+print r.status_code,r.headers, r.text
+
+# Obtained by putting the following code in modules/aaa/mod_auth_digest.c
+# in the function initialize_secret
+#    {
+#       const char *hex = "0123456789abcdef";
+#       char secbuff[SECRET_LEN * 4];
+#       char *hash = secbuff;
+#       int idx;
+
+#       for (idx=0; idx<sizeof(secret); idx++) {
+#       *hash++ = hex[secret[idx] >> 4];
+#       *hash++ = hex[secret[idx] & 0xF];
+#       }
+#       *hash = '\0';
+#       /* remove comment makings in below for apache-2.4+ */
+#       ap_log_error(APLOG_MARK, APLOG_NOTICE, 0, s, /*  APLOGNO(11759) */ "secret: %s", secbuff);
+#   }
+
+
+import sha
+import binascii
+import base64
+import struct
+
+apachesecret = binascii.unhexlify('497d8894adafa5ec7c8c981ddf9c8457da7a90ac')
+s = sha.sha(apachesecret)
+
+v=preauth()
+
+print v['nonce']
+realm = v['Digest realm'][1:-1]
+
+(t,) = struct.unpack('l',base64.b64decode(v['nonce'][1:13]))
+
+# whee, time travel
+t = t + 5540
+
+timepac = base64.b64encode(struct.pack('l',t))
+
+s.update(realm)
+s.update(timepac)
+
+v['nonce'] =  v['nonce'][0] + timepac + s.hexdigest() + v['nonce'][-1]
+
+print v
+
+r = auth(v)
+#[Mon Jul 29 02:12:55.539813 2013] [auth_digest:error] [pid 9647:tid 139895522670336] [client 127.0.0.1:58474] AH01777: invalid nonce 59QJppTiBAA=b08983fd166ade9840407df1b0f75b9e6e07d88d received - user attempted time travel
+print r.status_code,r.headers, r.text
+
+url='/digest_onetime/'
+v=preauth()
+
+# Need opaque header handling in auth
+r = auth(v)
+print r.status_code,r.headers, r.text
+r = auth(v)
+print r.status_code,r.headers, r.text

fail2ban/tests/config/apache-auth/digest/.htaccess

@@ -0,0 +1,6 @@
+AuthType Digest
+AuthName "digest private area"
+AuthDigestDomain /digest/
+AuthBasicProvider file
+AuthUserFile /var/www/html/digest/.htpasswd
+Require valid-user

fail2ban/tests/config/apache-auth/digest/.htpasswd

@@ -0,0 +1 @@
+username:digest private area:fad48d3a7c63f61b5b3567a4105bbb04

fail2ban/tests/config/apache-auth/digest_anon/.htaccess

@@ -0,0 +1,9 @@
+AuthType Digest
+AuthName "digest anon"
+AuthDigestDomain /digest_anon/
+AuthBasicProvider file anon
+AuthUserFile /var/www/html/digest_anon/.htpasswd
+Anonymous_NoUserID off
+Anonymous anonymous
+Anonymous_LogEmail on
+Require valid-user

fail2ban/tests/config/apache-auth/digest_anon/.htpasswd

@@ -0,0 +1,3 @@
+username:digest anon:25e4077a9344ceb1a88f2a62c9fb60d8
+05bbb04
+anonymous:digest anon:faa4e5870970cf935bb9674776e6b26a

fail2ban/tests/config/apache-auth/digest_time/.htaccess

@@ -0,0 +1,7 @@
+AuthType Digest
+AuthName "digest private area"
+AuthDigestDomain /digest_time/
+AuthBasicProvider file
+AuthUserFile /var/www/html/digest_time/.htpasswd
+AuthDigestNonceLifetime 1
+Require valid-user

fail2ban/tests/config/apache-auth/digest_time/.htpasswd

@@ -0,0 +1 @@
+username:digest private area:fad48d3a7c63f61b5b3567a4105bbb04

fail2ban/tests/config/apache-auth/digest_wrongrelm/.htaccess

@@ -0,0 +1,6 @@
+AuthType Digest
+AuthName "digest private area"
+AuthDigestDomain /digest_wrongrelm/
+AuthBasicProvider file
+AuthUserFile /var/www/html/digest_wrongrelm/.htpasswd
+Require valid-user

fail2ban/tests/config/apache-auth/digest_wrongrelm/.htpasswd

@@ -0,0 +1,2 @@
+username:wrongrelm:99cd340e1283c6d0ab34734bd47bdc30
+4105bbb04

fail2ban/tests/config/apache-auth/noentry/.htaccess

@@ -0,0 +1 @@
+Deny from all

fail2ban/tests/datedetectortestcase.py

@@ -92,8 +92,10 @@ class DateDetectorTest(unittest.TestCase):
 			# exclude
 
 			# yoh: on [:6] see in above test
-			self.assertEqual(self.__datedetector.getTime(log)[:6], date[:6])
+			logtime = self.__datedetector.getTime(log)
-			self.assertEqual(self.__datedetector.getUnixTime(log), dateUnix)
+			self.assertNotEqual(logtime, None, "getTime retrieved nothing: failure for %s" % sdate)
+			self.assertEqual(logtime[:6], date[:6], "getTime comparison failure for %s: \"%s\" is not \"%s\"" % (sdate, logtime[:6], date[:6]))
+			self.assertEqual(self.__datedetector.getUnixTime(log), dateUnix, "getUnixTime failure for %s: \"%s\" is not \"%s\"" % (sdate, logtime[:6], date[:6]))
 
 	def testStableSortTemplate(self):
 		old_names = [x.getName() for x in self.__datedetector.getTemplates()]

fail2ban/tests/files/logs/apache-auth

@@ -3,5 +3,114 @@
 [Sat Jun 01 02:17:42 2013] [error] [client 192.168.33.1] File does not exist: /srv/http/site/[client 192.168.0.1] user root not found
 
 # should match
+# failJSON: { "time": "2013-07-11T01:21:41", "match": true , "host": "194.228.20.113" } 
+[Thu Jul 11 01:21:41 2013] [error] [client 194.228.20.113] user  not found: /
+
+# failJSON: { "time": "2013-07-11T01:21:43", "match": true , "host": "194.228.20.113" } 
+[Thu Jul 11 01:21:43 2013] [error] [client 194.228.20.113] user dsfasdf not found: /
+
+# The failures below use the configuration described in fail2ban/tests/files/config/apache-auth
+#
+
+# wget  http://localhost/noentry/cant_get_me.html -O /dev/null
+# failJSON: { "time": "2013-07-17T23:20:45", "match": true , "host": "127.0.0.1" } 
+[Wed Jul 17 23:20:45 2013] [error] [client 127.0.0.1] client denied by server configuration: /var/www/html/noentry/cant_get_me.html
+
+# failJSON: { "time": "2013-07-20T21:34:49", "match": true , "host": "127.0.0.1" } 
+[Sat Jul 20 21:34:49.453232 2013] [access_compat:error] [pid 17512:tid 140123104306944] [client 127.0.0.1:51380] AH01797: client denied by server configuration: /var/www/html/noentry/cant_get_me.html
+
+# wget --http-user='' --http-password='' http://localhost/basic/file/cant_get_me.html -O /dev/null
+# failJSON: { "time": "2013-07-17T23:14:37", "match": true , "host": "127.0.0.1" }
+[Wed Jul 17 23:14:37 2013] [error] [client 127.0.0.1] user  not found: /basic/anon/cant_get_me.html
+
+# failJSON: { "time": "2013-07-20T21:37:32", "match": true , "host": "127.0.0.1" } 
+[Sat Jul 20 21:37:32.266605 2013] [auth_basic:error] [pid 17512:tid 140123079128832] [client 127.0.0.1:51386] AH01618: user  not found: /basic/file/cant_get_me.html
+
+# wget --http-user=username --http-password=wrongpass http://localhost/basic/file -O /dev/null
+# failJSON: { "time": "2013-07-17T22:18:52", "match": true , "host": "127.0.0.1" }
+[Wed Jul 17 22:18:52 2013] [error] [client 127.0.0.1] user username: authentication failure for "/basic/file": Password Mismatch
+
+# failJSON: { "time": "2013-07-20T21:39:11", "match": true , "host": "127.0.0.1" } 
+[Sat Jul 20 21:39:11.978080 2013] [auth_basic:error] [pid 17512:tid 140123053950720] [client 127.0.0.1:51390] AH01617: user username: authentication failure for "/basic/file": Password Mismatch
+
+# wget --http-user=wrongusername --http-password=wrongpass http://localhost/basic/file -O /dev/null
+# failJSON: { "time": "2013-07-17T22:32:48", "match": true , "host": "127.0.0.1" }
+[Wed Jul 17 22:32:48 2013] [error] [client 127.0.0.1] user wrongusername not found: /basic/file
+
+# failJSON: { "time": "2013-07-20T21:40:33", "match": true , "host": "127.0.0.1" } 
+[Sat Jul 20 21:40:33.803528 2013] [auth_basic:error] [pid 17540:tid 140123095914240] [client 127.0.0.1:51395] AH01618: user wrongusername not found: /basic/file
+
+# wget --header='Authorization: Digest username="Mufasa",realm="testrealm@host.com",nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093",uri="/dir/index.html",qop=auth,nc=00000001,cnonce="0a4f113b",response="6629fae49393a05397450978507c4ef1",opaque="5ccc069c403ebaf9f0171e9517f40e41"'  http://localhost/basic/file -O /dev/null
+# failJSON: { "time": "2013-07-17T22:39:55", "match": true , "host": "127.0.0.1" }
+[Wed Jul 17 22:39:55 2013] [error] [client 127.0.0.1] client used wrong authentication scheme: /basic/file
+
+# failJSON: { "time": "2013-07-20T21:41:52", "match": true , "host": "127.0.0.1" } 
+[Sat Jul 20 21:41:52.523931 2013] [auth_basic:error] [pid 17512:tid 140122964092672] [client 127.0.0.1:51396] AH01614: client used wrong authentication scheme: /basic/file
+
+# wget --http-user=username --http-password=password http://localhost/basic/authz_owner/cant_get_me.html -O /dev/null
+# failJSON: { "time": "2013-07-17T22:54:32", "match": true , "host": "127.0.0.1" }
+[Wed Jul 17 22:54:32 2013] [error] [client 127.0.0.1] Authorization of user username to access /basic/authz_owner/cant_get_me.html failed, reason: file owner dan does not match.
+
+# failJSON: { "time": "2013-07-20T22:11:43", "match": true , "host": "127.0.0.1" } 
+[Sat Jul 20 22:11:43.147674 2013] [authz_owner:error] [pid 17540:tid 140122922129152] [client 127.0.0.1:51548] AH01637: Authorization of user username to access /basic/authz_owner/cant_get_me.html failed, reason: file owner dan does not match
+
+# wget --http-user=username --http-password=password http://localhost/basic/authz_owner/cant_get_me.html -O /dev/null
+# failJSON: { "time": "2013-07-20T21:42:44", "match": true , "host": "127.0.0.1" } 
+[Sat Jul 20 21:42:44.304159 2013] [authz_core:error] [pid 17484:tid 140123095914240] [client 127.0.0.1:51397] AH01631: user username: authorization failure for "/basic/authz_owner/cant_get_me.html": 
+
+# wget --http-user='username' --http-password='wrongpassword' http://localhost/digest/cant_get_me.html -O /dev/null
+# failJSON: { "time": "2013-07-17T23:50:37", "match": true , "host": "127.0.0.1" }
+[Wed Jul 17 23:50:37 2013] [error] [client 127.0.0.1] Digest: user username: password mismatch: /digest/cant_get_me.html
+
+# failJSON: { "time": "2013-07-20T21:44:06", "match": true , "host": "127.0.0.1" } 
+[Sat Jul 20 21:44:06.867985 2013] [auth_digest:error] [pid 17540:tid 140123070736128] [client 127.0.0.1:51406] AH01792: user username: password mismatch: /digest/cant_get_me.html
+
+# wget --http-user='username' --http-password='password' http://localhost/digest_wrongrelm/cant_get_me.html -O /dev/null
+# failJSON: { "time": "2013-07-18T00:08:39", "match": true , "host": "127.0.0.1" }
+[Thu Jul 18 00:08:39 2013] [error] [client 127.0.0.1] Digest: user `username' in realm `digest private area' not found: /digest_wrongrelm/cant_get_me.html
+
+# failJSON: { "time": "2013-07-20T21:45:28", "match": true , "host": "127.0.0.1" } 
+[Sat Jul 20 21:45:28.890523 2013] [auth_digest:error] [pid 17540:tid 140122972485376] [client 127.0.0.1:51408] AH01790: user `username' in realm `digest private area' not found: /digest_wrongrelm/cant_get_me.html
+
+# ./tests/files/config/apache-auth/digest.py
+# failJSON: { "time": "2013-07-28T21:05:31", "match": true , "host": "127.0.0.1" }
+[Sun Jul 28 21:05:31.178340 2013] [auth_digest:error] [pid 24224:tid 139895539455744] [client 127.0.0.1:56906] AH01793: invalid qop `auth' received: /digest/qop_none/
+
+# ./tests/files/config/apache-auth/digest.py
+# failJSON: { "time": "2013-07-28T21:12:44", "match": true , "host": "127.0.0.1" }
+[Sun Jul 28 21:12:44 2013] [error] [client 127.0.0.1] Digest: invalid nonce JDiBAA=db9372522295196b7ac31db99e10cd1106c received - length is not 52
+
+
+# ./tests/files/config/apache-auth/digest.py
+# yoh: ATM it should not match because matching failregex is still under "investigation"
+# failJSON: { "time": "2013-07-28T21:16:37", "match": false , "host": "127.0.0.1" }
+[Sun Jul 28 21:16:37 2013] [error] [client 127.0.0.1] Digest: invalid nonce l19lgpDiBAZZZf1ec3d9613f3b3ef43660e3628d78455fd8b937 received - hash is not 6fda8bbcbcf85ff1ebfe7d1c43faba583bc53a02
+
+# ./tests/files/config/apache-auth/digest.py
+# failJSON: { "time": "2013-07-28T21:18:11", "match": false , "host": "127.0.0.1" }
+[Sun Jul 28 21:18:11.769228 2013] [auth_digest:error] [pid 24752:tid 139895505884928] [client 127.0.0.1:56964] AH01776: invalid nonce b9YAiJDiBAZZZ1b1abe02d20063ea3b16b544ea1b0d981c1bafe received - hash is not d42d824dee7aaf50c3ba0a7c6290bd453e3dd35b
+
+# ./tests/files/config/apache-auth/digest.py
+# failJSON: { "time": "2013-07-28T21:30:02", "match": true , "host": "127.0.0.1" }
+[Sun Jul 28 21:30:02 2013] [error] [client 127.0.0.1] Digest: realm mismatch - got `so far away' but expected `digest private area'
+
+# failJSON: { "time": "2013-07-28T21:27:56", "match": true , "host": "127.0.0.1" }
+[Sun Jul 28 21:27:56.549667 2013] [auth_digest:error] [pid 24835:tid 139895297222400] [client 127.0.0.1:57052] AH01788: realm mismatch - got `so far away' but expected `digest private area'
+
+
+# ./tests/files/config/apache-auth/digest.py
+# failJSON: { "time": "2013-07-28T21:41:20", "match": true , "host": "127.0.0.1" }
+[Sun Jul 28 21:41:20 2013] [error] [client 127.0.0.1] Digest: unknown algorithm `super funky chicken' received: /digest/
+
+# failJSON: { "time": "2013-07-28T21:42:03", "match": true , "host": "127.0.0.1" }
+[Sun Jul 28 21:42:03.930190 2013] [auth_digest:error] [pid 24835:tid 139895505884928] [client 127.0.0.1:57115] AH01789: unknown algorithm `super funky chicken' received: /digest/
+
+# ./tests/files/config/apache-auth/digest.py
+# failJSON: { "time": "2013-07-29T02:15:26", "match": true , "host": "127.0.0.1" }
+[Mon Jul 29 02:15:26 2013] [error] [client 127.0.0.1] Digest: invalid nonce LWEDr5TiBAA=ceddd011628c30e3646f7acda4f1a0ab6b7c5ae6 received - user attempted time travel
+
+# failJSON: { "time": "2013-07-29T02:12:55", "match": true , "host": "127.0.0.1" }
+[Mon Jul 29 02:12:55.539813 2013] [auth_digest:error] [pid 9647:tid 139895522670336] [client 127.0.0.1:58474] AH01777: invalid nonce 59QJppTiBAA=b08983fd166ade9840407df1b0f75b9e6e07d88d received - user attempted time travel
+
 # failJSON: { "time": "2013-06-01T02:17:42", "match": true , "host": "192.168.0.2" }
 [Sat Jun 01 02:17:42 2013] [error] [client 192.168.0.2] user root not found

fail2ban/tests/files/logs/apache-badbots

@@ -0,0 +1,2 @@
+# failJSON: { "time": "2007-03-05T14:39:21", "match": true , "host": "1.2.3.4" }
+1.2.3.4 - - [05/Mar/2007:14:39:21 +0100] "POST /123.html/trackback/ HTTP/1.0" 301 459 "http://www.mydomain.tld/123.html/trackback" "TrackBack/1.02"

fail2ban/tests/files/logs/apache-nohome

@@ -2,5 +2,5 @@
 # failJSON: { "time": "2013-06-01T11:23:08", "match": true , "host": "1.2.3.4" }
 [Sat Jun 01 11:23:08 2013] [error] [client 1.2.3.4] File does not exist: /xxx/~
 # Apache 2.4
-# failJSON: { "time": "2005-06-27T11:55:44", "match": true , "host": "192.0.2.12" }
+# failJSON: { "time": "2013-06-27T11:55:44", "match": true , "host": "192.0.2.12" }
 [Thu Jun 27 11:55:44.569531 2013] [core:info] [pid 4101:tid 2992634688] [client 192.0.2.12:46652] AH00128: File does not exist: /xxx/~

fail2ban/tests/files/logs/apache-noscript

@@ -1,2 +1,4 @@
 # failJSON: { "time": "2013-06-09T07:57:47", "match": true , "host": "192.0.43.10" }
 [Sun Jun 09 07:57:47 2013] [error] [client 192.0.43.10] script '/usr/lib/cgi-bin/gitweb.cgiwp-login.php' not found or unable to stat
+# failJSON: { "time": "2008-07-22T06:48:30", "match": true , "host": "198.51.100.86" }
+[Tue Jul 22 06:48:30 2008] [error] [client 198.51.100.86] File does not exist: /home/southern/public_html/azenv.php

fail2ban/tests/files/logs/asterisk

@@ -1,4 +1,6 @@
 # Sample log files for asterisk 
+# failJSON: { "time": "2013-07-25T07:26:43", "match": true , "host": "1.2.3.4" }
+[2013-07-25 07:26:43] NOTICE[26015][C-000006b2] chan_sip.c: Failed to authenticate device 101<sip:101@1.2.3.4>;tag=deadbeef
 # failJSON: { "time": "2012-02-13T17:21:54", "match": true , "host": "1.2.3.4" }
 [2012-02-13 17:21:54] NOTICE[1638] chan_sip.c: Registration from '<sip:301@example.com>' failed for '1.2.3.4' - Wrong password
 # failJSON: { "time": "2012-02-13T17:18:22", "match": true , "host": "1.2.3.4" }

fail2ban/tests/files/logs/courier-auth

@@ -0,0 +1,8 @@
+# failJSON: { "time": "2005-04-23T21:59:01", "match": true , "host": "1.2.3.4" }
+Apr 23 21:59:01 dns2 imapd: LOGIN FAILED, user=sales@example.com, ip=[::ffff:1.2.3.4]
+# failJSON: { "time": "2005-04-23T21:59:38", "match": true , "host": "198.51.100.76" }
+Apr 23 21:59:38 dns2 pop3d: LOGIN FAILED, user=info@example.com, ip=[::ffff:198.51.100.76]
+# failJSON: { "time": "2004-11-13T08:11:53", "match": true , "host": "198.51.100.33" }
+Nov 13 08:11:53 server imapd-ssl: LOGIN FAILED, user=user@domain.tld, ip=[::ffff:198.51.100.33]
+# failJSON: { "time": "2005-04-17T19:17:11", "match": true , "host": "1.2.3.4" }
+Apr 17 19:17:11 SERVER courierpop3login: LOGIN FAILED, user=USER@EXAMPLE.org, ip=[::ffff:1.2.3.4]

fail2ban/tests/files/logs/courier-smtp

@@ -0,0 +1,8 @@
+# failJSON: { "time": "2005-04-10T03:47:57", "match": true , "host": "1.2.3.4" }
+Apr 10 03:47:57 web courieresmtpd: error,relay=::ffff:1.2.3.4,ident=tmf,from=<tmf@example.com>,to=<mailman-subscribe@example.com>: 550 User unknown.
+# failJSON: { "time": "2005-07-06T03:42:28", "match": true , "host": "1.2.3.4" }
+Jul  6 03:42:28 whistler courieresmtpd: error,relay=::ffff:1.2.3.4,from=<>,to=<admin at memcpy>: 550 User unknown.
+# failJSON: { "time": "2004-11-21T23:16:17", "match": true , "host": "1.2.3.4" }
+Nov 21 23:16:17 server courieresmtpd: error,relay=::ffff:1.2.3.4,from=<>,to=<>: 550 User unknown.
+# failJSON: { "time": "2004-08-14T12:51:04", "match": true , "host": "1.2.3.4" }
+Aug 14 12:51:04 HOSTNAME courieresmtpd: error,relay=::ffff:1.2.3.4,from=<firozquarl@aclunc.org>,to=<BOGUSUSER@HOSTEDDOMAIN.org>: 550 User unknown.

fail2ban/tests/files/logs/cyrus-imap

@@ -0,0 +1,13 @@
+# failJSON: { "time": "2005-01-04T21:51:05", "match": true , "host": "127.0.0.1" }
+Jan 4 21:51:05 hostname cyrus/imap[5355]: badlogin: localhost.localdomain [127.0.0.1] plaintext cyrus@localdomain SASL(-13): authentication failure: checkpass failed
+# failJSON: { "time": "2005-02-20T17:23:32", "match": true , "host": "198.51.100.23" }
+Feb 20 17:23:32 domain cyrus/pop3[18635]: badlogin: localhost [198.51.100.23] plaintext administrator SASL(-13): authentication failure: checkpass failed
+# failJSON: { "time": "2005-02-20T17:23:32", "match": true , "host": "1.2.3.4" }
+Feb 20 17:23:32 cyrus/pop3[4297]: badlogin: example.com [1.2.3.4] plaintext mail0001 SASL(-13): authentication failure: checkpass failed
+# failJSON: { "time": "2005-06-08T18:11:13", "match": true , "host": "198.51.100.45" }
+Jun 8 18:11:13 lampserver imap[4480]: badlogin: example.com [198.51.100.45] DIGEST-MD5 [SASL(-13): authentication failure: client response doesn't match what we generated]
+# failJSON: { "time": "2004-12-21T10:01:57", "match": true , "host": "198.51.100.57" }
+Dec 21 10:01:57 hostname imapd[18454]: badlogin: example.com [198.51.100.57] CRAM-MD5 [SASL(-13): authentication failure: incorrect digest response]
+# failJSON: { "time": "2004-12-30T16:03:27", "match": true , "host": "1.2.3.4" }
+Dec 30 16:03:27 somehost imapd[2517]: badlogin: local-somehost[1.2.3.4] OTP [SASL(-13): authentication failure: External SSF not good enough]
+

fail2ban/tests/files/logs/dropbear

@@ -0,0 +1,15 @@
+# failJSON: { "time": "2005-03-24T15:25:51", "match": true , "host": "198.51.100.87" }
+Mar 24 15:25:51 buffalo1 dropbear[4092]: bad password attempt for 'root' from 198.51.100.87:5543
+# failJSON: { "time": "2005-02-11T15:23:17", "match": true , "host": "198.51.100.215" }
+Feb 11 15:23:17 dropbear[1252]: login attempt for nonexistent user from ::ffff:198.51.100.215:60495
+# failJSON: { "time": "2005-03-24T15:25:51", "match": true , "host": "198.51.100.87" }
+Mar 24 15:25:51 buffalo1 dropbear[4092]: bad password attempt for 'root' from 198.51.100.87:5543
+# failJSON: { "time": "2005-02-11T15:23:17", "match": true , "host": "198.51.100.215" }
+Feb 11 15:23:17 dropbear[1252]: login attempt for nonexistent user from ::ffff:198.51.100.215:60495
+
+# failJSON: { "time": "2005-07-27T01:04:12", "match": true , "host": "1.2.3.4" }
+Jul 27 01:04:12 fail2ban-test dropbear[1335]: Bad password attempt for 'root' from 1.2.3.4:60588
+# failJSON: { "time": "2005-07-27T01:04:22", "match": true , "host": "1.2.3.4" }
+Jul 27 01:04:22 fail2ban-test dropbear[1335]: Exit before auth (user 'root', 10 fails): Max auth tries reached - user 'root' from 1.2.3.4:60588
+# failJSON: { "time": "2005-07-27T01:18:59", "match": true , "host": "1.2.3.4" }
+Jul 27 01:18:59 fail2ban-test dropbear[1477]: Login attempt for nonexistent user from 1.2.3.4:60794

fail2ban/tests/files/logs/exim

@@ -29,3 +29,11 @@
 # failJSON: { "time": "2013-06-15T16:36:49", "match": true , "host": "111.67.203.116" }
 2013-06-15 16:36:49 H=altmx.marsukov.com [111.67.203.116] F=<kadrofutcheti@mail.ru> rejected RCPT <oksana@birzhatm.ua>: Unknown user
 
+# 'https://github.com/fail2ban/fail2ban/pull/251#issuecomment-23001227' 
+# failJSON: { "time": "2013-08-20T07:48:02", "match": true , "host": "85.25.92.177" }
+2013-08-20 07:48:02 login authenticator failed for static-ip-85-25-92-177.inaddr.ip-pool.com (USER) [85.25.92.177]: 535 Incorrect authentication data: 1 Time(s)
+# failJSON: { "time": "2013-08-20T23:30:05", "match": true , "host": "91.218.72.71" }
+2013-08-20 23:30:05 plain authenticator failed for ([192.168.2.102]) [91.218.72.71]: 535 Incorrect authentication data: 1 Time(s)
+
+# failJSON: { "time": "2013-09-02T09:19:07", "match": true , "host": "118.233.20.68" }
+2013-09-02 09:19:07 login authenticator failed for (gkzwsoju) [118.233.20.68]: 535 Incorrect authentication data

fail2ban/tests/files/logs/gssftpd

@@ -0,0 +1,2 @@
+# failJSON: { "time": "2005-01-22T18:09:46", "match": true , "host": "198.51.100.23" }
+Jan 22 18:09:46 host ftpd[132]: repeated login failures from 198.51.100.23 (example.com)

fail2ban/tests/files/logs/lighttpd-auth

@@ -3,3 +3,5 @@
 2011-12-25 17:09:20: (http_auth.c.875) password doesn't match for /gitweb/ username: francois, IP: 4.4.4.4
 # failJSON: { "time": "2012-09-26T10:24:35", "match": true , "host": "4.4.4.4" }
 2012-09-26 10:24:35: (http_auth.c.1136) digest: auth failed for  xxx : wrong password, IP: 4.4.4.4
+# failJSON: { "time": "2013-08-25T00:24:55", "match": true , "host": "4.4.4.4" }
+2013-08-25 00:24:55: (http_auth.c.877) get_password failed, IP: 4.4.4.4

fail2ban/tests/files/logs/named-refused

@@ -10,3 +10,16 @@ Jul 24 14:20:25 raid5 named[3935]: client 148.160.29.6#33081: query (cache) 'shi
 Jul 24 14:23:36 raid5 named[3935]: client 148.160.29.6#33081: query (cache) 'mietberatung.de/NS/IN' denied
 # failJSON: { "time": "2005-07-24T14:23:36", "match": true , "host": "62.109.4.89" }
 Jul 24 14:23:36 raid5 named[3935]: client 62.109.4.89#9334: view external: query (cache) './NS/IN' denied
+# failJSON: { "time": "2013-08-11T03:36:11", "match": true , "host": "1.2.3.4" }
+11-Aug-2013 03:36:11.372 error: client 1.2.3.4#52115: zone transfer 'domain.com/AXFR/IN' denied
+# failJSON: { "time": "2004-08-17T08:20:22", "match": true , "host": "223.252.23.219" }
+Aug 17 08:20:22 catinthehat named[2954]: client 223.252.23.219#56275: zone transfer 'openquery.eu/AXFR/IN' denied
+# https://github.com/fail2ban/fail2ban/issues/333
+# BIND9 ver: BIND 9.9.3-rpz2+rl.13208.13-P2-RedHat-9.9.3-4.P2.el6 (Extended Support Version)
+# failJSON: { "time": "2013-08-23T10:32:56", "match": true , "host": "82.207.95.42" }
+23-Aug-2013 10:32:56.621 client 82.207.95.42#40278 (redginseng.com.ua): query (cache) 'redginseng.com.ua/A/IN' denied
+
+# failJSON: { "time": "2013-08-27T17:49:45", "match": true , "host": "59.167.242.100" }
+27-Aug-2013 17:49:45.330 client 59.167.242.100#44281 (watt.kiev.ua): zone transfer 'watt.kiev.ua/AXFR/IN' denied
+# failJSON: { "time": "2004-08-27T16:58:31", "match": true , "host": "176.9.92.38" }
+Aug 27 16:58:31 vhost1-ua named[29206]: client 176.9.92.38#42592 (simmarket.com.ua): bad zone transfer request: 'simmarket.com.ua/IN': non-authoritative zone (NOTAUTH)

fail2ban/tests/files/logs/php-url-fopen

@@ -0,0 +1,2 @@
+# failJSON: { "time": "2009-03-26T08:44:20", "match": true , "host": "66.185.212.172" }
+66.185.212.172 - - [26/Mar/2009:08:44:20 -0500] "GET /index.php?n=http://eatmyfood.hostinginfive.com/pizza.htm? HTTP/1.1" 200 114 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"

fail2ban/tests/files/logs/postfix

@@ -4,3 +4,9 @@
 Feb 21 09:21:54 xxx postfix/smtpd[14398]: NOQUEUE: reject: RCPT from example.com[192.0.43.10]: 450 4.7.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo=
 # failJSON: { "time": "2005-07-12T07:47:48", "match": true , "host": "1.2.3.4" }
 Jul 12 07:47:48 saturn postfix/smtpd[8738]: NOQUEUE: reject: RCPT from 1-2-3-4-example.com[1.2.3.4]: 554 5.7.1 <smtp@example.com>: Relay access denied; from=<john@example.com> to=<smtp@example.org> proto=SMTP helo=<198.51.100.17>
+# failJSON: { "time": "2005-07-18T23:12:56", "match": true , "host": "192.51.100.65" }
+Jul 18 23:12:56 xxx postfix/smtpd[8738]: NOQUEUE: reject: RCPT from foo[192.51.100.65]: 554 5.7.1 <bad.domain>: Helo command rejected: match bad.domain; from=<foo@good.domain> to=<foo@porcupine.org> proto=SMTP helo=<bad.domain>
+# failJSON: { "time": "2005-07-18T23:12:56", "match": true , "host": "192.51.100.43" }
+Jul 18 23:12:56 xxx postfix/smtpd[8738]: NOQUEUE: reject: RCPT from foo[192.51.100.43]: 554 5.7.1 <foo@bad.domain>: Sender address rejected: match bad.domain; from=<foo@bad.domain> to=<foo@porcupine.org> proto=SMTP helo=<192.51.100.43>
+# failJSON: { "time": "2005-08-10T10:55:38", "match": true , "host": "72.53.132.234" }
+Aug 10 10:55:38 f-vanier-bourgeois postfix/smtpd[2162]: NOQUEUE: reject: VRFY from 72-53-132-234.cpe.distributel.net[72.53.132.234]: 550 5.1.1 : Recipient address rejected: User unknown in local recipient tab

fail2ban/tests/files/logs/proftpd

@@ -10,6 +10,7 @@ Jun 09 11:15:43 platypus.ace-hosting.com.au proftpd[17424] platypus.ace-hosting.
 Jun 13 22:07:23 platypus.ace-hosting.com.au proftpd[15719] platypus.ace-hosting.com.au (::ffff:59.167.242.100[::ffff:59.167.242.100]): SECURITY VIOLATION: root login attempted.
 # failJSON: { "time": "2005-06-14T00:09:59", "match": true , "host": "59.167.242.100" }
 Jun 14 00:09:59 platypus.ace-hosting.com.au proftpd[17839] platypus.ace-hosting.com.au (::ffff:59.167.242.100[::ffff:59.167.242.100]): USER platypus.ace-hosting.com.au proftpd[17424] platypus.ace-hosting.com.au (hihoinjection[1.2.3.44]): no such user found from ::ffff:59.167.242.100 [::ffff:59.167.242.100] to ::ffff:113.212.99.194:21
-
+# failJSON: { "time": "2005-05-31T10:53:25", "match": true , "host": "1.2.3.4" }
-
+May 31 10:53:25 mail proftpd[15302]: xxxxxxxxxx (::ffff:1.2.3.4[::ffff:1.2.3.4]) - Maximum login attempts (3) exceeded 
-
+# failJSON: { "time": "2004-12-05T15:44:32", "match": true , "host": "1.2.3.4" }
+Dec 5 15:44:32 serv1 proftpd[70944]: serv1.domain.com (example.com[1.2.3.4]) - USER jtittle@domain.org: no such user found from example.com [1.2.3.4] to 1.2.3.4:21 

fail2ban/tests/files/logs/qmail

@@ -0,0 +1,4 @@
+# failJSON: { "time": "2004-09-06T07:33:33", "match": true , "host": "198.51.100.77" }
+Sep  6 07:33:33 sd6 qmail: 1157520813.485077 rblsmtpd: 198.51.100.77 pid 19597 sbl-xbl.spamhaus.org: 451 http://www.spamhaus.org/query/bl?ip=198.51.100.77
+# failJSON: { "time": "2004-09-06T07:18:29", "match": true , "host": "198.51.100.54" }
+Sep  6 07:18:29 sd6 qmail: 1157519909.633171 qmail-smtpd: 421 badiprbl: ip 198.51.100.54 rbl: example.com

fail2ban/tests/files/logs/recidive

@@ -0,0 +1,6 @@
+# failJSON: { "time": "2006-02-13T15:52:30", "match": true , "host": "1.2.3.4" }
+2006-02-13 15:52:30,388 fail2ban.actions: WARNING [sendmail] Ban 1.2.3.4
+# failJSON: { "match": false }
+2006-02-13 16:07:31,183 fail2ban.actions: WARNING [sendmail] Unban 1.2.3.4
+# failJSON: { "match": false }
+2006-02-13 15:52:30,388 fail2ban.actions: WARNING [recidive] Ban 1.2.3.4

fail2ban/tests/files/logs/sieve

@@ -0,0 +1,6 @@
+# failJSON: { "time": "2004-12-01T20:36:56", "match": true , "host": "1.2.3.4" }
+Dec 1 20:36:56 mail sieve[23713]: badlogin: example.com[1.2.3.4] PLAIN authentication failure
+# failJSON: { "time": "2005-07-18T17:21:58", "match": true , "host": "1.2.3.4" }
+Jul 18 17:21:58 ophelia cyrus/timsieved[12305]: badlogin: example.com[1.2.3.4] PLAIN authentication failure
+# failJSON: { "time": "2004-09-25T22:07:38", "match": true , "host": "1.2.3.4" }
+Sep 25 22:07:38 web9 timsieved[21040]: badlogin: web4[1.2.3.4] OTP authentication failure

fail2ban/tests/files/logs/sshd

@@ -78,13 +78,21 @@ Apr 29 18:53:38 Jamess-iMac.local sshd[47831]: error: PAM: authentication error
 # failJSON: { "time": "2005-06-25T23:53:34", "match": true , "host": "1.2.3.4" }
 Jun 25 23:53:34 [sshd] User root from 1.2.3.4 not allowed because not listed in AllowUsers
 
-#12
 # failJSON: { "match": false }
 Apr 24 01:39:19 host sshd[3719]: User root not allowed because account is locked
 # failJSON: { "match": false }
 Apr 24 01:39:19 host sshd[3719]: input_userauth_request: invalid user root [preauth]
 # failJSON: { "time": "2005-04-24T01:39:19", "match": true , "host": "198.51.100.34" }
 Apr 24 01:39:19 host sshd[3719]: error: Received disconnect from 198.51.100.34: 11: Bye Bye [preauth]
+# failJSON: { "time": "2004-12-12T20:04:39", "match": true , "host": "10.215.4.227" }
+Dec 12 20:04:39 aragorn sshd[1328]: error: PAM: User not known to the underlying authentication module for illegal user kernelitshell from 10.215.4.227 
+# failJSON: { "time": "2005-03-26T04:56:27", "match": true , "host": "example.com" }
+Mar 26 04:56:27 angel sshd[9739]: User allena from example.com not allowed because not in any group
+# failJSON: { "time": "2005-02-07T16:01:07", "match": true , "host": "192.51.100.54" }
+Feb 7 16:01:07 linux-m899 sshd[5106]: User root from 192.51.100.54 not allowed because a group is listed in DenyGroups
+# failJSON: { "time": "2005-01-05T11:15:05", "match": true , "host": "10.0.0.40" }
+Jan 5 11:15:05 NAS sshd[1966]: User root from 10.0.0.40 not allowed because none of user's groups are listed in AllowGroups
+
 # failJSON: { "match": false }
 May 27 00:16:33 host sshd[2364]: User root not allowed because account is locked
 # failJSON: { "match": false }

fail2ban/tests/files/logs/suhosin

@@ -0,0 +1,4 @@
+# failJSON: { "time": "2005-03-11T22:52:12", "match": true , "host": "198.51.100.167" }
+Mar 11 22:52:12   lighttpd[53690]: (mod_fastcgi.c.2676) FastCGI-stderr: ALERT - configured request variable name length limit exceeded - dropped variable 'upqchi07vFfAFuBjnIKGIwiLrHo3Vt68T3yqvhQu2TqetQ78roy7Q6bpTfDUtYFR593/MA' (attacker '198.51.100.167', file '/usr/local/captiveportal/index.php')
+# failJSON: { "time": "2005-02-26T22:52:29", "match": true , "host": "198.51.100.77" }
+Feb 26 22:52:29 host suhosin[9636]: ALERT - script tried to increase memory_limit to 268435456 bytes which is above the allowed value (attacker '198.51.100.77', file '/var/www/wordpress/wp-admin/includes/image.php', line 161)

fail2ban/tests/files/logs/wuftpd

@@ -1,3 +1,5 @@
 # This login line is from syslog
 # failJSON: { "time": "2004-10-06T09:59:26", "match": true , "host": "202.108.145.173" }
 Oct  6 09:59:26 myserver wu-ftpd[18760]: failed login from hj-145-173-a8.bta.net.cn [202.108.145.173]
+# failJSON: { "time": "2004-10-11T16:45:07", "match": true , "host": "198.51.100.71" }
+Oct 11 16:45:07 ubuntu wu-ftpd[2360]: failed login from example.com [198.51.100.71]

fail2ban/tests/files/logs/xinetd-fail

@@ -0,0 +1,4 @@
+# failJSON: { "time": "2005-05-15T17:38:49", "match": true , "host": "198.51.100.169" }
+May 15 17:38:49 boo xinetd[16256]: FAIL: telnet address from=198.51.100.169
+# failJSON: { "time": "2005-08-03T14:38:49", "match": true , "host": "198.51.100.223" }
+Aug  3 14:38:49 backup xinetd[31234]: FAIL: amanda libwrap from=198.51.100.223

fail2ban/tests/files/testcase-usedns.log

@@ -1,2 +1,2 @@
 Aug 14 11:54:59 i60p295 sshd[12365]: Failed publickey for roehl from example.com port 51332 ssh2
-Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:192.0.43.10 port 51332 ssh2
+Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:93.184.216.119 port 51332 ssh2

fail2ban/tests/filtertestcase.py

@@ -94,7 +94,12 @@ def _assert_equal_entries(utest, found, output, count=None):
 	utest.assertEqual(found_time, output_time)
 	if len(output) > 3 and count is None: # match matches
 		# do not check if custom count (e.g. going through them twice)
-		utest.assertEqual(repr(found[3]), repr(output[3]))
+		if os.linesep != '\n' or sys.platform.startswith('cygwin'):
+			# on those where text file lines end with '\r\n', remove '\r'
+			srepr = lambda x: repr(x).replace(r'\r', '')
+		else:
+			srepr = repr
+		utest.assertEqual(srepr(found[3]), srepr(output[3]))
 
 def _ticket_tuple(ticket):
 	"""Create a tuple for easy comparison from fail ticket
@@ -811,12 +816,12 @@ class GetFailures(unittest.TestCase):
 
 	def testGetFailuresUseDNS(self):
 		# We should still catch failures with usedns = no ;-)
-		output_yes = ('192.0.43.10', 2, 1124013539.0,
+		output_yes = ('93.184.216.119', 2, 1124013539.0,
 					  [u'Aug 14 11:54:59 i60p295 sshd[12365]: Failed publickey for roehl from example.com port 51332 ssh2\n',
-					   u'Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:192.0.43.10 port 51332 ssh2\n'])
+					   u'Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:93.184.216.119 port 51332 ssh2\n'])
 
-		output_no = ('192.0.43.10', 1, 1124013539.0,
+		output_no = ('93.184.216.119', 1, 1124013539.0,
-					  [u'Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:192.0.43.10 port 51332 ssh2\n'])
+					  [u'Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:93.184.216.119 port 51332 ssh2\n'])
 
 		# Actually no exception would be raised -- it will be just set to 'no'
 		#self.assertRaises(ValueError,
@@ -917,9 +922,9 @@ class DNSUtilsTests(unittest.TestCase):
 		res = DNSUtils.textToIp('www.example.com', 'no')
 		self.assertEqual(res, [])
 		res = DNSUtils.textToIp('www.example.com', 'warn')
-		self.assertEqual(res, ['192.0.43.10'])
+		self.assertEqual(res, ['93.184.216.119'])
 		res = DNSUtils.textToIp('www.example.com', 'yes')
-		self.assertEqual(res, ['192.0.43.10'])
+		self.assertEqual(res, ['93.184.216.119'])
 
 	def testTextToIp(self):
 		# Test hostnames
@@ -931,7 +936,7 @@ class DNSUtilsTests(unittest.TestCase):
 		for s in hostnames:
 			res = DNSUtils.textToIp(s, 'yes')
 			if s == 'www.example.com':
-				self.assertEqual(res, ['192.0.43.10'])
+				self.assertEqual(res, ['93.184.216.119'])
 			else:
 				self.assertEqual(res, [])
 

fail2ban/tests/samplestestcase.py

@@ -74,19 +74,16 @@ def testSampleRegexsFactory(name):
 				self.filter.addFailRegex(opt[3])
 			elif opt[2] == "maxlines":
 				self.filter.setMaxLines(opt[3])
+			elif opt[2] == "addignoreregex":
+				self.filter.addIgnoreRegex(opt[3])
 
 		if not self.filter.getFailRegex():
 			# No fail regexs set: likely just common file for includes.
 			return
 
-		# TODO: Remove exception handling once sample logs obtained for all
+		self.assertTrue(
-		try:
+			os.path.isfile(os.path.join(TEST_FILES_DIR, "logs", name)),
-			self.assertTrue(
+			"No sample log file available for '%s' filter" % name)
-				os.path.isfile(os.path.join(TEST_FILES_DIR, "logs", name)),
-				"No sample log file available for '%s' filter" % name)
-		except AssertionError:
-			print "I: No sample log file available for '%s' filter" % name
-			return
 
 		logFile = fileinput.FileInput(
 			os.path.join(TEST_FILES_DIR, "logs", name))
@@ -118,30 +115,28 @@ def testSampleRegexsFactory(name):
 				self.assertTrue(faildata.get('match', False),
 					"Line matched when shouldn't have: %s:%i %r" %
 					(logFile.filename(), logFile.filelineno(), line))
-				self.assertEqual(len(ret), 1,
+				self.assertEqual(len(ret), 1, "Multiple regexs matched %r - %s:%i" %
-					"Multiple regexs matched: regexs %s for %s:%i" % (
+								 (map(lambda x: x[0], ret),logFile.filename(), logFile.filelineno()))
-						",".join(["%i" % a[0] for a in ret]),
+
-						logFile.filename(), logFile.filelineno()))
 				# Verify timestamp and host as expected
 				failregex, host, time = ret[0]
 				self.assertEqual(host, faildata.get("host", None))
-				self.assertEqual(
+				fail2banTime = datetime.datetime.fromtimestamp(time)
-					datetime.datetime.fromtimestamp(time),
+				jsonTime = datetime.datetime.strptime(
-					datetime.datetime.strptime(
+							faildata.get("time", None), "%Y-%m-%dT%H:%M:%S")
-						faildata.get("time", None), "%Y-%m-%dT%H:%M:%S"))
+				
+				self.assertEqual(fail2banTime, jsonTime,
+					"Time  mismatch %s != %s on: %s:%i %r:" % 
+					(fail2banTime, jsonTime, logFile.filename(), logFile.filelineno(), line ) )
 
 				regexsUsed.add(failregex)
 
 		# TODO: Remove exception handling once all regexs have samples
 		for failRegexIndex, failRegex in enumerate(self.filter.getFailRegex()):
-			try:
+			self.assertTrue(
-				self.assertTrue(
+				failRegexIndex in regexsUsed,
-					failRegexIndex in regexsUsed,
+				"Regex for filter '%s' has no samples: %i: %r" %
-					"Regex for filter '%s' has no samples: %i: %r" %
+					(name, failRegexIndex, failRegex))
-						(name, failRegexIndex, failRegex))
-			except AssertionError:
-				print "I: Regex for filter '%s' has no samples: %i: %r" % (
-					name, failRegexIndex, failRegex)
 
 	return testFilter
 

fail2ban/tests/servertestcase.py

@@ -24,7 +24,7 @@ __author__ = "Cyril Jaquier"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
-import unittest, socket, time, tempfile, os, locale
+import unittest, socket, time, tempfile, os, locale, sys
 
 from fail2ban.server.server import Server
 from fail2ban.server.jail import Jail
@@ -606,7 +606,8 @@ class TransmitterLogging(TransmitterBase):
 
 		self.setGetTest("logtarget", "STDOUT")
 		self.setGetTest("logtarget", "STDERR")
-		self.setGetTest("logtarget", "SYSLOG")
+		if sys.platform.lower().startswith('linux'):
+			self.setGetTest("logtarget", "SYSLOG")
 
 	def testLogLevel(self):
 		self.setGetTest("loglevel", "4", 4)

files/nagios/README

@@ -35,7 +35,7 @@ HELP:
 /etc/init.d/fail2ban stop
 
 2.) delete the socket if available
-rm /tmp/fail2ban.sock
+rm /var/run/fail2ban/fail2ban.sock
 
 3.) start the Service 
 /etc/init.d/fail2ban start

files/redhat-initd

@@ -1,38 +1,42 @@
 #!/bin/bash
 #
 # chkconfig: 345 92 08
-# description: Fail2ban daemon
+# processname: fail2ban-server
-#              http://fail2ban.sourceforge.net/wiki/index.php/Main_Page
+# config: /etc/fail2ban/fail2ban.conf
-# process name: fail2ban-server
+# pidfile: /var/run/fail2ban/fail2ban.pid
-#
+# description: fail2ban is a daemon to ban hosts that cause multiple authentication errors
-#
-# Author: Tyler Owen
 #
+### BEGIN INIT INFO
+# Provides: fail2ban
+# Required-Start: $local_fs $remote_fs
+# Required-Stop: $local_fs $remote_fs
+# Should-Start: $time $network $syslog iptables firehol shorewall ferm
+# Should-Stop: $network $syslog iptables firehol shorewall ferm
+# Default-Start: 3 4 5
+# Default-Stop: 0 1 6
+# Short-Description: Start/Stop fail2ban
+# Description: Start/Stop fail2ban, a daemon to ban hosts that cause multiple authentication errors
+### END INIT INFO
 
 # Source function library.
-. /etc/init.d/functions
+. /etc/rc.d/init.d/functions
 
 # Check that the config file exists
 [ -f /etc/fail2ban/fail2ban.conf ] || exit 0
 
 FAIL2BAN="/usr/bin/fail2ban-client"
-
+prog=fail2ban-server
+lockfile=${LOCKFILE-/var/lock/subsys/fail2ban}
+socket=${SOCKET-/var/run/fail2ban/fail2ban.sock}
+pidfile=${PIDFILE-/var/run/fail2ban/fail2ban.pid}
 RETVAL=0
 
-getpid() {
-    pid=`ps -eo pid,comm | grep fail2ban- | awk '{ print $1 }'`
-}
-
 start() {
     echo -n $"Starting fail2ban: "
-    getpid
+    ${FAIL2BAN} -x start > /dev/null
-    if [ -z "$pid" ]; then
+    RETVAL=$?
-	rm -rf /var/run/fail2ban/fail2ban.sock # in case of unclean shutdown
+    if [ $RETVAL = 0 ]; then
-        $FAIL2BAN start > /dev/null
+        touch ${lockfile}
-        RETVAL=$?
-    fi
-    if [ $RETVAL -eq 0 ]; then
-        touch /var/lock/subsys/fail2ban
         echo_success
     else
         echo_failure
@@ -43,51 +47,50 @@ start() {
 
 stop() {
     echo -n $"Stopping fail2ban: "
-    getpid
+    ${FAIL2BAN} stop > /dev/null
     RETVAL=$?
-    if [ -n "$pid" ]; then
+    if [ $RETVAL = 0 ]; then
-        $FAIL2BAN stop > /dev/null
+        rm -f ${lockfile} ${pidfile}
-    sleep 1
-    getpid
-    if [ -z "$pid" ]; then
-        rm -f /var/lock/subsys/fail2ban
         echo_success
     else
         echo_failure
     fi
-    else
+    echo
-        echo_failure
+    return $RETVAL
-    fi
+}
+
+reload() {
+    echo "Reloading fail2ban: "
+    ${FAIL2BAN} reload
+    RETVAL=$?
     echo
     return $RETVAL
 }
 
 # See how we were called.
 case "$1" in
-  start)
+    start)
+        status -p ${pidfile} ${prog} >/dev/null 2>&1 && exit 0
         start
         ;;
-  stop)
+    stop)
         stop
         ;;
-  status)
+    reload)
-        getpid
+        reload
-        if [ -n "$pid" ]; then
-                echo "Fail2ban (pid $pid) is running..."
-                $FAIL2BAN status
-        else
-                RETVAL=1
-                echo "Fail2ban is stopped"
-        fi
         ;;
-  restart)
+    restart)
         stop
         start
         ;;
-  *)
+    status)
-        echo $"Usage: $0 {start|stop|status|restart}"
+        status -p ${pidfile} ${prog}
-        exit 1
+        RETVAL=$?
+        [ $RETVAL = 0 ] && ${FAIL2BAN} status
         ;;
+    *)
+        echo $"Usage: fail2ban {start|stop|restart|reload|status}"
+        RETVAL=2
 esac
 
 exit $RETVAL

setup.py

@@ -122,6 +122,12 @@ setup(
 						('/etc/fail2ban/action.d',
 							glob("config/action.d/*.conf")
 						),
+						('/etc/fail2ban/fail2ban.d',
+							''
+						),
+						('/etc/fail2ban/jail.d',
+							''
+						),
 						('/var/run/fail2ban',
 							''
 						),

testcases/files/logs/sieve

@@ -1,2 +0,0 @@
-# failJSON: { "time": "2004-12-01T20:36:56", "match": true , "host": "1.2.3.4" }
-Dec 1 20:36:56 mail sieve[23713]: badlogin: example.com[1.2.3.4] PLAIN authentication failure