Long delayed and possibly incomplete 0.9.2 release:

ver. 0.9.2 (2015/04/26) - better-quick-now-than-later
 ----------
 
 - Fixes:
    * infinite busy loop on _escapedTags match in substituteRecursiveTags gh-907.
      Thanks TonyThompson
    * port[s] typo in jail.conf/nginx-http-auth gh-913. Thanks Frederik Wagner
      (fnerdwq)
    * $ typo in jail.conf. Thanks Skibbi. Debian bug #767255
    * grep'ing for IP in *mail-whois-lines.conf should now match also
      at the beginning and EOL.  Thanks Dean Lee
    * jail.conf
      - php-url-fopen: separate logpath entries by newline
    * failregex declared direct in jail was joined to single line (specifying of
      multiple expressions was not possible).
    * filters.d/exim.conf - cover different settings of exim logs
      details. Thanks bes.internal
    * filter.d/postfix-sasl.conf - failregex is now case insensitive
    * filters.d/postfix.conf - add 'Client host rejected error message' failregex
    * fail2ban/__init__.py - add strptime thread safety hack-around
    * recidive uses iptables-allports banaction by default now.
      Avoids problems with iptables versions not understanding 'all' for
      protocols and ports
    * filter.d/dovecot.conf
      - match pam_authenticate line from EL7
      - match unknown user line from EL7
    * Use use_poll=True for Python 2.7 and >=3.4 to overcome "Bad file
      descriptor" msgs issue (gh-161)
    * filter.d/postfix-sasl.conf - tweak failregex and add ignoreregex to ignore
      system authentication issues
    * fail2ban-regex reads filter file(s) completely, incl. '.local' file etc.
      (gh-954)
    * firewallcmd-* actions: split output into separate lines for grepping (gh-908)
    * Guard unicode encode/decode issues while storing records in the database.
      Fixes "binding parameter error (unsupported type)" (gh-973), thanks to kot
      for reporting
    * filter.d/sshd added regex for matching openSUSE ssh authentication failure
    * filter.d/asterisk.conf:
      - Dropped "Sending fake auth rejection" failregex since it incorrectly
        targets the asterisk server itself
      - match "hacking attempt detected" logs
 
 - New Features:
    - New filters:
      - postfix-rbl  Thanks Lee Clemens
      - apache-fakegooglebot.conf  Thanks Lee Clemens
      - nginx-botsearch  Thanks Frantisek Sumsal
    - New recursive embedded substitution feature added:
      - `<<PREF>HOST>` becomes `<IPV4HOST>` for PREF=`IPV4`;
      - `<<PREF>HOST>` becomes `1.2.3.4` for PREF=`IPV4` and IPV4HOST=`1.2.3.4`;
    - New interpolation feature for config readers - `%(known/parameter)s`.
      (means last known option with name `parameter`). This interpolation makes
      possible to extend a stock filter or jail regexp in .local file
      (opposite to simply set failregex/ignoreregex that overwrites it),
      see gh-867.
    - Monit config for fail2ban in files/monit/
    - New actions:
      - action.d/firewallcmd-multiport and action.d/firewallcmd-allports Thanks Donald Yandt
      - action.d/sendmail-geoip-lines.conf
      - action.d/nsupdate to update DNSBL. Thanks Andrew St. Jean
    - New status argument for fail2ban-client -- flavor:
      fail2ban-client status <jail> [flavor]
      - empty or "basic" works as-is
      - "cymru" additionally prints (ASN, Country RIR) per banned IP
        (requires dnspython or dnspython3)
    - Flush log at USR1 signal
 
 - Enhancements:
    * Enable multiport for firewallcmd-new action.  Closes gh-834
    * files/debian-initd migrated from the debian branch and should be
      suitable for manual installations now (thanks Juan Karlo de Guzman)
    * Define empty ignoreregex in filters which didn't have it to avoid
      warnings (gh-934)
    * action.d/{sendmail-*,xarf-login-attack}.conf - report local
      timezone not UTC time/zone. Closes gh-911
    * Conditionally log Ignore IP with reason (dns, ip, command). Closes gh-916
    * Absorbed DNSUtils.cidr into addr2bin in filter.py, added unittests
    * Added syslogsocket configuration to fail2ban.conf
    * Note in the jail.conf for the recidive jail to increase dbpurgeage (gh-964)
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iEYEABECAAYFAlU9lg8ACgkQjRFFY3XAJMgFTgCfeDp7M0Xh1J9sbnehVL5fnMT3
 xOoAnA0qN8bR/zGXf1ofDPsZuPEo90k6
 =Iyl6
 -----END PGP SIGNATURE-----

Merge tag '0.9.2' into debian

Long delayed and possibly incomplete 0.9.2 release:

ver. 0.9.2 (2015/04/26) - better-quick-now-than-later
----------

- Fixes:
   * infinite busy loop on _escapedTags match in substituteRecursiveTags gh-907.
     Thanks TonyThompson
   * port[s] typo in jail.conf/nginx-http-auth gh-913. Thanks Frederik Wagner
     (fnerdwq)
   * $ typo in jail.conf. Thanks Skibbi. Debian bug #767255
   * grep'ing for IP in *mail-whois-lines.conf should now match also
     at the beginning and EOL.  Thanks Dean Lee
   * jail.conf
     - php-url-fopen: separate logpath entries by newline
   * failregex declared direct in jail was joined to single line (specifying of
     multiple expressions was not possible).
   * filters.d/exim.conf - cover different settings of exim logs
     details. Thanks bes.internal
   * filter.d/postfix-sasl.conf - failregex is now case insensitive
   * filters.d/postfix.conf - add 'Client host rejected error message' failregex
   * fail2ban/__init__.py - add strptime thread safety hack-around
   * recidive uses iptables-allports banaction by default now.
     Avoids problems with iptables versions not understanding 'all' for
     protocols and ports
   * filter.d/dovecot.conf
     - match pam_authenticate line from EL7
     - match unknown user line from EL7
   * Use use_poll=True for Python 2.7 and >=3.4 to overcome "Bad file
     descriptor" msgs issue (gh-161)
   * filter.d/postfix-sasl.conf - tweak failregex and add ignoreregex to ignore
     system authentication issues
   * fail2ban-regex reads filter file(s) completely, incl. '.local' file etc.
     (gh-954)
   * firewallcmd-* actions: split output into separate lines for grepping (gh-908)
   * Guard unicode encode/decode issues while storing records in the database.
     Fixes "binding parameter error (unsupported type)" (gh-973), thanks to kot
     for reporting
   * filter.d/sshd added regex for matching openSUSE ssh authentication failure
   * filter.d/asterisk.conf:
     - Dropped "Sending fake auth rejection" failregex since it incorrectly
       targets the asterisk server itself
     - match "hacking attempt detected" logs

- New Features:
   - New filters:
     - postfix-rbl  Thanks Lee Clemens
     - apache-fakegooglebot.conf  Thanks Lee Clemens
     - nginx-botsearch  Thanks Frantisek Sumsal
   - New recursive embedded substitution feature added:
     - `<<PREF>HOST>` becomes `<IPV4HOST>` for PREF=`IPV4`;
     - `<<PREF>HOST>` becomes `1.2.3.4` for PREF=`IPV4` and IPV4HOST=`1.2.3.4`;
   - New interpolation feature for config readers - `%(known/parameter)s`.
     (means last known option with name `parameter`). This interpolation makes
     possible to extend a stock filter or jail regexp in .local file
     (opposite to simply set failregex/ignoreregex that overwrites it),
     see gh-867.
   - Monit config for fail2ban in files/monit/
   - New actions:
     - action.d/firewallcmd-multiport and action.d/firewallcmd-allports Thanks Donald Yandt
     - action.d/sendmail-geoip-lines.conf
     - action.d/nsupdate to update DNSBL. Thanks Andrew St. Jean
   - New status argument for fail2ban-client -- flavor:
     fail2ban-client status <jail> [flavor]
     - empty or "basic" works as-is
     - "cymru" additionally prints (ASN, Country RIR) per banned IP
       (requires dnspython or dnspython3)
   - Flush log at USR1 signal

- Enhancements:
   * Enable multiport for firewallcmd-new action.  Closes gh-834
   * files/debian-initd migrated from the debian branch and should be
     suitable for manual installations now (thanks Juan Karlo de Guzman)
   * Define empty ignoreregex in filters which didn't have it to avoid
     warnings (gh-934)
   * action.d/{sendmail-*,xarf-login-attack}.conf - report local
     timezone not UTC time/zone. Closes gh-911
   * Conditionally log Ignore IP with reason (dns, ip, command). Closes gh-916
   * Absorbed DNSUtils.cidr into addr2bin in filter.py, added unittests
   * Added syslogsocket configuration to fail2ban.conf
   * Note in the jail.conf for the recidive jail to increase dbpurgeage (gh-964)

* tag '0.9.2': (140 commits)
  DOC: Slight tune up to RELEASE doc -- no need for PYTHONPATH to run tests
  MANIFEST: updated for some new files, sorted all entries, removed some duplicates
  Initial changes for the release -- simplified ChangeLog header etc
  added \s after host
  replaced .* before rhost with regex matching all the previous fields
  Fixed typo in filter description authentification instead of authentication
  Fixed the UTC -> CEST difference...
  Added changes to ChangeLog & updated sample test cases
  updated filter.d/sshd.conf
  Do not run smtp tests if no_network set
  BF: if install pypy -- come back to original directory
  BF(OSX): apparently exceptions could not be compared for identity, use repr
  very long time resolving IP for address "abcdef" on some PDC, under NAT etc. - replaced via "abcdef.abcdef" to prevent searching in local domains;
  fix test for invalid IP (use TEST-NET-1 according to RFC 5737): since fef031b3cd failed, because on some platforms like vm:debian 10.0.0.0 returns 'localhost' (intern network).
  Match hacking attempt IP instead of asterisk server IP (closes #1000)
  BF: fixing up version comparison for pypy.  Issue appeared in 2.5.0
  ENH: minor formatting, no functional changes
  BF: do not expect setting logtarget to SYSLOG to work on non-Linuxes
  Added a comment about systemd backend for jails with logs outside of journal (Closes #959)
  DOC: make a warning for recidive jail to increase dbpurgeage (Closes #964)
  ...

.gitignore

@@ -9,3 +9,4 @@ htmlcov
 *.bak
 __pycache__
 .vagrant/
+.idea/

.travis.yml

@@ -8,12 +8,17 @@ python:
   - "3.3"
   - "3.4"
   - "pypy"
+  - "pypy3"
 before_install:
-  - if [[ $TRAVIS_PYTHON_VERSION == 2.7 ]]; then sudo apt-get update -qq; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == 2.7 ]]; then travis_retry sudo apt-get update -qq; fi
 install:
-  - pip install pyinotify
+  - travis_retry pip install pyinotify
-  - if [[ $TRAVIS_PYTHON_VERSION == 2.7 ]]; then sudo apt-get install -qq python-gamin; cp /usr/share/pyshared/gamin.py /usr/lib/pyshared/python2.7/_gamin.so $VIRTUAL_ENV/lib/python2.7/site-packages/; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == 2* || $TRAVIS_PYTHON_VERSION == 'pypy' ]]; then travis_retry pip install dnspython; fi
-  - if [[ $TRAVIS_PYTHON_VERSION == 2.7 ]]; then cd ..; pip install -q coveralls; cd -; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == 3* || $TRAVIS_PYTHON_VERSION == 'pypy3' ]]; then travis_retry pip install dnspython3; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == 2.7 ]]; then travis_retry sudo apt-get install -qq python-gamin; cp /usr/share/pyshared/gamin.py /usr/lib/pyshared/python2.7/_gamin.so $VIRTUAL_ENV/lib/python2.7/site-packages/; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == 2.7 ]]; then cd ..; travis_retry pip install -q coveralls; cd -; fi
+  # overcome buggy pypy
+  - if [[ $TRAVIS_PYTHON_VERSION == pypy ]] ; then dpkg --compare-versions $(pypy --version 2>&1 | awk '/PyPy/{print $2;}') ge 2.5.1 || { d=$PWD; cd /tmp; wget http://buildbot.pypy.org/nightly/trunk/pypy-c-jit-latest-linux64.tar.bz2; tar -xjvf pypy*bz2; cd pypy-*/bin/; export PATH=$PWD:$PATH; cd $d; } ; fi
 script:
   - if [[ $TRAVIS_PYTHON_VERSION == 2.7 ]]; then coverage run --rcfile=.travis_coveragerc setup.py test; else python setup.py test; fi
 # test installation

CONTRIBUTING.md

@@ -15,3 +15,13 @@ join the [mailing list](https://lists.sourceforge.net/lists/listinfo/fail2ban-us
 
 ### You would like to contribute (new filters/actions/code/documentation)?
 send a [pull request](https://github.com/fail2ban/fail2ban/pulls)
+
+Pull requests guidelines
+========================
+
+- If there is an issue on github to be closed by the pull request, include
+  ```Closes #ISSUE``` (where ISSUE is issue's number)
+  
+- Add a brief summary of the change to the ChangeLog file into a corresponding
+  section out of Fixes, New Features or Enhancements (improvements to existing
+  features)

ChangeLog

@@ -3,17 +3,20 @@
                        |  _/ _` | | |/ /| '_ \/ _` | ' \ 
                        |_| \__,_|_|_/___|_.__/\__,_|_||_|
 
-================================================================================
+Fail2Ban: Changelog
-Fail2Ban (version 0.9.1.dev)                                          2014/10/29
+===================
-================================================================================
 
-ver. 0.9.2 (2014/XX/XXX) - wanna-be-released
+ver. 0.9.2 (2015/04/26) - better-quick-now-than-later
------------
+----------
 
 - Fixes:
+   * infinite busy loop on _escapedTags match in substituteRecursiveTags gh-907.
+     Thanks TonyThompson
+   * port[s] typo in jail.conf/nginx-http-auth gh-913. Thanks Frederik Wagner
+     (fnerdwq)
    * $ typo in jail.conf. Thanks Skibbi. Debian bug #767255
    * grep'ing for IP in *mail-whois-lines.conf should now match also
-     at the begginning and EOL.  Thanks Dean Lee
+     at the beginning and EOL.  Thanks Dean Lee
    * jail.conf
      - php-url-fopen: separate logpath entries by newline
    * failregex declared direct in jail was joined to single line (specifying of
@@ -21,19 +24,67 @@ ver. 0.9.2 (2014/XX/XXX) - wanna-be-released
    * filters.d/exim.conf - cover different settings of exim logs
      details. Thanks bes.internal
    * filter.d/postfix-sasl.conf - failregex is now case insensitive
+   * filters.d/postfix.conf - add 'Client host rejected error message' failregex
+   * fail2ban/__init__.py - add strptime thread safety hack-around
+   * recidive uses iptables-allports banaction by default now.
+     Avoids problems with iptables versions not understanding 'all' for
+     protocols and ports
+   * filter.d/dovecot.conf
+     - match pam_authenticate line from EL7
+     - match unknown user line from EL7
+   * Use use_poll=True for Python 2.7 and >=3.4 to overcome "Bad file
+     descriptor" msgs issue (gh-161)
+   * filter.d/postfix-sasl.conf - tweak failregex and add ignoreregex to ignore
+     system authentication issues
+   * fail2ban-regex reads filter file(s) completely, incl. '.local' file etc.
+     (gh-954)
+   * firewallcmd-* actions: split output into separate lines for grepping (gh-908)
+   * Guard unicode encode/decode issues while storing records in the database.
+     Fixes "binding parameter error (unsupported type)" (gh-973), thanks to kot
+	 for reporting
+   * filter.d/sshd added regex for matching openSUSE ssh authentication failure
+   * filter.d/asterisk.conf:
+     - Dropped "Sending fake auth rejection" failregex since it incorrectly
+	   targets the asterisk server itself
+     - match "hacking attempt detected" logs
 
 - New Features:
+   - New filters:
+     - postfix-rbl  Thanks Lee Clemens
+     - apache-fakegooglebot.conf  Thanks Lee Clemens
+     - nginx-botsearch  Thanks Frantisek Sumsal
+   - New recursive embedded substitution feature added:
+     - `<<PREF>HOST>` becomes `<IPV4HOST>` for PREF=`IPV4`;
+     - `<<PREF>HOST>` becomes `1.2.3.4` for PREF=`IPV4` and IPV4HOST=`1.2.3.4`;
    - New interpolation feature for config readers - `%(known/parameter)s`.
      (means last known option with name `parameter`). This interpolation makes
      possible to extend a stock filter or jail regexp in .local file
      (opposite to simply set failregex/ignoreregex that overwrites it),
      see gh-867.
-   - Monit config for fail2ban in /files/monit
+   - Monit config for fail2ban in files/monit/
+   - New actions:
+     - action.d/firewallcmd-multiport and action.d/firewallcmd-allports Thanks Donald Yandt
+     - action.d/sendmail-geoip-lines.conf
+     - action.d/nsupdate to update DNSBL. Thanks Andrew St. Jean
+   - New status argument for fail2ban-client -- flavor:
+     fail2ban-client status <jail> [flavor]
+     - empty or "basic" works as-is
+     - "cymru" additionally prints (ASN, Country RIR) per banned IP
+       (requires dnspython or dnspython3)
+   - Flush log at USR1 signal
 
 - Enhancements:
    * Enable multiport for firewallcmd-new action.  Closes gh-834
    * files/debian-initd migrated from the debian branch and should be
      suitable for manual installations now (thanks Juan Karlo de Guzman)
+   * Define empty ignoreregex in filters which didn't have it to avoid
+     warnings (gh-934)
+   * action.d/{sendmail-*,xarf-login-attack}.conf - report local
+     timezone not UTC time/zone. Closes gh-911
+   * Conditionally log Ignore IP with reason (dns, ip, command). Closes gh-916
+   * Absorbed DNSUtils.cidr into addr2bin in filter.py, added unittests
+   * Added syslogsocket configuration to fail2ban.conf
+   * Note in the jail.conf for the recidive jail to increase dbpurgeage (gh-964)
 
 
 ver. 0.9.1 (2014/10/29) - better, faster, stronger

MANIFEST

@@ -9,14 +9,155 @@ RELEASE
 THANKS
 TODO
 Vagrantfile
+bin/fail2ban-client
+bin/fail2ban-regex
+bin/fail2ban-server
+bin/fail2ban-testcases
+config/action.d/apf.conf
+config/action.d/badips.conf
+config/action.d/badips.py
+config/action.d/blocklist_de.conf
+config/action.d/bsd-ipfw.conf
+config/action.d/cloudflare.conf
+config/action.d/complain.conf
+config/action.d/dshield.conf
+config/action.d/dummy.conf
+config/action.d/firewallcmd-allports.conf
+config/action.d/firewallcmd-ipset.conf
+config/action.d/firewallcmd-multiport.conf
+config/action.d/firewallcmd-new.conf
+config/action.d/hostsdeny.conf
+config/action.d/ipfilter.conf
+config/action.d/ipfw.conf
+config/action.d/iptables-allports.conf
+config/action.d/iptables-common.conf
+config/action.d/iptables-ipset-proto4.conf
+config/action.d/iptables-ipset-proto6-allports.conf
+config/action.d/iptables-ipset-proto6.conf
+config/action.d/iptables-multiport-log.conf
+config/action.d/iptables-multiport.conf
+config/action.d/iptables-new.conf
+config/action.d/iptables-xt_recent-echo.conf
+config/action.d/iptables.conf
+config/action.d/mail-buffered.conf
+config/action.d/mail-whois-lines.conf
+config/action.d/mail-whois.conf
+config/action.d/mail.conf
+config/action.d/mynetwatchman.conf
+config/action.d/nsupdate.conf
+config/action.d/nsupdate.conf
+config/action.d/osx-afctl.conf
+config/action.d/osx-ipfw.conf
+config/action.d/pf.conf
+config/action.d/route.conf
+config/action.d/sendmail-buffered.conf
+config/action.d/sendmail-common.conf
+config/action.d/sendmail-geoip-lines.conf
+config/action.d/sendmail-whois-ipjailmatches.conf
+config/action.d/sendmail-whois-ipmatches.conf
+config/action.d/sendmail-whois-lines.conf
+config/action.d/sendmail-whois-matches.conf
+config/action.d/sendmail-whois.conf
+config/action.d/sendmail.conf
+config/action.d/shorewall.conf
+config/action.d/smtp.py
+config/action.d/symbiosis-blacklist-allports.conf
+config/action.d/ufw.conf
+config/action.d/xarf-login-attack.conf
+config/fail2ban.conf
+config/filter.d/3proxy.conf
+config/filter.d/apache-auth.conf
+config/filter.d/apache-badbots.conf
+config/filter.d/apache-botsearch.conf
+config/filter.d/apache-common.conf
+config/filter.d/apache-fakegooglebot.conf
+config/filter.d/apache-modsecurity.conf
+config/filter.d/apache-nohome.conf
+config/filter.d/apache-noscript.conf
+config/filter.d/apache-overflows.conf
+config/filter.d/apache-shellshock.conf
+config/filter.d/assp.conf
+config/filter.d/asterisk.conf
+config/filter.d/botsearch-common.conf
+config/filter.d/common.conf
+config/filter.d/counter-strike.conf
+config/filter.d/courier-auth.conf
+config/filter.d/courier-smtp.conf
+config/filter.d/cyrus-imap.conf
+config/filter.d/directadmin.conf
+config/filter.d/dovecot.conf
+config/filter.d/dropbear.conf
+config/filter.d/ejabberd-auth.conf
+config/filter.d/exim-common.conf
+config/filter.d/exim-spam.conf
+config/filter.d/exim.conf
+config/filter.d/freeswitch.conf
+config/filter.d/groupoffice.conf
+config/filter.d/gssftpd.conf
+config/filter.d/guacamole.conf
+config/filter.d/horde.conf
+config/filter.d/ignorecommands
+config/filter.d/ignorecommands/apache-fakegooglebot
+config/filter.d/kerio.conf
+config/filter.d/lighttpd-auth.conf
+config/filter.d/monit.conf
+config/filter.d/mysqld-auth.conf
+config/filter.d/nagios.conf
+config/filter.d/named-refused.conf
+config/filter.d/nginx-botsearch.conf
+config/filter.d/nginx-http-auth.conf
+config/filter.d/nsd.conf
+config/filter.d/openwebmail.conf
+config/filter.d/oracleims.conf
+config/filter.d/pam-generic.conf
+config/filter.d/pam-generic.conf
+config/filter.d/pam-generic.conf
+config/filter.d/perdition.conf
+config/filter.d/php-url-fopen.conf
+config/filter.d/php-url-fopen.conf
+config/filter.d/php-url-fopen.conf
+config/filter.d/portsentry.conf
+config/filter.d/postfix-rbl.conf
+config/filter.d/postfix-sasl.conf
+config/filter.d/postfix-sasl.conf
+config/filter.d/postfix-sasl.conf
+config/filter.d/postfix.conf
+config/filter.d/proftpd.conf
+config/filter.d/pure-ftpd.conf
+config/filter.d/qmail.conf
+config/filter.d/recidive.conf
+config/filter.d/roundcube-auth.conf
+config/filter.d/selinux-common.conf
+config/filter.d/selinux-ssh.conf
+config/filter.d/sendmail-auth.conf
+config/filter.d/sendmail-reject.conf
+config/filter.d/sendmail-spam.conf
+config/filter.d/sieve.conf
+config/filter.d/sogo-auth.conf
+config/filter.d/solid-pop3d.conf
+config/filter.d/squid.conf
+config/filter.d/squirrelmail.conf
+config/filter.d/sshd-ddos.conf
+config/filter.d/sshd.conf
+config/filter.d/stunnel.conf
+config/filter.d/suhosin.conf
+config/filter.d/tine20.conf
+config/filter.d/uwimap-auth.conf
+config/filter.d/vsftpd.conf
+config/filter.d/webmin-auth.conf
+config/filter.d/wuftpd.conf
+config/filter.d/xinetd-fail.conf
+config/jail.conf
+config/paths-common.conf
+config/paths-debian.conf
+config/paths-fedora.conf
+config/paths-freebsd.conf
+config/paths-osx.conf
+doc/run-rootless.txt
 fail2ban-2to3
 fail2ban-testcases-all
 fail2ban-testcases-all-python3
-bin/fail2ban-client
+fail2ban/__init__.py
-bin/fail2ban-server
-bin/fail2ban-testcases
-bin/fail2ban-regex
-doc/run-rootless.txt
 fail2ban/client/__init__.py
 fail2ban/client/actionreader.py
 fail2ban/client/beautifier.py
@@ -28,6 +169,9 @@ fail2ban/client/fail2banreader.py
 fail2ban/client/filterreader.py
 fail2ban/client/jailreader.py
 fail2ban/client/jailsreader.py
+fail2ban/exceptions.py
+fail2ban/helpers.py
+fail2ban/protocol.py
 fail2ban/server/__init__.py
 fail2ban/server/action.py
 fail2ban/server/actions.py
@@ -64,6 +208,8 @@ fail2ban/tests/clientreadertestcase.py
 fail2ban/tests/config/action.d/brokenaction.conf
 fail2ban/tests/config/fail2ban.conf
 fail2ban/tests/config/filter.d/simple.conf
+fail2ban/tests/config/filter.d/test.conf
+fail2ban/tests/config/filter.d/test.local
 fail2ban/tests/config/jail.conf
 fail2ban/tests/config/paths-common.conf
 fail2ban/tests/config/paths-debian.conf
@@ -74,6 +220,7 @@ fail2ban/tests/datedetectortestcase.py
 fail2ban/tests/dummyjail.py
 fail2ban/tests/failmanagertestcase.py
 fail2ban/tests/files/action.d/action.py
+fail2ban/tests/files/action.d/action_checkainfo.py
 fail2ban/tests/files/action.d/action_errors.py
 fail2ban/tests/files/action.d/action_modifyainfo.py
 fail2ban/tests/files/action.d/action_noAction.py
@@ -104,6 +251,7 @@ fail2ban/tests/files/logs/apache-auth
 fail2ban/tests/files/logs/apache-badbots
 fail2ban/tests/files/logs/apache-botscripts
 fail2ban/tests/files/logs/apache-botsearch
+fail2ban/tests/files/logs/apache-fakegooglebot
 fail2ban/tests/files/logs/apache-modsecurity
 fail2ban/tests/files/logs/apache-nohome
 fail2ban/tests/files/logs/apache-noscript
@@ -135,6 +283,7 @@ fail2ban/tests/files/logs/monit
 fail2ban/tests/files/logs/mysqld-auth
 fail2ban/tests/files/logs/nagios
 fail2ban/tests/files/logs/named-refused
+fail2ban/tests/files/logs/nginx-botsearch
 fail2ban/tests/files/logs/nginx-http-auth
 fail2ban/tests/files/logs/nsd
 fail2ban/tests/files/logs/openwebmail
@@ -144,6 +293,7 @@ fail2ban/tests/files/logs/perdition
 fail2ban/tests/files/logs/php-url-fopen
 fail2ban/tests/files/logs/portsentry
 fail2ban/tests/files/logs/postfix
+fail2ban/tests/files/logs/postfix-rbl
 fail2ban/tests/files/logs/postfix-sasl
 fail2ban/tests/files/logs/proftpd
 fail2ban/tests/files/logs/pure-ftpd
@@ -182,170 +332,38 @@ fail2ban/tests/samplestestcase.py
 fail2ban/tests/servertestcase.py
 fail2ban/tests/sockettestcase.py
 fail2ban/tests/utils.py
-setup.py
-setup.cfg
-fail2ban/__init__.py
-fail2ban/exceptions.py
-fail2ban/helpers.py
 fail2ban/version.py
-fail2ban/protocol.py
+files/bash-completion
-kill-server
+files/cacti/README
-config/action.d/apf.conf
+files/cacti/cacti_host_template_fail2ban.xml
-config/action.d/badips.conf
+files/cacti/fail2ban_stats.sh
-config/action.d/badips.py
-config/action.d/blocklist_de.conf
-config/action.d/bsd-ipfw.conf
-config/action.d/cloudflare.conf
-config/action.d/complain.conf
-config/action.d/dshield.conf
-config/action.d/dummy.conf
-config/action.d/firewallcmd-ipset.conf
-config/action.d/firewallcmd-new.conf
-config/action.d/hostsdeny.conf
-config/action.d/ipfilter.conf
-config/action.d/ipfw.conf
-config/action.d/iptables-allports.conf
-config/action.d/iptables-common.conf
-config/action.d/iptables-ipset-proto4.conf
-config/action.d/iptables-ipset-proto6-allports.conf
-config/action.d/iptables-ipset-proto6.conf
-config/action.d/iptables-multiport-log.conf
-config/action.d/iptables-multiport.conf
-config/action.d/iptables-new.conf
-config/action.d/iptables-xt_recent-echo.conf
-config/action.d/iptables.conf
-config/action.d/mail-buffered.conf
-config/action.d/mail-whois-lines.conf
-config/action.d/mail-whois.conf
-config/action.d/mail.conf
-config/action.d/mynetwatchman.conf
-config/action.d/osx-afctl.conf
-config/action.d/osx-ipfw.conf
-config/action.d/pf.conf
-config/action.d/route.conf
-config/action.d/sendmail-buffered.conf
-config/action.d/sendmail-common.conf
-config/action.d/sendmail-whois-ipjailmatches.conf
-config/action.d/sendmail-whois-ipmatches.conf
-config/action.d/sendmail-whois-lines.conf
-config/action.d/sendmail-whois-matches.conf
-config/action.d/sendmail-whois.conf
-config/action.d/sendmail.conf
-config/action.d/shorewall.conf
-config/action.d/smtp.py
-config/action.d/symbiosis-blacklist-allports.conf
-config/action.d/ufw.conf
-config/action.d/xarf-login-attack.conf
-config/fail2ban.conf
-config/filter.d/3proxy.conf
-config/filter.d/apache-auth.conf
-config/filter.d/apache-badbots.conf
-config/filter.d/apache-botsearch.conf
-config/filter.d/apache-common.conf
-config/filter.d/apache-modsecurity.conf
-config/filter.d/apache-nohome.conf
-config/filter.d/apache-noscript.conf
-config/filter.d/apache-overflows.conf
-config/filter.d/apache-shellshock.conf
-config/filter.d/assp.conf
-config/filter.d/asterisk.conf
-config/filter.d/common.conf
-config/filter.d/counter-strike.conf
-config/filter.d/courier-auth.conf
-config/filter.d/courier-smtp.conf
-config/filter.d/cyrus-imap.conf
-config/filter.d/directadmin.conf
-config/filter.d/dovecot.conf
-config/filter.d/dropbear.conf
-config/filter.d/ejabberd-auth.conf
-config/filter.d/exim-common.conf
-config/filter.d/exim-spam.conf
-config/filter.d/exim.conf
-config/filter.d/freeswitch.conf
-config/filter.d/groupoffice.conf
-config/filter.d/gssftpd.conf
-config/filter.d/guacamole.conf
-config/filter.d/horde.conf
-config/filter.d/kerio.conf
-config/filter.d/lighttpd-auth.conf
-config/filter.d/monit.conf
-config/filter.d/mysqld-auth.conf
-config/filter.d/nagios.conf
-config/filter.d/named-refused.conf
-config/filter.d/nginx-http-auth.conf
-config/filter.d/nsd.conf
-config/filter.d/openwebmail.conf
-config/filter.d/oracleims.conf
-config/filter.d/pam-generic.conf
-config/filter.d/pam-generic.conf
-config/filter.d/pam-generic.conf
-config/filter.d/perdition.conf
-config/filter.d/php-url-fopen.conf
-config/filter.d/php-url-fopen.conf
-config/filter.d/php-url-fopen.conf
-config/filter.d/portsentry.conf
-config/filter.d/postfix-sasl.conf
-config/filter.d/postfix-sasl.conf
-config/filter.d/postfix-sasl.conf
-config/filter.d/postfix.conf
-config/filter.d/proftpd.conf
-config/filter.d/pure-ftpd.conf
-config/filter.d/qmail.conf
-config/filter.d/recidive.conf
-config/filter.d/roundcube-auth.conf
-config/filter.d/selinux-common.conf
-config/filter.d/selinux-ssh.conf
-config/filter.d/sendmail-auth.conf
-config/filter.d/sendmail-reject.conf
-config/filter.d/sendmail-spam.conf
-config/filter.d/sieve.conf
-config/filter.d/sogo-auth.conf
-config/filter.d/solid-pop3d.conf
-config/filter.d/squid.conf
-config/filter.d/squirrelmail.conf
-config/filter.d/sshd-ddos.conf
-config/filter.d/sshd.conf
-config/filter.d/stunnel.conf
-config/filter.d/suhosin.conf
-config/filter.d/tine20.conf
-config/filter.d/uwimap-auth.conf
-config/filter.d/vsftpd.conf
-config/filter.d/webmin-auth.conf
-config/filter.d/wuftpd.conf
-config/filter.d/xinetd-fail.conf
-config/jail.conf
-config/paths-common.conf
-config/paths-debian.conf
-config/paths-fedora.conf
-config/paths-freebsd.conf
-config/paths-osx.conf
-man/fail2ban-client.1
-man/fail2ban.1
-man/jail.conf.5
-man/fail2ban-client.h2m
-man/fail2ban-server.1
-man/fail2ban-server.h2m
-man/fail2ban-regex.1
-man/fail2ban-regex.h2m
-man/generate-man
 files/debian-initd
-files/gentoo-initd
+files/fail2ban-logrotate
+files/fail2ban-tmpfiles.conf
+files/fail2ban.service
+files/fail2ban.upstart
+files/gen_badbots
 files/gentoo-confd
-files/redhat-initd
+files/gentoo-initd
+files/ipmasq-ZZZzzz_fail2ban.rul
+files/logwatch/fail2ban
 files/macosx-initd
+files/monit/fail2ban
+files/nagios/README
+files/nagios/check_fail2ban
+files/redhat-initd
 files/solaris-fail2ban.xml
 files/solaris-svc-fail2ban
 files/suse-initd
-files/fail2ban-logrotate
+kill-server
-files/fail2ban.upstart
+man/fail2ban-client.1
-files/logwatch/fail2ban
+man/fail2ban-client.h2m
-files/cacti/fail2ban_stats.sh
+man/fail2ban-regex.1
-files/cacti/cacti_host_template_fail2ban.xml
+man/fail2ban-regex.h2m
-files/cacti/README
+man/fail2ban-server.1
-files/nagios/check_fail2ban
+man/fail2ban-server.h2m
-files/nagios/README
+man/fail2ban.1
-files/bash-completion
+man/generate-man
-files/fail2ban-tmpfiles.conf
+man/jail.conf.5
-files/fail2ban.service
+setup.cfg
-files/ipmasq-ZZZzzz_fail2ban.rul
+setup.py
-files/gen_badbots

MANIFEST.in

@@ -2,3 +2,4 @@ include ChangeLog COPYING DEVELOP FILTERS README.* THANKS TODO CONTRIBUTING* Vag
 graft doc
 graft files
 recursive-include config *.conf *.py
+recursive-include config/filter.d/ignorecommands *

README.md

@@ -2,7 +2,7 @@
                         / _|__ _(_) |_  ) |__  __ _ _ _  
                        |  _/ _` | | |/ /| '_ \/ _` | ' \ 
                        |_| \__,_|_|_/___|_.__/\__,_|_||_|
-                       v0.9.1.dev              2014/??/??
+                       v0.9.2                  2015/04/26
 
 ## Fail2Ban: ban hosts that cause multiple authentication errors
 
@@ -33,11 +33,12 @@ Optional:
   - Linux >= 2.6.13
 - [gamin >= 0.0.21](http://www.gnome.org/~veillard/gamin)
 - [systemd >= 204](http://www.freedesktop.org/wiki/Software/systemd)
+- [dnspython](http://www.dnspython.org/)
 
 To install, just do:
 
-    tar xvfj fail2ban-0.9.1.tar.bz2
+    tar xvfj fail2ban-0.9.2.tar.bz2
-    cd fail2ban-0.9.1
+    cd fail2ban-0.9.2
     python setup.py install
 
 This will install Fail2Ban into the python library directory. The executable

RELEASE

@@ -72,7 +72,7 @@ Preparation
 
 * Ensure the tests work from the tarball::
 
-    cd /tmp/fail2ban-0.9.2/ && export PYTHONPATH=`pwd` && bin/fail2ban-testcases
+    cd /tmp/fail2ban-0.9.2/ && bin/fail2ban-testcases
 
 * Add/finalize the corresponding entry in the ChangeLog
 

THANKS

@@ -13,6 +13,7 @@ ag4ve (Shawn)
 Alasdair D. Campbell
 Amir Caspi
 Amy
+Andrew St. Jean
 Andrey G. Grozin
 Andy Fragen
 Arturo 'Buanzo' Busleiman
@@ -33,11 +34,13 @@ Daniel B.
 Daniel Black
 David Nutter
 Derek Atkins
+Donald Yandt
 Eric Gerbier
 Enrico Labedzki
 Eugene Hopkinson (SlowRiot)
 ftoppi
 François Boulogne
+Frantisek Sumsal
 Frédéric
 Georgiy Mernov
 Guilhem Lettron
@@ -82,6 +85,7 @@ Michael Hanselmann
 Mika (mkl)
 Nick Munger
 onorua
+Orion Poplawski
 Paul Marrapese
 Paul Traina
 Noel Butler

bin/fail2ban-regex

@@ -220,6 +220,7 @@ class Fail2banRegex(object):
 		self._datepattern_set = False
 		self._journalmatch = None
 
+		self.share_config=dict()
 		self._filter = Filter(None)
 		self._ignoreregex = list()
 		self._failregex = list()
@@ -260,12 +261,24 @@ class Fail2banRegex(object):
 	def readRegex(self, value, regextype):
 		assert(regextype in ('fail', 'ignore'))
 		regex = regextype + 'regex'
-		if os.path.isfile(value):
+		if os.path.isfile(value) or os.path.isfile(value + '.conf'):
+			if os.path.basename(os.path.dirname(value)) == 'filter.d':
+				## within filter.d folder - use standard loading algorithm to load filter completely (with .local etc.):
+				basedir = os.path.dirname(os.path.dirname(value))
+				value = os.path.splitext(os.path.basename(value))[0]
+				print "Use %11s filter file : %s, basedir: %s" % (regex, value, basedir)
+				reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config, basedir=basedir)
+				if not reader.read():
+					print "ERROR: failed to load filter %s" % value
+					return False
+			else:
+				## foreign file - readexplicit this file and includes if possible:
 				print "Use %11s file : %s" % (regex, value)
-			reader = FilterReader(value, 'fail2ban-regex-jail', {})
+				reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config)
 				reader.setBaseDir(None)
-
+				if not reader.readexplicit():
-			if reader.readexplicit():
+					print "ERROR: failed to read %s" % value
+					return False
 			reader.getOptions(None)
 			readercommands = reader.convert()
 			regex_values = [
@@ -289,9 +302,6 @@ class Fail2banRegex(object):
 				elif command[2] == 'datepattern':
 					datepattern = command[3]
 					self.setDatePattern(datepattern)
-			else:
-				print "ERROR: failed to read %s" % value
-				return False
 		else:
 			print "Use %11s line : %s" % (regex, shortstr(value))
 			regex_values = [RegexStat(value)]

config/action.d/badips.py

@@ -111,6 +111,8 @@ class BadIPsAction(ActionBase):
 		------
 		HTTPError
 			Any issues with badips.com request.
+		ValueError
+			If badips.com response didn't contain necessary information
 		"""
 		try:
 			response = urlopen(
@@ -122,7 +124,13 @@ class BadIPsAction(ActionBase):
 				messages['err'])
 			raise
 		else:
-			categories = json.loads(response.read().decode('utf-8'))['categories']
+			response_json = json.loads(response.read().decode('utf-8'))
+			if not 'categories' in response_json:
+				err = "badips.com response lacked categories specification. Response was: %s" \
+				  % (response_json,)
+				self._logSys.error(err)
+				raise ValueError(err)
+			categories = response_json['categories']
 			categories_names = set(
 				value['Name'] for value in categories)
 			if incParents:

config/action.d/bsd-ipfw.conf

@@ -38,7 +38,7 @@ actioncheck =
 # Values:  CMD
 #
 # requires an ipfw rule like "deny ip from table(1) to me"
-actionban = ipfw table <table> add <ip>
+actionban = e=`ipfw table <table> add <ip> 2>&1`; x=$?; [ $x -eq 0 -o "$e" = 'ipfw: setsockopt(IP_FW_TABLE_XADD): File exists' ] || { echo "$e" 1>&2; exit $x; }
 
 
 # Option:  actionunban
@@ -47,7 +47,7 @@ actionban = ipfw table <table> add <ip>
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = ipfw table <table> delete <ip>
+actionunban = e=`ipfw table <table> delete <ip> 2>&1`; x=$?; [ $x -eq 0 -o "$e" = 'ipfw: setsockopt(IP_FW_TABLE_XDEL): No such process' ] || { echo "$e" 1>&2; exit $x; }
 
 [Init]
 # Option:  table

config/action.d/firewallcmd-allports.conf

@@ -0,0 +1,53 @@
+# Fail2Ban configuration file
+#
+# Author: Donald Yandt 
+# Because of the --remove-rules in stop this action requires firewalld-0.3.8+
+
+
+[INCLUDES]
+
+before = iptables-blocktype.conf
+
+[Definition]
+
+actionstart = firewall-cmd --direct --add-chain ipv4 filter f2b-<name>
+              firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 1000 -j RETURN
+              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -j f2b-<name>
+
+actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -j f2b-<name>
+             firewall-cmd --direct --remove-rules ipv4 filter f2b-<name>
+             firewall-cmd --direct --remove-chain ipv4 filter f2b-<name>
+
+
+# Example actioncheck: firewall-cmd --direct --get-chains ipv4 filter | sed -e 's, ,\n,g' | grep -q '^f2b-recidive$'
+
+actioncheck = firewall-cmd --direct --get-chains ipv4 filter | sed -e 's, ,\n,g' | grep -q '^f2b-<name>$'
+
+actionban = firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
+
+actionunban = firewall-cmd --direct --remove-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
+
+[Init]
+
+# Default name of the chain
+#
+name = default
+
+chain = INPUT_direct
+
+# DEV NOTES:
+#
+# Author: Donald Yandt 
+# Uses "FirewallD" instead of the "iptables daemon".
+#
+#
+# Output:
+
+# actionstart:
+# $ firewall-cmd --direct --add-chain ipv4 filter f2b-recidive
+# success
+# $ firewall-cmd --direct --add-rule ipv4 filter f2b-recidive 1000 -j RETURN
+# success
+# $ sudo firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 0 -j f2b-recidive
+# success
+

config/action.d/firewallcmd-multiport.conf

@@ -0,0 +1,63 @@
+# Fail2Ban configuration file
+#
+# Author: Donald Yandt 
+# Because of the --remove-rules in stop this action requires firewalld-0.3.8+
+
+[INCLUDES]
+
+before = iptables-blocktype.conf
+
+[Definition]
+
+actionstart = firewall-cmd --direct --add-chain ipv4 filter f2b-<name>
+              firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 1000 -j RETURN
+              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -m state --state NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
+
+actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -m state --state NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
+             firewall-cmd --direct --remove-rules ipv4 filter f2b-<name>
+             firewall-cmd --direct --remove-chain ipv4 filter f2b-<name>
+
+# Example actioncheck: firewall-cmd --direct --get-chains ipv4 filter | sed -e 's, ,\n,g' | grep -q '^f2b-apache-modsecurity$'
+
+actioncheck = firewall-cmd --direct --get-chains ipv4 filter | sed -e 's, ,\n,g' | grep -q '^f2b-<name>$'
+
+actionban = firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
+
+actionunban = firewall-cmd --direct --remove-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
+
+[Init]
+
+# Default name of the chain
+name = default
+
+chain = INPUT_direct
+
+# Could also use port numbers separated by a comma. 
+port = 1:65535
+
+
+# Option:  protocol
+# Values:  [ tcp | udp | icmp | all ]
+
+protocol = tcp
+
+
+
+# DEV NOTES:
+#
+# Author: Donald Yandt 
+# Uses "FirewallD" instead of the "iptables daemon".
+#
+#
+# Output:
+# actionstart:
+# $ firewall-cmd --direct --add-chain ipv4 filter f2b-apache-modsecurity
+# success
+# $ firewall-cmd --direct --add-rule ipv4 filter f2b-apache-modsecurity 1000 -j RETURN
+# success
+# $ sudo firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 0 -m state --state NEW -p tcp -m multiport --dports 80,443 -j f2b-apache-modsecurity
+# success
+# actioncheck:
+# $ firewall-cmd --direct --get-chains ipv4 filter f2b-apache-modsecurity | sed -e 's, ,\n,g' | grep -q '^f2b-apache-modsecurity$'
+# f2b-apache-modsecurity
+

config/action.d/nsupdate.conf

@@ -0,0 +1,114 @@
+# Fail2Ban configuration file
+#
+# Author: Andrew St. Jean
+#
+# Use nsupdate to perform dynamic DNS updates on a BIND zone file.
+# One may want to do this to update a local RBL with banned IP addresses.
+#
+# Options
+#
+# domain	DNS domain that will appear in nsupdate add and delete
+#		commands.
+#
+# ttl		The time to live (TTL) in seconds of the TXT resource
+#		record.
+#
+# rdata		Data portion of the TXT resource record.
+#
+# nsupdatecmd	Full path to the nsupdate command.
+#
+# keyfile	Full path to TSIG key file used for authentication between
+#		nsupdate and BIND.
+#
+# Create an nsupdate.local to set at least the <domain> and <keyfile>
+# options as they don't have default values.
+#
+# The ban and unban commands assume nsupdate will authenticate to the BIND
+# server using a TSIG key. The full path to the key file must be specified
+# in the <keyfile> parameter. Use this command to generate your TSIG key.
+#
+# dnssec-keygen -a HMAC-MD5 -b 256 -n HOST <key_name>
+#
+# Replace <key_name> with some meaningful name.
+#
+# This command will generate two files. Specify the .private file in the
+# <keyfile> option. Note that the .key file must also be present in the same
+# directory for nsupdate to use the key.
+#
+# Don't forget to add the key and appropriate allow-update or update-policy
+# option to your named.conf file.
+#
+
+[Definition]
+
+# Option:  actionstart
+# Notes.:  command executed once at the start of Fail2Ban.
+# Values:  CMD
+#
+actionstart =
+
+
+# Option:  actionstop
+# Notes.:  command executed once at the end of Fail2Ban
+# Values:  CMD
+#
+actionstop =
+
+
+# Option:  actioncheck
+# Notes.:  command executed once before each actionban command
+# Values:  CMD
+#
+actioncheck =
+
+# Option:  actionban
+# Notes.:  command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionban = echo <ip> | awk -F. '{print "prereq nxrrset "$4"."$3"."$2"."$1".<domain> TXT"; print "update add "$4"."$3"."$2"."$1".<domain> <ttl> IN TXT \"<rdata>\""; print "send"}' | <nsupdatecmd> -k <keyfile>
+
+# Option:  actionunban
+# Notes.:  command executed when unbanning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionunban = echo <ip> | awk -F. '{print "update delete "$4"."$3"."$2"."$1".<domain>"; print "send"}' | <nsupdatecmd> -k <keyfile>
+
+[Init]
+
+# Option:  domain
+# Notes.:  DNS domain that nsupdate will update.
+# Values:  STRING
+#
+domain = 
+
+# Option:  ttl
+# Notes.:  time to live (TTL) in seconds of TXT resource record
+#          added by nsupdate.
+# Values:  NUM
+#
+ttl = 60
+
+# Option:  rdata
+# Notes.:  data portion of the TXT resource record added by nsupdate.
+# Values:  STRING
+#
+rdata = Your IP has been banned
+
+# Option:  nsupdatecmd
+# Notes.:  specifies the full path to the nsupdate program that dynamically
+#          updates BIND zone files.
+# Values:  CMD
+#
+nsupdatecmd = /usr/bin/nsupdate
+
+# Option:  keyfile
+# Notes.:  specifies the full path to the file containing the
+#	   TSIG key for communicating with BIND.
+# Values:  STRING
+#
+keyfile = 
+

config/action.d/sendmail-common.conf

@@ -15,7 +15,7 @@ after = sendmail-common.local
 # Values:  CMD
 #
 actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
-              Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
+              Date: `LC_TIME=C date +"%%a, %%d %%h %%Y %%T %%z"`
               From: <sendername> <<sender>>
               To: <dest>\n
               Hi,\n
@@ -28,7 +28,7 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
 # Values:  CMD
 #
 actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on `uname -n`
-             Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
+             Date: `LC_TIME=C date +"%%a, %%d %%h %%Y %%T %%z"`
              From: <sendername> <<sender>>
              To: <dest>\n
              Hi,\n

config/action.d/sendmail-geoip-lines.conf

@@ -0,0 +1,49 @@
+# Fail2Ban configuration file
+#
+# Author: Viktor Szépe
+#
+#
+
+[INCLUDES]
+
+before = sendmail-common.conf
+
+[Definition]
+
+# Option:  actionban
+# Notes.:  Command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+#          You need to install geoiplookup and the GeoLite or GeoIP databases.
+#          (geoip-bin and geoip-database in Debian)
+#          The host command comes from bind9-host package.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
+            Date: `LC_TIME=C date +"%%a, %%d %%h %%Y %%T %%z"`
+            From: <sendername> <<sender>>
+            To: <dest>\n
+            Hi,\n
+            The IP <ip> has just been banned by Fail2Ban after
+            <failures> attempts against <name>.\n\n
+            Here is more information about <ip>:\n
+            http://bgp.he.net/ip/<ip>
+            http://www.projecthoneypot.org/ip_<ip>
+            http://whois.domaintools.com/<ip>\n\n
+            Country:`geoiplookup -f /usr/share/GeoIP/GeoIP.dat "<ip>" | cut -d':' -f2-`
+            AS:`geoiplookup -f /usr/share/GeoIP/GeoIPASNum.dat "<ip>" | cut -d':' -f2-`
+            hostname: `host -t A <ip> 2>&1`\n\n
+            Lines containing IP:<ip> in <logpath>\n
+            `grep -E '(^|[^0-9])<ip>([^0-9]|$)' <logpath>`\n\n
+            Regards,\n
+            Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
+
+[Init]
+
+# Default name of the chain
+#
+name = default
+
+# Path to the log files which contain relevant lines for the abuser IP
+#
+logpath = /dev/null

config/action.d/sendmail-whois-ipjailmatches.conf

@@ -17,13 +17,13 @@ before = sendmail-common.conf
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
-            Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
+            Date: `LC_TIME=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n
             Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
             <failures> attempts against <name>.\n\n
-            Here are more information about <ip>:\n
+            Here is more information about <ip>:\n
             `/usr/bin/whois <ip>`\n\n
             Matches for <name> with <ipjailfailures> failures IP:<ip>\n
             <ipjailmatches>\n\n

config/action.d/sendmail-whois-ipmatches.conf

@@ -17,13 +17,13 @@ before = sendmail-common.conf
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
-            Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
+            Date: `LC_TIME=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n
             Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
             <failures> attempts against <name>.\n\n
-            Here are more information about <ip>:\n
+            Here is more information about <ip>:\n
             `/usr/bin/whois <ip>`\n\n
             Matches with <ipfailures> failures IP:<ip>\n
             <ipmatches>\n\n

config/action.d/sendmail-whois-lines.conf

@@ -17,7 +17,7 @@ before = sendmail-common.conf
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
-            Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
+            Date: `LC_TIME=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n
             Hi,\n

config/action.d/sendmail-whois-matches.conf

@@ -17,13 +17,13 @@ before = sendmail-common.conf
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
-            Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
+            Date: `LC_TIME=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n
             Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
             <failures> attempts against <name>.\n\n
-            Here are more information about <ip>:\n
+            Here is more information about <ip>:\n
             `/usr/bin/whois <ip>`\n\n
             Matches:\n
             <matches>\n\n

config/action.d/sendmail-whois.conf

@@ -17,7 +17,7 @@ before = sendmail-common.conf
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
-            Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
+            Date: `LC_TIME=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n
             Hi,\n

config/action.d/sendmail.conf

@@ -17,7 +17,7 @@ before = sendmail-common.conf
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
-            Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
+            Date: `LC_TIME=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n
             Hi,\n

config/action.d/xarf-login-attack.conf

@@ -46,7 +46,7 @@ actionban = oifs=${IFS}; IFS=.;SEP_IP=( <ip> ); set -- ${SEP_IP}; ADDRESSES=$(di
             REPORTID=<time>@`uname -n`
             TLP=<tlp>
             PORT=<port>
-            DATE=`LC_TIME=C date -u --date=@<time> +"%%a, %%d %%h %%Y %%T +0000"`
+            DATE=`LC_TIME=C date --date=@<time> +"%%a, %%d %%h %%Y %%T %%z"`
             if [ ! -z "$ADDRESSES" ]; then
                 (printf -- %%b "<header>\n<message>\n<report>\n";
                  date '+Note: Local timezone is %%z (%%Z)';

config/fail2ban.conf

@@ -34,6 +34,12 @@ loglevel = INFO
 #
 logtarget = /var/log/fail2ban.log
 
+# Option: syslogsocket
+# Notes: Set the syslog socket file. Only used when logtarget is SYSLOG
+#        auto uses platform.system() to determine predefined paths
+# Values: [ auto | FILE ]  Default: auto
+syslogsocket = auto
+
 # Option: socket
 # Notes.: Set the socket file. This is used to communicate with the daemon. Do
 #         not remove this file when Fail2ban runs. It will not be possible to

config/filter.d/apache-botsearch.conf

@@ -17,7 +17,9 @@
 [INCLUDES]
 
 # overwrite with apache-common.local if _apache_error_client is incorrect.
+# Load regexes for filtering from botsearch-common.conf
 before = apache-common.conf
+         botsearch-common.conf
 
 [Definition]
 
@@ -31,16 +33,6 @@ ignoreregex =
 
 # Webroot represents the webroot on which all other files are based
 webroot = /var/www/
-# Block is the actual non-found directories to block
-block = (<webmail>|<phpmyadmin>|<wordpress>)[^,]*
-
-# These are just convient definitions that assist the blocking of stuff that 
-# isn't installed
-webmail = roundcube|(ext)?mail|horde|(v-?)?webmail
-
-phpmyadmin = (typo3/|xampp/|admin/|)(pma|(php)?[Mm]y[Aa]dmin)
-
-wordpress = wp-(login|signup)\.php
 
 
 # DEV Notes:

config/filter.d/apache-fakegooglebot.conf

@@ -0,0 +1,14 @@
+# Fail2Ban filter for fake Googlebot User Agents
+
+[Definition]
+
+failregex = ^<HOST> .*Googlebot.*$
+
+ignoreregex =
+
+
+# DEV Notes:
+#
+# Author: Lee Clemens
+# Thanks: Johannes B. Ullrich, Ph.D.
+# Reference: https://isc.sans.edu/forums/diary/When+Google+isnt+Google/15968/

config/filter.d/asterisk.conf

@@ -22,7 +22,7 @@ failregex = ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Registration from '[^']*'
             ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$
             ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$
             ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Failed to authenticate (user|device) [^@]+@<HOST>\S*$
-            ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s (?:handle_request_subscribe: )?Sending fake auth rejection for (device|user) \d*<sip:[^@]+@<HOST>>;tag=\w+\S*$
+            ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s hacking attempt detected '<HOST>'$
             ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s SecurityEvent="(FailedACL|InvalidAccountID|ChallengeResponseFailed|InvalidPassword)",EventTV="[\d-]+",Severity="[\w]+",Service="[\w]+",EventVersion="\d+",AccountID="\d*",SessionID="0x[\da-f]+",LocalAddress="IPV[46]/(UD|TC)P/[\da-fA-F:.]+/\d+",RemoteAddress="IPV[46]/(UD|TC)P/<HOST>/\d+"(,Challenge="\w+",ReceivedChallenge="\w+")?(,ReceivedHash="[\da-f]+")?(,ACLName="\w+")?$
             ^(%(__prefix_line)s|\[\]\s*WARNING%(__pid_re)s:?(?:\[C-[\da-f]*\])? )Ext\. s: "Rejecting unknown SIP connection from <HOST>"$
 

config/filter.d/botsearch-common.conf

@@ -0,0 +1,19 @@
+# Generic configuration file for -botsearch filters
+
+[Init]
+
+# Block is the actual non-found directories to block
+block = \/?(<webmail>|<phpmyadmin>|<wordpress>|cgi-bin|mysqladmin)[^,]*
+
+# These are just convient definitions that assist the blocking of stuff that 
+# isn't installed
+webmail = roundcube|(ext)?mail|horde|(v-?)?webmail
+
+phpmyadmin = (typo3/|xampp/|admin/|)(pma|(php)?[Mm]y[Aa]dmin)
+
+wordpress = wp-(login|signup)\.php
+
+# DEV Notes:
+# Taken from apache-botsearch filter
+# 
+# Author: Frantisek Sumsal

config/filter.d/common.conf

@@ -53,4 +53,8 @@ __bsd_syslog_verbose = (<[^.]+\.[^.]+>)
 # This can be optional (for instance if we match named native log files)
 __prefix_line = \s*%(__bsd_syslog_verbose)s?\s*(?:%(__hostname)s )?(?:%(__kernel_prefix)s )?(?:@vserver_\S+ )?%(__daemon_combs_re)s?\s%(__daemon_extra_re)s?\s*
 
+# PAM authentication mechanism check for failures, e.g.: pam_unix, pam_sss,
+# pam_ldap
+__pam_auth = pam_unix
+
 # Author: Yaroslav Halchenko

config/filter.d/counter-strike.conf

@@ -6,6 +6,7 @@
 
 failregex = ^: Bad Rcon: "rcon \d+ "\S+"  sv_contact ".*?"" from "<HOST>:\d+"$
 
+ignoreregex =
 
 [Init]
 

config/filter.d/dovecot.conf

@@ -9,9 +9,10 @@ before = common.conf
 
 _daemon = (auth|dovecot(-auth)?|auth-worker)
 
-failregex = ^%(__prefix_line)s(pam_unix(\(dovecot:auth\))?:)?\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* rhost=<HOST>(\s+user=\S*)?\s*$
+failregex = ^%(__prefix_line)s(%(__pam_auth)s(\(dovecot:auth\))?:)?\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* rhost=<HOST>(\s+user=\S*)?\s*$
             ^%(__prefix_line)s(pop3|imap)-login: (Info: )?(Aborted login|Disconnected)(: Inactivity)? \(((auth failed, \d+ attempts)( in \d+ secs)?|tried to use (disabled|disallowed) \S+ auth)\):( user=<\S*>,)?( method=\S+,)? rip=<HOST>(, lip=(\d{1,3}\.){3}\d{1,3})?(, TLS( handshaking(: SSL_accept\(\) failed: error:[\dA-F]+:SSL routines:[TLS\d]+_GET_CLIENT_HELLO:unknown protocol)?)?(: Disconnected)?)?(, session=<\S+>)?\s*$
-            ^%(__prefix_line)s(Info|dovecot: auth\(default\)): pam\(\S+,<HOST>\): pam_authenticate\(\) failed: (User not known to the underlying authentication module: \d+ Time\(s\)|Authentication failure \(password mismatch\?\))\s*$
+            ^%(__prefix_line)s(Info|dovecot: auth\(default\)|auth-worker\(\d+\)): pam\(\S+,<HOST>\): pam_authenticate\(\) failed: (User not known to the underlying authentication module: \d+ Time\(s\)|Authentication failure \(password mismatch\?\))\s*$
+            ^%(__prefix_line)sauth-worker\(\d+\): pam\(\S+,<HOST>\): unknown user\s*$
 
 ignoreregex = 
 

config/filter.d/groupoffice.conf

@@ -8,7 +8,7 @@
 
 failregex = ^\[\]LOGIN FAILED for user: "\S+" from IP: <HOST>$
 
-
+ignoreregex =
 
 # Author: Daniel Black
 

config/filter.d/ignorecommands/apache-fakegooglebot

@@ -0,0 +1,32 @@
+#!/usr/bin/python
+# Inspired by https://isc.sans.edu/forums/diary/When+Google+isnt+Google/15968/
+#
+# Written in Python to reuse built-in Python batteries and not depend on
+# presence of host and cut commands
+#
+import sys
+
+def process_args(argv):
+    if len(argv) != 2:
+       sys.stderr.write("Please provide a single IP as an argument. Got: %s\n"
+                        % (argv[1:]))
+       sys.exit(2)
+
+    ip = argv[1]
+
+    from fail2ban.server.filter import DNSUtils
+    if not DNSUtils.isValidIP(ip):
+       sys.stderr.write("Argument must be a single valid IP. Got: %s\n"
+                        % ip)
+       sys.exit(3)
+    return ip
+
+def is_googlebot(ip):
+    import re
+    from fail2ban.server.filter import DNSUtils
+
+    host = DNSUtils.ipToName(ip)
+    sys.exit(0 if (host and re.match('crawl-.*\.googlebot\.com', host)) else 1)
+
+if __name__ == '__main__':
+    is_googlebot(process_args(sys.argv))

config/filter.d/kerio.conf

@@ -6,6 +6,9 @@ failregex = ^ SMTP Spam attack detected from <HOST>,
             ^ IP address <HOST> found in DNS blacklist \S+, mail from \S+ to \S+$
             ^ Relay attempt from IP address <HOST>
             ^ Attempt to deliver to unknown recipient \S+, from \S+, IP address <HOST>$
+
+ignoreregex =
+
 [Init]
 
 datepattern = ^\[%%d/%%b/%%Y %%H:%%M:%%S\]

config/filter.d/monit.conf

@@ -7,3 +7,4 @@
 failregex = ^\[[A-Z]+\s+\]\s*error\s*:\s*Warning:\s+Client '<HOST>' supplied unknown user '\w+' accessing monit httpd$
             ^\[[A-Z]+\s+\]\s*error\s*:\s*Warning:\s+Client '<HOST>' supplied wrong password for user '\w+' accessing monit httpd$
 
+ignoreregex =

config/filter.d/nginx-botsearch.conf

@@ -0,0 +1,20 @@
+# Fail2Ban filter to match web requests for selected URLs that don't exist
+#
+
+[INCLUDES]
+
+# Load regexes for filtering
+before = botsearch-common.conf
+
+[Definition]
+
+failregex = ^<HOST> \- \S+ \[\] \"(GET|POST) \/<block> \S+\" 404 .+$
+            ^ \[error\] \d+#\d+: \*\d+ (\S+ )?\"\S+\" (failed|is not found) \(2\: No such file or directory\), client\: <HOST>\, server\: \S*\, request: \"(GET|POST) \/<block> \S+\"\, .*?$
+
+ignoreregex = 
+
+
+# DEV Notes:
+# Based on apache-botsearch filter
+# 
+# Author: Frantisek Sumsal

config/filter.d/nsd.conf

@@ -24,3 +24,5 @@ _daemon = nsd
 
 failregex =  ^\[\]%(__prefix_line)sinfo: ratelimit block .* query <HOST> TYPE255$
               ^\[\]%(__prefix_line)sinfo: .* <HOST> refused, no acl matches\.$
+
+ignoreregex =

config/filter.d/pam-generic.conf

@@ -13,7 +13,7 @@ before = common.conf
 # Default: catch all failed logins
 _ttys_re=\S*
 
-__pam_re=\(?pam_unix(?:\(\S+\))?\)?:?
+__pam_re=\(?%(__pam_auth)s(?:\(\S+\))?\)?:?
 _daemon = \S+
 
 failregex = ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=%(_ttys_re)s ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$

config/filter.d/portsentry.conf

@@ -6,5 +6,7 @@
 
 failregex = \/<HOST> Port\: [0-9]+ (TCP|UDP) Blocked$
 
+ignoreregex =
+
 # Author: Pacop <pacoparu@gmail.com>
 

config/filter.d/postfix-rbl.conf

@@ -0,0 +1,19 @@
+# Fail2Ban filter for Postfix's RBL based Blocked hosts
+#
+#
+
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+[Definition]
+
+_daemon = postfix/smtpd
+
+failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 454 4\.7\.1 Service unavailable; Client host \[\S+\] blocked using .* from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$
+
+ignoreregex =
+
+# Author: Lee Clemens

config/filter.d/postfix-sasl.conf

@@ -9,9 +9,9 @@ before = common.conf
 
 _daemon = postfix/(submission/)?smtp(d|s)
 
-failregex = ^%(__prefix_line)swarning: [-._\w]+\[<HOST>\]: SASL ((?i)LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(: [ A-Za-z0-9+/]*={0,2})?\s*$
+failregex = ^%(__prefix_line)swarning: [-._\w]+\[<HOST>\]: SASL ((?i)LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(: [ A-Za-z0-9+/:]*={0,2})?\s*$
 
-ignoreregex = 
+ignoreregex = authentication failed: Connection lost to authentication server$
 
 [Init]
 

config/filter.d/postfix.conf

@@ -13,6 +13,7 @@ before = common.conf
 _daemon = postfix/(submission/)?smtp(d|s)
 
 failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 554 5\.7\.1 .*$
+            ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 450 4\.7\.1 Client host rejected: cannot find your hostname, (\[\S*\]); from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$
             ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 450 4\.7\.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo= *$
             ^%(__prefix_line)sNOQUEUE: reject: VRFY from \S+\[<HOST>\]: 550 5\.1\.1 .*$
             ^%(__prefix_line)simproper command pipelining after \S+ from [^[]*\[<HOST>\]:?$

config/filter.d/squid.conf

@@ -7,7 +7,7 @@
 failregex = ^\s+\d\s<HOST>\s+[A-Z_]+_DENIED/403 .*$
             ^\s+\d\s<HOST>\s+NONE/405 .*$
 
-
+ignoreregex =
 
 # Author: Daniel Black
 

config/filter.d/squirrelmail.conf

@@ -3,6 +3,7 @@
 
 failregex = ^ \[LOGIN_ERROR\].*from <HOST>: Unknown user or password incorrect\.$
 
+ignoreregex =
 
 [Init]
 

config/filter.d/sshd.conf

@@ -33,6 +33,7 @@ failregex = ^%(__prefix_line)s(?:error: PAM: )?[aA]uthentication (?:failure|erro
             ^(?P<__prefix>%(__prefix_line)s)User .+ not allowed because account is locked<SKIPLINES>(?P=__prefix)(?:error: )?Received disconnect from <HOST>: 11: .+ \[preauth\]$
             ^(?P<__prefix>%(__prefix_line)s)Disconnecting: Too many authentication failures for .+? \[preauth\]<SKIPLINES>(?P=__prefix)(?:error: )?Connection closed by <HOST> \[preauth\]$
             ^(?P<__prefix>%(__prefix_line)s)Connection from <HOST> port \d+(?: on \S+ port \d+)?<SKIPLINES>(?P=__prefix)Disconnecting: Too many authentication failures for .+? \[preauth\]$
+            ^%(__prefix_line)spam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=\S*\s*rhost=<HOST>\s.*$
 
 ignoreregex = 
 

config/filter.d/stunnel.conf

@@ -4,6 +4,8 @@
 
 failregex = ^ LOG\d\[\d+:\d+\]:\ SSL_accept from <HOST>:\d+ : (?P<CODE>[\dA-F]+): error:(?P=CODE):SSL routines:SSL3_GET_CLIENT_CERTIFICATE:peer did not return a certificate$
 
+ignoreregex =
+
 # DEV NOTES:
 # 
 # Author: Daniel Black

config/filter.d/vsftpd.conf

@@ -10,7 +10,7 @@ before = common.conf
 
 [Definition]
 
-__pam_re=\(?pam_unix(?:\(\S+\))?\)?:?
+__pam_re=\(?%(__pam_auth)s(?:\(\S+\))?\)?:?
 _daemon =  vsftpd
 
 failregex = ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=(ftp)? ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$

config/filter.d/wuftpd.conf

@@ -11,7 +11,7 @@ before = common.conf
 [Definition]
 
 _daemon = wu-ftpd
-__pam_re=\(?pam_unix(?:\(wu-ftpd:auth\))?\)?:?
+__pam_re=\(?%(__pam_auth)s(?:\(wu-ftpd:auth\))?\)?:?
 
 failregex = ^%(__prefix_line)sfailed login from \S+ \[<HOST>\]\s*$
             ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=(ftp)? ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$

config/jail.conf

@@ -79,6 +79,11 @@ maxretry = 5
 #              See "journalmatch" in the jails associated filter config
 # auto:      will try to use the following backends, in order:
 #              pyinotify, gamin, polling.
+#
+# Note: if systemd backend is choses as the default but you enable a jail
+#       for which logs are present only in its own log files, specify some other
+#       backend for that jail (e.g. polling) and provide empty value for
+#       journalmatch. See https://github.com/fail2ban/fail2ban/issues/959#issuecomment-74901200
 backend = auto
 
 # "usedns" specifies if jails should trust hostnames in logs,
@@ -277,6 +282,14 @@ logpath  = %(apache_error_log)s
 maxretry = 2
 
 
+[apache-fakegooglebot]
+
+port     = http,https
+logpath  = %(apache_access_log)s
+maxretry = 1
+ignorecommand = %(ignorecommands_dir)s/apache-fakegooglebot <ip>
+
+
 [apache-modsecurity]
 
 port     = http,https
@@ -291,9 +304,14 @@ maxretry = 1
 
 [nginx-http-auth]
 
-ports   = http,https
+port    = http,https
 logpath = %(nginx_error_log)s
 
+[nginx-botsearch]
+
+port     = http,https
+logpath  = %(nginx_error_log)s
+maxretry = 2
 
 # Ban attackers that try to use PHP's URL-fopen() functionality
 # through GET/POST variables. - Experimental, with more than a year
@@ -465,6 +483,13 @@ port     = smtp,465,submission
 logpath  = %(postfix_log)s
 
 
+[postfix-rbl]
+
+port     = smtp,465,submission
+logpath  = %(syslog_mail)s
+maxretry = 1
+
+
 [sendmail-auth]
 
 port    = submission,465,smtp
@@ -649,15 +674,16 @@ maxretry = 5
 
 
 # Jail for more extended banning of persistent abusers
-# !!! WARNING !!!
+# !!! WARNINGS !!!
-#   Make sure that your loglevel specified in fail2ban.conf/.local
+# 1. Make sure that your loglevel specified in fail2ban.conf/.local
 #    is not at DEBUG level -- which might then cause fail2ban to fall into
 #    an infinite loop constantly feeding itself with non-informative lines
+# 2. Increase dbpurgeage defined in fail2ban.conf to e.g. 648000 (7.5 days)
+#    to maintain entries for failed logins for sufficient amount of time
 [recidive]
 
 logpath  = /var/log/fail2ban.log
-port     = all
+banaction = iptables-allports
-protocol = all
 bantime  = 604800  ; 1 week
 findtime = 86400   ; 1 day
 maxretry = 5

config/paths-common.conf

@@ -61,3 +61,6 @@ dovecot_log = %(syslog_mail_warn)s
 solidpop3d_log = %(syslog_local0)s
 
 mysql_log = %(syslog_daemon)s
+
+# Directory with ignorecommand scripts
+ignorecommands_dir = /etc/fail2ban/filter.d/ignorecommands

fail2ban/__init__.py

@@ -68,3 +68,7 @@ logging.notice = _root_notice
 
 # add NOTICE to the priority map of all the levels
 logging.handlers.SysLogHandler.priority_map['NOTICE'] = 'notice'
+
+from time import strptime
+# strptime thread safety hack-around - http://bugs.python.org/issue7980
+strptime("2012", "%Y")

fail2ban/client/beautifier.py

@@ -85,6 +85,9 @@ class Beautifier:
 						val = " ".join(res1[1]) if isinstance(res1[1], list) else res1[1]
 						msg.append("%s %s:\t%s" % (prefix1, res1[0], val))
 				msg = "\n".join(msg)
+			elif inC[1] == "syslogsocket":
+				msg = "Current syslog socket is:\n"
+				msg = msg + "`- " + response
 			elif inC[1] == "logtarget":
 				msg = "Current logging target is:\n"
 				msg = msg + "`- " + response

fail2ban/client/fail2banreader.py

@@ -46,6 +46,7 @@ class Fail2banReader(ConfigReader):
 	def getOptions(self):
 		opts = [["string", "loglevel", "INFO" ],
 				["string", "logtarget", "STDERR"],
+				["string", "syslogsocket", "auto"],
 				["string", "dbfile", "/var/lib/fail2ban/fail2ban.sqlite3"],
 				["int", "dbpurgeage", 86400]]
 		self.__opts = ConfigReader.getOptions(self, "Definition", opts)
@@ -57,6 +58,8 @@ class Fail2banReader(ConfigReader):
 				stream.append(["set", "loglevel", self.__opts[opt]])
 			elif opt == "logtarget":
 				stream.append(["set", "logtarget", self.__opts[opt]])
+			elif opt == "syslogsocket":
+				stream.append(["set", "syslogsocket", self.__opts[opt]])
 			elif opt == "dbfile":
 				stream.append(["set", "dbfile", self.__opts[opt]])
 			elif opt == "dbpurgeage":

fail2ban/client/filterreader.py

@@ -50,17 +50,17 @@ class FilterReader(DefinitionInitConfigReader):
 	def getCombined(self):
 		combinedopts = dict(list(self._opts.items()) + list(self._initOpts.items()))
 		if not len(combinedopts):
-			return {};
+			return {}
 		opts = CommandAction.substituteRecursiveTags(combinedopts)
 		if not opts:
 			raise ValueError('recursive tag definitions unable to be resolved')
-		return opts;
+		return opts
 	
 	def convert(self):
 		stream = list()
 		opts = self.getCombined()
 		if not len(opts):
-			return stream;
+			return stream
 		for opt, value in opts.iteritems():
 			if opt == "failregex":
 				for regex in value.split('\n'):

fail2ban/protocol.py

@@ -44,6 +44,8 @@ protocol = [
 ["get loglevel", "gets the logging level"], 
 ["set logtarget <TARGET>", "sets logging target to <TARGET>. Can be STDOUT, STDERR, SYSLOG or a file"], 
 ["get logtarget", "gets logging target"], 
+["set syslogsocket auto|<SOCKET>", "sets the syslog socket path to auto or <SOCKET>. Only used if logtarget is SYSLOG"],
+["get syslogsocket", "gets syslog socket path"],
 ["flushlogs", "flushes the logtarget if a file and reopens it. For log rotation."], 
 ['', "DATABASE", ""],
 ["set dbfile <FILE>", "set the location of fail2ban persistent datastore. Set to \"None\" to disable"], 
@@ -54,7 +56,7 @@ protocol = [
 ["add <JAIL> <BACKEND>", "creates <JAIL> using <BACKEND>"], 
 ["start <JAIL>", "starts the jail <JAIL>"], 
 ["stop <JAIL>", "stops the jail <JAIL>. The jail is removed"], 
-["status <JAIL>", "gets the current status of <JAIL>"],
+["status <JAIL> [FLAVOR]", "gets the current status of <JAIL>, with optional flavor or extended info"],
 ['', "JAIL CONFIGURATION", ""],
 ["set <JAIL> idle on|off", "sets the idle state of <JAIL>"], 
 ["set <JAIL> addignoreip <IP>", "adds <IP> to the ignore list of <JAIL>"], 

fail2ban/server/action.py

@@ -362,6 +362,7 @@ class CommandAction(ActionBase):
 	@classmethod
 	def substituteRecursiveTags(cls, tags):
 		"""Sort out tag definitions within other tags.
+		Since v.0.9.2 supports embedded interpolation (see test cases for examples).
 
 		so:		becomes:
 		a = 3		a = 3
@@ -378,12 +379,17 @@ class CommandAction(ActionBase):
 			Dictionary of tags(keys) and their values, with tags
 			within the values recursively replaced.
 		"""
-		t = re.compile(r'<([^ >]+)>')
+		t = re.compile(r'<([^ <>]+)>')
+		# repeat substitution while embedded-recursive (repFlag is True)
+		while True:
+			repFlag = False
+			# substitute each value:
 			for tag in tags.iterkeys():
 				if tag in cls._escapedTags:
 					# Escaped so won't match
 					continue
 				value = str(tags[tag])
+				# search and replace all tags within value, that can be interpolated using other tags:
 				m = t.search(value)
 				done = []
 				#logSys.log(5, 'TAG: %s, value: %s' % (tag, value))
@@ -394,22 +400,25 @@ class CommandAction(ActionBase):
 						# recursive definitions are bad
 						#logSys.log(5, 'recursion fail tag: %s value: %s' % (tag, value) )
 						return False
-				elif found_tag in cls._escapedTags:
+					if found_tag in cls._escapedTags or not tags.has_key(found_tag):
-					# Escaped so won't match
+						# Escaped or missing tags - just continue on searching after end of match
+						# Missing tags are ok - cInfo can contain aInfo elements like <HOST> and valid shell
+						# constructs like <STDIN>.
+						m = t.search(value, m.end())
 						continue
-				else:
-					if tags.has_key(found_tag):
 					value = value.replace('<%s>' % found_tag , tags[found_tag])
 					#logSys.log(5, 'value now: %s' % value)
 					done.append(found_tag)
 					m = t.search(value, m.start())
-					else:
-						# Missing tags are ok so we just continue on searching.
-						# cInfo can contain aInfo elements like <HOST> and valid shell
-						# constructs like <STDIN>.
-						m = t.search(value, m.start() + 1)
 				#logSys.log(5, 'TAG: %s, newvalue: %s' % (tag, value))
+				# was substituted?
+				if tags[tag] != value:
+					# check still contains any tag - should be repeated (possible embedded-recursive substitution):
+					if t.search(value):
+						repFlag = True
 					tags[tag] = value
+			if not repFlag:
+				break
 		return tags
 
 	@staticmethod

fail2ban/server/actions.py

@@ -370,11 +370,21 @@ class Actions(JailThread, Mapping):
 					self._jail.name, name, aInfo, e,
 					exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
 
-	@property
+	def status(self, flavor="basic"):
-	def status(self):
+		"""Status of current and total ban counts and current banned IP list.
-		"""Status of active bans, and total ban counts.
 		"""
+		# TODO: Allow this list to be printed as 'status' output
+		supported_flavors = ["basic", "cymru"]
+		if flavor is None or flavor not in supported_flavors:
+			logSys.warning("Unsupported extended jail status flavor %r. Supported: %s" % (flavor, supported_flavors))
+		# Always print this information (basic)
 		ret = [("Currently banned", self.__banManager.size()),
 			   ("Total banned", self.__banManager.getBanTotal()),
 			   ("Banned IP list", self.__banManager.getBanList())]
+		if flavor == "cymru":
+			cymru_info = self.__banManager.getBanListExtendedCymruInfo()
+			ret += \
+				[("Banned ASN list", self.__banManager.geBanListExtendedASN(cymru_info)),
+				 ("Banned Country list", self.__banManager.geBanListExtendedCountry(cymru_info)),
+				 ("Banned RIR list", self.__banManager.geBanListExtendedRIR(cymru_info))]
 		return ret

fail2ban/server/asyncserver.py

@@ -149,7 +149,11 @@ class AsyncServer(asyncore.dispatcher):
 		self.__init = True
 		# TODO Add try..catch
 		# There's a bug report for Python 2.6/3.0 that use_poll=True yields some 2.5 incompatibilities:
-		logSys.debug("Detected Python 2.6 or greater. asyncore.loop() not using poll")
+		if (sys.version_info >= (2, 7) and sys.version_info < (2, 8)) \
+		   or (sys.version_info >= (3, 4)): # if python 2.7 ...
+			logSys.debug("Detected Python 2.7. asyncore.loop() using poll")
+			asyncore.loop(use_poll=True) # workaround for the "Bad file descriptor" issue on Python 2.7, gh-161
+		else:
 			asyncore.loop(use_poll=False) # fixes the "Unexpected communication problem" issue on Python 2.6 and 3.0
 	
 	##

fail2ban/server/banmanager.py

@@ -118,6 +118,124 @@ class BanManager:
 		finally:
 			self.__lock.release()
 
+	##
+	# Returns normalized value
+	#
+	# @return value or "unknown" if value is None or empty string
+
+	@staticmethod
+	def handleBlankResult(value):
+		if value is None or len(value) == 0:
+			return "unknown"
+		else:
+			return value
+
+	##
+	# Returns Cymru DNS query information
+	#
+	# @return {"asn": [], "country": [], "rir": []} dict for self.__banList IPs
+
+	def getBanListExtendedCymruInfo(self):
+		return_dict = {"asn": [], "country": [], "rir": []}
+		try:
+			import dns.exception
+			import dns.resolver
+		except ImportError:
+			logSys.error("dnspython package is required but could not be imported")
+			return_dict["asn"].append("error")
+			return_dict["country"].append("error")
+			return_dict["rir"].append("error")
+			return return_dict
+		self.__lock.acquire()
+		try:
+			for banData in self.__banList:
+				ip = banData.getIP()
+				# Reference: http://www.team-cymru.org/Services/ip-to-asn.html#dns
+				# TODO: IPv6 compatibility
+				reversed_ip = ".".join(reversed(ip.split(".")))
+				question = "%s.origin.asn.cymru.com" % reversed_ip
+				try:
+					answers = dns.resolver.query(question, "TXT")
+					for rdata in answers:
+						asn, net, country, rir, changed =\
+							[answer.strip("'\" ") for answer in rdata.to_text().split("|")]
+						asn = self.handleBlankResult(asn)
+						country = self.handleBlankResult(country)
+						rir = self.handleBlankResult(rir)
+						return_dict["asn"].append(self.handleBlankResult(asn))
+						return_dict["country"].append(self.handleBlankResult(country))
+						return_dict["rir"].append(self.handleBlankResult(rir))
+				except dns.resolver.NXDOMAIN:
+					return_dict["asn"].append("nxdomain")
+					return_dict["country"].append("nxdomain")
+					return_dict["rir"].append("nxdomain")
+				except dns.exception.DNSException as dnse:
+					logSys.error("Unhandled DNSException querying Cymru for %s TXT" % question)
+					logSys.exception(dnse)
+				except Exception as e:
+					logSys.error("Unhandled Exception querying Cymru for %s TXT" % question)
+					logSys.exception(e)
+		except Exception as e:
+			logSys.error("Failure looking up extended Cymru info")
+			logSys.exception(e)
+		finally:
+			self.__lock.release()
+		return return_dict
+
+	##
+	# Returns list of Banned ASNs from Cymru info
+	#
+	# Use getBanListExtendedCymruInfo() to provide cymru_info
+	#
+	# @return list of Banned ASNs
+
+	def geBanListExtendedASN(self, cymru_info):
+		self.__lock.acquire()
+		try:
+			return [asn for asn in cymru_info["asn"]]
+		except Exception as e:
+			logSys.error("Failed to lookup ASN")
+			logSys.exception(e)
+			return []
+		finally:
+			self.__lock.release()
+
+	##
+	# Returns list of Banned Countries from Cymru info
+	#
+	# Use getBanListExtendedCymruInfo() to provide cymru_info
+	#
+	# @return list of Banned Countries
+
+	def geBanListExtendedCountry(self, cymru_info):
+		self.__lock.acquire()
+		try:
+			return [country for country in cymru_info["country"]]
+		except Exception as e:
+			logSys.error("Failed to lookup Country")
+			logSys.exception(e)
+			return []
+		finally:
+			self.__lock.release()
+
+	##
+	# Returns list of Banned RIRs from Cymru info
+	#
+	# Use getBanListExtendedCymruInfo() to provide cymru_info
+	#
+	# @return list of Banned RIRs
+
+	def geBanListExtendedRIR(self, cymru_info):
+		self.__lock.acquire()
+		try:
+			return [rir for rir in cymru_info["rir"]]
+		except Exception as e:
+			logSys.error("Failed to lookup RIR")
+			logSys.exception(e)
+			return []
+		finally:
+			self.__lock.release()
+
 	##
 	# Create a ban ticket.
 	#

fail2ban/server/database.py

@@ -37,17 +37,51 @@ from ..helpers import getLogger
 logSys = getLogger(__name__)
 
 if sys.version_info >= (3,):
-	sqlite3.register_adapter(
+	def _json_dumps_safe(x):
-		dict,
+		try:
-		lambda x: json.dumps(x, ensure_ascii=False).encode(
+			x = json.dumps(x, ensure_ascii=False).encode(
+				locale.getpreferredencoding(), 'replace')
+		except Exception, e: # pragma: no cover
+			logSys.error('json dumps failed: %s', e)
+			x = '{}'
+		return x
+	def _json_loads_safe(x):
+		try:
+			x = json.loads(x.decode(
 				locale.getpreferredencoding(), 'replace'))
-	sqlite3.register_converter(
+		except Exception, e: # pragma: no cover
-		"JSON",
+			logSys.error('json loads failed: %s', e)
-		lambda x: json.loads(x.decode(
+			x = {}
-			locale.getpreferredencoding(), 'replace')))
+		return x
 else:
-	sqlite3.register_adapter(dict, json.dumps)
+	def _normalize(x):
-	sqlite3.register_converter("JSON", json.loads)
+		if isinstance(x, dict):
+			return dict((_normalize(k), _normalize(v)) for k, v in x.iteritems())
+		elif isinstance(x, list):
+			return [_normalize(element) for element in x]
+		elif isinstance(x, unicode):
+			return x.encode(locale.getpreferredencoding())
+		else:
+			return x
+	def _json_dumps_safe(x):
+		try:
+			x = json.dumps(_normalize(x), ensure_ascii=False).decode(
+				locale.getpreferredencoding(), 'replace')
+		except Exception, e: # pragma: no cover
+			logSys.error('json dumps failed: %s', e)
+			x = '{}'
+		return x
+	def _json_loads_safe(x):
+		try:
+			x = _normalize(json.loads(x.decode(
+				locale.getpreferredencoding(), 'replace')))
+		except Exception, e: # pragma: no cover
+			logSys.error('json loads failed: %s', e)
+			x = {}
+		return x
+
+sqlite3.register_adapter(dict, _json_dumps_safe)
+sqlite3.register_converter("JSON", _json_loads_safe)
 
 def commitandrollback(f):
 	@wraps(f)
@@ -431,8 +465,8 @@ class Fail2BanDb(object):
 		tickets = []
 		for ip, timeofban, data in self._getBans(**kwargs):
 			#TODO: Implement data parts once arbitrary match keys completed
-			tickets.append(FailTicket(ip, timeofban, data['matches']))
+			tickets.append(FailTicket(ip, timeofban, data.get('matches')))
-			tickets[-1].setAttempt(data['failures'])
+			tickets[-1].setAttempt(data.get('failures', 1))
 		return tickets
 
 	def getBansMerged(self, ip=None, jail=None, bantime=None):
@@ -484,8 +518,8 @@ class Fail2BanDb(object):
 						prev_banip = banip
 						matches = []
 						failures = 0
-					matches.extend(data['matches'])
+					matches.extend(data.get('matches', []))
-					failures += data['failures']
+					failures += data.get('failures', 1)
 					prev_timeofban = timeofban
 				ticket = FailTicket(banip, prev_timeofban, matches)
 				ticket.setAttempt(failures)

fail2ban/server/filter.py

@@ -338,6 +338,10 @@ class Filter(JailThread):
 		logSys.debug("Remove " + ip + " from ignore list")
 		self.__ignoreIpList.remove(ip)
 
+	def logIgnoreIp(self, ip, log_ignore, ignore_source="unknown source"):
+		if log_ignore:
+			logSys.info("[%s] Ignore %s by %s" % (self.jail.name, ip, ignore_source))
+
 	def getIgnoreIP(self):
 		return self.__ignoreIpList
 
@@ -349,7 +353,7 @@ class Filter(JailThread):
 	# @param ip IP address
 	# @return True if IP address is in ignore list
 
-	def inIgnoreIPList(self, ip):
+	def inIgnoreIPList(self, ip, log_ignore=False):
 		for i in self.__ignoreIpList:
 			# An empty string is always false
 			if i == "":
@@ -363,22 +367,26 @@ class Filter(JailThread):
 					"(?<=b)1+", bin(DNSUtils.addr2bin(s[1]))).group())
 			s[1] = long(s[1])
 			try:
-				a = DNSUtils.cidr(s[0], s[1])
+				a = DNSUtils.addr2bin(s[0], cidr=s[1])
-				b = DNSUtils.cidr(ip, s[1])
+				b = DNSUtils.addr2bin(ip, cidr=s[1])
 			except Exception:
 				# Check if IP in DNS
 				ips = DNSUtils.dnsToIp(i)
 				if ip in ips:
+					self.logIgnoreIp(ip, log_ignore, ignore_source="dns")
 					return True
 				else:
 					continue
 			if a == b:
+				self.logIgnoreIp(ip, log_ignore, ignore_source="ip")
 				return True
 
 		if self.__ignoreCommand:
 			command = CommandAction.replaceTag(self.__ignoreCommand, { 'ip': ip } )
 			logSys.debug('ignore command: ' + command)
-			return CommandAction.executeCmd(command)
+			ret_ignore = CommandAction.executeCmd(command)
+			self.logIgnoreIp(ip, log_ignore and ret_ignore, ignore_source="command")
+			return ret_ignore
 
 		return False
 
@@ -418,8 +426,7 @@ class Filter(JailThread):
 				logSys.debug("Ignore line since time %s < %s - %s"
 							 % (unixTime, MyTime.time(), self.getFindTime()))
 				break
-			if self.inIgnoreIPList(ip):
+			if self.inIgnoreIPList(ip, log_ignore=True):
-				logSys.info("[%s] Ignore %s" % (self.jail.name, ip))
 				continue
 			logSys.info("[%s] Found %s" % (self.jail.name, ip))
 			## print "D: Adding a ticket for %s" % ((ip, unixTime, [line]),)
@@ -529,8 +536,7 @@ class Filter(JailThread):
 						logSys.error(e)
 		return failList
 
-	@property
+	def status(self, flavor="basic"):
-	def status(self):
 		"""Status of failures detected by filter.
 		"""
 		ret = [("Currently failed", self.failManager.size()),
@@ -686,11 +692,10 @@ class FileFilter(Filter):
 			db.updateLog(self.jail, container)
 		return True
 
-	@property
+	def status(self, flavor="basic"):
-	def status(self):
 		"""Status of Filter plus files being monitored.
 		"""
-		ret = super(FileFilter, self).status
+		ret = super(FileFilter, self).status(flavor=flavor)
 		path = [m.getFileName() for m in self.getLogPath()]
 		ret.append(("File list", path))
 		return ret
@@ -792,11 +797,13 @@ class FileContainer:
 			line = line.decode(self.getEncoding(), 'strict')
 		except UnicodeDecodeError:
 			logSys.warning(
-				"Error decoding line from '%s' with '%s'. Continuing "
+				"Error decoding line from '%s' with '%s'."
+				" Consider setting logencoding=utf-8 (or another appropriate"
+				" encoding) for this jail. Continuing"
 				" to process line ignoring invalid characters: %r" %
 				(self.getFileName(), self.getEncoding(), line))
-			if sys.version_info >= (3,): # In python3, must be decoded
+			# decode with replacing error chars:
-				line = line.decode(self.getEncoding(), 'ignore')
+			line = line.decode(self.getEncoding(), 'replace')
 		return line
 
 	def close(self):
@@ -854,6 +861,14 @@ class DNSUtils:
 						% (dns, e))
 			return list()
 
+	@staticmethod
+	def ipToName(ip):
+		try:
+			return socket.gethostbyaddr(ip)[0]
+		except socket.error, e:
+			logSys.debug("Unable to find a name for the IP %s: %s" % (ip, e))
+			return None
+
 	@staticmethod
 	def searchIP(text):
 		""" Search if an IP address if directly available and return
@@ -900,22 +915,18 @@ class DNSUtils:
 		return ipList
 
 	@staticmethod
-	def cidr(i, n):
+	def addr2bin(ipstring, cidr=None):
-		""" Convert an IP address string with a CIDR mask into a 32-bit
+		""" Convert a string IPv4 address into binary form.
-			integer.
+		If cidr is supplied, return the network address for the given block
 		"""
-		# 32-bit IPv4 address mask
+		if cidr is None:
+			return struct.unpack("!L", socket.inet_aton(ipstring))[0]
+		else:
 			MASK = 0xFFFFFFFFL
-		return ~(MASK >> n) & MASK & DNSUtils.addr2bin(i)
+			return ~(MASK >> cidr) & MASK & DNSUtils.addr2bin(ipstring)
 
 	@staticmethod
-	def addr2bin(string):
+	def bin2addr(ipbin):
-		""" Convert a string IPv4 address into an unsigned integer.
+		""" Convert a binary IPv4 address into string n.n.n.n form.
 		"""
-		return struct.unpack("!L", socket.inet_aton(string))[0]
+		return socket.inet_ntoa(struct.pack("!L", ipbin))
-
-	@staticmethod
-	def bin2addr(addr):
-		""" Convert a numeric IPv4 address into string n.n.n.n form.
-		"""
-		return socket.inet_ntoa(struct.pack("!L", addr))

fail2ban/server/filtersystemd.py

@@ -259,9 +259,8 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
                       or "jailless") +" filter terminated")
 		return True
 
-	@property
+	def status(self, flavor="basic"):
-	def status(self):
+		ret = super(FilterSystemd, self).status(flavor=flavor)
-		ret = super(FilterSystemd, self).status
 		ret.append(("Journal matches",
 			[" + ".join(" ".join(match) for match in self.__matches)]))
 		return ret

fail2ban/server/jail.py

@@ -174,13 +174,12 @@ class Jail:
 		self.filter.idle = value
 		self.actions.idle = value
 
-	@property
+	def status(self, flavor="basic"):
-	def status(self):
 		"""The status of the jail.
 		"""
 		return [
-			("Filter", self.filter.status),
+			("Filter", self.filter.status(flavor=flavor)),
-			("Actions", self.actions.status),
+			("Actions", self.actions.status(flavor=flavor)),
 			]
 
 	def putFailTicket(self, ticket):
@@ -214,7 +213,7 @@ class Jail:
 		if self.database is not None:
 			for ticket in self.database.getBansMerged(
 				jail=self, bantime=self.actions.getBanTime()):
-				if not self.filter.inIgnoreIPList(ticket.getIP()):
+				if not self.filter.inIgnoreIPList(ticket.getIP(), log_ignore=True):
 					self.__queue.put(ticket)
 		logSys.info("Jail '%s' started" % self.name)
 

fail2ban/server/jailthread.py

@@ -26,7 +26,7 @@ __license__ = "GPL"
 
 import sys
 from threading import Thread
-from abc import abstractproperty, abstractmethod
+from abc import abstractmethod
 
 from ..helpers import excepthook
 
@@ -66,8 +66,8 @@ class JailThread(Thread):
 				excepthook(*sys.exc_info())
 		self.run = run_with_except_hook
 
-	@abstractproperty
+	@abstractmethod
-	def status(self): # pragma: no cover - abstract
+	def status(self, flavor="basic"): # pragma: no cover - abstract
 		"""Abstract - Should provide status information.
 		"""
 		pass

fail2ban/server/server.py

@@ -25,7 +25,7 @@ __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
 from threading import Lock, RLock
-import logging, logging.handlers, sys, os, signal
+import logging, logging.handlers, sys, os, signal, stat
 
 from .jails import Jails
 from .filter import FileFilter, JournalFilter
@@ -55,20 +55,33 @@ class Server:
 		self.__asyncServer = AsyncServer(self.__transm)
 		self.__logLevel = None
 		self.__logTarget = None
+		self.__syslogSocket = None
+		self.__autoSyslogSocketPaths = {
+			'Darwin':  '/var/run/syslog',
+			'FreeBSD': '/var/run/log',
+			'Linux': '/dev/log',
+		}
 		# Set logging level
 		self.setLogLevel("INFO")
 		self.setLogTarget("STDOUT")
+		self.setSyslogSocket("auto")
+
 
 	def __sigTERMhandler(self, signum, frame):
 		logSys.debug("Caught signal %d. Exiting" % signum)
 		self.quit()
 	
+	def __sigUSR1handler(self, signum, fname):
+		logSys.debug("Caught signal %d. Flushing logs" % signum)
+		self.flushLogs()
+
 	def start(self, sock, pidfile, force = False):
 		logSys.info("Starting Fail2ban v" + version.version)
 		
 		# Install signal handlers
 		signal.signal(signal.SIGTERM, self.__sigTERMhandler)
 		signal.signal(signal.SIGINT, self.__sigTERMhandler)
+		signal.signal(signal.SIGUSR1, self.__sigUSR1handler)
 		
 		# Ensure unhandled exceptions are logged
 		sys.excepthook = excepthook
@@ -320,8 +333,8 @@ class Server:
 		finally:
 			self.__lock.release()
 	
-	def statusJail(self, name):
+	def statusJail(self, name, flavor="basic"):
-		return self.__jails[name].status
+		return self.__jails[name].status(flavor=flavor)
 
 	# Logging
 	
@@ -376,7 +389,21 @@ class Server:
 				# Syslog daemons already add date to the message.
 				formatter = logging.Formatter("%(name)s[%(process)d]: %(levelname)s %(message)s")
 				facility = logging.handlers.SysLogHandler.LOG_DAEMON
-				hdlr = logging.handlers.SysLogHandler("/dev/log", facility=facility)
+				if self.__syslogSocket == "auto":
+					import platform
+					self.__syslogSocket = self.__autoSyslogSocketPaths.get(
+						platform.system())
+				if self.__syslogSocket is not None\
+						and os.path.exists(self.__syslogSocket)\
+						and stat.S_ISSOCK(os.stat(
+								self.__syslogSocket).st_mode):
+					hdlr = logging.handlers.SysLogHandler(
+						self.__syslogSocket, facility=facility)
+				else:
+					logSys.error(
+						"Syslog socket file: %s does not exists"
+						" or is not a socket" % self.__syslogSocket)
+					return False
 			elif target == "STDOUT":
 				hdlr = logging.StreamHandler(sys.stdout)
 			elif target == "STDERR":
@@ -412,14 +439,30 @@ class Server:
 			logger.addHandler(hdlr)
 			# Does not display this message at startup.
 			if not self.__logTarget is None:
-				logSys.info("Changed logging target to %s for Fail2ban v%s" %
+				logSys.info(
-						(target, version.version))
+					"Changed logging target to %s for Fail2ban v%s"
+					% ((target
+						if target != "SYSLOG"
+						else "%s (%s)"
+							 % (target, self.__syslogSocket)),
+					   version.version))
 			# Sets the logging target.
 			self.__logTarget = target
 			return True
 		finally:
 			self.__loggingLock.release()
 
+	##
+	# Sets the syslog socket.
+	#
+	# syslogsocket is the full path to the syslog socket
+	# @param syslogsocket the syslog socket path
+	def setSyslogSocket(self, syslogsocket):
+		self.__syslogSocket = syslogsocket
+		# Conditionally reload, logtarget depends on socket path when SYSLOG
+		return self.__logTarget != "SYSLOG"\
+			   or self.setLogTarget(self.__logTarget)
+
 	def getLogTarget(self):
 		try:
 			self.__loggingLock.acquire()
@@ -427,6 +470,13 @@ class Server:
 		finally:
 			self.__loggingLock.release()
 
+	def getSyslogSocket(self):
+		try:
+			self.__loggingLock.acquire()
+			return self.__syslogSocket
+		finally:
+			self.__loggingLock.release()
+
 	def flushLogs(self):
 		if self.__logTarget not in ['STDERR', 'STDOUT', 'SYSLOG']:
 			for handler in getLogger("fail2ban").handlers:

fail2ban/server/ticket.py

@@ -55,7 +55,7 @@ class Ticket:
 	def __eq__(self, other):
 		try:
 			return self.__ip == other.__ip and \
-				round(self.__time,2) == round(other.__time,2) and \
+				round(self.__time, 2) == round(other.__time, 2) and \
 				self.__attempt == other.__attempt and \
 				self.__matches == other.__matches
 		except AttributeError:

fail2ban/server/transmitter.py

@@ -121,6 +121,12 @@ class Transmitter:
 				return self.__server.getLogTarget()
 			else:
 				raise Exception("Failed to change log target")
+		elif name == "syslogsocket":
+			value = command[1]
+			if self.__server.setSyslogSocket(value):
+				return self.__server.getSyslogSocket()
+			else:
+				raise Exception("Failed to change syslog socket")
 		#Database
 		elif name == "dbfile":
 			self.__server.setDatabase(command[1])
@@ -264,6 +270,8 @@ class Transmitter:
 			return self.__server.getLogLevel()
 		elif name == "logtarget":
 			return self.__server.getLogTarget()
+		elif name == "syslogsocket":
+			return self.__server.getSyslogSocket()
 		#Database
 		elif name == "dbfile":
 			db = self.__server.getDatabase()
@@ -333,5 +341,8 @@ class Transmitter:
 		elif len(command) == 1:
 			name = command[0]
 			return self.__server.statusJail(name)
+		elif len(command) == 2:
+			name = command[0]
+			flavor = command[1]
+			return self.__server.statusJail(name, flavor=flavor)
 		raise Exception("Invalid command (no status)")
-	

fail2ban/tests/actionstestcase.py

@@ -86,7 +86,7 @@ class ExecuteActions(LogCaptureTestCase):
 
 		self.__actions.stop()
 		self.__actions.join()
-		self.assertEqual(self.__actions.status,[("Currently banned", 0 ),
+		self.assertEqual(self.__actions.status(),[("Currently banned", 0 ),
                ("Total banned", 0 ), ("Banned IP list", [] )])
 
 

fail2ban/tests/actiontestcase.py

@@ -59,6 +59,8 @@ class CommandActionTest(LogCaptureTestCase):
 		self.assertEqual(CommandAction.substituteRecursiveTags({'A': '<C>'}), {'A': '<C>'})
 		self.assertEqual(CommandAction.substituteRecursiveTags({'A': '<C> <D> <X>','X':'fun'}), {'A': '<C> <D> fun', 'X':'fun'})
 		self.assertEqual(CommandAction.substituteRecursiveTags({'A': '<C> <B>', 'B': 'cool'}), {'A': '<C> cool', 'B': 'cool'})
+		# Escaped tags should be ignored
+		self.assertEqual(CommandAction.substituteRecursiveTags({'A': '<matches> <B>', 'B': 'cool'}), {'A': '<matches> cool', 'B': 'cool'})
 		# Multiple stuff on same line is ok
 		self.assertEqual(CommandAction.substituteRecursiveTags({'failregex': 'to=<honeypot> fromip=<IP> evilperson=<honeypot>', 'honeypot': 'pokie', 'ignoreregex': ''}),
 								{ 'failregex': "to=pokie fromip=<IP> evilperson=pokie",
@@ -71,6 +73,14 @@ class CommandActionTest(LogCaptureTestCase):
 									'ABC': '123 192.0.2.0',
 									'xyz': '890 123 192.0.2.0',
 								})
+		# obscure embedded case
+		self.assertEqual(CommandAction.substituteRecursiveTags({'A': '<<PREF>HOST>', 'PREF': 'IPV4'}),
+						 {'A': '<IPV4HOST>', 'PREF': 'IPV4'})
+		self.assertEqual(CommandAction.substituteRecursiveTags({'A': '<<PREF>HOST>', 'PREF': 'IPV4', 'IPV4HOST': '1.2.3.4'}),
+						 {'A': '1.2.3.4', 'PREF': 'IPV4', 'IPV4HOST': '1.2.3.4'})
+		# more embedded within a string and two interpolations
+		self.assertEqual(CommandAction.substituteRecursiveTags({'A': 'A <IP<PREF>HOST> B IP<PREF> C', 'PREF': 'V4', 'IPV4HOST': '1.2.3.4'}),
+						 {'A': 'A 1.2.3.4 B IPV4 C', 'PREF': 'V4', 'IPV4HOST': '1.2.3.4'})
 
 	def testReplaceTag(self):
 		aInfo = {

fail2ban/tests/banmanagertestcase.py

@@ -30,7 +30,6 @@ from ..server.banmanager import BanManager
 from ..server.ticket import BanTicket
 
 class AddFailure(unittest.TestCase):
-
 	def setUp(self):
 		"""Call before every test case."""
 		self.__ticket = BanTicket('193.168.0.128', 1167605999.0)
@@ -39,6 +38,7 @@ class AddFailure(unittest.TestCase):
 
 	def tearDown(self):
 		"""Call after every test case."""
+		pass
 
 	def testAdd(self):
 		self.assertEqual(self.__banManager.size(), 1)
@@ -55,3 +55,60 @@ class AddFailure(unittest.TestCase):
 		ticket = BanTicket('111.111.1.111', 1167605999.0)
 		self.assertFalse(self.__banManager._inBanList(ticket))
 
+
+class StatusExtendedCymruInfo(unittest.TestCase):
+	def setUp(self):
+		"""Call before every test case."""
+		self.__ban_ip = "93.184.216.34"
+		self.__asn = "15133"
+		self.__country = "EU"
+		self.__rir = "ripencc"
+		ticket = BanTicket(self.__ban_ip, 1167605999.0)
+		self.__banManager = BanManager()
+		self.assertTrue(self.__banManager.addBanTicket(ticket))
+
+	def tearDown(self):
+		"""Call after every test case."""
+		pass
+
+	def testCymruInfo(self):
+		cymru_info = self.__banManager.getBanListExtendedCymruInfo()
+		if "assertDictEqual" in dir(self):
+			self.assertDictEqual(cymru_info, {"asn": [self.__asn],
+											  "country": [self.__country],
+											  "rir": [self.__rir]})
+		else:
+			# Python 2.6 does not support assertDictEqual()
+			self.assertEqual(cymru_info["asn"], [self.__asn])
+			self.assertEqual(cymru_info["country"], [self.__country])
+			self.assertEqual(cymru_info["rir"], [self.__rir])
+
+	def testCymruInfoASN(self):
+		self.assertEqual(
+			self.__banManager.geBanListExtendedASN(self.__banManager.getBanListExtendedCymruInfo()),
+			[self.__asn])
+
+	def testCymruInfoCountry(self):
+		self.assertEqual(
+			self.__banManager.geBanListExtendedCountry(self.__banManager.getBanListExtendedCymruInfo()),
+			[self.__country])
+
+	def testCymruInfoRIR(self):
+		self.assertEqual(
+			self.__banManager.geBanListExtendedRIR(self.__banManager.getBanListExtendedCymruInfo()),
+			[self.__rir])
+
+	def testCymruInfoNxdomain(self):
+		ticket = BanTicket("10.0.0.0", 1167605999.0)
+		self.__banManager = BanManager()
+		self.assertTrue(self.__banManager.addBanTicket(ticket))
+		cymru_info = self.__banManager.getBanListExtendedCymruInfo()
+		if "assertDictEqual" in dir(self):
+			self.assertDictEqual(cymru_info, {"asn": ["nxdomain"],
+											  "country": ["nxdomain"],
+											  "rir": ["nxdomain"]})
+		else:
+			# Python 2.6 does not support assertDictEqual()
+			self.assertEqual(cymru_info["asn"], ["nxdomain"])
+			self.assertEqual(cymru_info["country"], ["nxdomain"])
+			self.assertEqual(cymru_info["rir"], ["nxdomain"])

fail2ban/tests/clientreadertestcase.py

@@ -459,6 +459,22 @@ class JailsReaderTest(LogCaptureTestCase):
 		self.assertTrue(self._is_logged("No file(s) found for glob /weapons/of/mass/destruction"))
 
 	if STOCK:
+		def testReadStockActionConf(self):
+			for actionConfig in glob.glob(os.path.join(CONFIG_DIR, 'action.d', '*.conf')):
+				actionName = os.path.basename(actionConfig).replace('.conf', '')
+				actionReader = ActionReader(actionName, "TEST", {}, basedir=CONFIG_DIR)
+				self.assertTrue(actionReader.read())
+				actionReader.getOptions({})	  # populate _opts
+				if not actionName.endswith('-common'):
+					self.assertTrue('Definition' in actionReader.sections(),
+						msg="Action file %r is lacking [Definition] section" % actionConfig)
+					# all must have some actionban defined
+					self.assertTrue(actionReader._opts.get('actionban', '').strip(),
+						msg="Action file %r is lacking actionban" % actionConfig)
+				self.assertTrue('Init' in actionReader.sections(),
+						msg="Action file %r is lacking [Init] section" % actionConfig)
+
+
 		def testReadStockJailConf(self):
 			jails = JailsReader(basedir=CONFIG_DIR, share_config=self.__share_cfg) # we are running tests from root project dir atm
 			self.assertTrue(jails.read())		  # opens fine
@@ -607,7 +623,8 @@ class JailsReaderTest(LogCaptureTestCase):
 								'/var/lib/fail2ban/fail2ban.sqlite3'],
 							  ['set', 'dbpurgeage', 86400],
 							  ['set', 'loglevel', "INFO"],
-							  ['set', 'logtarget', '/var/log/fail2ban.log']])
+							  ['set', 'logtarget', '/var/log/fail2ban.log'],
+							  ['set', 'syslogsocket', 'auto']])
 
 			# and if we force change configurator's fail2ban's baseDir
 			# there should be an error message (test visually ;) --

fail2ban/tests/databasetestcase.py

@@ -177,6 +177,37 @@ class DatabaseTest(LogCaptureTestCase):
 		self.assertTrue(
 			isinstance(self.db.getBans(jail=self.jail)[0], FailTicket))
 
+	def testAddBanInvalidEncoded(self):
+		if Fail2BanDb is None: # pragma: no cover
+			return
+		self.testAddJail()
+		# invalid + valid, invalid + valid unicode, invalid + valid dual converted (like in filter:readline by fallback) ...
+		tickets = [
+		  FailTicket("127.0.0.1", 0, ['user "\xd1\xe2\xe5\xf2\xe0"', 'user "\xc3\xa4\xc3\xb6\xc3\xbc\xc3\x9f"']),
+		  FailTicket("127.0.0.2", 0, ['user "\xd1\xe2\xe5\xf2\xe0"', u'user "\xc3\xa4\xc3\xb6\xc3\xbc\xc3\x9f"']),
+		  FailTicket("127.0.0.3", 0, ['user "\xd1\xe2\xe5\xf2\xe0"', b'user "\xc3\xa4\xc3\xb6\xc3\xbc\xc3\x9f"'.decode('utf-8', 'replace')])
+		]
+		self.db.addBan(self.jail, tickets[0])
+		self.db.addBan(self.jail, tickets[1])
+		self.db.addBan(self.jail, tickets[2])
+
+		readtickets = self.db.getBans(jail=self.jail)
+		self.assertEqual(len(readtickets), 3)
+		## python 2 or 3 :
+		invstr = u'user "\ufffd\ufffd\ufffd\ufffd\ufffd"'.encode('utf-8', 'replace')
+		self.assertTrue(
+			   readtickets[0] == FailTicket("127.0.0.1", 0, [invstr, 'user "\xc3\xa4\xc3\xb6\xc3\xbc\xc3\x9f"'])
+			or readtickets[0] == tickets[0]
+		)
+		self.assertTrue(
+			   readtickets[1] == FailTicket("127.0.0.2", 0, [invstr, u'user "\xc3\xa4\xc3\xb6\xc3\xbc\xc3\x9f"'.encode('utf-8', 'replace')])
+			or readtickets[1] == tickets[1]
+		)
+		self.assertTrue(
+			   readtickets[2] == FailTicket("127.0.0.3", 0, [invstr, 'user "\xc3\xa4\xc3\xb6\xc3\xbc\xc3\x9f"'])
+			or readtickets[2] == tickets[2]
+		)
+
 	def testDelBan(self):
 		self.testAddBan()
 		ticket = self.db.getBans(jail=self.jail)[0]

fail2ban/tests/files/logs/apache-fakegooglebot

@@ -0,0 +1,5 @@
+# Apache 2.2
+# failJSON: { "time": "2015-01-31T14:29:44", "match": true, "host": "66.249.66.1" }
+66.249.66.1 - - - [31/Jan/2015:14:29:44 ] example.com "GET / HTTP/1.1" 200 814 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" + 293 1149 546
+# failJSON: { "time": "2015-01-31T14:29:44", "match": false, "host": "93.184.216.34" }
+93.184.216.34 - - - [31/Jan/2015:14:29:44 ] example.com "GET / HTTP/1.1" 200 814 "-" "NOT A __GOOGLE_BOT__" + 293 1149 546

fail2ban/tests/files/logs/asterisk

@@ -29,15 +29,9 @@
 [2013-05-13 07:10:53] SECURITY[1204] res_security_log.c: SecurityEvent="InvalidAccountID",EventTV="1368439853-500975",Severity="Error",Service="SIP",EventVersion="1",AccountID="00972599580679",SessionID="0x7f8ecc0421f8",LocalAddress="IPV4/UDP/1.2.3.4/5060",RemoteAddress="IPV4/UDP/1.2.3.4/5070"
 # failJSON: { "time": "2013-06-10T18:15:03", "match": true , "host": "1.2.3.4" }
 [2013-06-10 18:15:03] NOTICE[2723] chan_sip.c: Registration from '"100"<sip:100@192.168.0.2:5060>' failed for '1.2.3.4' - Not a local domain
-# http://forum.4psa.com/showthread.php?t=6601
+# http://sourceforge.net/p/fail2ban/mailman/message/33603322/
-# failJSON: { "time": "2009-12-22T16:35:24", "match": true , "host": "192.168.2.102" }
+# failJSON: { "time": "2015-03-16T18:46:34", "match": true , "host": "192.168.2.102" }
-[2009-12-22 16:35:24] NOTICE[6163] chan_sip.c: Sending fake auth rejection for device <sip:0004*001@192.168.2.102>;tag=e3793a95e1acbc69o
+[2015-03-16 18:46:34] NOTICE[3453] chan_sip.c: hacking attempt detected '192.168.2.102'
-# http://www.freepbx.org/forum/general-help/fake-auth-rejection
-# failJSON: { "time": "2009-12-22T16:35:24", "match": true , "host": "192.168.2.102" }
-[2009-12-22 16:35:24] NOTICE[1570][C-00000086] chan_sip.c: Sending fake auth rejection for device 1022<sip:1022@192.168.2.102>;tag=5d8b6f92
-# http://www.spinics.net/lists/asterisk/msg127381.html
-# failJSON: { "time": "2009-12-22T16:35:24", "match": true , "host": "192.168.2.102" }
-[2009-12-22 16:35:24] NOTICE[14916]: chan_sip.c:15644 handle_request_subscribe: Sending fake auth rejection for user <sip:CS@192.168.2.102>;tag=6pwd6erg54
 # failJSON: { "time": "2013-07-06T09:09:25", "match": true , "host": "141.255.164.106" }
 [2013-07-06 09:09:25] SECURITY[3308] res_security_log.c: SecurityEvent="InvalidPassword",EventTV="1373098165-824497",Severity="Error",Service="SIP",EventVersion="2",AccountID="972592891005",SessionID="0x88aab6c",LocalAddress="IPV4/UDP/92.28.73.180/5060",RemoteAddress="IPV4/UDP/141.255.164.106/5084",Challenge="41d26de5",ReceivedChallenge="41d26de5",ReceivedHash="7a6a3a2e95a05260aee612896e1b4a39"
 # failJSON: { "time": "2014-01-10T16:39:06", "match": true , "host": "50.30.42.14" }

fail2ban/tests/files/logs/dovecot

@@ -31,6 +31,12 @@ Jul 02 13:49:32 hostname dovecot[442]: dovecot: auth(default): pam(account@MYSER
 # failJSON: { "time": "2013-08-11T03:56:40", "match": true , "host": "1.2.3.4" }
 2013-08-11 03:56:40 auth-worker(default): Info: pam(username,1.2.3.4): pam_authenticate() failed: Authentication failure (password mismatch?)
 
+# failJSON: { "time": "2005-01-29T05:32:50", "match": true , "host": "1.2.3.4" }
+Jan 29 05:32:50 mail dovecot: auth-worker(304): pam(username,1.2.3.4): pam_authenticate() failed: Authentication failure (password mismatch?)
+
+# failJSON: { "time": "2005-01-29T05:13:40", "match": true , "host": "1.2.3.4" }
+Jan 29 05:13:40 mail dovecot: auth-worker(31326): pam(username,1.2.3.4): unknown user
+
 # failJSON: { "time": "2005-04-19T05:22:20", "match": true , "host": "80.255.3.104" }
 Apr 19 05:22:20 vm5 auth: pam_unix(dovecot:auth): authentication failure; logname= uid=0 euid=0 tty=dovecot ruser=informix rhost=80.255.3.104
 

fail2ban/tests/files/logs/nginx-botsearch

@@ -0,0 +1,23 @@
+# failJSON: { "time": "2015-01-20T19:53:28", "match": true , "host": "12.34.56.78" }
+12.34.56.78 - - [20/Jan/2015:19:53:28 +0100] "GET //phpMyAdmin-2.8.2.3/scripts/setup.php HTTP/1.1" 404 47 "-" "-" "-"
+
+# failJSON: { "time": "2015-01-20T19:53:28", "match": true , "host": "12.34.56.78" }
+12.34.56.78 - - [20/Jan/2015:19:53:28 +0100] "GET //pma/scripts/setup.php HTTP/1.1" 404 47 "-" "-" "-"
+
+# failJSON: { "time": "2015-01-20T19:53:28", "match": true , "host": "12.34.56.78" }
+12.34.56.78 - - [20/Jan/2015:19:53:28 +0100] "GET //mysqladmin/scripts/setup.php HTTP/1.1" 404 47 "-" "-" "-"
+
+# failJSON: { "time": "2015-01-20T19:53:28", "match": true , "host": "12.34.56.78" }
+12.34.56.78 - - [20/Jan/2015:19:53:28 +0100] "GET //admin/pma/scripts/setup.php HTTP/1.1" 404 47 "-" "-" "-"
+
+# failJSON: { "time": "2015-01-20T01:17:07", "match": true , "host": "7.8.9.10" }
+7.8.9.10 - root [20/Jan/2015:01:17:07 +0100] "GET /cgi-bin/recent.cgi HTTP/1.1" 404 162 "-" "-" "-"
+
+# failJSON: { "time": "2014-12-12T22:59:02", "match": true , "host": "2.5.2.5" }
+2.5.2.5 - tomcat [12/Dec/2014:22:59:02 +0100] "GET /cgi-bin/tools/tools.pl HTTP/1.1" 404 162 "-" "-" "-"
+
+# failJSON: { "time": "2015-01-21T10:56:10", "match": true , "host": "5.7.9.2" }
+2015/01/21 10:56:10 [error] 2833#0: *16336 open() "/var/www/site/cgi-bin/php4" failed (2: No such file or directory), client: 5.7.9.2, server: localhost, request: "GET /cgi-bin/php4 HTTP/1.1", host: "1.2.3.4"
+
+# failJSON: { "time": "2015-01-21T15:02:27", "match": true , "host": "5.7.9.2" }
+2015/01/21 15:02:27 [error] 2833#0: *16813 "/var/www/site/roundcube/" is not found (2: No such file or directory), client: 5.7.9.2, server: localhost, request: "GET /roundcube/ HTTP/1.1", host: "1.2.3.4"

fail2ban/tests/files/logs/postfix

@@ -20,3 +20,6 @@ Dec 25 02:35:54 platypus postfix/smtpd[9144]: improper command pipelining after
 
 # failJSON: { "time": "2004-12-18T02:05:46", "match": true , "host": "216.245.198.245" }
 Dec 18 02:05:46 platypus postfix/smtpd[16349]: improper command pipelining after NOOP from unknown[216.245.198.245]
+
+# failJSON: { "time": "2004-12-21T21:17:29", "match": true , "host": "93.184.216.34" }
+Dec 21 21:17:29 xxx postfix/smtpd[7150]: NOQUEUE: reject: RCPT from badserver.example.com[93.184.216.34]: 450 4.7.1 Client host rejected: cannot find your hostname, [93.184.216.34]; from=<badactor@example.com> to=<goodguy@example.com> proto=ESMTP helo=<badserver.example.com>

fail2ban/tests/files/logs/postfix-rbl

@@ -0,0 +1,2 @@
+# failJSON: { "time": "2004-12-30T18:19:15", "match": true , "host": "93.184.216.34" }
+Dec 30 18:19:15 xxx postfix/smtpd[1574]: NOQUEUE: reject: RCPT from badguy.example.com[93.184.216.34]: 454 4.7.1 Service unavailable; Client host [93.184.216.34] blocked using rbl.example.com; http://www.example.com/query?ip=93.184.216.34; from=<spammer@example.com> to=<goodguy@example.com> proto=ESMTP helo=<badguy.example.com>

fail2ban/tests/files/logs/postfix-sasl

@@ -12,3 +12,12 @@ Sep  6 00:44:56 trianon postfix/submission/smtpd[11538]: warning: unknown[82.221
 #4 Example from postfix post-debian changes to rename to add "submission" to syslog name + downcase
 # failJSON: { "time": "2004-09-06T00:44:57", "match": true , "host": "82.221.106.233" }
 Sep  6 00:44:57 trianon postfix/submission/smtpd[11538]: warning: unknown[82.221.106.233]: SASL login authentication failed: UGFzc3dvcmQ6
+
+#5 Example to add :
+# failJSON: { "time": "2005-01-29T08:11:45", "match": true , "host": "1.1.1.1" }
+Jan 29 08:11:45 mail postfix/smtpd[10752]: warning: unknown[1.1.1.1]: SASL LOGIN authentication failed: Password:
+
+#6 Example to ignore because due to a failed attempt to connect to authentication service - no malicious activities whatsoever
+# failJSON: { "time": "2005-02-03T08:29:28", "match": false , "host": "1.1.1.1" }
+Feb  3 08:29:28 mail postfix/smtpd[21022]: warning: unknown[1.1.1.1]: SASL LOGIN authentication failed: Connection lost to authentication server
+

fail2ban/tests/files/logs/sshd

@@ -148,3 +148,8 @@ Apr 27 13:02:04 host sshd[29116]: User root not allowed because account is locke
 Apr 27 13:02:04 host sshd[29116]: input_userauth_request: invalid user root [preauth]
 # failJSON: { "time": "2005-04-27T13:02:04", "match": true , "host": "1.2.3.4", "desc": "No Bye-Bye" }
 Apr 27 13:02:04 host sshd[29116]: Received disconnect from 1.2.3.4: 11: Normal Shutdown, Thank you for playing [preauth]
+
+# Match sshd auth errors on OpenSUSE systems
+# failJSON: { "time": "2015-04-16T20:02:50", "match": true , "host": "222.186.21.217", "desc": "Authentication for user failed" }
+2015-04-16T18:02:50.321974+00:00 host sshd[2716]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=222.186.21.217  user=root
+

fail2ban/tests/filtertestcase.py

@@ -224,7 +224,7 @@ class IgnoreIP(LogCaptureTestCase):
 			self.assertTrue(self.filter.inIgnoreIPList(ip))
 
 	def testIgnoreIPNOK(self):
-		ipList = "", "999.999.999.999", "abcdef", "192.168.0."
+		ipList = "", "999.999.999.999", "abcdef.abcdef", "192.168.0."
 		for ip in ipList:
 			self.filter.addIgnoreIP(ip)
 			self.assertFalse(self.filter.inIgnoreIPList(ip))
@@ -266,6 +266,15 @@ class IgnoreIP(LogCaptureTestCase):
 		self.assertTrue(self.filter.inIgnoreIPList("10.0.0.1"))
 		self.assertFalse(self.filter.inIgnoreIPList("10.0.0.0"))
 
+	def testIgnoreCauseOK(self):
+		ip = "93.184.216.34"
+		for ignore_source in ["dns", "ip", "command"]:
+			self.filter.logIgnoreIp(ip, True, ignore_source=ignore_source)
+			self.assertTrue(self._is_logged("[%s] Ignore %s by %s" % (self.jail.name, ip, ignore_source)))
+
+	def testIgnoreCauseNOK(self):
+		self.filter.logIgnoreIp("example.com", False, ignore_source="NOT_LOGGED")
+		self.assertFalse(self._is_logged("[%s] Ignore %s by %s" % (self.jail.name, "example.com", "NOT_LOGGED")))
 
 class IgnoreIPDNS(IgnoreIP):
 
@@ -1011,6 +1020,29 @@ class DNSUtilsTests(unittest.TestCase):
 			else:
 				self.assertEqual(res, [])
 
+	def testIpToName(self):
+		res = DNSUtils.ipToName('66.249.66.1')
+		self.assertEqual(res, 'crawl-66-249-66-1.googlebot.com')
+		# invalid ip (TEST-NET-1 according to RFC 5737)
+		res = DNSUtils.ipToName('192.0.2.0')
+		self.assertEqual(res, None)
+
+	def testAddr2bin(self):
+		res = DNSUtils.addr2bin('10.0.0.0')
+		self.assertEqual(res, 167772160L)
+		res = DNSUtils.addr2bin('10.0.0.0', cidr=None)
+		self.assertEqual(res, 167772160L)
+		res = DNSUtils.addr2bin('10.0.0.0', cidr=32L)
+		self.assertEqual(res, 167772160L)
+		res = DNSUtils.addr2bin('10.0.0.1', cidr=32L)
+		self.assertEqual(res, 167772161L)
+		res = DNSUtils.addr2bin('10.0.0.1', cidr=31L)
+		self.assertEqual(res, 167772160L)
+
+	def testBin2addr(self):
+		res = DNSUtils.bin2addr(167772160L)
+		self.assertEqual(res, '10.0.0.0')
+
 class JailTests(unittest.TestCase):
 
 	def testSetBackend_gh83(self):

fail2ban/tests/misctestcase.py

@@ -113,7 +113,7 @@ class SetupTest(unittest.TestCase):
 			# clean up
 			shutil.rmtree(tmp)
 			# remove build directory
-			os.system("%s %s clean --all >/dev/null"
+			os.system("%s %s clean --all >/dev/null 2>&1"
 					  % (sys.executable, self.setup))
 
 class TestsUtilsTest(unittest.TestCase):

fail2ban/tests/samplestestcase.py

@@ -141,7 +141,8 @@ def testSampleRegexsFactory(name):
 
 	return testFilter
 
-for filter_ in filter(lambda x: not x.endswith('common.conf'), os.listdir(os.path.join(CONFIG_DIR, "filter.d"))):
+for filter_ in filter(lambda x: not x.endswith('common.conf') and x.endswith('.conf'),
+					  os.listdir(os.path.join(CONFIG_DIR, "filter.d"))):
 	filterName = filter_.rpartition(".")[0]
 	if not filterName.startswith('.'):
 		setattr(

fail2ban/tests/servertestcase.py

@@ -30,6 +30,7 @@ import tempfile
 import os
 import locale
 import sys
+import platform
 
 from ..server.failregex import Regex, FailRegex, RegexException
 from ..server.server import Server
@@ -70,17 +71,24 @@ class TransmitterBase(unittest.TestCase):
 		"""Call after every test case."""
 		self.server.quit()
 
-	def setGetTest(self, cmd, inValue, outValue=None, jail=None):
+	def setGetTest(self, cmd, inValue, outValue=None, outCode=0, jail=None, repr_=False):
 		setCmd = ["set", cmd, inValue]
 		getCmd = ["get", cmd]
 		if jail is not None:
 			setCmd.insert(1, jail)
 			getCmd.insert(1, jail)
+
 		if outValue is None:
 			outValue = inValue
 
-		self.assertEqual(self.transm.proceed(setCmd), (0, outValue))
+		def v(x):
-		self.assertEqual(self.transm.proceed(getCmd), (0, outValue))
+			"""Prepare value for comparison"""
+			return (repr(x) if repr_ else x)
+
+		self.assertEqual(v(self.transm.proceed(setCmd)), v((outCode, outValue)))
+		if not outCode:
+			# if we expected to get it set without problem, check new value
+			self.assertEqual(v(self.transm.proceed(getCmd)), v((0, outValue)))
 
 	def setGetTestNOK(self, cmd, inValue, jail=None):
 		setCmd = ["set", cmd, inValue]
@@ -474,6 +482,72 @@ class Transmitter(TransmitterBase):
 			)
 		)
 
+	def testJailStatusBasic(self):
+		self.assertEqual(self.transm.proceed(["status", self.jailName, "basic"]),
+			(0,
+				[
+					('Filter', [
+						('Currently failed', 0),
+						('Total failed', 0),
+						('File list', [])]
+					),
+					('Actions', [
+						('Currently banned', 0),
+						('Total banned', 0),
+						('Banned IP list', [])]
+					)
+				]
+			)
+		)
+
+	def testJailStatusBasicKwarg(self):
+		self.assertEqual(self.transm.proceed(["status", self.jailName, "INVALID"]),
+			(0,
+				[
+					('Filter', [
+						('Currently failed', 0),
+						('Total failed', 0),
+						('File list', [])]
+					),
+					('Actions', [
+						('Currently banned', 0),
+						('Total banned', 0),
+						('Banned IP list', [])]
+					)
+				]
+			)
+		)
+
+	def testJailStatusCymru(self):
+		try:
+			import dns.exception
+			import dns.resolver
+		except ImportError:
+			value = ['error']
+		else:
+			value = []
+
+		self.assertEqual(self.transm.proceed(["status", self.jailName, "cymru"]),
+			(0,
+				[
+					('Filter', [
+						('Currently failed', 0),
+						('Total failed', 0),
+						('File list', [])]
+					),
+					('Actions', [
+						('Currently banned', 0),
+						('Total banned', 0),
+						('Banned IP list', []),
+						('Banned ASN list', value),
+						('Banned Country list', value),
+						('Banned RIR list', value)]
+					)
+				]
+			)
+		)
+
+
 	def testAction(self):
 		action = "TestCaseAction"
 		cmdList = [
@@ -681,6 +755,7 @@ class TransmitterLogging(TransmitterBase):
 		self.server = Server()
 		self.server.setLogTarget("/dev/null")
 		self.server.setLogLevel("CRITICAL")
+		self.server.setSyslogSocket("auto")
 		super(TransmitterLogging, self).setUp()
 
 	def testLogTarget(self):
@@ -708,7 +783,27 @@ class TransmitterLogging(TransmitterBase):
 			raise unittest.SkipTest("'/dev/log' not present")
 		elif not os.path.exists("/dev/log"):
 			return
+		self.assertTrue(self.server.getSyslogSocket(), "auto")
 		self.setGetTest("logtarget", "SYSLOG")
+		self.assertTrue(self.server.getSyslogSocket(), "/dev/log")
+
+	def testSyslogSocket(self):
+		self.setGetTest("syslogsocket", "/dev/log/NEW/PATH")
+
+	def testSyslogSocketNOK(self):
+		self.setGetTest("syslogsocket", "/this/path/should/not/exist")
+		self.setGetTestNOK("logtarget", "SYSLOG")
+		# set back for other tests
+		self.setGetTest("syslogsocket", "/dev/log")
+		self.setGetTest("logtarget", "SYSLOG",
+			**{True: {},    # should work on Linux
+			   False: dict( # expect to fail otherwise
+				   outCode=1,
+				   outValue=Exception('Failed to change log target'),
+				   repr_=True # Exceptions are not comparable apparently
+                                  )
+			  }[platform.system() in ('Linux',)]
+		)
 
 	def testLogLevel(self):
 		self.setGetTest("loglevel", "HEAVYDEBUG")

fail2ban/tests/utils.py

@@ -107,6 +107,11 @@ def gatherTests(regexps=None, no_network=False):
 	tests.addTest(unittest.makeSuite(failmanagertestcase.AddFailure))
 	# BanManager
 	tests.addTest(unittest.makeSuite(banmanagertestcase.AddFailure))
+	try:
+		import dns
+		tests.addTest(unittest.makeSuite(banmanagertestcase.StatusExtendedCymruInfo))
+	except ImportError:
+		pass
 	# ClientReaders
 	tests.addTest(unittest.makeSuite(clientreadertestcase.ConfigReaderTest))
 	tests.addTest(unittest.makeSuite(clientreadertestcase.JailReaderTest))
@@ -148,7 +153,7 @@ def gatherTests(regexps=None, no_network=False):
 	for file_ in os.listdir(
 		os.path.abspath(os.path.dirname(action_d.__file__))):
 		if file_.startswith("test_") and file_.endswith(".py"):
-			if no_network and file_ in ['test_badips.py']: #pragma: no cover
+			if no_network and file_ in ['test_badips.py','test_smtp.py']: #pragma: no cover
 				# Test required network
 				continue
 			tests.addTest(testloader.loadTestsFromName(

fail2ban/version.py

@@ -21,7 +21,7 @@
 #
 
 __author__ = "Cyril Jaquier, Yaroslav Halchenko, Steven Hiscocks, Daniel Black"
-__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2011-2014 Yaroslav Halchenko, 2013-2013 Steven Hiscocks, Daniel Black"
+__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2005-2015 Yaroslav Halchenko, 2013-2014 Steven Hiscocks, Daniel Black"
 __license__ = "GPL-v2+"
 
-version = "0.9.1.dev"
+version = "0.9.2"

files/debian-initd

@@ -3,8 +3,8 @@
 # Provides:          fail2ban
 # Required-Start:    $local_fs $remote_fs
 # Required-Stop:     $local_fs $remote_fs
-# Should-Start:      $time $network $syslog iptables firehol shorewall ipmasq arno-iptables-firewall iptables-persistent ferm
+# Should-Start:      $time $network $syslog $named iptables firehol shorewall ipmasq arno-iptables-firewall iptables-persistent ferm ufw
-# Should-Stop:       $network $syslog iptables firehol shorewall ipmasq arno-iptables-firewall iptables-persistent ferm
+# Should-Stop:       $network $syslog $named iptables firehol shorewall ipmasq arno-iptables-firewall iptables-persistent ferm ufw
 # Default-Start:     2 3 4 5
 # Default-Stop:      0 1 6
 # Short-Description: Start/stop fail2ban

man/fail2ban-client.1

@@ -1,12 +1,12 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.46.4.
-.TH FAIL2BAN-CLIENT "1" "October 2014" "fail2ban-client v0.9.1" "User Commands"
+.TH FAIL2BAN-CLIENT "1" "April 2015" "fail2ban-client v0.9.2" "User Commands"
 .SH NAME
 fail2ban-client \- configure and control the server
 .SH SYNOPSIS
 .B fail2ban-client
 [\fI\,OPTIONS\/\fR] \fI\,<COMMAND>\/\fR
 .SH DESCRIPTION
-Fail2Ban v0.9.1 reads log file that contains password failure report
+Fail2Ban v0.9.2 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .SH OPTIONS
 .TP
@@ -93,6 +93,14 @@ file
 \fBget logtarget\fR
 gets logging target
 .TP
+\fBset syslogsocket auto|<SOCKET>\fR
+sets the syslog socket path to
+auto or <SOCKET>. Only used if
+logtarget is SYSLOG
+.TP
+\fBget syslogsocket\fR
+gets syslog socket path
+.TP
 \fBflushlogs\fR
 flushes the logtarget if a file
 and reopens it. For log rotation.
@@ -128,8 +136,10 @@ starts the jail <JAIL>
 stops the jail <JAIL>. The jail is
 removed
 .TP
-\fBstatus <JAIL>\fR
+\fBstatus <JAIL> [FLAVOR]\fR
-gets the current status of <JAIL>
+gets the current status of <JAIL>,
+with optional flavor or extended
+info
 .IP
 JAIL CONFIGURATION
 .TP

man/fail2ban-regex.1

@@ -1,5 +1,5 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.46.4.
-.TH FAIL2BAN-REGEX "1" "October 2014" "fail2ban-regex 0.9.1" "User Commands"
+.TH FAIL2BAN-REGEX "1" "April 2015" "fail2ban-regex 0.9.2" "User Commands"
 .SH NAME
 fail2ban-regex \- test Fail2ban "failregex" option
 .SH SYNOPSIS

man/fail2ban-server.1

@@ -1,12 +1,12 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.46.4.
-.TH FAIL2BAN-SERVER "1" "October 2014" "fail2ban-server v0.9.1" "User Commands"
+.TH FAIL2BAN-SERVER "1" "April 2015" "fail2ban-server v0.9.2" "User Commands"
 .SH NAME
 fail2ban-server \- start the server
 .SH SYNOPSIS
 .B fail2ban-server
 [\fI\,OPTIONS\/\fR]
 .SH DESCRIPTION
-Fail2Ban v0.9.1 reads log file that contains password failure report
+Fail2Ban v0.9.2 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .PP
 Only use this command for debugging purpose. Start the server with

setup.py

@@ -124,6 +124,9 @@ setup(
 		('/etc/fail2ban/filter.d',
 			glob("config/filter.d/*.conf")
 		),
+		('/etc/fail2ban/filter.d/ignorecommands',
+			glob("config/filter.d/ignorecommands/*")
+		),
 		('/etc/fail2ban/action.d',
 			glob("config/action.d/*.conf") +
 			glob("config/action.d/*.py")