Merge tag '0.10.2' into debian

* tag '0.10.2': (623 commits)
  prepare release: bump version, update ChangeLog, man's and MANIFEST etc.
  ChangeLog update
  action.d/pf.conf: compatibility fix - recognizes that parameter `port` specified as empty, with or without braces (should be more backwards compatible to 0.9 now).
  regex rewritten: a bit fewer vulnerable now and using non-capturing groups, test-cases extended in order to cover trying of injection on user name
  nginx-http-auth: match usernames with spaces
  regex updated using non-capturing groups
  extended test-cases to cover new log-format (http_auth -> mod_auth)
  Update lighttpd-auth.conf
  file-filter's: provide stop function in order to explicitly delete/stop monitoring of each file.
  Remove annoying error-message "rm_watch: cannot remove WD=2, Errno=Invalid argument (EINVAL)", logged from pyinotify-module if rm_watch called with non-existing watch file descriptor (probably multi-threaded issue by dual-remove). Closes gh-1865
  should fix sporadic coverage decrease (don't cover "return", because too sporadic to get idle in pyinotify-callback);
  fixed restoring sane environment (via stop/start) if invariant check failed: bypass possible errors in stop (if start/check succeeded hereafter); test cases extended to cover such situation. Closes gh-1997
  action.d/hostsdeny.conf: actionunban rewritten using sed, also dots in IP were escaped now.
  micro-fix: delete temporary file (forgotten in test-case `test_move_dir` by reassign to directory)
  Update ChangeLog
  stop ban of legitimate users with multiple public keys (e. g. git, etc), thereby differentiate between "invalid user" (going banned earlier) and valid users with public keys, for which the rejects of not valid public keys (failures) will be retarded up to "Too many authentication failures" resp. disconnect without success (accepted public key).
  filter.d/sendmail-auth.conf - extended daemon for Fedora 24/RHEL - the daemon name is "sendmail" (gh-1632)
  test cases extended in order to cover `firewallcmd-ipset` with `allports`
  Update ChangeLog
  firewallcmd-ipset-allports: implemented in `action.d/firewallcmd-ipset.conf` now (`action.d/firewallcmd-ipset-allports.conf` removed), usage:
  ...

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,5 +1,7 @@
 Before submitting your PR, please review the following checklist:
 
+- [ ] **CHOOSE CORRECT BRANCH**: if filing a bugfix/enhancement
+      against 0.9.x series, choose `master` branch
 - [ ] **CONSIDER adding a unit test** if your PR resolves an issue
 - [ ] **LIST ISSUES** this PR resolves
 - [ ] **MAKE SURE** this PR doesn't break existing tests

.travis.yml

@@ -10,12 +10,16 @@ python:
   # - 3.2
   - 3.3
   - 3.4
+  - 3.5
+  - 3.6
+  - 3.7-dev
   # disabled since setuptools dropped support for Python 3.0 - 3.2
   # - pypy3
-  - pypy3.3-5.2-alpha1
+  - pypy3.3-5.5-alpha
 before_install:
-  - if [[ $TRAVIS_PYTHON_VERSION == 2* || $TRAVIS_PYTHON_VERSION == 'pypy' ]]; then export F2B_PY_2=true && echo "Set F2B_PY_2"; fi
+  - echo "running under $TRAVIS_PYTHON_VERSION"
-  - if [[ $TRAVIS_PYTHON_VERSION == 3* || $TRAVIS_PYTHON_VERSION == 'pypy3' ]]; then export F2B_PY_3=true && echo "Set F2B_PY_3"; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == 2* || $TRAVIS_PYTHON_VERSION == pypy* && $TRAVIS_PYTHON_VERSION != pypy3* ]]; then export F2B_PY_2=true && echo "Set F2B_PY_2"; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == 3* || $TRAVIS_PYTHON_VERSION == pypy3* ]]; then export F2B_PY_3=true && echo "Set F2B_PY_3"; fi
   - travis_retry sudo apt-get update -qq
   # Set this so sudo executes the correct python binary
   #   Anything not using sudo will already have the correct environment
@@ -39,8 +43,8 @@ before_script:
 script:
   # Keep the legacy setup.py test approach of checking coverage for python2
   - if [[ "$F2B_PY_2" ]]; then coverage run setup.py test; fi
-  # Coverage doesn't pick up setup.py test with python3, so run it directly
+  # Coverage doesn't pick up setup.py test with python3, so run it directly (with same verbosity as from setup)
-  - if [[ "$F2B_PY_3" ]]; then coverage run bin/fail2ban-testcases; fi
+  - if [[ "$F2B_PY_3" ]]; then coverage run bin/fail2ban-testcases --verbosity=2; fi
   # Use $VENV_BIN (not python) or else sudo will always run the system's python (2.7)
   - sudo $VENV_BIN/pip install .
   # Doc files should get installed on Travis under Linux

ChangeLog

@@ -6,14 +6,490 @@
 Fail2Ban: Changelog
 ===================
 
+Incompatibility list (compared to v.0.9):
+-----------
 
-ver. 0.9.7 (2017/05/11) - awaiting-victory
+* Filter (or `failregex`) internal capture-groups:
+
+  - If you've your own `failregex` or custom filters using conditional match `(?P=host)`, you should
+    rewrite the regex like in example below resp. using `(?:(?P=ip4)|(?P=ip6)` instead of `(?P=host)`
+    (or `(?:(?P=ip4)|(?P=ip6)|(?P=dns))` corresponding your `usedns` and `raw` settings).
+
+    Of course you can always define your own capture-group (like below `_cond_ip_`) to do this.
+    ```
+    testln="1500000000 failure from 192.0.2.1: bad host 192.0.2.1"
+    fail2ban-regex "$testln" "^\s*failure from (?P<_cond_ip_><HOST>): bad host (?P=_cond_ip_)$"
+    ```
+  - New internal groups (currently reserved for internal usage):
+    `ip4`, `ip6`, `dns`, `fid`, `fport`, additionally `user` and another captures in lower case if
+    mapping from tag `<F-*>` used in failregex (e. g. `user` by `<F-USER>`).
+
+* v.0.10 uses more precise date template handling, that can be theoretically incompatible to some
+  user configurations resp. `datepattern`.
+
+* Since v0.10 fail2ban supports the matching of the IPv6 addresses, but not all ban actions are
+  IPv6-capable now.
+
+
+ver. 0.10.2 (2018/01/18) - nothing-burns-like-the-cold
+-----------
+
+### Incompatibility list:
+* The configuration for jails using banaction `pf` can be incompatible after upgrade, because pf-action uses
+  anchors now (see `action.d/pf.conf` for more information). If you want use obsolete handling without anchors,
+  just rewrite it in the `jail.local` by overwrite of `pfctl` parameter, e. g. like `banaction = pf[pfctl="pfctl"]`.
+
+### Fixes
+* Fixed logging to systemd-journal: new logtarget value SYSOUT can be used instead of STDOUT, to avoid 
+  write of the time-stamp, if logging to systemd-journal from foreground mode (gh-1876)
+* Fixed recognition of the new date-format on mysqld-auth filter (gh-1639)
+* jail.conf: port `imap3` replaced with `imap` everywhere, since imap3 is not a standard port and old rarely 
+  (if ever) used and can missing on some systems (e. g. debian stretch), see gh-1942.
+* config/paths-common.conf: added missing initial values (and small normalization in config/paths-*.conf)
+  in order to avoid errors while interpolating (e. g. starting with systemd-backend), see gh-1955.
+* `action.d/pf.conf`: 
+  - fixed syntax error in achnor definition (documentation, see gh-1919);
+  - enclose ports in braces for multiport jails (see gh-1925);
+* `action.d/firewallcmd-ipset.conf`: fixed create of set for ipv6 (missing `family inet6`, gh-1990)
+* `filter.d/sshd.conf`:
+  - extended failregex for modes "extra"/"aggressive": now finds all possible (also future)
+    forms of "no matching (cipher|mac|MAC|compression method|key exchange method|host key type) found", 
+    see "ssherr.c" for all possible SSH_ERR_..._ALG_MATCH errors (gh-1943, gh-1944);
+  - fixed failregex in order to avoid banning of legitimate users with multiple public keys (gh-2014, gh-1263);
+
+### New Features
+* datedetector: extended default date-patterns (allows extra space between the date and time stamps);
+  introduces 2 new format directives (with corresponding %Ex prefix for more precise parsing):
+  - %k - one- or two-digit number giving the hour of the day (0-23) on a 24-hour clock,
+    (corresponds %H, but allows space if not zero-padded).
+  - %l - one- or two-digit number giving the hour of the day (12-11) on a 12-hour clock,
+    (corresponds %I, but allows space if not zero-padded).
+* `filter.d/exim.conf`: added mode `aggressive` to ban flood resp. DDOS-similar failures (gh-1983);
+* New Actions:
+  - `action.d/nginx-block-map.conf` - in order to ban not IP-related tickets via nginx (session blacklisting in
+    nginx-location with map-file);
+
+### Enhancements
+* jail.conf: extended with new parameter `mode` for the filters supporting it (gh-1988);
+* action.d/pf.conf: extended with bulk-unban, command `actionflush` in order to flush all bans at once.
+* Introduced new parameters for logging within fail2ban-server (gh-1980).
+  Usage `logtarget = target[facility=..., datetime=on|off, format="..."]`:
+  - `facility` - specify syslog facility (default `daemon`, see https://docs.python.org/2/library/logging.handlers.html#sysloghandler
+     for the list of facilities);
+  - `datetime` - add date-time to the message (default on, ignored if `format` specified);
+  - `format` - specify own format how it will be logged, for example for short-log into STDOUT:
+      `fail2ban-server -f --logtarget 'stdout[format="%(relativeCreated)5d | %(message)s"]' start`;
+* Automatically recover or recreate corrupt persistent database (e. g. if failed to open with 
+  'database disk image is malformed'). Fail2ban will create a backup, try to repair the database,
+  if repair fails - recreate new database (gh-1465, gh-2004).
+
+
+ver. 0.10.1 (2017/10/12) - succeeded-before-friday-the-13th
+-----------
+
+### Fixes
+* fix Gentoo init script's shebang to use openrc-run instead of runscript (gh-1891)
+* jail "pass2allow-ftp" supply blocktype and returntype parameters to the action (gh-1884)
+* avoid using "ANSI_X3.4-1968" as preferred encoding (if missing environment variables 
+  'LANGUAGE', 'LC_ALL', 'LC_CTYPE', and 'LANG', see gh-1587).
+* action.d/pf.conf: several fixes for pf-action like anchoring, etc. (see gh-1866, gh-1867);
+* fixed ignorself issue "Retrieving own IPs of localhost failed: inet_pton() argument 2 must be string, not int" (see gh-1865);
+* fixed tags `<fq-hostname>` and `<sh-hostname>`, could be used without ticket (a. g. in `actionstart` etc., gh-1859).
+
+* setup.py: fixed several setup facilities (gh-1874):
+  - don't check return code by dry-run: returns 256 on some python/setuptool versions;
+  - `files/fail2ban.service` renamed as template to `files/fail2ban.service.in`;
+  - setup process generates `build/fail2ban.service` from `files/fail2ban.service.in` using distribution related bin-path;
+  - bug-fixing by running setup with option `--dry-run`;
+
+### New Features
+* introduced new command-line options `--dp`, `--dump-pretty` to dump the configuration using more
+  human readable representation (opposite to `-d`);
+
+### Enhancements
+* nftables actions are IPv6-capable now (gh-1893)
+* filter.d/dovecot.conf: introduced mode `aggressive` for cases like "disconnected before auth was ready" (gh-1880)
+
+
+ver. 0.10.0 (2017/08/09) - long-awaited 0.10th version
+-----------
+
+TODO: implementing of options resp. other tasks from PR #1346
+      documentation should be extended (new options, etc)
+      
+### Fixes
+* `filter.d/apache-auth.conf`:
+  - better failure recognition using short form of regex (url/referer are foreign inputs, see gh-1645)
+* `filter.d/apache-common.conf` (`filter.d/apache-*.conf`):
+  - support of apache log-format if logging into syslog/systemd (gh-1695), using parameter `logging`,
+    parameter usage for jail:
+      filter = apache-auth[logging=syslog]
+    parameter usage for `apache-common.local`:
+      logging = syslog
+* `filter.d/pam-generic.conf`:
+  - [grave] injection on user name to host fixed
+* `filter.d/sshd.conf`:
+  - rewritten using `prefregex` and used MLFID-related multi-line parsing
+    (by using tag `<F-MLFID>` instead of buffering with `maxlines`);
+  - optional parameter `mode` rewritten: normal (default), ddos, extra or aggressive (combines all),
+    see sshd for regex details)
+* `filter.d/sendmail-reject.conf`:
+  - rewritten using `prefregex` and used MLFID-related multi-line parsing;
+  - optional parameter `mode` introduced: normal (default), extra or aggressive
+* `filter.d/haproxy-http-auth`: do not mistake client port for part of an IPv6 address (gh-1745)
+* `filter.d/postfix.conf`:
+    - updated to latest postfix formats
+    - joined several postfix filter together (normalized and optimized version, gh-1825)
+    - introduced new parameter `mode` (see gh-1825): more (default, combines normal and rbl), auth, normal,
+      rbl, ddos, extra or aggressive (combines all)
+    - postfix postscreen (resp. other RBL's compatibility fix, gh-1764, gh-1825)
+* `filter.d/postfix-rbl.conf`: removed (replaced with `postfix[mode=rbl]`)
+* `filter.d/postfix-sasl.conf`: removed (replaced with `postfix[mode=auth]`)
+* `filter.d/roundcube-auth.conf`:
+    - fixed regex when `X-Real-IP` or/and `X-Forwarded-For` are present after host (gh-1303);
+    - fixed regex when logging authentication errors to journal instead to a local file (gh-1159);
+    - additionally fixed more complex injections on username (e. g. using dot after fake host).
+* `filter.d/ejabberd-auth.conf`: fixed failregex - accept new log-format (gh-993)
+* `action.d/complain.conf`
+  - fixed using new tag `<ip-rev>` (sh/dash compliant now)
+* `action.d/sendmail-geoip-lines.conf`
+  - fixed using new tag `<ip-host>` (without external command execution)
+* fail2ban-regex: fixed matched output by multi-line (buffered) parsing
+* fail2ban-regex: support for multi-line debuggex URL implemented (gh-422)
+* fixed ipv6-action errors on systems not supporting ipv6 and vice versa (gh-1741)
+* fixed directory-based log-rotate for pyinotify-backend (gh-1778)
+
+### New Features
+* New Actions:
+
+* New Filters:
+
+### Enhancements
+* Introduced new filter option `prefregex` for pre-filtering using single regular expression (gh-1698);
+* Many times faster and fewer CPU-hungry because of parsing with `maxlines=1`, so without 
+  line buffering (scrolling of the buffer-window).
+  Combination of tags `<F-MLFID>` and `<F-NOFAIL>` can be used now to process multi-line logs
+  using single-line expressions:
+  - tag `<F-MLFID>`: used to identify resp. store failure info for groups of log-lines with the same
+    identifier (e. g. combined failure-info for the same conn-id by `<F-MLFID>(?:conn-id)</F-MLFID>`,
+    see sshd.conf for example);
+  - tag `<F-MLFFORGET>`: can be used as mark to forget current multi-line MLFID (e. g. by connection 
+    closed, reset or disconnect etc);
+  - tag `<F-NOFAIL>`: used as mark for no-failure (helper to accumulate common failure-info, 
+    e. g. from lines that contain IP-address);
+  Opposite to obsolete multi-line parsing (using buffering with `maxlines`) it is more precise and 
+  can recognize multiple failure attempts within the same connection (MLFID).
+* Several filters optimized with pre-filtering using new option `prefregex`, and multiline filter 
+  using `<F-MLFID>` + `<F-NOFAIL>` combination;
+* Exposes filter group captures in actions (non-recursive interpolation of tags `<F-...>`, 
+  see gh-1698, gh-1110)
+* Some filters extended with user name (can be used in gh-1243 to distinguish IP and user,
+  resp. to remove after success login the user-related failures only);
+* Safer, more stable and faster replaceTag interpolation (switched from cycle over all tags
+  to re.sub with callable)
+* substituteRecursiveTags optimization + moved in helpers facilities (because currently used 
+  commonly in server and in client)
+* New tags (usable in actions):
+  - `<fid>` - failure identifier (if raw resp. failures without IP address)
+  - `<ip-rev>` - PTR reversed representation of IP address
+  - `<ip-host>` - host name of the IP address
+  - `<F-...>` - interpolates to the corresponding filter group capture `...`
+  - `<fq-hostname>` - fully-qualified name of host (the same as `$(hostname -f)`)
+  - `<sh-hostname>` - short hostname (the same as `$(uname -n)`)
+* Allow to use filter options by `fail2ban-regex`, example:
+  fail2ban-regex text.log "sshd[mode=aggressive]"
+* Samples test case factory extended with filter options - dict in JSON to control 
+  filter options (e. g. mode, etc.):
+  # filterOptions: {"mode": "aggressive"}
+* Introduced new jail option "ignoreself", specifies whether the local resp. own IP addresses 
+  should be ignored (default is true). Fail2ban will not ban a host which matches such addresses.
+  Option "ignoreip" affects additionally to "ignoreself" and don't need to include the DNS 
+  resp. IPs of the host self.
+* Regex will be compiled as MULTILINE only if needed (buffering with `maxlines` > 1), that enables:
+  - to improve performance by the single line parsing (see gh-1733);
+  - make regex more precise (because distinguish between anchors `^`/`$` for the begin/end of string
+    and the new-line character '\n', e. g. if coming from filters (like systemd journal) that allow
+    the parsing of log-entries contain new-line chars (as single entry);
+  - if multiline regex however expected (by single-line parsing without buffering) - prefix `(?m)` 
+    could be used in regex to enable it;
+* Implemented execution of `actionstart` on demand (conditional), if action depends on `family` (gh-1742):
+  - new action parameter `actionstart_on_demand` (bool) can be set to prevent/allow starting action
+    on demand (default retrieved automatically, if some conditional parameter `param?family=...` 
+    presents in action properties), see `action.d/pf.conf` for example;
+  - additionally `actionstop` will be executed only for families previously executing `actionstart`
+    (starting on demand only)
+* Introduced new command `actionflush`: executed in order to flush all bans at once 
+  e. g. by unban all, reload with removing action, stop, shutdown the system (gh-1743),
+  the actions having `actionflush` do not execute `actionunban` for each single ticket
+* Add new command `actionflush` default for several iptables/iptables-ipset actions (and common include);
+* Add new jail option `logtimezone` to force the timezone on log lines that don't have an explicit one (gh-1773)
+* Implemented zone abbreviations (like CET, CEST, etc.) and abbr+-offset functionality (accept zones
+  like 'CET+0100'), for the list of abbreviations see strptime.TZ_STR;
+* Introduced new option `--timezone` (resp. `--TZ`) for `fail2ban-regex`.
+* Tokens `%z` and `%Z` are changed (more precise now);
+* Introduced new tokens `%Exz` and `%ExZ` that fully support zone abbreviations and/or offset-based
+  zones (implemented as enhancement using custom `datepattern`, because may be too dangerous for default
+  patterns and tokens like `%z`);
+  Note: the extended tokens supported zone abbreviations, but it can parse 1 or 3-5 char(s) in lowercase.
+        Don't use them in default date-patterns (if not anchored, few precise resp. optional).
+        Because python currently does not support mixing of case-sensitive with case-insensitive matching,
+	the TZ (in uppercase) cannot be combined with `%a`/`%b` etc (that are currently case-insensitive),
+	to avoid invalid date-time recognition in strings like '11-Aug-2013 03:36:11.372 error ...' with
+	wrong TZ "error".
+        Hence `%z` currently match literal Z|UTC|GMT only (and offset-based), and `%Exz` - all zone 
+	abbreviations.
+* `filter.d/courier-auth.conf`: support failed logins with method only
+* Config reader's: introduced new syntax `%(section/option)s`, in opposite to extended interpolation of
+  python 3 `${section:option}` work with all supported python version in fail2ban and this syntax is 
+  like our another features like `%(known/option)s`, etc. (gh-1750)
+* Variable `default_backend` switched to `%(default/backend)s`, so totally backwards compatible now,
+  but now the setting of parameter `backend` in default section of `jail.local` can overwrite default
+  backend also (see gh-1750). In the future versions parameter `default_backend` can be removed (incompatibility, 
+  possibly some distributions affected).
+
+
+ver. 0.10.0-alpha-1 (2016/07/14) - ipv6-support-etc
+-----------
+
+### Fixes
+* [Grave] memory leak's fixed (gh-1277, gh-1234)
+* [Grave] Misleading date patterns defined more precisely (using extended syntax
+  `%Ex[mdHMS]` for exact two-digit match or e. g. `%ExY` as more precise year 
+  pattern, within same century of last year and the next 3 years)
+* [Grave] extends date detector template with distance (position of match in 
+  log-line), to prevent grave collision using (re)ordered template list (e.g.
+  find-spot of wrong date-match inside foreign input, misleading date patterns
+  by ambiguous formats, etc.)
+* Distance collision check always prefers template with shortest distance
+  (left for right) if date pattern is not anchored
+* Tricky bug fix: last position of log file will be never retrieved (gh-795),
+  because of CASCADE all log entries will be deleted from logs table together with jail, 
+  if used "INSERT OR REPLACE" statement
+* Asyncserver (asyncore) code fixed and test cases repaired (again gh-161)
+* testSocket: sporadical bug repaired - wait for server thread starts a socket (listener)
+* testExecuteTimeoutWithNastyChildren: sporadical bug repaired - wait for pid file inside bash,
+  kill tree in any case (gh-1155)
+* Fixed high-load of pyinotify-backend,
+  see https://github.com/fail2ban/fail2ban/issues/885#issuecomment-248964591
+* Database: stability fix - repack cursor iterator as long as locked
+* File filter backends: stability fix for sporadically errors - always close file
+  handle, otherwise may be locked (prevent log-rotate, etc.)
+* Pyinotify-backend: stability fix for sporadically errors in multi-threaded
+  environment (without lock)
+* Fixed sporadically error in testCymruInfoNxdomain, because of unsorted values
+* Misleading errors logged from ignorecommand in success case on retcode 1 (gh-1194)
+* fail2ban.service - systemd service updated (gh-1618):
+  - starting service in normal mode (without forking)
+  - does not restart if service exited normally (exit-code 0, e.g. stopped via fail2ban-client)
+  - does not restart if service can not start (exit-code 255, e.g. wrong configuration, etc.)
+  - service can be additionally started/stopped with commands (fail2ban-client, fail2ban-server)
+  - automatically creates `/var/run/fail2ban` directory before start fail2ban 
+    (systems with virtual resp. memory-based FS for `/var/run`), see gh-1531
+  - if fail2ban running as systemd-service, for logging to the systemd-journal, 
+    the `logtarget` could be set to STDOUT
+  - value `logtarget` for system targets allowed also in lowercase (stdout, stderr, syslog, etc.)
+* Fixed UTC/GMT named time zone, using `%Z` and `%z` patterns 
+  (special case with 0 zone offset, see gh-1575)
+* `filter.d/freeswitch.conf`
+    - Optional prefixes (server, daemon, dual time) if systemd daemon logs used (gh-1548)
+    - User part rewritten to accept IPv6 resp. domain after "@" (gh-1548)
+
+### New Features
+* IPv6 support:
+    - IP addresses are now handled as objects rather than strings capable for 
+      handling both address types IPv4 and IPv6
+    - iptables related actions have been amended to support IPv6 specific actions
+      additionally
+    - hostsdeny and route actions have been tested to be aware of v4 and v6 already
+    - pf action for *BSD systems has been improved and supports now also v4 and v6
+    - name resolution is now working for either address type
+    - new conditional section functionality used in config resp. includes:
+      - [Init?family=inet4] - IPv4 qualified hosts only
+      - [Init?family=inet6] - IPv6 qualified hosts only
+* New reload functionality (now totally without restart, unbanning/rebanning, etc.),
+  see gh-1557
+* Several commands extended and new commands introduced:
+  - `restart [--unban] [--if-exists] <JAIL>` - restarts the jail \<JAIL\>
+    (alias for `reload --restart ... <JAIL>`)
+  - `reload [--restart] [--unban] [--all]` - reloads the configuration without restarting
+    of the server, the option `--restart` activates completely restarting of affected jails,
+    thereby can unban IP addresses (if option `--unban` specified)
+  - `reload [--restart] [--unban] [--if-exists] <JAIL>` - reloads the jail \<JAIL\>,
+    or restarts it (if option `--restart` specified), at the same time unbans all IP addresses
+    banned in this jail, if option `--unban` specified
+  - `unban --all` - unbans all IP addresses (in all jails and database)
+  - `unban <IP> ... <IP>` - unbans \<IP\> (in all jails and database) (see gh-1388)
+  - introduced new option `-t` or `--test` to test configuration resp. start server only 
+    if configuration is clean (fails by wrong configured jails if option `-t` specified)
+* New command action parameter `actionrepair` - command executed in order to restore
+  sane environment in error case of `actioncheck`.
+* Reporting via abuseipdb.com:
+  - Bans can now be reported to abuseipdb
+  - Catagories must be set in the config
+  - Relevant log lines included in report
+
+### Enhancements
+* Huge increasing of fail2ban performance and especially test-cases performance (see gh-1109)
+* Datedetector: in-place reordering using hits and last used time: 
+  matchTime, template list etc. rewritten because of performance degradation
+* Prevent out of memory situation if many IP's makes extremely many failures (maxEntries)
+* Introduced string to seconds (str2seconds) for configuration entries with time,
+  use `1h` instead of `3600`, `1d` instead of `86400`, etc
+* seekToTime - prevent completely read of big files first time (after start of service), 
+  initial seek to start time using half-interval search algorithm (see issue gh-795)
+* Ticket and some other modules prepared to easy merge with newest version of 'ban-time-incr'
+* Cache dnsToIp, ipToName to prevent long wait during retrieving of ip/name,
+  especially for wrong dns or lazy dns-system
+* FailManager memory-optimization: increases performance, 
+  prevents memory leakage, because don't copy failures list on some operations
+* fail2ban-testcases - new options introduced:
+    - `-f`, `--fast` to decrease wait intervals, avoid passive waiting, and skip
+      few very slow test cases (implied memory database, see `-m` and no gamin tests `-g`)
+    - `-g`, `--no-gamin` to prevent running of tests that require the gamin (slow)
+    - `-m`, `--memory-db` - run database tests using memory instead of file
+    - `-i`, `--ignore` - negate [regexps] filter to ignore tests matched specified regexps
+* Background servicing: prevents memory leak on some platforms/python versions, using forced GC
+  in periodic intervals (latency and threshold)
+* executeCmd partially moved from action to new module utils
+* Several functionality of class `DNSUtils` moved to new class `IPAddr`, 
+  both classes moved to new module `ipdns`
+* Pseudo-conditional section introduced, for conditional substitution resp. 
+  evaluation of parameters for different family qualified hosts, 
+  syntax `[Section?family=inet6]` (currently use for IPv6-support only).
+* All the backends were rewritten to get reload-possibility, performance increased,
+  so fewer greedy regarding cpu- resp. system-load now
+* Numeric log-level allowed now in server (resp. fail2ban.conf);
+* Implemented better error handling in some multi-threaded routines; shutdown of jails
+  rewritten (faster and safer, does not breaks shutdown process if some error occurred)
+* Possibility for overwriting some configuration options (read with config-readers)
+  with command line option, e. g.:
+```bash
+## start server with DEBUG log-level (ignore level read from fail2ban.conf):
+fail2ban-client --loglevel DEBUG start
+## or
+fail2ban-server -c /cfg/path --loglevel DEBUG start
+## keep server log-level by reload (without restart it)
+fail2ban-client --loglevel DEBUG reload
+## switch log-level back to INFO:
+fail2ban-client set loglevel INFO
+```
+* Optimized BanManager: increase performance, fewer system load, try to prevent
+  memory leakage:
+  - better ban/unban handling within actions (e.g. used dict instead of list)
+  - don't copy bans resp. its list on some operations;
+  - added new unbantime handling to relieve unBanList (prevent permanent
+    searching for tickets to unban)
+  - prefer failure-ID as identifier of the ticket to its IP (most of the time
+    the same, but it can be something else e.g. user name in some complex jails,
+    as introduced in 0.10)
+* Regexp enhancements:
+  - build replacement of `<HOST>` substitution corresponding parameter
+    `usedns` - dns-part will be added only if `usedns` is not `no`,
+    also using fail2ban-regex
+  - new replacement for `<ADDR>` in opposition to `<HOST>`, for separate
+    usage of 2 address groups only (regardless of `usedns`), `ip4` and `ip6`
+    together, without host (dns)
+* Misconfigured jails don't prevent fail2ban from starting, server starts 
+  nevertheless, as long as one jail was successful configured (gh-1619)
+  Message about wrong jail configuration logged in client log (stdout, systemd
+  journal etc.) and in server log with error level
+* More precise date template handling (WARNING: theoretically possible incompatibilities):
+  - datedetector rewritten more strict as earlier;
+  - default templates can be specified exacter using prefix/suffix syntax (via `datepattern`);
+  - more as one date pattern can be specified using option `datepattern` now 
+    (new-line separated);
+  - some default options like `datepattern` can be specified directly in 
+    section `[Definition]`, that avoids contrary usage of unnecessarily `[Init]`
+    section, because of performance (each extra section costs time);
+  - option `datepattern` can be specified in jail also (e. g. jails without filters 
+    or custom log-format, new-line separated for multiple patterns);
+  - if first unnamed group specified in pattern, only this will be cut out from
+    search log-line (e. g.: `^date:[({DATE})]` will cut out only datetime match 
+    pattern, and leaves `date:[] ...` for searching in filter);
+  - faster match and fewer searching of appropriate templates
+    (DateDetector.matchTime calls rarer DateTemplate.matchDate now);
+  - several standard filters extended with exact prefixed or anchored date templates;
+* Added possibility to recognize restored state of the tickets (see gh-1669).
+  New option `norestored` introduced, to ignore restored tickets (after restart).
+  To avoid execution of ban/unban for the restored tickets, `norestored = true`
+  could be added in definition section of action.
+  For conditional usage in the shell-based actions an interpolation `<restored>` 
+  could be used also. E. g. it is enough to add following script-piece at begin
+  of `actionban` (or `actionunban`) to prevent execution:
+  `if [ '<restored>' = '1' ]; then exit 0; fi;`
+  Several actions extended now using `norestored` option:
+  - complain.conf
+  - dshield.conf
+  - mail-buffered.conf
+  - mail-whois-lines.conf
+  - mail-whois.conf
+  - mail.conf
+  - sendmail-buffered.conf
+  - sendmail-geoip-lines.conf
+  - sendmail-whois-ipjailmatches.conf
+  - sendmail-whois-ipmatches.conf
+  - sendmail-whois-lines.conf
+  - sendmail-whois-matches.conf
+  - sendmail-whois.conf
+  - sendmail.conf
+  - smtp.py
+  - xarf-login-attack.conf
+* fail2ban-testcases:
+  - `assertLogged` extended with parameter wait (to wait up to specified timeout,
+    before we throw assert exception) + test cases rewritten using that
+  - added `assertDictEqual` for compatibility to early python versions (< 2.7);
+  - new `with_foreground_server_thread` decorator to test several client/server commands
+
+
+ver. 0.9.8 (2016/XX/XXX) - wanna-be-released
 -----------
 
 0.9.x line is no longer heavily developed.  If you are interested in
 new features (e.g. IPv6 support), please consider 0.10 branch and its
 releases.
 
+
+### Fixes
+* Fix for systemd-backend: fail2ban hits the ulimit (out of file descriptors), see gh-991.
+  Partially back-ported from v.0.10.
+* action.d/bsd-ipfw.conf
+    - Make the rule number, the action starts looking for a free slot to insert
+      the new rule, configurable (gh-1689)
+    - Replace not posix-compliant grep option: fgrep with `-q` option can cause 
+      141 exit code in some cases (gh-1389)
+* filter.d/apache-overflows.conf:
+    - Fixes resources greedy expression (see gh-1790);
+    - Rewritten without end-anchor ($), because of potential vulnerability on very long URLs.
+* filter.d/apache-badbots.conf - extended to recognize Jorgee Vulnerability Scanner (gh-1882)
+* filter.d/asterisk.conf
+    - fixed failregex AMI Asterisk authentification failed (see gh-1302)
+    - removed invalid (vulnerable) regex blocking IPs using forign data (from header "from")
+      thus not the IP-address that really originates the request (see gh-1927)
+    - fixed failregex for the SQL-injection attempts with single-quotes in connect-string (see gh-2011)
+* filter.d/dovecot.conf:
+    - fixed failregex, see gh-1879 (partially cherry-picked from gh-1880)
+    - extended to match pam_authenticate failures with "Permission denied" (gh-1897)
+* filter.d/exim.conf
+    - fixed failregex for case of flood attempts with `D=0s` (gh-1887)
+    - fixed failregex of "AUTH command used when not advertised" to better handle the foreign
+      input SMTP command (lower/mixed case auth command, prevent injection) (gh-1979)
+* filter.d/postfix-*.conf - added optional port regex (gh-1902)
+* filter.d/sendmail-auth.conf - extended daemon for Fedora 24/RHEL - the daemon name is "sendmail" (gh-1632)
+* filter.d/nginx-http-auth.conf - match usernames with spaces (gh-2015)
+
+### New Features
+
+### Enhancements
+* action.d/cloudflare.conf - Cloudflare API v4 implementation (gh-1651)
+* action.d/firewallcmd-ipset.conf - new parameter `actiontype`, provides `allports` capability (gh-1167)
+* filter.d/kerio.conf - filter extended with new rules (see gh-1455)
+* filter.d/phpmyadmin-syslog.conf - new filter for phpMyAdmin using syslog for auth logging
+* filter.d/zoneminder.conf - new filter for ZoneMinder (gh-1376)
+
+
+ver. 0.9.7 (2017/05/11) - awaiting-victory
+-----------
+
 ### Fixes
 * Fixed a systemd-journal handling in fail2ban-regex (gh-1657)
 * filter.d/sshd.conf

MANIFEST

@@ -3,6 +3,7 @@ bin/fail2ban-regex
 bin/fail2ban-server
 bin/fail2ban-testcases
 ChangeLog
+config/action.d/abuseipdb.conf
 config/action.d/apf.conf
 config/action.d/badips.conf
 config/action.d/badips.py
@@ -13,11 +14,13 @@ config/action.d/complain.conf
 config/action.d/dshield.conf
 config/action.d/dummy.conf
 config/action.d/firewallcmd-allports.conf
+config/action.d/firewallcmd-common.conf
 config/action.d/firewallcmd-ipset.conf
 config/action.d/firewallcmd-multiport.conf
 config/action.d/firewallcmd-new.conf
 config/action.d/firewallcmd-rich-logging.conf
 config/action.d/firewallcmd-rich-rules.conf
+config/action.d/helpers-common.conf
 config/action.d/hostsdeny.conf
 config/action.d/ipfilter.conf
 config/action.d/ipfw.conf
@@ -41,6 +44,7 @@ config/action.d/netscaler.conf
 config/action.d/nftables-allports.conf
 config/action.d/nftables-common.conf
 config/action.d/nftables-multiport.conf
+config/action.d/nginx-block-map.conf
 config/action.d/npf.conf
 config/action.d/nsupdate.conf
 config/action.d/osx-afctl.conf
@@ -99,7 +103,6 @@ config/filter.d/gssftpd.conf
 config/filter.d/guacamole.conf
 config/filter.d/haproxy-http-auth.conf
 config/filter.d/horde.conf
-config/filter.d/ignorecommands
 config/filter.d/ignorecommands/apache-fakegooglebot
 config/filter.d/kerio.conf
 config/filter.d/lighttpd-auth.conf
@@ -118,11 +121,10 @@ config/filter.d/openwebmail.conf
 config/filter.d/oracleims.conf
 config/filter.d/pam-generic.conf
 config/filter.d/perdition.conf
+config/filter.d/phpmyadmin-syslog.conf
 config/filter.d/php-url-fopen.conf
 config/filter.d/portsentry.conf
 config/filter.d/postfix.conf
-config/filter.d/postfix-rbl.conf
-config/filter.d/postfix-sasl.conf
 config/filter.d/proftpd.conf
 config/filter.d/pure-ftpd.conf
 config/filter.d/qmail.conf
@@ -133,16 +135,13 @@ config/filter.d/selinux-common.conf
 config/filter.d/selinux-ssh.conf
 config/filter.d/sendmail-auth.conf
 config/filter.d/sendmail-reject.conf
-config/filter.d/sendmail-spam.conf
 config/filter.d/sieve.conf
 config/filter.d/slapd.conf
 config/filter.d/sogo-auth.conf
 config/filter.d/solid-pop3d.conf
 config/filter.d/squid.conf
 config/filter.d/squirrelmail.conf
-config/filter.d/sshd-aggressive.conf
 config/filter.d/sshd.conf
-config/filter.d/sshd-ddos.conf
 config/filter.d/stunnel.conf
 config/filter.d/suhosin.conf
 config/filter.d/tine20.conf
@@ -151,7 +150,9 @@ config/filter.d/vsftpd.conf
 config/filter.d/webmin-auth.conf
 config/filter.d/wuftpd.conf
 config/filter.d/xinetd-fail.conf
+config/filter.d/zoneminder.conf
 config/jail.conf
+config/paths-arch.conf
 config/paths-common.conf
 config/paths-debian.conf
 config/paths-fedora.conf
@@ -162,7 +163,6 @@ CONTRIBUTING.md
 COPYING
 .coveragerc
 DEVELOP
-doc/run-rootless.txt
 fail2ban-2to3
 fail2ban/client/actionreader.py
 fail2ban/client/beautifier.py
@@ -170,8 +170,11 @@ fail2ban/client/configparserinc.py
 fail2ban/client/configreader.py
 fail2ban/client/configurator.py
 fail2ban/client/csocket.py
+fail2ban/client/fail2banclient.py
+fail2ban/client/fail2bancmdline.py
 fail2ban/client/fail2banreader.py
 fail2ban/client/fail2banregex.py
+fail2ban/client/fail2banserver.py
 fail2ban/client/filterreader.py
 fail2ban/client/__init__.py
 fail2ban/client/jailreader.py
@@ -187,7 +190,6 @@ fail2ban/server/banmanager.py
 fail2ban/server/database.py
 fail2ban/server/datedetector.py
 fail2ban/server/datetemplate.py
-fail2ban/server/faildata.py
 fail2ban/server/failmanager.py
 fail2ban/server/failregex.py
 fail2ban/server/filtergamin.py
@@ -196,7 +198,7 @@ fail2ban/server/filter.py
 fail2ban/server/filterpyinotify.py
 fail2ban/server/filtersystemd.py
 fail2ban/server/__init__.py
-fail2ban/server/iso8601.py
+fail2ban/server/ipdns.py
 fail2ban/server/jail.py
 fail2ban/server/jails.py
 fail2ban/server/jailthread.py
@@ -205,6 +207,7 @@ fail2ban/server/server.py
 fail2ban/server/strptime.py
 fail2ban/server/ticket.py
 fail2ban/server/transmitter.py
+fail2ban/server/utils.py
 fail2ban/setup.py
 fail2ban-testcases-all
 fail2ban-testcases-all-python3
@@ -214,22 +217,20 @@ fail2ban/tests/action_d/test_smtp.py
 fail2ban/tests/actionstestcase.py
 fail2ban/tests/actiontestcase.py
 fail2ban/tests/banmanagertestcase.py
+fail2ban/tests/clientbeautifiertestcase.py
 fail2ban/tests/clientreadertestcase.py
 fail2ban/tests/config/action.d/brokenaction.conf
 fail2ban/tests/config/fail2ban.conf
-fail2ban/tests/config/filter.d/common.conf
 fail2ban/tests/config/filter.d/simple.conf
 fail2ban/tests/config/filter.d/test.conf
 fail2ban/tests/config/filter.d/test.local
 fail2ban/tests/config/filter.d/zzz-generic-example.conf
+fail2ban/tests/config/filter.d/zzz-sshd-obsolete-multiline.conf
 fail2ban/tests/config/jail.conf
-fail2ban/tests/config/paths-common.conf
-fail2ban/tests/config/paths-debian.conf
-fail2ban/tests/config/paths-freebsd.conf
-fail2ban/tests/config/paths-osx.conf
 fail2ban/tests/databasetestcase.py
 fail2ban/tests/datedetectortestcase.py
 fail2ban/tests/dummyjail.py
+fail2ban/tests/fail2banclienttestcase.py
 fail2ban/tests/fail2banregextestcase.py
 fail2ban/tests/failmanagertestcase.py
 fail2ban/tests/files/action.d/action_checkainfo.py
@@ -262,7 +263,6 @@ fail2ban/tests/files/ignorecommand.py
 fail2ban/tests/files/logs/3proxy
 fail2ban/tests/files/logs/apache-auth
 fail2ban/tests/files/logs/apache-badbots
-fail2ban/tests/files/logs/apache-botscripts
 fail2ban/tests/files/logs/apache-botsearch
 fail2ban/tests/files/logs/apache-fakegooglebot
 fail2ban/tests/files/logs/apache-modsecurity
@@ -312,11 +312,10 @@ fail2ban/tests/files/logs/openwebmail
 fail2ban/tests/files/logs/oracleims
 fail2ban/tests/files/logs/pam-generic
 fail2ban/tests/files/logs/perdition
+fail2ban/tests/files/logs/phpmyadmin-syslog
 fail2ban/tests/files/logs/php-url-fopen
 fail2ban/tests/files/logs/portsentry
 fail2ban/tests/files/logs/postfix
-fail2ban/tests/files/logs/postfix-rbl
-fail2ban/tests/files/logs/postfix-sasl
 fail2ban/tests/files/logs/proftpd
 fail2ban/tests/files/logs/pure-ftpd
 fail2ban/tests/files/logs/qmail
@@ -326,7 +325,6 @@ fail2ban/tests/files/logs/screensharingd
 fail2ban/tests/files/logs/selinux-ssh
 fail2ban/tests/files/logs/sendmail-auth
 fail2ban/tests/files/logs/sendmail-reject
-fail2ban/tests/files/logs/sendmail-spam
 fail2ban/tests/files/logs/sieve
 fail2ban/tests/files/logs/slapd
 fail2ban/tests/files/logs/sogo-auth
@@ -334,8 +332,6 @@ fail2ban/tests/files/logs/solid-pop3d
 fail2ban/tests/files/logs/squid
 fail2ban/tests/files/logs/squirrelmail
 fail2ban/tests/files/logs/sshd
-fail2ban/tests/files/logs/sshd-aggressive
-fail2ban/tests/files/logs/sshd-ddos
 fail2ban/tests/files/logs/stunnel
 fail2ban/tests/files/logs/suhosin
 fail2ban/tests/files/logs/tine20
@@ -344,7 +340,10 @@ fail2ban/tests/files/logs/vsftpd
 fail2ban/tests/files/logs/webmin-auth
 fail2ban/tests/files/logs/wuftpd
 fail2ban/tests/files/logs/xinetd-fail
+fail2ban/tests/files/logs/zoneminder
 fail2ban/tests/files/logs/zzz-generic-example
+fail2ban/tests/files/logs/zzz-sshd-obsolete-multiline
+fail2ban/tests/files/testcase01a.log
 fail2ban/tests/files/testcase01.log
 fail2ban/tests/files/testcase02.log
 fail2ban/tests/files/testcase03.log
@@ -353,12 +352,14 @@ fail2ban/tests/files/testcase-journal.log
 fail2ban/tests/files/testcase-multiline.log
 fail2ban/tests/files/testcase-usedns.log
 fail2ban/tests/files/testcase-wrong-char.log
+fail2ban/tests/files/zzz-sshd-obsolete-multiline.log
 fail2ban/tests/filtertestcase.py
 fail2ban/tests/__init__.py
 fail2ban/tests/misctestcase.py
 fail2ban/tests/samplestestcase.py
 fail2ban/tests/servertestcase.py
 fail2ban/tests/sockettestcase.py
+fail2ban/tests/tickettestcase.py
 fail2ban/tests/utils.py
 fail2ban/version.py
 files/bash-completion
@@ -367,7 +368,7 @@ files/cacti/fail2ban_stats.sh
 files/cacti/README
 files/debian-initd
 files/fail2ban-logrotate
-files/fail2ban.service
+files/fail2ban.service.in
 files/fail2ban-tmpfiles.conf
 files/fail2ban.upstart
 files/gen_badbots

README.md

@@ -2,7 +2,7 @@
                         / _|__ _(_) |_  ) |__  __ _ _ _  
                        |  _/ _` | | |/ /| '_ \/ _` | ' \ 
                        |_| \__,_|_|_/___|_.__/\__,_|_||_|
-                       v0.9.7                  2017/05/11
+                       v0.10.2                 2018/01/18
 
 ## Fail2Ban: ban hosts that cause multiple authentication errors
 
@@ -18,8 +18,12 @@ attempts, it cannot eliminate the risk that weak authentication presents.
 Configure services to use only two factor or public/private authentication
 mechanisms if you really want to protect services.
      
+<img src="http://www.worldipv6launch.org/wp-content/themes/ipv6/downloads/World_IPv6_launch_logo.svg" height="52pt"/> | Since v0.10 fail2ban supports the matching of the IPv6 addresses.
+------|------
+
 This README is a quick introduction to Fail2ban. More documentation, FAQ, HOWTOs
-are available in fail2ban(1) manpage and on the website http://www.fail2ban.org
+are available in fail2ban(1) manpage, [Wiki](https://github.com/fail2ban/fail2ban/wiki)
+and on the website http://www.fail2ban.org
 
 Installation:
 -------------
@@ -39,8 +43,8 @@ Optional:
 
 To install, just do:
 
-    tar xvfj fail2ban-0.9.6.tar.bz2
+    tar xvfj fail2ban-0.10.2.tar.bz2
-    cd fail2ban-0.9.6
+    cd fail2ban-0.10.2
     python setup.py install
 
 This will install Fail2Ban into the python library directory. The executable
@@ -73,11 +77,11 @@ fail2ban(1) and jail.conf(5)  manpages for further references.
 Code status:
 ------------
 
-* [![tests status](https://secure.travis-ci.org/fail2ban/fail2ban.png?branch=master)](https://travis-ci.org/fail2ban/fail2ban) travis-ci.org (master branch)
+* [![tests status](https://secure.travis-ci.org/fail2ban/fail2ban.png?branch=0.10)](https://travis-ci.org/fail2ban/fail2ban?branch=0.10) travis-ci.org (0.10 branch) / [![tests status](https://secure.travis-ci.org/fail2ban/fail2ban.png?branch=master)](https://travis-ci.org/fail2ban/fail2ban) travis-ci.org (master branch)
 
-* [![Coverage Status](https://coveralls.io/repos/fail2ban/fail2ban/badge.png?branch=master)](https://coveralls.io/r/fail2ban/fail2ban)
+* [![Coverage Status](https://coveralls.io/repos/fail2ban/fail2ban/badge.png?branch=0.10)](https://coveralls.io/github/fail2ban/fail2ban?branch=0.10)
 
-* [![codecov.io](https://codecov.io/github/fail2ban/fail2ban/coverage.svg?branch=master)](https://codecov.io/github/fail2ban/fail2ban?branch=master)
+* [![codecov.io](https://codecov.io/gh/fail2ban/fail2ban/coverage.svg?branch=0.10)](https://codecov.io/gh/fail2ban/fail2ban/branch/0.10)
 
 Contact:
 --------
@@ -86,7 +90,7 @@ Contact:
 See [CONTRIBUTING.md](https://github.com/fail2ban/fail2ban/blob/master/CONTRIBUTING.md)
 
 ### You just appreciate this program:
-send kudos to the original author ([Cyril Jaquier](mailto: Cyril Jaquier <cyril.jaquier@fail2ban.org>))
+send kudos to the original author ([Cyril Jaquier](mailto:cyril.jaquier@fail2ban.org))
 or *better* to the [mailing list](https://lists.sourceforge.net/lists/listinfo/fail2ban-users)
 since Fail2Ban is "community-driven" for years now.
 

RELEASE

@@ -53,7 +53,7 @@ Preparation
 
   or an alternative for comparison with previous release
 
-     git diff 0.9.6 | grep -B2 'index 0000000..' | grep -B1 'new file mode' | sed -n -e '/^diff /s,.* b/,,gp' >> MANIFEST
+     git diff 0.10.0 | grep -B2 'index 0000000..' | grep -B1 'new file mode' | sed -n -e '/^diff /s,.* b/,,gp' >> MANIFEST
      sort MANIFEST | uniq | sponge MANIFEST
 
 * Run::
@@ -70,7 +70,7 @@ Preparation
 
 * clean up current directory::
 
-    diff -rul --exclude \*.pyc . /tmp/fail2ban-0.9.6/
+    diff -rul --exclude \*.pyc . /tmp/fail2ban-0.10.0/
 
   * Only differences should be files that you don't want distributed.
 
@@ -83,7 +83,7 @@ Preparation
 
   * To generate a list of committers use e.g.::
 
-      git shortlog -sn 0.9.6.. | sed -e 's,^[ 0-9\t]*,,g' | tr '\n' '\|' | sed -e 's:|:, :g'
+      git shortlog -sn 0.10.0.. | sed -e 's,^[ 0-9\t]*,,g' | tr '\n' '\|' | sed -e 's:|:, :g'
 
   * Ensure the top of the ChangeLog has the right version and current date.
   * Ensure the top entry of the ChangeLog has the right version and current date.
@@ -106,7 +106,7 @@ Preparation
 * Tag the release by using a signed (and annotated) tag.  Cut/paste
   release ChangeLog entry as tag annotation::
 
-    git tag -s 0.9.6
+    git tag -s 0.10.0
 
 Pre Release
 ===========
@@ -190,7 +190,7 @@ Post Release
 
 Add the following to the top of the ChangeLog::
 
-  ver. 0.9.8 (2016/XX/XXX) - wanna-be-released
+  ver. 0.10.0 (2016/XX/XXX) - wanna-be-released
   -----------
   
   ### Fixes

THANKS

@@ -12,9 +12,11 @@ Adrien Clerc
 ache
 ag4ve (Shawn)
 Alasdair D. Campbell
+Alexander Koeppe (IPv6 support)
 Alexandre Perrin (kAworu)
 Amir Caspi
 Amy
+Andrew James Collett (ajcollett)
 Andrew St. Jean
 Andrey G. Grozin
 Andy Fragen
@@ -59,6 +61,7 @@ John Thoe
 Jacques Lav!gnotte
 Johannes Weberhofer
 Jason H Martin
+Jeaye Wilkerson
 Jisoo Park
 Joel M Snyder
 Jonathan Kamens
@@ -109,6 +112,8 @@ SATO Kentaro
 Sean DuBois
 Sebastian Arcus
 Serg G. Brester
+Sergey Safarov
+Shaun C.
 Sireyessire
 silviogarbes
 Stefan Tatschner

bin/fail2ban-client

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 # emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
 # vi: set ft=python sts=4 ts=4 sw=4 noet :
 
@@ -18,458 +18,20 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-__author__ = "Cyril Jaquier"
+"""
-__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
+Fail2Ban reads log file that contains password failure report
+and bans the corresponding IP addresses using firewall rules.
+
+This tools starts/stops fail2ban server or does client/server communication,
+to change/read parameters of the server or jails.
+
+"""
+
+__author__ = "Fail2Ban Developers"
+__copyright__ = "Copyright (c) 2004-2008 Cyril Jaquier, 2012-2014 Yaroslav Halchenko, 2014-2016 Serg G. Brester"
 __license__ = "GPL"
 
-import getopt
+from fail2ban.client.fail2banclient import exec_command_line, sys
-import logging
-import os
-import pickle
-import re
-import shlex
-import signal
-import socket
-import string
-import sys
-import time
 
-from fail2ban.version import version
+if __name__ == "__main__":
-from fail2ban.protocol import printFormatted
+	exec_command_line(sys.argv)
-from fail2ban.client.csocket import CSocket
-from fail2ban.client.configurator import Configurator
-from fail2ban.client.beautifier import Beautifier
-from fail2ban.helpers import getLogger
-
-# Gets the instance of the logger.
-logSys = getLogger("fail2ban")
-
-##
-#
-# @todo This class needs cleanup.
-
-class Fail2banClient:
-
-	SERVER = "fail2ban-server"
-	PROMPT = "fail2ban> "
-
-	def __init__(self):
-		self.__argv = None
-		self.__stream = None
-		self.__configurator = Configurator()
-		self.__conf = dict()
-		self.__conf["conf"] = "/etc/fail2ban"
-		self.__conf["dump"] = False
-		self.__conf["force"] = False
-		self.__conf["background"] = True
-		self.__conf["verbose"] = 1
-		self.__conf["interactive"] = False
-		self.__conf["socket"] = None
-		self.__conf["pidfile"] = None
-
-	def dispVersion(self):
-		print "Fail2Ban v" + version
-		print
-		print "Copyright (c) 2004-2008 Cyril Jaquier, 2008- Fail2Ban Contributors"
-		print "Copyright of modifications held by their respective authors."
-		print "Licensed under the GNU General Public License v2 (GPL)."
-		print
-		print "Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>."
-		print "Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>."
-
-	def dispUsage(self):
-		""" Prints Fail2Ban command line options and exits
-		"""
-		print "Usage: "+self.__argv[0]+" [OPTIONS] <COMMAND>"
-		print
-		print "Fail2Ban v" + version + " reads log file that contains password failure report"
-		print "and bans the corresponding IP addresses using firewall rules."
-		print
-		print "Options:"
-		print "    -c <DIR>                configuration directory"
-		print "    -s <FILE>               socket path"
-		print "    -p <FILE>               pidfile path"
-		print "    -d                      dump configuration. For debugging"
-		print "    -i                      interactive mode"
-		print "    -v                      increase verbosity"
-		print "    -q                      decrease verbosity"
-		print "    -x                      force execution of the server (remove socket file)"
-		print "    -b                      start server in background (default)"
-		print "    -f                      start server in foreground (note that the client forks once itself)"
-		print "    -h, --help              display this help message"
-		print "    -V, --version           print the version"
-		print
-		print "Command:"
-
-		# Prints the protocol
-		printFormatted()
-
-		print
-		print "Report bugs to https://github.com/fail2ban/fail2ban/issues"
-
-	def dispInteractive(self):
-		print "Fail2Ban v" + version + " reads log file that contains password failure report"
-		print "and bans the corresponding IP addresses using firewall rules."
-		print
-
-	def __sigTERMhandler(self, signum, frame):
-		# Print a new line because we probably come from wait
-		print
-		logSys.warning("Caught signal %d. Exiting" % signum)
-		sys.exit(-1)
-
-	def __getCmdLineOptions(self, optList):
-		""" Gets the command line options
-		"""
-		for opt in optList:
-			if opt[0] == "-c":
-				self.__conf["conf"] = opt[1]
-			elif opt[0] == "-s":
-				self.__conf["socket"] = opt[1]
-			elif opt[0] == "-p":
-				self.__conf["pidfile"] = opt[1]
-			elif opt[0] == "-d":
-				self.__conf["dump"] = True
-			elif opt[0] == "-v":
-				self.__conf["verbose"] = self.__conf["verbose"] + 1
-			elif opt[0] == "-q":
-				self.__conf["verbose"] = self.__conf["verbose"] - 1
-			elif opt[0] == "-x":
-				self.__conf["force"] = True
-			elif opt[0] == "-i":
-				self.__conf["interactive"] = True
-			elif opt[0] == "-b":
-				self.__conf["background"] = True
-			elif opt[0] == "-f":
-				self.__conf["background"] = False
-			elif opt[0] in ["-h", "--help"]:
-				self.dispUsage()
-				sys.exit(0)
-			elif opt[0] in ["-V", "--version"]:
-				self.dispVersion()
-				sys.exit(0)
-
-	def __ping(self):
-		return self.__processCmd([["ping"]], False)
-
-	def __processCmd(self, cmd, showRet = True):
-		client = None
-		try:
-			beautifier = Beautifier()
-			streamRet = True
-			for c in cmd:
-				beautifier.setInputCmd(c)
-				try:
-					if not client:
-						client = CSocket(self.__conf["socket"])
-					ret = client.send(c)
-					if ret[0] == 0:
-						logSys.debug("OK : " + `ret[1]`)
-						if showRet:
-							print beautifier.beautify(ret[1])
-					else:
-						logSys.error("NOK: " + `ret[1].args`)
-						if showRet:
-							print beautifier.beautifyError(ret[1])
-						streamRet = False
-				except socket.error:
-					if showRet:
-						self.__logSocketError()
-					return False
-				except Exception as e:
-					if showRet:
-						logSys.error(e)
-					return False
-		finally:
-			if client:
-				client.close()
-		return streamRet
-
-	def __logSocketError(self):
-		try:
-			if os.access(self.__conf["socket"], os.F_OK):
-				# This doesn't check if path is a socket,
-				#  but socket.error should be raised
-				if os.access(self.__conf["socket"], os.W_OK):
-					# Permissions look good, but socket.error was raised
-					logSys.error("Unable to contact server. Is it running?")
-				else:
-					logSys.error("Permission denied to socket: %s,"
-								 " (you must be root)", self.__conf["socket"])
-			else:
-				logSys.error("Failed to access socket path: %s."
-							 " Is fail2ban running?",
-							 self.__conf["socket"])
-		except Exception as e:
-			logSys.error("Exception while checking socket access: %s",
-						 self.__conf["socket"])
-			logSys.error(e)
-
-	##
-	# Process a command line.
-	#
-	# Process one command line and exit.
-	# @param cmd the command line
-
-	def __processCommand(self, cmd):
-		if len(cmd) == 1 and cmd[0] == "start":
-			if self.__ping():
-				logSys.error("Server already running")
-				return False
-			else:
-				# Read the config
-				ret = self.__readConfig()
-				# Do not continue if configuration is not 100% valid
-				if not ret:
-					return False
-				# verify that directory for the socket file exists
-				socket_dir = os.path.dirname(self.__conf["socket"])
-				if not os.path.exists(socket_dir):
-					logSys.error(
-						"There is no directory %s to contain the socket file %s."
-						% (socket_dir, self.__conf["socket"]))
-					return False
-				if not os.access(socket_dir, os.W_OK | os.X_OK):
-					logSys.error(
-						"Directory %s exists but not accessible for writing"
-						% (socket_dir,))
-					return False
-				# Start the server
-				self.__startServerAsync(self.__conf["socket"],
-										self.__conf["pidfile"],
-										self.__conf["force"],
-										self.__conf["background"])
-				try:
-					# Wait for the server to start
-					self.__waitOnServer()
-					# Configure the server
-					self.__processCmd(self.__stream, False)
-					return True
-				except ServerExecutionException:
-					logSys.error("Could not start server. Maybe an old "
-								 "socket file is still present. Try to "
-								 "remove " + self.__conf["socket"] + ". If "
-								 "you used fail2ban-client to start the "
-								 "server, adding the -x option will do it")
-					return False
-		elif len(cmd) == 1 and cmd[0] == "reload":
-			if self.__ping():
-				ret = self.__readConfig()
-				# Do not continue if configuration is not 100% valid
-				if not ret:
-					return False
-				self.__processCmd([['stop', 'all']], False)
-				# Configure the server
-				return self.__processCmd(self.__stream, False)
-			else:
-				logSys.error("Could not find server")
-				return False
-		elif len(cmd) == 2 and cmd[0] == "reload":
-			if self.__ping():
-				jail = cmd[1]
-				ret = self.__readConfig(jail)
-				# Do not continue if configuration is not 100% valid
-				if not ret:
-					return False
-				self.__processCmd([['stop', jail]], False)
-				# Configure the server
-				return self.__processCmd(self.__stream, False)
-			else:
-				logSys.error("Could not find server")
-				return False
-		else:
-			return self.__processCmd([cmd])
-
-
-	##
-	# Start Fail2Ban server.
-	#
-	# Start the Fail2ban server in daemon mode.
-
-	def __startServerAsync(self, socket, pidfile, force = False, background = True):
-		# Forks the current process.
-		pid = os.fork()
-		if pid == 0:
-			args = list()
-			args.append(self.SERVER)
-			# Set the socket path.
-			args.append("-s")
-			args.append(socket)
-			# Set the pidfile
-			args.append("-p")
-			args.append(pidfile)
-			# Force the execution if needed.
-			if force:
-				args.append("-x")
-			# Start in foreground mode if requested.
-			if background:
-				args.append("-b")
-			else:
-				args.append("-f")
-			
-			try:
-				# Use the current directory.
-				exe = os.path.abspath(os.path.join(sys.path[0], self.SERVER))
-				logSys.debug("Starting %r with args %r" % (exe, args))
-				os.execv(exe, args)
-			except OSError:
-				try:
-					# Use the PATH env.
-					logSys.warning("Initial start attempt failed.  Starting %r with the same args" % (self.SERVER,))
-					os.execvp(self.SERVER, args)
-				except OSError:
-					logSys.error("Could not start %s" % self.SERVER)
-					os.exit(-1)
-
-	def __waitOnServer(self):
-		# Wait for the server to start
-		cnt = 0
-		if self.__conf["verbose"] > 1:
-			pos = 0
-			delta = 1
-			mask = "[          ]"
-		while not self.__ping():
-			# Wonderful visual :)
-			if self.__conf["verbose"] > 1:
-				pos += delta
-				sys.stdout.write("\rINFO   " + mask[:pos] + '#' + mask[pos+1:] +
-								 " Waiting on the server...")
-				sys.stdout.flush()
-				if pos > len(mask)-3:
-					delta = -1
-				elif pos < 2:
-					delta = 1
-			# The server has 30 seconds to start.
-			if cnt >= 300:
-				if self.__conf["verbose"] > 1:
-					sys.stdout.write('\n')
-				raise ServerExecutionException("Failed to start server")
-			time.sleep(0.1)
-			cnt += 1
-		if self.__conf["verbose"] > 1:
-			sys.stdout.write('\n')
-
-
-	def start(self, argv):
-		# Command line options
-		self.__argv = argv
-
-		# Install signal handlers
-		signal.signal(signal.SIGTERM, self.__sigTERMhandler)
-		signal.signal(signal.SIGINT, self.__sigTERMhandler)
-
-		# Reads the command line options.
-		try:
-			cmdOpts = 'hc:s:p:xfbdviqV'
-			cmdLongOpts = ['help', 'version']
-			optList, args = getopt.getopt(self.__argv[1:], cmdOpts, cmdLongOpts)
-		except getopt.GetoptError:
-			self.dispUsage()
-			return False
-
-		self.__getCmdLineOptions(optList)
-
-		verbose = self.__conf["verbose"]
-		if verbose <= 0:
-			logSys.setLevel(logging.ERROR)
-		elif verbose == 1:
-			logSys.setLevel(logging.WARNING)
-		elif verbose == 2:
-			logSys.setLevel(logging.INFO)
-		elif verbose == 3:
-			logSys.setLevel(logging.DEBUG)
-		else:
-			logSys.setLevel(logging.HEAVYDEBUG)
-		# Add the default logging handler to dump to stderr
-		logout = logging.StreamHandler(sys.stderr)
-		# set a format which is simpler for console use
-		formatter = logging.Formatter('%(levelname)-6s %(message)s')
-		# tell the handler to use this format
-		logout.setFormatter(formatter)
-		logSys.addHandler(logout)
-
-		# Set the configuration path
-		self.__configurator.setBaseDir(self.__conf["conf"])
-
-		# Set socket path
-		self.__configurator.readEarly()
-		conf = self.__configurator.getEarlyOptions()
-		if self.__conf["socket"] is None:
-			self.__conf["socket"] = conf["socket"]
-		if self.__conf["pidfile"] is None:
-			self.__conf["pidfile"] = conf["pidfile"]
-		logSys.info("Using socket file " + self.__conf["socket"])
-
-		if self.__conf["dump"]:
-			ret = self.__readConfig()
-			self.dumpConfig(self.__stream)
-			return ret
-
-		# Interactive mode
-		if self.__conf["interactive"]:
-			try:
-				import readline
-			except ImportError:
-				logSys.error("Readline not available")
-				return False
-			try:
-				ret = True
-				if len(args) > 0:
-					ret = self.__processCommand(args)
-				if ret:
-					readline.parse_and_bind("tab: complete")
-					self.dispInteractive()
-					while True:
-						cmd = raw_input(self.PROMPT)
-						if cmd == "exit" or cmd == "quit":
-							# Exit
-							return True
-						if cmd == "help":
-							self.dispUsage()
-						elif not cmd == "":
-							try:
-								self.__processCommand(shlex.split(cmd))
-							except Exception as e:
-								logSys.error(e)
-			except (EOFError, KeyboardInterrupt):
-				print
-				return True
-		# Single command mode
-		else:
-			if len(args) < 1:
-				self.dispUsage()
-				return False
-			return self.__processCommand(args)
-
-	def __readConfig(self, jail=None):
-		# Read the configuration
-		# TODO: get away from stew of return codes and exception
-		# handling -- handle via exceptions
-		try:
-			self.__configurator.Reload()
-			self.__configurator.readAll()
-			ret = self.__configurator.getOptions(jail)
-			self.__configurator.convertToProtocol()
-			self.__stream = self.__configurator.getConfigStream()
-		except Exception as e:
-			logSys.error("Failed during configuration: %s" % e)
-			ret = False
-		return ret
-
-	@staticmethod
-	def dumpConfig(cmd):
-		for c in cmd:
-			print c
-		return True
-
-
-class ServerExecutionException(Exception):
-	pass
-
-if __name__ == "__main__": # pragma: no cover - can't test main
-	client = Fail2banClient()
-	# Exit with correct return value
-	if client.start(sys.argv):
-		sys.exit(0)
-	else:
-		sys.exit(-1)

bin/fail2ban-regex

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 # emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
 # vi: set ft=python sts=4 ts=4 sw=4 noet :
 #

bin/fail2ban-server

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 # emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
 # vi: set ft=python sts=4 ts=4 sw=4 noet :
 
@@ -18,123 +18,20 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-__author__ = "Cyril Jaquier"
+"""
-__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
+Fail2Ban reads log file that contains password failure report
+and bans the corresponding IP addresses using firewall rules.
+
+This tools starts/stops fail2ban server or does client/server communication,
+to change/read parameters of the server or jails.
+
+"""
+
+__author__ = "Fail2Ban Developers"
+__copyright__ = "Copyright (c) 2004-2008 Cyril Jaquier, 2012-2014 Yaroslav Halchenko, 2014-2016 Serg G. Brester"
 __license__ = "GPL"
 
-import getopt
+from fail2ban.client.fail2banserver import exec_command_line, sys
-import os
-import sys
-
-from fail2ban.version import version
-from fail2ban.server.server import Server
-from fail2ban.helpers import getLogger
-
-# Gets the instance of the logger.
-logSys = getLogger("fail2ban")
-
-##
-# \mainpage Fail2Ban
-#
-# \section Introduction
-#
-# Fail2ban is designed to protect your server against brute force attacks.
-# Its first goal was to protect a SSH server.
-
-class Fail2banServer:
-
-	def __init__(self):
-		self.__server = None
-		self.__argv = None
-		self.__conf = dict()
-		self.__conf["background"] = True
-		self.__conf["force"] = False
-		self.__conf["socket"] = "/var/run/fail2ban/fail2ban.sock"
-		self.__conf["pidfile"] = "/var/run/fail2ban/fail2ban.pid"
-
-	def dispVersion(self):
-		print "Fail2Ban v" + version
-		print
-		print "Copyright (c) 2004-2008 Cyril Jaquier, 2008- Fail2Ban Contributors"
-		print "Copyright of modifications held by their respective authors."
-		print "Licensed under the GNU General Public License v2 (GPL)."
-		print
-		print "Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>."
-		print "Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>."
-
-	def dispUsage(self):
-		""" Prints Fail2Ban command line options and exits
-		"""
-		print "Usage: "+self.__argv[0]+" [OPTIONS]"
-		print
-		print "Fail2Ban v" + version + " reads log file that contains password failure report"
-		print "and bans the corresponding IP addresses using firewall rules."
-		print
-		print "Only use this command for debugging purpose. Start the server with"
-		print "fail2ban-client instead. The default behaviour is to start the server"
-		print "in background."
-		print
-		print "Options:"
-		print "    -b                   start in background"
-		print "    -f                   start in foreground"
-		print "    -s <FILE>            socket path"
-		print "    -p <FILE>            pidfile path"
-		print "    -x                   force execution of the server (remove socket file)"
-		print "    -h, --help           display this help message"
-		print "    -V, --version        print the version"
-		print
-		print "Report bugs to https://github.com/fail2ban/fail2ban/issues"
-
-	def __getCmdLineOptions(self, optList):
-		""" Gets the command line options
-		"""
-		for opt in optList:
-			if opt[0] == "-b":
-				self.__conf["background"] = True
-			if opt[0] == "-f":
-				self.__conf["background"] = False
-			if opt[0] == "-s":
-				self.__conf["socket"] = opt[1]
-			if opt[0] == "-p":
-				self.__conf["pidfile"] = opt[1]
-			if opt[0] == "-x":
-				self.__conf["force"] = True
-			if opt[0] in ["-h", "--help"]:
-				self.dispUsage()
-				sys.exit(0)
-			if opt[0] in ["-V", "--version"]:
-				self.dispVersion()
-				sys.exit(0)
-
-	def start(self, argv):
-		# Command line options
-		self.__argv = argv
-
-		# Reads the command line options.
-		try:
-			cmdOpts = 'bfs:p:xhV'
-			cmdLongOpts = ['help', 'version']
-			optList, args = getopt.getopt(self.__argv[1:], cmdOpts, cmdLongOpts)
-		except getopt.GetoptError:
-			self.dispUsage()
-			sys.exit(-1)
-
-		self.__getCmdLineOptions(optList)
-
-		try:
-			self.__server = Server(self.__conf["background"])
-			self.__server.start(self.__conf["socket"],
-								self.__conf["pidfile"],
-								self.__conf["force"])
-			return True
-		except Exception as e:
-			logSys.exception(e)
-			self.__server.quit()
-			return False
 
 if __name__ == "__main__":
-	server = Fail2banServer()
+	exec_command_line(sys.argv)
-	if server.start(sys.argv):
-		sys.exit(0)
-	else:
-		sys.exit(-1)

bin/fail2ban-testcases

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python
 # emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
 # vi: set ft=python sts=4 ts=4 sw=4 noet :
 """Script to run Fail2Ban tests battery
@@ -37,12 +37,8 @@ if os.path.exists("fail2ban/__init__.py"):
 	sys.path.insert(0, ".")
 from fail2ban.version import version
 
-from fail2ban.tests.utils import gatherTests
+from fail2ban.tests.utils import getOptParser, initProcess, gatherTests
-from fail2ban.helpers import FormatterWithTraceBack, getLogger
 from fail2ban.setup import updatePyExec
-from fail2ban.server.mytime import MyTime
-
-from optparse import OptionParser, Option
 
 # Update fail2ban-python env to current python version (where f2b-modules located/installed)
 bindir = os.path.dirname(
@@ -51,84 +47,19 @@ bindir = os.path.dirname(
 )
 updatePyExec(bindir)
 
-def get_opt_parser():
+(opts, regexps) = getOptParser(__doc__).parse_args()
-	# use module docstring for help output
-	p = OptionParser(
-				usage="%s [OPTIONS] [regexps]\n" % sys.argv[0] + __doc__,
-				version="%prog " + version)
-
-	p.add_options([
-		Option('-l', "--log-level", type="choice",
-			   dest="log_level",
-			   choices=('heavydebug', 'debug', 'info', 'notice', 'warning', 'error', 'critical'),
-			   default=None,
-			   help="Log level for the logger to use during running tests"),
-		Option('-n', "--no-network", action="store_true",
-			   dest="no_network",
-			   help="Do not run tests that require the network"),
-		Option("-t", "--log-traceback", action='store_true',
-			   help="Enrich log-messages with compressed tracebacks"),
-		Option("--full-traceback", action='store_true',
-			   help="Either to make the tracebacks full, not compressed (as by default)"),
-
-		])
-
-	return p
-
-parser = get_opt_parser()
-(opts, regexps) = parser.parse_args()
 
 #
-# Logging
+# Process initialization corresponding options (logging, default options, etc.)
 #
-logSys = getLogger("fail2ban")
+opts = initProcess(opts)
-
+verbosity = opts.verbosity
-# Numerical level of verbosity corresponding to a log "level"
-verbosity = {'heavydebug': 4,
-			 'debug': 3,
-			 'info': 2,
-			 'notice': 2,
-			 'warning': 1,
-			 'error': 1,
-			 'critical': 0,
-			 None: 1}[opts.log_level]
-
-if opts.log_level is not None: # pragma: no cover
-	# so we had explicit settings
-	logSys.setLevel(getattr(logging, opts.log_level.upper()))
-else: # pragma: no cover
-	# suppress the logging but it would leave unittests' progress dots
-	# ticking, unless like with '-l critical' which would be silent
-	# unless error occurs
-	logSys.setLevel(getattr(logging, 'CRITICAL'))
-
-# Add the default logging handler
-stdout = logging.StreamHandler(sys.stdout)
-
-fmt = ' %(message)s'
-
-if opts.log_traceback:
-	Formatter = FormatterWithTraceBack
-	fmt = (opts.full_traceback and ' %(tb)s' or ' %(tbc)s') + fmt
-else:
-	Formatter = logging.Formatter
-
-# Custom log format for the verbose tests runs
-if verbosity > 1: # pragma: no cover
-	stdout.setFormatter(Formatter(' %(asctime)-15s %(thread)s' + fmt))
-else: # pragma: no cover
-	# just prefix with the space
-	stdout.setFormatter(Formatter(fmt))
-logSys.addHandler(stdout)
 
 #
-# Let know the version
+# Gather tests (and filter corresponding options)
 #
-if not opts.log_level or opts.log_level != 'critical': # pragma: no cover
+tests = gatherTests(regexps, opts)
-	print("Fail2ban %s test suite. Python %s. Please wait..." \
-	    % (version, str(sys.version).replace('\n', '')))
 
-tests = gatherTests(regexps, opts.no_network)
 #
 # Run the tests
 #

config/action.d/abuseipdb.conf

@@ -0,0 +1,105 @@
+# Fail2ban configuration file
+#
+# Action to report IP address to abuseipdb.com
+# You must sign up to obtain an API key from abuseipdb.com.
+#
+# NOTE: These reports may include sensitive Info.
+# If you want cleaner reports that ensure no user data see the helper script at the below website.
+#
+# IMPORTANT:
+#
+# Reporting an IP of abuse is a serious complaint. Make sure that it is
+# serious. Fail2ban developers and network owners recommend you only use this
+# action for:
+#   * The recidive where the IP has been banned multiple times
+#   * Where maxretry has been set quite high, beyond the normal user typing
+#     password incorrectly.
+#   * For filters that have a low likelihood of receiving human errors
+#
+# This action relies on a api_key being added to the above action conf,
+# and the appropriate categories set.
+#
+# Example, for ssh bruteforce (in section [sshd] of `jail.local`): 
+#   action = %(known/action)s
+#            %(action_abuseipdb)s[abuseipdb_apikey="my-api-key", abuseipdb_category="18,22"]
+#
+# See below for catagories.
+#
+# Original Ref: https://wiki.shaunc.com/wikka.php?wakka=ReportingToAbuseIPDBWithFail2Ban
+# Added to fail2ban by Andrew James Collett (ajcollett)
+
+## abuseIPDB Catagories, `the abuseipdb_category` MUST be set in the jail.conf action call.
+# Example, for ssh bruteforce: action = %(action_abuseipdb)s[abuseipdb_category="18,22"]
+# ID	Title	Description
+# 3	  Fraud Orders
+# 4	  DDoS Attack
+# 9	  Open Proxy
+# 10	Web Spam
+# 11	Email Spam
+# 14	Port Scan
+# 18	Brute-Force
+# 19	Bad Web Bot
+# 20	Exploited Host
+# 21	Web App Attack
+# 22	SSH	Secure Shell (SSH) abuse. Use this category in combination with more specific categories.
+# 23	IoT Targeted
+# See https://abuseipdb.com/categories for more descriptions
+
+[Definition]
+
+# Option:  actionstart
+# Notes.:  command executed once at the start of Fail2Ban.
+# Values:  CMD
+#
+actionstart =
+
+# Option:  actionstop
+# Notes.:  command executed once at the end of Fail2Ban
+# Values:  CMD
+#
+actionstop =
+
+# Option:  actioncheck
+# Notes.:  command executed once before each actionban command
+# Values:  CMD
+#
+actioncheck =
+
+# Option:  actionban
+# Notes.:  command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+#
+#          ** IMPORTANT! **
+#
+#          By default, this posts directly to AbuseIPDB's API, unfortunately
+#          this results in a lot of backslashes/escapes appearing in the
+#          reports. This also may include info like your hostname.
+#          If you have your own web server with PHP available, you can
+#          use my (Shaun's) helper PHP script by commenting out the first #actionban
+#          line below, uncommenting the second one, and pointing the URL at
+#          wherever you install the helper script. For the PHP helper script, see
+#          <https://wiki.shaunc.com/wikka.php?wakka=ReportingToAbuseIPDBWithFail2Ban>
+#
+#          --ciphers ecdhe_ecdsa_aes_256_sha is used to workaround a
+#          "NSS error -12286" from curl as it attempts to connect using
+#          SSLv3. See https://www.centos.org/forums/viewtopic.php?t=52732
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionban = curl --fail --ciphers ecdhe_ecdsa_aes_256_sha --data 'key=<abuseipdb_apikey>' --data-urlencode 'comment=<matches>' --data 'ip=<ip>' --data 'category=<abuseipdb_category>' "https://www.abuseipdb.com/report/json"
+
+# Option:  actionunban
+# Notes.:  command executed when unbanning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionunban =
+
+[Init]
+# Option:  abuseipdb_apikey
+# Notes    Your API key from abuseipdb.com
+# Values:  STRING  Default: None
+# Register for abuseipdb [https://www.abuseipdb.com], get api key and set below.
+# You will need to set the catagory in the action call.
+abuseipdb_apikey =

config/action.d/badips.py

@@ -34,7 +34,7 @@ else:
 from fail2ban.server.actions import ActionBase
 
 
-class BadIPsAction(ActionBase):
+class BadIPsAction(ActionBase): # pragma: no cover - may be unavailable
 	"""Fail2Ban action which reports bans to badips.com, and also
 	blacklist bad IPs listed on badips.com by using another action's
 	ban method.
@@ -105,6 +105,16 @@ class BadIPsAction(ActionBase):
 		# Used later for threading.Timer for updating badips
 		self._timer = None
 
+	@staticmethod
+	def isAvailable(timeout=1):
+		try:
+			response = urlopen(Request("/".join([BadIPsAction._badips]),
+					headers={'User-Agent': "Fail2Ban"}), timeout=timeout)
+			return True, ''
+		except Exception as e: # pragma: no cover
+			return False, e
+
+
 	def getCategories(self, incParents=False):
 		"""Get badips.com categories.
 

config/action.d/bsd-ipfw.conf

@@ -14,7 +14,7 @@
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = ipfw show | fgrep -q 'table(<table>)' || ( ipfw show | awk 'BEGIN { b = 1 } { if ($1 <= b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num <blocktype> <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
+actionstart = ipfw show | fgrep -c -m 1 -s 'table(<table>)' > /dev/null 2>&1 || ( ipfw show | awk 'BEGIN { b = <lowest_rule_num> } { if ($1 < b) {} else if ($1 == b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num <blocktype> <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
 
 
 # Option:  actionstop
@@ -81,3 +81,11 @@ block = ip
 # Values:  STRING
 #
 blocktype = unreach port
+
+# Option:  lowest_rule_num
+# Notes:   When fail2ban starts with action and there is no rule for the given table yet
+#          then fail2ban will start looking for an empty slot starting with this rule number.
+# Values:  NUM
+lowest_rule_num = 111
+
+

config/action.d/cloudflare.conf

@@ -40,7 +40,12 @@ actioncheck =
 #          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
-actionban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=ban' -d 'tkn=<cftoken>' -d 'email=<cfuser>' -d 'key=<ip>'
+# API v1
+#actionban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=ban' -d 'tkn=<cftoken>' -d 'email=<cfuser>' -d 'key=<ip>'
+# API v4
+actionban = curl -s -o /dev/null -X POST -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
+            -H 'Content-Type: application/json' -d '{ "mode": "block", "configuration": { "target": "ip", "value": "<ip>" } }' \
+            https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -50,7 +55,12 @@ actionban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=
 #          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
-actionunban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=nul' -d 'tkn=<cftoken>' -d 'email=<cfuser>' -d 'key=<ip>'
+# API v1
+#actionunban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=nul' -d 'tkn=<cftoken>' -d 'email=<cfuser>' -d 'key=<ip>'
+# API v4
+actionunban = curl -s -o /dev/null -X DELETE -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
+              https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules/$(curl -s -X GET -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
+              'https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules?mode=block&configuration_target=ip&configuration_value=<ip>&page=1&per_page=1' | cut -d'"' -f6)
 
 [Init]
 

config/action.d/complain.conf

@@ -28,8 +28,18 @@
 #
 
 
+[INCLUDES]
+
+before = helpers-common.conf
+
 [Definition]
 
+# Used in test cases for coverage internal transformations
+debug = 0
+
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
@@ -54,10 +64,18 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = oifs=${IFS}; IFS=.;SEP_IP=( <ip> ); set -- ${SEP_IP}; ADDRESSES=$(dig +short -t txt -q $4.$3.$2.$1.abuse-contacts.abusix.org); IFS=${oifs}
+actionban = oifs=${IFS};
+            RESOLVER_ADDR="%(addr_resolver)s"
+            if [ "<debug>" -gt 0 ]; then echo "try to resolve $RESOLVER_ADDR"; fi
+            ADDRESSES=$(dig +short -t txt -q $RESOLVER_ADDR | tr -d '"')
+            IFS=,; ADDRESSES=$(echo $ADDRESSES)
+            IFS=${oifs}
             IP=<ip>
             if [ ! -z "$ADDRESSES" ]; then
-                (printf %%b "<message>\n"; date '+Note: Local timezone is %%z (%%Z)'; grep -E '(^|[^0-9])<ip>([^0-9]|$)' <logpath>) | <mailcmd> "Abuse from <ip>" <mailargs> ${ADDRESSES//,/\" \"}
+                ( printf %%b "<message>\n"; date '+Note: Local timezone is %%z (%%Z)'; 
+                  printf %%b "\nLines containing failures of <ip> (max <grepmax>)\n";
+                  %(_grep_logs)s;
+                ) | <mailcmd> "Abuse from <ip>" <mailargs> $ADDRESSES
             fi
 
 # Option:  actionunban
@@ -68,7 +86,12 @@ actionban = oifs=${IFS}; IFS=.;SEP_IP=( <ip> ); set -- ${SEP_IP}; ADDRESSES=$(di
 #
 actionunban =
 
-[Init]
+# Server as resolver used in dig command
+#
+addr_resolver = <ip-rev>abuse-contacts.abusix.org
+
+# Default message used for abuse content
+#
 message = Dear Sir/Madam,\n\nWe have detected abuse from the IP address $IP, which according to a abusix.com is on your network. We would appreciate if you would investigate and take action as appropriate.\n\nLog lines are given below, but please ask if you require any further information.\n\n(If you are not the correct person to contact about this please accept our apologies - your e-mail address was extracted from the whois record by an automated process.)\n\n This mail was generated by Fail2Ban.\nThe recipient address of this report was provided by the Abuse Contact DB by abusix.com. abusix.com does not maintain the content of the database. All information which we pass out, derives from the RIR databases and is processed for ease of use. If you want to change or report non working abuse contacts please contact the appropriate RIR. If you have any further question, contact abusix.com directly via email (info@abusix.com). Information about the Abuse Contact Database can be found here: https://abusix.com/global-reporting/abuse-contact-db\nabusix.com is neither responsible nor liable for the content or accuracy of this message.\n
 
 # Path to the log files which contain relevant lines for the abuser IP
@@ -92,3 +115,7 @@ mailcmd = mail -s
 #
 mailargs =
 
+# Number of log lines to include in the email
+#
+#grepmax = 1000
+#grepopts = -m <grepmax>

config/action.d/dshield.conf

@@ -28,6 +28,9 @@
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD

config/action.d/dummy.conf

@@ -10,14 +10,23 @@
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = touch /var/run/fail2ban/fail2ban.dummy
+actionstart = if [ ! -z '<target>' ]; then touch <target>; fi;
-              printf %%b "<init>\n" >> /var/run/fail2ban/fail2ban.dummy
+              printf %%b "<init>\n" <to_target>
+              echo "%(debug)s started"
+
+# Option:  actionflush
+# Notes.:  command executed once to flush (clear) all IPS, by shutdown (resp. by stop of the jail or this action)
+# Values:  CMD
+#
+actionflush = printf %%b "-*\n" <to_target>
+              echo "%(debug)s clear all"
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = rm -f /var/run/fail2ban/fail2ban.dummy
+actionstop = if [ ! -z '<target>' ]; then rm -f <target>; fi;
+             echo "%(debug)s stopped"
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -31,7 +40,8 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "+<ip>\n" >> /var/run/fail2ban/fail2ban.dummy
+actionban = printf %%b "+<ip>\n" <to_target>
+            echo "%(debug)s banned <ip> (family: <family>)"
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -39,9 +49,15 @@ actionban = printf %%b "+<ip>\n" >> /var/run/fail2ban/fail2ban.dummy
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = printf %%b "-<ip>\n" >> /var/run/fail2ban/fail2ban.dummy
+actionunban = printf %%b "-<ip>\n" <to_target>
+              echo "%(debug)s unbanned <ip> (family: <family>)"
+
+
+debug = [<name>] <actname> <target> --
 
 [Init]
 
 init = 123
 
+target = /var/run/fail2ban/fail2ban.dummy
+to_target = >> <target>

config/action.d/firewallcmd-allports.conf

@@ -6,34 +6,26 @@
 
 [INCLUDES]
 
-before = iptables-common.conf
+before = firewallcmd-common.conf
 
 [Definition]
 
-actionstart = firewall-cmd --direct --add-chain ipv4 filter f2b-<name>
+actionstart = firewall-cmd --direct --add-chain <family> filter f2b-<name>
-              firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 1000 -j RETURN
+              firewall-cmd --direct --add-rule <family> filter f2b-<name> 1000 -j RETURN
-              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -j f2b-<name>
+              firewall-cmd --direct --add-rule <family> filter <chain> 0 -j f2b-<name>
 
-actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -j f2b-<name>
+actionstop = firewall-cmd --direct --remove-rule <family> filter <chain> 0 -j f2b-<name>
-             firewall-cmd --direct --remove-rules ipv4 filter f2b-<name>
+             firewall-cmd --direct --remove-rules <family> filter f2b-<name>
-             firewall-cmd --direct --remove-chain ipv4 filter f2b-<name>
+             firewall-cmd --direct --remove-chain <family> filter f2b-<name>
 
 
 # Example actioncheck: firewall-cmd --direct --get-chains ipv4 filter | sed -e 's, ,\n,g' | grep -q '^f2b-recidive$'
 
-actioncheck = firewall-cmd --direct --get-chains ipv4 filter | sed -e 's, ,\n,g' | grep -q '^f2b-<name>$'
+actioncheck = firewall-cmd --direct --get-chains <family> filter | sed -e 's, ,\n,g' | grep -q '^f2b-<name>$'
 
-actionban = firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
+actionban = firewall-cmd --direct --add-rule <family> filter f2b-<name> 0 -s <ip> -j <blocktype>
 
-actionunban = firewall-cmd --direct --remove-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
+actionunban = firewall-cmd --direct --remove-rule <family> filter f2b-<name> 0 -s <ip> -j <blocktype>
-
-[Init]
-
-# Default name of the chain
-#
-name = default
-
-chain = INPUT_direct
 
 # DEV NOTES:
 #

config/action.d/firewallcmd-common.conf

@@ -0,0 +1,76 @@
+# Fail2Ban configuration file
+#
+# Author: Donald Yandt
+#
+
+[Init]
+
+# Option:  name
+# Notes    Default name of the chain
+# Values:  STRING
+name = default
+
+# Option   port
+# Notes    Can also use port numbers separated by a comma and in rich-rules comma and/or space.
+# Value    STRING Default: 1:65535
+port = 1:65535
+
+# Option:  protocol
+# Notes    [ tcp | udp | icmp | all ]
+# Values:  STRING Default: tcp
+protocol = tcp
+
+# Option:  family(ipv4)
+# Notes    specifies the socket address family type
+# Values:  STRING  
+family = ipv4
+
+# Option:  chain
+# Notes    specifies the firewalld chain to which the Fail2Ban rules should be
+#          added
+# Values:  STRING  Default: INPUT_direct
+chain = INPUT_direct
+
+# Option:  zone
+# Notes    use command firewall-cmd --get-active-zones to see a list of all active zones. See firewalld man pages for more information on zones
+# Values:  STRING  Default: public
+zone = public
+
+# Option:  service
+# Notes    use command firewall-cmd --get-services to see a list of services available
+#          Examples services: amanda-client amanda-k5-client bacula bacula-client dhcp dhcpv6 dhcpv6-client dns freeipa-ldap freeipa-ldaps 
+#          freeipa-replication ftp high-availability http https imaps ipp ipp-client ipsec iscsi-target kadmin kerberos 
+#          kpasswd ldap ldaps libvirt libvirt-tls mdns mosh mountd ms-wbt mysql nfs ntp openvpn pmcd pmproxy pmwebapi pmwebapis pop3s 
+#          postgresql privoxy proxy-dhcp puppetmaster radius rpc-bind rsyncd samba samba-client sane smtp squid ssh synergy 
+#          telnet tftp tftp-client tinc tor-socks transmission-client vdsm vnc-server wbem-https xmpp-bosh xmpp-client xmpp-local xmpp-server
+# Values:  STRING Default: ssh 
+service = ssh
+
+# Option:  rejecttype (ipv4)
+# Notes    See iptables/firewalld man pages for ipv4 reject types. 
+# Values:  STRING
+rejecttype = icmp-port-unreachable
+
+# Option:  blocktype (ipv4/ipv6)
+# Notes    See iptables/firewalld man pages for jump targets. Common values are REJECT, 
+#          REJECT --reject-with icmp-port-unreachable, DROP
+# Values:  STRING
+blocktype = REJECT --reject-with <rejecttype>
+
+# Option:  rich-blocktype (ipv4/ipv6)
+# Notes    See firewalld man pages for jump targets. Common values are reject, 
+#          reject type="icmp-port-unreachable", drop
+# Values:  STRING
+rich-blocktype = reject type='<rejecttype>'
+
+[Init?family=inet6]
+
+# Option:  family(ipv6)
+# Notes    specifies the socket address family type
+# Values:  STRING  
+family = ipv6
+
+# Option:  rejecttype (ipv6)
+# Note:    See iptables/firewalld man pages for ipv6 reject types. 
+# Values:  STRING
+rejecttype = icmp6-port-unreachable

config/action.d/firewallcmd-ipset.conf

@@ -14,20 +14,22 @@
 
 [INCLUDES]
 
-before = iptables-common.conf
+before = firewallcmd-common.conf
 
 [Definition]
 
-actionstart = ipset create fail2ban-<name> hash:ip timeout <bantime>
+actionstart = ipset create <ipmset> hash:ip timeout <bantime><familyopt>
-              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j <blocktype>
+              firewall-cmd --direct --add-rule <family> filter <chain> 0 <actiontype> -m set --match-set <ipmset> src -j <blocktype>
 
-actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j <blocktype>
+actionflush = ipset flush <ipmset>
-             ipset flush fail2ban-<name>
-             ipset destroy fail2ban-<name>
 
-actionban = ipset add fail2ban-<name> <ip> timeout <bantime> -exist
+actionstop = firewall-cmd --direct --remove-rule <family> filter <chain> 0 <actiontype> -m set --match-set <ipmset> src -j <blocktype>
+             <actionflush>
+             ipset destroy <ipmset>
 
-actionunban = ipset del fail2ban-<name> <ip> -exist
+actionban = ipset add <ipmset> <ip> timeout <bantime> -exist
+
+actionunban = ipset del <ipmset> <ip> -exist
 
 [Init]
 
@@ -44,6 +46,31 @@ chain = INPUT_direct
 
 bantime = 600
 
+# Option: actiontype
+# Notes.: defines additions to the blocking rule
+# Values: leave empty to block all attempts from the host
+# Default: Value of the multiport
+actiontype = <multiport>
+
+# Option: allports
+# Notes.: default addition to block all ports
+# Usage.: use in jail config:  banaction = firewallcmd-ipset[actiontype=<allports>]
+#         for all protocols:   banaction = firewallcmd-ipset[actiontype=""]
+allports = -p <protocol>
+
+# Option: multiport
+# Notes.: addition to block access only to specific ports
+# Usage.: use in jail config:  banaction = firewallcmd-ipset[actiontype=<multiport>]
+multiport = -p <protocol> -m multiport --dports <port>
+
+ipmset = f2b-<name>
+familyopt =
+
+[Init?family=inet6]
+
+ipmset = f2b-<name>6
+familyopt = <sp>family inet6
+
 
 # DEV NOTES:
 #

config/action.d/firewallcmd-multiport.conf

@@ -5,59 +5,22 @@
 
 [INCLUDES]
 
-before = iptables-common.conf
+before = firewallcmd-common.conf
 
 [Definition]
 
-actionstart = firewall-cmd --direct --add-chain ipv4 filter f2b-<name>
+actionstart = firewall-cmd --direct --add-chain <family> filter f2b-<name>
-              firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 1000 -j RETURN
+              firewall-cmd --direct --add-rule <family> filter f2b-<name> 1000 -j RETURN
-              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -m conntrack --ctstate NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
+              firewall-cmd --direct --add-rule <family> filter <chain> 0 -m conntrack --ctstate NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
 
-actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -m conntrack --ctstate NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
+actionstop = firewall-cmd --direct --remove-rule <family> filter <chain> 0 -m conntrack --ctstate NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
-             firewall-cmd --direct --remove-rules ipv4 filter f2b-<name>
+             firewall-cmd --direct --remove-rules <family> filter f2b-<name>
-             firewall-cmd --direct --remove-chain ipv4 filter f2b-<name>
+             firewall-cmd --direct --remove-chain <family> filter f2b-<name>
 
 # Example actioncheck: firewall-cmd --direct --get-chains ipv4 filter | sed -e 's, ,\n,g' | grep -q '^f2b-apache-modsecurity$'
 
-actioncheck = firewall-cmd --direct --get-chains ipv4 filter | sed -e 's, ,\n,g' | grep -q '^f2b-<name>$'
+actioncheck = firewall-cmd --direct --get-chains <family> filter | sed -e 's, ,\n,g' | grep -q '^f2b-<name>$'
 
-actionban = firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
+actionban = firewall-cmd --direct --add-rule <family> filter f2b-<name> 0 -s <ip> -j <blocktype>
-
-actionunban = firewall-cmd --direct --remove-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
-
-[Init]
-
-# Default name of the chain
-name = default
-
-chain = INPUT_direct
-
-# Could also use port numbers separated by a comma. 
-port = 1:65535
-
-
-# Option:  protocol
-# Values:  [ tcp | udp | icmp | all ]
-
-protocol = tcp
-
-
-
-# DEV NOTES:
-#
-# Author: Donald Yandt 
-# Uses "FirewallD" instead of the "iptables daemon".
-#
-#
-# Output:
-# actionstart:
-# $ firewall-cmd --direct --add-chain ipv4 filter f2b-apache-modsecurity
-# success
-# $ firewall-cmd --direct --add-rule ipv4 filter f2b-apache-modsecurity 1000 -j RETURN
-# success
-# $ sudo firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 0 -m state --state NEW -p tcp -m multiport --dports 80,443 -j f2b-apache-modsecurity
-# success
-# actioncheck:
-# $ firewall-cmd --direct --get-chains ipv4 filter f2b-apache-modsecurity | sed -e 's, ,\n,g' | grep -q '^f2b-apache-modsecurity$'
-# f2b-apache-modsecurity
 
+actionunban = firewall-cmd --direct --remove-rule <family> filter f2b-<name> 0 -s <ip> -j <blocktype>

config/action.d/firewallcmd-new.conf

@@ -4,32 +4,23 @@
 
 [INCLUDES]
 
-before = iptables-common.conf
+before = firewallcmd-common.conf
 
 [Definition]
 
-actionstart = firewall-cmd --direct --add-chain ipv4 filter f2b-<name>
+actionstart = firewall-cmd --direct --add-chain <family> filter f2b-<name>
-              firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 1000 -j RETURN
+              firewall-cmd --direct --add-rule <family> filter f2b-<name> 1000 -j RETURN
-              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -m state --state NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
+              firewall-cmd --direct --add-rule <family> filter <chain> 0 -m state --state NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
 
-actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -m state --state NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
+actionstop = firewall-cmd --direct --remove-rule <family> filter <chain> 0 -m state --state NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
-             firewall-cmd --direct --remove-rules ipv4 filter f2b-<name>
+             firewall-cmd --direct --remove-rules <family> filter f2b-<name>
-             firewall-cmd --direct --remove-chain ipv4 filter f2b-<name>
+             firewall-cmd --direct --remove-chain <family> filter f2b-<name>
 
-actioncheck = firewall-cmd --direct --get-chains ipv4 filter | grep -q 'f2b-<name>$'
+actioncheck = firewall-cmd --direct --get-chains <family> filter | sed -e 's, ,\n,g' | grep -q 'f2b-<name>$'
 
-actionban = firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
+actionban = firewall-cmd --direct --add-rule <family> filter f2b-<name> 0 -s <ip> -j <blocktype>
 
-actionunban = firewall-cmd --direct --remove-rule ipv4 filter f2b-<name> 0 -s <ip> -j <blocktype>
+actionunban = firewall-cmd --direct --remove-rule <family> filter f2b-<name> 0 -s <ip> -j <blocktype>
-
-[Init]
-
-# Option:  chain
-# Notes    specifies the iptables chain to which the fail2ban rules should be
-#          added
-# Values:  [ STRING ]
-#
-chain = INPUT_direct
 
 # DEV NOTES:
 #

config/action.d/firewallcmd-rich-logging.conf

@@ -15,6 +15,10 @@
 # firewall-cmd [--zone=<zone>] --list-all
 # firewall-cmd [--zone=zone] --query-rich-rule='rule'
 
+[INCLUDES]
+
+before = firewallcmd-common.conf
+
 [Definition]
 
 actionstart = 
@@ -26,40 +30,22 @@ actioncheck =
 # you can also use zones and/or service names. 
 #
 # zone example: 
-# 	firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' port port='<port>' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>"
+# firewall-cmd --zone=<zone> --add-rich-rule="rule family='<family>' source address='<ip>' port port='<port>' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <rich-blocktype>"
+#
 # service name example:
-#	firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' service name='<service>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>"
+# firewall-cmd --zone=<zone> --add-rich-rule="rule family='<family>' source address='<ip>' service name='<service>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <rich-blocktype>"
+#
 # Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp 
 
-actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>"; done
+actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <rich-blocktype>"; done
 	   
-actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>"; done
+actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <rich-blocktype>"; done
 
 [Init]
 
-name = default
-
 # log levels are "emerg", "alert", "crit", "error", "warning", "notice", "info" or "debug"
 level = info
 
 # log rate per minute
 rate = 1
 
-zone = public
-
-# use command firewall-cmd --get-services to see a list of services available
-#
-# Examples:
-#
-# amanda-client amanda-k5-client bacula bacula-client dhcp dhcpv6 dhcpv6-client dns freeipa-ldap freeipa-ldaps 
-# freeipa-replication ftp high-availability http https imaps ipp ipp-client ipsec iscsi-target kadmin kerberos 
-# kpasswd ldap ldaps libvirt libvirt-tls mdns mosh mountd ms-wbt mysql nfs ntp openvpn pmcd pmproxy pmwebapi pmwebapis pop3s 
-# postgresql privoxy proxy-dhcp puppetmaster radius rpc-bind rsyncd samba samba-client sane smtp squid ssh synergy 
-# telnet tftp tftp-client tinc tor-socks transmission-client vdsm vnc-server wbem-https xmpp-bosh xmpp-client xmpp-local xmpp-server
-
-service = ssh
-
-# reject types: 'icmp-net-unreachable', 'icmp-host-unreachable', 'icmp-port-unreachable', 'icmp-proto-unreachable', 
-# 'icmp-net-prohibited', 'icmp-host-prohibited', 'icmp-admin-prohibited' or 'tcp-reset'
-
-blocktype = reject type='icmp-port-unreachable'

config/action.d/firewallcmd-rich-rules.conf

@@ -13,6 +13,10 @@
 # firewall-cmd [--zone=<zone>] --list-all
 # firewall-cmd [--zone=zone] --query-rich-rule='rule'
 
+[INCLUDES]
+
+before = firewallcmd-common.conf
+
 [Definition]
 
 actionstart = 
@@ -24,34 +28,15 @@ actioncheck =
 #you can also use zones and/or service names. 
 #
 # zone example: 
-# 	firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' port port='<port>' protocol='<protocol>' <blocktype>"
+# firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' port port='<port>' protocol='<protocol>' <rich-blocktype>"
+#
 # service name example:
-#	firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' service name='<service>' <blocktype>"
+# firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' service name='<service>' <rich-blocktype>"
+#
 # Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp 
 
-actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' <blocktype>"; done
+actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' <rich-blocktype>"; done
 	   
-actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' <blocktype>"; done
+actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' <rich-blocktype>"; done
 
-[Init]
 
-name = default
-
-zone = public
-
-# use command firewall-cmd --get-services to see a list of services available
-#
-# Examples:
-#
-# amanda-client amanda-k5-client bacula bacula-client dhcp dhcpv6 dhcpv6-client dns freeipa-ldap freeipa-ldaps 
-# freeipa-replication ftp high-availability http https imaps ipp ipp-client ipsec iscsi-target kadmin kerberos 
-# kpasswd ldap ldaps libvirt libvirt-tls mdns mosh mountd ms-wbt mysql nfs ntp openvpn pmcd pmproxy pmwebapi pmwebapis pop3s 
-# postgresql privoxy proxy-dhcp puppetmaster radius rpc-bind rsyncd samba samba-client sane smtp squid ssh synergy 
-# telnet tftp tftp-client tinc tor-socks transmission-client vdsm vnc-server wbem-https xmpp-bosh xmpp-client xmpp-local xmpp-server
-
-service = ssh
-
-# reject types: 'icmp-net-unreachable', 'icmp-host-unreachable', 'icmp-port-unreachable', 'icmp-proto-unreachable', 
-# 'icmp-net-prohibited', 'icmp-host-prohibited', 'icmp-admin-prohibited' or 'tcp-reset'
-
-blocktype = reject type='icmp-port-unreachable'

config/action.d/helpers-common.conf

@@ -0,0 +1,16 @@
+[DEFAULT]
+
+# Usage:
+#   _grep_logs_args = 'test'
+#   (printf %%b "Log-excerpt contains 'test':\n"; %(_grep_logs)s; printf %%b "Log-excerpt contains 'test':\n") | mail ...
+#
+_grep_logs = logpath="<logpath>"; grep <grepopts> -E %(_grep_logs_args)s $logpath | <greplimit>
+_grep_logs_args = "(^|[^0-9a-fA-F:])$(echo '<ip>' | sed 's/\./\\./g')([^0-9a-fA-F:]|$)"
+
+# Used for actions, that should not by executed if ticket was restored:
+_bypass_if_restored = if [ '<restored>' = '1' ]; then exit 0; fi;
+
+[Init]
+greplimit = tail -n <grepmax>
+grepmax = 1000
+grepopts = -m <grepmax>

config/action.d/hostsdeny.conf

@@ -31,8 +31,7 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = IP=<ip> &&
+actionban = IP=<ip> && printf %%b "<daemon_list>: $IP\n" >> <file>
-            printf %%b "<daemon_list>: $IP\n" >> <file>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -40,7 +39,7 @@ actionban = IP=<ip> &&
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = echo "/^<daemon_list>: <ip>$/<br>d<br>w<br>q" | ed <file>
+actionunban = IP=$(echo <ip> | sed 's/\./\\./g') && sed -i "/^<daemon_list>: $IP$/d" <file>
 
 [Init]
 

config/action.d/iptables-allports.conf

@@ -26,7 +26,7 @@ actionstart = <iptables> -N f2b-<name>
 # Values:  CMD
 #
 actionstop = <iptables> -D <chain> -p <protocol> -j f2b-<name>
-             <iptables> -F f2b-<name>
+             <actionflush>
              <iptables> -X f2b-<name>
 
 # Option:  actioncheck

config/action.d/iptables-common.conf

@@ -6,6 +6,9 @@
 # used in all iptables based actions by default.
 #
 # The user can override the defaults in iptables-common.local
+#
+# Modified: Alexander Koeppe <format_c@online.de>, Serg G. Brester <serg.brester@sebres.de>
+#       made config file IPv6 capable (see new section Init?family=inet6)
 
 [INCLUDES]
 
@@ -13,6 +16,15 @@ after = iptables-blocktype.local
         iptables-common.local
 # iptables-blocktype.local is obsolete
 
+[Definition]
+
+# Option:  actionflush
+# Notes.:  command executed once to flush IPS, by shutdown (resp. by stop of the jail or this action)
+# Values:  CMD
+#
+actionflush = <iptables> -F f2b-<name>
+
+
 [Init]
 
 # Option:  chain
@@ -62,3 +74,19 @@ lockingopt = -w
 # Notes.:  Actual command to be executed, including common to all calls options
 # Values:  STRING
 iptables = iptables <lockingopt>
+
+
+[Init?family=inet6]
+
+# Option:  blocktype (ipv6)
+# Note:    This is what the action does with rules. This can be any jump target
+#          as per the iptables man page (section 8). Common values are DROP
+#          REJECT, REJECT --reject-with icmp6-port-unreachable
+# Values:  STRING
+blocktype = REJECT --reject-with icmp6-port-unreachable
+
+# Option:  iptables (ipv6)
+# Notes.:  Actual command to be executed, including common to all calls options
+# Values:  STRING
+iptables = ip6tables <lockingopt>
+

config/action.d/iptables-ipset-proto4.conf

@@ -30,12 +30,19 @@ before = iptables-common.conf
 actionstart = ipset --create f2b-<name> iphash
               <iptables> -I <chain> -p <protocol> -m multiport --dports <port> -m set --match-set f2b-<name> src -j <blocktype>
 
+
+# Option:  actionflush
+# Notes.:  command executed once to flush IPS, by shutdown (resp. by stop of the jail or this action)
+# Values:  CMD
+#
+actionflush = ipset --flush f2b-<name>
+
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
 actionstop = <iptables> -D <chain> -p <protocol> -m multiport --dports <port> -m set --match-set f2b-<name> src -j <blocktype>
-             ipset --flush f2b-<name>
+             <actionflush>
              ipset --destroy f2b-<name>
 
 # Option:  actionban

config/action.d/iptables-ipset-proto6-allports.conf

@@ -12,6 +12,9 @@
 #
 # If you are running on an older kernel you make need to patch in external
 # modules which probably won't be protocol version 6.
+#
+# Modified: Alexander Koeppe <format_c@online.de>, Serg G. Brester <serg.brester@sebres.de>
+#       made config file IPv6 capable (see new section Init?family=inet6)
 
 [INCLUDES]
 
@@ -23,16 +26,22 @@ before = iptables-common.conf
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = ipset create f2b-<name> hash:ip timeout <bantime>
+actionstart = ipset create <ipmset> hash:ip timeout <bantime><familyopt>
-              <iptables> -I <chain> -m set --match-set f2b-<name> src -j <blocktype>
+              <iptables> -I <chain> -m set --match-set <ipmset> src -j <blocktype>
+
+# Option:  actionflush
+# Notes.:  command executed once to flush IPS, by shutdown (resp. by stop of the jail or this action)
+# Values:  CMD
+#
+actionflush = ipset flush <ipmset>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = <iptables> -D <chain> -m set --match-set f2b-<name> src -j <blocktype>
+actionstop = <iptables> -D <chain> -m set --match-set <ipmset> src -j <blocktype>
-             ipset flush f2b-<name>
+             <actionflush>
-             ipset destroy f2b-<name>
+             ipset destroy <ipmset>
 
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
@@ -40,7 +49,7 @@ actionstop = <iptables> -D <chain> -m set --match-set f2b-<name> src -j <blockty
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = ipset add f2b-<name> <ip> timeout <bantime> -exist
+actionban = ipset add <ipmset> <ip> timeout <bantime> -exist
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -48,7 +57,7 @@ actionban = ipset add f2b-<name> <ip> timeout <bantime> -exist
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = ipset del f2b-<name> <ip> -exist
+actionunban = ipset del <ipmset> <ip> -exist
 
 [Init]
 
@@ -57,3 +66,12 @@ actionunban = ipset del f2b-<name> <ip> -exist
 # Values:  [ NUM ]  Default: 600
 #
 bantime = 600
+
+ipmset = f2b-<name>
+familyopt =
+
+
+[Init?family=inet6]
+
+ipmset = f2b-<name>6
+familyopt = <sp>family inet6

config/action.d/iptables-ipset-proto6.conf

@@ -12,6 +12,9 @@
 #
 # If you are running on an older kernel you make need to patch in external
 # modules.
+#
+# Modified: Alexander Koeppe <format_c@online.de>, Serg G. Brester <serg.brester@sebres.de>
+#       made config file IPv6 capable (see new section Init?family=inet6)
 
 [INCLUDES]
 
@@ -23,16 +26,22 @@ before = iptables-common.conf
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = ipset create f2b-<name> hash:ip timeout <bantime>
+actionstart = ipset create <ipmset> hash:ip timeout <bantime><familyopt>
-              <iptables> -I <chain> -p <protocol> -m multiport --dports <port> -m set --match-set f2b-<name> src -j <blocktype>
+              <iptables> -I <chain> -p <protocol> -m multiport --dports <port> -m set --match-set <ipmset> src -j <blocktype>
+
+# Option:  actionflush
+# Notes.:  command executed once to flush IPS, by shutdown (resp. by stop of the jail or this action)
+# Values:  CMD
+#
+actionflush = ipset flush <ipmset>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = <iptables> -D <chain> -p <protocol> -m multiport --dports <port> -m set --match-set f2b-<name> src -j <blocktype>
+actionstop = <iptables> -D <chain> -p <protocol> -m multiport --dports <port> -m set --match-set <ipmset> src -j <blocktype>
-             ipset flush f2b-<name>
+             <actionflush>
-             ipset destroy f2b-<name>
+             ipset destroy <ipmset>
 
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
@@ -40,7 +49,7 @@ actionstop = <iptables> -D <chain> -p <protocol> -m multiport --dports <port> -m
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = ipset add f2b-<name> <ip> timeout <bantime> -exist
+actionban = ipset add <ipmset> <ip> timeout <bantime> -exist
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -48,7 +57,7 @@ actionban = ipset add f2b-<name> <ip> timeout <bantime> -exist
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = ipset del f2b-<name> <ip> -exist
+actionunban = ipset del <ipmset> <ip> -exist
 
 [Init]
 
@@ -57,3 +66,12 @@ actionunban = ipset del f2b-<name> <ip> -exist
 # Values:  [ NUM ]  Default: 600
 #
 bantime = 600
+
+ipmset = f2b-<name>
+familyopt =
+
+
+[Init?family=inet6]
+
+ipmset = f2b-<name>6
+familyopt = <sp>family inet6

config/action.d/iptables-multiport-log.conf

@@ -26,13 +26,19 @@ actionstart = <iptables> -N f2b-<name>
               <iptables> -I f2b-<name>-log -j LOG --log-prefix "$(expr f2b-<name> : '\(.\{1,23\}\)'):DROP " --log-level warning -m limit --limit 6/m --limit-burst 2
               <iptables> -A f2b-<name>-log -j <blocktype>
 
+# Option:  actionflush
+# Notes.:  command executed once to flush IPS, by shutdown (resp. by stop of the jail or this action)
+# Values:  CMD
+#
+actionflush = <iptables> -F f2b-<name>
+              <iptables> -F f2b-<name>-log
+
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
 actionstop = <iptables> -D <chain> -p <protocol> -m multiport --dports <port> -j f2b-<name>
-             <iptables> -F f2b-<name>
+             <actionflush>
-             <iptables> -F f2b-<name>-log
              <iptables> -X f2b-<name>
              <iptables> -X f2b-<name>-log
 

config/action.d/iptables-multiport.conf

@@ -23,7 +23,7 @@ actionstart = <iptables> -N f2b-<name>
 # Values:  CMD
 #
 actionstop = <iptables> -D <chain> -p <protocol> -m multiport --dports <port> -j f2b-<name>
-             <iptables> -F f2b-<name>
+             <actionflush>
              <iptables> -X f2b-<name>
 
 # Option:  actioncheck

config/action.d/iptables-new.conf

@@ -25,7 +25,7 @@ actionstart = <iptables> -N f2b-<name>
 # Values:  CMD
 #
 actionstop = <iptables> -D <chain> -m state --state NEW -p <protocol> --dport <port> -j f2b-<name>
-             <iptables> -F f2b-<name>
+             <actionflush>
              <iptables> -X f2b-<name>
 
 # Option:  actioncheck

config/action.d/iptables-xt_recent-echo.conf

@@ -2,7 +2,8 @@
 #
 # Author: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
 #
-#
+# Modified: Alexander Koeppe <format_c@online.de>, Serg G. Brester <serg.brester@sebres.de>
+#       made config file IPv6 capable
 
 [INCLUDES]
 
@@ -22,30 +23,36 @@ before = iptables-common.conf
 # iptables-persistent package).
 # 
 # Explanation of the rule below:
-#    Check if any packets coming from an IP on the f2b-<name>
+#    Check if any packets coming from an IP on the <iptname>
 #    list have been seen in the last 3600 seconds. If yes, update the
 #    timestamp for this IP and drop the packet. If not, let the packet
 #    through.
 #
-#    Fail2ban inserts blacklisted hosts into the f2b-<name> list
+#    Fail2ban inserts blacklisted hosts into the <iptname> list
 #    and removes them from the list after some time, according to its
 #    own rules. The 3600 second timeout is independent and acts as a
 #    safeguard in case the fail2ban process dies unexpectedly. The
 #    shorter of the two timeouts actually matters.
-actionstart = if [ `id -u` -eq 0 ];then <iptables> -I <chain> -m recent --update --seconds 3600 --name f2b-<name> -j <blocktype>;fi
+actionstart = if [ `id -u` -eq 0 ];then <iptables> -I <chain> -m recent --update --seconds 3600 --name <iptname> -j <blocktype>;fi
+
+# Option:  actionflush
+#
+# [TODO] Flushing is currently not implemented for xt_recent
+#
+actionflush = 
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = echo / > /proc/net/xt_recent/f2b-<name>
+actionstop = echo / > /proc/net/xt_recent/<iptname>
-             if [ `id -u` -eq 0 ];then <iptables> -D <chain> -m recent --update --seconds 3600 --name f2b-<name> -j <blocktype>;fi
+             if [ `id -u` -eq 0 ];then <iptables> -D <chain> -m recent --update --seconds 3600 --name <iptname> -j <blocktype>;fi
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
 # Values:  CMD
 #
-actioncheck = test -e /proc/net/xt_recent/f2b-<name>
+actioncheck = test -e /proc/net/xt_recent/<iptname>
 
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
@@ -53,7 +60,7 @@ actioncheck = test -e /proc/net/xt_recent/f2b-<name>
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = echo +<ip> > /proc/net/xt_recent/f2b-<name>
+actionban = echo +<ip> > /proc/net/xt_recent/<iptname>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -61,7 +68,12 @@ actionban = echo +<ip> > /proc/net/xt_recent/f2b-<name>
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = echo -<ip> > /proc/net/xt_recent/f2b-<name>
+actionunban = echo -<ip> > /proc/net/xt_recent/<iptname>
 
 [Init]
 
+iptname              = f2b-<name>
+
+[Init?family=inet6]
+
+iptname = f2b-<name>6

config/action.d/iptables.conf

@@ -23,7 +23,7 @@ actionstart = <iptables> -N f2b-<name>
 # Values:  CMD
 #
 actionstop = <iptables> -D <chain> -p <protocol> --dport <port> -j f2b-<name>
-             <iptables> -F f2b-<name>
+             <actionflush>
              <iptables> -X f2b-<name>
 
 # Option:  actioncheck

config/action.d/mail-buffered.conf

@@ -6,6 +6,9 @@
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
@@ -14,7 +17,7 @@ actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Output will be buffered until <lines> lines are available.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
+              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on <fq-hostname>" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -25,13 +28,13 @@ actionstop = if [ -f <tmpfile> ]; then
                  These hosts have been banned by Fail2Ban.\n
                  `cat <tmpfile>`
                  Regards,\n
-                 Fail2Ban"|mail -s "[Fail2Ban] <name>: Summary from `uname -n`" <dest>
+                 Fail2Ban"|mail -s "[Fail2Ban] <name>: Summary from <fq-hostname>" <dest>
                  rm <tmpfile>
              fi
              printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
+             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on <fq-hostname>" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command

config/action.d/mail-whois-lines.conf

@@ -7,9 +7,13 @@
 [INCLUDES]
 
 before = mail-whois-common.conf
+         helpers-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
@@ -17,7 +21,7 @@ before = mail-whois-common.conf
 actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
+              Fail2Ban" | <mailcmd> "[Fail2Ban] <name>: started on <fq-hostname>" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -26,7 +30,7 @@ actionstart = printf %%b "Hi,\n
 actionstop = printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
+             Fail2Ban" | <mailcmd> "[Fail2Ban] <name>: stopped on <fq-hostname>" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -40,15 +44,19 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Hi,\n
+
+_ban_mail_content = ( printf %%b "Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
             <failures> attempts against <name>.\n\n
-            Here is more information about <ip> :\n
+            Here is more information about <ip> :\n"
-            `%(_whois_command)s`\n\n
+            %(_whois_command)s;
-            Lines containing IP:<ip> in <logpath>\n
+            printf %%b "\nLines containing failures of <ip> (max <grepmax>)\n";
-            `grep -E <grepopts> '(^|[^0-9])<ip>([^0-9]|$)' <logpath>`\n\n
+            %(_grep_logs)s;
+            printf %%b "\n
             Regards,\n
-            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from  `uname -n`" <dest>
+            Fail2Ban" )
+
+actionban = %(_ban_mail_content)s | <mailcmd> "[Fail2Ban] <name>: banned <ip> from <fq-hostname>" <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -60,6 +68,12 @@ actionunban =
 
 [Init]
 
+# Option:  mailcmd
+# Notes.:  Your system mail command. Is passed 2 args: subject and recipient
+# Values:  CMD
+#
+mailcmd = mail -s
+
 # Default name of the chain
 #
 name = default
@@ -74,4 +88,5 @@ logpath = /dev/null
 
 # Number of log lines to include in the email
 #
-grepopts = -m 1000
+#grepmax = 1000
+#grepopts = -m <grepmax>

config/action.d/mail-whois.conf

@@ -10,6 +10,9 @@ before = mail-whois-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
@@ -17,7 +20,7 @@ before = mail-whois-common.conf
 actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
+              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on <fq-hostname>" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -26,7 +29,7 @@ actionstart = printf %%b "Hi,\n
 actionstop = printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
+             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on <fq-hostname>" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -46,7 +49,7 @@ actionban = printf %%b "Hi,\n
             Here is more information about <ip> :\n
             `%(_whois_command)s`\n
             Regards,\n
-            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>
+            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from <fq-hostname>" <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the

config/action.d/mail.conf

@@ -6,6 +6,9 @@
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
@@ -13,7 +16,7 @@
 actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started  on `uname -n`" <dest>
+              Fail2Ban"|mail -s "[Fail2Ban] <name>: started  on <fq-hostname>" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -22,7 +25,7 @@ actionstart = printf %%b "Hi,\n
 actionstop = printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
+             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on <fq-hostname>" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -40,7 +43,7 @@ actionban = printf %%b "Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
             <failures> attempts against <name>.\n
             Regards,\n
-            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>
+            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from <fq-hostname>" <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the

config/action.d/nftables-allports.conf

@@ -17,6 +17,6 @@ before = nftables-common.conf
 # Notes.:  additional expressions for nftables filter rule
 # Values:  nftables expressions
 #
-nftables_mode = ip protocol <protocol>
+nftables_mode = meta l4proto <protocol>
 
 [Init]

config/action.d/nftables-common.conf

@@ -28,11 +28,11 @@ nftables_mode = <protocol> dport \{ <port> \}
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = <nftables> add set <nftables_family> <nftables_table> f2b-<name> \{ type <nftables_type>\; \}
+actionstart = <nftables> add set <nftables_family> <nftables_table> <set_name> \{ type <nftables_type>\; \}
-              <nftables> insert rule <nftables_family> <nftables_table> <chain> %(nftables_mode)s ip saddr @f2b-<name> <blocktype>
+              <nftables> insert rule <nftables_family> <nftables_table> <chain> %(nftables_mode)s <address_family> saddr @<set_name> <blocktype>
 
 _nft_list = <nftables> --handle --numeric list chain <nftables_family> <nftables_table> <chain>
-_nft_get_handle_id = grep -m1 'ip saddr @f2b-<name> <blocktype> # handle' | grep -oe ' handle [0-9]*'
+_nft_get_handle_id = grep -m1 '<address_family> saddr @<set_name> <blocktype> # handle' | grep -oe ' handle [0-9]*'
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -40,13 +40,13 @@ _nft_get_handle_id = grep -m1 'ip saddr @f2b-<name> <blocktype> # handle' | grep
 #
 actionstop = HANDLE_ID=$(%(_nft_list)s | %(_nft_get_handle_id)s)
              <nftables> delete rule <nftables_family> <nftables_table> <chain> $HANDLE_ID
-             <nftables> delete set <nftables_family> <nftables_table> f2b-<name>
+             <nftables> delete set <nftables_family> <nftables_table> <set_name>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
 # Values:  CMD
 #
-actioncheck = <nftables> list chain <nftables_family> <nftables_table> <chain> | grep -q '@f2b-<name>[ \t]'
+actioncheck = <nftables> list chain <nftables_family> <nftables_table> <chain> | grep -q '@<set_name>[ \t]'
 
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
@@ -54,7 +54,7 @@ actioncheck = <nftables> list chain <nftables_family> <nftables_table> <chain> |
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = <nftables> add element <nftables_family> <nftables_table> f2b-<name> \{ <ip> \}
+actionban = <nftables> add element <nftables_family> <nftables_table> <set_name> \{ <ip> \}
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -62,7 +62,7 @@ actionban = <nftables> add element <nftables_family> <nftables_table> f2b-<name>
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = <nftables> delete element <nftables_family> <nftables_table> f2b-<name> \{ <ip> \}
+actionunban = <nftables> delete element <nftables_family> <nftables_table> <set_name> \{ <ip> \}
 
 [Init]
 
@@ -117,3 +117,19 @@ blocktype = reject
 # Notes.:  Actual command to be executed, including common to all calls options
 # Values:  STRING
 nftables = nft
+
+# Option: set_name
+# Notes.: The name of the nft set used to store banned addresses
+# Values: STRING
+set_name = f2b-<name>
+
+# Option: address_family
+# Notes.: The family of the banned addresses
+# Values: [ ip | ip6 ]
+address_family = ip
+
+[Init?family=inet6]
+
+nftables_type = ipv6_addr
+set_name = f2b-<name>6
+address_family = ip6

config/action.d/nginx-block-map.conf

@@ -0,0 +1,108 @@
+# Fail2Ban configuration file for black-listing via nginx
+#
+# Author: Serg G. Brester (aka sebres)
+#
+# To use 'nginx-block-map' action you should define some special blocks in your nginx configuration,
+# and use it hereafter in your locations (to notify fail2ban by failure, resp. nginx by ban).
+#
+# Example (argument "token_id" resp. cookie "session_id" used here as unique identifier for user):
+#
+#   http {
+#     ...
+#     # maps to check user is blacklisted (banned in f2b):
+#     #map $arg_token_id      $blck_lst_tok { include blacklisted-tokens.map; }
+#     map  $cookie_session_id $blck_lst_ses { include blacklisted-sessions.map; }
+#     ...
+#     # special log-format to notify fail2ban about failures:
+#     log_format f2b_session_errors '$msec failure "$cookie_session_id" - $remote_addr - $remote_user '
+#      ;#                  '"$request" $status $bytes_sent '
+#       #                  '"$http_referer" "$http_user_agent"';
+#
+#     # location checking blacklisted values:
+#     location ... {
+#       # check banned sessionid:
+#       if ($blck_lst_ses != "") {
+#         try_files "" @f2b-banned;
+#       }
+#       ...
+#       # notify fail2ban about a failure inside nginx:
+#       error_page 401 = @notify-f2b;
+#       ...
+#     }
+#     ...
+#     # location for return with "403 Forbidden" if banned:
+#     location @f2b-banned {
+#       default_type text/html;
+#       return 403 "<br/><center>
+#         <b style=\"color:red; font-size:18pt; border:2pt solid black; padding:5pt;\">
+#         You are banned!</b></center>";
+#     }
+#     ...
+#     # location to notify fail2ban about a failure inside nginx:
+#     location @notify-f2b {
+#       access_log /var/log/nginx/f2b-auth-errors.log f2b_session_errors;
+#     }
+#   }
+#   ...
+#
+# Note that quote-character (and possibly other special characters) are not allowed currently as session-id.
+# Thus please add any session-id validation rule in your locations (or in the corresponding backend-service), 
+# like in example below:
+#
+#   location ... {
+#     if ($cookie_session_id !~ "^[\w\-]+$") {
+#       return  403 "Wrong session-id"
+#     }
+#     ...
+#   }
+#
+# The parameters for jail corresponding log-format (f2b_session_errors):
+#
+#   [nginx-blck-lst]
+#   filter =
+#   datepattern = ^Epoch
+#   failregex = ^ failure "<F-ID>[^"]+</F-ID>" - <ADDR>
+#   usedns = no
+#
+# The same log-file can be used for IP-related jail (additionally to session-related, to ban very bad IPs):
+#
+#   [nginx-blck-ip]
+#   maxretry = 100
+#   filter =
+#   datepattern = ^Epoch
+#   failregex = ^ failure "[^"]+" - <ADDR>
+#   usedns = no
+#
+
+[Definition]
+
+# path to configuration of nginx (used to target nginx-instance in multi-instance system,
+# and as path for the blacklisted map):
+srv_cfg_path = /etc/nginx/
+
+# cmd-line arguments to supply to test/reload nginx:
+#srv_cmd = nginx -c %(srv_cfg_path)s/nginx.conf
+srv_cmd = nginx
+
+# first test configuration is correct, hereafter send reload signal:
+blck_lst_reload = %(srv_cmd)s -qt; if [ $? -eq 0 ]; then
+                    %(srv_cmd)s -s reload; if [ $? -ne 0 ]; then echo 'reload failed.'; fi;
+                  fi;
+
+# map-file for nginx, can be redefined using `action = nginx-block-map[blck_lst_file="/path/file.map"]`:
+blck_lst_file = %(srv_cfg_path)s/blacklisted-sessions.map
+
+# Action definition:
+
+actionstart_on_demand = false
+actionstart = touch '%(blck_lst_file)s'
+
+actionflush = truncate -s 0 '%(blck_lst_file)s'; %(blck_lst_reload)s
+
+actionstop = %(actionflush)s
+
+actioncheck = 
+
+actionban = echo "\\\\<fid> 1;" >> '%(blck_lst_file)s'; %(blck_lst_reload)s
+
+actionunban = id=$(echo "<fid>" | sed -e 's/[]\/$*.^|[]/\\&/g'); sed -i "/$id 1;/d" %(blck_lst_file)s; %(blck_lst_reload)s

config/action.d/pf.conf

@@ -3,6 +3,7 @@
 # OpenBSD pf ban/unban
 #
 # Author: Nick Hilliard <nick@foobar.org>
+# Modified by: Alexander Koeppe making PF work seamless and with IPv4 and IPv6
 #
 #
 
@@ -12,23 +13,49 @@
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-# we don't enable PF automatically, as it will be enabled elsewhere
+# we don't enable PF automatically; to enable run pfctl -e 
-actionstart = 
+# or add `pf_enable="YES"` to /etc/rc.conf (tested on FreeBSD)
+# also, these rulesets are loaded into (nested) anchors
+# to enable them, add as wildcard:
+#     anchor "f2b/*"
+# or using jail names:
+#     anchor f2b {
+#        anchor name1
+#        anchor name2
+#        ...
+#     }
+# to your main pf ruleset, where "namei" are the names of the jails
+# which invoke this action
+actionstart = echo "table <<tablename>-<name>> persist counters" | <pfctl> -f-
+              port="<port>"; if [ "$port" != "" ] && case "$port" in \{*) false;; esac; then port="{$port}"; fi
+              echo "<block> proto <protocol> from <<tablename>-<name>> to <actiontype>" | <pfctl> -f-
 
+# Option:  start_on_demand - to start action on demand
+# Example: `action=pf[actionstart_on_demand=true]`
+actionstart_on_demand = false
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-# we don't disable PF automatically either
+# we only disable PF rules we've installed prior
-actionstop = 
+actionstop = <pfctl> -sr 2>/dev/null | grep -v <tablename>-<name> | <pfctl> -f-
+             %(actionflush)s
+             <pfctl> -t <tablename>-<name> -T kill
+
+
+# Option:  actionflush
+# Notes.:  command executed once to flush IPS, by shutdown (resp. by stop of the jail or this action)
+# Values:  CMD
+#
+actionflush = <pfctl> -t <tablename>-<name> -T flush
 
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
 # Values:  CMD
 #
-actioncheck = 
+actioncheck = <pfctl> -sr | grep -q <tablename>-<name>
 
 
 # Option:  actionban
@@ -39,7 +66,7 @@ actioncheck =
 #          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
-actionban = /sbin/pfctl -t <tablename> -T add <ip>/32
+actionban = <pfctl> -t <tablename>-<name> -T add <ip>
 
 
 # Option:  actionunban
@@ -51,12 +78,47 @@ actionban = /sbin/pfctl -t <tablename> -T add <ip>/32
 # Values:  CMD
 #
 # note -r option used to remove matching rule
-actionunban = /sbin/pfctl -t <tablename> -T delete <ip>/32
+actionunban = <pfctl> -t <tablename>-<name> -T delete <ip>
+
+# Option: pfctl
+#
+# Use anchor as jailname to manipulate affected rulesets only.
+# If more parameter expected it can be extended with `pf[pfctl="<known/pfctl> ..."]`
+# 
+pfctl = pfctl -a f2b/<name>
 
 [Init]
 # Option:  tablename
 # Notes.:  The pf table name.
 # Values:  [ STRING ]
 #
-tablename = fail2ban
+tablename = f2b
+
+# Option: block
+#
+# The action you want pf to take.
+# Probably, you want "block quick", but adjust as needed.
+block = block quick
+
+# Option:  protocol
+# Notes.:  internally used by config reader for interpolations.
+# Values:  [ tcp | udp | icmp | ipv6-icmp ] Default: tcp
+#
+protocol = tcp
+
+# Option: actiontype
+# Notes.: defines additions to the blocking rule
+# Values: leave empty to block all attempts from the host
+# Default: Value of the multiport
+actiontype = <multiport>
+
+# Option: allports
+# Notes.: default addition to block all ports
+# Usage.: use in jail config: "banaction = pf[actiontype=<allports>]"
+allports = any
+
+# Option: multiport
+# Notes.: addition to block access only to specific ports
+# Usage.: use in jail config: "banaction = pf[actiontype=<multiport>]"
+multiport = any port $port
 

config/action.d/sendmail-buffered.conf

@@ -10,11 +10,14 @@ before = sendmail-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionstart
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
+actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on <fq-hostname>
               From: <sendername> <<sender>>
               To: <dest>\n
               Hi,\n
@@ -28,7 +31,7 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
 # Values:  CMD
 #
 actionstop = if [ -f <tmpfile> ]; then
-                 printf %%b "Subject: [Fail2Ban] <name>: summary from `uname -n`
+                 printf %%b "Subject: [Fail2Ban] <name>: summary from <fq-hostname>
                  From: <sendername> <<sender>>
                  To: <dest>\n
                  Hi,\n
@@ -38,7 +41,7 @@ actionstop = if [ -f <tmpfile> ]; then
                  Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
                  rm <tmpfile>
              fi
-             printf %%b "Subject: [Fail2Ban] <name>: stopped  on `uname -n`
+             printf %%b "Subject: [Fail2Ban] <name>: stopped  on <fq-hostname>
              From: Fail2Ban <<sender>>
              To: <dest>\n
              Hi,\n
@@ -61,7 +64,7 @@ actioncheck =
 actionban = printf %%b "`date`: <ip> (<failures> failures)\n" >> <tmpfile>
             LINE=$( wc -l <tmpfile> | awk '{ print $1 }' )
             if [ $LINE -ge <lines> ]; then
-                printf %%b "Subject: [Fail2Ban] <name>: summary from `uname -n`
+                printf %%b "Subject: [Fail2Ban] <name>: summary from <fq-hostname>
                 From: <sendername> <<sender>>
                 To: <dest>\n
                 Hi,\n

config/action.d/sendmail-common.conf

@@ -14,7 +14,7 @@ after = sendmail-common.local
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
+actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on <fq-hostname>
               Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
               From: <sendername> <<sender>>
               To: <dest>\n
@@ -27,7 +27,7 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
 # Notes.:  command executed once at the end of Fail2Ban
 # Values:  CMD
 #
-actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on `uname -n`
+actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on <fq-hostname>
              Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
              From: <sendername> <<sender>>
              To: <dest>\n

config/action.d/sendmail-geoip-lines.conf

@@ -7,9 +7,13 @@
 [INCLUDES]
 
 before = sendmail-common.conf
+         helpers-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionban
 # Notes.:  Command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
@@ -19,7 +23,7 @@ before = sendmail-common.conf
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
+actionban = ( printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from <fq-hostname>
             Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n
@@ -32,11 +36,12 @@ actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
             http://whois.domaintools.com/<ip>\n\n
             Country:`geoiplookup -f /usr/share/GeoIP/GeoIP.dat "<ip>" | cut -d':' -f2-`
             AS:`geoiplookup -f /usr/share/GeoIP/GeoIPASNum.dat "<ip>" | cut -d':' -f2-`
-            hostname: `host -t A <ip> 2>&1`\n\n
+            hostname: <ip-host>\n\n
-            Lines containing IP:<ip> in <logpath>\n
+            Lines containing failures of <ip>\n";
-            `grep -E <grepopts> '(^|[^0-9])<ip>([^0-9]|$)' <logpath>`\n\n
+            %(_grep_logs)s;
+            printf %%b "\n
             Regards,\n
-            Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
+            Fail2Ban" ) | /usr/sbin/sendmail -f <sender> <dest>
 
 [Init]
 
@@ -50,4 +55,5 @@ logpath = /dev/null
 
 # Number of log lines to include in the email
 #
-grepopts = -m 1000
+#grepmax = 1000
+#grepopts = -m <grepmax>

config/action.d/sendmail-whois-ipjailmatches.conf

@@ -10,13 +10,16 @@ before = sendmail-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
+actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from <fq-hostname>
             Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n

config/action.d/sendmail-whois-ipmatches.conf

@@ -10,13 +10,16 @@ before = sendmail-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
+actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from <fq-hostname>
             Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n

config/action.d/sendmail-whois-lines.conf

@@ -7,16 +7,20 @@
 [INCLUDES]
 
 before = sendmail-common.conf
+         helpers-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
+actionban = ( printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from <fq-hostname>
             Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n
@@ -25,10 +29,11 @@ actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
             <failures> attempts against <name>.\n\n
             Here is more information about <ip> :\n
             `/usr/bin/whois <ip> || echo missing whois program`\n\n
-            Lines containing IP:<ip> in <logpath>\n
+            Lines containing failures of <ip>\n";
-            `grep -E <grepopts> '(^|[^0-9])<ip>([^0-9]|$)' <logpath>`\n\n
+            %(_grep_logs)s;
+            printf %%b "\n
             Regards,\n
-            Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
+            Fail2Ban" ) | /usr/sbin/sendmail -f <sender> <dest>
 
 [Init]
 
@@ -42,4 +47,5 @@ logpath = /dev/null
 
 # Number of log lines to include in the email
 #
-grepopts = -m 1000
+#grepmax = 1000
+#grepopts = -m <grepmax>

config/action.d/sendmail-whois-matches.conf

@@ -10,13 +10,16 @@ before = sendmail-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
+actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from <fq-hostname>
             Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n

config/action.d/sendmail-whois.conf

@@ -10,13 +10,16 @@ before = sendmail-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
+actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from <fq-hostname>
             Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n

config/action.d/sendmail.conf

@@ -10,13 +10,16 @@ before = sendmail-common.conf
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
+actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from <fq-hostname>
             Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
             From: <sendername> <<sender>>
             To: <dest>\n

config/action.d/shorewall.conf

@@ -40,7 +40,7 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = shorewall <blocktype> <ip>
+actionban = shorewall<family> <blocktype> <ip>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -48,12 +48,26 @@ actionban = shorewall <blocktype> <ip>
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = shorewall allow <ip>
+actionunban = shorewall<family> allow <ip>
+
 
 [Init]
 
+# Option:  family
+# Note:    Control which version of command is executed
+# Values:  Empty or 6 in case of IPv6
+family = 
+
 # Option:  blocktype
 # Note:    This is what the action does with rules.
 #          See man page of shorewall for options that include drop, logdrop, reject, or logreject
 # Values:  STRING
 blocktype = reject
+
+[Init?family=inet6]
+
+# Option:  family
+# Note:    Control which version of command is executed
+# Values:  Empty or 6 in case of IPv6
+family = 6
+

config/action.d/smtp.py

@@ -123,9 +123,12 @@ class SMTPAction(ActionBase):
 		self.message_values = CallingMap(
 			jailname = self._jail.name,
 			hostname = socket.gethostname,
-			bantime = self._jail.actions.getBanTime,
+			bantime = lambda: self._jail.actions.getBanTime(),
 			)
 
+		# bypass ban/unban for restored tickets
+		self.norestored = 1
+
 	def _sendMessage(self, subject, text):
 		"""Sends message based on arguments and instance's properties.
 
@@ -211,6 +214,8 @@ class SMTPAction(ActionBase):
 			Dictionary which includes information in relation to
 			the ban.
 		"""
+		if aInfo.get('restored'):
+			return
 		aInfo.update(self.message_values)
 		message = "".join([
 			messages['ban']['head'],

config/action.d/xarf-login-attack.conf

@@ -22,7 +22,7 @@
 # Login-Attack, Malware-Attack, Fraud (Phishing, etc.), Info DNSBL
 #
 # For details see:
-# https://github.com/abusix/xarf-specification
+# https://github.com/xarf/xarf-specification
 # http://www.x-arf.org/schemata.html
 #
 # Author: Daniel Black
@@ -32,6 +32,9 @@
 
 [Definition]
 
+# bypass ban/unban for restored tickets
+norestored = 1
+
 actionstart =
 
 actionstop =
@@ -43,14 +46,14 @@ actionban = oifs=${IFS}; IFS=.;SEP_IP=( <ip> ); set -- ${SEP_IP}; ADDRESSES=$(di
             FROM=<sender>
             SERVICE=<service>
             FAILURES=<failures>
-            REPORTID=<time>@`uname -n`
+            REPORTID=<time>@<fq-hostname>
             TLP=<tlp>
             PORT=<port>
             DATE=`LC_ALL=C date --date=@<time> +"%%a, %%d %%h %%Y %%T %%z"`
             if [ ! -z "$ADDRESSES" ]; then
-                (printf -- %%b "<header>\n<message>\n<report>\n";
+                (printf -- %%b "<header>\n<message>\n<report>\n\n";
                  date '+Note: Local timezone is %%z (%%Z)';
-                 printf -- %%b "<ipmatches>\n\n<footer>") | <mailcmd> <mailargs> ${ADDRESSES//,/\" \"}
+                 printf -- %%b "\n<ipmatches>\n\n<footer>") | <mailcmd> <mailargs> ${ADDRESSES//,/\" \"}
             fi
 
 actionunban =
@@ -116,7 +119,7 @@ logpath = /dev/null
 
 # Option:  sender
 # Notes.:  This is the sender that is included in the XARF report
-sender = fail2ban@`uname -n`
+sender = fail2ban@<fq-hostname>
 
 # Option:  port
 # Notes.:  This is the port number that received the login-attack

config/fail2ban.conf

@@ -30,7 +30,7 @@ loglevel = INFO
 #         using logrotate -- also adjust or disable rotation in the
 #         corresponding configuration file
 #         (e.g. /etc/logrotate.d/fail2ban on Debian systems)
-# Values: [ STDOUT | STDERR | SYSLOG | FILE ]  Default: STDERR
+# Values: [ STDOUT | STDERR | SYSLOG | SYSOUT | FILE ]  Default: STDERR
 #
 logtarget = /var/log/fail2ban.log
 
@@ -66,4 +66,4 @@ dbfile = /var/lib/fail2ban/fail2ban.sqlite3
 # Options: dbpurgeage
 # Notes.: Sets age at which bans should be purged from the database
 # Values: [ SECONDS ] Default: 86400 (24hours)
-dbpurgeage = 86400
+dbpurgeage = 1d

config/filter.d/3proxy.conf

@@ -9,6 +9,8 @@ failregex = ^\s[+-]\d{4} \S+ \d{3}0[1-9] \S+ <HOST>:\d+ [\d.]+:\d+ \d+ \d+ \d+\s
 
 ignoreregex = 
 
+datepattern = {^LN-BEG}
+
 # DEV Notes:
 # http://www.3proxy.ru/howtoe.asp#ERRORS indicates that 01-09 are
 # all authentication problems (%E field)

config/filter.d/apache-auth.conf

@@ -9,20 +9,21 @@ before = apache-common.conf
 
 [Definition]
 
+prefregex = ^%(_apache_error_client)s (?:AH\d+: )?<F-CONTENT>.+</F-CONTENT>$
 
-failregex = ^%(_apache_error_client)s (AH(01797|01630): )?client denied by server configuration: (uri )?\S*(, referer: \S+)?\s*$
+# auth_type = ((?:Digest|Basic): )?
-            ^%(_apache_error_client)s (AH01617: )?user .*? authentication failure for "\S*": Password Mismatch(, referer: \S+)?$
+auth_type = ([A-Z]\w+: )?
-            ^%(_apache_error_client)s (AH01618: )?user .*? not found(: )?\S*(, referer: \S+)?\s*$
+
-            ^%(_apache_error_client)s (AH01614: )?client used wrong authentication scheme: \S*(, referer: \S+)?\s*$
+failregex = ^client (?:denied by server configuration|used wrong authentication scheme)\b
-            ^%(_apache_error_client)s (AH\d+: )?Authorization of user \S+ to access \S* failed, reason: .*$
+            ^user <F-USER>(?:\S*|.*?)</F-USER> (?:auth(?:oriz|entic)ation failure|not found|denied by provider)\b
-            ^%(_apache_error_client)s (AH0179[24]: )?(Digest: )?user .*?: password mismatch: \S*(, referer: \S+)?\s*$
+            ^Authorization of user <F-USER>(?:\S*|.*?)</F-USER> to access .*? failed\b
-            ^%(_apache_error_client)s (AH0179[01]: |Digest: )user `.*?' in realm `.+' (not found|denied by provider): \S*(, referer: \S+)?\s*$
+            ^%(auth_type)suser <F-USER>(?:\S*|.*?)</F-USER>: password mismatch\b
-            ^%(_apache_error_client)s (AH01631: )?user .*?: authorization failure for "\S*":(, referer: \S+)?\s*$
+            ^%(auth_type)suser `<F-USER>(?:[^']*|.*?)</F-USER>' in realm `.+' (not found|denied by provider)\b
-            ^%(_apache_error_client)s (AH01775: )?(Digest: )?invalid nonce .* received - length is not \S+(, referer: \S+)?\s*$
+            ^%(auth_type)sinvalid nonce .* received - length is not\b
-            ^%(_apache_error_client)s (AH01788: )?(Digest: )?realm mismatch - got `.*?' but expected `.+'(, referer: \S+)?\s*$
+            ^%(auth_type)srealm mismatch - got `(?:[^']*|.*?)' but expected\b
-            ^%(_apache_error_client)s (AH01789: )?(Digest: )?unknown algorithm `.*?' received: \S*(, referer: \S+)?\s*$
+            ^%(auth_type)sunknown algorithm `(?:[^']*|.*?)' received\b
-            ^%(_apache_error_client)s (AH01793: )?invalid qop `.*?' received: \S*(, referer: \S+)?\s*$
+            ^invalid qop `(?:[^']*|.*?)' received\b
-            ^%(_apache_error_client)s (AH01777: )?(Digest: )?invalid nonce .*? received - user attempted time travel(, referer: \S+)?\s*$
+            ^%(auth_type)sinvalid nonce .*? received - user attempted time travel\b
 
 ignoreregex = 
 
@@ -43,14 +44,17 @@ ignoreregex =
 # all of these expressions. Lots of submodules like mod_authz_* return back to mod_authz_core
 # to return the actual failure.
 #
+# Note that URI can contain spaces.
+#
 # See also: http://wiki.apache.org/httpd/ListOfErrors
 # Expressions that don't have tests and aren't common.
 # more be added with  https://issues.apache.org/bugzilla/show_bug.cgi?id=55284 
-#     ^%(_apache_error_client)s (AH01778: )?user .*: nonce expired \([\d.]+ seconds old - max lifetime [\d.]+\) - sending new nonce\s*$
+#     ^user .*: nonce expired \([\d.]+ seconds old - max lifetime [\d.]+\) - sending new nonce\s*$
-#     ^%(_apache_error_client)s (AH01779: )?user .*: one-time-nonce mismatch - sending new nonce\s*$
+#     ^user .*: one-time-nonce mismatch - sending new nonce\s*$
-#     ^%(_apache_error_client)s (AH02486: )?realm mismatch - got `.*' but no realm specified\s*$
+#     ^realm mismatch - got `(?:[^']*|.*?)' but no realm specified\s*$
 #
-# referer is always in error log messages if it exists added as per the log_error_core function in server/log.c
+# Because url/referer are foreign input, short form of regex used if long enough to idetify failure.
 # 
 # Author: Cyril Jaquier
-# Major edits by Daniel Black
+# Major edits by Daniel Black and Ben Rubson.
+# Rewritten for v.0.10 by Sergey Brester (sebres).

config/filter.d/apache-badbots.conf

@@ -7,13 +7,16 @@
 
 [Definition]
 
-badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider
+badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider|(?:Mozilla/\d+\.\d+ )?Jorgee
 badbots = Atomic_Email_Hunter/4\.0|atSpider/1\.0|autoemailspider|bwh3_user_agent|China Local Browse 2\.6|ContactBot/0\.2|ContentSmartz|DataCha0s/2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailSpider|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Franklin Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Guestbook Auto Submitter|Industry Program 1\.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsCrawl\.com/1\.0 \+http\://letscrawl\.com/|Lincoln State Web Browser|LMQueueBot/0\.2|LWP\:\:Simple/5\.803|Mac Finder 1\.0\.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control - 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missigua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|MVAClient|Mozilla/2\.0 \(compatible; NEWT ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/3\.0 \(compatible; scan4mail \(advanced version\) http\://www\.peterspages\.net/?scan4mail\)|Mozilla/4\.0 \(compatible; Advanced Email Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0 \(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Version\: xxxx Type\:xx\)|NameOfAgent \(CMS Spider\)|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google\.com|ShablastBot 1\.0|snap\.com beta crawler v0|Snapbot/1\.0|Snapbot/1\.0 \(Snap Shots, \+http\://www\.snap\.com\)|sogou develop spider|Sogou Orion spider/3\.0\(\+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sogou spider|Sogou web spider/3\.0\(\+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|VadixBot|WebVulnCrawl\.unknown/1\.0 libwww-perl/5\.803|Wells Search II|WEP Search 00
 
 failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*"(?:%(badbots)s|%(badbotscustom)s)"$
 
 ignoreregex =
 
+datepattern = ^[^\[]*\[({DATE})
+              {^LN-BEG}
+
 # DEV Notes:
 # List of bad bots fetched from http://www.user-agents.org
 # Generated on Thu Nov  7 14:23:35 PST 2013 by files/gen_badbots.

config/filter.d/apache-botsearch.conf

@@ -3,7 +3,7 @@
 # This filter is aimed at blocking specific URLs that don't exist. This
 # could be a set of URLs places in a Disallow: directive in robots.txt or
 # just some web services that don't exist caused bots are searching for
-# exploitable content. This filter is designed to have a low false postitive
+# exploitable content. This filter is designed to have a low false positive
 # rate due.
 #
 # An alternative to this is the apache-noscript filter which blocks all
@@ -23,14 +23,13 @@ before = apache-common.conf
 
 [Definition]
 
-failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): <webroot><block>(, referer: \S+)?\s*$
+prefregex = ^%(_apache_error_client)s (?:AH\d+: )?<F-CONTENT>.+</F-CONTENT>$
-            ^%(_apache_error_client)s script '<webroot><block>' not found or unable to stat(, referer: \S+)?\s*$
+
+failregex = ^(?:File does not exist|script not found or unable to stat): <webroot><block>(, referer: \S+)?\s*$
+            ^script '<webroot><block>' not found or unable to stat(, referer: \S+)?\s*$
 
 ignoreregex = 
 
-
-[Init]
-
 # Webroot represents the webroot on which all other files are based
 webroot = /var/www/
 

config/filter.d/apache-common.conf

@@ -3,12 +3,33 @@
 
 [INCLUDES]
 
+before = common.conf
 # Load customizations if any available
 after = apache-common.local
 
 [DEFAULT]
 
-_apache_error_client = \[\] \[(:?error|\S+:\S+)\]( \[pid \d+(:\S+ \d+)?\])? \[client <HOST>(:\d{1,5})?\]
+# Apache logging mode:
+#   all - universal prefix (logfile, syslog)
+#   logfile - logfile only
+#   syslog - syslog only
+# Use `filter = apache-auth[logging=syslog]` to get more precise regex if apache logs into syslog (ErrorLog syslog).
+# Use `filter = apache-auth[logging=all]` to get universal regex matches both logging variants.
+logging = logfile
+
+# Apache logging prefixes (date-pattern prefix, server, process etc.):
+apache-prefix-syslog = %(__prefix_line)s
+apache-prefix-logfile = \[\]\s
+apache-prefix-all = (?:%(apache-prefix-logfile)s|%(apache-prefix-syslog)s)?
+
+# Setting for __prefix_line (only `logging=syslog`):
+_daemon = (?:apache\d*|httpd(?:/\w+)?)
+
+apache-prefix = <apache-prefix-<logging>>
+
+_apache_error_client = <apache-prefix>\[(:?error|\S+:\S+)\]( \[pid \d+(:\S+ \d+)?\])? \[client <HOST>(:\d{1,5})?\]
+
+datepattern = {^LN-BEG}
 
 # Common prefix for [error] apache messages which also would include <HOST>
 # Depending on the version it could be

config/filter.d/apache-fakegooglebot.conf

@@ -6,6 +6,8 @@ failregex = ^<HOST> .*Googlebot.*$
 
 ignoreregex =
 
+datepattern = ^[^\[]*\[({DATE})
+              {^LN-BEG}
 
 # DEV Notes:
 #

config/filter.d/apache-overflows.conf

@@ -8,12 +8,16 @@ before = apache-common.conf
 
 [Definition]
 
-failregex = ^%(_apache_error_client)s ((AH0013[456]: )?Invalid (method|URI) in request .*( - possible attempt to establish SSL connection on non-SSL port)?|(AH00565: )?request failed: URI too long \(longer than \d+\)|request failed: erroneous characters after protocol string: .*|AH00566: request failed: invalid characters in URI)(, referer: \S+)?$
+failregex = ^%(_apache_error_client)s (?:(?:AH0013[456]: )?Invalid (method|URI) in request\b|(?:AH00565: )?request failed: URI too long \(longer than \d+\)|request failed: erroneous characters after protocol string:|(?:AH00566: )?request failed: invalid characters in URI\b)
 
 ignoreregex =
 
 # DEV Notes:
 #
+# [sebres] Because this apache-log could contain very long URLs (and/or referrer), 
+#          the parsing of it anchored way may be very vulnerable (at least as regards 
+#          the system resources, see gh-1790). Thus rewritten without end-anchor ($).
+# 
 # fgrep -r 'URI too long' httpd-2.*
 #   httpd-2.2.25/server/protocol.c:                          "request failed: URI too long (longer than %d)", r->server->limit_req_line);
 #   httpd-2.4.4/server/protocol.c:                              "request failed: URI too long (longer than %d)",

config/filter.d/apache-pass.conf

@@ -3,16 +3,15 @@
 #
 # The knocking request must have a referer.
 
-[INCLUDES]
-
-before = apache-common.conf
-
 [Definition]
 
 failregex = ^<HOST> - \w+ \[\] "GET <knocking_url> HTTP/1\.[01]" 200 \d+ ".*" "[^-].*"$
 
 ignoreregex =
 
+datepattern = ^[^\[]*\[({DATE})
+              {^LN-BEG}
+
 [Init]
 
 knocking_url = /knocking/

config/filter.d/apache-shellshock.conf

@@ -9,8 +9,10 @@ before = apache-common.conf
 
 [Definition]
 
-failregex = ^%(_apache_error_client)s (AH01215: )?/bin/(ba)?sh: warning: HTTP_.*?: ignoring function definition attempt(, referer: \S+)?\s*$
+prefregex = ^%(_apache_error_client)s (AH01215: )?/bin/([bd]a)?sh: <F-CONTENT>.+</F-CONTENT>$
-            ^%(_apache_error_client)s (AH01215: )?/bin/(ba)?sh: error importing function definition for `HTTP_.*?'(, referer: \S+)?\s*$
+
+failregex = ^warning: HTTP_[^:]+: ignoring function definition attempt(, referer: \S+)?\s*$
+            ^error importing function definition for `HTTP_[^']+'(, referer: \S+)?\s*$
 
 ignoreregex = 
 
@@ -23,4 +25,4 @@ ignoreregex =
 # [Thu Sep 25 09:27:18.813902 2014] [cgi:error] [pid 16860] [client 89.207.132.76:59635] AH01215: /bin/bash: warning: HTTP_TEST: ignoring function definition attempt
 # [Thu Sep 25 09:29:56.141832 2014] [cgi:error] [pid 16864] [client 162.247.73.206:41273] AH01215: /bin/bash: error importing function definition for `HTTP_TEST'
 #
-# Author: Eugene Hopkinson (riot@riot.so)
+# Author: Eugene Hopkinson (e.hopkinson@gmail.com)

config/filter.d/assp.conf

@@ -20,6 +20,9 @@ failregex = ^(:? \[SSL-out\])? <HOST> max sender authentication errors \(\d{,3}\
 
 ignoreregex = 
 
+datepattern = {^LN-BEG}%%b-%%d-%%Exy %%H:%%M:%%S
+              {^LN-BEG}
+
 # DEV Notes:
 # V1 Examples matches:
 #   Apr-27-13 02:33:09 Blocking 217.194.197.97 - too much AUTH errors (41);

config/filter.d/asterisk.conf

@@ -11,26 +11,30 @@ before = common.conf
 
 _daemon = asterisk
 
-__pid_re = (?:\[\d+\])
+__pid_re = (?:\s*\[\d+\])
 
 iso8601 = \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d+[+-]\d{4}
 
 # All Asterisk log messages begin like this:
 log_prefix= (?:NOTICE|SECURITY|WARNING)%(__pid_re)s:?(?:\[C-[\da-f]*\])? [^:]+:\d*(?:(?: in)? \w+:)?
 
-failregex = ^%(__prefix_line)s%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - (Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$
+prefregex = ^%(__prefix_line)s%(log_prefix)s <F-CONTENT>.+</F-CONTENT>$
-            ^%(__prefix_line)s%(log_prefix)s Call from '[^']*' \(<HOST>:\d+\) to extension '[^']*' rejected because extension not found in context
+
-            ^%(__prefix_line)s%(log_prefix)s Host <HOST> failed to authenticate as '[^']*'$
+failregex = ^Registration from '[^']*' failed for '<HOST>(:\d+)?' - (?:Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$
-            ^%(__prefix_line)s%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$
+            ^Call from '[^']*' \(<HOST>:\d+\) to extension '[^']*' rejected because extension not found in context
-            ^%(__prefix_line)s%(log_prefix)s Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$
+            ^(?:Host )?<HOST> (?:failed (?:to authenticate\b|MD5 authentication\b)|tried to authenticate with nonexistent user\b)
-            ^%(__prefix_line)s%(log_prefix)s Failed to authenticate (user|device) [^@]+@<HOST>\S*$
+            ^No registration for peer '[^']*' \(from <HOST>\)$
-            ^%(__prefix_line)s%(log_prefix)s hacking attempt detected '<HOST>'$
+            ^hacking attempt detected '<HOST>'$
-            ^%(__prefix_line)s%(log_prefix)s SecurityEvent="(FailedACL|InvalidAccountID|ChallengeResponseFailed|InvalidPassword)",EventTV="([\d-]+|%(iso8601)s)",Severity="[\w]+",Service="[\w]+",EventVersion="\d+",AccountID="(\d*|<unknown>)",SessionID=".+",LocalAddress="IPV[46]/(UDP|TCP|WS)/[\da-fA-F:.]+/\d+",RemoteAddress="IPV[46]/(UDP|TCP|WS)/<HOST>/\d+"(,Challenge="[\w/]+")?(,ReceivedChallenge="\w+")?(,Response="\w+",ExpectedResponse="\w*")?(,ReceivedHash="[\da-f]+")?(,ACLName="\w+")?$
+            ^SecurityEvent="(?:FailedACL|InvalidAccountID|ChallengeResponseFailed|InvalidPassword)"(?:(?:,(?!RemoteAddress=)\w+="[^"]*")*|.*?),RemoteAddress="IPV[46]/(UDP|TCP|WS)/<HOST>/\d+"(?:,(?!RemoteAddress=)\w+="[^"]*")*$
-            ^%(__prefix_line)s%(log_prefix)s "Rejecting unknown SIP connection from <HOST>"$
+            ^"Rejecting unknown SIP connection from <HOST>"$
-            ^%(__prefix_line)s%(log_prefix)s Request (?:'[^']*' )?from '[^']*' failed for '<HOST>(?::\d+)?'\s\(callid: [^\)]*\) - (?:No matching endpoint found|Not match Endpoint(?: Contact)? ACL|(?:Failed|Error) to authenticate)\s*$
+            ^Request (?:'[^']*' )?from '(?:[^']*|.*?)' failed for '<HOST>(?::\d+)?'\s\(callid: [^\)]*\) - (?:No matching endpoint found|Not match Endpoint(?: Contact)? ACL|(?:Failed|Error) to authenticate)\s*$
+
+# FreePBX (todo: make optional in v.0.10):
+#            ^(%(__prefix_line)s|\[\]\s*WARNING%(__pid_re)s:?(?:\[C-[\da-f]*\])? )[^:]+: Friendly Scanner from <HOST>$
 
 ignoreregex =
 
+datepattern = {^LN-BEG}
 
 # Author: Xavier Devlamynck / Daniel Black
 #

config/filter.d/botsearch-common.conf

@@ -5,7 +5,7 @@
 # Block is the actual non-found directories to block
 block = \/?(<webmail>|<phpmyadmin>|<wordpress>|cgi-bin|mysqladmin)[^,]*
 
-# These are just convient definitions that assist the blocking of stuff that 
+# These are just convenient definitions that assist the blocking of stuff that 
 # isn't installed
 webmail = roundcube|(ext)?mail|horde|(v-?)?webmail
 

config/filter.d/common.conf

@@ -61,4 +61,7 @@ __prefix_line = %(__date_ambit)s?\s*(?:%(__bsd_syslog_verbose)s\s+)?(?:%(__hostn
 # pam_ldap
 __pam_auth = pam_unix
 
+# standardly all formats using prefix have line-begin anchored date:
+datepattern = {^LN-BEG}
+
 # Author: Yaroslav Halchenko

config/filter.d/counter-strike.conf

@@ -8,8 +8,6 @@ failregex = ^: Bad Rcon: "rcon \d+ "\S+"  sv_contact ".*?"" from "<HOST>:\d+"$
 
 ignoreregex =
 
-[Init]
-
 datepattern = ^L %%d/%%m/%%Y - %%H:%%M:%%S
 
 

config/filter.d/courier-auth.conf

@@ -11,9 +11,11 @@ before = common.conf
 
 _daemon = (?:courier)?(?:imapd?|pop3d?)(?:login)?(?:-ssl)?
 
-failregex = ^%(__prefix_line)sLOGIN FAILED, user=.*, ip=\[<HOST>\]$
+failregex = ^%(__prefix_line)sLOGIN FAILED, (?:user|method)=.*, ip=\[<HOST>\]$
 
 ignoreregex = 
 
+datepattern = {^LN-BEG}
+
 # Author: Christoph Haas
 # Modified by: Cyril Jaquier

config/filter.d/courier-smtp.conf

@@ -12,8 +12,10 @@ before = common.conf
 
 _daemon = courieresmtpd
 
-failregex = ^%(__prefix_line)serror,relay=<HOST>,.*: 550 User (<.*> )?unknown\.?$
+prefregex = ^%(__prefix_line)serror,relay=<HOST>,<F-CONTENT>.+</F-CONTENT>$
-            ^%(__prefix_line)serror,relay=<HOST>,msg="535 Authentication failed\.",cmd:( AUTH \S+)?( [0-9a-zA-Z\+/=]+)?(?: \S+)$
+
+failregex = ^[^:]*: 550 User (<.*> )?unknown\.?$
+            ^msg="535 Authentication failed\.",cmd:( AUTH \S+)?( [0-9a-zA-Z\+/=]+)?(?: \S+)$
 
 ignoreregex = 
 

config/filter.d/directadmin.conf

@@ -13,7 +13,6 @@ failregex = ^: \'<HOST>\' \d{1,3} failed login attempt(s)?. \s*
 
 ignoreregex = 
 
-[Init]
 datepattern = ^%%Y:%%m:%%d-%%H:%%M:%%S
 
 #

config/filter.d/dovecot.conf

@@ -7,25 +7,39 @@ before = common.conf
 
 [Definition]
 
-_daemon = (auth|dovecot(-auth)?|auth-worker)
+_auth_worker = (?:dovecot: )?auth(?:-worker)?
+_daemon = (?:dovecot(?:-auth)?|auth)
 
-failregex = ^%(__prefix_line)s(?:%(__pam_auth)s(?:\(dovecot:auth\))?:)?\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* rhost=<HOST>(?:\s+user=\S*)?\s*$
+prefregex = ^%(__prefix_line)s(?:%(_auth_worker)s(?:\([^\)]+\))?: )?(?:%(__pam_auth)s(?:\(dovecot:auth\))?: |(?:pop3|imap)-login: )?(?:Info: )?<F-CONTENT>.+</F-CONTENT>$
-            ^%(__prefix_line)s(?:pop3|imap)-login: (?:Info: )?(?:Aborted login|Disconnected)(?::(?: [^ \(]+)+)? \((?:auth failed, \d+ attempts( in \d+ secs)?|tried to use (disabled|disallowed) \S+ auth)\):( user=<[^>]+>,)?( method=\S+,)? rip=<HOST>(?:, lip=\S+)?(?:, TLS(?: handshaking(?:: SSL_accept\(\) failed: error:[\dA-F]+:SSL routines:[TLS\d]+_GET_CLIENT_HELLO:unknown protocol)?)?(: Disconnected)?)?(, session=<\S+>)?\s*$
+
-            ^%(__prefix_line)s(?:Info|dovecot: auth\(default\)|auth-worker\(\d+\)): pam\(\S+,<HOST>\): pam_authenticate\(\) failed: (User not known to the underlying authentication module: \d+ Time\(s\)|Authentication failure \(password mismatch\?\))\s*$
+failregex = ^authentication failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* rhost=<HOST>(?:\s+user=\S*)?\s*$
-            ^%(__prefix_line)s(?:auth|auth-worker\(\d+\)): (?:pam|passwd-file)\(\S+,<HOST>\): unknown user\s*$
+            ^(?:Aborted login|Disconnected)(?::(?: [^ \(]+)+)? \((?:auth failed, \d+ attempts(?: in \d+ secs)?|tried to use (?:disabled|disallowed) \S+ auth)\):(?: user=<[^>]*>,)?(?: method=\S+,)? rip=<HOST>(?:[^>]*(?:, session=<\S+>)?)\s*$
-            ^%(__prefix_line)s(?:auth|auth-worker\(\d+\)): Info: ldap\(\S*,<HOST>,\S*\): invalid credentials\s*$
+            ^pam\(\S+,<HOST>(?:,\S*)?\): pam_authenticate\(\) failed: (?:User not known to the underlying authentication module: \d+ Time\(s\)|Authentication failure \(password mismatch\?\)|Permission denied)\s*$
+            ^[a-z\-]{3,15}\(\S*,<HOST>(?:,\S*)?\): (?:unknown user|invalid credentials)\s*$
+            <mdre-<mode>>
+
+mdre-aggressive = ^(?:Aborted login|Disconnected)(?::(?: [^ \(]+)+)? \((?:no auth attempts|disconnected before auth was ready,|client didn't finish \S+ auth,)(?: (?:in|waited) \d+ secs)?\):(?: user=<[^>]*>,)?(?: method=\S+,)? rip=<HOST>(?:[^>]*(?:, session=<\S+>)?)\s*$
+
+mdre-normal = 
+
+# Parameter `mode` - `normal` or `aggressive`.
+# Aggressive mode can be used to match log-entries like:
+#   'no auth attempts', 'disconnected before auth was ready', 'client didn't finish SASL auth'.
+# Note it may produce lots of false positives on misconfigured MTAs.
+# Ex.:
+# filter = dovecot[mode=aggressive]
+mode = normal
 
 ignoreregex = 
 
-[Init]
-
 journalmatch = _SYSTEMD_UNIT=dovecot.service
 
+datepattern = {^LN-BEG}TAI64N
+              {^LN-BEG}
+
 # DEV Notes:
 # * the first regex is essentially a copy of pam-generic.conf
 # * Probably doesn't do dovecot sql/ldap backends properly (resolved in edit 21/03/2016)
-# * Removed the 'no auth attempts' log lines from the matches because produces
-#    lots of false positives on misconfigured MTAs making regexp unusable
 #
 # Author: Martin Waschbuesch
 #         Daniel Black (rewrote with begin and end anchors)

config/filter.d/dropbear.conf

@@ -23,9 +23,11 @@ before = common.conf
 
 _daemon = dropbear
 
-failregex = ^%(__prefix_line)s[Ll]ogin attempt for nonexistent user ('.*' )?from <HOST>:\d+$
+prefregex = ^%(__prefix_line)s<F-CONTENT>(?:[Ll]ogin|[Bb]ad|[Ee]xit).+</F-CONTENT>$
-            ^%(__prefix_line)s[Bb]ad (PAM )?password attempt for .+ from <HOST>(:\d+)?$
+
-            ^%(__prefix_line)s[Ee]xit before auth \(user '.+', \d+ fails\): Max auth tries reached - user '.+' from <HOST>:\d+\s*$
+failregex = ^[Ll]ogin attempt for nonexistent user ('.*' )?from <HOST>:\d+$
+            ^[Bb]ad (PAM )?password attempt for .+ from <HOST>(:\d+)?$
+            ^[Ee]xit before auth \(user '.+', \d+ fails\): Max auth tries reached - user '.+' from <HOST>:\d+\s*$
 
 ignoreregex = 
 

config/filter.d/ejabberd-auth.conf

@@ -16,8 +16,8 @@
 #          searched for other failures. This tag can be used multiple times.
 # Values:  TEXT
 #
-failregex = ^=INFO REPORT====  ===\nI\(<0\.\d+\.0>:ejabberd_c2s:\d+\) : \([^)]+\) Failed authentication for .+ from IP <HOST> \({{(?:\d+,){3}\d+},\d+}\)$
+failregex = ^=INFO REPORT====  ===\nI\(<0\.\d+\.0>:ejabberd_c2s:\d+\) : \([^)]+\) Failed authentication for \S+ from (?:IP )?<HOST>(?: \({{(?:\d+,){3}\d+},\d+}\))?$
-            ^(?:\.\d+)? \[info\] <0\.\d+\.\d>@ejabberd_c2s:wait_for_feature_request:\d+ \([^\)]+\) Failed authentication for \S+ from IP <HOST>$
+            ^(?:\.\d+)? \[info\] <0\.\d+\.\d>@ejabberd_c2s:\w+:\d+ \([^\)]+\) Failed (?:c2s \w+ )?authentication for \S+ from (?:IP )?(?:::FFFF:)?<HOST>(?:: |$)
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.
@@ -25,8 +25,6 @@ failregex = ^=INFO REPORT====  ===\nI\(<0\.\d+\.0>:ejabberd_c2s:\d+\) : \([^)]+\
 #
 ignoreregex = 
 
-[Init]
-
 # "maxlines" is number of log lines to buffer for multi-line regex searches
 maxlines = 2
 
@@ -35,3 +33,8 @@ maxlines = 2
 # Values:  TEXT
 #
 journalmatch = 
+
+#datepattern = ^(?:=[^=]+={3,} )?({DATE})
+# explicit time format using prefix =...==== and no date in second string begins with I(...)... 
+datepattern = ^(?:=[^=]+={3,} )?(%%ExY(?P<_sep>[-/.])%%m(?P=_sep)%%d[T ]%%H:%%M:%%S(?:[.,]%%f)?(?:\s*%%z)?)
+              ^I\(()**

config/filter.d/exim.conf

@@ -13,14 +13,32 @@ before = exim-common.conf
 
 [Definition]
 
+# Fre-filter via "prefregex" is currently inactive because of too different failure syntax in exim-log (testing needed):
+#prefregex = ^%(pid)s <F-CONTENT>\b(?:\w+ authenticator failed|([\w\-]+ )?SMTP (?:(?:call|connection) from|protocol(?: synchronization)? error)|no MAIL in|(?:%(host_info_pre)s\[[^\]]+\]%(host_info_suf)s(?:sender verify fail|rejected RCPT|dropped|AUTH command))).+</F-CONTENT>$
+
 failregex = ^%(pid)s %(host_info)ssender verify fail for <\S+>: (?:Unknown user|Unrouteable address|all relevant MX records point to non-existent hosts)\s*$
             ^%(pid)s \w+ authenticator failed for (?:[^\[\( ]* )?(?:\(\S*\) )?\[<HOST>\](?::\d+)?(?: I=\[\S+\](:\d+)?)?: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
             ^%(pid)s %(host_info)srejected RCPT [^@]+@\S+: (?:relay not permitted|Sender verify failed|Unknown user|Unrouteable address)\s*$
             ^%(pid)s SMTP protocol synchronization error \([^)]*\): rejected (?:connection from|"\S+") %(host_info)s(?:next )?input=".*"\s*$
             ^%(pid)s SMTP call from \S+ %(host_info)sdropped: too many nonmail commands \(last was "\S+"\)\s*$
-            ^%(pid)s SMTP protocol error in "AUTH \S*(?: \S*)?" %(host_info)sAUTH command used when not advertised\s*$
+            ^%(pid)s SMTP protocol error in "[^"]+(?:"+[^"]*(?="))*?" %(host_info)sAUTH command used when not advertised\s*$
-            ^%(pid)s no MAIL in SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sD=\d\S+s(?: C=\S*)?\s*$
+            ^%(pid)s no MAIL in SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sD=\d\S*s(?: C=\S*)?\s*$
             ^%(pid)s (?:[\w\-]+ )?SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sclosed by DROP in ACL\s*$
+            <mdre-<mode>>
+
+mdre-aggressive = ^%(pid)s no host name found for IP address <HOST>$
+                  ^%(pid)s no IP address found for host \S+ \(during SMTP connection from \[<HOST>\]\)$
+
+mdre-normal = 
+
+# Parameter `mode` - `normal` or `aggressive`.
+# Aggressive mode can be used to match flood and ddos-similar log-entries like:
+#   'no host found for IP', 'no IP found for host'.
+# Note this is not an authentication failures, so it may produce lots of false 
+# positives on misconfigured MTAs.
+# Ex.:
+#   filter = exim[mode=aggressive]
+mode = normal
 
 ignoreregex = 
 

config/filter.d/freeswitch.conf

@@ -8,13 +8,26 @@
 # IP addresses on your LAN.
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
 [Definition]
 
-failregex = ^\.\d+ \[WARNING\] sofia_reg\.c:\d+ SIP auth (failure|challenge) \((REGISTER|INVITE)\) on sofia profile \'[^']+\' for \[.*\] from ip <HOST>$
+_daemon = freeswitch
-            ^\.\d+ \[WARNING\] sofia_reg\.c:\d+ Can't find user \[\d+@\d+\.\d+\.\d+\.\d+\] from <HOST>$
+
+# Prefix contains common prefix line (server, daemon, etc.) and 2 datetimes if used systemd backend
+_pref_line = ^%(__prefix_line)s(?:\d+-\d+-\d+ \d+:\d+:\d+\.\d+)?
+
+failregex = %(_pref_line)s \[WARNING\] sofia_reg\.c:\d+ SIP auth (failure|challenge) \((REGISTER|INVITE)\) on sofia profile \'[^']+\' for \[[^\]]*\] from ip <HOST>$
+            %(_pref_line)s \[WARNING\] sofia_reg\.c:\d+ Can't find user \[[^@]+@[^\]]+\] from <HOST>$
 
 ignoreregex =
 
+datepattern = {^LN-BEG}
+
 # Author: Rupa SChomaker, soapee01, Daniel Black
 # https://freeswitch.org/confluence/display/FREESWITCH/Fail2Ban
 # Thanks to Jim on mailing list of samples and guidance

config/filter.d/froxlor-auth.conf

@@ -25,8 +25,11 @@ _daemon = Froxlor
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = ^%(__prefix_line)s\[Login Action <HOST>\] Unknown user \S* tried to login.$
+
-            ^%(__prefix_line)s\[Login Action <HOST>\] User \S* tried to login with wrong password.$
+prefregex = ^%(__prefix_line)s\[Login Action <HOST>\] <F-CONTENT>.+</F-CONTENT>$
+
+failregex = ^Unknown user \S* tried to login.$
+            ^User \S* tried to login with wrong password.$
 
 
 # Option:  ignoreregex

config/filter.d/guacamole.conf

@@ -17,6 +17,9 @@ failregex = ^.*\nWARNING: Authentication attempt from <HOST> for user "[^"]*" fa
 #
 ignoreregex = 
 
-[Init]
 # "maxlines" is number of log lines to buffer for multi-line regex searches
 maxlines = 2
+
+datepattern = ^%%b %%d, %%ExY %%I:%%M:%%S %%p
+              ^WARNING:()**
+              {^LN-BEG}

config/filter.d/haproxy-http-auth.conf

@@ -28,7 +28,7 @@ _daemon = haproxy
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = ^%(__prefix_line)s<HOST>.*<NOSRV> -1/-1/-1/-1/\+*\d* 401
+failregex = ^%(__prefix_line)s<HOST>(?::\d+)?\s+.*<NOSRV> -1/-1/-1/-1/\+*\d* 401
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/ignorecommands/apache-fakegooglebot

@@ -5,31 +5,34 @@
 # presence of host and cut commands
 #
 import sys
+from fail2ban.server.ipdns import DNSUtils, IPAddr
 
 def process_args(argv):
     if len(argv) != 2:
-       sys.stderr.write("Please provide a single IP as an argument. Got: %s\n"
+       raise ValueError("Please provide a single IP as an argument. Got: %s\n"
                         % (argv[1:]))
-       sys.exit(2)
-
     ip = argv[1]
 
-    from fail2ban.server.filter import DNSUtils
+    if not IPAddr(ip).isValid:
-    if not DNSUtils.isValidIP(ip):
+       raise ValueError("Argument must be a single valid IP. Got: %s\n"
-       sys.stderr.write("Argument must be a single valid IP. Got: %s\n"
                         % ip)
-       sys.exit(3)
     return ip
 
+google_ips = None
+
 def is_googlebot(ip):
     import re
-    from fail2ban.server.filter import DNSUtils
 
     host = DNSUtils.ipToName(ip)
     if not host or not re.match('.*\.google(bot)?\.com$', host):
-       sys.exit(1)
+       return False
     host_ips = DNSUtils.dnsToIp(host)
-    sys.exit(0 if ip in host_ips else 1)
+    return (ip in host_ips)
 
-if __name__ == '__main__':
+if __name__ == '__main__': # pragma: no cover
-    is_googlebot(process_args(sys.argv))
+    try:
+      ret = is_googlebot(process_args(sys.argv))
+    except ValueError as e:
+      sys.stderr.write(str(e))
+      sys.exit(2)
+    sys.exit(0 if ret else 1)

config/filter.d/kerio.conf

@@ -3,18 +3,22 @@
 [Definition]
 
 failregex = ^ SMTP Spam attack detected from <HOST>,
-            ^ IP address <HOST> found in DNS blacklist \S+, mail from \S+ to \S+$
+            ^ IP address <HOST> found in DNS blacklist
             ^ Relay attempt from IP address <HOST>
             ^ Attempt to deliver to unknown recipient \S+, from \S+, IP address <HOST>$
+            ^ Failed SMTP login from <HOST>
+            ^ SMTP: User \S+ doesn't exist. Attempt from IP address <HOST>
+            ^ Client with IP address <HOST> has no reverse DNS entry, connection rejected before SMTP greeting$
+            ^ Administration login into Web Administration from <HOST> failed: IP address not allowed$
+            ^ Message from IP address <HOST>, sender \S+ rejected: sender domain does not exist$
 
 ignoreregex =
 
-[Init]
-
 datepattern = ^\[%%d/%%b/%%Y %%H:%%M:%%S\]
 
 # DEV NOTES:
 # 
 # Author: A.P. Lawrence
+# Updated by: M. Bischoff <https://github.com/herrbischoff>
 #
 # Based off: http://aplawrence.com/Kerio/fail2ban.html

config/filter.d/lighttpd-auth.conf

@@ -3,7 +3,7 @@
 
 [Definition]
 
-failregex = ^: \(http_auth\.c\.\d+\) (password doesn\'t match .* username: .*|digest: auth failed for .*: wrong password|get_password failed), IP: <HOST>\s*$
+failregex = ^: \((?:http|mod)_auth\.c\.\d+\) (?:password doesn\'t match .* username: .*|digest: auth failed for .*: wrong password|get_password failed), IP: <HOST>\s*$
 
 ignoreregex = 
 

config/filter.d/monit.conf

@@ -13,7 +13,7 @@ before = common.conf
 _daemon = monit
 
 # Regexp for previous (accessing monit httpd) and new (access denied) versions
-failregex = ^\[[A-Z]+\s+\]\s*error\s*:\s*Warning:\s+Client '<HOST>' supplied (?:unknown user '[^']+'|wrong password for user '[^']*') accessing monit httpd$
+failregex = ^\[\s*\]\s*error\s*:\s*Warning:\s+Client '<HOST>' supplied (?:unknown user '[^']+'|wrong password for user '[^']*') accessing monit httpd$
             ^%(__prefix_line)s\w+: access denied -- client <HOST>: (?:unknown user '[^']+'|wrong password for user '[^']*'|empty password)$
 
 # Ignore login with empty user (first connect, no user specified)

config/filter.d/murmur.conf

@@ -15,13 +15,16 @@ _daemon = murmurd
 #      variable in your server config file (murmur.ini / mumble-server.ini).
 _usernameregex = [^>]+
 
-_prefix = <W>[\n\s]*(\.\d{3})?\s+\d+ => <\d+:%(_usernameregex)s\(-1\)> Rejected connection from <HOST>:\d+:
+_prefix = \s+\d+ => <\d+:%(_usernameregex)s\(-1\)> Rejected connection from <HOST>:\d+:
 
-failregex = ^%(_prefix)s Invalid server password$
+prefregex = ^%(_prefix)s <F-CONTENT>.+</F-CONTENT>$
-            ^%(_prefix)s Wrong certificate or password for existing user$
+
+failregex = ^Invalid server password$
+            ^Wrong certificate or password for existing user$
 
 ignoreregex =
 
+datepattern = ^<W>{DATE}
 
 # DEV Notes:
 #

config/filter.d/named-refused.conf

@@ -34,9 +34,11 @@ __daemon_combs_re=(?:%(__pid_re)s?:\s+%(__daemon_re)s|%(__daemon_re)s%(__pid_re)
 # this can be optional (for instance if we match named native log files)
 __line_prefix=(?:\s\S+ %(__daemon_combs_re)s\s+)?
 
-failregex = ^%(__line_prefix)s( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: (view (internal|external): )?query(?: \(cache\))? '.*' denied\s*$
+prefregex = ^%(__line_prefix)s( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: <F-CONTENT>.+</F-CONTENT>$
-            ^%(__line_prefix)s( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: zone transfer '\S+/AXFR/\w+' denied\s*$
+
-            ^%(__line_prefix)s( error:)?\s*client <HOST>#\S+( \([\S.]+\))?: bad zone transfer request: '\S+/IN': non-authoritative zone \(NOTAUTH\)\s*$
+failregex = ^(view (internal|external): )?query(?: \(cache\))? '.*' denied\s*$
+            ^zone transfer '\S+/AXFR/\w+' denied\s*$
+            ^bad zone transfer request: '\S+/IN': non-authoritative zone \(NOTAUTH\)\s*$
 
 ignoreregex =
 

config/filter.d/nginx-botsearch.conf

@@ -13,6 +13,9 @@ failregex = ^<HOST> \- \S+ \[\] \"(GET|POST|HEAD) \/<block> \S+\" 404 .+$
 
 ignoreregex = 
 
+datepattern = {^LN-BEG}%%ExY(?P<_sep>[-/.])%%m(?P=_sep)%%d[T ]%%H:%%M:%%S(?:[.,]%%f)?(?:\s*%%z)?
+              ^[^\[]*\[({DATE})
+              {^LN-BEG}
 
 # DEV Notes:
 # Based on apache-botsearch filter

config/filter.d/nginx-http-auth.conf

@@ -4,10 +4,12 @@
 [Definition]
 
 
-failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (password mismatch|was not found in ".*"), client: <HOST>, server: \S*, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"(, referrer: "\S+")?\s*$
+failregex = ^ \[error\] \d+#\d+: \*\d+ user "(?:[^"]+|.*?)":? (?:password mismatch|was not found in "[^\"]*"), client: <HOST>, server: \S*, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"(?:, referrer: "\S+")?\s*$
 
 ignoreregex = 
 
+datepattern = {^LN-BEG}
+
 # DEV NOTES:
 # Based on samples in https://github.com/fail2ban/fail2ban/pull/43/files
 # Extensive search of all nginx auth failures not done yet.

config/filter.d/nginx-limit-req.conf

@@ -36,10 +36,11 @@ ngx_limit_req_zones = [^"]+
 # Use following full expression if you should range limit request to specified 
 # servers, requests, referrers etc. only :
 #
-# failregex = ^\s*\[error\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:%(ngx_limit_req_zones)s)", client: <HOST>, server: \S*, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"(, referrer: "\S+")?\s*$
+# failregex = ^\s*\[[a-z]+\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:%(ngx_limit_req_zones)s)", client: <HOST>, server: \S*, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"(, referrer: "\S+")?\s*$
 
 # Shortly, much faster and stable version of regexp:
-failregex = ^\s*\[error\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:%(ngx_limit_req_zones)s)", client: <HOST>
+failregex = ^\s*\[[a-z]+\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:%(ngx_limit_req_zones)s)", client: <HOST>,
 
 ignoreregex = 
 
+datepattern = {^LN-BEG}

config/filter.d/nsd.conf

@@ -26,3 +26,6 @@ failregex =  ^%(__prefix_line)sinfo: ratelimit block .* query <HOST> TYPE255$
              ^%(__prefix_line)sinfo: .* <HOST> refused, no acl matches\.$
 
 ignoreregex =
+
+datepattern = {^LN-BEG}Epoch
+              {^LN-BEG}

config/filter.d/openhab.conf

@@ -9,7 +9,6 @@
 [Definition] 
 failregex = ^<HOST>\s+-\s+-\s+\[\]\s+"[A-Z]+ .*" 401 \d+\s*$
 
-[Init]
 datepattern = %%d/%%b[^/]*/%%Y:%%H:%%M:%%S %%z
 
 

config/filter.d/oracleims.conf

@@ -52,10 +52,12 @@ before = common.conf
 # Note that you MUST have LOG_FORMAT=4 for this to work!
 #
 
-failregex = ^.*tr="[A-Z]+\|[0-9.]+\|\d+\|<HOST>\|\d+" ap="[^"]*" mi="Bad password" us="[^"]*" di="535 5.7.8 Bad username or password( \(Authentication failed\))?\."/>$
+failregex = tr="[A-Z]+\|[0-9.]+\|\d+\|<HOST>\|\d+" ap="[^"]*" mi="Bad password" us="[^"]*" di="535 5.7.8 Bad username or password( \(Authentication failed\))?\."/>$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.
 # Values:  TEXT
 #
 ignoreregex =
+
+datepattern = ^<co ts="{DATE}"\s+

config/filter.d/pam-generic.conf

@@ -16,7 +16,12 @@ _ttys_re=\S*
 __pam_re=\(?%(__pam_auth)s(?:\(\S+\))?\)?:?
 _daemon = \S+
 
-failregex = ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=%(_ttys_re)s ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$
+prefregex = ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=%(_ttys_re)s <F-CONTENT>.+</F-CONTENT>$
+
+failregex = ^ruser=<F-USER>\S*</F-USER> rhost=<HOST>\s*$
+            ^ruser= rhost=<HOST>\s+user=<F-USER>\S*</F-USER>\s*$
+            ^ruser= rhost=<HOST>\s+user=<F-USER>.*?</F-USER>\s*$
+            ^ruser=<F-USER>.*?</F-USER> rhost=<HOST>\s*$
 
 ignoreregex = 
 

config/filter.d/php-url-fopen.conf

@@ -18,3 +18,6 @@ ignoreregex =
 # http://blogs.buanzo.com.ar/2009/04/fail2ban-filter-for-php-injection-attacks.html#comment-1489
 #
 # Author: Arturo 'Buanzo' Busleiman <buanzo@buanzo.com.ar>
+
+datepattern = ^[^\[]*\[({DATE})
+              {^LN-BEG}

config/filter.d/phpmyadmin-syslog.conf

@@ -0,0 +1,18 @@
+# Fail2Ban fitler for the phpMyAdmin-syslog
+#
+
+[INCLUDES]
+
+before = common.conf
+
+[Definition]
+
+_daemon = phpMyAdmin
+
+failregex = ^%(__prefix_line)suser denied: (?:\S+|.*?) \(mysql-denied\) from <HOST>\s*$
+
+ignoreregex =
+
+
+# Author: Pavel Mihadyuk
+# Regex fixes: Serg G. Brester

config/filter.d/portsentry.conf

@@ -8,5 +8,8 @@ failregex = \/<HOST> Port\: [0-9]+ (TCP|UDP) Blocked$
 
 ignoreregex =
 
+datepattern = {^LN-BEG}Epoch
+              {^LN-BEG}
+
 # Author: Pacop <pacoparu@gmail.com>
 

config/filter.d/postfix-rbl.conf

@@ -1,19 +0,0 @@
-# Fail2Ban filter for Postfix's RBL based Blocked hosts
-#
-#
-
-[INCLUDES]
-
-# Read common prefixes. If any customizations available -- read them from
-# common.local
-before = common.conf
-
-[Definition]
-
-_daemon = postfix(-\w+)?/smtpd
-
-failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 454 4\.7\.1 Service unavailable; Client host \[\S+\] blocked using .* from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$
-
-ignoreregex =
-
-# Author: Lee Clemens