make up leeway of ChangeLog (prepare release of 1.0)

ChangeLog

@@ -15,6 +15,10 @@ ver. 1.0.1-dev-1 (20??/??/??) - development nightly edition
   you can use the 0.11 version of fail2ban or upgrade python (or even build it from source).
 * potential incompatibility by parsing of options of `backend`, `filter` and `action` parameters (if they
   are partially incorrect), because fail2ban could throw an error now (doesn't silently bypass it anymore).
+* due to fix for CVE-2021-32749 (GHSA-m985-3f3v-cwmm) the mailing action using mailutils may require extra configuration,
+  if it is not compatible or doesn't support `-E 'set escape'` (e. g. with `mailcmd` parameter), see gh-3059
+* automatic invocation of 2to3 is removed in setup now (gh-3098), there is also no option `--disable-2to3` anymore,
+  `./fail2ban-2to3` should be called outside before setup
 * to v.0.11:
   - due to change of `actioncheck` behavior (gh-488), some actions can be incompatible as regards
     the invariant check, if `actionban` or `actionunban` would not throw an error (exit code 
@@ -24,29 +28,144 @@ ver. 1.0.1-dev-1 (20??/??/??) - development nightly edition
     and the values are different (gh-3217)
 
 ### Fixes
+* theoretical RCE vulnerability in mailing action using mailutils (mail-whois), CVE-2021-32749, GHSA-m985-3f3v-cwmm
 * readline fixed to consider interim new-line character as part of code point in multi-byte logs
   (e. g. unicode encoding like utf-16be, utf-16le);
+* [stability] solves race condition with uncontrolled growth of failure list (jail with too many matches,
+  that did not cause ban), behavior changed to ban ASAP, gh-2945
+* fixes search for the best datepattern - e. g. if line is too short, boundaries check for previously known
+  unprecise pattern may fail on incomplete lines (logging break-off, no flush, etc), gh-3020
+* [stability, performance] backend `systemd`:
+  - fixes error "local variable 'line' referenced before assignment", introduced in 55d7d9e2, gh-3097
+  - don't update database too often (every 10 ticks or ~ 10 seconds in production)
+  - fixes wrong time point of "in operation" mode, gh-2882
+  - better avoidance of landing in dead space by seeks over journals (improved seek to time)
+  - fixes missing space in message (tag `<matches>`) between timestamp and host if the message read from systemd journal, gh-3293
+* [stability] backend `pyinotify`: fixes sporadic runtime error "dictionary changed size during iteration"
+* several backends optimizations (in file and journal filters):
+  - don't need to wait if we still had log-entries from last iteration (which got interrupted for servicing)
+  - rewritten update log/journal position, it is more stable and faster now (fewer DB access and surely up-to-date at end)
+* `paths-debian.conf`:
+  - add debian path to roundcube error logs
+* `action.d/firewallcmd-*.conf` (multiport only): fixed port range selector, replacing `:` with `-`;"
+  reverted the incompatibility gh-3047 introduced in a038fd5, gh-2821, because this depends now on firewalld backend
+  (e. g. `-` vs. `:` related to `iptables` vs. `nftables`)
+* `action.d/nginx-block-map.conf`: reload nginx only if it is running (also avoid error in nginx-errorlog, gh-2949)
 * `action.d/ufw.conf`:
   - fixed handling on IPv6 (using prepend, gh-2331, gh-3018)
   - application names containing spaces can be used now (gh-656, gh-1532, gh-3018)
-* `filter.d/drupal-auth.conf` more strict regex, extended to match "Login attempt failed from" (gh-2742)
+* `filter.d/apache-fakegooglebot.conf`:
+  - better, more precise regex and datepattern (closes possible weakness like gh-3013)
+  - `filter.d/ignorecommands/apache-fakegooglebot` - added timeout parameter (default 55 seconds), avoid fail with timeout
+    (default 1 minute) by reverse lookup on some slow DNS services (googlebots must be resolved fast), gh-2951
+* `filter.d/apache-overflows.conf` - extended to match AH00126 error (Invalid URI ...), gh-2908
+* `filter.d/asterisk.conf` - add transport to asterisk RE: call rejection messages can have the transport prefixed to the IP address, gh-2913
+* `filter.d/courier-auth.conf`:
+  - consider optional port after IP, gh-3211
+  - regex is rewritten without catch-all's and right anchor, so it is more stable against further modifications now
+* `filter.d/dovecot.conf`:
+  - adjusted for updated dovecot log format with `read(size=...)` in message (gh-3210)
+  - parse everything in parenthesis by auth-worker info, e. g. can match (pid=...,uid=...) too (amend to gh-2553)
+  - extended to match prefix like `conn unix:auth-worker (uid=143): auth-worker<13247>:`
+    (authenticate from external service like exim), gh-2553
+  - fixed "Authentication failure" regex, matches "Password mismatch" in title case (gh-2880)
+* `filter.d/drupal-auth.conf` - more strict regex, extended to match "Login attempt failed from" (gh-2742)
+* `filter.d/exim-common.conf` - pid-prefix extended to match `mx1 exim[...]:` (gh-2553)
+* `filter.d/lighttpd-auth.conf` - adjusted to the current source code + avoiding catch-all's, etc (gh-3116)
+* `filter.d/named-refused.conf`:
+  - added support for alternate names (suffix), FreeIPA renames the BIND9 named daemon to named-pkcs11, gh-2636
+  - fixes prefix for messages from systemd journal (no mandatory space ahead, because don't have timestamp), gh-2899
+* `filter.d/nginx-*.conf` - added journalmatch to nginx filters, gh-2935
+* `filter.d/nsd.conf` - support for current log format, gh-2965
+* `filter.d/postfix.conf`: fixes and new vectors, review and combining several regex to single RE:
+    - mode `ddos` (and `aggressive`) extended:
+      * to consider abusive handling of clients hitting command limit, gh-3040
+      * to handle postscreen's PREGREET and HANGUP messages, gh-2898
+    - matches rejects with "undeliverable address" (sender/recipient verification) additionally to "Unknown user", gh-3039
+      both are configurable now via extended parameter and can be disabled using `exre-user=` supplied in filter parameters
+    - reject: BDAT/DATA from, gh-2927
+    - (since regex is more precise now) token selector changed to `[A-Z]{4}`, e. g. no matter what a command is supplied now 
+      (RCPT, EHLO, VRFY, DATA, BDAT or something else)
+    - matches "Command rejected" and "Data command rejected" now
+    - matches RCPT from unknown, 504 5.5.2, need fully-qualified hostname, gh-2995
+    - matches 550 5.7.25 Client host rejected, gh-2996
+* `filter.d/sendmail-auth.conf`:
+  - detect several "authentication failure" messages, sendmail 8.16.1, gh-2757
+  - detect user not found, gh-3030
+  - detect failures without user part, gh-3324
+* `filter.d/sendmail-reject.conf`:
+  - fix reverse DNS for ... (gh-3012)
+  - fixed regex to consider "Connection rate limit exceeded" with different combination of arguments
+* `filter.d/sshd.conf`:
+  - mode `ddos` extended - recognizes messages "kex_exchange_identification: Connection closed / reset by pear", gh-3086
+    (fixed possible regression of f77398c)
+  - mode `ddos` extended - recognizes new message "banner exchange: invalid format" generated by port scanner
+    (https payload on ssh port), gh-3169
+* `filter.d/zoneminder.conf` - support new log format (ERR instead of WAR), add detection of non-existent user login attempts, gh-2984
+* amend to gh-980 fixing several actions (correctly supporting new enhancements now)
+* fixed typo by `--dump-pretty` option which did never work (only `--dp` was working)
+* fixes start of fail2ban-client in docker: speedup daemonization process by huge open files limit, gh-3334
+* provides details of failed regex compilation in the error message we throw in Regex-constructor 
+  (it's good to know what exactly is wrong)
+* fixed failed update of database didn't signal with an error, gh-3352:
+  - client and server exit with error code by failure during start process (in foreground mode)
+  - added fallback to repair if database cannot be upgraded
 
 ### New Features and Enhancements
+* python 3.10 and 3.11 compatibility (and GHA-CI support)
 * `actioncheck` behavior is changed now (gh-488), so invariant check as well as restore or repair
    of sane environment (in case of recognized unsane state) would only occur on action errors (e. g.
    if ban or unban operations are exiting with other code as 0)
 * better recognition of log rotation, better performance by reopen: avoid unnecessary seek to begin of file
   (and hash calculation)
 * file filter reads only complete lines (ended with new-line) now, so waits for end of line (for its completion)
+* datedetector:
+  - token `%Z` must recognize zone abbreviation `Z` (GMT/UTC) also (similar to `%z`)
+  - token `%Z` recognizes all known zone abbreviation besides Z, GMT, UTC correctly, if it is matching
+    (`%z` remains unchanged for backwards-compatibility, see comment in code)
+  - date patterns `%ExY` and `%Exy` accept every year from 19xx up to current century (+3 years) in `fail2ban-regex`
+  - better grouping algorithm for resulting century RE for `%ExY` and `%Exy`
 * actions differentiate tags `<ip>` and `<fid>` (`<F-ID>`), if IP-address deviates from ID then the value
   of `<ip>` is not equal `<fid>` anymore (gh-3217)
+* action info extended with new members for jail info (usable as tags in command actions), gh-10:
+  - `<jail.found>`, `<jail.found_total>` - current and total found failures
+  - `<jail.banned>`, `<jail.banned_total>` - current and total bans
+* `filter.d/monitorix.conf` - added new filter and jail for Monitorix, gh-2679
+* `filter.d/mssql-auth.conf` - new filter and jail for Microsoft SQL Server, gh-2642
+* `filter.d/nginx-bad-request.conf` - added filter to find bad requests (400), gh-2750
+* `filter.d/nginx-http-auth.conf` - extended with parameter mode, so additionally to `auth` (or `normal`) 
+   mode `fallback` (or combined as `aggressive`) can find SSL errors while SSL handshaking, gh-2881
+* `filter.d/scanlogd.conf` - new filter and jail, add support for filtering out detected port scans via scanlogd, gh-2950
+* `action.d/apprise.conf` - added Apprise support (50+ Notifications), gh-2565
+* `action.d/badips.*` - removed actions, badips.com is no longer active, gh-2889
+* `action.d/cloudflare.conf` - better IPv6 capability, gh-2891
+* `action.d/cloudflare-token.conf` - added support for Cloudflare Token APIs.  This method is more restrictive and therefore safter than using API Keys.
+* `action.d/ipthreat.conf` - new action for IPThreat integration, gh-3349
 * `action.d/ufw.conf` (gh-3018):
   - new option `add` (default `prepend`), can be supplied as `insert 1` for ufw versions before v.0.36 (gh-2331, gh-3018)
   - new options `kill-mode` and `kill` to drop established connections of intruder (see action for details, gh-3018)
-* `filter.d/nginx-http-auth.conf` - extended with parameter mode, so additionally to `auth` (or `normal`) 
-   mode `fallback` (or combined as `aggressive`) can find SSL errors while SSL handshaking, gh-2881
-* `action.d/cloudflare-token.conf` - added support for Cloudflare Token APIs.  This method is more restrictive and therefore safter than using API Keys.
-* new logtarget SYSTEMD-JOURNAL
+* `iptables` and `iptables-ipset` actions extended to support multiple protocols with single action 
+  for multiport or oneport type (back-ported from nftables action);
+* `iptables` actions are more breakdown-safe: start wouldn't fail if chain or rule already exists
+  (e. g. created by previous instance and doesn't get purged properly); ultimately closes gh-980
+* `ipset` actions are more breakdown-safe: start wouldn't fail if set with this name already exists
+  (e. g. created by previous instance and don't deleted properly)
+* replace internals of several `iptables` and `iptables-ipset` actions using internals of iptables include:
+  - better check mechanism (using `-C`, option `--check` is available long time);
+  - additionally iptables-ipset is a common action for `iptables-ipset-proto6-*` now (which become obsolete now);
+  - many features of different iptables actions are combinable as single chain/rule (can be supplied to action as parameters);
+  - iptables is a replacement for iptables-common now, several actions using this as include now become obsolete;
+* new logtarget SYSTEMD-JOURNAL, gh-1403
+* fail2ban.conf: new fail2ban configuration option `allowipv6` (default `auto`), can be used to allow or disallow IPv6 
+  interface in fail2ban immediately by start (e. g. if fail2ban starts before network interfaces), gh-2804
+* invalidate IP/DNS caches by reload, so inter alia would allow to recognize IPv6IsAllowed immediately, previously
+  retarded up to cache max-time (5m), gh-2804
+* OpenRC (Gentoo, mainly) service script improvements, gh-2182
+* suppress unneeded info "Jail is not a JournalFilter instance" (moved to debug level), gh-3186
+* implements new interpolation variable `%(fail2ban_confpath)s` (automatically substituted from config-reader path, 
+  default `/etc/fail2ban` or `/usr/local/etc/fail2ban` depending on distribution); `ignorecommands_dir` is unneeded anymore,
+  thus removed from `paths-common.conf`, fixes gh-3005
+* `fail2ban-regex`: accepts filter parameters containing new-line
 
 
 ver. 0.11.2 (2020/11/23) - heal-the-world-with-security-tools