- avoid duplicates in "matches" (previously always added matches of pending failures to every next real failure, or nofail-helper recognized IP, now first failure only);
- several optimizations of merge mechanism (multi-line parsing);
fail2ban-regex: better output handling, extended with tag substitution (ex.: `-o 'fail <ip>, user <F-USER>: <msg>'`); consider a string containing new-line as multi-line log-excerpt (not as a single log-line)
filter.d/sshd.conf: introduced parameter `publickey` (allowing change behavior of "Failed publickey" failures):
- `nofail` (default) - consider failed publickey (legitimate users) as no failure (helper to get IP and user-name only)
- `invalid` - consider failed publickey for invalid users only;
- `any` - consider failed publickey for valid users too;
- `ignore` - ignore "Failed publickey ..." failures (don't consider failed publickey at all)
tests/samplestestcase.py: SampleRegexsFactory gets new failJSON option `constraint` to allow ignore of some tests depending on filter name, options and test parameters
avoid overwrite of data with empty tags by ticket constructed from multi-line failures;
amend to d1b7e2b5fb2b389d04845369d7d29db65425dcf2: better output (as well as ignoring of pending lines) using `--out msg`;
filter.d/sshd.conf: don't forget mlf-cache on "disconnecting: too many authentication failures" - message does not have IP (must be followed by "closed [preauth]" to obtain host-IP).
- common.conf: rewritten using section-based handling round about option logtype;
- option `logtype` extended with `rfc5424` to cover RFC 5424 log-format (see #2309);
filter.d/sshd.conf: extended with new rules:
- Disconnecting ...: Change of username or service not allowed
- Disconnected from ... [preauth] (extra/aggressive mode only)
at least using both modes can ban port-scanners and prevent for other annoying "intruders", closing connection within preauth-stage (see gh-2085 for example).
intruder (as legitimate user) firstly tries to login with another user-name (brute-force), so hopes to reset failure counter by succeeded login;
this is fixed and covered in tests now;
sshd-filter extended to cover multiple-login attempts (also fully implements gh-2070);
differentiate between "invalid user" (going banned earlier) and valid users with public keys, for which the rejects of not valid public keys (failures) will be retarded up to "Too many authentication failures" resp. disconnect without success (accepted public key).
- optional parameter `mode` rewritten: normal (default), ddos, extra or aggressive (combines all), see sshd for regex details);
test cases reformatted (since "filterOptions", we don't need multiple test log-files anymore);
Many times faster and fewer CPU-hungry because of parsing with `maxlines=1`, so without line buffering (scrolling of the buffer-window).
Combination of tags `<F-MLFID>` and `<F-NOFAIL>` can be used now to process multi-line logs using single-line expressions:
- tag `<F-MLFID>`: used to identify resp. store failure info for groups of log-lines with the same identifier (e. g. combined failure-info for the same conn-id by `<F-MLFID>(?:conn-id)</F-MLFID>`, see sshd.conf for example)
- tag `<F-NOFAIL>`: used as mark for no-failure (helper to accumulate common failure-info);
filter.d/sshd.conf: [sshd], [sshd-ddos], [sshd-aggressive] optimized with pre-filtering using new option `prefregex` and new multi-line handling.
Some filters extended with user name;
[filter.d/pam-generic.conf]: grave fix injection on user name to host fixed;
test-cases in testSampleRegexsFactory can now check the captured groups (using additionally fields in failJSON structure)
new ssh rule(s) added:
- Connection reset by peer (multi-line rule during authorization process);
- No supported authentication methods available;
Single line and multi-line expression optimized, added optional prefixes and suffix (logged from several ssh versions);
closes gh-864
Sergey G. Brester