Commit Graph

5475 Commits (2367ad115c1d20daaca886ebe4db81edf06df577)

Author SHA1 Message Date
sebres e3737bb7c0 filter stability fix: prevent race condition - no ban if filter (backend) is continuously busy if too many messages will be found in log, e. g. initial scan of large log-file or journal (gh-2660) 2020-03-13 17:20:19 +01:00
Sergey G. Brester 428c75d1cd
Merge pull request #2651 from fail2ban/0.10-travis-3.9-dev
python 3.9 compatibility + CI
2020-03-06 20:46:02 +01:00
Sergey G. Brester d4da9afd7f
Update ChangeLog 2020-03-06 20:29:48 +01:00
Sergey G. Brester 9d7388e684
Thread: is_alive instead of isAlive (removed in py-3.9) 2020-03-06 20:04:18 +01:00
Sergey G. Brester 55e76c0b80
restore isAlive method removed in python 3.9 2020-03-06 19:41:16 +01:00
Sergey G. Brester 781a25512b
travis CI: add 3.9-dev as target 2020-03-06 19:04:39 +01:00
sebres 8b43d54878 Merge branch '0.11' 2020-03-05 14:32:42 +01:00
sebres 32f02ef3b3 Merge branch '0.10' into 0.11 2020-03-05 14:01:14 +01:00
sebres 42714d0849 filter.d/common.conf: closes gh-2650, avoid substitute of default values in related `lt_*` section, `__prefix_line` should be interpolated in definition section (after the config considers all sections that can overwrite it);
amend to 62b1712d22 (PR #2387, backend-related option `logtype`);
testSampleRegexsZZZ-GENERIC-EXAMPLE covering now negative case also (other daemon in prefix line)
2020-03-05 13:47:11 +01:00
sebres 2ddf687c31 Merge branch '0.10' into 0.11 - test cases only (add ban to database was moved to observer in 0.11) 2020-03-02 19:17:16 +01:00
sebres 15158e4474 closes gh-2647: add ban to database is moved from jail.putFailTicket to actions.__CheckBan; be sure manual ban is written to database, so can be restored by restart; reload/restart test extended 2020-03-02 18:58:59 +01:00
sebres f088e7bf76 Merge branch '0.10' into 0.11 2020-03-02 17:10:48 +01:00
sebres 6281dc3633 failmanager, ticket: avoid reset of retry count by pause between attempts near to findTime - adjust time of ticket will now change current attempts considering findTime as an estimation from rate by previous known interval (if it exceeds the findTime);
this should avoid some false positives as well as provide more safe handling around `maxretry/findtime` relation especially on busy circumstances.
2020-03-02 17:05:00 +01:00
sebres 4766547e1f performance optimization of `datepattern` (better search algorithm);
datetemplate: improved anchor detection for capturing groups `(^...)`; introduced new prefix `{UNB}` for `datepattern` to disable word boundaries in regex;
datedetector: speedup special case if only one template is defined (every match wins - no collision, no sorting, no other best match possible)
2020-02-28 14:27:21 +01:00
sebres ef1eaf9b37 Merge branch '0.11' 2020-02-25 17:18:50 +01:00
sebres c15c300d2a Merge branch '0.10' into 0.11 2020-02-25 17:11:29 +01:00
sebres e6ca04ca9d Merge branch '0.10' into 0.11 + version bump (back to dev) 2020-02-25 16:10:31 +01:00
Sergey G. Brester 2e42b98cd3
Merge pull request #2638 from gurnec/pypy-ulimit-fix
close Popen() pipes explicitly for PyPy
2020-02-25 15:31:31 +01:00
sebres 6c6cf2a956 small amend (avoid possible error by close of not existing pipe) 2020-02-25 15:06:04 +01:00
Christopher Gurnee df885586d4 close Popen() pipes explicitly for PyPy
Waiting for garbage collection to close pipes opened by Popen() can
lead to "Too many open files" errors with PyPy; close them explicitly.
2020-02-25 14:55:10 +01:00
sebres e57e950ef5 version bump (back to dev) 2020-02-25 14:51:54 +01:00
sebres ab3a7fc6d2 filter.d/sshd.conf: mode `ddos` (and aggressive) extended to detect port scanner sending unexpected ident string after connect 2020-02-17 16:24:42 +01:00
sebres 35591db3e8 Merge branch '0.11' 2020-02-14 12:14:51 +01:00
sebres 7282cf91b0 Merge branch '0.10' into 0.11 2020-02-14 12:13:29 +01:00
sebres b3644ad413 code normalization and optimization (strip of trailing new-line, date parsing, ignoreregex mechanism, etc) 2020-02-13 21:52:54 +01:00
sebres 91eca4fdeb automatically create not-existing path (last level folder only) for pidfile, socket and database (with default permissions) 2020-02-13 13:50:17 +01:00
sebres 14e68eed72 performance: set fetch handler getGroups depending on presence of alternate tags in RE (simplest variant or merged with alt-tags) in regex constructor 2020-02-13 12:31:15 +01:00
sebres 9137c7bb23 filter processing:
- avoid duplicates in "matches" (previously always added matches of pending failures to every next real failure, or nofail-helper recognized IP, now first failure only);
  - several optimizations of merge mechanism (multi-line parsing);
fail2ban-regex: better output handling, extended with tag substitution (ex.: `-o 'fail <ip>, user <F-USER>: <msg>'`); consider a string containing new-line as multi-line log-excerpt (not as a single log-line)
filter.d/sshd.conf: introduced parameter `publickey` (allowing change behavior of "Failed publickey" failures):
  - `nofail` (default) - consider failed publickey (legitimate users) as no failure (helper to get IP and user-name only)
  - `invalid` - consider failed publickey for invalid users only;
  - `any` - consider failed publickey for valid users too;
  - `ignore` - ignore "Failed publickey ..." failures (don't consider failed publickey at all)
tests/samplestestcase.py: SampleRegexsFactory gets new failJSON option `constraint` to allow ignore of some tests depending on filter name, options and test parameters
2020-02-13 12:28:07 +01:00
sebres 1492ab2247 improve processing of pending failures (lines without ID/IP) - fail2ban-regex would show those in matched lines now (as well as increase count of matched RE);
avoid overwrite of data with empty tags by ticket constructed from multi-line failures;
amend to d1b7e2b5fb2b389d04845369d7d29db65425dcf2: better output (as well as ignoring of pending lines) using `--out msg`;
filter.d/sshd.conf: don't forget mlf-cache on "disconnecting: too many authentication failures" - message does not have IP (must be followed by "closed [preauth]" to obtain host-IP).
2020-02-11 18:44:36 +01:00
Sergey G. Brester ac8e8db814
travis: switch 3.8-dev to 3.8 (released) 2020-02-11 14:18:58 +01:00
Sergey G. Brester d7643fe538
Merge pull request #2630 from fail2ban/gh-2200-postfix
filter.d/postfix.conf: extended mode ddos and aggressive covering multiple disconnects without auth
2020-02-11 12:44:21 +01:00
Sergey G. Brester 88cf5bcd93
Update postfix 2020-02-10 13:41:28 +01:00
Sergey G. Brester 774dda6105
filter.d/postfix.conf: extended mode ddos and aggressive covering multiple disconnects without auth 2020-02-10 13:29:16 +01:00
Sergey G. Brester 34d63fccfe
close gh-2629 - jail.conf (action_blocklist_de interpolation): replace service parameter (use jail name instead of filter, which can be empty) 2020-02-10 13:03:55 +01:00
sebres 65ce64ba6d Merge branch '0.11' 2020-02-07 13:57:00 +01:00
sebres 01333fc3e2 Merge branch '0.10' into 0.11 2020-02-07 13:55:20 +01:00
sebres 7a28861fc7 review of command line: more long-named options can be supplied via command line 2020-02-07 13:52:45 +01:00
sebres 3f48907064 amend to f3dbc9dda10e52610e3de26f538b5581fd905505: change main thread-name back to `fail2ban-server`;
implements new command line option `--pname` to specify it by start of server (default `fail2ban-server`);
closes gh-2623 (revert change of main thread-name, because it can affect process-name too, so `pgrep` & co. may be confused)
2020-02-07 11:08:01 +01:00
sebres 9c7bd80807 fail2ban-regex: stop endless logging on closed streams (redirected pipes like `... | head -n 100`), exit if stdout channel is closed 2020-02-03 20:09:13 +01:00
sebres ccbc0c7405 Merge branch '0.11' 2020-01-30 10:23:14 +01:00
sebres a7c68ea19f Merge branch '0.10' into 0.11 2020-01-28 21:47:55 +01:00
sebres 12b3ac684a closes #2615: systemd backend would seek to last known position (or `now - findtime`) in journal at start. 2020-01-28 21:45:30 +01:00
sebres cd42cb26d6 database: try to fix `out of sequence` error on some old platform / sqlite versions (#2613) - repack iterator as long as in lock (although dirty read has no matter here and only writing operations should be serialized, but to be sure and exclude this as source of that errors). 2020-01-27 12:57:29 +01:00
sebres 569dea2b19 filter.d/mysqld-auth.conf: capture user name in filter (can be more strict if user switched, used in action or fail2ban-regex output);
also add coverage for mariadb 10.4 log format (gh-2611)
2020-01-22 17:24:40 +01:00
sebres 9e6d07d928 testSampleRegexsFactory: `time` is not mandatory anymore (check time only if set in json), allows usage of same line(s) matching different `logtype` option:
`# filterOptions: [{"logtype": "file"}, {"logtype": "short"}, {"logtype": "journal"}]`
2020-01-22 17:19:35 +01:00
sebres 3befbb1770 improved wait for observer stop on server quit (second stop would force quit), this also cause reset db in observer (to avoid out of sequence errors) before database gets ultimately closed at end of server stop process (gh-2608) 2020-01-20 16:45:01 +01:00
sebres 7f1b578af4 Merge branch 'gh-488-check-on-error' 2020-01-16 16:52:44 +01:00
sebres 8cbc1e0ebb ChangeLog (change actioncheck behavior) 2020-01-16 16:51:57 +01:00
sebres 546fff8dc0 Merge branch '0.11' 2020-01-16 16:25:20 +01:00
Sergey G. Brester d2d3762ba9
Merge pull request #2605 from angeloc/0.11
Fixing --withouth-test install option
2020-01-16 13:45:49 +01:00