Large set of fixes and enhancements for `systemd` and `auto` backends:
* fixes `systemd` bug with missing journal descriptor after rotation by reopening of journal if it is recognized as not alive (gh-3929)
* improve threaded clean-up of all filters, new thread functions `afterStop` (to force clean-up after stop) and `done`, invoking `afterStop` once
* ensure journal-reader is always closed (additional prevention against leaks and "too many open files"), thereby avoid sporadic segfault in systemd module (see https://github.com/systemd/python-systemd/issues/143)
* fixes `systemd` causing "too many open files" error for a lot of journal files and large amout of systemd jails (see new parameter `rotated` below, gh-3391);
* backend `systemd` extended with new parameter `rotated` (default `false`, as prevention against "too many open files"),
that allows to monitor only actual journals and ignore now a lot of rotated files by default; so can drastically reduce
amount of used file descriptors, normally to 1 or 2 descriptors per jail (gh-3391)
* implements automatic switch `backend = auto` to backend `systemd`, when the following is true (RFE gh-3768):
- no files matching `logpath` found for this jail;
- no `systemd_if_nologs = false` is specified for the jail (`true` by default);
- option `journalmatch` is set for the jail or its filter (otherwise it'd be too heavy to allow all auto-jails,
even if they have never been foreseen for journal monitoring);
(option `skip_if_nologs` will be ignored if we could switch backend to `systemd`)
- no files matching `logpath` found for this jail;
- no `systemd_if_nologs = false` (`true` by default) is specified for the jail;
- option `journalmatch` is set for the jail or its filter (otherwise it'd be too heavy to allow all auto-jails, even if they have never been foreseen for journal);
- option `skip_if_nologs` will be ignored if we could switch backend to `systemd`;
closes gh-3768
since line 7 matches successfully now (it was disabled in gh-358 because of obsolete format), it is marked as match:true (line can be removed later if unneeded)
configuration `ignoreip` and fail2ban-client commands `addignoreip`/`delignoreip` extended with `file:...` syntax to ignore IPs from file-ip-set (containing IP, subnet, dns/fqdn or raw strings);
the file would be read lazy on demand, by first ban (and automatically reloaded by update after small latency to avoid expensive stats check on every compare);
the entries inside the file can be separated by comma, space or new line with optional comments (text following chars # or ; after space or newline would be ignored up to next newline)
Sergey G. Brester