When a file is deleted, the watcher for it is automatically removed.
FilterPyinotify is detecting a new file being created with the same
name, and in turn attempts to remove the watch for the deleted file
(which has already been removed automatically).
Also, IN_IGNORED events are generated when a file is deleted, but these
weren't being caught, causing an non-existent file path to be passed to
FilterPyinotify._process_file (which caught the exceptions in
Filter.getFailures).
Realized --print-all-{missed,ignored} aren't meant to be simple on/off,
but are meant to enable printing all lines even when there are more
than 20.
This restores the behavior of --print-all-*, but preserves the
memory-savings when --print-all-* are not specified.
Also adds --print-no-{missed,ignored} which can be used to suppress
printing of a line type entirely.
When testing proccessing a large logfile (430 MB, 2M lines) without
--print-all-missed or --print-all-ignored, fail2ban-regex consumes
just over 2 GB of RAM before completing. Even though it does not need
to retain any of the missed or ignored log lines, it does anyway.
With this patch, memory use never grows beyond about 11 MB (unless
--print-all-* are enabled).
Previously, fail2ban-regex always printed both regardless of whether
those options were present or not.
This commit duplicates the patch I submitted at
https://sourceforge.net/p/fail2ban/patches/25/
I removed the ignoreip setting from the nagios section. As pointed out, it is redundant here. Nagios server, under normal circumstances should not trigger any access errors, and would be included in the global ignoreips anyway.
The second failregex was supposed to catch an error concerning an ACL denial over IPv6, but this message is no more generated by the nrpe version (v2.15) that introduced the IPv6 support, so the first failregex seems to be sufficient.
The dot at the ignoregex did not belong there. Somehow it was added during the copying and pasting. Thanks for reporting it, I did not see it. Otherwise, empty ignoregexes are in all filters, and if they are missing, fail2ban client shows warnings when starting the filter, which I prefer avoiding.