man/jail.conf.5: documentation extended (prefregex, etc)

closes gh-2820

man/jail.conf.5

@@ -276,6 +276,9 @@ It defaults to "auto" which will try "pyinotify", "gamin", "systemd" before "pol
 .B usedns
 use DNS to resolve HOST names that appear in the logs. By default it is "warn" which will resolve hostnames to IPs however it will also log a warning. If you are using DNS here you could be blocking the wrong IPs due to the asymmetric nature of reverse DNS (that the application used to write the domain name to log) compared to forward DNS that fail2ban uses to resolve this back to an IP (but not necessarily the same one). Ideally you should configure your applications to log a real IP. This can be set to "yes" to prevent warnings in the log or "no" to disable DNS resolution altogether (thus ignoring entries where hostname, not an IP is logged)..
 .TP
+.B prefregex
+regex (Python \fBreg\fRular \fBex\fRpression) to parse a common part containing in every message (see \fBprefregex\fR in section FILTER FILES for details).
+.TP
 .B failregex
 regex (Python \fBreg\fRular \fBex\fRpression) to be added to the filter's failregexes (see \fBfailregex\fR in section FILTER FILES for details). If this is useful for others using your application please share you regular expression with the fail2ban developers by reporting an issue (see REPORTING BUGS below).
 .TP
@@ -432,7 +435,36 @@ These are used to identify failed authentication attempts in log files and to ex
 
 Like action files, filter files are ini files. The main section is the [Definition] section.
 
-There are two filter definitions used in the [Definition] section:
+There are several standard filter definitions used in the [Definition] section:
+.TP
+.B prefregex
+is the regex (\fBreg\fRular \fBex\fRpression) to parse a common part containing in every message, which is applied after \fBdatepattern\fR found a match, before the search for any \fBfailregex\fR or \fBignoreregex\fR would start.
+.br
+If this regex doesn't match the process is starting immediately with next message and search for any \fBfailregex\fR does not occur.
+.br
+If \fBprefregex\fR contains \fI<F-CONTENT>...</F-CONTENT>\fR, the part of message enclosed between this tags will be extracted and herafter used as whole message for search with \fBfailregex\fR or \fBignoreregex\fR.
+.IP
+For example:
+.nf
+        prefregex = ^%(__prefix_line)s (?:ERROR|FAILURE) <F-CONTENT>.+</F-CONTENT>$
+        failregex = ^user not found
+                    ^authentication failed
+                    ^unknown authentication method
+.fi
+.IP
+You can use \fBprefregex\fR in order to:
+.RS
+.IP
+- specify 1 common regex to match some common part present in every messages (do avoid unneeded match in every \fBfailregex\fR if you have more as one);
+.IP
+- to cut some interesting part of message only (to simplify \fBfailregex\fR) enclosed between tags \fI<F-CONTENT>\fI and \fI</F-CONTENT>\fR;
+.IP
+- to gather some failure identifier (e. g. some prefix matched by \fI<F-MLFID>...<F-MLFID/>\fR tag) to identify several messages belonging to same session, where a connect message containing IP followed by failure message(s) that are not contain IP;
+this provides a new multi-line parsing method as replacement for old (slow an ugly) multi-line parsing using buffering window (\fImaxlines\fR > 1 and \fI<SKIPLINES>\fR);
+.IP
+- to ignore some wrong, too long or even unneeded messages (a.k.a. parasite log traffic) which can be also present in journal, before \fBfailregex\fR search would take place.
+.RE
+
 .TP
 .B failregex
 is the regex (\fBreg\fRular \fBex\fRpression) that will match failed attempts. The standard replacement tags can be used as part of the regex:
@@ -451,17 +483,18 @@ is the regex (\fBreg\fRular \fBex\fRpression) that will match failed attempts. T
 \fI<CIDR>\fR - helper regex to match CIDR (simple integer form of net-mask).
 .IP
 \fI<SUBNET>\fR - regex to match sub-net adresses (in form of IP/CIDR, also single IP is matched, so part /CIDR is optional).
+.PP
+\fBNOTE:\fR the \fBfailregex\fR will be applied to the remaining part of message after \fBprefregex\fR processing (if specified), which in turn takes place after \fBdatepattern\fR processing (whereby the string of timestamp matching the best pattern, cut out from the message).
+.PP
+For multiline regexs (parsing with \fImaxlines\fR greater that 1) the tag \fI<SKIPLINES>\fR can be used to separate lines. This allows lines between the matched lines to continue to be searched for other failures. The tag can be used multiple times.
+.br
+This is an obsolete handling and if the lines contain some common identifier, better would be to use new handling (with tags \fI<F-MLFID>...<F-MLFID/>\fR).
 .RE
-.TP
-For multiline regexs the tag \fI<SKIPLINES>\fR should be used to separate lines. This allows lines between the matched lines to continue to be searched for other failures. The tag can be used multiple times.
 
 .TP
 .B ignoreregex
 is the regex to identify log entries that should be ignored by Fail2Ban, even if they match failregex.
 
-
-.PP
-Similar to actions, filters have an [Init] section which can be overridden in \fIjail.conf/jail.local\fR. Besides the filter-specific settings, the filter [Init] section can be used to set following standard options:
 .TP
 \fBmaxlines\fR
 specifies the maximum number of lines to buffer to match multi-line regexs. For some log formats this will not required to be changed. Other logs may require to increase this value if a particular log file is frequently written to.
@@ -492,7 +525,9 @@ There are several prefixes and words with special meaning that could be specifie
 \fBjournalmatch\fR
 specifies the systemd journal match used to filter the journal entries. See \fBjournalctl(1)\fR and \fBsystemd.journal-fields(7)\fR for matches syntax and more details on special journal fields. This option is only valid for the \fIsystemd\fR backend.
 .PP
-Similar to actions [Init] section enables filter-specific settings. All parameters specified in [Init] section can be redefined or extended in \fIjail.conf/jail.local\fR.
+Similar to actions, filters may have an [Init] section also (optional since v.0.10). All parameters of both sections [Definition] and [Init] can be overridden (redefined or extended) in \fIjail.conf\fR or \fIjail.local\fR (or in related \fIfilter.d/filter-name.local\fR).
+Every option supplied in the jail to the filter overwrites the value specified in [Init] section, which in turm would overwrite the value in [Definition] section.
+Besides the standard settings of filter both sections can be used to initialize filter-specific options.
 
 Filters can also have a section called [INCLUDES]. This is used to read other configuration files.