github
/
fail2ban
mirror of https://github.com/fail2ban/fail2ban
1
0
Fork 0
Code Issues Releases Wiki Activity

Created Developing Regex in Fail2ban (markdown)

master
Egbert 2020-09-20 18:25:15 -04:00
parent 0518c517bc
commit 21e9bd76ab
1 changed files with 248 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

248
Developing-Regex-in-Fail2ban.md Normal file

@ -0,0 +1,248 @@
So, you’re eager to write a new fail2ban filter and it failed … miserably or you have a new case but unsure how to get the best regex ... fastest.
If the `fail2ban` couldn’t match anything … regardless of whether it is standard fail2ban config or your highly, purportedly, hapzardly-concoted filter config file: this page is for you.
That is what this page offers, specifically developing as well as troubleshooting Regex used by `fail2ban`.
WHAT ARE THE STAGES OF REGEX?
=============================
Fail2ban has several components of regex in which to apply toward the log text, these components/subcomponents are:
* datepattern
* prefregex
* failregex
* ignoreregex
Usually, the date starts at the beginning of each log line that `fail2ban` searches against. For this article, we shall assume that date comes firstly before anything.
ACTUAL EXAMPLES!
================
The actual examples were obtained during a DDOS against my Bind9 master nameserver. And a regex is needed ... fast.
Actual log file is (after privacy redactions):
```log
19-Sep-2020 11:47:00.116 query-errors: info: client @0x7f0410000e40 123.123.123.123#80 (sl): view red: query failed (REFUSED) for sl/IN/ANY at query.c:5445
19-Sep-2020 11:47:01.120 query-errors: info: client @0x7f0410000e40 123.123.123.123#80 (sl): view red: query failed (REFUSED) for sl/IN/ANY at query.c:5445
19-Sep-2020 11:47:02.020 query-errors: info: client @0x7f0410000e40 123.123.123.123#80 (sl): view red: query failed (REFUSED) for sl/IN/ANY at query.c:5445
19-Sep-2020 11:47:03.356 query-errors: info: client @0x7f0410000e40 123.123.123.123#80 (sl): view red: query failed (REFUSED) for sl/IN/ANY at query.c:5445
19-Sep-2020 11:47:04.988 query-errors: info: client @0x7f0410000e40 123.123.123.123#80 (sl): view red: query failed (REFUSED) for sl/IN/ANY at query.c:5445
19-Sep-2020 11:47:05.576 query-errors: info: client @0x7f0410000e40 123.123.123.123#80 (sl): view red: query failed (REFUSED) for sl/IN/ANY at query.c:5445
```
Note: A little history, the sl TLD went off-line and IoTs were spraying invalid DNS-QUERY records with falsified source IP address toward selected DNS servers, resulting in a mild DNS amplification attack via DNS-QUERY-REFUSED error message all being sent to the target victim.
Sadly, latest Bind9 daemon has no configurable field to suppress these false DNS-QUERY-REFUSED acknowledgement messages (ISC Bind team claim it is not kosher to do this, but I still have a problem and intend fail2ban to deal with it).
FIRST PATTERN, FIRST
====================
Hopefully you got a ‘date’ hit. Something like from your `fail2ban-regex -v` (please note the important -v command line option):
```bash
fail2ban-regex -v /tmp/captured.log /etc/fail2ban/filter.d/named-refused.conf
```
which outputted the following:
```console
...
Date template hits:
|- [# of hits] date format
| [6] {^LN-BEG}Day(?P<_sep>[-/])MON(?P=_sep)ExYear[ :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
...
```
whose output shows `[6]` lines that have matched the date timestamp at the beginning of each line. That’s an excellent good start for troubleshooting.
In the sad case of [0] match for a date pattern hit, use the `--VD` option along with `-l HEAVYDEBUG` option in your `fail2ban-regex`. Having a [0] means you are dealing with a log text whose `datepattern` that fail2ban has never dealt with before. You’ll need to craft your own `datepattern`.
Such unknown `datepattern` shall be a subject for another blog, not here.
PRE-FILTER MATCHED
==================
In every filter file, `prefregex` defaults to `^(?P<content>.+)$`. If you haven’t touch or set the `prefregex`, move on to the next section.
Otherwise, `prefregex` becomes your focus in troubleshooting.
You can tell that the (default or customized) `prefregex` actually works if you added `-l HEAVYDEBUG` to your `fail2ban-regex` command line:
```bash
fail2ban-regex \
-v \
-l HEAVYDEBUG \
/tmp/captured.log \
/etc/fail2ban/filter.d/named-refused.conf
``
Remember the above command; we are going to use it each time we modified the filter configuration file: and quite very often. Use your bash history buffer and recall that command, over and over again. Remember.
and its output shows a line starting with `T: Pre-filter matched`:
H: Looking for prefregex '^(?P<content>.+)$'
T: Pre-filter matched {'content': ' query-errors: info: client @0x7f01e00004e0 123.123.123.123#80 (sl): view red: query failed (REFUSED) for sl/IN/ANY at query.c:5445'}
and note the value of `'content:'`. This content comes after the `datepattern`; we have successfully parse the date timestamp. Next, remaining content is then fed into the `failregex` patterns.
Note: Please note in 'content': value that there is an extra space at the beginning of that value so be careful with the ‘`^`‘ and make sure it starts with ‘`^ `‘ (note a space after caret symbol.)
But, with regard to that extra space char, do what I do; incorporate that space into your `prefregex`. Your customized `prefregex` will take away that beginning but lone space character from all your future (and current) `failregex` filter patterns. This makes for an easier-to-read `failpregex` pattern(s).
```ini
prefregex = ^ <F-CONTENT>.+</F-CONTENT>$
```
The above custom `prefregex` will ensure that that beginning space character is removed before sending the remaining content to the `failregex`. This new `prefregex` returns just the interesting '`<F-CONTENT>.+</F-CONTENT>$`' which is basically everything after that lone (but unwanted) space char.
Running that fail2ban-regex with the '`-l HEAVYDEBUG`', the new output shows:
```console
T: Pre-filter matched {'content': 'query-errors: info: client @0x7f0410000e40 123.123.123.123#80 (sl): view red: query failed (REFUSED) for sl/IN/ANY at query.c:5445'}
```
Notice that a space no longer exist before '`query-errors`'.
Everything from the beginning of the first non-space to the end of the line can then be dealt with by our yet-to-be-defined `failregex`.
FAILREGEX MATCHED
==================
Focus on the `failregex` portion of the filter config file. They're under `[Definition]`.
The catch of using `failregex` is that there MUST be at least one regex group match such as '`<HOST>`', '`<ADDR>`', or '`<F-USER>`'.
So, do what I do… Make a generic failregex` in your filter confing file like this:
```ini
failregex = query.+<HOST>
```
Notice that there is no '`$`' to catch end-of-line match condition? We’ll do those '`$`' lastly because we’re trying to just match … ANYTHING!
Re-run fail2ban-regex with -l HEAVYDEBUG and notice the '`T: Matched FailRegex part`':
```console
T: Matched FailRegex('query.+(?:(?:::f{4,6}:)?(?P<ip4>(?:\\d{1,3}\\.){3}\\d{1,3})|\\[?(?P<ip6>(?:[0-9a-fA-F]{1,4}::?|::){1,7}(?:[0-9a-fA-F]{1,4}|(?<=:):))\\]?|(?P<dns>[\\w\\-.^_]*\\w))')
```
Now I am matching SOMETHING!
Notice the convoluted patterns after '`query.+?`' These long patterns represent '`<HOST>`' part. We can safely ignore that for now.
Most importantly, I am MATCHING something that starts with '`^query`'! Yippee!
GYRATING TOWARD FULL MATCH
With a working matching pattern (albeit a failed but overly-broad pattern), we can then work toward a full-blown concise (yet flexible) pattern.
Let’s start by adding more static pattern. I am pretty sure from my intensive examination of that line 5445 in Bind9 `query.c` source file that '`query-error: info:`' is something that will not change for my target condition. This log output may have other variance like '`query-error: warn`' or '`query-error: debug`' but I am ignoring those.
First iteration of `failregex` expansion:
```ini
failregex = ^query-errors: info: .+<HOST>
```
Execute the command:
```bash
fail2ban-regex \
-l HEAVYDEBUG \
--print-no-missed \
/tmp/query-errors.log named-refused.local
``
and notice the output:
```console
Results
=======
Failregex: 6 total
|- #) [# of hits] regular expression
| 1) [6] ^query-errors: info: .+<HOST>
`-
```
See the '`[6]`'? I have six matches out of 6 lines give in log text file. I am getting close to a full-blown pattern! Don’t forget, we have to close that pattern out with a $ but not yet, save that for the end of this tutorial.
CAUTION: Every time you make a change to your filter file, PAY VERY CLOSE ATTENTION to this part of the output:
```console
Failregex: X total tabulation.
```
Once you get that '`Failregex: 0 total`', you know you have done something HORRIBLE, busted and broke your pattern, so roll that pattern back to its simpler pattern and start again.
INCREMENTS, INCREMENTS, INCREMENTS
==================================
As we add more and more increments of pattern and ensuring that '`Failregex: 6 matches`' still appears:
```console
failregex: query-errors: info: client.+<HOST>
failregex: query-errors: info: client @0x[0-9a-fA-F]{8,12}.+<HOST>
```
Whoa, my pattern is getting too long… so I made a variable to contain this entire pattern and called it '`_client`'.
```ini
_client = query-error: info: client @0x[0-9a-f]{8,12}
```
Now I can shorten the '`failregex`' a bit:
```ini
failregex = ^%(_client)s <HOST>
```
It’s the same thing, but oh it so readable, onward to matching the rest of the line.
NOTE: You are running `fail2ban-regex` between each modification, aren’t you?
NOW FOR THE ENDING PART
=======================
We have FINALLY reached the '`<HOST>`' part of the failregex/log text.
Now it is closing time! Let’s race to the '`$`' (end).
Add that port number after the host:
```ini
failregex = ^%(_client)s <HOST>#\d{1,5}
```
NOTE: You are still re-running fail2ban-regex between each modification, aren’t you?
REPETITION
==========
Notice that the domain name '`sl`' got used twice in each of the same log line?
Let us make a pattern called '`_domain`' to reduce our typing errors a bit.
```ini
_domain = [0-9a-zA-Z\._\-]{2,256}
```
Our new `failregex` becomes:
```ini
failregex = ^%(_client)s <HOST>#\d{1,5} \(%(_domain)s\):
```
NOTE: You are running `fail2ban-regex` between each modification, still getting that exact same match '`[6]`' (or whatever count you’re aiming for.)
SIMPLIFICATION
==============
Now for the view part of nameserver error output where `ISC Bind9` handles the view name.
View name is optional: We may not get a view name on some nameserver installation.
```ini
_view_name = [0-9a-zA-Z\._\-]{1,64}
_view = ( \%(_domain)s\))?: view %(_view_name)s
```
Our latest `failregex` becomes:
```ini
failregex = ^%(_client)s <HOST>#\d{1,5}%(_view)s
```
Still have a long way to go before we add that '`$`' ending pattern.
NOTE: `fail2ban-regex` between each modification still?
FINAL STRETCH
=============
We have the remaining of log text left to go:
```console
query failed (REFUSED) for example.tld/IN/ANY at query.c:5445
```
We’re impatient lot, aren’t we? Rush it up with:
```ini
_query_refused = query failed \(REFUSED\) for %(_dns_tuple)s at %(_codeloc)s$
```
and supply missing defines:
```ini
_domain = [0-9a-zA-Z\._\-]{1,254}
_dns_tuple = %(_domain)s\/IN\/ANY
_filespec = [0-9a-zA-Z\._\-]{1,254}
_codeloc = %(_filespec)s:\d{1,6}
```
NOTE: Guess? You are running fail2ban-regex between each modification? You still getting that non-zero '`Failregex: 6 total match`' under Results?
```console
Failregex: 6 total
|- #) [# of hits] regular expression
| 1) [6] ^query-errors: info: client @0x[0-9a-f]{8,12} <HOST>#\d{1,5}( \([0-9a-zA-Z\._\-]{1,254}\))?: view [0-9a-zA-Z\._\-]{1,64}: query failed \(REFUSED\) for [0-9a-zA-Z\._\-]{1,254}\/IN\/ANY at [0-9a-zA-Z\._\-]{1,254}:\d{1,6}$
`-
```
Ok, you could have paid attention to the last line of the output:
```console
Lines: 6 lines, 0 ignored, 6 matched, 0 missed
```
But I’ve find this to be easily overlooked hence the focal point within the Failregex as being more informative.
CONCLUSION
==========
Now we can add the `'$'` to the end of `failregex`.
Execute `fail2ban-client reload` and watch the blocking begin.