Merge pull request #1710 from sebres/0.10-test-with-filter-options

0.10 filter options extension
pull/1715/head
Serg G. Brester 8 years ago committed by GitHub
commit 1bcde678c6

@ -15,6 +15,14 @@ TODO: implementing of options resp. other tasks from PR #1346
### Fixes
* `filter.d/pam-generic.conf`:
- [grave] injection on user name to host fixed
* `filter.d/sshd.conf`:
- rewritten using `prefregex` and used MLFID-related multi-line parsing
(by using tag `<F-MLFID>` instead of buffering with `maxlines`);
- optional parameter `mode` rewritten: normal (default), ddos, extra or aggressive (combines all),
see sshd for regex details)
* filter.d/sendmail-reject.conf:
- rewritten using `prefregex` and used MLFID-related multi-line parsing;
- optional parameter `mode` introduced: normal (default), extra or aggressive
* `action.d/complain.conf`
- fixed using new tag `<ip-rev>` (sh/dash compliant now)
* `action.d/sendmail-geoip-lines.conf`
@ -51,6 +59,11 @@ TODO: implementing of options resp. other tasks from PR #1346
- `<ip-rev>` - PTR reversed representation of IP address
- `<ip-host>` - host name of the IP address
- `<F-...>` - interpolates to the corresponding filter group capture `...`
* Allow to use filter options by `fail2ban-regex`, example:
fail2ban-regex text.log "sshd[mode=aggressive]"
* Samples test case factory extended with filter options - dict in JSON to control
filter options (e. g. mode, etc.):
# filterOptions: {"mode": "aggressive"}
ver. 0.10.0-alpha-1 (2016/07/14) - ipv6-support-etc

@ -138,7 +138,6 @@ config/filter.d/solid-pop3d.conf
config/filter.d/squid.conf
config/filter.d/squirrelmail.conf
config/filter.d/sshd.conf
config/filter.d/sshd-ddos.conf
config/filter.d/stunnel.conf
config/filter.d/suhosin.conf
config/filter.d/tine20.conf
@ -327,7 +326,6 @@ fail2ban/tests/files/logs/solid-pop3d
fail2ban/tests/files/logs/squid
fail2ban/tests/files/logs/squirrelmail
fail2ban/tests/files/logs/sshd
fail2ban/tests/files/logs/sshd-ddos
fail2ban/tests/files/logs/stunnel
fail2ban/tests/files/logs/suhosin
fail2ban/tests/files/logs/tine20

@ -21,30 +21,45 @@ before = common.conf
_daemon = (?:(sm-(mta|acceptingconnections)|sendmail))
failregex = ^%(__prefix_line)s\w{14}: ruleset=check_rcpt, arg1=(?P<email><\S+@\S+>), relay=(\S+ )?\[<HOST>\]( \(may be forged\))?, reject=(550 5\.7\.1 (?P=email)\.\.\. Relaying denied\. (IP name possibly forged \[(\d+\.){3}\d+\]|Proper authentication required\.|IP name lookup failed \[(\d+\.){3}\d+\])|553 5\.1\.8 (?P=email)\.\.\. Domain of sender address \S+ does not exist|550 5\.[71]\.1 (?P=email)\.\.\. (Rejected: .*|User unknown))$
^%(__prefix_line)sruleset=check_relay, arg1=(?P<dom>\S+), arg2=<HOST>, relay=((?P=dom) )?\[(\d+\.){3}\d+\]( \(may be forged\))?, reject=421 4\.3\.2 (Connection rate limit exceeded\.|Too many open connections\.)$
^%(__prefix_line)s\w{14}: rejecting commands from (\S* )?\[<HOST>\] due to pre-greeting traffic after \d+ seconds$
^%(__prefix_line)s\w{14}: (\S+ )?\[<HOST>\]: ((?i)expn|vrfy) \S+ \[rejected\]$
^(?P<__prefix>%(__prefix_line)s\w+: )<[^@]+@[^>]+>\.\.\. No such user here$<SKIPLINES>^(?P=__prefix)from=<[^@]+@[^>]+>, size=\d+, class=\d+, nrcpts=\d+, bodytype=\w+, proto=E?SMTP, daemon=MTA, relay=\S+ \[<HOST>\]$
prefregex = ^<F-MLFID>%(__prefix_line)s(?:\w{14}: )?</F-MLFID><F-CONTENT>.+</F-CONTENT>$
cmnfailre = ^ruleset=check_rcpt, arg1=(?P<email><\S+@\S+>), relay=(\S+ )?\[<HOST>\](?: \(may be forged\))?, reject=(550 5\.7\.1 (?P=email)\.\.\. Relaying denied\. (IP name possibly forged \[(\d+\.){3}\d+\]|Proper authentication required\.|IP name lookup failed \[(\d+\.){3}\d+\])|553 5\.1\.8 (?P=email)\.\.\. Domain of sender address \S+ does not exist|550 5\.[71]\.1 (?P=email)\.\.\. (Rejected: .*|User unknown))$
^ruleset=check_relay, arg1=(?P<dom>\S+), arg2=<HOST>, relay=((?P=dom) )?\[(\d+\.){3}\d+\](?: \(may be forged\))?, reject=421 4\.3\.2 (Connection rate limit exceeded\.|Too many open connections\.)$
^rejecting commands from (\S* )?\[<HOST>\] due to pre-greeting traffic after \d+ seconds$
^(?:\S+ )?\[<HOST>\]: (?:(?i)expn|vrfy) \S+ \[rejected\]$
^<[^@]+@[^>]+>\.\.\. No such user here$
^<F-NOFAIL>from=<[^@]+@[^>]+></F-NOFAIL>, size=\d+, class=\d+, nrcpts=\d+, bodytype=\w+, proto=E?SMTP, daemon=MTA, relay=\S+ \[<HOST>\]$
ignoreregex =
mdre-normal =
mdre-extra = ^(?:\S+ )?\[<HOST>\](?: \(may be forged\))? did not issue (?:[A-Z]{4}[/ ]?)+during connection to M(?:TA|SP)(?:-\w+)?$
[Init]
mdre-aggressive = %(mdre-extra)s
failregex = %(cmnfailre)s
<mdre-<mode>>
# Parameter "mode": normal (default), extra or aggressive
# Usage example (for jail.local):
# [sendmail-reject]
# filter = sendmail-reject[mode=extra]
#
mode = normal
ignoreregex =
# "maxlines" is number of log lines to buffer for multi-line regex searches
maxlines = 10
# DEV NOTES:
#
# Regarding the last multiline regex:
# Regarding the multiline regex:
#
# There can be a nunber of non-related lines between the first and second part
# of this regex maxlines of 10 is quite generious. Only one of the
# "No such user" lines needs to be matched before the line with the HOST.
# "No such user" lines generate a failure and needs to be matched together with
# another line with the HOST, therefore no-failure line was added as regex, that
# contains HOST (see line with tag <F-NOFAIL>).
#
# Note the capture __prefix, includes both the __prefix_lines (which includes
# the sendmail PID), but also the \w+ which the the sendmail assigned mail ID.
# Note the capture <F-MLFID>, includes both the __prefix_lines (which includes
# the sendmail PID), but also the `\w{14}` which the the sendmail assigned
# mail ID (todo: check this is necessary, possible obsolete).
#
# Author: Daniel Black and Fabian Wenk
# Author: Daniel Black, Fabian Wenk and Sergey Brester aka sebres.
# Rewritten using prefregex by Serg G. Brester.

@ -1,11 +0,0 @@
# Fail2Ban aggressive ssh filter for at attempted exploit
#
# Includes failregex of both sshd and sshd-ddos filters
#
[INCLUDES]
before = sshd.conf
[Definition]
mode = %(aggressive)s

@ -1,17 +0,0 @@
# Fail2Ban ssh filter for at attempted exploit
#
# The regex here also relates to a exploit:
#
# http://www.securityfocus.com/bid/17958/exploit
# The example code here shows the pushing of the exploit straight after
# reading the server version. This is where the client version string normally
# pushed. As such the server will read this unparsible information as
# "Did not receive identification string".
[INCLUDES]
before = sshd.conf
[Definition]
mode = %(ddos)s

@ -24,45 +24,59 @@ __pref = (?:(?:error|fatal): (?:PAM: )?)?
__suff = (?: \[preauth\])?\s*
__on_port_opt = (?: port \d+)?(?: on \S+(?: port \d+)?)?
prefregex = ^<F-MLFID>%(__prefix_line)s</F-MLFID>%(__pref)s<F-CONTENT>.+</F-CONTENT>$
[Definition]
mode = %(normal)s
normal = ^[aA]uthentication (?:failure|error|failed) for <F-USER>.*</F-USER> from <HOST>( via \S+)?\s*%(__suff)s$
^User not known to the underlying authentication module for <F-USER>.*</F-USER> from <HOST>\s*%(__suff)s$
^Failed \S+ for (?P<cond_inv>invalid user )?<F-USER>(?P<cond_user>\S+)|(?(cond_inv)(?:(?! from ).)*?|[^:]+)</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$)
^<F-USER>ROOT</F-USER> LOGIN REFUSED.* FROM <HOST>\s*%(__suff)s$
^[iI](?:llegal|nvalid) user <F-USER>.*?</F-USER> from <HOST>%(__on_port_opt)s\s*$
^User <F-USER>.+</F-USER> from <HOST> not allowed because not listed in AllowUsers\s*%(__suff)s$
^User <F-USER>.+</F-USER> from <HOST> not allowed because listed in DenyUsers\s*%(__suff)s$
^User <F-USER>.+</F-USER> from <HOST> not allowed because not in any group\s*%(__suff)s$
^refused connect from \S+ \(<HOST>\)\s*%(__suff)s$
^Received disconnect from <HOST>%(__on_port_opt)s:\s*3: .*: Auth fail%(__suff)s$
^User <F-USER>.+</F-USER> from <HOST> not allowed because a group is listed in DenyGroups\s*%(__suff)s$
^User <F-USER>.+</F-USER> from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*%(__suff)s$
^pam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=<F-USER>\S*</F-USER>\s*rhost=<HOST>\s.*%(__suff)s$
^(error: )?maximum authentication attempts exceeded for <F-USER>.*</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)? \[preauth\]$
^User <F-USER>.+</F-USER> not allowed because account is locked%(__suff)s
^Disconnecting: Too many authentication failures for <F-USER>.+?</F-USER>%(__suff)s
^<F-NOFAIL>Received disconnect</F-NOFAIL> from <HOST>: 11:
^<F-NOFAIL>Connection closed</F-NOFAIL> by <HOST>%(__suff)s$
ddos = ^Did not receive identification string from <HOST>%(__suff)s$
^Received disconnect from <HOST>%(__on_port_opt)s:\s*14: No supported authentication methods available%(__suff)s$
^Unable to negotiate with <HOST>%(__on_port_opt)s: no matching (?:cipher|key exchange method) found.
^Unable to negotiate a (?:cipher|key exchange method)%(__suff)s$
^<F-NOFAIL>SSH: Server;Ltype:</F-NOFAIL> (?:Authname|Version|Kex);Remote: <HOST>-\d+;[A-Z]\w+:
^Read from socket failed: Connection reset by peer \[preauth\]
common = ^<F-NOFAIL>Connection from</F-NOFAIL> <HOST>
aggressive = %(normal)s
%(ddos)s
prefregex = ^<F-MLFID>%(__prefix_line)s</F-MLFID>%(__pref)s<F-CONTENT>.+</F-CONTENT>$
[Definition]
cmnfailre = ^[aA]uthentication (?:failure|error|failed) for <F-USER>.*</F-USER> from <HOST>( via \S+)?\s*%(__suff)s$
^User not known to the underlying authentication module for <F-USER>.*</F-USER> from <HOST>\s*%(__suff)s$
^Failed \S+ for (?P<cond_inv>invalid user )?<F-USER>(?P<cond_user>\S+)|(?(cond_inv)(?:(?! from ).)*?|[^:]+)</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$)
^<F-USER>ROOT</F-USER> LOGIN REFUSED.* FROM <HOST>\s*%(__suff)s$
^[iI](?:llegal|nvalid) user <F-USER>.*?</F-USER> from <HOST>%(__on_port_opt)s\s*$
^User <F-USER>.+</F-USER> from <HOST> not allowed because not listed in AllowUsers\s*%(__suff)s$
^User <F-USER>.+</F-USER> from <HOST> not allowed because listed in DenyUsers\s*%(__suff)s$
^User <F-USER>.+</F-USER> from <HOST> not allowed because not in any group\s*%(__suff)s$
^refused connect from \S+ \(<HOST>\)\s*%(__suff)s$
^Received disconnect from <HOST>%(__on_port_opt)s:\s*3: .*: Auth fail%(__suff)s$
^User <F-USER>.+</F-USER> from <HOST> not allowed because a group is listed in DenyGroups\s*%(__suff)s$
^User <F-USER>.+</F-USER> from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*%(__suff)s$
^pam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=<F-USER>\S*</F-USER>\s*rhost=<HOST>\s.*%(__suff)s$
^(error: )?maximum authentication attempts exceeded for <F-USER>.*</F-USER> from <HOST>%(__on_port_opt)s(?: ssh\d*)? \[preauth\]$
^User <F-USER>.+</F-USER> not allowed because account is locked%(__suff)s
^Disconnecting: Too many authentication failures for <F-USER>.+?</F-USER>%(__suff)s
^<F-NOFAIL>Received disconnect</F-NOFAIL> from <HOST>: 11:
^<F-NOFAIL>Connection closed</F-NOFAIL> by <HOST>%(__suff)s$
mdre-normal =
mdre-ddos = ^Did not receive identification string from <HOST>%(__suff)s$
^<F-NOFAIL>SSH: Server;Ltype:</F-NOFAIL> (?:Authname|Version|Kex);Remote: <HOST>-\d+;[A-Z]\w+:
^Read from socket failed: Connection reset by peer \[preauth\]
mdre-extra = ^Received disconnect from <HOST>%(__on_port_opt)s:\s*14: No supported authentication methods available%(__suff)s$
^Unable to negotiate with <HOST>%(__on_port_opt)s: no matching (?:cipher|key exchange method) found.
^Unable to negotiate a (?:cipher|key exchange method)%(__suff)s$
mdre-aggressive = %(mdre-ddos)s
%(mdre-extra)s
cfooterre = ^<F-NOFAIL>Connection from</F-NOFAIL> <HOST>
failregex = %(cmnfailre)s
<mdre-<mode>>
%(cfooterre)s
# Parameter "mode": normal (default), ddos, extra or aggressive (combines all)
# Usage example (for jail.local):
# [sshd]
# mode = extra
# # or another jail (rewrite filter parameters of jail):
# [sshd-aggressive]
# filter = sshd[mode=aggressive]
#
mode = normal
failregex = %(mode)s
%(common)s
#filter = sshd[mode=aggressive]
ignoreregex =
@ -79,5 +93,5 @@ datepattern = {^LN-BEG}
# and later catch-all's could contain user-provided input, which need to be greedily
# matched away first.
#
# Author: Cyril Jaquier, Yaroslav Halchenko, Petr Voralek, Daniel Black
# Author: Cyril Jaquier, Yaroslav Halchenko, Petr Voralek, Daniel Black and Sergey Brester aka sebres
# Rewritten using prefregex (and introduced "mode" parameter) by Serg G. Brester.

@ -229,17 +229,11 @@ action = %(action_)s
[sshd]
# To use more aggressive sshd filter (inclusive sshd-ddos failregex):
#filter = sshd-aggressive
port = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s
[sshd-ddos]
# This jail corresponds to the standard configuration in Fail2ban.
# The mail-whois action send a notification e-mail with a whois request
# in the body.
# To use more aggressive sshd modes set filter parameter "mode" in jail.local:
# normal (default), ddos, extra or aggressive (combines all).
# See "tests/files/logs/sshd" or "filter.d/sshd.conf" for usage example and details.
mode = normal
filter = sshd[mode=%(mode)s]
port = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s
@ -555,7 +549,11 @@ backend = %(syslog_backend)s
[sendmail-reject]
# To use more aggressive modes set filter parameter "mode" in jail.local:
# normal (default), extra or aggressive
# See "tests/files/logs/sendmail-reject" or "filter.d/sendmail-reject.conf" for usage example and details.
mode = normal
filter = sendmail-reject[mode=%(mode)s]
port = smtp,465,submission
logpath = %(syslog_mail)s
backend = %(syslog_backend)s

@ -176,6 +176,8 @@ class ConfigReaderUnshared(SafeConfigParserWithIncludes):
if not os.path.exists(self._basedir):
raise ValueError("Base configuration directory %s does not exist "
% self._basedir)
if filename.startswith("./"): # pragma: no cover
filename = os.path.abspath(filename)
basename = os.path.join(self._basedir, filename)
logSys.debug("Reading configs for %s under %s " , filename, self._basedir)
config_files = [ basename + ".conf" ]
@ -277,6 +279,8 @@ class DefinitionInitConfigReader(ConfigReader):
def __init__(self, file_, jailName, initOpts, **kwargs):
ConfigReader.__init__(self, **kwargs)
if file_.startswith("./"): # pragma: no cover
file_ = os.path.abspath(file_)
self.setFile(file_)
self.setJailName(jailName)
self._initOpts = initOpts

@ -120,6 +120,8 @@ Report bugs to https://github.com/fail2ban/fail2ban/issues
version="%prog " + version)
p.add_options([
Option("-c", "--config", default='/etc/fail2ban',
help="set alternate config directory"),
Option("-d", "--datepattern",
help="set custom pattern used to match date/times"),
Option("-e", "--encoding", default=PREFER_ENC,
@ -271,24 +273,60 @@ class Fail2banRegex(object):
def readRegex(self, value, regextype):
assert(regextype in ('fail', 'ignore'))
regex = regextype + 'regex'
if regextype == 'fail' and (os.path.isfile(value) or os.path.isfile(value + '.conf')):
if os.path.basename(os.path.dirname(value)) == 'filter.d':
# try to check - we've case filter?[options...]?:
basedir = self._opts.config
fltFile = None
fltOpt = {}
if regextype == 'fail':
fltName, fltOpt = JailReader.extractOptions(value)
if fltName is not None:
if "." in fltName[~5:]:
tryNames = (fltName,)
else:
tryNames = (fltName, fltName + '.conf', fltName + '.local')
for fltFile in tryNames:
if not "/" in fltFile:
if os.path.basename(basedir) == 'filter.d':
fltFile = os.path.join(basedir, fltFile)
else:
fltFile = os.path.join(basedir, 'filter.d', fltFile)
else:
basedir = os.path.dirname(fltFile)
if os.path.isfile(fltFile):
break
fltFile = None
# if it is filter file:
if fltFile is not None:
if (basedir == self._opts.config
or os.path.basename(basedir) == 'filter.d'
or ("." not in fltName[~5:] and "/" not in fltName)
):
## within filter.d folder - use standard loading algorithm to load filter completely (with .local etc.):
basedir = os.path.dirname(os.path.dirname(value))
value = os.path.splitext(os.path.basename(value))[0]
output( "Use %11s filter file : %s, basedir: %s" % (regex, value, basedir) )
reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config, basedir=basedir)
if not reader.read(): # pragma: no cover
output( "ERROR: failed to load filter %s" % value )
return False
else: # pragma: no cover
if os.path.basename(basedir) == 'filter.d':
basedir = os.path.dirname(basedir)
fltName = os.path.splitext(os.path.basename(fltName))[0]
output( "Use %11s filter file : %s, basedir: %s" % (regex, fltName, basedir) )
else:
## foreign file - readexplicit this file and includes if possible:
output( "Use %11s file : %s" % (regex, value) )
reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config)
reader.setBaseDir(None)
if not reader.readexplicit():
output( "ERROR: failed to read %s" % value )
return False
output( "Use %11s file : %s" % (regex, fltName) )
basedir = None
if fltOpt:
output( "Use filter options : %r" % fltOpt )
reader = FilterReader(fltName, 'fail2ban-regex-jail', fltOpt, share_config=self.share_config, basedir=basedir)
ret = None
try:
if basedir is not None:
ret = reader.read()
else:
## foreign file - readexplicit this file and includes if possible:
reader.setBaseDir(None)
ret = reader.readexplicit()
except Exception as e:
output("Wrong config file: %s" % (str(e),))
if self._verbose: raise(e)
if not ret:
output( "ERROR: failed to load filter %s" % value )
return False
reader.getOptions(None)
readercommands = reader.convert()

@ -43,7 +43,7 @@ logSys = getLogger(__name__)
class JailReader(ConfigReader):
# regex, to extract list of options:
optionCRE = re.compile(r"^([\w\-_\.]+)(?:\[(.*)\])?\s*$", re.DOTALL)
optionCRE = re.compile(r"^([^\[]+)(?:\[(.*)\])?\s*$", re.DOTALL)
# regex, to iterate over single option in option list, syntax:
# `action = act[p1="...", p2='...', p3=...]`, where the p3=... not contains `,` or ']'
# since v0.10 separator extended with `]\s*[` for support of multiple option groups, syntax

@ -23,9 +23,9 @@ __prefix_line_sl = %(__prefix_line)s%(__pref)s
__prefix_line_ml1 = (?P<__prefix>%(__prefix_line)s)%(__pref)s
__prefix_line_ml2 = %(__suff)s$<SKIPLINES>^(?P=__prefix)%(__pref)s
mode = %(normal)s
[Definition]
normal = ^%(__prefix_line_sl)s[aA]uthentication (?:failure|error|failed) for .* from <HOST>( via \S+)?\s*%(__suff)s$
cmnfailre = ^%(__prefix_line_sl)s[aA]uthentication (?:failure|error|failed) for .* from <HOST>( via \S+)?\s*%(__suff)s$
^%(__prefix_line_sl)sUser not known to the underlying authentication module for .* from <HOST>\s*%(__suff)s$
^%(__prefix_line_sl)sFailed \S+ for (?P<cond_inv>invalid user )?(?P<user>(?P<cond_user>\S+)|(?(cond_inv)(?:(?! from ).)*?|[^:]+)) from <HOST>%(__on_port_opt)s(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$)
^%(__prefix_line_sl)sROOT LOGIN REFUSED.* FROM <HOST>\s*%(__suff)s$
@ -43,18 +43,30 @@ normal = ^%(__prefix_line_sl)s[aA]uthentication (?:failure|error|failed) for .*
^%(__prefix_line_ml1)sDisconnecting: Too many authentication failures for .+?%(__prefix_line_ml2)sConnection closed by <HOST>%(__suff)s$
^%(__prefix_line_ml1)sConnection from <HOST>%(__on_port_opt)s%(__prefix_line_ml2)sDisconnecting: Too many authentication failures for .+%(__suff)s$
ddos = ^%(__prefix_line_sl)sDid not receive identification string from <HOST>%(__suff)s$
^%(__prefix_line_sl)sReceived disconnect from <HOST>%(__on_port_opt)s:\s*14: No supported authentication methods available%(__suff)s$
^%(__prefix_line_sl)sUnable to negotiate with <HOST>%(__on_port_opt)s: no matching (?:cipher|key exchange method) found.
^%(__prefix_line_ml1)sConnection from <HOST>%(__on_port_opt)s%(__prefix_line_ml2)sUnable to negotiate a (?:cipher|key exchange method)%(__suff)s$
^%(__prefix_line_ml1)sSSH: Server;Ltype: (?:Authname|Version|Kex);Remote: <HOST>-\d+;[A-Z]\w+:.*%(__prefix_line_ml2)sRead from socket failed: Connection reset by peer%(__suff)s$
mdre-normal =
aggressive = %(normal)s
%(ddos)s
mdre-ddos = ^%(__prefix_line_sl)sDid not receive identification string from <HOST>%(__suff)s$
^%(__prefix_line_ml1)sSSH: Server;Ltype: (?:Authname|Version|Kex);Remote: <HOST>-\d+;[A-Z]\w+:.*%(__prefix_line_ml2)sRead from socket failed: Connection reset by peer%(__suff)s$
[Definition]
mdre-extra = ^%(__prefix_line_sl)sReceived disconnect from <HOST>%(__on_port_opt)s:\s*14: No supported authentication methods available%(__suff)s$
^%(__prefix_line_sl)sUnable to negotiate with <HOST>%(__on_port_opt)s: no matching (?:cipher|key exchange method) found.
^%(__prefix_line_ml1)sConnection from <HOST>%(__on_port_opt)s%(__prefix_line_ml2)sUnable to negotiate a (?:cipher|key exchange method)%(__suff)s$
mdre-aggressive = %(mdre-ddos)s
%(mdre-extra)s
failregex = %(mode)s
failregex = %(cmnfailre)s
<mdre-<mode>>
# Parameter "mode": normal (default), ddos, extra or aggressive (combines all)
# Usage example (for jail.local):
# [sshd]
# mode = extra
# # or another jail (rewrite filter parameters of jail):
# [sshd-aggressive]
# filter = sshd[mode=aggressive]
#
mode = normal
ignoreregex =

@ -209,7 +209,8 @@ class Fail2banRegexTest(LogCaptureTestCase):
(opts, args, fail2banRegex) = _Fail2banRegex(
"-l", "notice", # put down log-level, because of too many debug-messages
"-v", "--verbose-date", "--print-all-matched",
Fail2banRegexTest.FILENAME_SSHD, Fail2banRegexTest.FILTER_SSHD
"-c", CONFIG_DIR,
Fail2banRegexTest.FILENAME_SSHD, "sshd"
)
self.assertTrue(fail2banRegex.start(args))
# test failure line and not-failure lines both presents:
@ -220,7 +221,8 @@ class Fail2banRegexTest(LogCaptureTestCase):
(opts, args, fail2banRegex) = _Fail2banRegex(
"-l", "notice", # put down log-level, because of too many debug-messages
"--print-all-matched",
Fail2banRegexTest.FILENAME_ZZZ_SSHD, Fail2banRegexTest.FILTER_SSHD
"-c", CONFIG_DIR,
Fail2banRegexTest.FILENAME_ZZZ_SSHD, "sshd.conf[mode=normal]"
)
self.assertTrue(fail2banRegex.start(args))
# test failure line and all not-failure lines presents:
@ -234,7 +236,8 @@ class Fail2banRegexTest(LogCaptureTestCase):
(opts, args, fail2banRegex) = _Fail2banRegex(
"-l", "notice", # put down log-level, because of too many debug-messages
"--print-all-matched", "--print-all-missed",
Fail2banRegexTest.FILENAME_ZZZ_SSHD, Fail2banRegexTest.FILTER_ZZZ_SSHD
"-c", os.path.dirname(Fail2banRegexTest.FILTER_ZZZ_SSHD),
Fail2banRegexTest.FILENAME_ZZZ_SSHD, os.path.basename(Fail2banRegexTest.FILTER_ZZZ_SSHD)
)
self.assertTrue(fail2banRegex.start(args))
# test "failure" line presents (2nd part only, because multiline fewer precise):
@ -245,10 +248,17 @@ class Fail2banRegexTest(LogCaptureTestCase):
# by the way test of ignoreregex (specified in filter file)...
(opts, args, fail2banRegex) = _Fail2banRegex(
"-l", "notice", # put down log-level, because of too many debug-messages
Fail2banRegexTest.FILENAME_ZZZ_GEN, Fail2banRegexTest.FILTER_ZZZ_GEN
Fail2banRegexTest.FILENAME_ZZZ_GEN, Fail2banRegexTest.FILTER_ZZZ_GEN+"[mode=test]"
)
self.assertTrue(fail2banRegex.start(args))
def testWrongFilterFile(self):
# use test log as filter file to cover eror cases...
(opts, args, fail2banRegex) = _Fail2banRegex(
Fail2banRegexTest.FILENAME_ZZZ_GEN, Fail2banRegexTest.FILENAME_ZZZ_GEN
)
self.assertFalse(fail2banRegex.start(args))
def _reset(self):
# reset global warn-counter:
from ..server.filter import _decode_line_warn

@ -1,3 +1,5 @@
# normal mode # filterOptions: {"mode": "normal"}
# failJSON: { "time": "2005-02-25T03:01:10", "match": true , "host": "128.68.136.133" }
Feb 25 03:01:10 kismet sm-acceptingconnections[27713]: s1P819mk027713: ruleset=check_rcpt, arg1=<asservnew@freemailhost.ru>, relay=128-68-136-133.broadband.corbina.ru [128.68.136.133], reject=550 5.7.1 <asservnew@freemailhost.ru>... Relaying denied. Proper authentication required.
@ -69,20 +71,27 @@ Feb 22 14:02:44 batman sm-mta[4030]: s1MD2hsd004030: rrcs-24-73-201-194.se.biz.r
# failJSON: { "match": false }
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026254: <arhipov@domain.com>... No such user here
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026250: <arhipov@domain.com>... No such user here
# failJSON: { "match": false }
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026254: <anatoliy@domain.com>... No such user here
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026251: <anatoliy@domain.com>... No such user here
# failJSON: { "match": false }
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026254: <artem@domain.com>... No such user here
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026252: <artem@domain.com>... No such user here
# failJSON: { "match": false }
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026254: <anto@domain.com>... No such user here
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026252: <anto@domain.com>... No such user here
# failJSON: { "match": false }
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026254: <anton@domain.com>... No such user here
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026254: <davaojk25@domain.com>... No such user here
# failJSON: { "time": "2004-11-03T11:35:30", "match": true , "host": "95.32.23.163" }
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026254: from=<davaojk25@domain.com>, size=0, class=0, nrcpts=0, bodytype=8BITMIME, proto=ESMTP, daemon=MTA, relay=163.23.32.95.dsl-dynamic.vsi.ru [95.32.23.163]
# failJSON: { "match": false }
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026254: <anton@domain.com>... No such user here
# Different mail ID shouldn't match
# failJSON: { "match": false }
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026255: from=<davaojk25@domain.com>, size=0, class=0, nrcpts=0, bodytype=8BITMIME, proto=ESMTP, daemon=MTA, relay=163.23.32.95.dsl-dynamic.vsi.ru [95.32.23.163]
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026252: <anton@domain.com>... No such user here
# failJSON: { "match": false, "desc": "Different mail ID shouldn't match" }
Nov 3 11:35:30 Microsoft sendmail[26254]: rA37ZTSC026255: from=<anton@domain.com>, size=0, class=0, nrcpts=0, bodytype=8BITMIME, proto=ESMTP, daemon=MTA, relay=163.23.32.95.dsl-dynamic.vsi.ru [95.32.23.163]
# filterOptions: {"mode": "extra"}
# failJSON: { "time": "2005-03-06T16:55:28", "match": true , "host": "192.0.2.194", "desc": "wrong resp. non RFC compiant (ddos prelude?), MTA-mode" }
Mar 6 16:55:28 s192-168-0-1 sm-mta[20949]: v26LtRA0020949: some-host-24.example.org [192.0.2.194] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
# failJSON: { "time": "2005-03-07T15:04:37", "match": true , "host": "192.0.2.195", "desc": "wrong resp. non RFC compiant (ddos prelude?), MSP-mode, (may be forged)" }
Mar 7 15:04:37 s192-168-0-1 sm-mta[18624]: v27K4Vj8018624: some-host-24.example.org [192.0.2.195] (may be forged) did not issue MAIL/EXPN/VRFY/ETRN during connection to MSP-v4

@ -183,3 +183,49 @@ Apr 27 13:02:04 host sshd[29116]: Received disconnect from 1.2.3.4: 11: Normal S
# Match sshd auth errors on OpenSUSE systems
# failJSON: { "time": "2015-04-16T20:02:50", "match": true , "host": "222.186.21.217", "desc": "Authentication for user failed" }
2015-04-16T18:02:50.321974+00:00 host sshd[2716]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=222.186.21.217 user=root
# filterOptions: {"mode": "ddos"}
# http://forums.powervps.com/showthread.php?t=1667
# failJSON: { "time": "2005-06-07T01:10:56", "match": true , "host": "69.61.56.114" }
Jun 7 01:10:56 host sshd[5937]: Did not receive identification string from 69.61.56.114
# gh-864(1):
# failJSON: { "match": false }
Nov 24 23:46:39 host sshd[32686]: SSH: Server;Ltype: Version;Remote: 127.0.0.1-1780;Protocol: 2.0;Client: libssh2_1.4.3
# failJSON: { "time": "2004-11-24T23:46:43", "match": true , "host": "127.0.0.1", "desc": "Multiline for connection reset by peer (1)" }
Nov 24 23:46:43 host sshd[32686]: fatal: Read from socket failed: Connection reset by peer [preauth]
# gh-864(2):
# failJSON: { "match": false }
Nov 24 23:46:40 host sshd[32686]: SSH: Server;Ltype: Kex;Remote: 127.0.0.1-1780;Enc: aes128-ctr;MAC: hmac-sha1;Comp: none [preauth]
# failJSON: { "time": "2004-11-24T23:46:43", "match": true , "host": "127.0.0.1", "desc": "Multiline for connection reset by peer (2)" }
Nov 24 23:46:43 host sshd[32686]: fatal: Read from socket failed: Connection reset by peer [preauth]
# gh-864(3):
# failJSON: { "match": false }
Nov 24 23:46:41 host sshd[32686]: SSH: Server;Ltype: Authname;Remote: 127.0.0.1-1780;Name: root [preauth]
# failJSON: { "time": "2004-11-24T23:46:43", "match": true , "host": "127.0.0.1", "desc": "Multiline for connection reset by peer (3)" }
Nov 24 23:46:43 host sshd[32686]: fatal: Read from socket failed: Connection reset by peer [preauth]
# filterOptions: {"mode": "extra"}
# several other cases from gh-864:
# failJSON: { "time": "2004-11-25T01:34:12", "match": true , "host": "127.0.0.1", "desc": "No supported authentication methods" }
Nov 25 01:34:12 srv sshd[123]: Received disconnect from 127.0.0.1: 14: No supported authentication methods available [preauth]
# failJSON: { "time": "2004-11-25T01:35:13", "match": true , "host": "127.0.0.1", "desc": "No supported authentication methods" }
Nov 25 01:35:13 srv sshd[123]: error: Received disconnect from 127.0.0.1: 14: No supported authentication methods available [preauth]
# failJSON: { "time": "2004-11-25T01:35:14", "match": true , "host": "192.168.2.92", "desc": "Optional space after port" }
Nov 25 01:35:14 srv sshd[3625]: error: Received disconnect from 192.168.2.92 port 1684:14: No supported authentication methods available [preauth]
# gh-1545:
# failJSON: { "time": "2004-11-26T13:03:29", "match": true , "host": "192.0.2.1", "desc": "No matching cipher" }
Nov 26 13:03:29 srv sshd[45]: Unable to negotiate with 192.0.2.1 port 55419: no matching cipher found. Their offer: aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,arcfour128,arcfour,3des-cbc,none [preauth]
# gh-1117:
# failJSON: { "time": "2004-11-26T13:03:30", "match": true , "host": "192.0.2.2", "desc": "No matching key exchange method" }
Nov 26 13:03:30 srv sshd[45]: fatal: Unable to negotiate with 192.0.2.2 port 55419: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1
# failJSON: { "match": false }
Nov 26 15:03:30 host sshd[22440]: Connection from 192.0.2.3 port 39678 on 192.168.1.9 port 22
# failJSON: { "time": "2004-11-26T15:03:31", "match": true , "host": "192.0.2.3", "desc": "Multiline - no matching key exchange method" }
Nov 26 15:03:31 host sshd[22440]: fatal: Unable to negotiate a key exchange method [preauth]

@ -1,3 +0,0 @@
# sshd-aggressive includes sshd and sshd-ddos failregex's:
# addFILE: "sshd"
# addFILE: "sshd-ddos"

@ -1,41 +0,0 @@
# http://forums.powervps.com/showthread.php?t=1667
# failJSON: { "time": "2005-06-07T01:10:56", "match": true , "host": "69.61.56.114" }
Jun 7 01:10:56 host sshd[5937]: Did not receive identification string from 69.61.56.114
# gh-864(1):
# failJSON: { "match": false }
Nov 24 23:46:39 host sshd[32686]: SSH: Server;Ltype: Version;Remote: 127.0.0.1-1780;Protocol: 2.0;Client: libssh2_1.4.3
# failJSON: { "time": "2004-11-24T23:46:43", "match": true , "host": "127.0.0.1", "desc": "Multiline for connection reset by peer (1)" }
Nov 24 23:46:43 host sshd[32686]: fatal: Read from socket failed: Connection reset by peer [preauth]
# gh-864(2):
# failJSON: { "match": false }
Nov 24 23:46:40 host sshd[32686]: SSH: Server;Ltype: Kex;Remote: 127.0.0.1-1780;Enc: aes128-ctr;MAC: hmac-sha1;Comp: none [preauth]
# failJSON: { "time": "2004-11-24T23:46:43", "match": true , "host": "127.0.0.1", "desc": "Multiline for connection reset by peer (2)" }
Nov 24 23:46:43 host sshd[32686]: fatal: Read from socket failed: Connection reset by peer [preauth]
# gh-864(3):
# failJSON: { "match": false }
Nov 24 23:46:41 host sshd[32686]: SSH: Server;Ltype: Authname;Remote: 127.0.0.1-1780;Name: root [preauth]
# failJSON: { "time": "2004-11-24T23:46:43", "match": true , "host": "127.0.0.1", "desc": "Multiline for connection reset by peer (3)" }
Nov 24 23:46:43 host sshd[32686]: fatal: Read from socket failed: Connection reset by peer [preauth]
# several other cases from gh-864:
# failJSON: { "time": "2004-11-25T01:34:12", "match": true , "host": "127.0.0.1", "desc": "No supported authentication methods" }
Nov 25 01:34:12 srv sshd[123]: Received disconnect from 127.0.0.1: 14: No supported authentication methods available [preauth]
# failJSON: { "time": "2004-11-25T01:35:13", "match": true , "host": "127.0.0.1", "desc": "No supported authentication methods" }
Nov 25 01:35:13 srv sshd[123]: error: Received disconnect from 127.0.0.1: 14: No supported authentication methods available [preauth]
# failJSON: { "time": "2004-11-25T01:35:14", "match": true , "host": "192.168.2.92", "desc": "Optional space after port" }
Nov 25 01:35:14 srv sshd[3625]: error: Received disconnect from 192.168.2.92 port 1684:14: No supported authentication methods available [preauth]
# gh-1545:
# failJSON: { "time": "2004-11-26T13:03:29", "match": true , "host": "192.0.2.1", "desc": "No matching cipher" }
Nov 26 13:03:29 srv sshd[45]: Unable to negotiate with 192.0.2.1 port 55419: no matching cipher found. Their offer: aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,arcfour128,arcfour,3des-cbc,none [preauth]
# gh-1117:
# failJSON: { "time": "2004-11-26T13:03:30", "match": true , "host": "192.0.2.2", "desc": "No matching key exchange method" }
Nov 26 13:03:30 srv sshd[45]: fatal: Unable to negotiate with 192.0.2.2 port 55419: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1
# failJSON: { "match": false }
Nov 26 15:03:30 host sshd[22440]: Connection from 192.0.2.3 port 39678 on 192.168.1.9 port 22
# failJSON: { "time": "2004-11-26T15:03:31", "match": true , "host": "192.0.2.3", "desc": "Multiline - no matching key exchange method" }
Nov 26 15:03:31 host sshd[22440]: fatal: Unable to negotiate a key exchange method [preauth]

@ -49,12 +49,7 @@ class FilterSamplesRegex(unittest.TestCase):
def setUp(self):
"""Call before every test case."""
super(FilterSamplesRegex, self).setUp()
self.filter = Filter(None)
self.filter.returnRawHost = True
self.filter.checkAllRegex = True
self.filter.checkFindTime = False
self.filter.active = True
self.filter = None
setUpMyTime()
def tearDown(self):
@ -83,11 +78,15 @@ class FilterSamplesRegex(unittest.TestCase):
RE_WRONG_GREED.search('non-greedy .+? test' + RE_HOST + ' test vary catch-all .* anchored$'))
def testSampleRegexsFactory(name, basedir):
def testFilter(self):
def _readFilter(self, name, basedir, opts=None):
self.filter = Filter(None)
self.filter.returnRawHost = True
self.filter.checkAllRegex = True
self.filter.checkFindTime = False
self.filter.active = True
if opts is None: opts = dict()
# Check filter exists
filterConf = FilterReader(name, "jail", {},
filterConf = FilterReader(name, "jail", opts,
basedir=basedir, share_config=unittest.F2B.share_config)
self.assertEqual(filterConf.getFile(), name)
self.assertEqual(filterConf.getJailName(), "jail")
@ -113,6 +112,17 @@ def testSampleRegexsFactory(name, basedir):
elif opt[2] == "datepattern":
self.filter.setDatePattern(optval)
# test regexp contains greedy catch-all before <HOST>, that is
# not hard-anchored at end or has not precise sub expression after <HOST>:
for fr in self.filter.getFailRegex():
if RE_WRONG_GREED.search(fr): # pragma: no cover
raise AssertionError("Following regexp of \"%s\" contains greedy catch-all before <HOST>, "
"that is not hard-anchored at end or has not precise sub expression after <HOST>:\n%s" %
(name, str(fr).replace(RE_HOST, '<HOST>')))
def testSampleRegexsFactory(name, basedir):
def testFilter(self):
self.assertTrue(
os.path.isfile(os.path.join(TEST_FILES_DIR, "logs", name)),
"No sample log file available for '%s' filter" % name)
@ -125,22 +135,21 @@ def testSampleRegexsFactory(name, basedir):
logFile = fileinput.FileInput(os.path.join(TEST_FILES_DIR, "logs",
filename))
# test regexp contains greedy catch-all before <HOST>, that is
# not hard-anchored at end or has not precise sub expression after <HOST>:
for fr in self.filter.getFailRegex():
if RE_WRONG_GREED.search(fr): # pragma: no cover
raise AssertionError("Following regexp of \"%s\" contains greedy catch-all before <HOST>, "
"that is not hard-anchored at end or has not precise sub expression after <HOST>:\n%s" %
(name, str(fr).replace(RE_HOST, '<HOST>')))
for line in logFile:
jsonREMatch = re.match("^# ?(failJSON|addFILE):(.+)$", line)
jsonREMatch = re.match("^#+ ?(failJSON|filterOptions|addFILE):(.+)$", line)
if jsonREMatch:
try:
faildata = json.loads(jsonREMatch.group(2))
# filterOptions - dict in JSON to control filter options (e. g. mode, etc.):
if jsonREMatch.group(1) == 'filterOptions':
self.filter = None
self._readFilter(name, basedir, opts=faildata)
continue
# addFILE - filename to "include" test-files should be additionally parsed:
if jsonREMatch.group(1) == 'addFILE':
filenames.append(faildata)
continue
# failJSON - faildata contains info of the failure to check it.
except ValueError as e:
raise ValueError("%s: %s:%i" %
(e, logFile.filename(), logFile.filelineno()))
@ -150,6 +159,9 @@ def testSampleRegexsFactory(name, basedir):
else:
faildata = {}
if self.filter is None:
self._readFilter(name, basedir, opts=None)
try:
ret = self.filter.processLine(line)
if not ret:

@ -27,6 +27,9 @@ a string representing a 'failregex'
.TP
filename
path to a filter file (filter.d/sshd.conf)
.TP
filtername[option=value, ..., option=value]
short path to a filter relative filter.d in configuration base (sshd[mode=aggressive])
.SS "IGNOREREGEX:"
.TP
string
@ -42,6 +45,9 @@ show program's version number and exit
\fB\-h\fR, \fB\-\-help\fR
show this help message and exit
.TP
\fB\-c\fR CONFIGBASE, \fB\-\-config\fR=\fI\,CONFIGBASE\/\fR
set alternate config base directory (default /etc/fail2ban)
.TP
\fB\-d\fR DATEPATTERN, \fB\-\-datepattern\fR=\fI\,DATEPATTERN\/\fR
set custom pattern used to match date/times
.TP

Loading…
Cancel
Save