Merge branch '0.10' into 0.11

pull/1913/head
sebres 2017-09-08 11:33:19 +02:00
commit e0fede621e
16 changed files with 122 additions and 26 deletions

View File

@ -45,10 +45,25 @@ ver. 0.10.1-dev-1 (2017/??/??) - development current edition
-----------
### Fixes
* jail "pass2allow-ftp" supply blocktype and returntype parameters to the action (gh-1884)
* avoid using "ANSI_X3.4-1968" as preferred encoding (if missing environment variables
'LANGUAGE', 'LC_ALL', 'LC_CTYPE', and 'LANG', see gh-1587).
* action.d/pf.conf: several fixes for pf-action like anchoring, etc. (see gh-1866, gh-1867);
* fixed ignorself issue "Retrieving own IPs of localhost failed: inet_pton() argument 2 must be string, not int" (see gh-1865);
* fixed tags `<fq-hostname>` and `<sh-hostname>`, could be used without ticket (a. g. in `actionstart` etc., gh-1859).
* setup.py: fixed several setup facilities (gh-1874):
- don't check return code by dry-run: returns 256 on some python/setuptool versions;
- `files/fail2ban.service` renamed as template to `files/fail2ban.service.in`;
- setup process generates `build/fail2ban.service` from `files/fail2ban.service.in` using distribution related bin-path;
- bug-fixing by running setup with option `--dry-run`;
### New Features
* introduced new command-line options `--dp`, `--dump-pretty` to dump the configuration using more
human readable representation (opposite to `-d`);
### Enhancements
* filter.d/dovecot.conf: introduced mode `aggressive` for cases like "disconnected before auth was ready" (gh-1880)
ver. 0.10.0 (2017/08/09) - long-awaited 0.10th version
@ -411,13 +426,18 @@ releases.
* filter.d/apache-overflows.conf:
- Fixes resources greedy expression (see gh-1790);
- Rewritten without end-anchor ($), because of potential vulnerability on very long URLs.
* filter.d/apache-badbots.conf - extended to recognize Jorgee Vulnerability Scanner (gh-1882)
* filter.d/asterisk.conf - fixed failregex AMI Asterisk authentification failed (see gh-1302)
* filter.d/dovecot.conf - fixed failregex, see gh-1879 (partially cherry-picked from gh-1880)
* filter.d/exim.conf - fixed failregex for case of flood attempts with `D=0s` (gh-1887)
### New Features
### Enhancements
* action.d/cloudflare.conf - Cloudflare API v4 implementation (gh-1651)
* filter.d/kerio.conf - filter extended with new rules (see gh-1455)
* filter.d/phpmyadmin-syslog.conf - new filter for phpMyAdmin using syslog for auth logging
* filter.d/zoneminder.conf - new filter for ZoneMinder (gh-1376)
ver. 0.9.7 (2017/05/11) - awaiting-victory

View File

@ -7,7 +7,7 @@
[Definition]
badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider
badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider|(?:Mozilla/\d+\.\d+ )?Jorgee
badbots = Atomic_Email_Hunter/4\.0|atSpider/1\.0|autoemailspider|bwh3_user_agent|China Local Browse 2\.6|ContactBot/0\.2|ContentSmartz|DataCha0s/2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailSpider|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Franklin Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Guestbook Auto Submitter|Industry Program 1\.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsCrawl\.com/1\.0 \+http\://letscrawl\.com/|Lincoln State Web Browser|LMQueueBot/0\.2|LWP\:\:Simple/5\.803|Mac Finder 1\.0\.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control - 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missigua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|MVAClient|Mozilla/2\.0 \(compatible; NEWT ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/3\.0 \(compatible; scan4mail \(advanced version\) http\://www\.peterspages\.net/?scan4mail\)|Mozilla/4\.0 \(compatible; Advanced Email Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0 \(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Version\: xxxx Type\:xx\)|NameOfAgent \(CMS Spider\)|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google\.com|ShablastBot 1\.0|snap\.com beta crawler v0|Snapbot/1\.0|Snapbot/1\.0 \(Snap Shots&#44; \+http\://www\.snap\.com\)|sogou develop spider|Sogou Orion spider/3\.0\(\+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sogou spider|Sogou web spider/3\.0\(\+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|VadixBot|WebVulnCrawl\.unknown/1\.0 libwww-perl/5\.803|Wells Search II|WEP Search 00
failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*"(?:%(badbots)s|%(badbotscustom)s)"$

View File

@ -22,7 +22,7 @@ failregex = ^%(pid)s %(host_info)ssender verify fail for <\S+>: (?:Unknown user|
^%(pid)s SMTP protocol synchronization error \([^)]*\): rejected (?:connection from|"\S+") %(host_info)s(?:next )?input=".*"\s*$
^%(pid)s SMTP call from \S+ %(host_info)sdropped: too many nonmail commands \(last was "\S+"\)\s*$
^%(pid)s SMTP protocol error in "AUTH \S*(?: \S*)?" %(host_info)sAUTH command used when not advertised\s*$
^%(pid)s no MAIL in SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sD=\d\S+s(?: C=\S*)?\s*$
^%(pid)s no MAIL in SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sD=\d\S*s(?: C=\S*)?\s*$
^%(pid)s (?:[\w\-]+ )?SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sclosed by DROP in ACL\s*$
ignoreregex =

View File

@ -0,0 +1,18 @@
# Fail2Ban fitler for the phpMyAdmin-syslog
#
[INCLUDES]
before = common.conf
[Definition]
_daemon = phpMyAdmin
failregex = ^%(__prefix_line)suser denied: (?:\S+|.*?) \(mysql-denied\) from <HOST>\s*$
ignoreregex =
# Author: Pavel Mihadyuk
# Regex fixes: Serg G. Brester

View File

@ -0,0 +1,21 @@
# Fail2Ban filter for Zoneminder login failures
[INCLUDES]
before = apache-common.conf
[Definition]
# pattern: [Wed Apr 27 23:12:07.736196 2016] [:error] [pid 2460] [client 10.1.1.1:47296] WAR [Login denied for user "test"], referer: https://zoneminderurl/index.php
#
#
# Option: failregex
# Notes.: regex to match the password failure messages in the logfile.
failregex = ^%(_apache_error_client)s WAR \[Login denied for user "[^"]*"\]
ignoreregex =
# Notes:
# Tested on Zoneminder 1.29.0
#
# Author: John Marzella

View File

@ -877,6 +877,7 @@ filter = apache-pass[knocking_url="%(knocking_url)s"]
logpath = %(apache_access_log)s
blocktype = RETURN
returntype = DROP
action = %(action_)s[blocktype=%(blocktype)s, returntype=%(returntype)s]
bantime = 1h
maxretry = 1
findtime = 1
@ -911,3 +912,17 @@ logpath = /var/log/slapd.log
port = smtp,ssmtp
filter = domino-smtp
logpath = /home/domino01/data/IBM_TECHNICAL_SUPPORT/console.log
[phpmyadmin-syslog]
port = http,https
filter = phpmyadmin-syslog
logpath = %(syslog_authpriv)s
backend = %(syslog_backend)s
[zoneminder]
# Zoneminder HTTP/HTTPS web interface auth
# Logs auth failures to apache2 error log
port = http,https
logpath = %(apache_error_log)s

View File

@ -89,11 +89,11 @@ class ActionReader(DefinitionInitConfigReader):
stream.append(head + ["addaction", self._name])
multi = []
for opt, optval in opts.iteritems():
if opt in self._configOpts:
if opt in self._configOpts and not opt.startswith('known/'):
multi.append([opt, optval])
if self._initOpts:
for opt, optval in self._initOpts.iteritems():
if opt not in self._configOpts:
if opt not in self._configOpts and not opt.startswith('known/'):
multi.append([opt, optval])
if len(multi) > 1:
stream.append(["multi-set", self._jailName, "action", self._name, multi])

View File

@ -20,8 +20,8 @@
# Author: Yaroslav Halchenko
# Modified: Cyril Jaquier
__author__ = 'Yaroslav Halhenko'
__copyright__ = 'Copyright (c) 2007 Yaroslav Halchenko'
__author__ = 'Yaroslav Halhenko, Serg G. Brester (aka sebres)'
__copyright__ = 'Copyright (c) 2007 Yaroslav Halchenko, 2015 Serg G. Brester (aka sebres)'
__license__ = 'GPL'
import os
@ -150,7 +150,7 @@ after = 1.conf
if seclwr == 'known':
# try get raw value from known options:
try:
v = self._sections['KNOWN'][opt]
v = self._sections['KNOWN/'+section][opt]
except KeyError:
# fallback to default:
try:
@ -297,7 +297,6 @@ after = 1.conf
# merge defaults and all sections to self:
alld.update(cfg.get_defaults())
for n, s in cfg.get_sections().iteritems():
curalls = alls
# conditional sections
cond = SafeConfigParserWithIncludes.CONDITIONAL_RE.match(n)
if cond:
@ -313,7 +312,7 @@ after = 1.conf
s2 = alls.get(n)
if isinstance(s2, dict):
# save previous known values, for possible using in local interpolations later:
self.merge_section('KNOWN', s2, '')
self.merge_section('KNOWN/'+n, s2, '')
# merge section
s2.update(s)
else:

View File

@ -20,8 +20,8 @@
# Author: Cyril Jaquier
# Modified by: Yaroslav Halchenko (SafeConfigParserWithIncludes)
__author__ = "Cyril Jaquier"
__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
__author__ = "Cyril Jaquier, Yaroslav Halchenko, Serg G. Brester (aka sebres)"
__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2007 Yaroslav Halchenko, 2015 Serg G. Brester (aka sebres)"
__license__ = "GPL"
import glob
@ -110,7 +110,7 @@ class ConfigReader():
def sections(self):
try:
return (n for n in self._cfg.sections() if n != 'KNOWN')
return (n for n in self._cfg.sections() if not n.startswith('KNOWN/'))
except AttributeError:
return []

View File

@ -76,9 +76,9 @@ class Configurator:
self.__fail2ban.getOptions(updateMainOpt)
return self.__jails.getOptions(jail, ignoreWrong=ignoreWrong)
def convertToProtocol(self):
def convertToProtocol(self, allow_no_files=False):
self.__streams["general"] = self.__fail2ban.convert()
self.__streams["jails"] = self.__jails.convert()
self.__streams["jails"] = self.__jails.convert(allow_no_files=allow_no_files)
def getConfigStream(self):
cmds = list()

View File

@ -102,6 +102,7 @@ class Fail2banCmdLine():
output(" --logtarget <FILE>|STDOUT|STDERR|SYSLOG")
output(" --syslogsocket auto|<FILE>")
output(" -d dump configuration. For debugging")
output(" --dp, --dump-pretty dump the configuration using more human readable representation")
output(" -t, --test test configuration (can be also specified with start parameters)")
output(" -i interactive mode")
output(" -v increase verbosity")
@ -137,8 +138,8 @@ class Fail2banCmdLine():
self._conf["pidfile"] = opt[1]
elif o.startswith("--log") or o.startswith("--sys"):
self._conf[ o[2:] ] = opt[1]
elif o == "-d":
self._conf["dump"] = True
elif o in ["-d", "--dp", "--dump-pretty"]:
self._conf["dump"] = True if o == "-d" else 2
elif o == "-t" or o == "--test":
self.cleanConfOnly = True
self._conf["test"] = True
@ -184,7 +185,8 @@ class Fail2banCmdLine():
# Reads the command line options.
try:
cmdOpts = 'hc:s:p:xfbdtviqV'
cmdLongOpts = ['loglevel=', 'logtarget=', 'syslogsocket=', 'test', 'async', 'timeout=', 'str2sec=', 'help', 'version']
cmdLongOpts = ['loglevel=', 'logtarget=', 'syslogsocket=', 'test', 'async',
'timeout=', 'str2sec=', 'help', 'version', 'dp', '--dump-pretty']
optList, self._args = getopt.getopt(self._argv[1:], cmdOpts, cmdLongOpts)
except getopt.GetoptError:
self.dispUsage()
@ -240,7 +242,10 @@ class Fail2banCmdLine():
if readcfg:
ret, stream = self.readConfig()
readcfg = False
self.dumpConfig(stream)
if stream is not None:
self.dumpConfig(stream, self._conf["dump"] == 2)
else: # pragma: no cover
output("ERROR: The configuration stream failed because of the invalid syntax.")
if not self._conf.get("test", False):
return ret
@ -275,7 +280,8 @@ class Fail2banCmdLine():
self.configurator.readAll()
ret = self.configurator.getOptions(jail, self._conf,
ignoreWrong=not self.cleanConfOnly)
self.configurator.convertToProtocol()
self.configurator.convertToProtocol(
allow_no_files=self._conf.get("dump", False))
stream = self.configurator.getConfigStream()
except Exception as e:
logSys.error("Failed during configuration: %s" % e)
@ -283,9 +289,15 @@ class Fail2banCmdLine():
return ret, stream
@staticmethod
def dumpConfig(cmd):
def dumpConfig(cmd, pretty=False):
if pretty:
from pprint import pformat
def _output(s):
output(pformat(s, width=1000, indent=2))
else:
_output = output
for c in cmd:
output(c)
_output(c)
return True
#

View File

@ -242,9 +242,12 @@ class JailReader(ConfigReader):
found_files += 1
stream.append(
["set", self.__name, "addlogpath", p, tail])
if not (found_files or allow_no_files):
raise ValueError(
"Have not found any log file for %s jail" % self.__name)
if not found_files:
msg = "Have not found any log file for %s jail" % self.__name
if not allow_no_files:
raise ValueError(msg)
logSys.warning(msg)
elif opt == "logencoding":
stream.append(["set", self.__name, "logencoding", value])
elif opt == "backend":

View File

@ -458,7 +458,11 @@ class Fail2banClientTest(Fail2banClientServerBase):
startparams = _start_params(tmp, True)
self.execSuccess(startparams, "-vvd")
self.assertLogged("Loading files")
self.assertLogged("logtarget")
self.assertLogged("['set', 'logtarget',")
self.pruneLog()
# pretty dump:
self.execSuccess(startparams, "--dp")
self.assertLogged("['set', 'logtarget',")
@with_tmpdir
@with_kill_srv

View File

@ -53,7 +53,7 @@
# failJSON: { "time": "2016-03-21T06:38:05", "match": true , "host": "49.212.207.15" }
2016-03-21 06:38:05 [5718] no MAIL in SMTP connection from www3005.sakura.ne.jp [49.212.207.15]:28890 I=[172.89.0.6]:25 D=21s C=EHLO,STARTTLS
# failJSON: { "time": "2016-03-21T06:57:36", "match": true , "host": "122.165.71.116" }
2016-03-21 06:57:36 [5908] no MAIL in SMTP connection from [122.165.71.116]:2056 I=[172.89.0.6]:25 D=10s
2016-03-21 06:57:36 [5908] no MAIL in SMTP connection from [122.165.71.116]:2056 I=[172.89.0.6]:25 D=0s
# failJSON: { "time": "2016-03-21T06:57:36", "match": true , "host": "122.165.71.116" }
2016-03-21 06:57:36 [5908] no MAIL in SMTP connection from [122.165.71.116] I=[172.89.0.6]:25 D=10s
# failJSON: { "time": "2016-03-21T04:07:49", "match": true , "host": "174.137.147.204" }

View File

@ -0,0 +1,2 @@
# failJSON: { "time": "2004-08-22T14:50:22", "match": true , "host": "192.0.2.1" }
Aug 22 14:50:22 eurostream phpMyAdmin[16358]: user denied: root (mysql-denied) from 192.0.2.1

View File

@ -0,0 +1,2 @@
# failJSON: { "time": "2016-03-28T16:50:49", "match": true , "host": "10.1.1.1" }
[Mon Mar 28 16:50:49.522240 2016] [:error] [pid 1795] [client 10.1.1.1:50700] WAR [Login denied for user "username1"], referer: https://zoneminder/