mirror of https://github.com/fail2ban/fail2ban
Merge branch 'fail2ban:master' into feature-abuseipdb-integration
Hasan ÇALIŞIR
GitHub
commit
4a1e854080
|
|
@ -1,12 +1,10 @@
|
|||
Before submitting your PR, please review the following checklist:
|
||||
|
||||
- [ ] **CHOOSE CORRECT BRANCH**: if filing a bugfix/enhancement
|
||||
against certain release version, choose `0.9`, `0.10` or `0.11` branch,
|
||||
for dev-edition use `master` branch
|
||||
- [ ] **CONSIDER adding a unit test** if your PR resolves an issue
|
||||
- [ ] **LIST ISSUES** this PR resolves
|
||||
- [ ] **LIST ISSUES** this PR resolves or describe the approach in detail
|
||||
- [ ] **MAKE SURE** this PR doesn't break existing tests
|
||||
- [ ] **KEEP PR small** so it could be easily reviewed.
|
||||
- [ ] **KEEP PR small** so it could be easily reviewed
|
||||
- [ ] **AVOID** making unnecessary stylistic changes in unrelated code
|
||||
- [ ] **ACCOMPANY** each new `failregex` for filter `X` with sample log lines
|
||||
within `fail2ban/tests/files/logs/X` file
|
||||
(and `# failJSON`) within `fail2ban/tests/files/logs/X` file
|
||||
- [ ] **PROVIDE ChangeLog** entry describing the pull request
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ ver. 1.1.1-dev-1 (20??/??/??) - development nightly edition
|
|||
* `filter.d/exim.conf` - mode `aggressive` extended to catch dropped by ACL failures, e.g. "ACL: Country is banned"
|
||||
* `filter.d/freeswitch.conf` - bypass some new info in prefix before [WARNING] (changed default `_pref_line`),
|
||||
FreeSWITCH log line prefix has changed in newer versions (gh-3143)
|
||||
* `filter.d/lighttpd-auth.conf` - fixed regex (if failures generated by systemd-journal), bypass several prefixes now (gh-3955)
|
||||
* `filter.d/postfix.conf` - consider CONNECT and other rejected commands as a valid `_pref` (gh-3800)
|
||||
* `filter.d/dropbear.conf`:
|
||||
- recognizes extra pid/timestamp if logged into stdout/journal, added `journalmatch` (gh-3597)
|
||||
|
|
@ -36,10 +37,16 @@ ver. 1.1.1-dev-1 (20??/??/??) - development nightly edition
|
|||
- adapted to conform possible new daemon name sshd-session, since OpenSSH 9.8
|
||||
several log messages will be tagged with as originating from a process named "sshd-session" rather than "sshd" (gh-3782)
|
||||
- `ddos` and `aggressive` modes: regex extended for timeout before authentication (optional connection from part, gh-3907)
|
||||
* `filter.d/vsftpd.conf` - fixed regex (if failures generated by systemd-journal, gh-3954)
|
||||
|
||||
### New Features and Enhancements
|
||||
* new jail option `skip_if_nologs` to ignore jail if no `logpath` matches found, fail2ban continue to start with warnings/errors,
|
||||
thus other jails become running (gh-2756)
|
||||
* configuration `ignoreip` and fail2ban-client commands `addignoreip`/`delignoreip` extended with `file:...` syntax
|
||||
to ignore IPs from file-ip-set (containing IP, subnet, dns/fqdn or raw strings); the file would be read lazy on demand,
|
||||
by first ban (and automatically reloaded by update after small latency to avoid expensive stats check on every compare);
|
||||
the entries inside the file can be separated by comma, space or new line with optional comments (text following chars
|
||||
`#` or `;` after space or newline would be ignored up to next newline)
|
||||
* `action.d/*-ipset.conf`:
|
||||
- parameter `ipsettype` to set type of ipset, e. g. hash:ip, hash:net, etc (gh-3760)
|
||||
* `action.d/firewallcmd-rich-*.conf` - fixed incorrect quoting, disabling port variable expansion
|
||||
|
|
|
|||
|
|
@ -3,8 +3,8 @@
|
|||
|
||||
[Definition]
|
||||
|
||||
failregex = ^\s*(?:: )?\(?(?:http|mod)_auth\.c\.\d+\) (?:password doesn\'t match for (?:\S+|.*?) username:\s+<F-USER>(?:\S+|.*?)</F-USER>\s*|digest: auth failed(?: for\s+<F-ALT_USER>(?:\S+|.*?)</F-ALT_USER>\s*)?: (?:wrong password|uri mismatch \([^\)]*\))|get_password failed),? IP: <HOST>\s*$
|
||||
failregex = ^[^\)]*\(?(?:http|mod)_auth\.c\.\d+\) (?:password doesn\'t match for (?:\S+|.*?) username:\s+<F-USER>(?:\S+|.*?)</F-USER>\s*|digest: auth failed(?: for\s+<F-ALT_USER>(?:\S+|.*?)</F-ALT_USER>\s*)?: (?:wrong password|uri mismatch \([^\)]*\))|get_password failed),? IP: <HOST>\s*$
|
||||
|
||||
ignoreregex =
|
||||
ignoreregex =
|
||||
|
||||
# Author: Francois Boulogne <fboulogne@april.org>
|
||||
# Authors: Francois Boulogne <fboulogne@april.org>, Lucian Maly <lmaly@redhat.com>
|
||||
|
|
|
|||
|
|
@ -10,13 +10,13 @@ before = common.conf
|
|||
|
||||
[Definition]
|
||||
|
||||
__pam_re=\(?%(__pam_auth)s(?:\(\S+\))?\)?:?
|
||||
__pam_re=(?:\(?%(__pam_auth)s(?:\(\S+\))?\)?:?\s+)?
|
||||
_daemon = vsftpd
|
||||
|
||||
failregex = ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=(ftp)? ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$
|
||||
^ \[pid \d+\] \[[^\]]+\] FAIL LOGIN: Client "<HOST>"(?:\s*$|,)
|
||||
failregex = ^%(__prefix_line)s%(__pam_re)sauthentication failure; logname=<F-ALT_USER1>\S*</F-ALT_USER1> uid=\S* euid=\S* tty=(?:ftp)? ruser=<F-USER>\S*</F-USER> rhost=<HOST>(?:\s+user=<F-ALT_USER>\S*</F-ALT_USER>)?\s*$
|
||||
^(?:\s*\[pid \d+\] |%(__prefix_line)s)\[<F-USER>[^\]]+</F-USER>\] FAIL LOGIN: Client "<HOST>"(?:\s*$|,)
|
||||
|
||||
ignoreregex =
|
||||
ignoreregex =
|
||||
|
||||
# Author: Cyril Jaquier
|
||||
# Authors: Cyril Jaquier, Lucian Maly <lmaly@redhat.com>
|
||||
# Documentation from fail2ban wiki
|
||||
|
|
|
|||
|
|
@ -282,7 +282,18 @@ def excepthook(exctype, value, traceback):
|
|||
"Unhandled exception in Fail2Ban:", exc_info=True)
|
||||
return sys.__excepthook__(exctype, value, traceback)
|
||||
|
||||
def splitwords(s):
|
||||
RE_REM_COMMENTS = re.compile(r'(?m)(?:^|\s)[\#;].*')
|
||||
def removeComments(s):
|
||||
"""Helper to remove comments:
|
||||
# comment ...
|
||||
; comment ...
|
||||
no comment # comment ...
|
||||
no comment ; comment ...
|
||||
"""
|
||||
return RE_REM_COMMENTS.sub('', s)
|
||||
|
||||
RE_SPLT_WORDS = re.compile(r'[\s,]+')
|
||||
def splitwords(s, ignoreComments=False):
|
||||
"""Helper to split words on any comma, space, or a new line
|
||||
|
||||
Returns empty list if input is empty (or None) and filters
|
||||
|
|
@ -290,7 +301,9 @@ def splitwords(s):
|
|||
"""
|
||||
if not s:
|
||||
return []
|
||||
return list(filter(bool, [v.strip() for v in re.split(r'[\s,]+', s)]))
|
||||
if ignoreComments:
|
||||
s = removeComments(s)
|
||||
return list(filter(bool, [v.strip() for v in RE_SPLT_WORDS.split(s)]))
|
||||
|
||||
def _merge_dicts(x, y):
|
||||
"""Helper to merge dicts.
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ import time
|
|||
|
||||
from .actions import Actions
|
||||
from .failmanager import FailManagerEmpty, FailManager
|
||||
from .ipdns import DNSUtils, IPAddr
|
||||
from .ipdns import DNSUtils, IPAddr, FileIPAddrSet
|
||||
from .observer import Observers
|
||||
from .ticket import FailTicket
|
||||
from .jailthread import JailThread
|
||||
|
|
@ -510,6 +510,12 @@ class Filter(JailThread):
|
|||
# An empty string is always false
|
||||
if ipstr == "":
|
||||
return
|
||||
# File?
|
||||
ip = FileIPAddrSet.RE_FILE_IGN_IP.match(ipstr)
|
||||
if ip:
|
||||
ip = DNSUtils.getIPsFromFile(ip.group(1)) # FileIPAddrSet
|
||||
self.__ignoreIpList.append(ip)
|
||||
return
|
||||
# Create IP address object
|
||||
ip = IPAddr(ipstr)
|
||||
# Avoid exact duplicates
|
||||
|
|
@ -532,6 +538,11 @@ class Filter(JailThread):
|
|||
return
|
||||
# delete by ip:
|
||||
logSys.debug(" Remove %r from ignore list", ip)
|
||||
# File?
|
||||
if FileIPAddrSet.RE_FILE_IGN_IP.match(ip):
|
||||
self.__ignoreIpList.remove(ip)
|
||||
return
|
||||
# IP / DNS
|
||||
if ip in self.__ignoreIpSet:
|
||||
self.__ignoreIpSet.remove(ip)
|
||||
else:
|
||||
|
|
@ -588,7 +599,7 @@ class Filter(JailThread):
|
|||
return True
|
||||
for net in self.__ignoreIpList:
|
||||
if ip.isInNet(net):
|
||||
self.logIgnoreIp(ip, log_ignore, ignore_source=("ip" if net.isValid else "dns"))
|
||||
self.logIgnoreIp(ip, log_ignore, ignore_source=(net.instanceType))
|
||||
if self.__ignoreCache: c.set(key, True)
|
||||
return True
|
||||
|
||||
|
|
|
|||
|
|
@ -23,10 +23,11 @@ __license__ = "GPL"
|
|||
|
||||
import socket
|
||||
import struct
|
||||
import os
|
||||
import re
|
||||
|
||||
from .utils import Utils
|
||||
from ..helpers import getLogger
|
||||
from ..helpers import getLogger, MyTime, splitwords
|
||||
|
||||
# Gets the instance of the logger.
|
||||
logSys = getLogger(__name__)
|
||||
|
|
@ -79,6 +80,8 @@ class DNSUtils:
|
|||
# todo: make configurable the expired time and max count of cache entries:
|
||||
CACHE_nameToIp = Utils.Cache(maxCount=1000, maxTime=5*60)
|
||||
CACHE_ipToName = Utils.Cache(maxCount=1000, maxTime=5*60)
|
||||
# static cache used to hold sets read from files:
|
||||
CACHE_fileToIp = Utils.Cache(maxCount=100, maxTime=5*60)
|
||||
|
||||
@staticmethod
|
||||
def dnsToIp(dns):
|
||||
|
|
@ -229,6 +232,20 @@ class DNSUtils:
|
|||
DNSUtils.CACHE_nameToIp.set(DNSUtils._getSelfIPs_key, ips)
|
||||
return ips
|
||||
|
||||
@staticmethod
|
||||
def getIPsFromFile(fileName, noError=True):
|
||||
"""Get set of IP addresses or subnets from file"""
|
||||
# to find cached IPs:
|
||||
ips = DNSUtils.CACHE_fileToIp.get(fileName)
|
||||
if ips is not None:
|
||||
return ips
|
||||
# try to obtain set from file:
|
||||
ips = FileIPAddrSet(fileName)
|
||||
#ips.load() - load on demand
|
||||
# cache and return :
|
||||
DNSUtils.CACHE_fileToIp.set(fileName, ips)
|
||||
return ips
|
||||
|
||||
_IPv6IsAllowed = None
|
||||
|
||||
@staticmethod
|
||||
|
|
@ -457,6 +474,10 @@ class IPAddr(object):
|
|||
def familyStr(self):
|
||||
return IPAddr.FAM2STR.get(self._family)
|
||||
|
||||
@property
|
||||
def instanceType(self):
|
||||
return "ip" if self.isValid else "dns"
|
||||
|
||||
@property
|
||||
def plen(self):
|
||||
return self._plen
|
||||
|
|
@ -598,6 +619,9 @@ class IPAddr(object):
|
|||
def isInNet(self, net):
|
||||
"""Return either the IP object is in the provided network
|
||||
"""
|
||||
# if addr-set:
|
||||
if isinstance(net, IPAddrSet):
|
||||
return self in net
|
||||
# if it isn't a valid IP address, try DNS resolution
|
||||
if not net.isValid and net.raw != "":
|
||||
# Check if IP in DNS
|
||||
|
|
@ -675,15 +699,32 @@ IPAddr.IP6_4COMPAT = IPAddr("::ffff:0:0", 96)
|
|||
|
||||
class IPAddrSet(set):
|
||||
|
||||
hasSubNet = False
|
||||
hasSubNet = 0
|
||||
|
||||
def __init__(self, ips=[]):
|
||||
ips, subnet = IPAddrSet._list2set(ips)
|
||||
set.__init__(self, ips)
|
||||
self.hasSubNet = subnet
|
||||
|
||||
@staticmethod
|
||||
def _list2set(ips):
|
||||
ips2 = set()
|
||||
subnet = 0
|
||||
for ip in ips:
|
||||
if not isinstance(ip, IPAddr): ip = IPAddr(ip)
|
||||
ips2.add(ip)
|
||||
self.hasSubNet |= not ip.isSingle
|
||||
set.__init__(self, ips2)
|
||||
subnet += not ip.isSingle
|
||||
return ips2, subnet
|
||||
|
||||
@property
|
||||
def instanceType(self):
|
||||
return "ip-set"
|
||||
|
||||
def set(self, ips):
|
||||
ips, subnet = IPAddrSet._list2set(ips)
|
||||
self.clear()
|
||||
self.update(ips)
|
||||
self.hasSubNet = subnet
|
||||
|
||||
def add(self, ip):
|
||||
if not isinstance(ip, IPAddr): ip = IPAddr(ip)
|
||||
|
|
@ -696,6 +737,82 @@ class IPAddrSet(set):
|
|||
return set.__contains__(self, ip) or (self.hasSubNet and any(n.contains(ip) for n in self))
|
||||
|
||||
|
||||
class FileIPAddrSet(IPAddrSet):
|
||||
|
||||
# RE matching file://...
|
||||
RE_FILE_IGN_IP = re.compile(r'^file:/{0,2}(.*)$')
|
||||
|
||||
fileName = ''
|
||||
_shortRepr = None
|
||||
maxUpdateLatency = 1 # latency in seconds to update by changes
|
||||
_nextCheck = 0
|
||||
_fileStats = ()
|
||||
|
||||
def __init__(self, fileName=''):
|
||||
self.fileName = fileName
|
||||
# self.load() - lazy load on demand by first check (in, __contains__ etc)
|
||||
|
||||
@property
|
||||
def instanceType(self):
|
||||
return repr(self)
|
||||
|
||||
def __eq__(self, other):
|
||||
if id(self) == id(other): return 1
|
||||
# to allow remove file-set from list (delIgnoreIP) by its name:
|
||||
if isinstance(other, FileIPAddrSet):
|
||||
return self.fileName == other.fileName
|
||||
m = FileIPAddrSet.RE_FILE_IGN_IP.match(other)
|
||||
if m:
|
||||
return self.fileName == m.group(1)
|
||||
|
||||
def _isModified(self):
|
||||
"""Check whether the file is modified (file stats changed)
|
||||
|
||||
Side effect: if modified, _fileStats will be updated to last known stats of file
|
||||
"""
|
||||
tm = MyTime.time()
|
||||
# avoid to check it always (not often than maxUpdateLatency):
|
||||
if tm <= self._nextCheck:
|
||||
return None; # no check needed
|
||||
self._nextCheck = tm + self.maxUpdateLatency
|
||||
stats = os.stat(self.fileName)
|
||||
stats = stats.st_mtime, stats.st_ino, stats.st_size
|
||||
if self._fileStats != stats:
|
||||
self._fileStats = stats
|
||||
return True; # modified, needs to be reloaded
|
||||
return False; # unmodified
|
||||
|
||||
def load(self, forceReload=False, noError=True):
|
||||
"""Load set from file (on demand if needed or by forceReload)
|
||||
"""
|
||||
try:
|
||||
# load only if needed and modified (or first time load on demand)
|
||||
if self._isModified() or forceReload:
|
||||
with open(self.fileName, 'r') as f:
|
||||
ips = f.read()
|
||||
ips = splitwords(ips, ignoreComments=True)
|
||||
self.set(ips)
|
||||
except Exception as e: # pragma: no cover
|
||||
self._nextCheck += 60; # increase interval to check (to 1 minute, to avoid log flood on errors)
|
||||
if not noError: raise e
|
||||
logSys.warning("Retrieving IPs set from %r failed: %s", self.fileName, e)
|
||||
|
||||
def __repr__(self):
|
||||
if not self._shortRepr:
|
||||
shortfn = os.path.basename(self.fileName)
|
||||
if shortfn != self.fileName:
|
||||
shortfn = '.../' + shortfn
|
||||
self._shortRepr = 'file:' + shortfn + ')'
|
||||
return self._shortRepr
|
||||
|
||||
def __contains__(self, ip):
|
||||
# load if needed:
|
||||
if self.fileName:
|
||||
self.load()
|
||||
# inherited contains:
|
||||
return IPAddrSet.__contains__(self, ip)
|
||||
|
||||
|
||||
def _NetworkInterfacesAddrs(withMask=False):
|
||||
|
||||
# Closure implementing lazy load modules and libc and define _NetworkInterfacesAddrs on demand:
|
||||
|
|
|
|||
|
|
@ -12,3 +12,7 @@
|
|||
2021-09-30 17:44:37: (mod_auth.c.791) digest: auth failed for tester : wrong password, IP: 192.0.2.3
|
||||
# failJSON: { "time": "2021-09-30T17:44:37", "match": true , "host": "192.0.2.4", "desc": "gh-3116" }
|
||||
2021-09-30 17:44:37: (mod_auth.c.791) digest: auth failed: uri mismatch (/uri1 != /uri2), IP: 192.0.2.4
|
||||
|
||||
# systemd-journal
|
||||
# failJSON: { "time": "2025-03-04T02:11:57", "match": true , "host": "192.0.2.211", "desc": "gh-3955" }
|
||||
2025-03-04T02:11:57.602061 ip-172-31-3-150.ap-southeast-2.compute.internal lighttpd[764]: (mod_auth.c.853) password doesn't match for / username: user1 IP: 192.0.2.211
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@
|
|||
# failJSON: { "time": "2004-10-11T01:06:47", "match": true , "host": "209.67.1.67" }
|
||||
Oct 11 01:06:47 ServerJV vsftpd: (pam_unix) authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=209.67.1.67
|
||||
|
||||
# Pam pre 0.99.2.0 - https://github.com/fail2ban/fail2ban/pull/358
|
||||
# failJSON: { "time": "2005-02-06T12:02:29", "match": false , "host": "64.168.103.1" }
|
||||
# Pam pre 0.99.2.0 - https://github.com/fail2ban/fail2ban/pull/358 (format is obsolete, can be removed, but still match right now)
|
||||
# failJSON: { "time": "2005-02-06T12:02:29", "match": true , "host": "64.168.103.1", "desc": "obsolete, can be removed, but still match right now" }
|
||||
Feb 6 12:02:29 server vsftpd(pam_unix)[15522]: authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=64.168.103.1 user=user1
|
||||
|
||||
#2 Internal
|
||||
|
|
@ -15,3 +15,13 @@ Oct 23 21:15:42 vps vsftpd: pam_unix(vsftpd:auth): authentication failure; logna
|
|||
|
||||
# failJSON: { "time": "2016-09-08T00:39:49", "match": true , "host": "192.0.2.1" }
|
||||
Thu Sep 8 00:39:49 2016 [pid 15019] [guest] FAIL LOGIN: Client "::ffff:192.0.2.1", "User is not in the allow user list."
|
||||
|
||||
# fileOptions: {"logtype": "journal"}
|
||||
|
||||
# failJSON: { "match": true , "host": "192.0.2.222", "desc": "gh-3954" }
|
||||
2025-03-04T01:06:36.645577 ip-172-31-3-150.ap-southeast-2.compute.internal vsftpd[1658]: [username] FAIL LOGIN: Client "192.0.2.222"
|
||||
|
||||
# failJSON: { "match": true , "host": "192.0.2.223", "desc": "gh-3954, more tests, without part `pam_unix(vsftpd:auth): ` (unknown if it is needed)" }
|
||||
2025-03-04T01:06:37.123456 ip-172-31-3-150.ap-southeast-2.compute.internal vsftpd[1659]: authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=192.0.2.223 user=tester
|
||||
# failJSON: { "match": true , "host": "192.0.2.224", "desc": "gh-3954, more tests, with part `pam_unix(vsftpd:auth): ` (unknown if it is needed, but it matches)" }
|
||||
2025-03-04T01:06:38.123456 ip-172-31-3-150.ap-southeast-2.compute.internal vsftpd[1660]: pam_unix(vsftpd:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=192.0.2.224 user=tester
|
||||
|
|
|
|||
|
|
@ -0,0 +1,4 @@
|
|||
test-local-net ; test local subnet
|
||||
test-subnet-a, test-subnet-b ; further test subnets
|
||||
192.0.2.200, 2001:0db8::00c8 # 2 IPs
|
||||
192.0.2.216/29, 2001:db8::d8/125 # 2 subnets by IP/CIDR notation
|
||||
|
|
@ -399,6 +399,82 @@ class IgnoreIP(LogCaptureTestCase):
|
|||
self.filter.addIgnoreIP('192.168.1.0/255.255.0.0')
|
||||
self.assertRaises(ValueError, self.filter.addIgnoreIP, '192.168.1.0/255.255.0.128')
|
||||
|
||||
def testIgnoreIPDNS(self):
|
||||
# test subnets are pre-cached (as IPAddrSet), so it shall work even without network:
|
||||
for dns in ("test-subnet-a", "test-subnet-b"):
|
||||
self.filter.addIgnoreIP(dns)
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('192.0.2.1')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('192.0.2.7')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('192.0.2.16')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('192.0.2.23')))
|
||||
self.assertFalse(self.filter.inIgnoreIPList(IPAddr('192.0.2.8')))
|
||||
self.assertFalse(self.filter.inIgnoreIPList(IPAddr('192.0.2.15')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('2001:db8::00')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('2001:db8::07')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('2001:0db8:0000:0000:0000:0000:0000:0000')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('2001:0db8:0000:0000:0000:0000:0000:0007')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('2001:db8::10')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('2001:db8::17')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('2001:0db8:0000:0000:0000:0000:0000:0010')))
|
||||
self.assertTrue(self.filter.inIgnoreIPList(IPAddr('2001:0db8:0000:0000:0000:0000:0000:0017')))
|
||||
self.assertFalse(self.filter.inIgnoreIPList(IPAddr('2001:db8::08')))
|
||||
self.assertFalse(self.filter.inIgnoreIPList(IPAddr('2001:db8::0f')))
|
||||
self.assertFalse(self.filter.inIgnoreIPList(IPAddr('2001:0db8:0000:0000:0000:0000:0000:0008')))
|
||||
self.assertFalse(self.filter.inIgnoreIPList(IPAddr('2001:0db8:0000:0000:0000:0000:0000:000f')))
|
||||
|
||||
# to test several IPs in ip-set from file "files/test-ign-ips-file":
|
||||
TEST_IPS_IGN_FILE = {
|
||||
'127.0.0.1': True,
|
||||
'127.255.255.255': True,
|
||||
'127.0.0.1/8': True,
|
||||
'192.0.2.1': True,
|
||||
'192.0.2.7': True,
|
||||
'192.0.2.0/29': True,
|
||||
'192.0.2.16': True,
|
||||
'192.0.2.23': True,
|
||||
'192.0.2.200': True,
|
||||
'192.0.2.216': True,
|
||||
'192.0.2.223': True,
|
||||
'192.0.2.216/29': True,
|
||||
'192.0.2.8': False,
|
||||
'192.0.2.15': False,
|
||||
'192.0.2.100': False,
|
||||
'192.0.2.224': False,
|
||||
'::1': True,
|
||||
'2001:db8::00': True,
|
||||
'2001:db8::07': True,
|
||||
'2001:db8::0/125': True,
|
||||
'2001:0db8:0000:0000:0000:0000:0000:0000': True,
|
||||
'2001:0db8:0000:0000:0000:0000:0000:0007': True,
|
||||
'2001:db8::10': True,
|
||||
'2001:db8::17': True,
|
||||
'2001:0db8:0000:0000:0000:0000:0000:0010': True,
|
||||
'2001:0db8:0000:0000:0000:0000:0000:0017': True,
|
||||
'2001:db8::c8': True,
|
||||
'2001:db8::d8': True,
|
||||
'2001:db8::df': True,
|
||||
'2001:db8::d8/125': True,
|
||||
'2001:0db8:0000:0000:0000:0000:0000:00d8': True,
|
||||
'2001:0db8:0000:0000:0000:0000:0000:00df': True,
|
||||
'2001:db8::08': False,
|
||||
'2001:db8::0f': False,
|
||||
'2001:0db8:0000:0000:0000:0000:0000:0008': False,
|
||||
'2001:0db8:0000:0000:0000:0000:0000:000f': False,
|
||||
'2001:db8::e0': False,
|
||||
'2001:0db8:0000:0000:0000:0000:0000:00e0': False,
|
||||
}
|
||||
|
||||
def testIgnoreIPFileIPAddr(self):
|
||||
fname = 'file://' + os.path.join(TEST_FILES_DIR, "test-ign-ips-file")
|
||||
self.filter.ignoreSelf = False
|
||||
self.filter.addIgnoreIP(fname)
|
||||
for ip, v in IgnoreIP.TEST_IPS_IGN_FILE.items():
|
||||
self.assertEqual(self.filter.inIgnoreIPList(IPAddr(ip)), v, ("for %r in ignoreip, file://test-ign-ips-file)" % (ip,)))
|
||||
# now remove it:
|
||||
self.filter.delIgnoreIP(fname)
|
||||
for ip in IgnoreIP.TEST_IPS_IGN_FILE.keys():
|
||||
self.assertEqual(self.filter.inIgnoreIPList(IPAddr(ip)), False, ("for %r ignoreip, without file://test-ign-ips-file)" % (ip,)))
|
||||
|
||||
def testIgnoreInProcessLine(self):
|
||||
setUpMyTime()
|
||||
try:
|
||||
|
|
@ -2427,6 +2503,63 @@ class DNSUtilsNetworkTests(unittest.TestCase):
|
|||
DNSUtils.CACHE_nameToIp.unset(DNSUtils._getSelfIPs_key)
|
||||
DNSUtils.CACHE_nameToIp.unset(DNSUtils._getNetIntrfIPs_key)
|
||||
|
||||
def test_FileIPAddrSet(self):
|
||||
fname = os.path.join(TEST_FILES_DIR, "test-ign-ips-file")
|
||||
ips = DNSUtils.getIPsFromFile(fname)
|
||||
for ip, v in IgnoreIP.TEST_IPS_IGN_FILE.items():
|
||||
self.assertEqual(IPAddr(ip) in ips, v, ("for %r in test-ign-ips-file\n containing %s)" % (ip, set(ips))))
|
||||
|
||||
def test_FileIPAddrSet_Update(self):
|
||||
fname = tempfile.mktemp(prefix='tmp_fail2ban', suffix='.ips')
|
||||
f = open(fname, 'wb')
|
||||
try:
|
||||
f.write(b"192.0.2.200, 192.0.2.201\n")
|
||||
f.flush()
|
||||
ips = DNSUtils.getIPsFromFile(fname)
|
||||
self.assertTrue(IPAddr('192.0.2.200') in ips)
|
||||
self.assertTrue(IPAddr('192.0.2.201') in ips)
|
||||
self.assertFalse(IPAddr('192.0.2.202') in ips)
|
||||
# +1m, jump to next minute to force next check for update:
|
||||
MyTime.setTime(MyTime.time() + 60)
|
||||
# add .202, some comment and check all 3 IPs are there:
|
||||
f.write(b"""192.0.2.202\n
|
||||
# 2001:db8::ca/127 ; IPv6 commented yet
|
||||
""")
|
||||
f.flush()
|
||||
self.assertTrue(IPAddr('192.0.2.200') in ips)
|
||||
self.assertTrue(IPAddr('192.0.2.201') in ips)
|
||||
self.assertTrue(IPAddr('192.0.2.202') in ips)
|
||||
self.assertFalse(IPAddr('2001:db8::ca') in ips)
|
||||
self.assertFalse(IPAddr('2001:db8::cb') in ips)
|
||||
# +1m, jump to next minute to force next check for update:
|
||||
MyTime.setTime(MyTime.time() + 60)
|
||||
# remove .200, add IPv6-subnet and check all new IPs are there:
|
||||
f.seek(0); f.truncate()
|
||||
f.write(b"""
|
||||
# 192.0.2.200 ; commented
|
||||
192.0.2.201, 192.0.2.202 # no .200 anymore
|
||||
2001:db8::ca/127 ; but 2 new IPv6
|
||||
""")
|
||||
f.flush()
|
||||
self.assertFalse(IPAddr('192.0.2.200') in ips)
|
||||
self.assertTrue(IPAddr('192.0.2.201') in ips)
|
||||
self.assertTrue(IPAddr('192.0.2.202') in ips)
|
||||
self.assertTrue(IPAddr('2001:db8::ca') in ips)
|
||||
self.assertTrue(IPAddr('2001:db8::cb') in ips)
|
||||
# +1m, jump to next minute to force next check for update:
|
||||
MyTime.setTime(MyTime.time() + 60)
|
||||
self.assertFalse(ips._isModified()); # must be unchanged
|
||||
self.assertEqual(ips._isModified(), None); # not checked by latency (same time)
|
||||
f.write(b"""#END of file\n""")
|
||||
f.flush()
|
||||
# +1m, jump to next minute to force next check for update:
|
||||
MyTime.setTime(MyTime.time() + 60)
|
||||
self.assertTrue(ips._isModified()); # must be modified
|
||||
self.assertEqual(ips._isModified(), None); # not checked by latency (same time)
|
||||
finally:
|
||||
tearDownMyTime()
|
||||
_killfile(f, fname)
|
||||
|
||||
def testFQDN(self):
|
||||
unittest.F2B.SkipIfNoNetwork()
|
||||
sname = DNSUtils.getHostname(fqdn=False)
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@ from io import StringIO
|
|||
from .utils import LogCaptureTestCase, logSys as DefLogSys
|
||||
|
||||
from ..helpers import formatExceptionInfo, mbasename, TraceBack, FormatterWithTraceBack, getLogger, \
|
||||
getVerbosityFormat, splitwords, uni_decode, uni_string
|
||||
getVerbosityFormat, removeComments, splitwords, uni_decode, uni_string
|
||||
from ..server.mytime import MyTime
|
||||
|
||||
|
||||
|
|
@ -68,6 +68,16 @@ class HelpersTest(unittest.TestCase):
|
|||
self.assertEqual(splitwords(' 1\n 2, 3'), ['1', '2', '3'])
|
||||
self.assertEqual(splitwords('\t1\t 2,\r\n 3\n'), ['1', '2', '3']); # other spaces
|
||||
|
||||
def testSplitNoComments(self):
|
||||
s = '''
|
||||
# comment ...
|
||||
; comment ...
|
||||
line1 A # comment ...
|
||||
line2 B ; comment ...
|
||||
'''
|
||||
self.assertEqual(splitwords(s, ignoreComments=True), ['line1', 'A', 'line2', 'B'])
|
||||
self.assertEqual(splitwords(removeComments(s)), ['line1', 'A', 'line2', 'B'])
|
||||
|
||||
|
||||
def _sh_call(cmd):
|
||||
import subprocess
|
||||
|
|
|
|||
|
|
@ -163,6 +163,7 @@ def testSampleRegexsFactory(name, basedir):
|
|||
ignoreBlock = False
|
||||
lnnum = 0
|
||||
for line in logFile:
|
||||
jsonline = ''
|
||||
lnnum += 1
|
||||
jsonREMatch = re.match("^#+ ?(failJSON|(?:file|filter)Options|addFILE):(.+)$", line)
|
||||
if jsonREMatch:
|
||||
|
|
@ -204,7 +205,9 @@ def testSampleRegexsFactory(name, basedir):
|
|||
except ValueError as e: # pragma: no cover - we've valid json's
|
||||
raise ValueError("%s: %s:%i" %
|
||||
(e, logFile.getFileName(), lnnum))
|
||||
jsonline = line
|
||||
line = next(logFile)
|
||||
lnnum += 1
|
||||
elif ignoreBlock or line.startswith("#") or not line.strip():
|
||||
continue
|
||||
else: # pragma: no cover - normally unreachable
|
||||
|
|
@ -299,8 +302,9 @@ def testSampleRegexsFactory(name, basedir):
|
|||
import pprint
|
||||
raise AssertionError("%s: %s on: %s:%i, line:\n %s\nregex (%s):\n %s\n"
|
||||
"faildata: %s\nfail: %s" % (
|
||||
fltName, e, logFile.getFileName(), lnnum,
|
||||
line, failregex, regexList[failregex] if failregex != -1 else None,
|
||||
fltName, e, logFile.getFileName(), lnnum,
|
||||
(("%s\n\u25ba %s" % (jsonline, line)) if jsonline else line),
|
||||
failregex, regexList[failregex] if failregex != -1 else None,
|
||||
'\n'.join(pprint.pformat(faildata).splitlines()),
|
||||
'\n'.join(pprint.pformat(fail).splitlines())))
|
||||
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ from io import StringIO
|
|||
from functools import wraps
|
||||
|
||||
from ..helpers import getLogger, str2LogLevel, getVerbosityFormat, uni_decode
|
||||
from ..server.ipdns import IPAddr, DNSUtils
|
||||
from ..server.ipdns import IPAddr, IPAddrSet, DNSUtils
|
||||
from ..server.mytime import MyTime
|
||||
from ..server.utils import Utils
|
||||
# for action_d.test_smtp :
|
||||
|
|
@ -335,6 +335,12 @@ def initTests(opts):
|
|||
ips = set([IPAddr('127.0.0.1'), IPAddr('::1')]); # DNSUtils.dnsToIp('localhost')
|
||||
for i in DNSUtils.getSelfNames():
|
||||
c.set(i, ips)
|
||||
# some test subnets (although normally they are not resolved to addr/cidr,
|
||||
# we'll use IPAddrSet here to seek through the resolved subnet in tests):
|
||||
c = DNSUtils.CACHE_nameToIp
|
||||
c.set('test-local-net', IPAddrSet([IPAddr('127.0.0.1/8'), IPAddr('::1')]))
|
||||
c.set('test-subnet-a', IPAddrSet([IPAddr('192.0.2.0/29'), IPAddr('2001:db8::0/125')])); # 192.0.2.0 .. 192.0.2.7, 2001:db8::00 .. 2001:db8::07
|
||||
c.set('test-subnet-b', IPAddrSet([IPAddr('192.0.2.16/29'), IPAddr('2001:db8::10/125')])); # 192.0.2.16 .. 192.0.2.23, 2001:db8::10 .. 2001:db8::17
|
||||
|
||||
|
||||
def mtimesleep():
|
||||
|
|
|
|||
|
|
@ -247,7 +247,8 @@ Values can also be quoted (required when value includes a ","). More that one ac
|
|||
boolean value (default true) indicates the banning of own IP addresses should be prevented
|
||||
.TP
|
||||
.B ignoreip
|
||||
list of IPs not to ban. They can include a DNS resp. CIDR mask too. The option affects additionally to \fBignoreself\fR (if true) and don't need to contain own DNS resp. IPs of the running host.
|
||||
list of IPs not to ban. They can also include CIDR mask or can be DNS (FQDN), or even raw string (if jail banning IDs instead of IPs). The option affects additionally to \fBignoreself\fR (if true) and don't need to contain own DNS resp. IPs of the running host.
|
||||
This can also contain a filename (prefixed with "file:") with entries to ignore, which will be lazy loaded to the runtime on demand by first ban and automatically reloaded by update after small latency.
|
||||
.TP
|
||||
.B ignorecommand
|
||||
command that is executed to determine if the current candidate IP for banning (or failure-ID for raw IDs) should not be banned. This option operates alongside the \fBignoreself\fR and \fBignoreip\fR options. It is executed first, only if neither \fBignoreself\fR nor \fBignoreip\fR match the criteria.
|
||||
|
|
|
|||
Loading…
Reference in New Issue