Merge pull request #1557 from sebres/_0.10/fix-reload-bug

0.10/reload-and-more: reload without restart, stability and performance fixes

ChangeLog

@@ -12,14 +12,22 @@ ver. 0.10.0 (2016/XX/XXX) - gonna-be-released-some-time-shining
 TODO: implementing of options resp. other tasks from PR #1346
 
 ### Fixes
-* [grave] memory leak's fixed (gh-1277, gh-1234)
+* [Grave] memory leak's fixed (gh-1277, gh-1234)
-* tricky bug fix: last position of log file will be never retrieved (gh-795),
+* Tricky bug fix: last position of log file will be never retrieved (gh-795),
   because of CASCADE all log entries will be deleted from logs table together with jail, 
   if used "INSERT OR REPLACE" statement
-* asyncserver (asyncore) code fixed and test cases repaired (again gh-161)
+* Asyncserver (asyncore) code fixed and test cases repaired (again gh-161)
 * testSocket: sporadical bug repaired - wait for server thread starts a socket (listener)
 * testExecuteTimeoutWithNastyChildren: sporadical bug repaired - wait for pid file inside bash,
   kill tree in any case (gh-1155)
+* Fixed high-load of pyinotify-backend,
+  see https://github.com/fail2ban/fail2ban/issues/885#issuecomment-248964591
+* Database: stability fix - repack cursor iterator as long as locked
+* File filter backends: stability fix for sporadically errors - always close file
+  handle, otherwise may be locked (prevent log-rotate, etc.)
+* Pyinotify-backend: stability fix for sporadically errors in multi-threaded
+  environment (without lock)
+* Fixed sporadically error in testCymruInfoNxdomain, because of unsorted values
 
 ### New Features
 * IPv6 support:
@@ -33,18 +41,33 @@ TODO: implementing of options resp. other tasks from PR #1346
     - new conditional section functionality used in config resp. includes:
       - [Init?family=inet4] - IPv4 qualified hosts only
       - [Init?family=inet6] - IPv6 qualified hosts only
+* New reload functionality (now totally without restart, unbanning/rebanning, etc.),
+  see gh-1557
+* Several commands extended and new commands introduced:
+  - `restart [--unban] [--if-exists] <JAIL>` - restarts the jail \<JAIL\>
+    (alias for `reload --restart ... <JAIL>`)
+  - `reload [--restart] [--unban] [--all]` - reloads the configuration without restarting
+    of the server, the option `--restart` activates completely restarting of affected jails,
+    thereby can unban IP addresses (if option `--unban` specified)
+  - `reload [--restart] [--unban] [--if-exists] <JAIL>` - reloads the jail \<JAIL\>,
+    or restarts it (if option `--restart` specified), at the same time unbans all IP addresses
+    banned in this jail, if option `--unban` specified
+  - `unban --all` - unbans all IP addresses (in all jails and database)
+  - `unban <IP> ... <IP>` - unbans \<IP\> (in all jails and database) (see gh-1388)
+* New command action parameter `actionrepair` - command executed in order to restore
+  sane environment in error case of `actioncheck`.
 
 ### Enhancements
-* huge increasing of fail2ban performance and especially test-cases performance (see gh-1109)
+* Huge increasing of fail2ban performance and especially test-cases performance (see gh-1109)
-* datedetector: in-place reordering using hits and last used time: 
+* Datedetector: in-place reordering using hits and last used time: 
   matchTime, template list etc. rewritten because of performance degradation
-* prevent out of memory situation if many IP's makes extremely many failures (maxEntries)
+* Prevent out of memory situation if many IP's makes extremely many failures (maxEntries)
-* introduced string to seconds (str2seconds) for configuration entries with time,
+* Introduced string to seconds (str2seconds) for configuration entries with time,
   use `1h` instead of `3600`, `1d` instead of `86400`, etc
 * seekToTime - prevent completely read of big files first time (after start of service), 
   initial seek to start time using half-interval search algorithm (see issue gh-795)
-* ticket and some other modules prepared to easy merge with newest version of 'ban-time-incr'
+* Ticket and some other modules prepared to easy merge with newest version of 'ban-time-incr'
-* cache dnsToIp, ipToName to prevent long wait during retrieving of ip/name,
+* Cache dnsToIp, ipToName to prevent long wait during retrieving of ip/name,
   especially for wrong dns or lazy dns-system
 * FailManager memory-optimization: increases performance, 
   prevents memory leakage, because don't copy failures list on some operations
@@ -54,14 +77,52 @@ TODO: implementing of options resp. other tasks from PR #1346
     - `-g`, `--no-gamin` to prevent running of tests that require the gamin (slow)
     - `-m`, `--memory-db` - run database tests using memory instead of file
     - `-i`, `--ignore` - negate [regexps] filter to ignore tests matched specified regexps
-* background servicing: prevents memory leak on some platforms/python versions, using forced GC
+* Background servicing: prevents memory leak on some platforms/python versions, using forced GC
   in periodic intervals (latency and threshold)
 * executeCmd partially moved from action to new module utils
-* several functionality of class `DNSUtils` moved to new class `IPAddr`, 
+* Several functionality of class `DNSUtils` moved to new class `IPAddr`, 
   both classes moved to new module `ipdns`
-* pseudo-conditional section introduced, for conditional substitution resp. 
+* Pseudo-conditional section introduced, for conditional substitution resp. 
   evaluation of parameters for different family qualified hosts, 
   syntax `[Section?family=inet6]` (currently use for IPv6-support only).
+* All the backends were rewritten to get reload-possibility, performance increased,
+  so fewer greedy regarding cpu- resp. system-load now
+* Numeric log-level allowed now in server (resp. fail2ban.conf);
+* Implemented better error handling in some multi-threaded routines; shutdown of jails
+  rewritten (faster and safer, does not breaks shutdown process if some error occurred)
+* Possibility for overwriting some configuration options (read with config-readers)
+  with command line option, e. g.:
+```bash
+## start server with DEBUG log-level (ignore level read from fail2ban.conf):
+fail2ban-client --loglevel DEBUG start
+## or
+fail2ban-server -c /cfg/path --loglevel DEBUG start
+## keep server log-level by reload (without restart it)
+fail2ban-client --loglevel DEBUG reload
+## switch log-level back to INFO:
+fail2ban-client set loglevel INFO
+```
+* Optimized BanManager: increase performance, fewer system load, try to prevent
+  memory leakage:
+  - better ban/unban handling within actions (e.g. used dict instead of list)
+  - don't copy bans resp. its list on some operations;
+  - added new unbantime handling to relieve unBanList (prevent permanent
+    searching for tickets to unban)
+  - prefer failure-ID as identifier of the ticket to its IP (most of the time
+    the same, but it can be something else e.g. user name in some complex jails,
+    as introduced in 0.10)
+* Regexp enhancements:
+  - build replacement of `<HOST>` substitution corresponding parameter
+    `usedns` - dns-part will be added only if `usedns` is not `no`,
+    also using fail2ban-regex
+  - new replacement for `<ADDR>` in opposition to `<HOST>`, for separate
+    usage of 2 address groups only (regardless of `usedns`), `ip4` and `ip6`
+    together, without host (dns)
+* fail2ban-testcases:
+  - `assertLogged` extended with parameter wait (to wait up to specified timeout,
+    before we throw assert exception) + test cases rewritten using that
+  - added `assertDictEqual` for compatibility to early python versions (< 2.7);
+  - new `with_foreground_server_thread` decorator to test several client/server commands
 
 
 ver. 0.9.6 (2016/XX/XX) - wanna-be-released

fail2ban/__init__.py

@@ -26,8 +26,11 @@ __license__ = "GPL"
 
 import logging.handlers
 
-# Custom debug level
+# Custom debug levels
+logging.TRACEDEBUG = 7
 logging.HEAVYDEBUG = 5
+logging.addLevelName(logging.TRACEDEBUG, 'TRACE')
+logging.addLevelName(logging.HEAVYDEBUG, 'HEAVY')
 
 """
 Below derived from:

fail2ban/client/actionreader.py

@@ -38,7 +38,9 @@ class ActionReader(DefinitionInitConfigReader):
 	_configOpts = {
 		"actionstart": ["string", None],
 		"actionstop": ["string", None],
+		"actionreload": ["string", None],
 		"actioncheck": ["string", None],
+		"actionrepair": ["string", None],
 		"actionban": ["string", None],
 		"actionunban": ["string", None],
 	}

fail2ban/client/beautifier.py

@@ -46,7 +46,7 @@ class Beautifier:
 		return self.__inputCmd
 
 	def beautify(self, response):
-		logSys.debug(
+		logSys.log(5,
 			"Beautify " + repr(response) + " with " + repr(self.__inputCmd))
 		inC = self.__inputCmd
 		msg = response

fail2ban/client/configurator.py

@@ -72,8 +72,8 @@ class Configurator:
 	def getEarlyOptions(self):
 		return self.__fail2ban.getEarlyOptions()
 
-	def getOptions(self, jail = None):
+	def getOptions(self, jail=None, updateMainOpt=None):
-		self.__fail2ban.getOptions()
+		self.__fail2ban.getOptions(updateMainOpt)
 		return self.__jails.getOptions(jail)
 		
 	def convertToProtocol(self):

fail2ban/client/fail2banclient.py

@@ -91,7 +91,7 @@ class Fail2banClient(Fail2banCmdLine, Thread):
 						client = CSocket(self._conf["socket"])
 					ret = client.send(c)
 					if ret[0] == 0:
-						logSys.debug("OK : %r", ret[1])
+						logSys.log(5, "OK : %r", ret[1])
 						if showRet or c[0] == 'echo':
 							output(beautifier.beautify(ret[1]))
 					else:
@@ -104,7 +104,7 @@ class Fail2banClient(Fail2banCmdLine, Thread):
 						if showRet or c != ["ping"]:
 							self.__logSocketError()
 						else:
-							logSys.debug(" -- ping failed -- %r", e)
+							logSys.log(5, " -- ping failed -- %r", e)
 					return False
 				except Exception as e: # pragma: no cover
 					if showRet or self._conf["verbose"] > 1:
@@ -226,11 +226,11 @@ class Fail2banClient(Fail2banCmdLine, Thread):
 		# prepare: read config, check configuration is valid, etc.:
 		if phase is not None:
 			phase['start'] = True
-			logSys.debug('  client phase %s', phase)
+			logSys.log(5, '  client phase %s', phase)
 		stream = self.__prepareStartServer()
 		if phase is not None:
 			phase['ready'] = phase['start'] = (True if stream else False)
-			logSys.debug('  client phase %s', phase)
+			logSys.log(5, '  client phase %s', phase)
 		if not stream:
 			return False
 		# configure server with config stream:
@@ -246,6 +246,10 @@ class Fail2banClient(Fail2banCmdLine, Thread):
 	# @param cmd the command line
 
 	def __processCommand(self, cmd):
+		# wrap tuple to list (because could be modified here):
+		if not isinstance(cmd, list):
+			cmd = list(cmd)
+		# process:
 		if len(cmd) == 1 and cmd[0] == "start":
 
 			ret = self.__startServer(self._conf["background"])
@@ -253,8 +257,12 @@ class Fail2banClient(Fail2banCmdLine, Thread):
 				return False
 			return ret
 
-		elif len(cmd) == 1 and cmd[0] == "restart":
+		elif len(cmd) >= 1 and cmd[0] == "restart":
-
+			# if restart jail - re-operate via "reload --restart ...":
+			if len(cmd) > 1:
+				cmd[0:1] = ["reload", "--restart"]
+				return self.__processCommand(cmd)
+			# restart server:
 			if self._conf.get("interactive", False):
 				output('  ## stop ... ')
 			self.__processCommand(['stop'])
@@ -273,9 +281,21 @@ class Fail2banClient(Fail2banCmdLine, Thread):
 			return self.__processCommand(['start'])
 
 		elif len(cmd) >= 1 and cmd[0] == "reload":
+			# reload options:
+			opts = []
+			while len(cmd) >= 2:
+				if cmd[1] in ('--restart', "--unban", "--if-exists"):
+					opts.append(cmd[1])
+					del cmd[1]
+				else:
+					if len(cmd) > 2:
+						logSys.error("Unexpected argument(s) for reload: %r", cmd[1:])
+						return False
+					# stop options - jail name or --all
+					break
 			if self.__ping():
 				if len(cmd) == 1:
-					jail = 'all'
+					jail = '--all'
 					ret, stream = self.readConfig()
 				else:
 					jail = cmd[1]
@@ -283,9 +303,10 @@ class Fail2banClient(Fail2banCmdLine, Thread):
 				# Do not continue if configuration is not 100% valid
 				if not ret:
 					return False
-				self.__processCmd([['stop', jail]], False)
+				if self._conf.get("interactive", False):
-				# Configure the server
+					output('  ## reload ... ')
-				return self.__processCmd(stream, True)
+				# Reconfigure the server
+				return self.__processCmd([['reload', jail, opts, stream]], True)
 			else:
 				logSys.error("Could not find server")
 				return False
@@ -320,7 +341,7 @@ class Fail2banClient(Fail2banCmdLine, Thread):
 			maxtime = self._conf["timeout"]
 		# Wait for the server to start (the server has 30 seconds to answer ping)
 		starttime = time.time()
-		logSys.debug("__waitOnServer: %r", (alive, maxtime))
+		logSys.log(5, "__waitOnServer: %r", (alive, maxtime))
 		test = lambda: os.path.exists(self._conf["socket"]) and self.__ping()
 		with VisualWait(self._conf["verbose"]) as vis:
 			sltime = 0.0125 / 2

fail2ban/client/fail2bancmdline.py

@@ -242,7 +242,7 @@ class Fail2banCmdLine():
 		try:
 			self.configurator.Reload()
 			self.configurator.readAll()
-			ret = self.configurator.getOptions(jail)
+			ret = self.configurator.getOptions(jail, self._conf)
 			self.configurator.convertToProtocol()
 			stream = self.configurator.getConfigStream()
 		except Exception as e:

fail2ban/client/fail2banreader.py

@@ -25,7 +25,7 @@ __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
 from .configreader import ConfigReader
-from ..helpers import getLogger
+from ..helpers import getLogger, str2LogLevel
 
 # Gets the instance of the logger.
 logSys = getLogger(__name__)
@@ -49,13 +49,17 @@ class Fail2banReader(ConfigReader):
 		]
 		return ConfigReader.getOptions(self, "Definition", opts)
 	
-	def getOptions(self):
+	def getOptions(self, updateMainOpt=None):
 		opts = [["string", "loglevel", "INFO" ],
 				["string", "logtarget", "STDERR"],
 				["string", "syslogsocket", "auto"],
 				["string", "dbfile", "/var/lib/fail2ban/fail2ban.sqlite3"],
 				["string", "dbpurgeage", "1d"]]
 		self.__opts = ConfigReader.getOptions(self, "Definition", opts)
+		if updateMainOpt:
+			self.__opts.update(updateMainOpt)
+		# check given log-level:
+		str2LogLevel(self.__opts.get('loglevel', 0))
 	
 	def convert(self):
 		# Ensure logtarget/level set first so any db errors are captured

fail2ban/client/fail2banregex.py

@@ -29,7 +29,6 @@ __copyright__ = "Copyright (c) 2004-2008 Cyril Jaquier, 2012-2014 Yaroslav Halch
 __license__ = "GPL"
 
 import getopt
-import locale
 import logging
 import os
 import shlex
@@ -52,7 +51,7 @@ from .filterreader import FilterReader
 from ..server.filter import Filter, FileContainer
 from ..server.failregex import RegexException
 
-from ..helpers import FormatterWithTraceBack, getLogger
+from ..helpers import FormatterWithTraceBack, getLogger, PREFER_ENC
 # Gets the instance of the logger.
 logSys = getLogger("fail2ban")
 
@@ -127,6 +126,9 @@ Report bugs to https://github.com/fail2ban/fail2ban/issues
 			   help="File encoding. Default: system locale"),
 		Option("-r", "--raw", action='store_true',
 			   help="Raw hosts, don't resolve dns"),
+		Option("--usedns", action='store', default=None,
+			   help="DNS specified replacement of tags <HOST> in regexp "
+			        "('yes' - matches all form of hosts, 'no' - IP addresses only)"),
 		Option("-L", "--maxlines", type=int, default=0,
 			   help="maxlines for multi-line regex"),
 		Option("-m", "--journalmatch",
@@ -239,8 +241,10 @@ class Fail2banRegex(object):
 		if opts.encoding:
 			self.encoding = opts.encoding
 		else:
-			self.encoding = locale.getpreferredencoding()
+			self.encoding = PREFER_ENC
 		self.raw = True if opts.raw else False
+		if opts.usedns:
+			self._filter.setUseDns(opts.usedns)
 
 	def decode_line(self, line):
 		return FileContainer.decode_line('<LOG>', self.encoding, line)

fail2ban/client/jailreader.py

@@ -106,7 +106,7 @@ class JailReader(ConfigReader):
 				["int",    "maxretry", None],
 				["string", "findtime", None],
 				["string", "bantime", None],
-				["string", "usedns", None],
+				["string", "usedns", None], # be sure usedns is before all regex(s) in stream
 				["string", "failregex", None],
 				["string", "ignoreregex", None],
 				["string", "ignorecommand", None],

fail2ban/helpers.py

@@ -21,6 +21,7 @@ __author__ = "Cyril Jaquier, Arturo 'Buanzo' Busleiman, Yaroslav Halchenko"
 __license__ = "GPL"
 
 import gc
+import locale
 import logging
 import os
 import re
@@ -32,6 +33,9 @@ from threading import Lock
 from .server.mytime import MyTime
 
 
+PREFER_ENC = locale.getpreferredencoding()
+
+
 def formatExceptionInfo():
 	""" Consistently format exception information """
 	cla, exc = sys.exc_info()[:2]
@@ -125,6 +129,16 @@ def getLogger(name):
 		name = "fail2ban.%s" % name.rpartition(".")[-1]
 	return logging.getLogger(name)
 
+def str2LogLevel(value):
+	try:
+		if isinstance(value, int) or value.isdigit():
+			ll = int(value)
+		else:
+			ll = getattr(logging, value)
+	except AttributeError:
+		raise ValueError("Invalid log level %r" % value)
+	return ll
+
 
 def excepthook(exctype, value, traceback):
 	"""Except hook used to log unhandled exceptions to Fail2Ban log
@@ -144,6 +158,36 @@ def splitwords(s):
 	return filter(bool, map(str.strip, re.split('[ ,\n]+', s)))
 
 
+#
+# Following "uni_decode" function unified python independent any to string converting
+#
+# Typical example resp. work-case for understanding the coding/decoding issues:
+#
+#   [isinstance('', str), isinstance(b'', str), isinstance(u'', str)]
+#   [True, True, False]; # -- python2
+#	  [True, False, True]; # -- python3
+#
+if sys.version_info >= (3,):
+	def uni_decode(x, enc=PREFER_ENC, errors='strict'):
+		try:
+			if isinstance(x, bytes):
+				return x.decode(enc, errors)
+			return x
+		except (UnicodeDecodeError, UnicodeEncodeError): # pragma: no cover - unsure if reachable
+			if errors != 'strict': 
+				raise
+			return uni_decode(x, enc, 'replace')
+else:
+	def uni_decode(x, enc=PREFER_ENC, errors='strict'):
+		try:
+			if isinstance(x, unicode):
+				return x.encode(enc, errors)
+			return x
+		except (UnicodeDecodeError, UnicodeEncodeError): # pragma: no cover - unsure if reachable
+			if errors != 'strict':
+				raise
+			return uni_decode(x, enc, 'replace')
+
 class BgService(object):
 	"""Background servicing
 

fail2ban/protocol.py

@@ -49,16 +49,20 @@ protocol = [
 ['', "BASIC", ""],
 ["start", "starts the server and the jails"], 
 ["restart", "restarts the server"], 
-["reload", "reloads the configuration without restart"], 
+["restart [--unban] [--if-exists] <JAIL>", "restarts the jail <JAIL> (alias for 'reload --restart ... <JAIL>')"], 
-["reload <JAIL>", "reloads the jail <JAIL>"], 
+["reload [--restart] [--unban] [--all]", "reloads the configuration without restarting of the server, the option '--restart' activates completely restarting of affected jails, thereby can unban IP addresses (if option '--unban' specified)"],
+["reload [--restart] [--unban] [--if-exists] <JAIL>", "reloads the jail <JAIL>, or restarts it (if option '--restart' specified)"],
 ["stop", "stops all jails and terminate the server"], 
+["unban --all", "unbans all IP addresses (in all jails and database)"],
+["unban <IP> ... <IP>", "unbans <IP> (in all jails and database)"],
 ["status", "gets the current status of the server"], 
 ["ping", "tests if the server is alive"],
 ["echo", "for internal usage, returns back and outputs a given string"],
 ["help", "return this output"], 
 ["version", "return the server version"],
 ['', "LOGGING", ""],
-["set loglevel <LEVEL>", "sets logging level to <LEVEL>. Levels: CRITICAL, ERROR, WARNING, NOTICE, INFO, DEBUG"], 
+["set loglevel <LEVEL>", "sets logging level to <LEVEL>. Levels: CRITICAL, ERROR, WARNING, NOTICE, INFO, "
+	"DEBUG, TRACEDEBUG, HEAVYDEBUG or corresponding numeric value (50-5)"], 
 ["get loglevel", "gets the logging level"], 
 ["set logtarget <TARGET>", "sets logging target to <TARGET>. Can be STDOUT, STDERR, SYSLOG or a file"], 
 ["get logtarget", "gets logging target"], 

fail2ban/server/action.py

@@ -200,6 +200,9 @@ class CommandAction(ActionBase):
 	Attributes
 	----------
 	actionban
+	actioncheck
+	actionreload
+	actionrepair
 	actionstart
 	actionstop
 	actionunban
@@ -208,22 +211,35 @@ class CommandAction(ActionBase):
 
 	_escapedTags = set(('matches', 'ipmatches', 'ipjailmatches'))
 
-	timeout = 60
+	def clearAllParams(self):
-	## Command executed in order to initialize the system.
+		""" Clear all lists/dicts parameters (used by reloading)
-	actionstart = ''
+		"""
-	## Command executed when an IP address gets banned.
+		self.__init = 1
-	actionban = ''
+		try:
-	## Command executed when an IP address gets removed.
+			self.timeout = 60
-	actionunban = ''
+			## Command executed in order to initialize the system.
-	## Command executed in order to check requirements.
+			self.actionstart = ''
-	actioncheck = ''
+			## Command executed when an IP address gets banned.
-	## Command executed in order to stop the system.
+			self.actionban = ''
-	actionstop = ''
+			## Command executed when an IP address gets removed.
+			self.actionunban = ''
+			## Command executed in order to check requirements.
+			self.actioncheck = ''
+			## Command executed in order to restore sane environment in error case.
+			self.actionrepair = ''
+			## Command executed in order to stop the system.
+			self.actionstop = ''
+			## Command executed in case of reloading action.
+			self.actionreload = ''
+		finally:
+			self.__init = 0
 
 	def __init__(self, jail, name):
 		super(CommandAction, self).__init__(jail, name)
+		self.__init = 1
 		self.__properties = None
 		self.__substCache = {}
+		self.clearAllParams()
 		self._logSys.debug("Created %s" % self.__class__)
 
 	@classmethod
@@ -231,7 +247,7 @@ class CommandAction(ActionBase):
 		return NotImplemented # Standard checks
 
 	def __setattr__(self, name, value):
-		if not name.startswith('_') and not callable(value):
+		if not name.startswith('_') and not self.__init and not callable(value):
 			# special case for some pasrameters:
 			if name in ('timeout', 'bantime'):
 				value = str(MyTime.str2seconds(value))
@@ -264,28 +280,38 @@ class CommandAction(ActionBase):
 	def _substCache(self):
 		return self.__substCache
 
+	def _executeOperation(self, tag, operation):
+		"""Executes the operation commands (like "actionstart", "actionstop", etc).
+
+		Replace the tags in the action command with actions properties
+		and executes the resulting command.
+		"""
+		# check valid tags in properties (raises ValueError if self recursion, etc.):
+		res = True
+		try:
+			# common (resp. ipv4):
+			startCmd = self.replaceTag(tag, self._properties, 
+				conditional='family=inet4', cache=self.__substCache)
+			if startCmd:
+				res &= self.executeCmd(startCmd, self.timeout)
+			# start ipv6 actions if available:
+			if allowed_ipv6:
+				startCmd6 = self.replaceTag(tag, self._properties, 
+					conditional='family=inet6', cache=self.__substCache)
+				if startCmd6 and startCmd6 != startCmd:
+					res &= self.executeCmd(startCmd6, self.timeout)
+			if not res:
+				raise RuntimeError("Error %s action %s/%s" % (operation, self._jail, self._name,))
+		except ValueError as e:
+			raise RuntimeError("Error %s action %s/%s: %r" % (operation, self._jail, self._name, e))
+
 	def start(self):
 		"""Executes the "actionstart" command.
 
 		Replace the tags in the action command with actions properties
 		and executes the resulting command.
 		"""
-		# check valid tags in properties (raises ValueError if self recursion, etc.):
+		return self._executeOperation('<actionstart>', 'starting')
-		try:
-			# common (resp. ipv4):
-			startCmd = self.replaceTag('<actionstart>', self._properties, 
-				conditional='family=inet4', cache=self.__substCache)
-			res = self.executeCmd(startCmd, self.timeout)
-			# start ipv6 actions if available:
-			if allowed_ipv6:
-				startCmd6 = self.replaceTag('<actionstart>', self._properties, 
-					conditional='family=inet6', cache=self.__substCache)
-				if startCmd6 != startCmd:
-					res &= self.executeCmd(startCmd6, self.timeout)
-			if not res:
-				raise RuntimeError("Error starting action %s/%s" % (self._jail, self._name,))
-		except ValueError as e:
-			raise RuntimeError("Error starting action %s/%s: %r" % (self._jail, self._name, e))
 
 	def ban(self, aInfo):
 		"""Executes the "actionban" command.
@@ -323,18 +349,20 @@ class CommandAction(ActionBase):
 		Replaces the tags in the action command with actions properties
 		and executes the resulting command.
 		"""
-		# common (resp. ipv4):
+		return self._executeOperation('<actionstop>', 'stopping')
-		stopCmd = self.replaceTag('<actionstop>', self._properties, 
+
-			conditional='family=inet4', cache=self.__substCache)
+	def reload(self, **kwargs):
-		res = self.executeCmd(stopCmd, self.timeout)
+		"""Executes the "actionreload" command.
-		# ipv6 actions if available:
+
-		if allowed_ipv6:
+		Parameters
-			stopCmd6 = self.replaceTag('<actionstop>', self._properties, 
+		----------
-				conditional='family=inet6', cache=self.__substCache)
+		kwargs : dict
-			if stopCmd6 != stopCmd:
+		  Currently unused, because CommandAction do not support initOpts
-				res &= self.executeCmd(stopCmd6, self.timeout)
+
-		if not res:
+		Replaces the tags in the action command with actions properties
-			raise RuntimeError("Error stopping action")
+		and executes the resulting command.
+		"""
+		return self._executeOperation('<actionreload>', 'reloading')
 
 	@classmethod
 	def substituteRecursiveTags(cls, inptags, conditional=''):
@@ -520,14 +548,28 @@ class CommandAction(ActionBase):
 
 		checkCmd = self.replaceTag('<actioncheck>', self._properties, 
 			conditional=conditional, cache=self.__substCache)
-		if not self.executeCmd(checkCmd, self.timeout):
+		if checkCmd:
-			self._logSys.error(
-				"Invariant check failed. Trying to restore a sane environment")
-			self.stop()
-			self.start()
 			if not self.executeCmd(checkCmd, self.timeout):
-				self._logSys.critical("Unable to restore environment")
+				self._logSys.error(
-				return False
+					"Invariant check failed. Trying to restore a sane environment")
+				# try to find repair command, if exists - exec it:
+				repairCmd = self.replaceTag('<actionrepair>', self._properties, 
+					conditional=conditional, cache=self.__substCache)
+				if repairCmd:
+					if not self.executeCmd(repairCmd, self.timeout):
+						self._logSys.critical("Unable to restore environment")
+						return False
+				else:
+					# no repair command, try to restart action...
+					# [WARNING] TODO: be sure all banactions get a repair command, because
+					#    otherwise stop/start will theoretically remove all the bans,
+					#    but the tickets are still in BanManager, so in case of new failures
+					#    it will not be banned, because "already banned" will happen.
+					self.stop()
+					self.start()
+				if not self.executeCmd(checkCmd, self.timeout):
+					self._logSys.critical("Unable to restore environment")
+					return False
 
 		# Replace static fields
 		realCmd = self.replaceTag(cmd, self._properties, 

fail2ban/server/actions.py

@@ -36,7 +36,7 @@ from collections import Mapping
 try:
 	from collections import OrderedDict
 except ImportError:
-	OrderedDict = None
+	OrderedDict = dict
 
 from .banmanager import BanManager
 from .jailthread import JailThread
@@ -81,14 +81,11 @@ class Actions(JailThread, Mapping):
 		JailThread.__init__(self)
 		## The jail which contains this action.
 		self._jail = jail
-		if OrderedDict is not None:
+		self._actions = OrderedDict()
-			self._actions = OrderedDict()
-		else:
-			self._actions = dict()
 		## The ban manager.
 		self.__banManager = BanManager()
 
-	def add(self, name, pythonModule=None, initOpts=None):
+	def add(self, name, pythonModule=None, initOpts=None, reload=False):
 		"""Adds a new action.
 
 		Add a new action if not already present, defaulting to standard
@@ -116,7 +113,17 @@ class Actions(JailThread, Mapping):
 		"""
 		# Check is action name already exists
 		if name in self._actions:
-			raise ValueError("Action %s already exists" % name)
+			if not reload:
+				raise ValueError("Action %s already exists" % name)
+			# don't create new action if reload supported:
+			action = self._actions[name]
+			if hasattr(action, 'reload'):
+				# don't execute reload right now, reload after all parameters are actualized
+				if hasattr(action, 'clearAllParams'):
+					action.clearAllParams()
+					self._reload_actions[name] = initOpts
+				return
+		## Create new action:
 		if pythonModule is None:
 			action = CommandAction(self._jail, name)
 		else:
@@ -138,6 +145,27 @@ class Actions(JailThread, Mapping):
 			action = customActionModule.Action(self._jail, name, **initOpts)
 		self._actions[name] = action
 
+	def reload(self, begin=True):
+		""" Begin or end of reloading resp. refreshing of all parameters
+		"""
+		if begin:
+			self._reload_actions = dict()
+		else:
+			if hasattr(self, '_reload_actions'):
+				# reload actions after all parameters set via stream:
+				for name, initOpts in self._reload_actions.iteritems():
+					if name in self._actions:
+						self._actions[name].reload(**initOpts if initOpts else {})
+				# remove obsolete actions (untouched by reload process):
+				delacts = OrderedDict((name, action) for name, action in self._actions.iteritems()
+					if name not in self._reload_actions)
+				if len(delacts):
+					# unban all tickets using remove action only:
+					self.__flushBan(db=False, actions=delacts)
+					# stop and remove it:
+					self.stopActions(actions=delacts)
+				delattr(self, '_reload_actions')
+
 	def __getitem__(self, name):
 		try:
 			return self._actions[name]
@@ -180,7 +208,7 @@ class Actions(JailThread, Mapping):
 	def getBanTime(self):
 		return self.__banManager.getBanTime()
 
-	def removeBannedIP(self, ip):
+	def removeBannedIP(self, ip=None, db=True, ifexists=False):
 		"""Removes banned IP calling actions' unban method
 
 		Remove a banned IP now, rather than waiting for it to expire,
@@ -188,24 +216,50 @@ class Actions(JailThread, Mapping):
 
 		Parameters
 		----------
-		ip : str or IPAddr
+		ip : str or IPAddr or None
-			The IP address  to unban
+			The IP address to unban or all IPs if None
 
 		Raises
 		------
 		ValueError
 			If `ip` is not banned
 		"""
+		# Unban all?
+		if ip is None:
+			return self.__flushBan(db)
+		# Single IP:
 		# Always delete ip from database (also if currently not banned)
-		if self._jail.database is not None:
+		if db and self._jail.database is not None:
 			self._jail.database.delBan(self._jail, ip)
 		# Find the ticket with the IP.
-		ticket = self.__banManager.getTicketByIP(ip)
+		ticket = self.__banManager.getTicketByID(ip)
 		if ticket is not None:
 			# Unban the IP.
 			self.__unBan(ticket)
 		else:
-			raise ValueError("IP %s is not banned" % ip)
+			if ifexists:
+				return 0
+			raise ValueError("%s is not banned" % ip)
+		return 1
+
+
+	def stopActions(self, actions=None):
+		"""Stops the actions in reverse sequence (optionally filtered)
+		"""
+		if actions is None:
+			actions = self._actions
+		revactions = actions.items()
+		revactions.reverse()
+		for name, action in revactions:
+			try:
+				action.stop()
+			except Exception as e:
+				logSys.error("Failed to stop jail '%s' action '%s': %s",
+					self._jail.name, name, e,
+					exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
+			del self._actions[name]
+			logSys.debug("%s: action %s terminated", self._jail.name, name)
+
 
 	def run(self):
 		"""Main loop for Threading.
@@ -227,22 +281,14 @@ class Actions(JailThread, Mapping):
 					exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
 		while self.active:
 			if self.idle:
-				time.sleep(self.sleeptime)
+				Utils.wait_for(lambda: not self.active or not self.idle,
+					self.sleeptime * 10, self.sleeptime)
 				continue
-			if not Utils.wait_for(self.__checkBan, self.sleeptime):
+			if not Utils.wait_for(lambda: not self.active or self.__checkBan(), self.sleeptime):
 				self.__checkUnBan()
+		
 		self.__flushBan()
-
+		self.stopActions()
-		actions = self._actions.items()
-		actions.reverse()
-		for name, action in actions:
-			try:
-				action.stop()
-			except Exception as e:
-				logSys.error("Failed to stop jail '%s' action '%s': %s",
-					self._jail.name, name, e,
-					exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
-		logSys.debug(self._jail.name + ": action terminated")
 		return True
 
 	def __getBansMerged(self, mi, overalljails=False):
@@ -295,8 +341,11 @@ class Actions(JailThread, Mapping):
 		bool
 			True if an IP address get banned.
 		"""
-		ticket = self._jail.getFailTicket()
+		cnt = 0
-		if ticket:
+		while cnt < 100:
+			ticket = self._jail.getFailTicket()
+			if not ticket:
+				break
 			aInfo = CallingMap()
 			bTicket = BanManager.createBanTicket(ticket)
 			ip = bTicket.getIP()
@@ -311,8 +360,10 @@ class Actions(JailThread, Mapping):
 				aInfo["ipjailmatches"]  = lambda: "\n".join(mi4ip().getMatches())
 				aInfo["ipfailures"]     = lambda: mi4ip(True).getAttempt()
 				aInfo["ipjailfailures"] = lambda: mi4ip().getAttempt()
-			if self.__banManager.addBanTicket(bTicket):
+			reason = {}
-				logSys.notice("[%s] Ban %s" % (self._jail.name, aInfo["ip"]))
+			if self.__banManager.addBanTicket(bTicket, reason=reason):
+				cnt += 1
+				logSys.notice("[%s] %sBan %s", self._jail.name, ('' if not bTicket.restored else 'Restore '), ip)
 				for name, action in self._actions.iteritems():
 					try:
 						action.ban(aInfo.copy())
@@ -322,30 +373,68 @@ class Actions(JailThread, Mapping):
 							"info '%r': %s",
 							self._jail.name, name, aInfo, e,
 							exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
-				return True
+				# after all actions are processed set banned flag:
+				bTicket.banned = True
 			else:
-				logSys.notice("[%s] %s already banned" % (self._jail.name,
+				bTicket = reason['ticket']
-														aInfo["ip"]))
+				# if already banned (otherwise still process some action)
-		return False
+				if bTicket.banned:
+					# compare time of failure occurrence with time ticket was really banned:
+					diftm = ticket.getTime() - bTicket.getTime()
+					# log already banned with following level:
+					#   DEBUG   - before 3 seconds - certain interval for it, because of possible latency by recognizing in backends, etc.
+					#   NOTICE  - before 60 seconds - may still occurre if action are slow, or very high load in backend,
+					#   WARNING - after 60 seconds - very long time, something may be wrong
+					ll = logging.DEBUG   if diftm < 3 \
+					else logging.NOTICE  if diftm < 60 \
+					else logging.WARNING
+					logSys.log(ll, "[%s] %s already banned", self._jail.name, ip)
+		if cnt:
+			logSys.debug("Banned %s / %s, %s ticket(s) in %r", cnt, 
+				self.__banManager.getBanTotal(), self.__banManager.size(), self._jail.name)
+		return cnt
 
 	def __checkUnBan(self):
 		"""Check for IP address to unban.
 
 		Unban IP addresses which are outdated.
 		"""
-		for ticket in self.__banManager.unBanList(MyTime.time()):
+		lst = self.__banManager.unBanList(MyTime.time())
+		for ticket in lst:
 			self.__unBan(ticket)
+		cnt = len(lst)
+		if cnt:
+			logSys.debug("Unbanned %s, %s ticket(s) in %r", 
+				cnt, self.__banManager.size(), self._jail.name)
+		return cnt
 
-	def __flushBan(self):
+	def __flushBan(self, db=False, actions=None):
 		"""Flush the ban list.
 
 		Unban all IP address which are still in the banning list.
-		"""
-		logSys.debug("Flush ban list")
-		for ticket in self.__banManager.flushBanList():
-			self.__unBan(ticket)
 
-	def __unBan(self, ticket):
+		If actions specified, don't flush list - just execute unban for 
+		given actions (reload, obsolete resp. removed actions).
+		"""
+		if actions is None:
+			logSys.debug("Flush ban list")
+			lst = self.__banManager.flushBanList()
+		else:
+			lst = iter(self.__banManager)
+		cnt = 0
+		for ticket in lst:
+			# delete ip from database also:
+			if db and self._jail.database is not None:
+				ip = str(ticket.getIP())
+				self._jail.database.delBan(self._jail, ip)
+			# unban ip:
+			self.__unBan(ticket, actions=actions)
+			cnt += 1
+		logSys.debug("Unbanned %s, %s ticket(s) in %r", 
+			cnt, self.__banManager.size(), self._jail.name)
+		return cnt
+
+	def __unBan(self, ticket, actions=None):
 		"""Unbans host corresponding to the ticket.
 
 		Executes the actions in order to unban the host given in the
@@ -356,14 +445,20 @@ class Actions(JailThread, Mapping):
 		ticket : FailTicket
 			Ticket of failures of which to unban
 		"""
+		if actions is None:
+			unbactions = self._actions
+		else:
+			unbactions = actions
 		aInfo = dict()
 		aInfo["ip"] = ticket.getIP()
 		aInfo["failures"] = ticket.getAttempt()
 		aInfo["time"] = ticket.getTime()
 		aInfo["matches"] = "".join(ticket.getMatches())
-		logSys.notice("[%s] Unban %s" % (self._jail.name, aInfo["ip"]))
+		if actions is None:
-		for name, action in self._actions.iteritems():
+			logSys.notice("[%s] Unban %s", self._jail.name, aInfo["ip"])
+		for name, action in unbactions.iteritems():
 			try:
+				logSys.debug("[%s] action %r: unban %s", self._jail.name, name, aInfo["ip"])
 				action.unban(aInfo.copy())
 			except Exception as e:
 				logSys.error(

fail2ban/server/asyncserver.py

@@ -67,23 +67,28 @@ class RequestHandler(asynchat.async_chat):
 	# This method is called once we have a complete request.
 
 	def found_terminator(self):
-		# Pop whole buffer
+		try:
-		message = self.__buffer
+			# Pop whole buffer
-		self.__buffer = []		
+			message = self.__buffer
-		# Joins the buffer items.
+			self.__buffer = []		
-		message = CSPROTO.EMPTY.join(message)
+			# Joins the buffer items.
-		# Closes the channel if close was received
+			message = CSPROTO.EMPTY.join(message)
-		if message == CSPROTO.CLOSE:
+			# Closes the channel if close was received
-			self.close_when_done()
+			if message == CSPROTO.CLOSE:
-			return
+				self.close_when_done()
-		# Deserialize
+				return
-		message = loads(message)
+			# Deserialize
-		# Gives the message to the transmitter.
+			message = loads(message)
-		message = self.__transmitter.proceed(message)
+			# Gives the message to the transmitter.
-		# Serializes the response.
+			message = self.__transmitter.proceed(message)
-		message = dumps(message, HIGHEST_PROTOCOL)
+			# Serializes the response.
-		# Sends the response to the client.
+			message = dumps(message, HIGHEST_PROTOCOL)
-		self.push(message + CSPROTO.END)
+			# Sends the response to the client.
+			self.push(message + CSPROTO.END)
+		except Exception as e: # pragma: no cover
+			logSys.error("Caught unhandled exception: %r", e,
+				exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
+
 		
 	def handle_error(self):
 		e1, e2 = formatExceptionInfo()
@@ -199,6 +204,7 @@ class AsyncServer(asyncore.dispatcher):
 
 
 	def close(self):
+		stopflg = False
 		if self.__active:
 			self.__loop = False
 			asyncore.dispatcher.close(self)
@@ -206,11 +212,13 @@ class AsyncServer(asyncore.dispatcher):
 			# for the server leaves loop, before remove socket
 			if threading.current_thread() != self.__worker:
 				Utils.wait_for(lambda: not self.__active, 1)
+			stopflg = True
 		# Remove socket (file) only if it was created:
 		if self.__init and os.path.exists(self.__sock):
 			self._remove_sock()
 			logSys.debug("Removed socket file " + self.__sock)
-		logSys.debug("Socket shutdown")
+		if stopflg:
+			logSys.debug("Socket shutdown")
 		self.__active = False
 
 	##

fail2ban/server/banmanager.py

@@ -51,11 +51,13 @@ class BanManager:
 		## Mutex used to protect the ban list.
 		self.__lock = Lock()
 		## The ban list.
-		self.__banList = list()
+		self.__banList = dict()
 		## The amount of time an IP address gets banned.
 		self.__banTime = 600
 		## Total number of banned IP address
 		self.__banTotal = 0
+		## The time for next unban process (for performance and load reasons):
+		self.__nextUnbanTime = BanTicket.MAX_TIME
 	
 	##
 	# Set the ban time.
@@ -64,11 +66,8 @@ class BanManager:
 	# @param value the time
 	
 	def setBanTime(self, value):
-		try:
+		with self.__lock:
-			self.__lock.acquire()
 			self.__banTime = int(value)
-		finally:
-			self.__lock.release()
 	
 	##
 	# Get the ban time.
@@ -77,11 +76,8 @@ class BanManager:
 	# @return the time
 	
 	def getBanTime(self):
-		try:
+		with self.__lock:
-			self.__lock.acquire()
 			return self.__banTime
-		finally:
-			self.__lock.release()
 	
 	##
 	# Set the total number of banned address.
@@ -89,11 +85,8 @@ class BanManager:
 	# @param value total number
 	
 	def setBanTotal(self, value):
-		try:
+		with self.__lock:
-			self.__lock.acquire()
 			self.__banTotal = value
-		finally:
-			self.__lock.release()
 	
 	##
 	# Get the total number of banned address.
@@ -101,11 +94,8 @@ class BanManager:
 	# @return the total number
 	
 	def getBanTotal(self):
-		try:
+		with self.__lock:
-			self.__lock.acquire()
 			return self.__banTotal
-		finally:
-			self.__lock.release()
 
 	##
 	# Returns a copy of the IP list.
@@ -113,11 +103,17 @@ class BanManager:
 	# @return IP list
 	
 	def getBanList(self):
-		try:
+		with self.__lock:
-			self.__lock.acquire()
+			return self.__banList.keys()
-			return [m.getIP() for m in self.__banList]
+
-		finally:
+	##
-			self.__lock.release()
+	# Returns a iterator to ban list (used in reload, so idle).
+	#
+	# @return ban list iterator
+	
+	def __iter__(self):
+		with self.__lock:
+			return self.__banList.itervalues()
 
 	##
 	# Returns normalized value
@@ -149,7 +145,7 @@ class BanManager:
 			return return_dict
 		self.__lock.acquire()
 		try:
-			for banData in self.__banList:
+			for banData in self.__banList.values():
 				ip = banData.getIP()
 				# Reference: http://www.team-cymru.org/Services/ip-to-asn.html#dns
 				question = ip.getPTR(
@@ -260,30 +256,33 @@ class BanManager:
 	# @param ticket the ticket
 	# @return True if the IP address is not in the ban list
 	
-	def addBanTicket(self, ticket):
+	def addBanTicket(self, ticket, reason={}):
-		try:
+		eob = ticket.getEndOfBanTime(self.__banTime)
-			self.__lock.acquire()
+		with self.__lock:
 			# check already banned
-			for oldticket in self.__banList:
+			fid = ticket.getID()
-				if ticket.getIP() == oldticket.getIP():
+			oldticket = self.__banList.get(fid)
-					# if already permanent
+			if oldticket:
-					btold, told = oldticket.getBanTime(self.__banTime), oldticket.getTime()
+				reason['ticket'] = oldticket
-					if btold == -1:
+				# if new time for end of ban is larger than already banned end-time:
-						return False
+				if eob > oldticket.getEndOfBanTime(self.__banTime):
-					# if given time is less than already banned time
-					btnew, tnew = ticket.getBanTime(self.__banTime), ticket.getTime()
-					if btnew != -1 and tnew + btnew <= told + btold:
-						return False
 					# we have longest ban - set new (increment) ban time
-					oldticket.setTime(tnew)
+					reason['prolong'] = 1
-					oldticket.setBanTime(btnew)
+					btm = ticket.getBanTime(self.__banTime)
-					return False
+					# if not permanent:
-			# not yet banned - add new
+					if btm != -1:
-			self.__banList.append(ticket)
+						diftm = ticket.getTime() - oldticket.getTime()
+						if diftm > 0:
+							btm += diftm
+					oldticket.setBanTime(btm)
+				return False
+			# not yet banned - add new one:
+			self.__banList[fid] = ticket
 			self.__banTotal += 1
+			# correct next unban time:
+			if self.__nextUnbanTime > eob:
+				self.__nextUnbanTime = eob
 			return True
-		finally:
-			self.__lock.release()
 
 	##
 	# Get the size of the ban list.
@@ -291,11 +290,7 @@ class BanManager:
 	# @return the size
 
 	def size(self):
-		try:
+		return len(self.__banList)
-			self.__lock.acquire()
-			return len(self.__banList)
-		finally:
-			self.__lock.release()
 
 	##
 	# Check if a ticket is in the list.
@@ -306,10 +301,7 @@ class BanManager:
 	# @return True if a ticket already exists
 	
 	def _inBanList(self, ticket):
-		for i in self.__banList:
+		return ticket.getID() in self.__banList
-			if ticket.getIP() == i.getIP():
-				return True
-		return False
 	
 	##
 	# Get the list of IP address to unban.
@@ -319,22 +311,39 @@ class BanManager:
 	# @return the list of ticket to unban
 	
 	def unBanList(self, time):
-		try:
+		with self.__lock:
-			self.__lock.acquire()
 			# Permanent banning
 			if self.__banTime < 0:
 				return list()
 
-			# Gets the list of ticket to remove.
+			# Check next unban time:
-			unBanList = [ticket for ticket in self.__banList if ticket.isTimedOut(time, self.__banTime)]
+			if self.__nextUnbanTime > time:
-			
+				return list()
+
+			# Gets the list of ticket to remove (thereby correct next unban time).
+			unBanList = {}
+			self.__nextUnbanTime = BanTicket.MAX_TIME
+			for fid,ticket in self.__banList.iteritems():
+				# current time greater as end of ban - timed out:
+				eob = ticket.getEndOfBanTime(self.__banTime)
+				if time > eob:
+					unBanList[fid] = ticket
+				elif self.__nextUnbanTime > eob:
+					self.__nextUnbanTime = eob
+
 			# Removes tickets.
-			self.__banList = [ticket for ticket in self.__banList
+			if len(unBanList):
-							  if ticket not in unBanList]
+				if len(unBanList) / 2.0 <= len(self.__banList) / 3.0:
+					# few as 2/3 should be removed - remove particular items:
+					for fid in unBanList.iterkeys():
+						del self.__banList[fid]
+				else:
+					# create new dictionary without items to be deleted:
+					self.__banList = dict((fid,ticket) for fid,ticket in self.__banList.iteritems() \
+						if fid not in unBanList)
 						
-			return unBanList
+			# return list of tickets:
-		finally:
+			return unBanList.values()
-			self.__lock.release()
 
 	##
 	# Flush the ban list.
@@ -343,28 +352,21 @@ class BanManager:
 	# @return the complete ban list
 	
 	def flushBanList(self):
-		try:
+		with self.__lock:
-			self.__lock.acquire()
+			uBList = self.__banList.values()
-			uBList = self.__banList
+			self.__banList = dict()
-			self.__banList = list()
 			return uBList
-		finally:
-			self.__lock.release()
 
 	##
-	# Gets the ticket for the specified IP.
+	# Gets the ticket for the specified ID (most of the time it is IP-address).
 	#
-	# @return the ticket for the IP or False.
+	# @return the ticket or False.
-	def getTicketByIP(self, ip):
+	def getTicketByID(self, fid):
-		try:
+		with self.__lock:
-			self.__lock.acquire()
+			try:
-
+				# Return the ticket after removing (popping)
-			# Find the ticket the IP goes with and return it
+				# if from the ban list.
-			for i, ticket in enumerate(self.__banList):
+				return self.__banList.pop(fid)
-				if ticket.getIP() == ip:
+			except KeyError:
-					# Return the ticket after removing (popping)
+				pass
-					# if from the ban list.
-					return self.__banList.pop(i)
-		finally:
-			self.__lock.release()
 		return None						  # if none found

fail2ban/server/database.py

@@ -22,7 +22,6 @@ __copyright__ = "Copyright (c) 2013 Steven Hiscocks"
 __license__ = "GPL"
 
 import json
-import locale
 import shutil
 import sqlite3
 import sys
@@ -32,7 +31,7 @@ from threading import RLock
 
 from .mytime import MyTime
 from .ticket import FailTicket
-from ..helpers import getLogger
+from ..helpers import getLogger, PREFER_ENC
 
 # Gets the instance of the logger.
 logSys = getLogger(__name__)
@@ -41,7 +40,7 @@ if sys.version_info >= (3,):
 	def _json_dumps_safe(x):
 		try:
 			x = json.dumps(x, ensure_ascii=False).encode(
-				locale.getpreferredencoding(), 'replace')
+				PREFER_ENC, 'replace')
 		except Exception as e: # pragma: no cover
 			logSys.error('json dumps failed: %s', e)
 			x = '{}'
@@ -50,7 +49,7 @@ if sys.version_info >= (3,):
 	def _json_loads_safe(x):
 		try:
 			x = json.loads(x.decode(
-				locale.getpreferredencoding(), 'replace'))
+				PREFER_ENC, 'replace'))
 		except Exception as e: # pragma: no cover
 			logSys.error('json loads failed: %s', e)
 			x = {}
@@ -62,14 +61,14 @@ else:
 		elif isinstance(x, list):
 			return [_normalize(element) for element in x]
 		elif isinstance(x, unicode):
-			return x.encode(locale.getpreferredencoding())
+			return x.encode(PREFER_ENC)
 		else:
 			return x
 
 	def _json_dumps_safe(x):
 		try:
 			x = json.dumps(_normalize(x), ensure_ascii=False).decode(
-				locale.getpreferredencoding(), 'replace')
+				PREFER_ENC, 'replace')
 		except Exception as e: # pragma: no cover
 			logSys.error('json dumps failed: %s', e)
 			x = '{}'
@@ -78,7 +77,7 @@ else:
 	def _json_loads_safe(x):
 		try:
 			x = _normalize(json.loads(x.decode(
-				locale.getpreferredencoding(), 'replace')))
+				PREFER_ENC, 'replace')))
 		except Exception as e: # pragma: no cover
 			logSys.error('json loads failed: %s', e)
 			x = {}
@@ -212,7 +211,7 @@ class Fail2BanDb(object):
 				if newversion == Fail2BanDb.__version__:
 					logSys.warning( "Database updated from '%i' to '%i'",
 						version, newversion)
-				else:
+				else: # pragma: no cover
 					logSys.error( "Database update failed to achieve version '%i'"
 						": updated from '%i' to '%i'",
 						Fail2BanDb.__version__, version, newversion)
@@ -223,6 +222,11 @@ class Fail2BanDb(object):
 				cur.execute("PRAGMA journal_mode = MEMORY")
 			cur.close()
 
+	def close(self):
+		logSys.debug("Close connection to database ...")
+		self._db.close()
+		logSys.info("Connection to database closed.")
+
 	@property
 	def filename(self):
 		"""File name of SQLite3 database file.
@@ -477,7 +481,8 @@ class Fail2BanDb(object):
 			queryArgs.append(str(ip))
 		query += " ORDER BY ip, timeofban desc"
 
-		return cur.execute(query, queryArgs)
+		# repack iterator as long as in lock:
+		return list(cur.execute(query, queryArgs))
 
 	def getBans(self, **kwargs):
 		"""Get bans from the database.
@@ -576,6 +581,43 @@ class Fail2BanDb(object):
 				self._bansMergedCache[cacheKey] = tickets if ip is None else ticket
 			return tickets if ip is None else ticket
 
+	def _getCurrentBans(self, cur, jail = None, ip = None, forbantime=None, fromtime=None):
+		if fromtime is None:
+			fromtime = MyTime.time()
+		queryArgs = []
+		if jail is not None:
+			query = "SELECT ip, timeofban, data FROM bans WHERE jail=?"
+			queryArgs.append(jail.name)
+		else:
+			query = "SELECT ip, max(timeofban), data FROM bans WHERE 1"
+		if ip is not None:
+			query += " AND ip=?"
+			queryArgs.append(ip)
+		if forbantime is not None:
+			query += " AND timeofban > ?"
+			queryArgs.append(fromtime - forbantime)
+		if ip is None:
+			query += " GROUP BY ip ORDER BY ip, timeofban DESC"
+		cur = self._db.cursor()
+		return cur.execute(query, queryArgs)
+
+	def getCurrentBans(self, jail = None, ip = None, forbantime=None, fromtime=None):
+		tickets = []
+		ticket = None
+
+		with self._lock:
+			results = list(self._getCurrentBans(self._db.cursor(), 
+				jail=jail, ip=ip, forbantime=forbantime, fromtime=fromtime))
+
+		if results:
+			for banip, timeofban, data in results:
+				# logSys.debug('restore ticket   %r, %r, %r', banip, timeofban, data)
+				ticket = FailTicket(banip, timeofban, data=data)
+				# logSys.debug('restored ticket: %r', ticket)
+				tickets.append(ticket)
+
+		return tickets if ip is None else ticket
+
 	@commitandrollback
 	def purge(self, cur):
 		"""Purge old bans, jails and log files from database.

fail2ban/server/failregex.py

@@ -40,11 +40,11 @@ class Regex:
 	# avoid construction of invalid object.
 	# @param value the regular expression
 	
-	def __init__(self, regex):
+	def __init__(self, regex, **kwargs):
 		self._matchCache = None
 		# Perform shortcuts expansions.
 		# Resolve "<HOST>" tag using default regular expression for host:
-		regex = Regex._resolveHostTag(regex)
+		regex = Regex._resolveHostTag(regex, **kwargs)
 		# Replace "<SKIPLINES>" with regular expression for multiple lines.
 		regexSplit = regex.split("<SKIPLINES>")
 		regex = regexSplit[0]
@@ -69,22 +69,29 @@ class Regex:
 	# @return the replaced regular expression as string
 
 	@staticmethod
-	def _resolveHostTag(regex):
+	def _resolveHostTag(regex, useDns="yes"):
-		# 3 groups instead of <HOST> - separated ipv4, ipv6 and host
-		regex = regex.replace("<HOST>",
-		 	r"""(?:(?:::f{4,6}:)?(?P<ip4>(?:\d{1,3}\.){3}\d{1,3})|\[?(?P<ip6>(?:[0-9a-fA-F]{1,4}::?|::){1,7}(?:[0-9a-fA-F]{1,4}|(?<=:):))\]?|(?P<dns>[\w\-.^_]*\w))""")
 		# separated ipv4:
+		r_host = []
 		r = r"""(?:::f{4,6}:)?(?P<ip4>(?:\d{1,3}\.){3}\d{1,3})"""
 		regex = regex.replace("<IP4>", r); # self closed
 		regex = regex.replace("<F-IP4/>", r); # closed
+		r_host.append(r)
 		# separated ipv6:
 		r = r"""(?P<ip6>(?:[0-9a-fA-F]{1,4}::?|::){1,7}(?:[0-9a-fA-F]{1,4}?|(?<=:):))"""
 		regex = regex.replace("<IP6>", r); # self closed
 		regex = regex.replace("<F-IP6/>", r); # closed
+		r_host.append(r"""\[?%s\]?""" % (r,)); # enclose ipv6 in optional [] in host-regex
+		# 2 address groups instead of <ADDR> - in opposition to `<HOST>`, 
+		# for separate usage of 2 address groups only (regardless of `usedns`), `ip4` and `ip6` together
+		regex = regex.replace("<ADDR>", "(?:%s)" % ("|".join(r_host),))
 		# separated dns:
 		r = r"""(?P<dns>[\w\-.^_]*\w)"""
 		regex = regex.replace("<DNS>", r); # self closed
 		regex = regex.replace("<F-DNS/>", r); # closed
+		if useDns not in ("no",):
+			r_host.append(r)
+		# 3 groups instead of <HOST> - separated ipv4, ipv6 and host (dns)
+		regex = regex.replace("<HOST>", "(?:%s)" % ("|".join(r_host),))
 		# default failure-id as no space tag:
 		regex = regex.replace("<F-ID/>", r"""(?P<fid>\S+)"""); # closed
 		# default failure port, like 80 or http :
@@ -249,9 +256,9 @@ class FailRegex(Regex):
 	# avoid construction of invalid object.
 	# @param value the regular expression
 
-	def __init__(self, regex):
+	def __init__(self, regex, **kwargs):
 		# Initializes the parent.
-		Regex.__init__(self, regex)
+		Regex.__init__(self, regex, **kwargs)
 		# Check for group "dns", "ip4", "ip6", "fid"
 		if not [grp for grp in FAILURE_ID_GROPS if grp in self._regexObj.groupindex]:
 			raise RegexException("No failure-id group in '%s'" % self._regex)

fail2ban/server/filter.py

@@ -24,7 +24,6 @@ __license__ = "GPL"
 import codecs
 import datetime
 import fcntl
-import locale
 import logging
 import os
 import re
@@ -40,7 +39,7 @@ from .datetemplate import DatePatternRegex, DateEpoch, DateTai64n
 from .mytime import MyTime
 from .failregex import FailRegex, Regex, RegexException
 from .action import CommandAction
-from ..helpers import getLogger
+from ..helpers import getLogger, PREFER_ENC
 
 # Gets the instance of the logger.
 logSys = getLogger(__name__)
@@ -87,9 +86,10 @@ class Filter(JailThread):
 		## External command
 		self.__ignoreCommand = False
 		## Default or preferred encoding (to decode bytes from file or journal):
-		self.__encoding = locale.getpreferredencoding()
+		self.__encoding = PREFER_ENC
-		## Error counter
+		## Error counter (protected, so can be used in filter implementations)
-		self.__errors = 0
+		## if it reached 100 (at once), run-cycle will go idle
+		self._errors = 0
 		## Ticks counter
 		self.ticks = 0
 
@@ -100,6 +100,31 @@ class Filter(JailThread):
 	def __repr__(self):
 		return "%s(%r)" % (self.__class__.__name__, self.jail)
 
+	@property
+	def jailName(self):
+		return (self.jail is not None and self.jail.name or "~jailless~")
+
+	def clearAllParams(self):
+		""" Clear all lists/dicts parameters (used by reloading)
+		"""
+		self.delFailRegex()
+		self.delIgnoreRegex()
+		self.delIgnoreIP()
+
+	def reload(self, begin=True):
+		""" Begin or end of reloading resp. refreshing of all parameters
+		"""
+		if begin:
+			self.clearAllParams()
+			if hasattr(self, 'getLogPaths'):
+				self._reload_logs = dict((k, 1) for k in self.getLogPaths())
+		else:
+			if hasattr(self, '_reload_logs'):
+				# if it was not reloaded - remove obsolete log file:
+				for path in self._reload_logs:
+					self.delLogPath(path)
+				delattr(self, '_reload_logs')
+
 	##
 	# Add a regular expression which matches the failure.
 	#
@@ -109,18 +134,23 @@ class Filter(JailThread):
 
 	def addFailRegex(self, value):
 		try:
-			regex = FailRegex(value)
+			regex = FailRegex(value, useDns=self.__useDns)
 			self.__failRegex.append(regex)
 			if "\n" in regex.getRegex() and not self.getMaxLines() > 1:
 				logSys.warning(
-					"Mutliline regex set for jail '%s' "
+					"Mutliline regex set for jail %r "
-					"but maxlines not greater than 1")
+					"but maxlines not greater than 1", self.jailName)
 		except RegexException as e:
 			logSys.error(e)
 			raise e
 
-	def delFailRegex(self, index):
+	def delFailRegex(self, index=None):
 		try:
+			# clear all:
+			if index is None:
+				del self.__failRegex[:]
+				return
+			# delete by index:
 			del self.__failRegex[index]
 		except IndexError:
 			logSys.error("Cannot remove regular expression. Index %d is not "
@@ -146,14 +176,19 @@ class Filter(JailThread):
 
 	def addIgnoreRegex(self, value):
 		try:
-			regex = Regex(value)
+			regex = Regex(value, useDns=self.__useDns)
 			self.__ignoreRegex.append(regex)
 		except RegexException as e:
 			logSys.error(e)
 			raise e 
 
-	def delIgnoreRegex(self, index):
+	def delIgnoreRegex(self, index=None):
 		try:
+			# clear all:
+			if index is None:
+				del self.__ignoreRegex[:]
+				return
+			# delete by index:
 			del self.__ignoreRegex[index]
 		except IndexError:
 			logSys.error("Cannot remove regular expression. Index %d is not "
@@ -203,7 +238,7 @@ class Filter(JailThread):
 		value = MyTime.str2seconds(value)
 		self.__findTime = value
 		self.failManager.setMaxTime(value)
-		logSys.info("Set findtime = %s" % value)
+		logSys.info("  findtime: %s", value)
 
 	##
 	# Get the time needed to find a failure.
@@ -232,10 +267,10 @@ class Filter(JailThread):
 			template = DatePatternRegex(pattern)
 		self.dateDetector = DateDetector()
 		self.dateDetector.appendTemplate(template)
-		logSys.info("Date pattern set to `%r`: `%s`" %
+		logSys.info("  date pattern `%r`: `%s`",
-			(pattern, template.name))
+			pattern, template.name)
-		logSys.debug("Date pattern regex for %r: %s" %
+		logSys.debug("  date pattern regex for %r: %s",
-			(pattern, template.regex))
+			pattern, template.regex)
 
 	##
 	# Get the date detector pattern, or Default Detectors if not changed
@@ -261,7 +296,7 @@ class Filter(JailThread):
 
 	def setMaxRetry(self, value):
 		self.failManager.setMaxRetry(value)
-		logSys.info("Set maxRetry = %s" % value)
+		logSys.info("  maxRetry: %s", value)
 
 	##
 	# Get the maximum retry value.
@@ -280,7 +315,7 @@ class Filter(JailThread):
 		if int(value) <= 0:
 			raise ValueError("maxlines must be integer greater than zero")
 		self.__lineBufferSize = int(value)
-		logSys.info("Set maxlines = %i" % self.__lineBufferSize)
+		logSys.info("  maxLines: %i", self.__lineBufferSize)
 
 	##
 	# Get the maximum line buffer size.
@@ -297,10 +332,10 @@ class Filter(JailThread):
 
 	def setLogEncoding(self, encoding):
 		if encoding.lower() == "auto":
-			encoding = locale.getpreferredencoding()
+			encoding = PREFER_ENC
 		codecs.lookup(encoding) # Raise LookupError if invalid codec
 		self.__encoding = encoding
-		logSys.info("Set jail log file encoding to %s" % encoding)
+		logSys.info("  encoding: %s" % encoding)
 		return encoding
 
 	##
@@ -375,16 +410,21 @@ class Filter(JailThread):
 		ip = IPAddr(ipstr)
 
 		# log and append to ignore list
-		logSys.debug("Add %r to ignore list (%r)", ip, ipstr)
+		logSys.debug("  Add %r to ignore list (%r)", ip, ipstr)
 		self.__ignoreIpList.append(ip)
 
-	def delIgnoreIP(self, ip):
+	def delIgnoreIP(self, ip=None):
-		logSys.debug("Remove %r from ignore list", ip)
+		# clear all:
+		if ip is None:
+			del self.__ignoreIpList[:]
+			return
+		# delete by ip:
+		logSys.debug("  Remove %r from ignore list", ip)
 		self.__ignoreIpList.remove(ip)
 
 	def logIgnoreIp(self, ip, log_ignore, ignore_source="unknown source"):
 		if log_ignore:
-			logSys.info("[%s] Ignore %s by %s" % (self.jail.name, ip, ignore_source))
+			logSys.info("[%s] Ignore %s by %s" % (self.jailName, ip, ignore_source))
 
 	def getIgnoreIP(self):
 		return self.__ignoreIpList
@@ -415,29 +455,6 @@ class Filter(JailThread):
 
 		return False
 
-	if sys.version_info >= (3,):
-		@staticmethod
-		def uni_decode(x, enc, errors='strict'):
-			try:
-				if isinstance(x, bytes):
-					return x.decode(enc, errors)
-				return x
-			except (UnicodeDecodeError, UnicodeEncodeError): # pragma: no cover - unsure if reachable
-				if errors != 'strict': 
-					raise
-				return uni_decode(x, enc, 'replace')
-	else:
-		@staticmethod
-		def uni_decode(x, enc, errors='strict'):
-			try:
-				if isinstance(x, unicode):
-					return x.encode(enc, errors)
-				return x
-			except (UnicodeDecodeError, UnicodeEncodeError): # pragma: no cover - unsure if reachable
-				if errors != 'strict':
-					raise
-				return uni_decode(x, enc, 'replace')
-
 	def processLine(self, line, date=None, returnRawHost=False,
 		checkAllRegex=False, checkFindTime=False):
 		"""Split the time portion from log msg and return findFailures on them
@@ -478,24 +495,28 @@ class Filter(JailThread):
 				if self.inIgnoreIPList(ip, log_ignore=True):
 					continue
 				logSys.info(
-					"[%s] Found %s - %s", self.jail.name, ip, datetime.datetime.fromtimestamp(unixTime).strftime("%Y-%m-%d %H:%M:%S")
+					"[%s] Found %s - %s", self.jailName, ip, datetime.datetime.fromtimestamp(unixTime).strftime("%Y-%m-%d %H:%M:%S")
 				)
 				tick = FailTicket(ip, unixTime, lines, data=fail)
 				self.failManager.addFailure(tick)
 			# reset (halve) error counter (successfully processed line):
-			if self.__errors:
+			if self._errors:
-				self.__errors //= 2
+				self._errors //= 2
 		except Exception as e:
 			logSys.error("Failed to process line: %r, caught exception: %r", line, e,
 				exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
-			# incr error counter, stop processing (going idle) after 100th error :
+			# incr common error counter:
-			self.__errors += 1
+			self.commonError()
-			# sleep a little bit (to get around time-related errors):
+
-			time.sleep(self.sleeptime)
+	def commonError(self):
-			if self.__errors >= 100:
+		# incr error counter, stop processing (going idle) after 100th error :
-				logSys.error("Too many errors at once (%s), going idle", self.__errors)
+		self._errors += 1
-				self.__errors //= 2
+		# sleep a little bit (to get around time-related errors):
-				self.idle = True
+		time.sleep(self.sleeptime)
+		if self._errors >= 100:
+			logSys.error("Too many errors at once (%s), going idle", self._errors)
+			self._errors //= 2
+			self.idle = True
 
 	##
 	# Returns true if the line should be ignored.
@@ -657,7 +678,10 @@ class FileFilter(Filter):
 
 	def addLogPath(self, path, tail=False, autoSeek=True):
 		if path in self.__logs:
-			logSys.error(path + " already exists")
+			if hasattr(self, '_reload_logs') and path in self._reload_logs:
+				del self._reload_logs[path]
+			else:
+				logSys.error(path + " already exists")
 		else:
 			log = FileContainer(path, self.getLogEncoding(), tail)
 			db = self.jail.database
@@ -666,7 +690,7 @@ class FileFilter(Filter):
 				if lastpos and not tail:
 					log.setPos(lastpos)
 			self.__logs[path] = log
-			logSys.info("Added logfile = %s (pos = %s, hash = %s)" , path, log.getPos(), log.getHash())
+			logSys.info("Added logfile: %r (pos = %s, hash = %s)" , path, log.getPos(), log.getHash())
 			if autoSeek:
 				# if default, seek to "current time" - "find time":
 				if isinstance(autoSeek, bool):
@@ -692,7 +716,7 @@ class FileFilter(Filter):
 		db = self.jail.database
 		if db is not None:
 			db.updateLog(self.jail, log)
-		logSys.info("Removed logfile = %s" % path)
+		logSys.info("Removed logfile: %r" % path)
 		self._delLogPath(path)
 		return
 
@@ -759,49 +783,48 @@ class FileFilter(Filter):
 		if log is None:
 			logSys.error("Unable to get failures in " + filename)
 			return False
-		# Try to open log file.
+		# We should always close log (file), otherwise may be locked (log-rotate, etc.)
 		try:
-			has_content = log.open()
+			# Try to open log file.
-		# see http://python.org/dev/peps/pep-3151/
-		except IOError as e:
-			logSys.error("Unable to open %s" % filename)
-			logSys.exception(e)
-			return False
-		except OSError as e: # pragma: no cover - requires race condition to tigger this
-			logSys.error("Error opening %s" % filename)
-			logSys.exception(e)
-			return False
-		except Exception as e: # pragma: no cover - Requires implemention error in FileContainer to generate
-			logSys.error("Internal error in FileContainer open method - please report as a bug to https://github.com/fail2ban/fail2ban/issues")
-			logSys.exception(e)
-			return False
-
-		# seek to find time for first usage only (prevent performance decline with polling of big files)
-		if self.__autoSeek.get(filename):
-			startTime = self.__autoSeek[filename]
-			del self.__autoSeek[filename]
-			# prevent completely read of big files first time (after start of service), 
-			# initial seek to start time using half-interval search algorithm:
 			try:
-				self.seekToTime(log, startTime)
+				has_content = log.open()
-			except Exception as e: # pragma: no cover
+			# see http://python.org/dev/peps/pep-3151/
-				logSys.error("Error during seek to start time in \"%s\"", filename)
+			except IOError as e:
-				raise
+				logSys.error("Unable to open %s" % filename)
+				logSys.exception(e)
+				return False
+			except OSError as e: # pragma: no cover - requires race condition to tigger this
+				logSys.error("Error opening %s" % filename)
+				logSys.exception(e)
+				return False
+			except Exception as e: # pragma: no cover - Requires implemention error in FileContainer to generate
+				logSys.error("Internal error in FileContainer open method - please report as a bug to https://github.com/fail2ban/fail2ban/issues")
 				logSys.exception(e)
 				return False
 
-		# yoh: has_content is just a bool, so do not expect it to
+			# seek to find time for first usage only (prevent performance decline with polling of big files)
-		# change -- loop is exited upon break, and is not entered at
+			if self.__autoSeek.get(filename):
-		# all if upon container opening that one was empty.  If we
+				startTime = self.__autoSeek[filename]
-		# start reading tested to be empty container -- race condition
+				del self.__autoSeek[filename]
-		# might occur leading at least to tests failures.
+				# prevent completely read of big files first time (after start of service), 
-		while has_content:
+				# initial seek to start time using half-interval search algorithm:
-			line = log.readline()
+				try:
-			if not line or not self.active:
+					self.seekToTime(log, startTime)
-				# The jail reached the bottom or has been stopped
+				except Exception as e: # pragma: no cover
-				break
+					logSys.error("Error during seek to start time in \"%s\"", filename)
-			self.processLineAndAdd(line)
+					raise
-		log.close()
+					logSys.exception(e)
+					return False
+
+			if has_content:
+				while not self.idle:
+					line = log.readline()
+					if not line or not self.active:
+						# The jail reached the bottom or has been stopped
+						break
+					self.processLineAndAdd(line)
+		finally:
+			log.close()
 		db = self.jail.database
 		if db is not None:
 			db.updateLog(self.jail, log)
@@ -1055,10 +1078,14 @@ _decode_line_warn = {}
 
 class JournalFilter(Filter): # pragma: systemd no cover
 
+	def clearAllParams(self):
+		super(JournalFilter, self).clearAllParams()
+		self.delJournalMatch()
+
 	def addJournalMatch(self, match): # pragma: no cover - Base class, not used
 		pass
 
-	def delJournalMatch(self, match): # pragma: no cover - Base class, not used
+	def delJournalMatch(self, match=None): # pragma: no cover - Base class, not used
 		pass
 
 	def getJournalMatch(self, match): # pragma: no cover - Base class, not used

fail2ban/server/filtergamin.py

@@ -124,14 +124,15 @@ class FilterGamin(FileFilter):
 		while self.active:
 			if self.idle:
 				# wait a little bit here for not idle, to prevent hi-load:
-				if not Utils.wait_for(lambda: not self.idle, 
+				if not Utils.wait_for(lambda: not self.active or not self.idle,
 					self.sleeptime * 10, self.sleeptime
 				):
 					self.ticks += 1
 					continue
-			Utils.wait_for(self._handleEvents, self.sleeptime)
+			Utils.wait_for(lambda: not self.active or self._handleEvents(),
+				self.sleeptime)
 			self.ticks += 1
-		logSys.debug(self.jail.name + ": filter terminated")
+		logSys.debug("[%s] filter terminated", self.jailName)
 		return True
 
 	def stop(self):

fail2ban/server/filterpoll.py

@@ -31,7 +31,7 @@ from .failmanager import FailManagerEmpty
 from .filter import FileFilter
 from .mytime import MyTime
 from .utils import Utils
-from ..helpers import getLogger
+from ..helpers import getLogger, logging
 
 
 # Gets the instance of the logger.
@@ -101,14 +101,15 @@ class FilterPoll(FileFilter):
 				logSys.log(6, "Woke up idle=%s with %d files monitored",
 						   self.idle, self.getLogCount())
 			if self.idle:
-				if not Utils.wait_for(lambda: not self.idle, 
+				if not Utils.wait_for(lambda: not self.active or not self.idle, 
 					self.sleeptime * 10, self.sleeptime
 				):
 					self.ticks += 1
 					continue
 			# Get file modification
 			modlst = []
-			Utils.wait_for(lambda: self.getModified(modlst), self.sleeptime)
+			Utils.wait_for(lambda: not self.active or self.getModified(modlst),
+				self.sleeptime)
 			for filename in modlst:
 				self.getFailures(filename)
 				self.__modified = True
@@ -122,9 +123,7 @@ class FilterPoll(FileFilter):
 				except FailManagerEmpty:
 					self.failManager.cleanup(MyTime.time())
 				self.__modified = False
-		logSys.debug(
+		logSys.debug("[%s] filter terminated", self.jailName)
-			(self.jail is not None and self.jail.name or "jailless") +
-					 " filter terminated")
 		return True
 
 	##
@@ -137,28 +136,34 @@ class FilterPoll(FileFilter):
 		try:
 			logStats = os.stat(filename)
 			stats = logStats.st_mtime, logStats.st_ino, logStats.st_size
-			pstats = self.__prevStats.get(filename, ())
+			pstats = self.__prevStats.get(filename, (0))
-			self.__file404Cnt[filename] = 0
+			if logSys.getEffectiveLevel() <= 5:
-			if logSys.getEffectiveLevel() <= 7:
 				# we do not want to waste time on strftime etc if not necessary
 				dt = logStats.st_mtime - pstats[0]
-				logSys.log(7, "Checking %s for being modified. Previous/current stats: %s / %s. dt: %s",
+				logSys.log(5, "Checking %s for being modified. Previous/current stats: %s / %s. dt: %s",
 				           filename, pstats, stats, dt)
 				# os.system("stat %s | grep Modify" % filename)
+			self.__file404Cnt[filename] = 0
 			if pstats == stats:
 				return False
 			logSys.debug("%s has been modified", filename)
 			self.__prevStats[filename] = stats
 			return True
-		except OSError as e:
+		except Exception as e:
-			logSys.error("Unable to get stat on %s because of: %s"
+			# stil alive (may be deleted because multi-threaded):
-						 % (filename, e))
+			if not self.getLog(filename):
+				logSys.warning("Log %r seems to be down: %s", filename, e)
+				return
+			# log error:
+			if self.__file404Cnt[filename] < 2:
+				logSys.error("Unable to get stat on %s because of: %s",
+							 filename, e, 
+							 exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
+			# increase file and common error counters:
 			self.__file404Cnt[filename] += 1
-			if self.__file404Cnt[filename] > 2:
+			self.commonError()
-				logSys.warning("Too many errors. Setting the jail idle")
+			if self.__file404Cnt[filename] > 50:
-				if self.jail is not None:
+				logSys.warning("Too many errors. Remove file %r from monitoring process", filename)
-					self.jail.idle = True
-				else:
-					logSys.warning("No jail is assigned to %s" % self)
 				self.__file404Cnt[filename] = 0
+				self.delLogPath(filename)
 			return False

fail2ban/server/filterpyinotify.py

@@ -76,7 +76,7 @@ class FilterPyinotify(FileFilter):
 		logSys.debug("Created FilterPyinotify")
 
 	def callback(self, event, origin=''):
-		logSys.debug("%sCallback for Event: %s", origin, event)
+		logSys.log(7, "[%s] %sCallback for Event: %s", self.jailName, origin, event)
 		path = event.pathname
 		if event.mask & ( pyinotify.IN_CREATE | pyinotify.IN_MOVED_TO ):
 			# skip directories altogether
@@ -119,14 +119,15 @@ class FilterPyinotify(FileFilter):
 		logSys.debug("Added file watcher for %s", path)
 
 	def _delFileWatcher(self, path):
-		wdInt = self.__watches[path]
+		try:
-		wd = self.__monitor.rm_watch(wdInt)
+			wdInt = self.__watches.pop(path)
-		if wd[wdInt]:
+			wd = self.__monitor.rm_watch(wdInt)
-			del self.__watches[path]
+			if wd[wdInt]:
-			logSys.debug("Removed file watcher for %s", path)
+				logSys.debug("Removed file watcher for %s", path)
-			return True
+				return True
-		else:
+		except KeyError: # pragma: no cover
-			return False
+			pass
+		return False
 
 	##
 	# Add a log file path
@@ -158,8 +159,11 @@ class FilterPyinotify(FileFilter):
 					if k.startswith(path_dir + pathsep)]):
 			# Remove watches for the directory
 			# since there is no other monitored file under this directory
-			wdInt = self.__watches.pop(path_dir)
+			try:
-			self.__monitor.rm_watch(wdInt)
+				wdInt = self.__watches.pop(path_dir)
+				self.__monitor.rm_watch(wdInt)
+			except KeyError: # pragma: no cover
+				pass
 			logSys.debug("Removed monitor for the parent directory %s", path_dir)
 
 	# pyinotify.ProcessEvent default handler:
@@ -174,7 +178,7 @@ class FilterPyinotify(FileFilter):
 	# slow check events while idle:
 	def __check_events(self, *args, **kwargs):
 		if self.idle:
-			if Utils.wait_for(lambda: not self.idle, 
+			if Utils.wait_for(lambda: not self.active or not self.idle,
 				self.sleeptime * 10, self.sleeptime
 			):
 				pass
@@ -190,11 +194,12 @@ class FilterPyinotify(FileFilter):
 	def run(self):
 		prcevent = pyinotify.ProcessEvent()
 		prcevent.process_default = self.__process_default
+		## timeout for pyinotify must be set in milliseconds (our time values are floats contain seconds)
 		self.__notifier = pyinotify.ThreadedNotifier(self.__monitor,
-			prcevent, timeout=self.sleeptime)
+			prcevent, timeout=self.sleeptime * 1000)
 		self.__notifier.check_events = self.__check_events
 		self.__notifier.start()
-		logSys.debug("pyinotifier started for %s.", self.jail.name)
+		logSys.debug("[%s] filter started (pyinotifier)", self.jailName)
 		return True
 
 	##
@@ -202,15 +207,22 @@ class FilterPyinotify(FileFilter):
 
 	def stop(self):
 		super(FilterPyinotify, self).stop()
-
 		# Stop the notifier thread
 		self.__notifier.stop()
-		self.__notifier.join()			# to not exit before notifier does
+
-		self.__cleanup()				# for pedantic ones
+	##
+	# Wait for exit with cleanup.
+
+	def join(self):
+		self.__cleanup()
+		super(FilterPyinotify, self).join()
+		logSys.debug("[%s] filter terminated (pyinotifier)", self.jailName)
 
 	##
 	# Deallocates the resources used by pyinotify.
 
 	def __cleanup(self):
-		self.__notifier = None
+		if self.__notifier:
+			self.__notifier.join()			# to not exit before notifier does
+			self.__notifier = None
 		self.__monitor = None

fail2ban/server/filtersystemd.py

@@ -34,7 +34,7 @@ from .failmanager import FailManagerEmpty
 from .filter import JournalFilter, Filter
 from .mytime import MyTime
 from .utils import Utils
-from ..helpers import getLogger, logging, splitwords
+from ..helpers import getLogger, logging, splitwords, uni_decode
 
 # Gets the instance of the logger.
 logSys = getLogger(__name__)
@@ -130,7 +130,8 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
 			self.resetJournalMatches()
 			raise
 		else:
-			logSys.info("Added journal match for: %r", " ".join(match))
+			logSys.info("[%s] Added journal match for: %r", self.jailName, 
+				" ".join(match))
 	##
 	# Reset a journal match filter called on removal or failure
 	#
@@ -138,7 +139,7 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
 
 	def resetJournalMatches(self):
 		self.__journal.flush_matches()
-		logSys.debug("Flushed all journal matches")
+		logSys.debug("[%s] Flushed all journal matches", self.jailName)
 		match_copy = self.__matches[:]
 		self.__matches = []
 		try:
@@ -154,13 +155,20 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
 	#
 	# @param match journalctl syntax matches
 
-	def delJournalMatch(self, match):
+	def delJournalMatch(self, match=None):
-		if match in self.__matches:
+		# clear all:
+		if match is None:
+			if not self.__matches:
+				return
+			del self.__matches[:]
+		# delete by index:
+		elif match in self.__matches:
 			del self.__matches[self.__matches.index(match)]
-			self.resetJournalMatches()
 		else:
-			raise ValueError("Match not found")
+			raise ValueError("Match %r not found" % match)
-		logSys.info("Removed journal match for: %r" % " ".join(match))
+		self.resetJournalMatches()
+		logSys.info("[%s] Removed journal match for: %r", self.jailName, 
+			match if match else '*')
 
 	##
 	# Get current journal match filter
@@ -170,10 +178,6 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
 	def getJournalMatch(self):
 		return self.__matches
 
-	def uni_decode(self, x):
-		v = Filter.uni_decode(x, self.getLogEncoding())
-		return v
-
 	##
 	# Format journal log entry into syslog style
 	#
@@ -182,16 +186,16 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
 
 	def formatJournalEntry(self, logentry):
 		# Be sure, all argument of line tuple should have the same type:
-		uni_decode = self.uni_decode
+		enc = self.getLogEncoding()
 		logelements = []
 		v = logentry.get('_HOSTNAME')
 		if v:
-			logelements.append(uni_decode(v))
+			logelements.append(uni_decode(v, enc))
 		v = logentry.get('SYSLOG_IDENTIFIER')
 		if not v:
 			v = logentry.get('_COMM')
 		if v:
-			logelements.append(uni_decode(v))
+			logelements.append(uni_decode(v, enc))
 			v = logentry.get('SYSLOG_PID')
 			if not v:
 				v = logentry.get('_PID')
@@ -206,16 +210,16 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
 				logelements.append("[%12.6f]" % monotonic.total_seconds())
 		msg = logentry.get('MESSAGE','')
 		if isinstance(msg, list):
-			logelements.append(" ".join(uni_decode(v) for v in msg))
+			logelements.append(" ".join(uni_decode(v, enc) for v in msg))
 		else:
-			logelements.append(uni_decode(msg))
+			logelements.append(uni_decode(msg, enc))
 
 		logline = " ".join(logelements)
 
 		date = logentry.get('_SOURCE_REALTIME_TIMESTAMP',
 				logentry.get('__REALTIME_TIMESTAMP'))
-		logSys.debug("Read systemd journal entry: %r" %
+		logSys.log(5, "[%s] Read systemd journal entry: %s %s", self.jailName,
-			"".join([date.isoformat(), logline]))
+			date.isoformat(), logline)
 		## use the same type for 1st argument:
 		return ((logline[:0], date.isoformat(), logline),
 			time.mktime(date.timetuple()) + date.microsecond/1.0E6)
@@ -252,40 +256,64 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
 
 		while self.active:
 			# wait for records (or for timeout in sleeptime seconds):
-			self.__journal.wait(self.sleeptime)
+			try:
-			if self.idle:
+				## todo: find better method as wait_for to break (e.g. notify) journal.wait(self.sleeptime),
-				# because journal.wait will returns immediatelly if we have records in journal,
+				## don't use `journal.close()` for it, because in some python/systemd implementation it may 
-				# just wait a little bit here for not idle, to prevent hi-load:
+				## cause abnormal program termination
-				if not Utils.wait_for(lambda: not self.idle, 
+				#self.__journal.wait(self.sleeptime) != journal.NOP
-					self.sleeptime * 10, self.sleeptime
+				## 
-				):
+				## wait for entries without sleep in intervals, because "sleeping" in journal.wait:
+				Utils.wait_for(lambda: not self.active or \
+					self.__journal.wait(Utils.DEFAULT_SLEEP_INTERVAL) != journal.NOP,
+					self.sleeptime, 0.00001)
+				if self.idle:
+					# because journal.wait will returns immediatelly if we have records in journal,
+					# just wait a little bit here for not idle, to prevent hi-load:
+					if not Utils.wait_for(lambda: not self.active or not self.idle, 
+						self.sleeptime * 10, self.sleeptime
+					):
+						self.ticks += 1
+						continue
+				self.__modified = 0
+				while self.active:
+					logentry = None
+					try:
+						logentry = self.__journal.get_next()
+					except OSError as e:
+						logSys.error("Error reading line from systemd journal: %s",
+							e, exc_info=logSys.getEffectiveLevel() <= logging.DEBUG)
 					self.ticks += 1
-					continue
+					if logentry:
-			self.__modified = 0
+						self.processLineAndAdd(
-			while self.active:
+							*self.formatJournalEntry(logentry))
-				logentry = None
+						self.__modified += 1
-				try:
+						if self.__modified >= 100: # todo: should be configurable
-					logentry = self.__journal.get_next()
+							break
-				except OSError as e:
+					else:
-					logSys.error("Error reading line from systemd journal: %s",
-						e, exc_info=logSys.getEffectiveLevel() <= logging.DEBUG)
-				self.ticks += 1
-				if logentry:
-					self.processLineAndAdd(
-						*self.formatJournalEntry(logentry))
-					self.__modified += 1
-					if self.__modified >= 100: # todo: should be configurable
 						break
-				else:
+				if self.__modified:
+					try:
+						while True:
+							ticket = self.failManager.toBan()
+							self.jail.putFailTicket(ticket)
+					except FailManagerEmpty:
+						self.failManager.cleanup(MyTime.time())
+			except Exception as e: # pragma: no cover
+				if not self.active: # if not active - error by stop...
 					break
-			if self.__modified:
+				logSys.error("Caught unhandled exception in main cycle: %r", e,
-				try:
+					exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
-					while True:
+				# incr common error counter:
-						ticket = self.failManager.toBan()
+				self.commonError()
-						self.jail.putFailTicket(ticket)
-				except FailManagerEmpty:
-					self.failManager.cleanup(MyTime.time())
 
+		logSys.debug("[%s] filter terminated", self.jailName)
+		# close journal:
+		try:
+			if self.__journal:
+				self.__journal.close()
+		except Exception as e: # pragma: no cover
+			logSys.error("Close journal failed: %r", e,
+				exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
 		logSys.debug((self.jail is not None and self.jail.name
                       or "jailless") +" filter terminated")
 		return True

fail2ban/server/jail.py

@@ -28,7 +28,7 @@ import Queue
 
 from .actions import Actions
 from ..client.jailreader import JailReader
-from ..helpers import getLogger
+from ..helpers import getLogger, MyTime
 
 # Gets the instance of the logger.
 logSys = getLogger(__name__)
@@ -79,6 +79,7 @@ class Jail(object):
 		logSys.info("Creating new jail '%s'" % self.name)
 		if backend is not None:
 			self._setBackend(backend)
+		self.backend = backend
 
 	def __repr__(self):
 		return "%s(%r)" % (self.__class__.__name__, self.name)
@@ -193,7 +194,7 @@ class Jail(object):
 		Used by filter to add a failure for banning.
 		"""
 		self.__queue.put(ticket)
-		if self.database is not None:
+		if not ticket.restored and self.database is not None:
 			self.database.addBan(self, ticket)
 
 	def getFailTicket(self):
@@ -202,34 +203,66 @@ class Jail(object):
 		Used by actions to get a failure for banning.
 		"""
 		try:
-			return self.__queue.get(False)
+			ticket = self.__queue.get(False)
+			return ticket
 		except Queue.Empty:
 			return False
 
+	def restoreCurrentBans(self):
+		"""Restore any previous valid bans from the database.
+		"""
+		try:
+			if self.database is not None:
+				forbantime = self.actions.getBanTime()
+				for ticket in self.database.getCurrentBans(jail=self, forbantime=forbantime):
+					#logSys.debug('restored ticket: %s', ticket)
+					if not self.filter.inIgnoreIPList(ticket.getIP(), log_ignore=True):
+						# mark ticked was restored from database - does not put it again into db:
+						ticket.restored = True
+						# correct start time / ban time (by the same end of ban):
+						btm = ticket.getBanTime(forbantime)
+						diftm = MyTime.time() - ticket.getTime()
+						if btm != -1 and diftm > 0:
+							btm -= diftm
+						# ignore obsolete tickets:
+						if btm != -1 and btm <= 0:
+							continue
+						ticket.setTime(MyTime.time())
+						ticket.setBanTime(btm)
+						self.putFailTicket(ticket)
+		except Exception as e: # pragma: no cover
+			logSys.error('%s', e, exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
+
 	def start(self):
 		"""Start the jail, by starting filter and actions threads.
 
 		Once stated, also queries the persistent database to reinstate
 		any valid bans.
 		"""
+		logSys.debug("Starting jail %r", self.name)
 		self.filter.start()
 		self.actions.start()
-		# Restore any previous valid bans from the database
+		self.restoreCurrentBans()
-		if self.database is not None:
+		logSys.info("Jail %r started", self.name)
-			for ticket in self.database.getBansMerged(
-				jail=self, bantime=self.actions.getBanTime()):
-				if not self.filter.inIgnoreIPList(ticket.getIP(), log_ignore=True):
-					self.__queue.put(ticket)
-		logSys.info("Jail '%s' started" % self.name)
 
-	def stop(self):
+	def stop(self, stop=True, join=True):
 		"""Stop the jail, by stopping filter and actions threads.
 		"""
-		self.filter.stop()
+		if stop:
-		self.actions.stop()
+			logSys.debug("Stopping jail %r", self.name)
-		self.filter.join()
+		for obj in (self.filter, self.actions):
-		self.actions.join()
+			try:
-		logSys.info("Jail '%s' stopped" % self.name)
+				## signal to stop filter / actions:
+				if stop:
+					obj.stop()
+				## wait for end of threads:
+				if join:
+					obj.join()
+			except Exception as e:
+				logSys.error("Stop %r of jail %r failed: %s", obj, self.name, e,
+					exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
+		if join:
+			logSys.info("Jail %r stopped", self.name)
 
 	def isAlive(self):
 		"""Check jail "isAlive" by checking filter and actions threads.

fail2ban/server/jails.py

@@ -62,14 +62,15 @@ class Jails(Mapping):
 		DuplicateJailException
 			If jail name is already present.
 		"""
-		try:
+		with self.__lock:
-			self.__lock.acquire()
 			if name in self._jails:
-				raise DuplicateJailException(name)
+				if noduplicates:
+					raise DuplicateJailException(name)
 			else:
 				self._jails[name] = Jail(name, backend, db)
-		finally:
+
-			self.__lock.release()
+	def exists(self, name):
+		return name in self._jails
 
 	def __getitem__(self, name):
 		try:

fail2ban/server/jailthread.py

@@ -53,8 +53,8 @@ class JailThread(Thread):
 		super(JailThread, self).__init__(name=name)
 		## Should going with main thread also:
 		self.daemon = True
-		## Control the state of the thread.
+		## Control the state of the thread (None - was not started, True - active, False - stopped).
-		self.active = False
+		self.active = None
 		## Control the idle state of the thread.
 		self.idle = False
 		## The time the thread sleeps in the loop.
@@ -93,3 +93,14 @@ class JailThread(Thread):
 		"""Abstract - Called when thread starts, thread stops when returns.
 		"""
 		pass
+
+	def join(self):
+		""" Safer join, that could be called also for not started (or ended) threads (used for cleanup).
+		"""
+		## if cleanup needed - create derivate and call it before join...
+
+		## if was really started - should call join:
+		if self.active is not None:
+			super(JailThread, self).join()
+
+

fail2ban/server/server.py

@@ -38,7 +38,7 @@ from .filter import FileFilter, JournalFilter
 from .transmitter import Transmitter
 from .asyncserver import AsyncServer, AsyncServerException
 from .. import version
-from ..helpers import getLogger, excepthook
+from ..helpers import getLogger, str2LogLevel, excepthook
 
 # Gets the instance of the logger.
 logSys = getLogger(__name__)
@@ -67,6 +67,7 @@ class Server:
 		self.__db = None
 		self.__daemon = daemon
 		self.__transm = Transmitter(self)
+		self.__reload_state = {}
 		#self.__asyncServer = AsyncServer(self.__transm)
 		self.__asyncServer = None
 		self.__logLevel = None
@@ -170,6 +171,12 @@ class Server:
 		# Now stop all the jails
 		self.stopAllJail()
 
+		# Explicit close database (server can leave in a thread, 
+		# so delayed GC can prevent commiting changes)
+		if self.__db:
+			self.__db.close()
+			self.__db = None
+
 		# Only now shutdown the logging.
 		if self.__logTarget is not None:
 			with self.__loggingLock:
@@ -184,41 +191,111 @@ class Server:
 		self.quit = lambda: False
 
 	def addJail(self, name, backend):
-		self.__jails.add(name, backend, self.__db)
+		addflg = True
+		if self.__reload_state.get(name) and self.__jails.exists(name):
+			jail = self.__jails[name]
+			# if backend switch - restart instead of reload:
+			if jail.backend == backend:
+				addflg = False
+				logSys.info("Reload jail %r", name)
+				# prevent to reload the same jail twice (temporary keep it in state, needed to commit reload):
+				self.__reload_state[name] = None
+			else:
+				logSys.info("Restart jail %r (reason: %r != %r)", name, jail.backend, backend)
+				self.delJail(name, stop=True)
+				# prevent to start the same jail twice (no reload more - restart):
+				del self.__reload_state[name]
+		if addflg:
+			self.__jails.add(name, backend, self.__db)
 		if self.__db is not None:
 			self.__db.addJail(self.__jails[name])
 		
-	def delJail(self, name):
+	def delJail(self, name, stop=True, join=True):
-		if self.__db is not None:
+		jail = self.__jails[name]
-			self.__db.delJail(self.__jails[name])
+		if join or jail.isAlive():
-		del self.__jails[name]
+			jail.stop(stop=stop, join=join)
+		if join:
+			if self.__db is not None:
+				self.__db.delJail(jail)
+			del self.__jails[name]
 
 	def startJail(self, name):
-		try:
+		with self.__lock:
-			self.__lock.acquire()
+			jail = self.__jails[name]
-			if not self.__jails[name].isAlive():
+			if not jail.isAlive():
-				self.__jails[name].start()
+				jail.start()
-		finally:
+			elif name in self.__reload_state:
-			self.__lock.release()
+				logSys.info("Jail %r reloaded", name)
+				del self.__reload_state[name]
+			if jail.idle:
+				jail.idle = False
 	
 	def stopJail(self, name):
-		logSys.debug("Stopping jail %s" % name)
+		with self.__lock:
-		try:
+			self.delJail(name, stop=True)
-			self.__lock.acquire()
-			if self.__jails[name].isAlive():
-				self.__jails[name].stop()
-				self.delJail(name)
-		finally:
-			self.__lock.release()
 	
 	def stopAllJail(self):
 		logSys.info("Stopping all jails")
-		try:
+		with self.__lock:
-			self.__lock.acquire()
+			# 1st stop all jails (signal and stop actions/filter thread):
-			for jail in self.__jails.keys():
+			for name in self.__jails.keys():
-				self.stopJail(jail)
+				self.delJail(name, stop=True, join=False)
-		finally:
+			# 2nd wait for end and delete jails:
-			self.__lock.release()
+			for name in self.__jails.keys():
+				self.delJail(name, stop=False, join=True)
+
+	def reloadJails(self, name, opts, begin):
+		if begin:
+			# begin reload:
+			if self.__reload_state and (name == '--all' or self.__reload_state.get(name)):
+				raise ValueError('Reload already in progress')
+			logSys.info("Reload " + (("jail %s" % name) if name != '--all' else "all jails"))
+			with self.__lock:
+				# if single jail:
+				if name != '--all':
+					jail = None
+					# test jail exists (throws exception if not):
+					if "--if-exists" not in opts or self.__jails.exists(name):
+						jail = self.__jails[name]
+					if jail:
+						# first unban all ips (will be not restored after (re)start):
+						if "--unban" in opts:
+							self.setUnbanIP(name)
+						# stop if expected:
+						if "--restart" in opts:
+							self.stopJail(name)
+				else:
+					# first unban all ips (will be not restored after (re)start):
+					if "--unban" in opts:
+						self.setUnbanIP()
+					# stop if expected:
+					if "--restart" in opts:
+						self.stopAllJail()
+				# first set all affected jail(s) to idle and reset filter regex and other lists/dicts:
+				for jn, jail in self.__jails.iteritems():
+					if name == '--all' or jn == name:
+						jail.idle = True
+						self.__reload_state[jn] = jail
+						jail.filter.reload(begin=True)
+						jail.actions.reload(begin=True)
+				pass
+		else:
+			# end reload, all affected (or new) jails have already all new parameters (via stream) and (re)started:
+			with self.__lock:
+				deljails = []
+				for jn, jail in self.__jails.iteritems():
+					# still in reload state:
+					if jn in self.__reload_state:
+						# remove jails that are not reloaded (untouched, so not in new configuration)
+						deljails.append(jn)
+					else:
+						# commit (reload was finished):
+						jail.filter.reload(begin=False)
+						jail.actions.reload(begin=False)
+				for jn in deljails:
+					self.delJail(jn)
+			self.__reload_state = {}
+			logSys.info("Reload finished.")
 
 	def setIdleJail(self, name, value):
 		self.__jails[name].idle = value
@@ -309,7 +386,7 @@ class Server:
 			logSys.debug("  failregex: %r", value)
 			flt.addFailRegex(value)
 	
-	def delFailRegex(self, name, index):
+	def delFailRegex(self, name, index=None):
 		self.__jails[name].filter.delFailRegex(index)
 	
 	def getFailRegex(self, name):
@@ -351,7 +428,9 @@ class Server:
 	
 	# Action
 	def addAction(self, name, value, *args):
-		self.__jails[name].actions.add(value, *args)
+		## create (or reload) jail action:
+		self.__jails[name].actions.add(value, *args, 
+			reload=name in self.__reload_state)
 	
 	def getActions(self, name):
 		return self.__jails[name].actions
@@ -368,8 +447,20 @@ class Server:
 	def setBanIP(self, name, value):
 		return self.__jails[name].filter.addBannedIP(value)
 		
-	def setUnbanIP(self, name, value):
+	def setUnbanIP(self, name=None, value=None):
-		self.__jails[name].actions.removeBannedIP(value)
+		if name is not None:
+			# in all jails:
+			jails = [self.__jails[name]]
+		else:
+			# single jail:
+			jails = self.__jails.values()
+		# unban given or all (if value is None):
+		cnt = 0
+		for jail in jails:
+			cnt += jail.actions.removeBannedIP(value, ifexists=(name is None))
+		if value and not cnt:
+			logSys.info("%s is not banned", value)
+		return cnt
 		
 	def getBanTime(self, name):
 		return self.__jails[name].actions.getBanTime()
@@ -419,11 +510,11 @@ class Server:
 		with self.__loggingLock:
 			if self.__logLevel == value:
 				return
-			try:
+			ll = str2LogLevel(value)
-				getLogger("fail2ban").setLevel(getattr(logging, value))
+			# don't change real log-level if running from the test cases:
-				self.__logLevel = value
+			getLogger("fail2ban").setLevel(
-			except AttributeError:
+				ll if DEF_LOGTARGET != "INHERITED" or ll < logging.DEBUG else DEF_LOGLEVEL)
-				raise ValueError("Invalid log level %r" % value)
+			self.__logLevel = value
 	
 	##
 	# Get the logging level.

fail2ban/server/ticket.py

@@ -34,8 +34,13 @@ from .mytime import MyTime
 logSys = getLogger(__name__)
 
 
-class Ticket:
+class Ticket(object):
+
+	MAX_TIME = 0X7FFFFFFFFFFF ;# 4461763-th year
 	
+	RESTORED = 0x01
+	BANNED   = 0x08
+
 	def __init__(self, ip=None, time=None, matches=None, data={}, ticket=None):
 		"""Ticket constructor
 
@@ -49,13 +54,12 @@ class Ticket:
 		self._banCount = 0;
 		self._banTime = None;
 		self._time = time if time is not None else MyTime.time()
-		self._data = {'matches': [], 'failures': 0}
+		self._data = {'matches': matches or [], 'failures': 0}
-		self._data.update(data)
+		if data is not None:
+			self._data.update(data)
 		if ticket:
 			# ticket available - copy whole information from ticket:
 			self.__dict__.update(i for i in ticket.__dict__.iteritems() if i[0] in self.__dict__)
-		else:
-			self._data['matches'] = matches or []
 
 	def __str__(self):
 		return "%s: ip=%s time=%s #attempts=%d matches=%r" % \
@@ -94,8 +98,8 @@ class Ticket:
 	def setBanTime(self, value):
 		self._banTime = value;
 
-	def getBanTime(self, defaultBT = None):
+	def getBanTime(self, defaultBT=None):
-		return (self._banTime if not self._banTime is None else defaultBT);
+		return (self._banTime if self._banTime is not None else defaultBT)
 
 	def setBanCount(self, value):
 		self._banCount = value;
@@ -106,8 +110,16 @@ class Ticket:
 	def getBanCount(self):
 		return self._banCount;
 
-	def isTimedOut(self, time, defaultBT = None):
+	def getEndOfBanTime(self, defaultBT=None):
-		bantime = (self._banTime if not self._banTime is None else defaultBT);
+		bantime = (self._banTime if self._banTime is not None else defaultBT)
+		# permanent
+		if bantime == -1:
+			return Ticket.MAX_TIME
+		# unban time (end of ban):
+		return self._time + bantime
+
+	def isTimedOut(self, time, defaultBT=None):
+		bantime = (self._banTime if self._banTime is not None else defaultBT)
 		# permanent
 		if bantime == -1:
 			return False
@@ -126,6 +138,26 @@ class Ticket:
 	def getMatches(self):
 		return self._data.get('matches', [])
 
+	@property
+	def restored(self):
+		return self._flags & Ticket.RESTORED
+	@restored.setter
+	def restored(self, value):
+		if value:
+			self._flags |= Ticket.RESTORED
+		else:
+			self._flags &= ~(Ticket.RESTORED)
+	
+	@property
+	def banned(self):
+		return self._flags & Ticket.BANNED
+	@banned.setter
+	def banned(self, value):
+		if value:
+			self._flags |= Ticket.BANNED
+		else:
+			self._flags &= ~(Ticket.BANNED)
+
 	def setData(self, *args, **argv):
 		# if overwrite - set data and filter None values:
 		if len(args) == 1:

fail2ban/server/transmitter.py

@@ -27,7 +27,7 @@ __license__ = "GPL"
 import time
 import json
 
-from ..helpers import getLogger
+from ..helpers import getLogger, logging
 from .. import version
 
 # Gets the instance of the logger.
@@ -52,13 +52,14 @@ class Transmitter:
 	
 	def proceed(self, command):
 		# Deserialize object
-		logSys.debug("Command: %r", command)
+		logSys.log(5, "Command: %r", command)
 		try:
 			ret = self.__commandHandler(command)
 			ack = 0, ret
 		except Exception as e:
-			logSys.warning("Command %r has failed. Received %r"
+			logSys.warning("Command %r has failed. Received %r",
-						% (command, e))
+						command, e, 
+						exc_info=logSys.getEffectiveLevel()<=logging.DEBUG)
 			ack = 1, e
 		return ack
 	
@@ -72,8 +73,8 @@ class Transmitter:
 			return "pong"
 		elif command[0] == "add":
 			name = command[1]
-			if name == "all":
+			if name == "--all":
-				raise Exception("Reserved name")
+				raise Exception("Reserved name %r" % (name,))
 			try:
 				backend = command[2]
 			except IndexError:
@@ -87,12 +88,31 @@ class Transmitter:
 		elif command[0] == "stop":
 			if len(command) == 1:
 				self.__server.quit()
-			elif command[1] == "all":
+			elif command[1] == "--all":
 				self.__server.stopAllJail()
 			else:
 				name = command[1]
 				self.__server.stopJail(name)
 			return None
+		elif command[0] == "reload":
+			opts = command[1:3]
+			try:
+				self.__server.reloadJails(*opts, begin=True)
+				for cmd in command[3]:
+					self.__commandHandler(cmd)
+			finally:
+				self.__server.reloadJails(*opts, begin=False)
+			return 'OK'
+		elif len(command) >= 2 and command[0] == "unban":
+			# unban in all jails:
+			value = command[1:]
+			# if all ips:
+			if len(value) == 1 and value[0] == "--all":
+				self.__server.setUnbanIP()
+				return
+			for value in value:
+				self.__server.setUnbanIP(None, value)
+			return None
 		elif command[0] == "echo":
 			return command[1:]
 		elif command[0] == "sleep":
@@ -265,7 +285,7 @@ class Transmitter:
 			action = self.__server.getAction(name, actionname)
 			if multiple:
 				for cmd in command[3]:
-					logSys.debug("  %r", cmd)
+					logSys.log(5, "  %r", cmd)
 					actionkey = cmd[0]
 					if callable(getattr(action, actionkey, None)):
 						actionvalue = json.loads(cmd[1]) if len(cmd)>1 else {}

fail2ban/server/utils.py

@@ -21,8 +21,14 @@ __author__ = "Serg G. Brester (sebres) and Fail2Ban Contributors"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2011-2012 Yaroslav Halchenko, 2012-2015 Serg G. Brester"
 __license__ = "GPL"
 
-import logging, os, fcntl, subprocess, time, signal
+import fcntl
-from ..helpers import getLogger
+import logging
+import os
+import signal
+import subprocess
+import sys
+import time
+from ..helpers import getLogger, uni_decode
 
 # Gets the instance of the logger.
 logSys = getLogger(__name__)
@@ -46,8 +52,8 @@ class Utils():
 	"""Utilities provide diverse static methods like executes OS shell commands, etc.
 	"""
 
-	DEFAULT_SLEEP_TIME = 0.1
+	DEFAULT_SLEEP_TIME = 2
-	DEFAULT_SLEEP_INTERVAL = 0.01
+	DEFAULT_SLEEP_INTERVAL = 0.2
 
 
 	class Cache(object):
@@ -179,7 +185,7 @@ class Utils():
 				if stdout is not None and stdout != '' and std_level >= logSys.getEffectiveLevel():
 					logSys.log(std_level, "%s -- stdout:", realCmd)
 					for l in stdout.splitlines():
-						logSys.log(std_level, " -- stdout: %r", l)
+						logSys.log(std_level, " -- stdout: %r", uni_decode(l))
 				popen.stdout.close()
 			if popen.stderr:
 				try:
@@ -191,7 +197,7 @@ class Utils():
 				if stderr is not None and stderr != '' and std_level >= logSys.getEffectiveLevel():
 					logSys.log(std_level, "%s -- stderr:", realCmd)
 					for l in stderr.splitlines():
-						logSys.log(std_level, " -- stderr: %r", l)
+						logSys.log(std_level, " -- stderr: %r", uni_decode(l))
 				popen.stderr.close()
 
 		success = False

fail2ban/tests/actiontestcase.py

@@ -277,6 +277,24 @@ class CommandActionTest(LogCaptureTestCase):
 		self.assertRaises(RuntimeError, self.__action.ban, {'ip': None})
 		self.assertLogged('Unable to restore environment')
 
+	def testExecuteActionCheckRepairEnvironment(self):
+		self.__action.actionstart = ""
+		self.__action.actionstop = ""
+		self.__action.actionban = "rm /tmp/fail2ban.test"
+		self.__action.actioncheck = "[ -e /tmp/fail2ban.test ]"
+		self.__action.actionrepair = "echo 'repair ...'; touch /tmp/fail2ban.test"
+		# 1st time with success repair:
+		self.__action.ban({'ip': None})
+		self.assertLogged("Invariant check failed. Trying", "echo 'repair ...'", all=True)
+		self.pruneLog()
+		# 2nd time failed (not really repaired):
+		self.__action.actionrepair = "echo 'repair ...'"
+		self.assertRaises(RuntimeError, self.__action.ban, {'ip': None})
+		self.assertLogged(
+			"Invariant check failed. Trying", 
+			"echo 'repair ...'", 
+			"Unable to restore environment", all=True)
+
 	def testExecuteActionChangeCtags(self):
 		self.assertRaises(AttributeError, getattr, self.__action, "ROST")
 		self.__action.ROST = "192.0.2.0"
@@ -294,7 +312,12 @@ class CommandActionTest(LogCaptureTestCase):
 	def testExecuteActionStartEmpty(self):
 		self.__action.actionstart = ""
 		self.__action.start()
+		self.assertTrue(self.__action.executeCmd(""))
 		self.assertLogged('Nothing to do')
+		self.pruneLog()
+		self.assertTrue(self.__action._processCmd(""))
+		self.assertLogged('Nothing to do')
+		self.pruneLog()
 
 	def testExecuteIncorrectCmd(self):
 		CommandAction.executeCmd('/bin/ls >/dev/null\nbogusXXX now 2>/dev/null')
@@ -376,11 +399,11 @@ class CommandActionTest(LogCaptureTestCase):
 
 	def testCaptureStdOutErr(self):
 		CommandAction.executeCmd('echo "How now brown cow"')
-		self.assertLogged("stdout: 'How now brown cow'\n", "stdout: b'How now brown cow'\n")
+		self.assertLogged("stdout: 'How now brown cow'\n")
 		CommandAction.executeCmd(
 			'echo "The rain in Spain stays mainly in the plain" 1>&2')
 		self.assertLogged(
-			"stderr: 'The rain in Spain stays mainly in the plain'\n", "stderr: b'The rain in Spain stays mainly in the plain'\n")
+			"stderr: 'The rain in Spain stays mainly in the plain'\n")
 
 	def testCallingMap(self):
 		mymap = CallingMap(callme=lambda: str(10), error=lambda: int('a'),

fail2ban/tests/banmanagertestcase.py

@@ -28,7 +28,6 @@ import unittest
 
 from ..server.banmanager import BanManager
 from ..server.ticket import BanTicket
-from .utils import assert_dict_equal
 
 class AddFailure(unittest.TestCase):
 	def setUp(self):
@@ -53,11 +52,15 @@ class AddFailure(unittest.TestCase):
 		self.assertEqual(self.__banManager.size(), 1)
 
 	def testAddDuplicateWithTime(self):
+		defBanTime = self.__banManager.getBanTime()
+		prevEndOfBanTime = 0
 		# add again a duplicate :
-		#   1) with newer start time and the same ban time
+		#   0) with same start time and the same (default) ban time
+		#   1) with newer start time and the same (default) ban time
 		#   2) with same start time and longer ban time
     #   3) with permanent ban time (-1)
 		for tnew, btnew in (
+			(1167605999.0,       None),
 			(1167605999.0 + 100, None),
 			(1167605999.0,       24*60*60),
 			(1167605999.0,       -1),
@@ -70,10 +73,15 @@ class AddFailure(unittest.TestCase):
 			self.assertFalse(self.__banManager.addBanTicket(ticket2))
 			self.assertEqual(self.__banManager.size(), 1)
 			# pop ticket and check it was prolonged :
-			banticket = self.__banManager.getTicketByIP(ticket2.getIP())
+			banticket = self.__banManager.getTicketByID(ticket2.getID())
-			self.assertEqual(banticket.getTime(), ticket2.getTime())
+			self.assertEqual(banticket.getEndOfBanTime(defBanTime), ticket2.getEndOfBanTime(defBanTime))
-			self.assertEqual(banticket.getTime(), ticket2.getTime())
+			self.assertTrue(banticket.getEndOfBanTime(defBanTime) > prevEndOfBanTime)
-			self.assertEqual(banticket.getBanTime(), ticket2.getBanTime(self.__banManager.getBanTime()))
+			prevEndOfBanTime = ticket1.getEndOfBanTime(defBanTime)
+			# but the start time should not be changed (+ 100 is ignored):
+			self.assertEqual(banticket.getTime(), 1167605999.0)
+			# if prolong to permanent, it should also have permanent ban time:
+			if btnew == -1:
+				self.assertEqual(banticket.getBanTime(defBanTime), -1)
 
 	def testInListOK(self):
 		self.assertTrue(self.__banManager.addBanTicket(self.__ticket))
@@ -87,9 +95,28 @@ class AddFailure(unittest.TestCase):
 
 	def testUnban(self):
 		btime = self.__banManager.getBanTime()
+		stime = self.__ticket.getTime()
 		self.assertTrue(self.__banManager.addBanTicket(self.__ticket))
 		self.assertTrue(self.__banManager._inBanList(self.__ticket))
-		self.assertEqual(self.__banManager.unBanList(self.__ticket.getTime() + btime + 1), [self.__ticket])
+		self.assertEqual(self.__banManager.unBanList(stime), [])
+		self.assertEqual(self.__banManager.unBanList(stime + btime + 1), [self.__ticket])
+		self.assertEqual(self.__banManager.size(), 0)
+		## again, but now we will prolong ban-time and then try to unban again (1st too early):
+		self.assertTrue(self.__banManager.addBanTicket(self.__ticket))
+		# prolong ban:
+		ticket = BanTicket(self.__ticket.getID(), stime + 600)
+		self.assertFalse(self.__banManager.addBanTicket(ticket))
+		# try unban too early:
+		self.assertEqual(len(self.__banManager.unBanList(stime + btime + 1)), 0)
+		# try unban using correct time:
+		self.assertEqual(len(self.__banManager.unBanList(stime + btime + 600 + 1)), 1)
+		## again, but now we test removing tickets particular (to test < 2/3-rule):
+		for i in range(5):
+			ticket = BanTicket('193.168.0.%s' % i, stime)
+			ticket.setBanTime(ticket.getBanTime(btime) + i*10)
+			self.assertTrue(self.__banManager.addBanTicket(ticket))
+		self.assertEqual(len(self.__banManager.unBanList(stime + btime + 1*10 + 1)), 2)
+		self.assertEqual(len(self.__banManager.unBanList(stime + btime + 5*10 + 1)), 3)
 		self.assertEqual(self.__banManager.size(), 0)
 
 	def testUnbanPermanent(self):
@@ -122,7 +149,7 @@ class StatusExtendedCymruInfo(unittest.TestCase):
 
 	def testCymruInfo(self):
 		cymru_info = self.__banManager.getBanListExtendedCymruInfo()
-		assert_dict_equal(cymru_info,
+		self.assertDictEqual(cymru_info,
 						  {"asn": [self.__asn],
 						   "country": [self.__country],
 						   "rir": [self.__rir]})
@@ -149,7 +176,7 @@ class StatusExtendedCymruInfo(unittest.TestCase):
 		ticket = BanTicket("0.0.0.0", 1167605999.0)
 		self.assertTrue(self.__banManager.addBanTicket(ticket))
 		cymru_info = self.__banManager.getBanListExtendedCymruInfo()
-		assert_dict_equal(cymru_info,
+		self.assertDictEqual(cymru_info,
 						  {"asn": ["nxdomain"],
 						   "country": ["nxdomain"],
 						   "rir": ["nxdomain"]})
@@ -160,7 +187,7 @@ class StatusExtendedCymruInfo(unittest.TestCase):
 		ticket = BanTicket("10.0.0.0", 1167606000.0)
 		self.assertTrue(self.__banManager.addBanTicket(ticket))
 		cymru_info = self.__banManager.getBanListExtendedCymruInfo()
-		assert_dict_equal(cymru_info,
+		self.assertDictEqual(dict((k, sorted(v)) for k, v in cymru_info.iteritems()),
-						  {"asn": ["nxdomain", "4565",],
+						  {"asn": sorted(["nxdomain", "4565",]),
-						   "country": ["nxdomain", "unknown"],
+						   "country": sorted(["nxdomain", "unknown"]),
-						   "rir": ["nxdomain", "other"]})
+						   "rir": sorted(["nxdomain", "other"])})

fail2ban/tests/databasetestcase.py

@@ -353,6 +353,11 @@ class DatabaseTest(LogCaptureTestCase):
 		# be returned
 		tickets = self.db.getBansMerged(bantime=-1)
 		self.assertEqual(len(tickets), 2)
+		# getCurrentBans:
+		tickets = self.db.getCurrentBans(jail=self.jail)
+		self.assertEqual(len(tickets), 2)
+		ticket = self.db.getCurrentBans(jail=None, ip="127.0.0.1");
+		self.assertEqual(ticket.getIP(), "127.0.0.1")
 
 	def testActionWithDB(self):
 		# test action together with database functionality

fail2ban/tests/fail2banclienttestcase.py

@@ -41,8 +41,9 @@ from ..client.fail2banclient import exec_command_line as _exec_client, VisualWai
 from ..client.fail2banserver import Fail2banServer, exec_command_line as _exec_server
 from .. import protocol
 from ..server import server
+from ..server.mytime import MyTime
 from ..server.utils import Utils
-from .utils import LogCaptureTestCase, with_tmpdir, shutil, logging
+from .utils import LogCaptureTestCase, logSys as DefLogSys, with_tmpdir, shutil, logging
 
 from ..helpers import getLogger
 
@@ -57,6 +58,7 @@ SERVER = "fail2ban-server"
 BIN = dirname(Fail2banServer.getServerPath())
 
 MAX_WAITTIME = 30 if not unittest.F2B.fast else 5
+MID_WAITTIME = MAX_WAITTIME
 
 ##
 # Several wrappers and settings for proper testing:
@@ -68,7 +70,8 @@ fail2bancmdline.logSys = \
 fail2banclient.logSys = \
 fail2banserver.logSys = logSys
 
-server.DEF_LOGTARGET = "/dev/null"
+SRV_DEF_LOGTARGET = server.DEF_LOGTARGET
+SRV_DEF_LOGLEVEL = server.DEF_LOGLEVEL
 
 def _test_output(*args):
 	logSys.info(args[0])
@@ -110,17 +113,25 @@ fail2bancmdline.PRODUCTION = \
 fail2banserver.PRODUCTION = False
 
 
-def _out_file(fn):
+def _out_file(fn, handle=logSys.debug):
 	"""Helper which outputs content of the file at HEAVYDEBUG loglevels"""
-	logSys.debug('---- ' + fn + ' ----')
+	handle('---- ' + fn + ' ----')
 	for line in fileinput.input(fn):
 		line = line.rstrip('\n')
-		logSys.debug(line)
+		handle(line)
-	logSys.debug('-'*30)
+	handle('-'*30)
 
 
-def _start_params(tmp, use_stock=False, logtarget="/dev/null"):
+def _write_file(fn, mode, *lines):
+	f = open(fn, mode)
+	f.write('\n'.join(lines))
+	f.close()
+
+
+def _start_params(tmp, use_stock=False, logtarget="/dev/null", db=":memory:"):
 	cfg = pjoin(tmp, "config")
+	if db == 'auto':
+		db = pjoin(tmp, "f2b-db.sqlite3")
 	if use_stock and STOCK:
 		# copy config (sub-directories as alias):
 		def ig_dirs(dir, files):
@@ -146,8 +157,7 @@ def _start_params(tmp, use_stock=False, logtarget="/dev/null"):
 	else:
 		# just empty config directory without anything (only fail2ban.conf/jail.conf):
 		os.mkdir(cfg)
-		f = open(pjoin(cfg, "fail2ban.conf"), "w")
+		_write_file(pjoin(cfg, "fail2ban.conf"), "w",
-		f.write('\n'.join((
 			"[Definition]",
 			"loglevel = INFO",
 			"logtarget = " + logtarget,
@@ -155,19 +165,16 @@ def _start_params(tmp, use_stock=False, logtarget="/dev/null"):
 			"socket = " + pjoin(tmp, "f2b.sock"),
 			"pidfile = " + pjoin(tmp, "f2b.pid"),
 			"backend = polling",
-			"dbfile = :memory:",
+			"dbfile = " + db,
 			"dbpurgeage = 1d",
 			"",
-		)))
+		)
-		f.close()
+		_write_file(pjoin(cfg, "jail.conf"), "w",
-		f = open(pjoin(cfg, "jail.conf"), "w")
-		f.write('\n'.join((
 			"[INCLUDES]", "",
 			"[DEFAULT]", "",
 			"",
-		)))
+		)
-		f.close()
+		if DefLogSys.level < logging.DEBUG:  # if HEAVYDEBUG
-		if logSys.level < logging.DEBUG:  # if HEAVYDEBUG
 			_out_file(pjoin(cfg, "fail2ban.conf"))
 			_out_file(pjoin(cfg, "jail.conf"))
 	# parameters (sock/pid and config, increase verbosity, set log, etc.):
@@ -237,19 +244,78 @@ def with_kill_srv(f):
 			_kill_srv(pidfile)
 	return wrapper
 
+def with_foreground_server_thread(startextra={}):
+	"""Helper to decorate tests uses foreground server (as thread), started directly in test-cases
+
+	To be used only in subclasses
+	"""
+	def _deco_wrapper(f):
+		@with_tmpdir
+		@wraps(f)
+		def wrapper(self, tmp, *args, **kwargs):
+			th = None
+			phase = dict()
+			try:
+				# started directly here, so prevent overwrite test cases logger with "INHERITED"
+				startparams = _start_params(tmp, logtarget="INHERITED", **startextra)
+				# because foreground block execution - start it in thread:
+				th = Thread(
+					name="_TestCaseWorker",
+					target=self._testStartForeground,
+					args=(tmp, startparams, phase)
+				)
+				th.daemon = True
+				th.start()
+				try:
+					# wait for start thread:
+					Utils.wait_for(lambda: phase.get('start', None) is not None, MAX_WAITTIME)
+					self.assertTrue(phase.get('start', None))
+					# wait for server (socket and ready):
+					self._wait_for_srv(tmp, True, startparams=startparams)
+					DefLogSys.info('=== within server: begin ===')
+					self.pruneLog()
+					# several commands to server in body of decorated function:
+					return f(self, tmp, startparams, *args, **kwargs)
+				finally:
+					DefLogSys.info('=== within server: end.  ===')
+					self.pruneLog()
+					# stop:
+					self.execSuccess(startparams, "stop")
+					# wait for end:
+					Utils.wait_for(lambda: phase.get('end', None) is not None, MAX_WAITTIME)
+					self.assertTrue(phase.get('end', None))
+					self.assertLogged("Shutdown successful", "Exiting Fail2ban")
+			finally:
+				if th:
+					# we start client/server directly in current process (new thread),
+					# so don't kill (same process) - if success, just wait for end of worker:
+					if phase.get('end', None):
+						th.join()
+		return wrapper
+	return _deco_wrapper
+
 
 class Fail2banClientServerBase(LogCaptureTestCase):
 
 	_orig_exit = Fail2banCmdLine._exit
 
+	def _setLogLevel(self, *args, **kwargs):
+		pass
+
 	def setUp(self):
 		"""Call before every test case."""
 		LogCaptureTestCase.setUp(self)
+		# prevent to switch the logging in the test cases (use inherited one):
+		server.DEF_LOGTARGET = "INHERITED"
+		server.DEF_LOGLEVEL = DefLogSys.level
 		Fail2banCmdLine._exit = staticmethod(self._test_exit)
 
 	def tearDown(self):
 		"""Call after every test case."""
 		Fail2banCmdLine._exit = self._orig_exit
+		# restore server log target:
+		server.DEF_LOGTARGET = SRV_DEF_LOGTARGET
+		server.DEF_LOGLEVEL = SRV_DEF_LOGLEVEL
 		LogCaptureTestCase.tearDown(self)
 
 	@staticmethod
@@ -303,47 +369,12 @@ class Fail2banClientServerBase(LogCaptureTestCase):
 		phase['end'] = True
 		logSys.debug("end of test worker")
 
-	@with_tmpdir
+	@with_foreground_server_thread()
-	def testStartForeground(self, tmp):
+	def testStartForeground(self, tmp, startparams):
-		# intended to be ran only in subclasses
+		# several commands to server:
-		th = None
+		self.execSuccess(startparams, "ping")
-		phase = dict()
+		self.execFailed(startparams, "~~unknown~cmd~failed~~")
-		try:
+		self.execSuccess(startparams, "echo", "TEST-ECHO")
-			# started directly here, so prevent overwrite test cases logger with "INHERITED"
-			startparams = _start_params(tmp, logtarget="INHERITED")
-			# because foreground block execution - start it in thread:
-			th = Thread(
-				name="_TestCaseWorker",
-				target=self._testStartForeground,
-				args=(tmp, startparams, phase)
-			)
-			th.daemon = True
-			th.start()
-			try:
-				# wait for start thread:
-				Utils.wait_for(lambda: phase.get('start', None) is not None, MAX_WAITTIME)
-				self.assertTrue(phase.get('start', None))
-				# wait for server (socket and ready):
-				self._wait_for_srv(tmp, True, startparams=startparams)
-				self.pruneLog()
-				# several commands to server:
-				self.execSuccess(startparams, "ping")
-				self.execFailed(startparams, "~~unknown~cmd~failed~~")
-				self.execSuccess(startparams, "echo", "TEST-ECHO")
-			finally:
-				self.pruneLog()
-				# stop:
-				self.execSuccess(startparams, "stop")
-				# wait for end:
-				Utils.wait_for(lambda: phase.get('end', None) is not None, MAX_WAITTIME)
-				self.assertTrue(phase.get('end', None))
-				self.assertLogged("Shutdown successful", "Exiting Fail2ban")
-		finally:
-			if th:
-				# we start client/server directly in current process (new thread),
-				# so don't kill (same process) - if success, just wait for end of worker:
-				if phase.get('end', None):
-					th.join()
 
 
 class Fail2banClientTest(Fail2banClientServerBase):
@@ -508,6 +539,24 @@ class Fail2banClientTest(Fail2banClientServerBase):
 		self.assertLogged("Usage: ")
 		self.pruneLog()
 
+	@with_tmpdir
+	def testClientFailCommands(self, tmp):
+		# started directly here, so prevent overwrite test cases logger with "INHERITED"
+		startparams = _start_params(tmp, logtarget="INHERITED")
+
+		# not started:
+		self.execFailed(startparams,
+			"reload", "jail")
+		self.assertLogged("Could not find server")
+		self.pruneLog()
+
+		# unexpected arg:
+		self.execFailed(startparams,
+			"--async", "reload", "--xxx", "jail")
+		self.assertLogged("Unexpected argument(s) for reload:")
+		self.pruneLog()
+
+
 	def testVisualWait(self):
 		sleeptime = 0.035
 		for verbose in (2, 0):
@@ -612,3 +661,315 @@ class Fail2banServerTest(Fail2banClientServerBase):
 		# again:
 		self.assertTrue(_kill_srv(tmp))
 		self.assertLogged("cleanup: no pidfile for")
+
+	@with_foreground_server_thread(startextra={'db': 'auto'})
+	def testServerReloadTest(self, tmp, startparams):
+		# Very complicated test-case, that expected running server (foreground in thread).
+		#
+		# In this test-case, each phase is related from previous one, 
+		# so it cannot be splitted in multiple test cases.
+		# Additionaly many log-messages used as ready-sign (to wait for end of phase).
+		#
+		# Used file database (instead of :memory:), to restore bans and log-file positions,
+		# after restart/reload between phases.
+		cfg = pjoin(tmp, "config")
+		test1log = pjoin(tmp, "test1.log")
+		test2log = pjoin(tmp, "test2.log")
+		test3log = pjoin(tmp, "test3.log")
+
+		os.mkdir(pjoin(cfg, "action.d"))
+		def _write_action_cfg(actname="test-action1", allow=True, 
+			start="", reload="", ban="", unban="", stop=""):
+			fn = pjoin(cfg, "action.d", "%s.conf" % actname)
+			if not allow:
+				os.remove(fn)
+				return
+			_write_file(fn, "w",
+				"[Definition]",
+				"actionstart =  echo '[<name>] %s: ** start'" % actname, start,
+				"actionreload = echo '[<name>] %s: .. reload'" % actname, reload,
+				"actionban =    echo '[<name>] %s: ++ ban <ip>'" % actname, ban,
+				"actionunban =  echo '[<name>] %s: -- unban <ip>'" % actname, unban,
+				"actionstop =   echo '[<name>] %s: __ stop'" % actname, stop,
+			)
+			if DefLogSys.level <= logging.DEBUG:  # if DEBUG
+				_out_file(fn)
+
+		def _write_jail_cfg(enabled=(1, 2), actions=()):
+			_write_file(pjoin(cfg, "jail.conf"), "w",
+				"[INCLUDES]", "",
+				"[DEFAULT]", "",
+				"usedns = no",
+				"maxretry = 3",
+				"findtime = 10m",
+				"failregex = ^\s*failure (401|403) from <HOST>",
+				"",
+				"[test-jail1]", "backend = polling", "filter =", 
+				"action = ",
+				"         test-action1[name='%(__name__)s']" if 1 in actions else "",
+				"         test-action2[name='%(__name__)s']" if 2 in actions else "",
+				"logpath = " + test1log,
+				"          " + test2log if 2 in enabled else "",
+				"          " + test3log if 2 in enabled else "",
+				"failregex = ^\s*failure (401|403) from <HOST>",
+				"            ^\s*error (401|403) from <HOST>" if 2 in enabled else "",
+				"enabled = true" if 1 in enabled else "",
+				"",
+				"[test-jail2]", "backend = polling", "filter =", 
+				"action =",
+				"logpath = " + test2log,
+				"enabled = true" if 2 in enabled else "",
+			)
+			if DefLogSys.level <= logging.DEBUG:  # if DEBUG
+				_out_file(pjoin(cfg, "jail.conf"))
+
+		# create default test actions:
+		_write_action_cfg(actname="test-action1")
+		_write_action_cfg(actname="test-action2")
+
+		_write_jail_cfg(enabled=[1], actions=[1,2])
+		_write_file(test1log, "w", *((str(int(MyTime.time())) + " failure 401 from 192.0.2.1: test 1",) * 3))
+		_write_file(test2log, "w")
+		_write_file(test3log, "w")
+		
+		# reload and wait for ban:
+		self.pruneLog("[test-phase 1a]")
+		if DefLogSys.level < logging.DEBUG:  # if HEAVYDEBUG
+			_out_file(test1log)
+		self.execSuccess(startparams, "reload")
+		self.assertLogged(
+			"Reload finished.",
+			"1 ticket(s) in 'test-jail1", all=True, wait=MID_WAITTIME)
+		self.assertLogged("Added logfile: %r" % test1log)
+		self.assertLogged("[test-jail1] Ban 192.0.2.1")
+		# test actions started:
+		self.assertLogged(
+			"stdout: '[test-jail1] test-action1: ** start'", 
+			"stdout: '[test-jail1] test-action2: ** start'", all=True)
+		
+		# enable both jails, 3 logs for jail1, etc...
+		# truncate test-log - we should not find unban/ban again by reload:
+		self.pruneLog("[test-phase 1b]")
+		_write_jail_cfg(actions=[1,2])
+		_write_file(test1log, "w+")
+		if DefLogSys.level < logging.DEBUG:  # if HEAVYDEBUG
+			_out_file(test1log)
+		self.execSuccess(startparams, "reload")
+		self.assertLogged("Reload finished.", all=True, wait=MID_WAITTIME)
+		# test not unbanned / banned again:
+		self.assertNotLogged(
+			"[test-jail1] Unban 192.0.2.1", 
+			"[test-jail1] Ban 192.0.2.1", all=True)
+		# test 2 new log files:
+		self.assertLogged(
+			"Added logfile: %r" % test2log, 
+			"Added logfile: %r" % test3log, all=True)
+		# test actions reloaded:
+		self.assertLogged(
+			"stdout: '[test-jail1] test-action1: .. reload'", 
+			"stdout: '[test-jail1] test-action2: .. reload'", all=True)
+		# test 1 new jail:
+		self.assertLogged(
+			"Creating new jail 'test-jail2'",
+			"Jail 'test-jail2' started", all=True)
+		
+		# update action1, delete action2 (should be stopped via configuration)...
+		self.pruneLog("[test-phase 2a]")
+		_write_jail_cfg(actions=[1])
+		_write_action_cfg(actname="test-action1", 
+			start= "               echo '[<name>] %s: started.'" % "test-action1",
+			reload="               echo '[<name>] %s: reloaded.'" % "test-action1", 
+			stop=  "               echo '[<name>] %s: stopped.'" % "test-action1")
+		self.execSuccess(startparams, "reload")
+		self.assertLogged("Reload finished.", all=True, wait=MID_WAITTIME)
+		# test not unbanned / banned again:
+		self.assertNotLogged(
+			"[test-jail1] Unban 192.0.2.1", 
+			"[test-jail1] Ban 192.0.2.1", all=True)
+		# no new log files:
+		self.assertNotLogged("Added logfile:")
+		# test action reloaded (update):
+		self.assertLogged(
+			"stdout: '[test-jail1] test-action1: .. reload'",
+			"stdout: '[test-jail1] test-action1: reloaded.'", all=True)
+		# test stopped action unbans:
+		self.assertLogged(
+			"stdout: '[test-jail1] test-action2: -- unban 192.0.2.1'")
+		# test action stopped:
+		self.assertLogged(
+			"stdout: '[test-jail1] test-action2: __ stop'")
+		self.assertNotLogged(
+			"stdout: '[test-jail1] test-action1: -- unban 192.0.2.1'")
+		
+		# don't need both actions anymore:
+		_write_action_cfg(actname="test-action1", allow=False)
+		_write_action_cfg(actname="test-action2", allow=False)
+		_write_jail_cfg(actions=[])
+		
+		# write new failures:
+		self.pruneLog("[test-phase 2b]")
+		_write_file(test2log, "w+", *(
+			(str(int(MyTime.time())) + "   error 403 from 192.0.2.2: test 2",) * 3 +
+		  (str(int(MyTime.time())) + "   error 403 from 192.0.2.3: test 2",) * 3 +
+		  (str(int(MyTime.time())) + " failure 401 from 192.0.2.4: test 2",) * 3 +
+		  (str(int(MyTime.time())) + " failure 401 from 192.0.2.8: test 2",) * 3
+		))
+		if DefLogSys.level < logging.DEBUG:  # if HEAVYDEBUG
+			_out_file(test2log)
+		# test all will be found in jail1 and one in jail2:
+		self.assertLogged(
+			"2 ticket(s) in 'test-jail2",
+			"5 ticket(s) in 'test-jail1", all=True, wait=MID_WAITTIME)
+		self.assertLogged(
+			"[test-jail1] Ban 192.0.2.2",
+			"[test-jail1] Ban 192.0.2.3",
+			"[test-jail1] Ban 192.0.2.4",
+			"[test-jail1] Ban 192.0.2.8",
+			"[test-jail2] Ban 192.0.2.4",
+			"[test-jail2] Ban 192.0.2.8", all=True)
+		# test ips at all not visible for jail2:
+		self.assertNotLogged(
+			"[test-jail2] Found 192.0.2.2", 
+			"[test-jail2] Ban 192.0.2.2",
+			"[test-jail2] Found 192.0.2.3", 
+			"[test-jail2] Ban 192.0.2.3", all=True)
+
+		# rotate logs:
+		_write_file(test1log, "w+")
+		_write_file(test2log, "w+")
+
+		# restart jail without unban all:
+		self.pruneLog("[test-phase 2c]")
+		self.execSuccess(startparams,
+			"restart", "test-jail2")
+		self.assertLogged(
+			"Reload finished.",
+			"Restore Ban",
+			"2 ticket(s) in 'test-jail2", all=True, wait=MID_WAITTIME)
+		# stop/start and unban/restore ban:
+		self.assertLogged(
+			"Jail 'test-jail2' stopped",
+			"Jail 'test-jail2' started",
+			"[test-jail2] Unban 192.0.2.4",
+			"[test-jail2] Unban 192.0.2.8",
+			"[test-jail2] Restore Ban 192.0.2.4",
+			"[test-jail2] Restore Ban 192.0.2.8", all=True
+		)
+
+		# restart jail with unban all:
+		self.pruneLog("[test-phase 2d]")
+		self.execSuccess(startparams,
+			"restart", "--unban", "test-jail2")
+		self.assertLogged(
+			"Reload finished.",
+			"Jail 'test-jail2' started", all=True, wait=MID_WAITTIME)
+		self.assertLogged(
+			"Jail 'test-jail2' stopped",
+			"Jail 'test-jail2' started",
+			"[test-jail2] Unban 192.0.2.4",
+			"[test-jail2] Unban 192.0.2.8", all=True
+		)
+		# no more ban (unbanned all):
+		self.assertNotLogged(
+			"[test-jail2] Ban 192.0.2.4",
+			"[test-jail2] Ban 192.0.2.8", all=True
+		)
+
+		# reload jail1 without restart (without ban/unban):
+		self.pruneLog("[test-phase 3]")
+		self.execSuccess(startparams, "reload", "test-jail1")
+		self.assertLogged(
+			"Reload finished.", all=True, wait=MID_WAITTIME)
+		self.assertLogged(
+			"Reload jail 'test-jail1'",
+			"Jail 'test-jail1' reloaded", all=True)
+		self.assertNotLogged(
+			"Reload jail 'test-jail2'",
+			"Jail 'test-jail2' reloaded",
+			"Jail 'test-jail1' started", all=True
+		)
+
+		# whole reload, but this time with jail1 only (jail2 should be stopped via configuration):
+		self.pruneLog("[test-phase 4]")
+		_write_jail_cfg(enabled=[1])
+		self.execSuccess(startparams, "reload")
+		self.assertLogged("Reload finished.", all=True, wait=MID_WAITTIME)
+		# test both jails should be reloaded:
+		self.assertLogged(
+			"Reload jail 'test-jail1'")
+		# test jail2 goes down:
+		self.assertLogged(
+			"Stopping jail 'test-jail2'", 
+			"Jail 'test-jail2' stopped", all=True)
+		# test 2 log files removed:
+		self.assertLogged(
+			"Removed logfile: %r" % test2log, 
+			"Removed logfile: %r" % test3log, all=True)
+
+		# now write failures again and check already banned (jail1 was alive the whole time) and new bans occurred (jail1 was alive the whole time):
+		self.pruneLog("[test-phase 5]")
+		_write_file(test1log, "w+", *(
+			(str(int(MyTime.time())) + " failure 401 from 192.0.2.1: test 5",) * 3 + 
+			(str(int(MyTime.time())) + "   error 403 from 192.0.2.5: test 5",) * 3 +
+			(str(int(MyTime.time())) + " failure 401 from 192.0.2.6: test 5",) * 3
+		))
+		if DefLogSys.level < logging.DEBUG:  # if HEAVYDEBUG
+			_out_file(test1log)
+		self.assertLogged(
+			"6 ticket(s) in 'test-jail1",
+			"[test-jail1] 192.0.2.1 already banned", all=True, wait=MID_WAITTIME)
+		# test "failure" regexp still available:
+		self.assertLogged(
+			"[test-jail1] Found 192.0.2.1",
+			"[test-jail1] Found 192.0.2.6",
+			"[test-jail1] 192.0.2.1 already banned",
+			"[test-jail1] Ban 192.0.2.6", all=True)
+		# test "error" regexp no more available:
+		self.assertNotLogged("[test-jail1] Found 192.0.2.5")
+
+		# unban single ips:
+		self.pruneLog("[test-phase 6]")
+		self.execSuccess(startparams,
+			"--async", "unban", "192.0.2.5", "192.0.2.6")
+		self.assertLogged(
+			"192.0.2.5 is not banned",
+			"[test-jail1] Unban 192.0.2.6", all=True
+		)
+
+		# reload all (one jail) with unban all:
+		self.pruneLog("[test-phase 7]")
+		self.execSuccess(startparams,
+			"reload", "--unban")
+		self.assertLogged("Reload finished.", all=True, wait=MID_WAITTIME)
+		# reloads unbanned all:
+		self.assertLogged(
+			"Jail 'test-jail1' reloaded",
+			"[test-jail1] Unban 192.0.2.1",
+			"[test-jail1] Unban 192.0.2.2",
+			"[test-jail1] Unban 192.0.2.3",
+			"[test-jail1] Unban 192.0.2.4", all=True
+		)
+		# no restart occurred, no more ban (unbanned all using option "--unban"):
+		self.assertNotLogged(
+			"Jail 'test-jail1' stopped",
+			"Jail 'test-jail1' started",
+			"[test-jail1] Ban 192.0.2.1",
+			"[test-jail1] Ban 192.0.2.2",
+			"[test-jail1] Ban 192.0.2.3",
+			"[test-jail1] Ban 192.0.2.4", all=True
+		)
+
+		# several small cases (cover several parts):
+		self.pruneLog("[test-phase end-1]")
+		# wrong jail (not-started):
+		self.execFailed(startparams,
+			"--async", "reload", "test-jail2")
+		self.assertLogged("the jail 'test-jail2' does not exist")
+		self.pruneLog()
+		# unavailable jail (but exit 0), using --if-exists option:
+		self.execSuccess(startparams,
+			"--async", "reload", "--if-exists", "test-jail2")
+		self.assertNotLogged(
+			"Creating new jail 'test-jail2'",
+			"Jail 'test-jail2' started", all=True)
+		self.pruneLog()

fail2ban/tests/fail2banregextestcase.py

@@ -131,6 +131,15 @@ class Fail2banRegexTest(LogCaptureTestCase):
 		self.assertTrue(fail2banRegex.start(opts, args))
 		self.assertLogged('Lines: 19 lines, 0 ignored, 16 matched, 3 missed')
 
+	def testDirectRE_1raw_noDns(self):
+		(opts, args, fail2banRegex) = _Fail2banRegex(
+			"--print-all-matched", "--raw", "--usedns=no",
+			Fail2banRegexTest.FILENAME_01, 
+			Fail2banRegexTest.RE_00
+		)
+		self.assertTrue(fail2banRegex.start(opts, args))
+		self.assertLogged('Lines: 19 lines, 0 ignored, 13 matched, 6 missed')
+
 	def testDirectRE_2(self):
 		(opts, args, fail2banRegex) = _Fail2banRegex(
 			"--print-all-matched",

fail2ban/tests/filtertestcase.py

@@ -38,11 +38,11 @@ except ImportError:
 
 from ..server.jail import Jail
 from ..server.filterpoll import FilterPoll
-from ..server.filter import Filter, FileFilter, FileContainer, locale
+from ..server.filter import Filter, FileFilter, FileContainer
 from ..server.failmanager import FailManagerEmpty
 from ..server.ipdns import DNSUtils, IPAddr
 from ..server.mytime import MyTime
-from ..server.utils import Utils
+from ..server.utils import Utils, uni_decode
 from .utils import setUpMyTime, tearDownMyTime, mtimesleep, LogCaptureTestCase
 from .dummyjail import DummyJail
 
@@ -311,8 +311,7 @@ class BasicFilter(unittest.TestCase):
 					b'Fail for "g\xc3\xb6ran" from 192.0.2.1'
 				):
 					# join should work if all arguments have the same type:
-					enc = locale.getpreferredencoding()
+					"".join([uni_decode(v) for v in (a1, a2, a3)])
-					"".join([Filter.uni_decode(v, enc) for v in (a1, a2, a3)])
 
 
 class IgnoreIP(LogCaptureTestCase):

fail2ban/tests/misctestcase.py

@@ -35,7 +35,7 @@ from StringIO import StringIO
 
 from utils import LogCaptureTestCase, logSys as DefLogSys
 
-from ..helpers import formatExceptionInfo, mbasename, TraceBack, FormatterWithTraceBack, getLogger
+from ..helpers import formatExceptionInfo, mbasename, TraceBack, FormatterWithTraceBack, getLogger, uni_decode
 from ..helpers import splitwords
 from ..server.datedetector import DateDetector
 from ..server.datetemplate import DatePatternRegex
@@ -74,16 +74,14 @@ class HelpersTest(unittest.TestCase):
 
 if sys.version_info >= (2,7):
 	def _sh_call(cmd):
-		import subprocess, locale
+		import subprocess
 		ret = subprocess.check_output(cmd, shell=True)
-		if sys.version_info >= (3,):
+		return uni_decode(ret).rstrip()
-			ret = ret.decode(locale.getpreferredencoding(), 'replace')
-		return str(ret).rstrip()
 else:
 	def _sh_call(cmd):
 		import subprocess
 		ret = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read()
-		return str(ret).rstrip()
+		return uni_decode(ret).rstrip()
 
 def _getSysPythonVersion():
 	return _sh_call("fail2ban-python -c 'import sys; print(tuple(sys.version_info))'")
@@ -286,6 +284,10 @@ class TestsUtilsTest(LogCaptureTestCase):
 			self.assertLogged, 'test_zyx', 'zyx', all=False)
 		self._testAssertionErrorRE(r"All of the .* were found present in the log",
 			self.assertNotLogged, 'test', 'xyz', all=False)
+		## assertDictEqual:
+		self.assertDictEqual({'A': [1, 2]}, {'A': [1, 2]})
+		self.assertRaises(AssertionError, self.assertDictEqual, 
+			{'A': [1, 2]}, {'A': [2, 1]})
 
 	def testFormatterWithTraceBack(self):
 		strout = StringIO()

fail2ban/tests/servertestcase.py

@@ -28,7 +28,6 @@ import unittest
 import time
 import tempfile
 import os
-import locale
 import sys
 import platform
 
@@ -40,7 +39,7 @@ from ..server.jail import Jail
 from ..server.jailthread import JailThread
 from ..server.utils import Utils
 from .utils import LogCaptureTestCase
-from ..helpers import getLogger
+from ..helpers import getLogger, PREFER_ENC
 from .. import version
 
 try:
@@ -240,7 +239,7 @@ class Transmitter(TransmitterBase):
 			self.transm.proceed(["add", self.jailName, "polling"])[0], 1)
 		# All name is reserved
 		self.assertEqual(
-			self.transm.proceed(["add", "all", "polling"])[0], 1)
+			self.transm.proceed(["add", "--all", "polling"])[0], 1)
 
 	def testStartStopJail(self):
 		self.assertEqual(
@@ -267,7 +266,7 @@ class Transmitter(TransmitterBase):
 		self.assertTrue( Utils.wait_for(
 			lambda: self.server.isAlive(2) and not isinstance(self.transm.proceed(["status", self.jailName]), RuntimeError),
 			3) )
-		self.assertEqual(self.transm.proceed(["stop", "all"]), (0, None))
+		self.assertEqual(self.transm.proceed(["stop", "--all"]), (0, None))
 		self.assertTrue( Utils.wait_for( lambda: not len(self.server._Server__jails), 3) )
 		self.assertNotIn(self.jailName, self.server._Server__jails)
 		self.assertNotIn("TestJail2", self.server._Server__jails)
@@ -354,7 +353,7 @@ class Transmitter(TransmitterBase):
 	def testJailLogEncoding(self):
 		self.setGetTest("logencoding", "UTF-8", jail=self.jailName)
 		self.setGetTest("logencoding", "ascii", jail=self.jailName)
-		self.setGetTest("logencoding", "auto", locale.getpreferredencoding(),
+		self.setGetTest("logencoding", "auto", PREFER_ENC,
 			jail=self.jailName)
 		self.setGetTestNOK("logencoding", "Monkey", jail=self.jailName)
 
@@ -843,6 +842,8 @@ class TransmitterLogging(TransmitterBase):
 
 	def testLogLevel(self):
 		self.setGetTest("loglevel", "HEAVYDEBUG")
+		self.setGetTest("loglevel", "TRACEDEBUG")
+		self.setGetTest("loglevel", "9")
 		self.setGetTest("loglevel", "DEBUG")
 		self.setGetTest("loglevel", "INFO")
 		self.setGetTest("loglevel", "NOTICE")

fail2ban/tests/tickettestcase.py

@@ -108,6 +108,24 @@ class TicketTests(unittest.TestCase):
     self.assertEqual(ft2.getLastTime(), ft.getLastTime())
     self.assertEqual(ft2.getBanTime(), ft.getBanTime())
 
+  def testTicketFlags(self):
+    flags = ('restored', 'banned')
+    ticket = Ticket('test', 0)
+    trueflags = []
+    for v in (True, False, True):
+      for f in flags:
+        setattr(ticket, f, v)
+        if v:
+          trueflags.append(f)
+        else:
+          trueflags.remove(f)
+        for f2 in flags:
+          self.assertEqual(bool(getattr(ticket, f2)), f2 in trueflags)
+    ## inherite props from another tockets:
+    ticket = FailTicket(ticket=ticket)
+    for f2 in flags:
+      self.assertTrue(bool(getattr(ticket, f2)))
+
   def testTicketData(self):
     t = BanTicket('193.168.0.128', None, ['first', 'second'])
     # expand data (no overwrites, matches are available) :

fail2ban/tests/utils.py

@@ -119,6 +119,7 @@ def getOptParser(doc=""):
 
 def initProcess(opts):
 	# Logger:
+	global logSys
 	logSys = getLogger("fail2ban")
 
 	# Numerical level of verbosity corresponding to a log "level"
@@ -242,6 +243,9 @@ def initTests(opts):
 			raise unittest.SkipTest('Skip test because of "--fast"')
 		unittest.F2B.SkipIfFast = F2B_SkipIfFast
 	else:
+		# smaller inertance inside test-cases (litle speedup):
+		Utils.DEFAULT_SLEEP_TIME = 0.25
+		Utils.DEFAULT_SLEEP_INTERVAL = 0.025
 		# sleep intervals are large - use replacement for sleep to check time to sleep:
 		_org_sleep = time.sleep
 		def _new_sleep(v):
@@ -462,6 +466,20 @@ def gatherTests(regexps=None, opts=None):
 # Forwards compatibility of unittest.TestCase for some early python versions
 #
 
+if not hasattr(unittest.TestCase, 'assertDictEqual'):
+	import difflib, pprint
+	def assertDictEqual(self, d1, d2, msg=None):
+		self.assert_(isinstance(d1, dict), 'First argument is not a dictionary')
+		self.assert_(isinstance(d2, dict), 'Second argument is not a dictionary')
+		if d1 != d2:
+			standardMsg = '%r != %r' % (d1, d2)
+			diff = ('\n' + '\n'.join(difflib.ndiff(
+				pprint.pformat(d1).splitlines(),
+				pprint.pformat(d2).splitlines())))
+			msg = msg or (standardMsg + diff)
+			self.fail(msg)
+	unittest.TestCase.assertDictEqual = assertDictEqual
+
 if not hasattr(unittest.TestCase, 'assertRaisesRegexp'):
 	def assertRaisesRegexp(self, exccls, regexp, fun, *args, **kwargs):
 		try:
@@ -577,7 +595,8 @@ class LogCaptureTestCase(unittest.TestCase):
 			print("")
 			logSys.handlers += self._old_handlers
 			logSys.debug('='*10 + ' %s ' + '='*20, self.id())
-		logSys.setLevel(logging.DEBUG)
+		else:
+			logSys.setLevel(logging.DEBUG)
 
 	def tearDown(self):
 		"""Call after every test case."""
@@ -587,8 +606,21 @@ class LogCaptureTestCase(unittest.TestCase):
 		logSys.handlers = self._old_handlers
 		logSys.level = self._old_level
 
-	def _is_logged(self, s):
+	def _is_logged(self, *s, **kwargs):
-		return s in self._log.getvalue()
+		logged = self._log.getvalue()
+		if not kwargs.get('all', False):
+			# at least one entry should be found:
+			for s_ in s:
+				if s_ in logged:
+					return True
+			if True: # pragma: no cover
+				return False
+		else:
+			# each entry should be found:
+			for s_ in s:
+				if s_ not in logged: # pragma: no cover
+					return False
+			return True
 
 	def assertLogged(self, *s, **kwargs):
 		"""Assert that one of the strings was logged
@@ -602,19 +634,23 @@ class LogCaptureTestCase(unittest.TestCase):
 		  Test should succeed if string (or any of the listed) is present in the log
 		all : boolean (default False) if True should fail if any of s not logged
 		"""
-		logged = self._log.getvalue()
+		wait = kwargs.get('wait', None)
+		if wait:
+			res = Utils.wait_for(lambda: self._is_logged(*s, **kwargs), wait)
+		else:
+			res = self._is_logged(*s, **kwargs)
 		if not kwargs.get('all', False):
 			# at least one entry should be found:
-			for s_ in s:
+			if not res: # pragma: no cover
-				if s_ in logged:
+				logged = self._log.getvalue()
-					return
-			if True: # pragma: no cover
 				self.fail("None among %r was found in the log: ===\n%s===" % (s, logged))
 		else:
 			# each entry should be found:
-			for s_ in s:
+			if not res: # pragma: no cover
-				if s_ not in logged: # pragma: no cover
+				logged = self._log.getvalue()
-					self.fail("%r was not found in the log: ===\n%s===" % (s_, logged))
+				for s_ in s:
+					if s_ not in logged:
+						self.fail("%r was not found in the log: ===\n%s===" % (s_, logged))
 
 	def assertNotLogged(self, *s, **kwargs):
 		"""Assert that strings were not logged
@@ -638,8 +674,10 @@ class LogCaptureTestCase(unittest.TestCase):
 				if s_ in logged: # pragma: no cover
 					self.fail("%r was found in the log: ===\n%s===" % (s_, logged))
 
-	def pruneLog(self):
+	def pruneLog(self, logphase=None):
 		self._log.truncate(0)
+		if logphase:
+			logSys.debug('='*5 + ' %s ' + '='*5, logphase)
 
 	def getLog(self):
 		return self._log.getvalue()
@@ -649,9 +687,3 @@ class LogCaptureTestCase(unittest.TestCase):
 
 
 pid_exists = Utils.pid_exists
-
-# Python 2.6 compatibility. in 2.7 assertDictEqual
-def assert_dict_equal(a, b):
-	assert isinstance(a, dict), "Object is not dictionary: %r" % a
-	assert isinstance(b, dict), "Object is not dictionary: %r" % b
-	assert a==b, "Dictionaries differ:\n%r !=\n%r" % (a, b)

man/fail2ban-client.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.44.1.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.47.3.
-.TH FAIL2BAN-CLIENT "1" "July 2016" "fail2ban-client v0.10.0a1" "User Commands"
+.TH FAIL2BAN-CLIENT "1" "September 2016" "fail2ban-client v0.10.0a2" "User Commands"
 .SH NAME
 fail2ban-client \- configure and control the server
 .SH SYNOPSIS
 .B fail2ban-client
-[\fIOPTIONS\fR] \fI<COMMAND>\fR
+[\fI\,OPTIONS\/\fR] \fI\,<COMMAND>\/\fR
 .SH DESCRIPTION
-Fail2Ban v0.10.0a1 reads log file that contains password failure report
+Fail2Ban v0.10.0a2 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .SH OPTIONS
 .TP
@@ -68,17 +68,36 @@ starts the server and the jails
 \fBrestart\fR
 restarts the server
 .TP
-\fBreload\fR
+\fBrestart [\-\-unban] [\-\-if\-exists] <JAIL>\fR
-reloads the configuration without
+restarts the jail <JAIL> (alias
-restart
+for 'reload \fB\-\-restart\fR ... <JAIL>')
 .TP
-\fBreload <JAIL>\fR
+\fBreload [\-\-restart] [\-\-unban] [\-\-all]\fR
-reloads the jail <JAIL>
+reloads the configuration without
+restarting of the server, the
+option '\-\-restart' activates
+completely restarting of affected
+jails, thereby can unban IP
+addresses (if option '\-\-unban'
+specified)
+.TP
+\fBreload [\-\-restart] [\-\-unban] [\-\-if\-exists] <JAIL>\fR
+reloads the jail <JAIL>, or
+restarts it (if option '\-\-restart'
+specified)
 .TP
 \fBstop\fR
 stops all jails and terminate the
 server
 .TP
+\fBunban \fB\-\-all\fR\fR
+unbans all IP addresses (in all
+jails and database)
+.TP
+\fBunban <IP> ... <IP>\fR
+unbans <IP> (in all jails and
+database)
+.TP
 \fBstatus\fR
 gets the current status of the
 server
@@ -101,7 +120,9 @@ LOGGING
 \fBset loglevel <LEVEL>\fR
 sets logging level to <LEVEL>.
 Levels: CRITICAL, ERROR, WARNING,
-NOTICE, INFO, DEBUG
+NOTICE, INFO, DEBUG, TRACEDEBUG,
+HEAVYDEBUG or corresponding
+numeric value (50\-5)
 .TP
 \fBget loglevel\fR
 gets the logging level
@@ -248,9 +269,8 @@ for <JAIL>
 \fBset <JAIL> maxlines <LINES>\fR
 sets the number of <LINES> to
 buffer for regex search for <JAIL>
-.IP
+.TP
-set <JAIL> addaction <ACT>[ <PYTHONFILE> <JSONKWARGS>]
+\fBset <JAIL> addaction <ACT>[ <PYTHONFILE> <JSONKWARGS>]\fR
-.IP
 adds a new action named <ACT> for
 <JAIL>. Optionally for a Python
 based action, a <PYTHONFILE> and
@@ -262,45 +282,38 @@ removes the action <ACT> from
 <JAIL>
 .IP
 COMMAND ACTION CONFIGURATION
-.IP
+.TP
-set <JAIL> action <ACT> actionstart <CMD>
+\fBset <JAIL> action <ACT> actionstart <CMD>\fR
-.IP
 sets the start command <CMD> of
 the action <ACT> for <JAIL>
-.IP
+.TP
-set <JAIL> action <ACT> actionstop <CMD> sets the stop command <CMD> of the
+\fBset <JAIL> action <ACT> actionstop <CMD> sets the stop command <CMD> of the\fR
-.IP
 action <ACT> for <JAIL>
-.IP
+.TP
-set <JAIL> action <ACT> actioncheck <CMD>
+\fBset <JAIL> action <ACT> actioncheck <CMD>\fR
-.IP
 sets the check command <CMD> of
 the action <ACT> for <JAIL>
 .TP
 \fBset <JAIL> action <ACT> actionban <CMD>\fR
 sets the ban command <CMD> of the
 action <ACT> for <JAIL>
-.IP
+.TP
-set <JAIL> action <ACT> actionunban <CMD>
+\fBset <JAIL> action <ACT> actionunban <CMD>\fR
-.IP
 sets the unban command <CMD> of
 the action <ACT> for <JAIL>
-.IP
+.TP
-set <JAIL> action <ACT> timeout <TIMEOUT>
+\fBset <JAIL> action <ACT> timeout <TIMEOUT>\fR
-.IP
 sets <TIMEOUT> as the command
 timeout in seconds for the action
 <ACT> for <JAIL>
 .IP
 GENERAL ACTION CONFIGURATION
-.IP
+.TP
-set <JAIL> action <ACT> <PROPERTY> <VALUE>
+\fBset <JAIL> action <ACT> <PROPERTY> <VALUE>\fR
-.IP
 sets the <VALUE> of <PROPERTY> for
 the action <ACT> for <JAIL>
-.IP
+.TP
-set <JAIL> action <ACT> <METHOD>[ <JSONKWARGS>]
+\fBset <JAIL> action <ACT> <METHOD>[ <JSONKWARGS>]\fR
-.IP
 calls the <METHOD> with
 <JSONKWARGS> for the action <ACT>
 for <JAIL>

man/fail2ban-regex.1

@@ -1,10 +1,10 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.44.1.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.47.3.
-.TH FAIL2BAN-REGEX "1" "July 2016" "fail2ban-regex 0.10.0a1" "User Commands"
+.TH FAIL2BAN-REGEX "1" "September 2016" "fail2ban-regex 0.10.0a2" "User Commands"
 .SH NAME
 fail2ban-regex \- test Fail2ban "failregex" option
 .SH SYNOPSIS
 .B fail2ban-regex
-[\fIOPTIONS\fR] \fI<LOG> <REGEX> \fR[\fIIGNOREREGEX\fR]
+[\fI\,OPTIONS\/\fR] \fI\,<LOG> <REGEX> \/\fR[\fI\,IGNOREREGEX\/\fR]
 .SH DESCRIPTION
 Fail2Ban  reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
@@ -16,7 +16,7 @@ string
 a string representing a log line
 .TP
 filename
-path to a log file (\fI/var/log/auth.log\fP)
+path to a log file (\fI\,/var/log/auth.log\/\fP)
 .TP
 "systemd\-journal"
 search systemd journal (systemd\-python required)
@@ -42,23 +42,28 @@ show program's version number and exit
 \fB\-h\fR, \fB\-\-help\fR
 show this help message and exit
 .TP
-\fB\-d\fR DATEPATTERN, \fB\-\-datepattern\fR=\fIDATEPATTERN\fR
+\fB\-d\fR DATEPATTERN, \fB\-\-datepattern\fR=\fI\,DATEPATTERN\/\fR
 set custom pattern used to match date/times
 .TP
-\fB\-e\fR ENCODING, \fB\-\-encoding\fR=\fIENCODING\fR
+\fB\-e\fR ENCODING, \fB\-\-encoding\fR=\fI\,ENCODING\/\fR
 File encoding. Default: system locale
 .TP
 \fB\-r\fR, \fB\-\-raw\fR
 Raw hosts, don't resolve dns
 .TP
-\fB\-L\fR MAXLINES, \fB\-\-maxlines\fR=\fIMAXLINES\fR
+\fB\-\-usedns\fR=\fI\,USEDNS\/\fR
+DNS specified replacement of tags <HOST> in regexp
+('yes' \- matches all form of hosts, 'no' \- IP
+addresses only)
+.TP
+\fB\-L\fR MAXLINES, \fB\-\-maxlines\fR=\fI\,MAXLINES\/\fR
 maxlines for multi\-line regex
 .TP
-\fB\-m\fR JOURNALMATCH, \fB\-\-journalmatch\fR=\fIJOURNALMATCH\fR
+\fB\-m\fR JOURNALMATCH, \fB\-\-journalmatch\fR=\fI\,JOURNALMATCH\/\fR
 journalctl style matches overriding filter file.
 "systemd\-journal" only
 .TP
-\fB\-l\fR LOG_LEVEL, \fB\-\-log\-level\fR=\fILOG_LEVEL\fR
+\fB\-l\fR LOG_LEVEL, \fB\-\-log\-level\fR=\fI\,LOG_LEVEL\/\fR
 Log level for the Fail2Ban logger to use
 .TP
 \fB\-v\fR, \fB\-\-verbose\fR

man/fail2ban-server.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.44.1.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.47.3.
-.TH FAIL2BAN-SERVER "1" "July 2016" "fail2ban-server v0.10.0a1" "User Commands"
+.TH FAIL2BAN-SERVER "1" "September 2016" "fail2ban-server v0.10.0a2" "User Commands"
 .SH NAME
 fail2ban-server \- start the server
 .SH SYNOPSIS
 .B fail2ban-server
-[\fIOPTIONS\fR]
+[\fI\,OPTIONS\/\fR]
 .SH DESCRIPTION
-Fail2Ban v0.10.0a1 reads log file that contains password failure report
+Fail2Ban v0.10.0a2 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .SH OPTIONS
 .TP

man/fail2ban-testcases.1

@@ -1,10 +1,10 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.44.1.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.47.3.
-.TH FAIL2BAN-TESTCASES "1" "July 2016" "fail2ban-testcases 0.10.0a1" "User Commands"
+.TH FAIL2BAN-TESTCASES "1" "September 2016" "fail2ban-testcases 0.10.0a2" "User Commands"
 .SH NAME
 fail2ban-testcases \- run Fail2Ban unit-tests
 .SH SYNOPSIS
 .B fail2ban-testcases
-[\fIOPTIONS\fR] [\fIregexps\fR]
+[\fI\,OPTIONS\/\fR] [\fI\,regexps\/\fR]
 .SH DESCRIPTION
 Script to run Fail2Ban tests battery
 .SH OPTIONS
@@ -15,9 +15,15 @@ show program's version number and exit
 \fB\-h\fR, \fB\-\-help\fR
 show this help message and exit
 .TP
-\fB\-l\fR LOG_LEVEL, \fB\-\-log\-level\fR=\fILOG_LEVEL\fR
+\fB\-l\fR LOG_LEVEL, \fB\-\-log\-level\fR=\fI\,LOG_LEVEL\/\fR
 Log level for the logger to use during running tests
 .TP
+\fB\-v\fR VERBOSITY, \fB\-\-verbosity\fR=\fI\,VERBOSITY\/\fR
+Set numerical level of verbosity (0..4)
+.TP
+\fB\-\-log\-direct\fR
+Prevent lazy logging inside tests
+.TP
 \fB\-n\fR, \fB\-\-no\-network\fR
 Do not run tests that require the network
 .TP

man/jail.conf.5

@@ -127,7 +127,7 @@ These files have one section, [Definition].
 The items that can be set are:
 .TP
 .B loglevel
-verbosity level of log output: CRITICAL, ERROR, WARNING, NOTICE, INFO, DEBUG. Default: ERROR
+verbosity level of log output: CRITICAL, ERROR, WARNING, NOTICE, INFO, DEBUG, TRACEDEBUG, HEAVYDEBUG or corresponding numeric value (50-5). Default: ERROR (equal 40)
 .TP
 .B logtarget
 log target: filename, SYSLOG, STDERR or STDOUT. Default: STDERR