Merge commit '0.8.8-145-g72b0647' into debian

* commit '0.8.8-145-g72b0647': (114 commits)
  ENH: Slight tune ups for fresh SOGo filter + comment into the sample log file
  ENH: postfix filter -- react also on (450 4.7.1) with empty from/to. fixes #126
  TST: basic testing of reading the shipped jail.conf (forcing all jails to be enabled)
  ENH: allow to force enable all jails (for testing), do not crash for jails without actions (just warn)
  ENH: minor -- add default value into the warning if option had none provided
  ENH: _copy_lines_between_files -- read all needed, and only then write/flush at once
  ENH: move pyinotify callback debug message into callback + delay string interpolations
  ENH: adding ability to incorporate tracebacks into log lines while running tests
  ENH: FailManager -- improve log message to report total # of detected failures as well
  BF: allow to wait longer for FilterPoll in test_move_file
  ENH: elaborated debug log message about already detected failures
  BF: Remove custom __str__ for MonitorFailures and just adjust __name__ of the generated class
  DOC: Added suggested by @beilber description of .d/ + added I formatting to all filenames
  ENH: increase timeout to 20 sec from 10 sec in assert_correct_last_attempt
  BF: fixing up for handling of TAI64N timestamps and adding some unittest for prev commit (not effective much though)
  An example of failed logins against sogo
  Update sogo-auth.conf
  Added Daniel and Steven to THANKS
  My improvements to manpages
  PKG: change email that I want in RPMs
  ...

.coveragerc

@@ -0,0 +1,4 @@
+
+[run]
+branch = True
+omit = /usr*

.gitignore

@@ -1 +1,8 @@
 *~
+build
+dist
+*.pyc
+htmlcov
+.coverage
+*.orig
+*.rej

DEVELOP

@@ -24,14 +24,100 @@ Request feature. You can find more details on the Fail2Ban wiki
 Testing
 =======
 
-Existing tests can be run by executing `fail2ban-testcases`.
+Existing tests can be run by executing `fail2ban-testcases`. This has options
+like --log-level that will probably be useful. `fail2ban-testcases --help` for
+full options.
+
+Test cases should cover all usual cases, all exception cases and all inside
+/ outside boundary conditions.
+
+Test cases should cover all branches. The coverage tool will help identify
+missing branches. Also see http://nedbatchelder.com/code/coverage/branch.html
+for more details.
+
+Install the package python-coverage to visualise your test coverage. Run the
+following (note: on Debian-based systems, the script is called
+`python-coverage`):
+
+coverage run fail2ban-testcases
+coverage html
+
+Then look at htmlcov/index.html and see how much coverage your test cases
+exert over the codebase. Full coverage is a good thing however it may not be
+complete. Try to ensure tests cover as many independent paths through the
+code.
+
+Manual Execution. To run in a development environment do:
+
+./fail2ban-client -c config/ -s /tmp/f2b.sock -i start
+
+some quick commands:
+
+status
+add test pyinotify
+status test
+set test addaction iptables
+set test actionban iptables echo <ip> <cidr> >> /tmp/ban
+set test actionunban iptables echo <ip> <cidr> >> /tmp/unban
+get test actionban iptables
+get test actionunban iptables
+set test banip 192.168.2.2
+status test
+
 
-Documentation about creating tests (when tests are required and some guidelines
-for creating good tests) will be added soon.
 
 Coding Standards
 ================
-Coming Soon.
+
+Style
+-----
+
+Please use tabs for now. Keep to 80 columns, at least for readable text.
+
+Tests
+-----
+
+Add tests. They should test all the code you add in a meaning way.
+
+Coverage
+--------
+
+Test coverage should always increase as you add code.
+
+You may use "# pragma: no cover" in the code for branches of code that support
+older versions on python. For all other uses of "pragma: no cover" or
+"pragma: no branch" document the reason why its not covered. "I haven't written
+a test case" isn't a sufficient reason.
+
+Documentation
+-------------
+
+Ensure this documentation is up to date after changes. Also ensure that the man
+pages still are accurate. Ensure that there is sufficient documentation for
+your new features to be used.
+
+Bugs
+----
+
+Remove them and don't add any more.
+
+Git
+---
+
+Use the following tags in your commit messages:
+
+'BF:' for bug fixes
+'DOC:' for documentation fixes
+'ENH:' for enhancements
+'TST:' for commits concerning tests only (thus not touching the main code-base)
+
+Multiple tags could be joined with +, e.g. "BF+TST:".
+
+Adding Actions
+--------------
+
+If you add an action.d/*.conf file also add a example in config/jail.conf
+with enabled=false and maxretry=5 for ssh.
 
 
 Design
@@ -127,12 +213,14 @@ FileContainer
   .__pos
     Keeps the position pointer
 
+
+dnsutils.py
+~~~~~~~~~~~
+
 DNSUtils
 
   Utility class for DNS and IP handling
 
-  RF-Note: convert to functions within a separate submodule
-
 
 filter*.py
 ~~~~~~~~~~
@@ -156,3 +244,35 @@ action.py
 ~~~~~~~~~
 
 Takes care about executing start/check/ban/unban/stop commands
+
+
+Releasing
+=========
+
+# Ensure the version is correct in ./common/version.py
+
+# Add/finalize the corresponding entry in the ChangeLog
+
+# Update man pages
+
+  (cd man ; ./generate-man )
+  git commit -m 'update man pages for release' man/*
+
+# Make sure the tests pass
+
+  ./fail2ban-testcases-all
+
+# Prepare/upload source and rpm binary distributions
+
+  python setup.py check
+  python setup.py sdist
+  python setup.py bdist_rpm
+  python setup.py upload
+
+# Run the following and update the wiki with output:
+
+  python -c 'import common.protocol; common.protocol.printWiki()'
+
+# Email users and development list of release
+
+TODO notifying distributors etc.

MANIFEST

@@ -3,6 +3,8 @@ ChangeLog
 TODO
 THANKS
 COPYING
+DEVELOP
+doc/run-rootless.txt
 fail2ban-client
 fail2ban-server
 fail2ban-testcases
@@ -40,6 +42,7 @@ server/banmanager.py
 server/datetemplate.py
 server/mytime.py
 server/failregex.py
+testcases/files/testcase-usedns.log
 testcases/banmanagertestcase.py
 testcases/failmanagertestcase.py
 testcases/clientreadertestcase.py
@@ -48,6 +51,7 @@ testcases/__init__.py
 testcases/datedetectortestcase.py
 testcases/actiontestcase.py
 testcases/servertestcase.py
+testcases/sockettestcase.py
 testcases/files/testcase01.log
 testcases/files/testcase02.log
 testcases/files/testcase03.log
@@ -55,6 +59,7 @@ testcases/files/testcase04.log
 setup.py
 setup.cfg
 common/__init__.py
+common/exceptions.py
 common/helpers.py
 common/version.py
 common/protocol.py
@@ -86,6 +91,17 @@ config/filter.d/vsftpd.conf
 config/filter.d/webmin-auth.conf
 config/filter.d/wuftpd.conf
 config/filter.d/xinetd-fail.conf
+config/filter.d/asterisk.conf
+config/filter.d/dovecot.conf
+config/filter.d/dropbear.conf
+config/filter.d/lighttpd-auth.conf
+config/filter.d/recidive.conf
+config/filter.d/roundcube-auth.conf
+config/action.d/dummy.conf
+config/action.d/iptables-ipset-proto4.conf
+config/action.d/iptables-ipset-proto6.conf
+config/action.d/iptables-xt_recent-echo.conf
+config/action.d/route.conf
 config/action.d/complain.conf
 config/action.d/dshield.conf
 config/action.d/hostsdeny.conf
@@ -108,6 +124,8 @@ config/action.d/sendmail-whois-lines.conf
 config/action.d/shorewall.conf
 config/fail2ban.conf
 man/fail2ban-client.1
+man/fail2ban.1
+man/jail.conf.5
 man/fail2ban-client.h2m
 man/fail2ban-server.1
 man/fail2ban-server.h2m

README

@@ -91,5 +91,5 @@ WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
 PARTICULAR PURPOSE. See the GNU General Public License for more details.
 
 You should have received a copy of the GNU General Public License along with
-Fail2Ban; if not, write to the Free Software Foundation, Inc., 59 Temple Place,
+Fail2Ban; if not, write to the Free Software Foundation, Inc., 51 Franklin
-Suite 330, Boston, MA  02111-1307  USA
+Street, Fifth Floor, Boston, MA 02110, USA

THANKS

@@ -13,6 +13,7 @@ Christian Rauch
 Christoph Haas
 Christos Psonis
 Daniel B. Cid
+Daniel Black
 David Nutter
 Eric Gerbier
 Guillaume Delvit
@@ -38,6 +39,7 @@ Robert Edeker
 Russell Odom
 Sireyessire
 Stephen Gildea
+Steven Hiscocks
 Tom Pike
 Tyler
 Vaclav Misek

TODO

@@ -13,9 +13,12 @@ Legend:
 # partially done
 * done
 
-- Removed relative imports
+- Run tests though all filters/examples files - (see sshd example file) as unit
+  test
 
-- Cleanup fail2ban-client and fail2ban-server. Move code to server/ and client/
+* Removed relative imports
+
+* Cleanup fail2ban-client and fail2ban-server. Move code to server/ and client/
 
 - Add timeout to external commands (signal alarm, watchdog thread, etc)
 

client/actionreader.py

@@ -35,8 +35,8 @@ logSys = logging.getLogger("fail2ban.client.config")
 
 class ActionReader(ConfigReader):
 	
-	def __init__(self, action, name):
+	def __init__(self, action, name, **kwargs):
-		ConfigReader.__init__(self)
+		ConfigReader.__init__(self, **kwargs)
 		self.__file = action[0]
 		self.__cInfo = action[1]
 		self.__name = name

client/beautifier.py

@@ -17,20 +17,14 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-# Author: Cyril Jaquier
+__author__ = "Cyril Jaquier, Yaroslav Halchenko"
-# 
+__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2013- Yaroslav Halchenko"
-# $Revision$
-
-__author__ = "Cyril Jaquier"
-__version__ = "$Revision$"
-__date__ = "$Date$"
-__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
-from server.jails import UnknownJailException
-from server.jails import DuplicateJailException
 import logging
 
+from common.exceptions import UnknownJailException, DuplicateJailException
+
 # Gets the instance of the logger.
 logSys = logging.getLogger("fail2ban.client.config")
 

client/configreader.py

@@ -27,7 +27,7 @@ __date__ = "$Date$"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
-import logging, os
+import glob, logging, os
 from configparserinc import SafeConfigParserWithIncludes
 from ConfigParser import NoOptionError, NoSectionError
 
@@ -36,33 +36,61 @@ logSys = logging.getLogger("fail2ban.client.config")
 
 class ConfigReader(SafeConfigParserWithIncludes):
 
-	BASE_DIRECTORY = "/etc/fail2ban/"
+	DEFAULT_BASEDIR = '/etc/fail2ban'
 	
-	def __init__(self):
+	def __init__(self, basedir=None):
 		SafeConfigParserWithIncludes.__init__(self)
+		self.setBaseDir(basedir)
 		self.__opts = None
 	
-	#@staticmethod
+	def setBaseDir(self, basedir):
-	def setBaseDir(folderName):
+		if basedir is None:
-		path = folderName.rstrip('/')
+			basedir = ConfigReader.DEFAULT_BASEDIR	# stock system location
-		ConfigReader.BASE_DIRECTORY = path + '/'
+		if not (os.path.exists(basedir) and os.access(basedir, os.R_OK | os.X_OK)):
-	setBaseDir = staticmethod(setBaseDir)
+			raise ValueError("Base configuration directory %s either does not exist "
+							 "or is not accessible" % basedir)
+		self._basedir = basedir.rstrip('/')
 	
-	#@staticmethod
+	def getBaseDir(self):
-	def getBaseDir():
+		return self._basedir
-		return ConfigReader.BASE_DIRECTORY
-	getBaseDir = staticmethod(getBaseDir)
 	
 	def read(self, filename):
-		basename = ConfigReader.BASE_DIRECTORY + filename
+		basename = os.path.join(self._basedir, filename)
-		logSys.debug("Reading " + basename)
+		logSys.debug("Reading configs for %s under %s "  % (basename, self._basedir))
-		bConf = basename + ".conf"
+		config_files = [ basename + ".conf",
-		bLocal = basename + ".local"
+						 basename + ".local" ]
-		if os.path.exists(bConf) or os.path.exists(bLocal):
+
-			SafeConfigParserWithIncludes.read(self, [bConf, bLocal])
+		# choose only existing ones
+		config_files = filter(os.path.exists, config_files)
+
+		# possible further customizations under a .conf.d directory
+		config_dir = basename + '.d'
+		if os.path.exists(config_dir):
+			if os.path.isdir(config_dir) and os.access(config_dir, os.X_OK | os.R_OK):
+				# files must carry .conf suffix as well
+				config_files += sorted(glob.glob('%s/*.conf' % config_dir))
+			else:
+				logSys.warn("%s exists but not a directory or not accessible"
+							 % config_dir)
+
+		# check if files are accessible, warn if any is not accessible
+		# and remove it from the list
+		config_files_accessible = []
+		for f in config_files:
+			if os.access(f, os.R_OK):
+				config_files_accessible.append(f)
+			else:
+				logSys.warn("%s exists but not accessible - skipping" % f)
+
+		if len(config_files_accessible):
+			# at least one config exists and accessible
+			SafeConfigParserWithIncludes.read(self, config_files_accessible)
 			return True
 		else:
-			logSys.error(bConf + " and " + bLocal + " do not exist")
+			logSys.error("Found no accessible config files for %r " % filename
+						 + (["under %s" % self.getBaseDir(),
+							 "among existing ones: " + ', '.join(config_files)][bool(len(config_files))]))
+
 			return False
 
 	##
@@ -94,8 +122,8 @@ class ConfigReader(SafeConfigParserWithIncludes):
 				values[option[1]] = option[2]
 			except NoOptionError:
 				if not option[2] == None:
-					logSys.warn("'%s' not defined in '%s'. Using default value"
+					logSys.warn("'%s' not defined in '%s'. Using default one: %r"
-								% (option[1], sec))
+								% (option[1], sec, option[2]))
 					values[option[1]] = option[2]
 			except ValueError:
 				logSys.warn("Wrong value for '" + option[1] + "' in '" + sec +

client/fail2banreader.py

@@ -35,14 +35,15 @@ logSys = logging.getLogger("fail2ban.client.config")
 
 class Fail2banReader(ConfigReader):
 	
-	def __init__(self):
+	def __init__(self, **kwargs):
-		ConfigReader.__init__(self)
+		ConfigReader.__init__(self, **kwargs)
 	
 	def read(self):
 		ConfigReader.read(self, "fail2ban")
 	
 	def getEarlyOptions(self):
-		opts = [["string", "socket", "/tmp/fail2ban.sock"]]
+		opts = [["string", "socket", "/tmp/fail2ban.sock"],
+				["string", "pidfile", "/var/run/fail2ban/fail2ban.pid"]]
 		return ConfigReader.getOptions(self, "Definition", opts)
 	
 	def getOptions(self):

client/filterreader.py

@@ -35,8 +35,8 @@ logSys = logging.getLogger("fail2ban.client.config")
 
 class FilterReader(ConfigReader):
 	
-	def __init__(self, fileName, name):
+	def __init__(self, fileName, name, **kwargs):
-		ConfigReader.__init__(self)
+		ConfigReader.__init__(self, **kwargs)
 		self.__file = fileName
 		self.__name = name
 	

client/jailreader.py

@@ -40,10 +40,11 @@ class JailReader(ConfigReader):
 	
 	actionCRE = re.compile("^((?:\w|-|_|\.)+)(?:\[(.*)\])?$")
 	
-	def __init__(self, name):
+	def __init__(self, name, force_enable=False, **kwargs):
-		ConfigReader.__init__(self)
+		ConfigReader.__init__(self, **kwargs)
 		self.__name = name
 		self.__filter = None
+		self.__force_enable = force_enable
 		self.__actions = list()
 	
 	def setName(self, value):
@@ -53,10 +54,10 @@ class JailReader(ConfigReader):
 		return self.__name
 	
 	def read(self):
-		ConfigReader.read(self, "jail")
+		return ConfigReader.read(self, "jail")
 	
 	def isEnabled(self):
-		return self.__opts["enabled"]
+		return self.__force_enable or self.__opts["enabled"]
 	
 	def getOptions(self):
 		opts = [["bool", "enabled", "false"],
@@ -75,7 +76,8 @@ class JailReader(ConfigReader):
 		
 		if self.isEnabled():
 			# Read filter
-			self.__filter = FilterReader(self.__opts["filter"], self.__name)
+			self.__filter = FilterReader(self.__opts["filter"], self.__name,
+										 basedir=self.getBaseDir())
 			ret = self.__filter.read()
 			if ret:
 				self.__filter.getOptions(self.__opts)
@@ -86,8 +88,10 @@ class JailReader(ConfigReader):
 			# Read action
 			for act in self.__opts["action"].split('\n'):
 				try:
+					if not act:			  # skip empty actions
+						continue
 					splitAct = JailReader.splitAction(act)
-					action = ActionReader(splitAct, self.__name)
+					action = ActionReader(splitAct, self.__name, basedir=self.getBaseDir())
 					ret = action.read()
 					if ret:
 						action.getOptions(self.__opts)
@@ -96,8 +100,10 @@ class JailReader(ConfigReader):
 						raise AttributeError("Unable to read action")
 				except Exception, e:
 					logSys.error("Error in action definition " + act)
-					logSys.debug(e)
+					logSys.debug("Caught exception: %s" % (e,))
 					return False
+			if not len(self.__actions):
+				logSys.warn("No actions were defined for %s" % self.__name)
 		return True
 	
 	def convert(self):
@@ -142,12 +148,20 @@ class JailReader(ConfigReader):
 	def splitAction(action):
 		m = JailReader.actionCRE.match(action)
 		d = dict()
-		if not m.group(2) == None:
+		mgroups = m.groups()
+		if len(mgroups) == 2:
+			action_name, action_opts = mgroups
+		elif len(mgroups) == 1:
+			action_name, action_opts = mgroups[0], None
+		else:
+			raise ValueError("While reading action %s we should have got up to "
+							 "2 groups. Got: %r" % (action, mgroups))
+		if not action_opts is None:
 			# Huge bad hack :( This method really sucks. TODO Reimplement it.
 			actions = ""
 			escapeChar = None
 			allowComma = False
-			for c in m.group(2):
+			for c in action_opts:
 				if c in ('"', "'") and not allowComma:
 					# Start
 					escapeChar = c
@@ -172,6 +186,6 @@ class JailReader(ConfigReader):
 				try:
 					d[p[0].strip()] = p[1].strip()
 				except IndexError:
-					logSys.error("Invalid argument %s in '%s'" % (p, m.group(2)))
+					logSys.error("Invalid argument %s in '%s'" % (p, action_opts))
-		return [m.group(1), d]
+		return [action_name, d]
 	splitAction = staticmethod(splitAction)

client/jailsreader.py

@@ -36,12 +36,20 @@ logSys = logging.getLogger("fail2ban.client.config")
 
 class JailsReader(ConfigReader):
 	
-	def __init__(self):
+	def __init__(self, force_enable=False, **kwargs):
-		ConfigReader.__init__(self)
+		"""
+		Parameters
+		----------
+		force_enable : bool, optional
+		  Passed to JailReader to force enable the jails.
+		  It is for internal use
+		"""
+		ConfigReader.__init__(self, **kwargs)
 		self.__jails = list()
+		self.__force_enable = force_enable
 	
 	def read(self):
-		ConfigReader.read(self, "jail")
+		return ConfigReader.read(self, "jail")
 	
 	def getOptions(self, section = None):
 		opts = []
@@ -49,7 +57,7 @@ class JailsReader(ConfigReader):
 
 		if section:
 			# Get the options of a specific jail.
-			jail = JailReader(section)
+			jail = JailReader(section, basedir=self.getBaseDir(), force_enable=self.__force_enable)
 			jail.read()
 			ret = jail.getOptions()
 			if ret:
@@ -62,7 +70,7 @@ class JailsReader(ConfigReader):
 		else:
 			# Get the options of all jails.
 			for sec in self.sections():
-				jail = JailReader(sec)
+				jail = JailReader(sec, basedir=self.getBaseDir(), force_enable=self.__force_enable)
 				jail.read()
 				ret = jail.getOptions()
 				if ret:

common/exceptions.py

@@ -0,0 +1,36 @@
+# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
+# vi: set ft=python sts=4 ts=4 sw=4 noet :
+"""Fail2Ban exceptions used by both client and server
+
+"""
+# This file is part of Fail2Ban.
+#
+# Fail2Ban is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# Fail2Ban is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Fail2Ban; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+
+__author__ = "Cyril Jaquier, Yaroslav Halchenko"
+__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2011-2012 Yaroslav Halchenko"
+__license__ = "GPL"
+
+#
+# Jails
+#
+class DuplicateJailException(Exception):
+	pass
+
+class UnknownJailException(Exception):
+	pass
+
+
+

common/protocol.py

@@ -40,6 +40,7 @@ protocol = [
 ["stop", "stops all jails and terminate the server"], 
 ["status", "gets the current status of the server"], 
 ["ping", "tests if the server is alive"], 
+["help", "return this output"], 
 ['', "LOGGING", ""],
 ["set loglevel <LEVEL>", "sets logging level to <LEVEL>. 0 is minimal, 4 is debug"], 
 ["get loglevel", "gets the logging level"], 
@@ -90,6 +91,7 @@ protocol = [
 ["get <JAIL> actioncheck <ACT>", "gets the check command for the action <ACT> for <JAIL>"],
 ["get <JAIL> actionban <ACT>", "gets the ban command for the action <ACT> for <JAIL>"],
 ["get <JAIL> actionunban <ACT>", "gets the unban command for the action <ACT> for <JAIL>"],
+["get <JAIL> cinfo <ACT> <KEY>", "gets the value for <KEY> for the action <ACT> for <JAIL>"],
 ]
 
 ##

config/action.d/complain.conf

@@ -52,10 +52,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <failtime>  unix timestamp of the last failure
-#          <bantime>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = ADDRESSES=`whois <ip> | perl -e 'while (<STDIN>) { next if /^changed|@(ripe|apnic)\.net/io; $m += (/abuse|trouble:|report|spam|security/io?3:0); if (/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)/io) { while (s/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)//io) { if ($m) { $a{lc($1)}=$m } else { $b{lc($1)}=$m } } $m=0 } else { $m && --$m } } if (%%a) {print join(",",keys(%%a))} else {print join(",",keys(%%b))}'`
@@ -67,9 +64,7 @@ actionban = ADDRESSES=`whois <ip> | perl -e 'while (<STDIN>) { next if /^changed
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <bantime>  unix timestamp of the ban time
-#          <unbantime>  unix timestamp of the unban time
 # Values:  CMD
 #
 actionunban =

config/action.d/dshield.conf

@@ -54,9 +54,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 # See http://www.dshield.org/specs.html for more on report format/notes
@@ -91,9 +89,7 @@ actionban = TZONE=`date +%%z | sed 's/\([+-]..\)\(..\)/\1:\2/'`
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = if [ -f <tmpfile>.first ]; then
@@ -116,7 +112,7 @@ actionunban = if [ -f <tmpfile>.first ]; then
 port = ???
 
 # Option:  userid
-# Notes.:  Your DSheild user ID. Should be provided either in the jail config or
+# Notes.:  Your DShield user ID. Should be provided either in the jail config or
 #          in a .local file.
 #          Register at https://secure.dshield.org/register.html
 # Values:  [ NUM ]  Default: 0
@@ -124,13 +120,13 @@ port = ???
 userid = 0
 
 # Option:  myip
-# Notes.:  TThe target IP for the attack (your public IP). Should be provided
+# Notes.:  The target IP for the attack (your public IP). Should be provided
 #          either in the jail config or in a .local file unless your PUBLIC IP
 #          is the first IP assigned to eth0
 # Values:  [ an IP address ]  Default: Tries to find the IP address of eth0,
 #          which in most cases will be a private IP, and therefore incorrect
 #
-myip = `ip -4 addr show dev eth0 | grep inet | head -1 | sed -r 's/.*inet ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/'`
+myip = `ip -4 addr show dev eth0 | grep inet | head -n 1 | sed -r 's/.*inet ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/'`
 
 # Option:  protocol
 # Notes.:  The protocol over which the attack is happening
@@ -159,7 +155,6 @@ minreportinterval = 3600
 #          submit the batch, even if we haven't reached <lines> yet. Note that
 #          this is only checked on each ban/unban, and that we always send
 #          anything in the buffer on shutdown. Must be greater than
-#          <minreportinterval>.
 # Values:  [ NUM ]  Default: 21600 (6 hours)
 #
 maxbufferage = 21600

config/action.d/dummy.conf

@@ -29,9 +29,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = printf %%b "+<ip>\n" >> /tmp/fail2ban.dummy
@@ -39,9 +37,7 @@ actionban = printf %%b "+<ip>\n" >> /tmp/fail2ban.dummy
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = printf %%b "-<ip>\n" >> /tmp/fail2ban.dummy

config/action.d/hostsdeny.conf

@@ -28,9 +28,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = IP=<ip> &&
@@ -39,9 +37,7 @@ actionban = IP=<ip> &&
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = IP=<ip> && sed -i.old /ALL:\ $IP/d <file>

config/action.d/ipfilter.conf

@@ -34,9 +34,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = echo block in quick from <ip>/32 | /sbin/ipf -f -
@@ -45,9 +43,7 @@ actionban = echo block in quick from <ip>/32 | /sbin/ipf -f -
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 # note -r option used to remove matching rule

config/action.d/ipfw.conf

@@ -32,9 +32,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = ipfw add deny tcp from <ip> to <localhost> <port>
@@ -43,9 +41,7 @@ actionban = ipfw add deny tcp from <ip> to <localhost> <port>
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = ipfw delete `ipfw list | grep -i <ip> | awk '{print $1;}'`

config/action.d/iptables-allports.conf

@@ -34,9 +34,7 @@ actioncheck = iptables -n -L <chain> | grep -q 'fail2ban-<name>[ \t]'
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
@@ -44,16 +42,14 @@ actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables-ipset-proto4.conf

@@ -0,0 +1,71 @@
+# Fail2Ban configuration file
+#
+# Author: Daniel Black
+#
+# This is for ipset protocol 4 (ipset v4.2). If you have a later version
+# of ipset try to use the iptables-ipset-proto6.conf as it does some things
+# nicer.
+# 
+# This requires the program ipset which is normally in package called ipset.
+#
+# IPset was a feature introduced in the linux kernel 2.6.39 and 3.0.0 kernels.
+#
+# If you are running on an older kernel you make need to patch in external
+# modules.
+#
+# On Debian machines this can be done with:
+#
+# apt-get install ipset xtables-addons-source 
+# module-assistant auto-install xtables-addons
+
+[Definition]
+
+# Option:  actionstart
+# Notes.:  command executed once at the start of Fail2Ban.
+# Values:  CMD
+#
+actionstart = ipset --create fail2ban-<name> iphash
+              iptables -I INPUT -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j DROP
+
+# Option:  actionstop
+# Notes.:  command executed once at the end of Fail2Ban
+# Values:  CMD
+#
+actionstop = iptables -D INPUT -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j DROP
+             ipset --flush fail2ban-<name>
+             ipset --destroy fail2ban-<name>
+
+# Option:  actionban
+# Notes.:  command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionban = ipset --test fail2ban-<name> <ip> ||  ipset --add fail2ban-<name> <ip>
+
+# Option:  actionunban
+# Notes.:  command executed when unbanning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionunban = ipset --test fail2ban-<name> <ip> && ipset --del fail2ban-<name> <ip>
+
+[Init]
+
+# Default name of the ipset
+#
+name = default
+
+# Option:  port
+# Notes.:  specifies port to monitor
+# Values:  [ NUM | STRING ]  Default: ssh
+#
+port = ssh
+
+# Option:  protocol
+# Notes.:  internally used by config reader for interpolations.
+# Values:  [ tcp | udp | icmp | all ] Default: tcp
+#
+protocol = tcp
+

config/action.d/iptables-ipset-proto6.conf

@@ -0,0 +1,78 @@
+# Fail2Ban configuration file
+#
+# Author: Daniel Black
+#
+# This is for ipset protocol 6 (and hopefully later) (ipset v6.14).
+# Use ipset -V to see the protocol and version. Version 4 should use
+# iptables-ipset-proto4.conf.
+#
+# This requires the program ipset which is normally in package called ipset.
+#
+# IPset was a feature introduced in the linux kernel 2.6.39 and 3.0.0 kernels.
+#
+# If you are running on an older kernel you make need to patch in external
+# modules.
+#
+# On Debian machines this can be done with:
+#
+# apt-get install ipset xtables-addons-source 
+# module-assistant auto-install xtables-addons
+
+[Definition]
+
+# Option:  actionstart
+# Notes.:  command executed once at the start of Fail2Ban.
+# Values:  CMD
+#
+actionstart = ipset create fail2ban-<name> hash:ip timeout <bantime>
+              iptables -I INPUT -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j DROP
+
+# Option:  actionstop
+# Notes.:  command executed once at the end of Fail2Ban
+# Values:  CMD
+#
+actionstop = iptables -D INPUT -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j DROP
+             ipset flush fail2ban-<name>
+             ipset destroy fail2ban-<name>
+
+# Option:  actionban
+# Notes.:  command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionban = ipset add fail2ban-<name> <ip> timeout <bantime> -exist
+
+# Option:  actionunban
+# Notes.:  command executed when unbanning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionunban = ipset del fail2ban-<name> <ip> -exist
+
+[Init]
+
+# Default name of the ipset
+#
+name = default
+
+# Option:  port
+# Notes.:  specifies port to monitor
+# Values:  [ NUM | STRING ]  Default: ssh
+#
+port = ssh
+
+# Option:  protocol
+# Notes.:  internally used by config reader for interpolations.
+# Values:  [ tcp | udp | icmp | all ] Default: tcp
+#
+protocol = tcp
+
+# Option: bantime
+# Notes:  specifies the bantime in seconds (handled internally rather than by fail2ban)
+# Values:  [ NUM ]  Default: 600
+
+bantime = 600
+
+

config/action.d/iptables-multiport-log.conf

@@ -42,9 +42,7 @@ actioncheck = iptables -n -L fail2ban-<name>-log >/dev/null
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = iptables -I fail2ban-<name> 1 -s <ip> -j fail2ban-<name>-log
@@ -52,16 +50,14 @@ actionban = iptables -I fail2ban-<name> 1 -s <ip> -j fail2ban-<name>-log
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = iptables -D fail2ban-<name> -s <ip> -j fail2ban-<name>-log
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables-multiport.conf

@@ -32,9 +32,7 @@ actioncheck = iptables -n -L <chain> | grep -q 'fail2ban-<name>[ \t]'
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
@@ -42,16 +40,14 @@ actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables-new.conf

@@ -34,9 +34,7 @@ actioncheck = iptables -n -L <chain> | grep -q 'fail2ban-<name>[ \t]'
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
@@ -44,16 +42,14 @@ actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables-xt_recent-echo.conf

@@ -11,7 +11,7 @@
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-# Changing iptables rules requires root priviledges. If fail2ban is
+# Changing iptables rules requires root privileges. If fail2ban is
 # configured to run as root, firewall setup can be performed by
 # fail2ban automatically. However, if fail2ban is configured to run as
 # a normal user, the configuration must be done by some other means
@@ -46,9 +46,7 @@ actioncheck = test -e /proc/net/xt_recent/fail2ban-<name>
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = echo +<ip> > /proc/net/xt_recent/fail2ban-<name>
@@ -56,16 +54,14 @@ actionban = echo +<ip> > /proc/net/xt_recent/fail2ban-<name>
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = echo -<ip> > /proc/net/xt_recent/fail2ban-<name>
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables.conf

@@ -32,9 +32,7 @@ actioncheck = iptables -n -L <chain> | grep -q 'fail2ban-<name>[ \t]'
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
@@ -42,16 +40,14 @@ actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/mail-buffered.conf

@@ -43,9 +43,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = printf %%b "`date`: <ip> (<failures> failures)\n" >> <tmpfile>
@@ -62,9 +60,7 @@ actionban = printf %%b "`date`: <ip> (<failures> failures)\n" >> <tmpfile>
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = 

config/action.d/mail-whois-lines.conf

@@ -34,10 +34,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <failtime>  unix timestamp of the last failure
-#          <bantime>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = printf %%b "Hi,\n
@@ -53,16 +50,14 @@ actionban = printf %%b "Hi,\n
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <bantime>  unix timestamp of the ban time
-#          <unbantime>  unix timestamp of the unban time
 # Values:  CMD
 #
 actionunban = 
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/mail-whois.conf

@@ -34,9 +34,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = printf %%b "Hi,\n
@@ -50,16 +48,14 @@ actionban = printf %%b "Hi,\n
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = 
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/mail.conf

@@ -34,9 +34,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = printf %%b "Hi,\n
@@ -48,16 +46,14 @@ actionban = printf %%b "Hi,\n
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = 
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/mynetwatchman.conf

@@ -49,9 +49,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 #
@@ -71,9 +69,7 @@ actionban = MNWLOGIN=`perl -e '$s=shift;$s=~s/([\W])/"%%".uc(sprintf("%%2.2x",or
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban =
@@ -102,13 +98,13 @@ mnwlogin =
 mnwpass =
 
 # Option:  myip
-# Notes.:  TThe target IP for the attack (your public IP). Should be overridden
+# Notes.:  The target IP for the attack (your public IP). Should be overridden
 #          either in the jail config or in a .local file unless your PUBLIC IP
 #          is the first IP assigned to eth0
 # Values:  [ an IP address ]  Default: Tries to find the IP address of eth0,
 #          which in most cases will be a private IP, and therefore incorrect
 #
-myip = `ip -4 addr show dev eth0 | grep inet | head -1 | sed -r 's/.*inet ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/'`
+myip = `ip -4 addr show dev eth0 | grep inet | head -n 1 | sed -r 's/.*inet ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/'`
 
 # Option:  protocol
 # Notes.:  The protocol over which the attack is happening

config/action.d/route.conf

@@ -0,0 +1,25 @@
+# Fail2Ban configuration file
+#
+# Author: Michael Gebetsroither
+#
+# This is for blocking whole hosts through blackhole routes.
+#
+# PRO:
+#   - Works on all kernel versions and as no compatibility problems (back to debian lenny and WAY further).
+#   - It's FAST for very large numbers of blocked ips.
+#   - It's FAST because it Blocks traffic before it enters common iptables chains used for filtering.
+#   - It's per host, ideal as action against ssh password bruteforcing to block further attack attempts.
+#   - No additional software required beside iproute/iproute2
+#
+# CON:
+#   - Blocking is per IP and NOT per service, but ideal as action against ssh password bruteforcing hosts
+
+[Definition]
+actionban   = ip route add <type> <ip>
+actionunban = ip route del <type> <ip>
+
+# Type of blocking
+#
+# Type can be blackhole, unreachable and prohibit. Unreachable and prohibit correspond to the ICMP reject messages.
+
+type = blackhole

config/action.d/sendmail-buffered.conf

@@ -52,9 +52,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = printf %%b "`date`: <ip> (<failures> failures)\n" >> <tmpfile>
@@ -74,16 +72,14 @@ actionban = printf %%b "`date`: <ip> (<failures> failures)\n" >> <tmpfile>
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = 
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/sendmail-whois-lines.conf

@@ -42,9 +42,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
@@ -57,23 +55,21 @@ actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
             Here are more information about <ip>:\n
             `/usr/bin/whois <ip>`\n\n
             Lines containing IP:<ip> in <logpath>\n
-            `/bin/grep '\<<ip>\>' <logpath>`\n\n
+            `grep '\<<ip>\>' <logpath>`\n\n
             Regards,\n
             Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = 
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/sendmail-whois.conf

@@ -42,9 +42,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
@@ -62,16 +60,14 @@ actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = 
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/sendmail.conf

@@ -42,9 +42,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
@@ -60,16 +58,14 @@ actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = 
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/shorewall.conf

@@ -36,9 +36,7 @@ actioncheck =
 # Option:  actionban
 # Notes.:  command executed when banning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionban = shorewall drop <ip>
@@ -46,9 +44,7 @@ actionban = shorewall drop <ip>
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
 #          command is executed with Fail2Ban user rights.
-# Tags:    <ip>  IP address
+# Tags:    See jail.conf(5) man page
-#          <failures>  number of failures
-#          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
 actionunban = shorewall allow <ip>

config/fail2ban.conf

@@ -24,6 +24,10 @@ loglevel = 3
 # Option:  logtarget
 # Notes.:  Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
 #          Only one log target can be specified.
+#          If you change logtarget from the default value and you are
+#          using logrotate -- also adjust or disable rotation in the
+#          corresponding configuration file
+#          (e.g. /etc/logrotate.d/fail2ban on Debian systems)
 # Values:  STDOUT STDERR SYSLOG file  Default:  /var/log/fail2ban.log
 #
 logtarget = /var/log/fail2ban.log
@@ -36,3 +40,10 @@ logtarget = /var/log/fail2ban.log
 #
 socket = /var/run/fail2ban/fail2ban.sock
 
+# Option: pidfile
+# Notes.: Set the PID file. This is used to store the process ID of the
+#         fail2ban server.
+# Values: FILE  Default:  /var/run/fail2ban/fail2ban.sock
+#
+pidfile = /var/run/fail2ban/fail2ban.pid
+

config/filter.d/dovecot.conf

@@ -14,7 +14,8 @@
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = .*(?:pop3-login|imap-login):.*(?:Authentication failure|Aborted login \(auth failed|Aborted login \(tried to use disabled|Disconnected \(auth failed).*rip=(?P<host>\S*),.*
+failregex = .*(?:pop3-login|imap-login):.*(?:Authentication failure|Aborted login \(auth failed|Aborted login \(tried to use disabled|Disconnected \(auth failed).*\s+rip=(?P<host>\S*),.*
+            pam.*dovecot.*(?:authentication failure).*\s+rhost=<HOST>(?:\s+user=.*)?\s*$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/exim.conf

@@ -15,6 +15,7 @@
 # Values:  TEXT
 #
 failregex = \[<HOST>\] .*(?:rejected by local_scan|Unrouteable address)
+             login authenticator failed for .* \[<HOST>\]: 535 Incorrect authentication data \(set_id=.*\)\s*$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/postfix.conf

@@ -15,6 +15,7 @@
 # Values:  TEXT
 #
 failregex = reject: RCPT from (.*)\[<HOST>\]: 554
+            reject: RCPT from (.*)\[<HOST>\]: 450 4\.7\.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo= *$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/roundcube-auth.conf

@@ -0,0 +1,22 @@
+# Fail2Ban configuration file for roundcube web server
+#
+# Author: Teodor Micu & Yaroslav Halchenko
+#
+#
+
+[Definition]
+
+# Option:  failregex
+# Notes.:  regex to match the password failure messages in the logfile. The
+#          host must be matched by a group named "host". The tag "<HOST>" can
+#          be used for standard IP/hostname matching and is only an alias for
+#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
+# Values:  TEXT
+#
+failregex = FAILED login for .*. from <HOST>\s*$
+
+# Option:  ignoreregex
+# Notes.:  regex to ignore. If this regex matches, the line is ignored.
+# Values:  TEXT
+#
+ignoreregex = 

config/filter.d/sogo-auth.conf

@@ -0,0 +1,20 @@
+# /etc/fail2ban/filter.d/sogo-auth.conf
+#
+# Fail2Ban configuration file
+# By Arnd Brandes
+# SOGo
+#
+
+[Definition]
+# Option: failregex
+# Filter Ban in /var/log/sogo/sogo.log
+# Note: the error log may contain multiple hosts, whereas the first one 
+# is the client and all others are poxys. We match the first one, only
+
+failregex = Login from '<HOST>' for user '.*' might not have worked( - password policy: \d*  grace: -?\d*  expire: -?\d*  bound: -?\d*)?\s*$
+
+# Option: ignoreregex
+# Notes.: regex to ignore. If this regex matches, the line is ignored.
+# Values: TEXT
+#
+ignoreregex = 

config/filter.d/sshd-ddos.conf

@@ -2,11 +2,17 @@
 #
 # Author: Yaroslav Halchenko
 #
-# $Revision$
+
-#
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
 
 [Definition]
 
+_daemon = sshd
+
 # Option:  failregex
 # Notes.:  regex to match the password failures messages in the logfile. The
 #          host must be matched by a group named "host". The tag "<HOST>" can
@@ -14,7 +20,7 @@
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = sshd(?:\[\d+\])?: Did not receive identification string from <HOST>$
+failregex = ^%(__prefix_line)sDid not receive identification string from <HOST>\s*$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/filter.d/sshd.conf

@@ -30,7 +30,6 @@ failregex = ^%(__prefix_line)s(?:error: PAM: )?Authentication failure for .* fro
             ^%(__prefix_line)s[iI](?:llegal|nvalid) user .* from <HOST>\s*$
             ^%(__prefix_line)sUser .+ from <HOST> not allowed because not listed in AllowUsers\s*$
             ^%(__prefix_line)sUser .+ from <HOST> not allowed because listed in DenyUsers\s*$
-            ^%(__prefix_line)s(?:pam_unix\(sshd:auth\):\s)?authentication failure; logname=\S* uid=\S* euid=\S* tty=\S* ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$
             ^%(__prefix_line)srefused connect from \S+ \(<HOST>\)\s*$
             ^%(__prefix_line)sUser .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*$
 

config/filter.d/webmin-auth.conf

@@ -18,8 +18,8 @@
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 #
-failregex = webmin.* Non-existent login as .+ from <HOST>$
+failregex = webmin.* Non-existent login as .+ from <HOST>\s*$
-            webmin.* Invalid login as .+ from <HOST>$
+            webmin.* Invalid login as .+ from <HOST>\s*$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

config/jail.conf

@@ -101,6 +101,37 @@ action      = hostsdeny
 ignoreregex = for myuser from
 logpath     = /var/log/sshd.log
 
+# Here we use blackhole routes for not requiring any additional kernel support
+# to store large volumes of banned IPs
+
+[ssh-route]
+
+enabled = false
+filter = sshd
+action = route
+logpath = /var/log/sshd.log
+maxretry = 5
+
+# Here we use a combination of Netfilter/Iptables and IPsets
+# for storing large volumes of banned IPs
+#
+# IPset comes in two versions. See ipset -V for which one to use
+# requires the ipset package and kernel support.
+[ssh-iptables-ipset4]
+
+enabled  = false
+filter   = sshd
+action   = iptables-ipset-proto4[name=SSH, port=ssh, protocol=tcp]
+logpath  = /var/log/sshd.log
+maxretry = 5
+
+[ssh-iptables-ipset6]
+enabled  = false
+filter   = sshd
+action   = iptables-ipset-proto6[name=SSH, port=ssh, protocol=tcp, bantime=600]
+logpath  = /var/log/sshd.log
+maxretry = 5
+
 # This jail demonstrates the use of wildcards in "logpath".
 # Moreover, it is possible to give other files on a new line.
 
@@ -172,6 +203,29 @@ action   = shorewall
            sendmail[name=Postfix, dest=you@example.com]
 logpath  = /var/log/apache2/error_log
 
+# Monitor roundcube server
+
+[roundcube-iptables]
+
+enabled  = false
+filter   = roundcube-auth
+action   = iptables[name=RoundCube, port="http,https"]
+logpath  = /var/log/roundcube/userlogins
+
+
+# Monitor SOGo groupware server
+
+[sogo-iptables]
+
+enabled  = false
+filter   = sogo-auth
+port     = http, https
+# without proxy this would be:
+# port    = 20000
+
+action   = iptables[name=SOGo, port="http,https"]
+logpath  = /var/log/sogo/sogo.log
+
 # Ban attackers that try to use PHP's URL-fopen() functionality
 # through GET/POST variables. - Experimental, with more than a year
 # of usage in production environments.
@@ -201,7 +255,7 @@ logpath = /var/log/lighttpd/error.log
 maxretry = 2
 
 # Same as above for mod_auth
-# It catches wrong authentifications
+# It catches wrong authentications
 
 [lighttpd-auth]
 

doc/run-rootless.txt

@@ -1,7 +1,7 @@
-Fail2ban normally requires root priviledges to insert iptables rules
+Fail2ban normally requires root privileges to insert iptables rules
 through calls to /sbin/iptables and also to read the logfiles.
-Fail2ban can run as an unpriviledged user provided that those two
+Fail2ban can run as an unprivileged user provided that those two
-capabilites are preserved. The idea is to run fail2ban as a normal
+capabilities are preserved. The idea is to run fail2ban as a normal
 user (e.g. fail2ban) who belongs to a group which is allowed to read
 logfiles. The user should also be allowed to write to
 /proc/net/xt_recent/fail2ban-<name> (name is specified in the iptables
@@ -20,14 +20,14 @@ Another way to use xt_recent is by inserting the rules by writing to
 action. Files in /proc/net/xt_recent/ are protected by normal
 filesystem rules, so can be chown'ed and chmod'ed to be writable by a
 certain user. After the necessary iptables rules are inserted (which
-requires root priviledges), blacklisting can be perfomed by an
+requires root privileges), blacklisting can be performed by an
-unpriviledged user.
+unprivileged user.
 
 Using fail2ban with xt_recent allows smarter filtering than normal
 iptables rules with the xt_recent module can provide.
 
 The disadvantage is that fail2ban cannot perform the setup by itself,
-which would require the priviledge to call /sbin/iptables, and it must
+which would require the privilege to call /sbin/iptables, and it must
 be done through other means.
 
 The primary advantage is obvious: it's generally better to run
@@ -46,7 +46,7 @@ some user and thus allow delisting IPs by helper administrators
 without the ability to mess up other iptables rules.
 
 The xt_recent-echo jail can be used under the root user without
-further configuration. To run not as root, futher setup is necessary:
+further configuration. To run not as root, further setup is necessary:
 
 - Create user:
 

fail2ban-client

@@ -27,12 +27,13 @@ import getopt, time, shlex, socket
 
 # Inserts our own modules path first in the list
 # fix for bug #343821
-if os.path.abspath(__file__).startswith('/usr/'):
+try:
-	# makes sense to use system-wide library iff -client is also under /usr/
-	sys.path.insert(1, "/usr/share/fail2ban")
-
-# Now we can import our modules
 	from common.version import version
+except ImportError, e:
+	sys.path.insert(1, "/usr/share/fail2ban")
+	from common.version import version
+
+# Now we can import the rest of modules
 from common.protocol import printFormatted
 from client.csocket import CSocket
 from client.configurator import Configurator
@@ -61,6 +62,7 @@ class Fail2banClient:
 		self.__conf["verbose"] = 1
 		self.__conf["interactive"] = False
 		self.__conf["socket"] = None
+		self.__conf["pidfile"] = None
 		
 	def dispVersion(self):
 		print "Fail2Ban v" + version
@@ -83,6 +85,7 @@ class Fail2banClient:
 		print "Options:"
 		print "    -c <DIR>                configuration directory"
 		print "    -s <FILE>               socket path"
+		print "    -p <FILE>               pidfile path"
 		print "    -d                      dump configuration. For debugging"
 		print "    -i                      interactive mode"
 		print "    -v                      increase verbosity"
@@ -118,6 +121,8 @@ class Fail2banClient:
 				self.__conf["conf"] = opt[1]
 			elif opt[0] == "-s":
 				self.__conf["socket"] = opt[1]
+			elif opt[0] == "-p":
+				self.__conf["pidfile"] = opt[1]
 			elif opt[0] == "-d":
 				self.__conf["dump"] = True
 			elif opt[0] == "-v":
@@ -180,8 +185,21 @@ class Fail2banClient:
 				# Do not continue if configuration is not 100% valid
 				if not ret:
 					return False
+				# verify that directory for the socket file exists
+				socket_dir = os.path.dirname(self.__conf["socket"])
+				if not os.path.exists(socket_dir):
+					logSys.error(
+						"There is no directory %s to contain the socket file %s."
+						% (socket_dir, self.__conf["socket"]))
+					return False
+				if not os.access(socket_dir, os.W_OK | os.X_OK):
+					logSys.error(
+						"Directory %s exists but not accessible for writing"
+						% (socket_dir,))
+					return False
 				# Start the server
 				self.__startServerAsync(self.__conf["socket"],
+										self.__conf["pidfile"],
 										self.__conf["force"])
 				try:
 					# Wait for the server to start
@@ -190,10 +208,10 @@ class Fail2banClient:
 					self.__processCmd(self.__stream, False)
 					return True
 				except ServerExecutionException:
-					logSys.error("Could not start server. Maybe an old " +
+					logSys.error("Could not start server. Maybe an old "
-								 "socket file is still present. Try to " +
+								 "socket file is still present. Try to "
-								 "remove " + self.__conf["socket"] + ". If " +
+								 "remove " + self.__conf["socket"] + ". If "
-								 "you used fail2ban-client to start the " +
+								 "you used fail2ban-client to start the "
 								 "server, adding the -x option will do it")
 					return False
 		elif len(cmd) == 1 and cmd[0] == "reload":
@@ -230,7 +248,7 @@ class Fail2banClient:
 	#
 	# Start the Fail2ban server in daemon mode.
 	
-	def __startServerAsync(self, socket, force = False):
+	def __startServerAsync(self, socket, pidfile, force = False):
 		# Forks the current process.
 		pid = os.fork()
 		if pid == 0:
@@ -241,22 +259,26 @@ class Fail2banClient:
 			# Set the socket path.
 			args.append("-s")
 			args.append(socket)
+			# Set the pidfile
+			args.append("-p")
+			args.append(pidfile)
 			# Force the execution if needed.
 			if force:
 				args.append("-x")
 			try:
 				# Use the current directory.
 				exe = os.path.abspath(os.path.join(sys.path[0], self.SERVER))
+				logSys.debug("Starting %r with args %r" % (exe, args))
 				os.execv(exe, args)
 			except OSError:
 				try:
 					# Use the PATH env.
+					logSys.warning("Initial start attempt failed.  Starting %r with the same args" % (self.SERVER,))
 					os.execvp(self.SERVER, args)
 				except OSError:
-					print "Could not find %s" % self.SERVER
+					logSys.error("Could not start %s" % self.SERVER)
 					os.exit(-1)
 	
-			
 	def __waitOnServer(self):
 		# Wait for the server to start
 		cnt = 0
@@ -275,7 +297,7 @@ class Fail2banClient:
 					delta = -1
 				elif pos < 2:
 					delta = 1
-			# The server has 30 secondes to start.
+			# The server has 30 seconds to start.
 			if cnt >= 300:
 				if self.__conf["verbose"] > 1:
 					sys.stdout.write('\n')
@@ -296,7 +318,7 @@ class Fail2banClient:
 		
 		# Reads the command line options.
 		try:
-			cmdOpts = 'hc:s:xdviqV'
+			cmdOpts = 'hc:s:p:xdviqV'
 			cmdLongOpts = ['help', 'version']
 			optList, args = getopt.getopt(self.__argv[1:], cmdOpts, cmdLongOpts)
 		except getopt.GetoptError:
@@ -327,9 +349,11 @@ class Fail2banClient:
 		
 		# Set socket path
 		self.__configurator.readEarly()
-		socket = self.__configurator.getEarlyOptions()
+		conf = self.__configurator.getEarlyOptions()
 		if self.__conf["socket"] == None:
-			self.__conf["socket"] = socket["socket"]
+			self.__conf["socket"] = conf["socket"]
+		if self.__conf["pidfile"] == None:
+			self.__conf["pidfile"] = conf["pidfile"]
 		logSys.info("Using socket file " + self.__conf["socket"])
 
 		if self.__conf["dump"]:
@@ -356,7 +380,9 @@ class Fail2banClient:
 						if cmd == "exit" or cmd == "quit":
 							# Exit
 							return True
-						if not cmd == "":
+						if cmd == "help":
+							self.dispUsage()
+						elif not cmd == "":
 							self.__processCommand(shlex.split(cmd))
 			except (EOFError, KeyboardInterrupt):
 				print
@@ -394,7 +420,7 @@ class Fail2banClient:
 class ServerExecutionException(Exception):
 	pass
 
-if __name__ == "__main__":
+if __name__ == "__main__": # pragma: no cover - can't test main
 	client = Fail2banClient()
 	# Exit with correct return value
 	if client.start(sys.argv):

fail2ban-regex

@@ -1,4 +1,7 @@
 #!/usr/bin/python
+# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
+# vi: set ft=python sts=4 ts=4 sw=4 noet :
+#
 # This file is part of Fail2Ban.
 #
 # Fail2Ban is free software; you can redistribute it and/or modify
@@ -23,13 +26,14 @@ import getopt, sys, time, logging, os
 
 # Inserts our own modules path first in the list
 # fix for bug #343821
-if os.path.abspath(__file__).startswith('/usr/'):
+try:
-	# makes sense to use system-wide library iff -regex is also under /usr/
+	from common.version import version
+except ImportError, e:
 	sys.path.insert(1, "/usr/share/fail2ban")
+	from common.version import version
 
 from client.configparserinc import SafeConfigParserWithIncludes
 from ConfigParser import NoOptionError, NoSectionError, MissingSectionHeaderError
-from common.version import version
 from server.filter import Filter
 from server.failregex import RegexException
 
@@ -43,6 +47,10 @@ class RegexStat:
 		self.__failregex = failregex
 		self.__ipList = list()
 
+	def __str__(self):
+		return "%s(%r) %d failed: %s" \
+		  % (self.__class__, self.__failregex, self.__stats, self.__ipList)
+	
 	def inc(self):
 		self.__stats += 1
 	

fail2ban-server

@@ -26,11 +26,12 @@ import getopt, sys, logging, os
 
 # Inserts our own modules path first in the list
 # fix for bug #343821
-if os.path.abspath(__file__).startswith('/usr/'):
+try:
-	# makes sense to use system-wide library iff -server is also under /usr/
-	sys.path.insert(1, "/usr/share/fail2ban")
-
 	from common.version import version
+except ImportError, e:
+	sys.path.insert(1, "/usr/share/fail2ban")
+	from common.version import version
+
 from server.server import Server
 
 # Gets the instance of the logger.
@@ -53,6 +54,7 @@ class Fail2banServer:
 		self.__conf["background"] = True
 		self.__conf["force"] = False
 		self.__conf["socket"] = "/var/run/fail2ban/fail2ban.sock"
+		self.__conf["pidfile"] = "/var/run/fail2ban/fail2ban.pid"
 	
 	def dispVersion(self):
 		print "Fail2Ban v" + version
@@ -80,6 +82,7 @@ class Fail2banServer:
 		print "    -b                   start in background"
 		print "    -f                   start in foreground"
 		print "    -s <FILE>            socket path"
+		print "    -p <FILE>            pidfile path"
 		print "    -x                   force execution of the server (remove socket file)"
 		print "    -h, --help           display this help message"
 		print "    -V, --version        print the version"
@@ -96,6 +99,8 @@ class Fail2banServer:
 				self.__conf["background"] = False
 			if opt[0] == "-s":
 				self.__conf["socket"] = opt[1]
+			if opt[0] == "-p":
+				self.__conf["pidfile"] = opt[1]
 			if opt[0] == "-x":
 				self.__conf["force"] = True
 			if opt[0] in ["-h", "--help"]:
@@ -111,7 +116,7 @@ class Fail2banServer:
 		
 		# Reads the command line options.
 		try:
-			cmdOpts = 'bfs:xhV'
+			cmdOpts = 'bfs:p:xhV'
 			cmdLongOpts = ['help', 'version']
 			optList, args = getopt.getopt(self.__argv[1:], cmdOpts, cmdLongOpts)
 		except getopt.GetoptError:
@@ -122,7 +127,9 @@ class Fail2banServer:
 		
 		try:
 			self.__server = Server(self.__conf["background"])
-			self.__server.start(self.__conf["socket"], self.__conf["force"])
+			self.__server.start(self.__conf["socket"],
+								self.__conf["pidfile"],
+								self.__conf["force"])
 			return True
 		except Exception, e:
 			logSys.exception(e)

fail2ban-testcases

@@ -35,6 +35,9 @@ from testcases import filtertestcase
 from testcases import servertestcase
 from testcases import datedetectortestcase
 from testcases import actiontestcase
+from testcases import sockettestcase
+
+from testcases.utils import FormatterWithTraceBack
 from server.mytime import MyTime
 
 from optparse import OptionParser, Option
@@ -42,7 +45,7 @@ from optparse import OptionParser, Option
 def get_opt_parser():
 	# use module docstring for help output
 	p = OptionParser(
-				usage="%s [OPTIONS]\n" % sys.argv[0] + __doc__,
+				usage="%s [OPTIONS] [regexps]\n" % sys.argv[0] + __doc__,
 				version="%prog " + version)
 
 	p.add_options([
@@ -51,13 +54,20 @@ def get_opt_parser():
 			   choices=('debug', 'info', 'warn', 'error', 'fatal'),
 			   default=None,
 			   help="Log level for the logger to use during running tests"),
+		Option('-n', "--no-network", action="store_true",
+			   dest="no_network",
+			   help="Do not run tests that require the network"),
+		Option("-t", "--log-traceback", action='store_true',
+			   help="Enrich log-messages with compressed tracebacks"),
+		Option("--full-traceback", action='store_true',
+			   help="Either to make the tracebacks full, not compressed (as by default)"),
+
 		])
 
 	return p
 
 parser = get_opt_parser()
-(opts, files) = parser.parse_args()
+(opts, regexps) = parser.parse_args()
-assert(not len(files))
 
 #
 # Logging
@@ -72,10 +82,10 @@ verbosity = {'debug': 3,
 			 'fatal': 0,
 			 None: 1}[opts.log_level]
 
-if opts.log_level is not None:
+if opts.log_level is not None: # pragma: no cover
 	# so we had explicit settings
 	logSys.setLevel(getattr(logging, opts.log_level.upper()))
-else:
+else: # pragma: no cover
 	# suppress the logging but it would leave unittests' progress dots
 	# ticking, unless like with '-l fatal' which would be silent
 	# unless error occurs
@@ -83,19 +93,27 @@ else:
 
 # Add the default logging handler
 stdout = logging.StreamHandler(sys.stdout)
-# Custom log format for the verbose tests runs
-if verbosity > 1:
-	stdout.setFormatter(logging.Formatter(' %(asctime)-15s %(thread)s %(message)s'))
-else:
-	# just prefix with the space
-	stdout.setFormatter(logging.Formatter(' %(message)s'))
-logSys.addHandler(stdout)
 
+fmt = ' %(message)s'
+
+if opts.log_traceback:
+	Formatter = FormatterWithTraceBack
+	fmt = (opts.full_traceback and ' %(tb)s' or ' %(tbc)s') + fmt
+else:
+	Formatter = logging.Formatter
+
+# Custom log format for the verbose tests runs
+if verbosity > 1: # pragma: no cover
+	stdout.setFormatter(Formatter(' %(asctime)-15s %(thread)s' + fmt))
+else: # pragma: no cover
+	# just prefix with the space
+	stdout.setFormatter(Formatter(fmt))
+logSys.addHandler(stdout)
 
 #
 # Let know the version
 #
-if not opts.log_level or opts.log_level != 'fatal':
+if not opts.log_level or opts.log_level != 'fatal': # pragma: no cover
 	print "Fail2ban %s test suite. Python %s. Please wait..." \
 	    % (version, str(sys.version).replace('\n', ''))
 
@@ -103,23 +121,42 @@ if not opts.log_level or opts.log_level != 'fatal':
 #
 # Gather the tests
 #
+if not len(regexps): # pragma: no cover
 	tests = unittest.TestSuite()
+else: # pragma: no cover
+	import re
+	class FilteredTestSuite(unittest.TestSuite):
+		_regexps = [re.compile(r) for r in regexps]
+		def addTest(self, suite):
+			suite_str = str(suite)
+			for r in self._regexps:
+				if r.search(suite_str):
+					super(FilteredTestSuite, self).addTest(suite)
+					return
+
+	tests = FilteredTestSuite()
 
 # Server
 #tests.addTest(unittest.makeSuite(servertestcase.StartStop))
-#tests.addTest(unittest.makeSuite(servertestcase.Transmitter))
+tests.addTest(unittest.makeSuite(servertestcase.Transmitter))
 tests.addTest(unittest.makeSuite(actiontestcase.ExecuteAction))
 # FailManager
 tests.addTest(unittest.makeSuite(failmanagertestcase.AddFailure))
 # BanManager
 tests.addTest(unittest.makeSuite(banmanagertestcase.AddFailure))
-# ClientReader
+# ClientReaders
+tests.addTest(unittest.makeSuite(clientreadertestcase.ConfigReaderTest))
 tests.addTest(unittest.makeSuite(clientreadertestcase.JailReaderTest))
+tests.addTest(unittest.makeSuite(clientreadertestcase.JailsReaderTest))
+# CSocket and AsyncServer
+tests.addTest(unittest.makeSuite(sockettestcase.Socket))
 
 # Filter
+if not opts.no_network:
     tests.addTest(unittest.makeSuite(filtertestcase.IgnoreIP))
 tests.addTest(unittest.makeSuite(filtertestcase.LogFile))
 tests.addTest(unittest.makeSuite(filtertestcase.LogFileMonitor))
+if not opts.no_network:
     tests.addTest(unittest.makeSuite(filtertestcase.GetFailures))
     tests.addTest(unittest.makeSuite(filtertestcase.DNSUtilsTests))
 tests.addTest(unittest.makeSuite(filtertestcase.JailTests))
@@ -140,19 +177,22 @@ filters = [FilterPoll]					  # always available
 try:
 	from server.filtergamin import FilterGamin
 	filters.append(FilterGamin)
-except Exception, e:
+except Exception, e: # pragma: no cover
 	print "I: Skipping gamin backend testing. Got exception '%s'" % e
 
 try:
 	from server.filterpyinotify import FilterPyinotify
 	filters.append(FilterPyinotify)
-except Exception, e:
+except Exception, e: # pragma: no cover
 	print "I: Skipping pyinotify backend testing. Got exception '%s'" % e
 
 for Filter_ in filters:
 	tests.addTest(unittest.makeSuite(
 		filtertestcase.get_monitor_failures_testcase(Filter_)))
 
+# Server test for logging elements which break logging used to support
+# testcases analysis
+tests.addTest(unittest.makeSuite(servertestcase.TransmitterLogging))
 
 #
 # Run the tests
@@ -170,7 +210,7 @@ try:
 
 	tests_results = testRunner.run(tests)
 
-finally:
+finally: # pragma: no cover
 	# Just for the sake of it reset the TZ
 	# yoh: move all this into setup/teardown methods within tests
 	os.environ.pop('TZ')
@@ -178,5 +218,5 @@ finally:
 		os.environ['TZ'] = old_TZ
 	time.tzset()
 
-if not tests_results.wasSuccessful():
+if not tests_results.wasSuccessful(): # pragma: no cover
 	sys.exit(1)

files/cacti/README

@@ -49,5 +49,5 @@ details.
 
 You should have received a copy of the GNU General Public
 License along with Fail2Ban; if not, write to the Free
-Software Foundation, Inc., 59 Temple Place, Suite 330,
+Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
-Boston, MA  02111-1307  USA
+Boston, MA 02110, USA

man/fail2ban-client.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.36.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.40.10.
-.TH FAIL2BAN-CLIENT "1" "March 2008" "fail2ban-client v0.8.2" "User Commands"
+.TH FAIL2BAN-CLIENT "1" "March 2013" "fail2ban-client v0.8.8" "User Commands"
 .SH NAME
 fail2ban-client \- configure and control the server
 .SH SYNOPSIS
 .B fail2ban-client
 [\fIOPTIONS\fR] \fI<COMMAND>\fR
 .SH DESCRIPTION
-Fail2Ban v0.8.2 reads log file that contains password failure report
+Fail2Ban v0.8.8 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .SH OPTIONS
 .TP
@@ -16,6 +16,9 @@ configuration directory
 \fB\-s\fR <FILE>
 socket path
 .TP
+\fB\-p\fR <FILE>
+pidfile path
+.TP
 \fB\-d\fR
 dump configuration. For debugging
 .TP
@@ -110,7 +113,7 @@ adds <FILE> to the monitoring list
 of <JAIL>
 .TP
 \fBset <JAIL> dellogpath <FILE>\fR
-removes <FILE> to the monitoring
+removes <FILE> from the monitoring
 list of <JAIL>
 .TP
 \fBset <JAIL> addfailregex <REGEX>\fR
@@ -140,6 +143,15 @@ back for <JAIL>
 sets the number of seconds <TIME>
 a host will be banned for <JAIL>
 .TP
+\fBset <JAIL> usedns <VALUE>\fR
+sets the usedns mode for <JAIL>
+.TP
+\fBset <JAIL> banip <IP>\fR
+manually Ban <IP> for <JAIL>
+.TP
+\fBset <JAIL> unbanip <IP>\fR
+manually Unban <IP> in <JAIL>
+.TP
 \fBset <JAIL> maxretry <RETRY>\fR
 sets the number of failures
 <RETRY> before banning the host
@@ -191,14 +203,6 @@ files for <JAIL>
 gets the list of ignored IP
 addresses for <JAIL>
 .TP
-\fBget <JAIL> timeregex\fR
-gets the regular expression used
-for the time detection for <JAIL>
-.TP
-\fBget <JAIL> timepattern\fR
-gets the pattern used for the time
-detection for <JAIL>
-.TP
 \fBget <JAIL> failregex\fR
 gets the list of regular
 expressions which matches the
@@ -218,6 +222,9 @@ will look back for failures for
 gets the time a host is banned for
 <JAIL>
 .TP
+\fBget <JAIL> usedns\fR
+gets the usedns setting for <JAIL>
+.TP
 \fBget <JAIL> maxretry\fR
 gets the number of failures
 allowed for <JAIL>
@@ -245,15 +252,19 @@ action <ACT> for <JAIL>
 \fBget <JAIL> actionunban <ACT>\fR
 gets the unban command for the
 action <ACT> for <JAIL>
+.TP
+\fBget <JAIL> cinfo <ACT> <KEY>\fR
+gets the value for <KEY> for the
+action <ACT> for <JAIL>
 .SH FILES
 \fI/etc/fail2ban/*\fR
 .SH AUTHOR
 Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
 Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>.
 .SH "REPORTING BUGS"
-Report bugs on https://github.com/fail2ban/fail2ban/issues
+Report bugs to https://github.com/fail2ban/fail2ban/issues
 .SH COPYRIGHT
-Copyright \(co 2004-2008 Cyril Jaquier
+Copyright \(co 2004\-2008 Cyril Jaquier, 2008\- Fail2Ban Contributors
 .br
 Copyright of modifications held by their respective authors.
 Licensed under the GNU General Public License v2 (GPL).

man/fail2ban-regex.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.36.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.40.10.
-.TH FAIL2BAN-REGEX "1" "March 2008" "fail2ban-regex v0.8.2" "User Commands"
+.TH FAIL2BAN-REGEX "1" "March 2013" "fail2ban-regex v0.8.8" "User Commands"
 .SH NAME
 fail2ban-regex \- test Fail2ban "failregex" option
 .SH SYNOPSIS
 .B fail2ban-regex
 [\fIOPTIONS\fR] \fI<LOG> <REGEX> \fR[\fIIGNOREREGEX\fR]
 .SH DESCRIPTION
-Fail2Ban v0.8.2 reads log file that contains password failure report
+Fail2Ban v0.8.8 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .PP
 This tools can test regular expressions for "fail2ban".
@@ -17,6 +17,9 @@ display this help message
 .TP
 \fB\-V\fR, \fB\-\-version\fR
 print the version
+.TP
+\fB\-v\fR, \fB\-\-verbose\fR
+verbose output
 .SH LOG
 .TP
 \fBstring\fR
@@ -42,9 +45,9 @@ path to a filter file (filter.d/sshd.conf)
 Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
 Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>.
 .SH "REPORTING BUGS"
-Report bugs on https://github.com/fail2ban/fail2ban/issues
+Report bugs to https://github.com/fail2ban/fail2ban/issues
 .SH COPYRIGHT
-Copyright \(co 2004-2008 Cyril Jaquier
+Copyright \(co 2004\-2008 Cyril Jaquier
 .br
 Copyright of modifications held by their respective authors.
 Licensed under the GNU General Public License v2 (GPL).

man/fail2ban-server.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.36.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.40.10.
-.TH FAIL2BAN-SERVER "1" "March 2008" "fail2ban-server v0.8.2" "User Commands"
+.TH FAIL2BAN-SERVER "1" "March 2013" "fail2ban-server v0.8.8" "User Commands"
 .SH NAME
 fail2ban-server \- start the server
 .SH SYNOPSIS
 .B fail2ban-server
 [\fIOPTIONS\fR]
 .SH DESCRIPTION
-Fail2Ban v0.8.2 reads log file that contains password failure report
+Fail2Ban v0.8.8 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .PP
 Only use this command for debugging purpose. Start the server with
@@ -23,6 +23,9 @@ start in foreground
 \fB\-s\fR <FILE>
 socket path
 .TP
+\fB\-p\fR <FILE>
+pidfile path
+.TP
 \fB\-x\fR
 force execution of the server (remove socket file)
 .TP
@@ -35,9 +38,9 @@ print the version
 Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
 Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>.
 .SH "REPORTING BUGS"
-Report bugs on https://github.com/fail2ban/fail2ban/issues
+Report bugs to https://github.com/fail2ban/fail2ban/issues
 .SH COPYRIGHT
-Copyright \(co 2004-2008 Cyril Jaquier
+Copyright \(co 2004\-2008 Cyril Jaquier, 2008\- Fail2Ban Contributors
 .br
 Copyright of modifications held by their respective authors.
 Licensed under the GNU General Public License v2 (GPL).

man/fail2ban.1

@@ -0,0 +1,44 @@
+.TH FAIL2BAN "1"
+.SH NAME
+fail2ban \- a set of server and client programs to limit brute force authentication attempts.
+.SH DESCRIPTION
+Fail2Ban consists of a client, server and configuration files to limit
+brute force authentication attempts.
+
+The server program \fBfail2ban-server\fR is responsible for monitoring
+log files and issuing ban/unban commands.  It gets configured through
+a simple protocol by \fBfail2ban-client\fR, which can also read
+configuration files and issue corresponding configuration commands to
+the server.
+
+For details on the configuration of fail2ban see the jail.conf(5)
+manual page.  A jail (as specified in jail.conf) couples filters and
+actions definitions for any given list of files to get monitored.
+
+For details on the command-line options of fail2ban-server see the
+fail2ban-server(1) manual page.
+
+For details on the command-line options and commands for configuring
+the server via fail2ban-client see the fail2ban-client(1) manual page.
+
+For testing regular expressions specified in a filter using the
+fail2ban-regex program may be of use and its manual page is
+fail2ban-regex(1).
+
+.SH FILES
+\fI/etc/fail2ban/*\fR
+.SH AUTHOR
+Manual page written by Daniel Black and Yaroslav Halchenko
+.SH "REPORTING BUGS"
+Report bugs to https://github.com/fail2ban/fail2ban/issues
+.SH COPYRIGHT
+Copyright \(co 2013
+.br
+Copyright of modifications held by their respective authors.
+Licensed under the GNU General Public License v2 (GPL).
+.SH "SEE ALSO"
+.br 
+fail2ban-server(1)
+fail2ban-client(1)
+fail2ban-regex(1)
+jail.conf(5)

man/jail.conf.5

@@ -0,0 +1,168 @@
+.TH JAIL.CONF.5 "fail2ban" "jail.conf(5)"
+.SH NAME
+jail.conf \- configuration for the fail2ban server
+.SH SYNOPSIS
+.I jail.conf / jail.local
+
+.I action.d/*.conf action.d/*.local
+
+.I filter.d/*.conf filter.d/*.local
+.SH DESCRIPTION
+Fail2ban has three configuration file types. Action files are the commands for banning and unbanning of IP address,
+Filter files tell fail2ban how to detect authentication failures, and Jail configurations combine filters with actions into jails.
+
+There are *.conf files that are distributed by fail2ban and *.local file that contain user customizations.
+It is recommended that *.conf files should remain unchanged.  If needed, customizations should be provided in *.local files.
+For instance, if you would like to customize the [ssh-iptables-ipset] jail, create a jail.local to extend jail.conf
+(the configuration for the fail2ban server).  The jail.local file will be the following if you only need to enable
+it:
+
+.TP
+\fIjail.local\fR
+[ssh-iptables-ipset]
+
+enabled = true
+
+.PP
+Override only the settings you need to change and the rest of the configuration will come from the corresponding
+*.conf file.
+
+\fI*.d/\fR
+.RS
+In addition to .local, for any .conf file there can be a corresponding
+.d directory to contain additional .conf files that will be read after the
+appropriate .local file.  Last parsed file will take precidence over
+identical entries, parsed alphabetically, e.g.
+
+.RS
+\fIjail.d/01_enable.conf\fR - to enable a specific jail
+.RE
+.RS
+\fIjail.d/02_custom_port.conf\fR - containing specific configuration entry to change the port of the jail specified in the configuration
+.RE
+.RS
+\fIfail2ban.d/01_custom_log.conf\fR - containing specific configuration entry to use a different log path.
+.RE
+.RE
+
+.SH DEFAULT
+The following options are applicable to all jails. Their meaning is described in the default \fIjail.conf\fR file.
+.TP
+\fBignoreip\fR 
+.TP
+\fBbantime\fR
+.TP
+\fBfindtime\fR
+.TP
+\fBmaxretry\fR
+.TP
+\fBbackend\fR
+.TP
+\fBusedns\fR
+
+
+.SH "ACTION FILES"
+Action files specify which commands are executed to ban and unban an IP address. They are located under \fI/etc/fail2ban/action.d\fR.
+
+Like with jail.conf files, if you desire local changes create an \fI[actionname].local\fR file in the \fI/etc/fail2ban/action.d\fR directory
+and override the required settings.
+
+Action files are ini files that have two sections, \fBDefinition\fR and \fBInit\fR . 
+
+The [Init] section allows for action-specific settings. In \fIjail.conf/jail.local\fR these can be overwritten for a particular jail as options to the jail.
+
+The following commands can be present in the [Definition] section.
+.TP
+\fBactionstart\fR
+command(s) executed when the jail starts.
+.TP
+\fBactionstop\fR
+command(s) executed when the jail stops.
+.TP
+\fBactioncheck\fR
+the command ran before any other action. It aims to verify if the environment is still ok.
+.TP
+\fBactionban\fR
+command(s) that bans the IP address after \fBmaxretry\fR log lines matches within last \fBfindtime\fR seconds.
+.TP
+\fBactionunban\fR
+command(s) that unbans the IP address after \fBbantime\fR.
+
+Commands specified in the [Definition] section are executed through a system shell so shell redirection and process control is allowed. The commands should
+return 0, otherwise error would be logged.  Moreover if \fBactioncheck\fR exits with non-0 status, it is taken as indication that firewall status has changed and fail2ban needs to reinitialize itself (i.e. issue \fBactionstop\fR and \fBactionstart\fR commands).
+
+Tags are enclosed in <>.  All the elements of [Init] are tags that are replaced in all action commands.  Tags can be added by the
+\fBfail2ban-client\fR using the setctag command.
+
+More than a single command is allowed to be specified. Each command needs to be on a separate line and indented with whitespaces without blank lines. The following example defines
+two commands to be executed.
+
+ actionban = iptables -I fail2ban-<name> --source <ip> -j DROP
+             echo ip=<ip>, match=<match>, time=<time> >> /var/log/fail2ban.log
+
+.SS "Action Tags"
+The following tags are substituted in the actionban, actionunban and actioncheck (when called before actionban/actionunban) commands.
+.TP
+\fBip\fR
+An IPv4 ip address to be banned. e.g. 192.168.0.2
+.TP
+\fBfailures\fR
+The number of times the failure occurred in the log file. e.g. 3
+.TP
+\fBtime\fR
+The unix time of the ban. e.g. 1357508484
+.TP
+\fBmatches\fR
+The concatenated string of the log file lines of the matches that generated the ban. Many characters interpreted by shell get escaped.
+
+.SH FILTER FILES
+
+Filter definitions are those in \fI/etc/fail2ban/filter.d/*.conf\fR and \fIfilter.d/*.local\fR.
+
+These are used to identify failed authentication attempts in logs and to extract the host IP address (or hostname if \fBusedns\fR is \fBtrue\fR).
+
+Like action files, filter files are ini files. The main section is the [Definition] section.
+
+There are two filter definitions used in the [Definition] section:
+
+.TP
+\fBfailregex\fR
+is the regex (\fBreg\fRular \fBex\fRpression) that will match failed attempts. The tag <HOST> is used as part of the regex and is itself a regex
+for IPv4 addresses and hostnames. fail2ban will work out which one of these it actually is.
+
+.TP
+\fBignoreregex\fR
+is the regex to identify log entries that should be ignored by fail2ban, even if they match failregex.
+
+
+Using Python "string interpolation" mechanisms, other definitions are allowed and can later be used within other definitions as %(defnname)s. For example.
+
+ baduseragents = IE|wget
+ failregex = useragent=%(baduseragents)s
+
+.PP
+Filters can also have a section called [INCLUDES]. This is used to read other configuration files.
+
+.TP
+\fBbefore\fR
+indicates that this file is read before the [Definition] section.
+
+.TP
+\fBafter\fR
+indicates that this file is read after the [Definition] section.
+
+.SH AUTHOR
+Fail2ban was originally written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
+At the moment it is maintained and further developed by Yaroslav O. Halchenko <debian@onerussian.com> and a number of contributors.  See \fBTHANKS\fR file shipped with Fail2Ban for a full list.
+.
+Manual page written by Daniel Black and Yaroslav Halchenko.
+.SH "REPORTING BUGS"
+Report bugs to https://github.com/fail2ban/fail2ban/issues
+.SH COPYRIGHT
+Copyright \(co 2013 Daniel Black
+.br
+Copyright of modifications held by their respective authors.
+Licensed under the GNU General Public License v2 (GPL).
+.SH "SEE ALSO"
+.br
+fail2ban-server(1)

server/action.py

@@ -243,7 +243,7 @@ class Action:
 		return Action.executeCmd(stopCmd)
 
 	def escapeTag(tag):
-		for c in '\\#&;`|*?~<>^()[]{}$\n':
+		for c in '\\#&;`|*?~<>^()[]{}$\n\'"':
 			if c in tag:
 				tag = tag.replace(c, '\\' + c)
 		return tag
@@ -277,8 +277,8 @@ class Action:
 	# Executes a command with preliminary checks and substitutions.
 	#
 	# Before executing any commands, executes the "check" command first
-	# in order to check if prerequirements are met. If this check fails,
+	# in order to check if pre-requirements are met. If this check fails,
-	# it tries to restore a sane environnement before executing the real
+	# it tries to restore a sane environment before executing the real
 	# command.
 	# Replaces "aInfo" and "cInfo" in the query too.
 	#

server/actions.py

@@ -77,7 +77,8 @@ class Actions(JailThread):
 		for action in self.__actions:
 			if action.getName() == name:
 				self.__actions.remove(action)
-				break
+				return
+		raise KeyError("Invalid Action name: %s" % name)
 	
 	##
 	# Returns an action.
@@ -91,7 +92,7 @@ class Actions(JailThread):
 		for action in self.__actions:
 			if action.getName() == name:
 				return action
-		raise KeyError
+		raise KeyError("Invalid Action name")
 	
 	##
 	# Returns the last defined action.
@@ -131,7 +132,7 @@ class Actions(JailThread):
 			# Unban the IP.
 			self.__unBan(ticket)
 			return ip
-		return 'None'
+		raise ValueError("IP %s is not banned" % ip)
 
 	##
 	# Main loop.

server/asyncserver.py

@@ -142,7 +142,7 @@ class AsyncServer(asyncore.dispatcher):
 		if sys.version_info >= (2, 6): # if python 2.6 or greater...
 			logSys.debug("Detected Python 2.6 or greater. asyncore.loop() not using poll")
 			asyncore.loop(use_poll = False) # fixes the "Unexpected communication problem" issue on Python 2.6 and 3.0
-		else:
+		else: # pragma: no cover
 			logSys.debug("NOT Python 2.6/3.* - asyncore.loop() using poll")
 			asyncore.loop(use_poll = True)
 	

server/datedetector.py

@@ -40,6 +40,14 @@ class DateDetector:
 	def __init__(self):
 		self.__lock = Lock()
 		self.__templates = list()
+		self.__known_names = set()
+
+	def _appendTemplate(self, template):
+		name = template.getName()
+		if name in self.__known_names:
+			raise ValueError("There is already a template with name %s" % name)
+		self.__known_names.add(name)
+		self.__templates.append(template)
 	
 	def addDefaultTemplate(self):
 		self.__lock.acquire()
@@ -49,104 +57,104 @@ class DateDetector:
 			template.setName("MONTH Day Hour:Minute:Second")
 			template.setRegex("\S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%b %d %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# asctime
 			template = DateStrptime()
 			template.setName("WEEKDAY MONTH Day Hour:Minute:Second Year")
 			template.setRegex("\S{3} \S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2} \d{4}")
 			template.setPattern("%a %b %d %H:%M:%S %Y")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# asctime without year
 			template = DateStrptime()
 			template.setName("WEEKDAY MONTH Day Hour:Minute:Second")
 			template.setRegex("\S{3} \S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%a %b %d %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# simple date
 			template = DateStrptime()
 			template.setName("Year/Month/Day Hour:Minute:Second")
 			template.setRegex("\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%Y/%m/%d %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# simple date too (from x11vnc)
 			template = DateStrptime()
 			template.setName("Day/Month/Year Hour:Minute:Second")
 			template.setRegex("\d{2}/\d{2}/\d{4} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%d/%m/%Y %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# previous one but with year given by 2 digits
 			# (See http://bugs.debian.org/537610)
 			template = DateStrptime()
-			template.setName("Day/Month/Year Hour:Minute:Second")
+			template.setName("Day/Month/Year2 Hour:Minute:Second")
 			template.setRegex("\d{2}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%d/%m/%y %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# Apache format [31/Oct/2006:09:22:55 -0000]
 			template = DateStrptime()
 			template.setName("Day/MONTH/Year:Hour:Minute:Second")
 			template.setRegex("\d{2}/\S{3}/\d{4}:\d{2}:\d{2}:\d{2}")
 			template.setPattern("%d/%b/%Y:%H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# CPanel 05/20/2008:01:57:39
 			template = DateStrptime()
 			template.setName("Month/Day/Year:Hour:Minute:Second")
 			template.setRegex("\d{2}/\d{2}/\d{4}:\d{2}:\d{2}:\d{2}")
 			template.setPattern("%m/%d/%Y:%H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# Exim 2006-12-21 06:43:20
 			template = DateStrptime()
 			template.setName("Year-Month-Day Hour:Minute:Second")
 			template.setRegex("\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%Y-%m-%d %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# custom for syslog-ng 2006.12.21 06:43:20
 			template = DateStrptime()
 			template.setName("Year.Month.Day Hour:Minute:Second")
 			template.setRegex("\d{4}.\d{2}.\d{2} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%Y.%m.%d %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# named 26-Jul-2007 15:20:52.252 
 			template = DateStrptime()
 			template.setName("Day-MONTH-Year Hour:Minute:Second[.Millisecond]")
 			template.setRegex("\d{2}-\S{3}-\d{4} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%d-%b-%Y %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# 17-07-2008 17:23:25
 			template = DateStrptime()
 			template.setName("Day-Month-Year Hour:Minute:Second")
 			template.setRegex("\d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%d-%m-%Y %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# 01-27-2012 16:22:44.252
 			template = DateStrptime()
 			template.setName("Month-Day-Year Hour:Minute:Second[.Millisecond]")
 			template.setRegex("\d{2}-\d{2}-\d{4} \d{2}:\d{2}:\d{2}")
 			template.setPattern("%m-%d-%Y %H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# TAI64N
 			template = DateTai64n()
 			template.setName("TAI64N")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# Epoch
 			template = DateEpoch()
 			template.setName("Epoch")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# ISO 8601
 			template = DateISO8601()
 			template.setName("ISO 8601")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# Only time information in the log
 			template = DateStrptime()
 			template.setName("Hour:Minute:Second")
 			template.setRegex("^\d{2}:\d{2}:\d{2}")
 			template.setPattern("%H:%M:%S")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 			# <09/16/08@05:03:30>
 			template = DateStrptime()
 			template.setName("<Month/Day/Year@Hour:Minute:Second>")
 			template.setRegex("^<\d{2}/\d{2}/\d{2}@\d{2}:\d{2}:\d{2}>")
 			template.setPattern("<%m/%d/%y@%H:%M:%S>")
-			self.__templates.append(template)
+			self._appendTemplate(template)
 		finally:
 			self.__lock.release()
 	
@@ -158,7 +166,8 @@ class DateDetector:
 		try:
 			for template in self.__templates:
 				match = template.matchDate(line)
-				if not match == None:
+				if not match is None:
+					logSys.debug("Matched time template %s" % template.getName())
 					return match
 			return None
 		finally:
@@ -170,8 +179,9 @@ class DateDetector:
 			for template in self.__templates:
 				try:
 					date = template.getDate(line)
-					if date == None:
+					if date is None:
 						continue
+					logSys.debug("Got time using template %s" % template.getName())
 					return date
 				except ValueError:
 					pass
@@ -194,7 +204,8 @@ class DateDetector:
 		self.__lock.acquire()
 		try:
 			logSys.debug("Sorting the template list")
-			self.__templates.sort(lambda x, y: cmp(x.getHits(), y.getHits()))
+			self.__templates.sort(lambda x, y: cmp(x.getHits(), y.getHits()), reverse=True)
-			self.__templates.reverse()
+			t = self.__templates[0]
+			logSys.debug("Winning template: %s with %d hits" % (t.getName(), t.getHits()))
 		finally:
 			self.__lock.release()

server/datetemplate.py

@@ -50,8 +50,11 @@ class DateTemplate:
 	def getName(self):
 		return self.__name
 	
-	def setRegex(self, regex):
+	def setRegex(self, regex, wordBegin=True):
-		self.__regex = regex.strip()
+		regex = regex.strip()
+		if (wordBegin and not re.search(r'^\^', regex)):
+			regex = r'\b' + regex
+		self.__regex = regex
 		self.__cRegex = re.compile(regex)
 		
 	def getRegex(self):
@@ -158,6 +161,7 @@ class DateStrptime(DateTemplate):
 							"pattern" % (opattern, e))
 			if date[0] < 2000:
 				# There is probably no year field in the logs
+				# NOTE: Possibly makes week/year day incorrect
 				date[0] = MyTime.gmtime()[0]
 				# Bug fix for #1241756
 				# If the date is greater than the current time, we suppose
@@ -166,10 +170,12 @@ class DateStrptime(DateTemplate):
 					logSys.debug(
 						u"Correcting deduced year from %d to %d since %f > %f" %
 						(date[0], date[0]-1, time.mktime(date), MyTime.time()))
+					# NOTE: Possibly makes week/year day incorrect
 					date[0] -= 1
 				elif date[1] == 1 and date[2] == 1:
 					# If it is Jan 1st, it is either really Jan 1st or there
 					# is neither month nor day in the log.
+					# NOTE: Possibly makes week/year day incorrect
 					date[1] = MyTime.gmtime()[1]
 					date[2] = MyTime.gmtime()[2]
 		return date
@@ -180,7 +186,8 @@ class DateTai64n(DateTemplate):
 	def __init__(self):
 		DateTemplate.__init__(self)
 		# We already know the format for TAI64N
-		self.setRegex("@[0-9a-f]{24}")
+		# yoh: we should not add an additional front anchor
+		self.setRegex("@[0-9a-f]{24}", wordBegin=False)
 	
 	def getDate(self, line):
 		date = None

server/failmanager.py

@@ -105,9 +105,17 @@ class FailManager:
 				fData.setLastReset(unixTime)
 				fData.setLastTime(unixTime)
 				self.__failList[ip] = fData
-			logSys.debug("Currently have failures from %d IPs: %s"
+
-						 % (len(self.__failList), self.__failList.keys()))
 			self.__failTotal += 1
+
+			if logSys.getEffectiveLevel() <= logging.DEBUG:
+				# yoh: Since composing this list might be somewhat time consuming
+				# in case of having many active failures, it should be ran only
+				# if debug level is "low" enough
+				failures_summary = ', '.join(['%s:%d' % (k, v.getRetry())
+											  for k,v in  self.__failList.iteritems()])
+				logSys.debug("Total # of detected failures: %d. Current failures from %d IPs (IP:count): %s"
+							 % (self.__failTotal, len(self.__failList), failures_summary))
 		finally:
 			self.__lock.release()
 	

server/failregex.py

@@ -130,4 +130,4 @@ class FailRegex(Regex):
 			s = self._matchCache.string
 			r = self._matchCache.re
 			raise RegexException("No 'host' found in '%s' using '%s'" % (s, r))
-		return host
+		return str(host)

server/filter.py

@@ -44,7 +44,7 @@ logSys = logging.getLogger("fail2ban.filter")
 # Log reader class.
 #
 # This class reads a log file and detects login failures or anything else
-# that matches a given regular expression. This class is instanciated by
+# that matches a given regular expression. This class is instantiated by
 # a Jail object.
 
 class Filter(JailThread):
@@ -93,6 +93,7 @@ class Filter(JailThread):
 			self.__failRegex.append(regex)
 		except RegexException, e:
 			logSys.error(e)
+			raise e
 
 
 	def delFailRegex(self, index):
@@ -117,7 +118,7 @@ class Filter(JailThread):
 	# Add the regular expression which matches the failure.
 	#
 	# The regular expression can also match any other pattern than failures
-	# and thus can be used for many purporse.
+	# and thus can be used for many purpose.
 	# @param value the regular expression
 
 	def addIgnoreRegex(self, value):
@@ -126,6 +127,7 @@ class Filter(JailThread):
 			self.__ignoreRegex.append(regex)
 		except RegexException, e:
 			logSys.error(e)
+			raise e 
 
 	def delIgnoreRegex(self, index):
 		try:
@@ -211,7 +213,7 @@ class Filter(JailThread):
 	# file has been modified and looks for failures.
 	# @return True when the thread exits nicely
 
-	def run(self):
+	def run(self): # pragma: no cover
 		raise Exception("run() is abstract")
 
 	##
@@ -226,7 +228,7 @@ class Filter(JailThread):
 			self.failManager.addFailure(FailTicket(ip, unixTime))
 
 		# Perform the banning of the IP now.
-		try:
+		try: # pragma: no branch - exception is the only way out
 			while True:
 				ticket = self.failManager.toBan()
 				self.jail.putFailTicket(ticket)
@@ -373,7 +375,7 @@ class Filter(JailThread):
 								failList.append([ip, date])
 							# We matched a regex, it is enough to stop.
 							break
-					except RegexException, e:
+					except RegexException, e: # pragma: no cover - unsure if reachable
 						logSys.error(e)
 		return failList
 
@@ -414,7 +416,7 @@ class FileFilter(Filter):
 
 	def _addLogPath(self, path):
 		# nothing to do by default
-		# to be overriden by backends
+		# to be overridden by backends
 		pass
 
 
@@ -433,7 +435,7 @@ class FileFilter(Filter):
 
 	def _delLogPath(self, path):
 		# nothing to do by default
-		# to be overriden by backends
+		# to be overridden by backends
 		pass
 
 	##
@@ -477,10 +479,19 @@ class FileFilter(Filter):
 		# Try to open log file.
 		try:
 			container.open()
-		except Exception, e:
+		# see http://python.org/dev/peps/pep-3151/
+		except IOError, e:
 			logSys.error("Unable to open %s" % filename)
 			logSys.exception(e)
 			return False
+		except OSError, e: # pragma: no cover - requires race condition to tigger this
+			logSys.error("Error opening %s" % filename)
+			logSys.exception(e)
+			return False
+		except OSError, e: # pragma: no cover - Requires implemention error in FileContainer to generate
+			logSys.error("Internal errror in FileContainer open method - please report as a bug to https://github.com/fail2ban/fail2ban/issues")
+			logSys.exception(e)
+			return False
 
 		while True:
 			line = container.readline()
@@ -507,7 +518,7 @@ class FileFilter(Filter):
 try:
 	import hashlib
 	md5sum = hashlib.md5
-except ImportError:
+except ImportError: # pragma: no cover
 	# hashlib was introduced in Python 2.5.  For compatibility with those
 	# elderly Pythons, import from md5
 	import md5
@@ -550,7 +561,7 @@ class FileContainer:
 		stats = os.fstat(self.__handler.fileno())
 		# Compare hash and inode
 		if self.__hash != myHash or self.__ino != stats.st_ino:
-			logSys.info("Log rotation detected for %s" % self.__filename)
+			logSys.debug("Log rotation detected for %s" % self.__filename)
 			self.__hash = myHash
 			self.__ino = stats.st_ino
 			self.__pos = 0
@@ -639,7 +650,7 @@ class DNSUtils:
 			ip = DNSUtils.dnsToIp(text)
 			ipList.extend(ip)
 			if ip and useDns == "warn":
-				logSys.warning("Determined IP using DNS Reverse Lookup: %s = %s",
+				logSys.warning("Determined IP using DNS Lookup: %s = %s",
 					text, ipList)
 
 		return ipList

server/filterpoll.py

@@ -39,7 +39,7 @@ logSys = logging.getLogger("fail2ban.filter")
 # Log reader class.
 #
 # This class reads a log file and detects login failures or anything else
-# that matches a given regular expression. This class is instanciated by
+# that matches a given regular expression. This class is instantiated by
 # a Jail object.
 
 class FilterPoll(FileFilter):

server/filterpyinotify.py

@@ -63,12 +63,17 @@ class FilterPyinotify(FileFilter):
 		logSys.debug("Created FilterPyinotify")
 
 
-	def callback(self, event):
+	def callback(self, event, origin=''):
+		logSys.debug("%sCallback for Event: %s", origin, event)
 		path = event.pathname
-		if event.mask == pyinotify.IN_CREATE:
+		if event.mask & pyinotify.IN_CREATE:
+			# skip directories altogether
+			if event.mask & pyinotify.IN_ISDIR:
+				logSys.debug("Ignoring creation of directory %s", path)
+				return
 			# check if that is a file we care about
 			if not path in self.__watches:
-				logSys.debug("Ignoring creation of %s we do not monitor" % path)
+				logSys.debug("Ignoring creation of %s we do not monitor", path)
 				return
 			else:
 				# we need to substitute the watcher with a new one, so first
@@ -100,7 +105,7 @@ class FilterPyinotify(FileFilter):
 	def _addFileWatcher(self, path):
 		wd = self.__monitor.add_watch(path, pyinotify.IN_MODIFY)
 		self.__watches.update(wd)
-		logSys.debug("Added file watcher for %s" % path)
+		logSys.debug("Added file watcher for %s", path)
 		# process the file since we did get even
 		self._process_file(path)
 
@@ -110,7 +115,7 @@ class FilterPyinotify(FileFilter):
 		wd = self.__monitor.rm_watch(wdInt)
 		if wd[wdInt]:
 			del self.__watches[path]
-			logSys.debug("Removed file watcher for %s" % path)
+			logSys.debug("Removed file watcher for %s", path)
 			return True
 		else:
 			return False
@@ -126,7 +131,7 @@ class FilterPyinotify(FileFilter):
 			# we need to watch also  the directory for IN_CREATE
 			self.__watches.update(
 				self.__monitor.add_watch(path_dir, pyinotify.IN_CREATE))
-			logSys.debug("Added monitor for the parent directory %s" % path_dir)
+			logSys.debug("Added monitor for the parent directory %s", path_dir)
 
 		self._addFileWatcher(path)
 
@@ -147,7 +152,7 @@ class FilterPyinotify(FileFilter):
 			# since there is no other monitored file under this directory
 			wdInt = self.__watches.pop(path_dir)
 			_ = self.__monitor.rm_watch(wdInt)
-			logSys.debug("Removed monitor for the parent directory %s" % path_dir)
+			logSys.debug("Removed monitor for the parent directory %s", path_dir)
 
 
 	##
@@ -161,7 +166,7 @@ class FilterPyinotify(FileFilter):
 		self.__notifier = pyinotify.ThreadedNotifier(self.__monitor,
 			ProcessPyinotify(self))
 		self.__notifier.start()
-		logSys.debug("pyinotifier started for %s." % self.jail.getName())
+		logSys.debug("pyinotifier started for %s.", self.jail.getName())
 		# TODO: verify that there is nothing really to be done for
 		#       idle jails
 		return True
@@ -197,5 +202,4 @@ class ProcessPyinotify(pyinotify.ProcessEvent):
 
 	# just need default, since using mask on watch to limit events
 	def process_default(self, event):
-		logSys.debug("Callback for Event: %s" % event)
+		self.__FileFilter.callback(event, origin='Default ')
-		self.__FileFilter.callback(event)

server/jails.py

@@ -17,16 +17,11 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-# Author: Cyril Jaquier
+__author__ = "Cyril Jaquier, Yaroslav Halchenko"
-# 
+__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2013- Yaroslav Halchenko"
-# $Revision$
-
-__author__ = "Cyril Jaquier"
-__version__ = "$Revision$"
-__date__ = "$Date$"
-__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
+from common.exceptions import DuplicateJailException, UnknownJailException
 
 from jail import Jail
 from threading import Lock
@@ -160,9 +155,3 @@ class Jails:
 		finally:
 			self.__lock.release()
 
-
-class DuplicateJailException(Exception):
-	pass
-
-class UnknownJailException(Exception):
-	pass

server/server.py

@@ -40,8 +40,6 @@ logSys = logging.getLogger("fail2ban.server")
 
 class Server:
 	
-	PID_FILE = "/var/run/fail2ban/fail2ban.pid"
-
 	def __init__(self, daemon = False):
 		self.__loggingLock = Lock()
 		self.__lock = RLock()
@@ -59,7 +57,7 @@ class Server:
 		logSys.debug("Caught signal %d. Exiting" % signum)
 		self.quit()
 	
-	def start(self, sock, force = False):
+	def start(self, sock, pidfile, force = False):
 		logSys.info("Starting Fail2ban v" + version.version)
 		
 		# Install signal handlers
@@ -79,8 +77,8 @@ class Server:
 		
 		# Creates a PID file.
 		try:
-			logSys.debug("Creating PID file %s" % Server.PID_FILE)
+			logSys.debug("Creating PID file %s" % pidfile)
-			pidFile = open(Server.PID_FILE, 'w')
+			pidFile = open(pidfile, 'w')
 			pidFile.write("%s\n" % os.getpid())
 			pidFile.close()
 		except IOError, e:
@@ -94,17 +92,11 @@ class Server:
 			logSys.error("Could not start server: %s", e)
 		# Removes the PID file.
 		try:
-			logSys.debug("Remove PID file %s" % Server.PID_FILE)
+			logSys.debug("Remove PID file %s" % pidfile)
-			os.remove(Server.PID_FILE)
+			os.remove(pidfile)
 		except OSError, e:
 			logSys.error("Unable to remove PID file: %s" % e)
 		logSys.info("Exiting Fail2ban")
-		# Shutdowns the logging.
-		try:
-			self.__loggingLock.acquire()
-			logging.shutdown()
-		finally:
-			self.__loggingLock.release()
 	
 	def quit(self):
 		# Stop communication first because if jail's unban action
@@ -114,9 +106,18 @@ class Server:
 		# are exiting)
 		# See https://github.com/fail2ban/fail2ban/issues/7
 		self.__asyncServer.stop()
+
 		# Now stop all the jails
 		self.stopAllJail()
 
+		# Only now shutdown the logging.
+		try:
+			self.__loggingLock.acquire()
+			logging.shutdown()
+		finally:
+			self.__loggingLock.release()
+
+	
 	def addJail(self, name, backend):
 		self.__jails.add(name, backend)
 		
@@ -369,11 +370,20 @@ class Server:
 					logSys.error("Unable to log to " + target)
 					logSys.info("Logging to previous target " + self.__logTarget)
 					return False
-			# Removes previous handlers
+			# Removes previous handlers -- in reverse order since removeHandler
-			for handler in logging.getLogger("fail2ban").handlers:
+			# alter the list in-place and that can confuses the iterable
-				# Closes the handler.
+			for handler in logging.getLogger("fail2ban").handlers[::-1]:
+				# Remove the handler.
 				logging.getLogger("fail2ban").removeHandler(handler)
+				# And try to close -- it might be closed already
+				try:
+					handler.flush()
 					handler.close()
+				except (ValueError, KeyError):
+					if sys.version_info >= (2,6):
+						raise
+					# is known to be thrown after logging was shutdown once
+					# with older Pythons -- seems to be safe to ignore there
 			# tell the handler to use this format
 			hdlr.setFormatter(formatter)
 			logging.getLogger("fail2ban").addHandler(hdlr)

server/transmitter.py

@@ -112,14 +112,18 @@ class Transmitter:
 			return self.__server.getLogLevel()
 		elif name == "logtarget":
 			value = command[1]
-			self.__server.setLogTarget(value)
+			if self.__server.setLogTarget(value):
 				return self.__server.getLogTarget()
+			else:
+				raise Exception("Failed to change log target")
 		# Jail
 		elif command[1] == "idle":
 			if command[2] == "on":
 				self.__server.setIdleJail(name, True)
 			elif command[2] == "off":
 				self.__server.setIdleJail(name, False)
+			else:
+				raise Exception("Invalid idle option, must be 'yes' or 'no'")
 			return self.__server.getIdleJail(name)
 		# Filter
 		elif command[1] == "addignoreip":
@@ -183,12 +187,13 @@ class Transmitter:
 			self.__server.addAction(name, value)
 			return self.__server.getLastAction(name).getName()
 		elif command[1] == "delaction":
+			value = command[2]
 			self.__server.delAction(name, value)
 			return None
 		elif command[1] == "setcinfo":
 			act = command[2]
 			key = command[3]
-			value = command[4]
+			value = " ".join(command[4:])
 			self.__server.setCInfo(name, act, key, value)
 			return self.__server.getCInfo(name, act, key)
 		elif command[1] == "delcinfo":
@@ -198,27 +203,27 @@ class Transmitter:
 			return None
 		elif command[1] == "actionstart":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionStart(name, act, value)
 			return self.__server.getActionStart(name, act)
 		elif command[1] == "actionstop":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionStop(name, act, value)
 			return self.__server.getActionStop(name, act)
 		elif command[1] == "actioncheck":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionCheck(name, act, value)
 			return self.__server.getActionCheck(name, act)
 		elif command[1] == "actionban":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionBan(name, act, value)
 			return self.__server.getActionBan(name, act)
 		elif command[1] == "actionunban":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionUnban(name, act, value)
 			return self.__server.getActionUnban(name, act)
 		raise Exception("Invalid command (no set action or not yet implemented)")
@@ -265,12 +270,16 @@ class Transmitter:
 		elif command[1] == "actionunban":
 			act = command[2]
 			return self.__server.getActionUnban(name, act)
+		elif command[1] == "cinfo":
+			act = command[2]
+			key = command[3]
+			return self.__server.getCInfo(name, act, key)
 		raise Exception("Invalid command (no get action or not yet implemented)")
 	
 	def status(self, command):
 		if len(command) == 0:
 			return self.__server.status()
-		else:
+		elif len(command) == 1:
 			name = command[0]
 			return self.__server.statusJail(name)
 		raise Exception("Invalid command (no status)")

setup.cfg

@@ -3,3 +3,11 @@ install-purelib=/usr/share/fail2ban
 
 [sdist]
 formats=bztar
+
+[bdist_rpm]
+release = 1
+packager = Yaroslav Halchenko <debian@onerussian.com>, Daniel Black <grooverdan@users.sourceforge.net>
+doc_files = DEVELOP
+            README
+            THANKS
+            doc/run-rootless.txt

setup.py

@@ -67,6 +67,9 @@ setup(
 						),
 						('/var/run/fail2ban',
 							''
+						),
+						('/usr/share/doc/fail2ban',
+							['README', 'DEVELOP', 'doc/run-rootless.txt']
 						)
 					]
 )
@@ -83,12 +86,6 @@ elements =	{
 					[
 						"fail2ban.py"
 					], 
-				"/usr/lib/fail2ban/firewall/":
-					[
-						"iptables.py", 
-						"ipfwadm.py", 
-						"ipfw.py"
-					],
 				"/usr/lib/fail2ban/":
 					[
 						"version.py",

testcases/clientreadertestcase.py

@@ -17,26 +17,91 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-# Author: Cyril Jaquier
+__author__ = "Cyril Jaquier, Yaroslav Halchenko"
-# 
+__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2011-2013 Yaroslav Halchenko"
-# $Revision$
-
-__author__ = "Cyril Jaquier"
-__version__ = "$Revision$"
-__date__ = "$Date$"
-__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
-import unittest
+import os, shutil, tempfile, unittest
+from client.configreader import ConfigReader
 from client.jailreader import JailReader
+from client.jailsreader import JailsReader
 
-class JailReaderTest(unittest.TestCase):
+class ConfigReaderTest(unittest.TestCase):
 
 	def setUp(self):
 		"""Call before every test case."""
+		self.d = tempfile.mkdtemp(prefix="f2b-temp")
+		self.c = ConfigReader(basedir=self.d)
 
 	def tearDown(self):
 		"""Call after every test case."""
+		shutil.rmtree(self.d)
+
+	def _write(self, fname, value):
+		# verify if we don't need to create .d directory
+		if os.path.sep in fname:
+			d = os.path.dirname(fname)
+			d_ = os.path.join(self.d, d)
+			if not os.path.exists(d_):
+				os.makedirs(d_)
+		open("%s/%s" % (self.d, fname), "w").write("""
+[section]
+option = %s
+""" % value)
+
+	def _remove(self, fname):
+		os.unlink("%s/%s" % (self.d, fname))
+		self.assertTrue(self.c.read('c'))	# we still should have some
+
+
+	def _getoption(self, f='c'):
+		self.assertTrue(self.c.read(f))	# we got some now
+		return self.c.getOptions('section', [("int", 'option')])['option']
+
+
+	def testInaccessibleFile(self):
+		f = os.path.join(self.d, "d.conf")  # inaccessible file
+		self._write('d.conf', 0)
+		self.assertEqual(self._getoption('d'), 0)
+		os.chmod(f, 0)
+		self.assertFalse(self.c.read('d'))	# should not be readable BUT present
+
+
+	def testOptionalDotDDir(self):
+		self.assertFalse(self.c.read('c'))	# nothing is there yet
+		self._write("c.conf", "1")
+		self.assertEqual(self._getoption(), 1)
+		self._write("c.conf", "2")		# overwrite
+		self.assertEqual(self._getoption(), 2)
+		self._write("c.local", "3")		# add override in .local
+		self.assertEqual(self._getoption(), 3)
+		self._write("c.d/98.conf", "998") # add 1st override in .d/
+		self.assertEqual(self._getoption(), 998)
+		self._write("c.d/90.conf", "990") # add previously sorted override in .d/
+		self.assertEqual(self._getoption(), 998) #  should stay the same
+		self._write("c.d/99.conf", "999") # now override in a way without sorting we possibly get a failure
+		self.assertEqual(self._getoption(), 999)
+		self._remove("c.d/99.conf")
+		self.assertEqual(self._getoption(), 998)
+		self._remove("c.d/98.conf")
+		self.assertEqual(self._getoption(), 990)
+		self._remove("c.d/90.conf")
+		self.assertEqual(self._getoption(), 3)
+		self._remove("c.conf")			#  we allow to stay without .conf
+		self.assertEqual(self._getoption(), 3)
+		self._write("c.conf", "1")
+		self._remove("c.local")
+		self.assertEqual(self._getoption(), 1)
+
+
+class JailReaderTest(unittest.TestCase):
+
+	def testStockSSHJail(self):
+		jail = JailReader('ssh-iptables', basedir='config') # we are running tests from root project dir atm
+		self.assertTrue(jail.read())
+		self.assertTrue(jail.getOptions())
+		self.assertFalse(jail.isEnabled())
+		self.assertEqual(jail.getName(), 'ssh-iptables')
 
 	def testSplitAction(self):
 		action = "mail-whois[name=SSH]"
@@ -44,3 +109,40 @@ class JailReaderTest(unittest.TestCase):
 		result = JailReader.splitAction(action)
 		self.assertEquals(expected, result)
 
+class JailsReaderTest(unittest.TestCase):
+
+	def testProvidingBadBasedir(self):
+		if not os.path.exists('/XXX'):
+			self.assertRaises(ValueError, JailsReader, basedir='/XXX')
+
+	def testReadStockJailConf(self):
+		jails = JailsReader(basedir='config') # we are running tests from root project dir atm
+		self.assertTrue(jails.read())		  # opens fine
+		self.assertTrue(jails.getOptions())	  # reads fine
+		comm_commands = jails.convert()
+		# by default None of the jails is enabled and we get no
+		# commands to communicate to the server
+		self.assertEqual(comm_commands, [])
+
+	def testReadStockJailConfForceEnabled(self):
+		# more of a smoke test to make sure that no obvious surprises
+		# on users' systems when enabling shipped jails
+		jails = JailsReader(basedir='config', force_enable=True) # we are running tests from root project dir atm
+		self.assertTrue(jails.read())		  # opens fine
+		self.assertTrue(jails.getOptions())	  # reads fine
+		comm_commands = jails.convert()
+
+		# by default we have lots of jails ;)
+		self.assertTrue(len(comm_commands))
+
+		# and we know even some of them by heart
+		for j in ['ssh-iptables', 'recidive']:
+			# by default we have 'auto' backend ATM
+			self.assertTrue(['add', j, 'auto'] in comm_commands)
+			# and warn on useDNS
+			self.assertTrue(['set', j, 'usedns', 'warn'] in comm_commands)
+			self.assertTrue(['start', j] in comm_commands)
+		# last commands should be the 'start' commands
+		self.assertEqual(comm_commands[-1][0], 'start')
+		# TODO: make sure that all of the jails have actions assigned,
+		#       otherwise it makes little to no sense

testcases/datedetectortestcase.py

@@ -51,33 +51,79 @@ class DateDetectorTest(unittest.TestCase):
 	
 	def testGetTime(self):
 		log = "Jan 23 21:59:59 [sshd] error: PAM: Authentication failure"
-		date = [2005, 1, 23, 21, 59, 59, 1, 23, -1]
+		date = [2005, 1, 23, 21, 59, 59, 6, 23, -1]
 		dateUnix = 1106513999.0
-	
+		# yoh: testing only up to 6 elements, since the day of the week
-		self.assertEqual(self.__datedetector.getTime(log), date)
+		#      is not correctly determined atm, since year is not present
+		#      in the log entry.  Since this doesn't effect the operation
+		#      of fail2ban -- we just ignore incorrect day of the week
+		self.assertEqual(self.__datedetector.getTime(log)[:6], date[:6])
 		self.assertEqual(self.__datedetector.getUnixTime(log), dateUnix)
 
 	def testVariousTimes(self):
 		"""Test detection of various common date/time formats f2b should understand
 		"""
-		date = [2005, 1, 23, 21, 59, 59, 1, 23, -1]
+		date = [2005, 1, 23, 21, 59, 59, 6, 23, -1]
 		dateUnix = 1106513999.0
 
 		for sdate in (
 			"Jan 23 21:59:59",
+			"Sun Jan 23 21:59:59 2005",
+			"Sun Jan 23 21:59:59",
+			"2005/01/23 21:59:59",
 			"2005.01.23 21:59:59",
 			"23/01/2005 21:59:59",
+			"23/01/05 21:59:59",
+			"23/Jan/2005:21:59:59",
+			"01/23/2005:21:59:59",
+			"2005-01-23 21:59:59",
+			"23-Jan-2005 21:59:59",
+			"23-01-2005 21:59:59",
 			"01-23-2005 21:59:59.252", # reported on f2b, causes Feb29 fix to break
+			"@4000000041f4104f00000000", # TAI64N
+			"2005-01-23T21:59:59.252Z", #ISO 8601
+			"2005-01-23T21:59:59-05:00Z", #ISO 8601 with TZ
+			"<01/23/05@21:59:59>",
 			):
 			log = sdate + "[sshd] error: PAM: Authentication failure"
 			# exclude
 
-			# TODO (Yarik is confused): figure out why for above it is
+			# yoh: on [:6] see in above test
-			#      "1" as day of the week which would be Tue, although it
-			#      was Sun
 			self.assertEqual(self.__datedetector.getTime(log)[:6], date[:6])
 			self.assertEqual(self.__datedetector.getUnixTime(log), dateUnix)
 
+	def testStableSortTemplate(self):
+		old_names = [x.getName() for x in self.__datedetector.getTemplates()]
+		self.__datedetector.sortTemplate()
+		# If there were no hits -- sorting should not change the order
+		for old_name, n in zip(old_names, self.__datedetector.getTemplates()):
+			self.assertEqual(old_name, n.getName()) # "Sort must be stable"
+
+	def testAllUniqueTemplateNames(self):
+		self.assertRaises(ValueError, self.__datedetector._appendTemplate,
+						  self.__datedetector.getTemplates()[0])
+
+	def testFullYearMatch_gh130(self):
+		# see https://github.com/fail2ban/fail2ban/pull/130
+		# yoh: unfortunately this test is not really effective to reproduce the
+		#      situation but left in place to assure consistent behavior
+		m1 = [2012, 10, 11, 2, 37, 17]
+		self.assertEqual(
+			self.__datedetector.getTime('2012/10/11 02:37:17 [error] 18434#0')[:6],
+			m1)
+		self.__datedetector.sortTemplate()
+		# confuse it with year being at the end
+		for i in xrange(10):
+			self.assertEqual(
+				self.__datedetector.getTime('11/10/2012 02:37:17 [error] 18434#0')[:6],
+				m1)
+		self.__datedetector.sortTemplate()
+		# and now back to the original
+		self.assertEqual(
+			self.__datedetector.getTime('2012/10/11 02:37:17 [error] 18434#0')[:6],
+			m1)
+
+
 #	def testDefaultTempate(self):
 #		self.__datedetector.setDefaultRegex("^\S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}")
 #		self.__datedetector.setDefaultPattern("%b %d %H:%M:%S")

testcases/files/logs/dovecot

@@ -1 +1,8 @@
 @400000004c91b044077a9e94 imap-login: Info: Aborted login (auth failed, 1 attempts): user=<martin@waschbuesch.de>, method=CRAM-MD5, rip=80.187.101.33, lip=80.254.129.240, TLS
+
+@e040c6d8a3bfa62d358083300119c259cd44dcd0 dovecot-auth: pam_unix(dovecot:auth): authentication failure; logname= uid=0 euid=0 tty=dovecot ruser=web rhost=176.61.140.224
+# Above example with injected rhost into ruser -- should not match for 1.2.3.4
+@e040c6d8a3bfa62d358083300119c259cd44dcd0 dovecot-auth: pam_unix(dovecot:auth): authentication failure; logname= uid=0 euid=0 tty=dovecot ruser=rhost=1.2.3.4 rhost=192.0.43.10
+@e040c6d8a3bfa62d358083300119c259cd44dcd0 dovecot-auth: pam_unix(dovecot:auth): authentication failure; logname= uid=0 euid=0 tty=dovecot ruser=root rhost=176.61.140.224  user=root
+
+Dec 12 11:19:11 dunnart dovecot: pop3-login: Aborted login (tried to use disabled plaintext auth): rip=190.210.136.21, lip=113.212.99.193

testcases/files/logs/exim

@@ -0,0 +1,2 @@
+# From IRC 2013-01-04
+2013-01-04 17:03:46 login authenticator failed for rrcs-24-106-174-74.se.biz.rr.com ([192.168.2.33]) [24.106.174.74]: 535 Incorrect authentication data (set_id=brian)

testcases/files/logs/postfix

@@ -0,0 +1,3 @@
+# per https://github.com/fail2ban/fail2ban/issues/125
+# and https://github.com/fail2ban/fail2ban/issues/126
+Feb 21 09:21:54 xxx postfix/smtpd[14398]: NOQUEUE: reject: RCPT from example.com[192.0.43.10]: 450 4.7.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo=

testcases/files/logs/roundcube-auth

@@ -0,0 +1 @@
+[22-Jan-2013 22:28:21 +0200]: FAILED login for user1 from 192.0.43.10

testcases/files/logs/sogo-auth

@@ -0,0 +1,17 @@
+# yoh: Kept original apache log lines as well, just in case they might come useful
+#      for (testing) multiline regular expressions which would further constraint
+#      SOGo log lines
+Mar 24 08:58:32 sogod [26818]: <0x0xb8537990[LDAPSource]> <NSException: 0xb87c3008> NAME:LDAPException REASON:operation bind failed: Invalid credentials (0x31) INFO:{login = "uid=hack0r,ou=users,dc=mail,dc=example,dc=org"; }
+Mar 24 08:58:32 sogod [26818]: SOGoRootPage Login from '173.194.44.31' for user 'hack0r' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
+173.194.44.31 - - [24/Mar/2013:08:58:32 GMT] "POST /SOGo/connect HTTP/1.1" 403 34/38 0.311 - - 2M
+Mar 24 08:58:40 sogod [26818]: <0x0xb8537990[LDAPSource]> <NSException: 0xb87bb8d8> NAME:LDAPException REASON:operation bind failed: Invalid credentials (0x31) INFO:{login = "uid=kiddy,ou=users,dc=mail,dc=example,dc=org"; }
+Mar 24 08:58:40 sogod [26818]: SOGoRootPage Login from '173.194.44.31' for user 'kiddy' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
+173.194.44.31 - - [24/Mar/2013:08:58:40 GMT] "POST /SOGo/connect HTTP/1.1" 403 34/37 0.007 - - 32K
+Mar 24 08:58:50 sogod [26818]: <0x0xb8537990[LDAPSource]> <NSException: 0xb87c27f8> NAME:LDAPException REASON:operation bind failed: Invalid credentials (0x31) INFO:{login = "uid=plsBanMe,ou=users,dc=mail,dc=example,dc=org"; }
+Mar 24 08:58:50 sogod [26818]: SOGoRootPage Login from '173.194.44.31' for user 'plsBanMe' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
+173.194.44.31 - - [24/Mar/2013:08:58:50 GMT] "POST /SOGo/connect HTTP/1.1" 403 34/40 0.008 - - 0
+Mar 24 08:58:59 sogod [26818]: <0x0xb8537990[LDAPSource]> <NSException: 0xb87be830> NAME:LDAPException REASON:operation bind failed: Invalid credentials (0x31) INFO:{login = "uid=root,ou=users,dc=mail,dc=example,dc=org"; }
+Mar 24 08:58:59 sogod [26818]: SOGoRootPage Login from '173.194.44.31' for user 'root' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0
+173.194.44.31 - - [24/Mar/2013:08:58:59 GMT] "POST /SOGo/connect HTTP/1.1" 403 34/36 0.007 - - 0
+Mar 24 08:59:04 sogod [26818]: <0x0xb8537990[LDAPSource]> <NSException: 0xb87bc088> NAME:LDAPException REASON:operation bind failed: Invalid credentials (0x31) INFO:{login = "uid=admin,ou=users,dc=mail,dc=example,dc=org"; }
+Mar 24 08:59:04 sogod [26818]: SOGoRootPage Login from '173.194.44.31' for user 'admin' might not have worked - password policy: 65535  grace: -1  expire: -1  bound: 0

testcases/files/logs/sshd-ddos

@@ -0,0 +1,2 @@
+# http://forums.powervps.com/showthread.php?t=1667
+Jun 7 01:10:56 host sshd[5937]: Did not receive identification string from 69.61.56.114

testcases/files/logs/webmin-auth

@@ -0,0 +1,7 @@
+#Webmin authentication failures from /var/log/auth.log
+
+#1 User exists, bad password
+Dec 13 08:15:18 sb1 webmin[25875]: Invalid login as root from 89.2.49.230 
+
+#2 User does not exists
+Dec 12 23:14:19 sb1 webmin[22134]: Non-existent login as robert from 188.40.105.142 

testcases/filtertestcase.py

@@ -99,26 +99,29 @@ def _copy_lines_between_files(fin, fout, n=None, skip=0, mode='a', terminal_line
 
 	Returns open fout
 	"""
-	if sys.version_info[:2] <= (2,4):
+	if sys.version_info[:2] <= (2,4): # pragma: no cover
 		# on old Python st_mtime is int, so we should give at least 1 sec so
 		# polling filter could detect the change
 		time.sleep(1)
-	if isinstance(fin, str):
+	if isinstance(fin, str): # pragma: no branch - only used with str in test cases
 		fin = open(fin, 'r')
-	if isinstance(fout, str):
-		fout = open(fout, mode)
 	# Skip
 	for i in xrange(skip):
 		_ = fin.readline()
-	# Read/Write
+	# Read
 	i = 0
+	lines = []
 	while n is None or i < n:
 		l = fin.readline()
 		if terminal_line is not None and l == terminal_line:
 			break
-		fout.write(l)
+		lines.append(l)
-		fout.flush()
 		i += 1
+	# Write: all at once and flush
+	if isinstance(fout, str):
+		fout = open(fout, mode)
+	fout.write('\n'.join(lines))
+	fout.flush()
 	# to give other threads possibly some time to crunch
 	time.sleep(0.1)
 	return fout
@@ -324,11 +327,15 @@ def get_monitor_failures_testcase(Filter_):
 	"""Generator of TestCase's for different filters/backends
 	"""
 
+	_, testclass_name = tempfile.mkstemp('fail2ban', 'monitorfailures')
+
 	class MonitorFailures(unittest.TestCase):
+		count = 0
 		def setUp(self):
 			"""Call before every test case."""
 			self.filter = self.name = 'NA'
-			_, self.name = tempfile.mkstemp('fail2ban', 'monitorfailures')
+			self.name = '%s-%d' % (testclass_name, self.count)
+			MonitorFailures.count += 1 # so we have unique filenames across tests
 			self.file = open(self.name, 'a')
 			self.jail = DummyJail()
 			self.filter = Filter_(self.jail)
@@ -351,12 +358,9 @@ def get_monitor_failures_testcase(Filter_):
 			self.filter.join()		  # wait for the thread to terminate
 			#print "D: KILLING THE FILE"
 			_killfile(self.file, self.name)
+			#time.sleep(0.2)			  # Give FS time to ack the removal
 			pass
 
-		def __str__(self):
-			return "MonitorFailures%s(%s)" \
-			  % (Filter_, hasattr(self, 'name') and self.name or 'tempfile')
-
 		def isFilled(self, delay=2.):
 			"""Wait up to `delay` sec to assure that it was modified or not
 			"""
@@ -379,7 +383,7 @@ def get_monitor_failures_testcase(Filter_):
 			return not self.isFilled(delay)
 
 		def assert_correct_last_attempt(self, failures, count=None):
-			self.assertTrue(self.isFilled(10)) # give Filter a chance to react
+			self.assertTrue(self.isFilled(20)) # give Filter a chance to react
 			_assert_correct_last_attempt(self, self.jail, failures, count=count)
 
 
@@ -431,9 +435,10 @@ def get_monitor_failures_testcase(Filter_):
 			# if we move file into a new location while it has been open already
 			self.file = _copy_lines_between_files(GetFailures.FILENAME_01, self.name,
 												  n=14, mode='w')
-			self.assertTrue(self.isEmpty(2))
+			# Poll might need more time
+			self.assertTrue(self.isEmpty(2 + int(isinstance(self.filter, FilterPoll))*4))
 			self.assertRaises(FailManagerEmpty, self.filter.failManager.toBan)
-			self.assertEqual(self.filter.failManager.getFailTotal(), 2) # Fails with Poll from time to time
+			self.assertEqual(self.filter.failManager.getFailTotal(), 2)
 
 			# move aside, but leaving the handle still open...
 			os.rename(self.name, self.name + '.bak')
@@ -488,7 +493,8 @@ def get_monitor_failures_testcase(Filter_):
 			# yoh: not sure why count here is not 9... TODO
 			self.assert_correct_last_attempt(GetFailures.FAILURES_01)#, count=9)
 
-
+	MonitorFailures.__name__ = "MonitorFailures<%s>(%s)" \
+			  % (Filter_.__name__, testclass_name) # 'tempfile')
 	return MonitorFailures
 
 

testcases/servertestcase.py

@@ -27,8 +27,9 @@ __date__ = "$Date$"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
-import unittest, socket, time
+import unittest, socket, time, tempfile, os
 from server.server import Server
+from common.exceptions import UnknownJailException
 
 class StartStop(unittest.TestCase):
 
@@ -49,82 +50,461 @@ class StartStop(unittest.TestCase):
 		time.sleep(1)
 		self.__server.stopJail(name)
 
+class TestServer(Server):
+	def setLogLevel(self, *args, **kwargs):
+		pass
+	def setLogTarget(self, *args, **kwargs):
+		pass
 
-class Transmitter(unittest.TestCase):
+class TransmitterBase(unittest.TestCase):
 	
 	def setUp(self):
 		"""Call before every test case."""
-		self.__server = Server()
+		self.transm = self.server._Server__transm
-		self.__server.setLogLevel(0)
+		sock_fd, sock_name = tempfile.mkstemp('fail2ban.sock', 'transmitter')
-		self.__server.start(False)
+		os.close(sock_fd)
+		pidfile_fd, pidfile_name = tempfile.mkstemp(
+			'fail2ban.pid', 'transmitter')
+		os.close(pidfile_fd)
+		self.server.start(sock_name, pidfile_name, force=False)
+		self.jailName = "TestJail1"
+		self.server.addJail(self.jailName, "auto")
 
 	def tearDown(self):
 		"""Call after every test case."""
-		self.__server.quit()
+		self.server.quit()
 
-	def testSetActionOK(self):
+	def setGetTest(self, cmd, inValue, outValue=None, jail=None):
-		name = "TestCase"
+		setCmd = ["set", cmd, inValue]
-		cmdList = [["add", name],
+		getCmd = ["get", cmd]
-				   ["set", name, "actionstart", "Action Start"],
+		if jail is not None:
-				   ["set", name, "actionstop", "Action Stop"],
+			setCmd.insert(1, jail)
-				   ["set", name, "actioncheck", "Action Check"],
+			getCmd.insert(1, jail)
-				   ["set", name, "actionban", "Action Ban"],
+		if outValue is None:
-				   ["set", name, "actionunban", "Action Unban"],
+			outValue = inValue
-				   ["quit"]]
 
-		outList = [(0, name),
+		self.assertEqual(self.transm.proceed(setCmd), (0, outValue))
-				   (0, 'Action Start'),
+		self.assertEqual(self.transm.proceed(getCmd), (0, outValue))
-				   (0, 'Action Stop'),
-				   (0, 'Action Check'),
-				   (0, 'Action Ban'),
-				   (0, 'Action Unban'),
-				   (0, None)]
 
-		cnt = 0
+	def setGetTestNOK(self, cmd, inValue, jail=None):
-		for cmd in cmdList:
+		setCmd = ["set", cmd, inValue]
-			self.assertEqual(self.__server.transm.proceed(cmd), outList[cnt])
+		getCmd = ["get", cmd]
-			cnt += 1
+		if jail is not None:
+			setCmd.insert(1, jail)
+			getCmd.insert(1, jail)
 
-	def testSetActionNOK(self):
+		# Get initial value before trying invalid value
-		name = "TestCase"
+		initValue = self.transm.proceed(getCmd)[1]
-		cmdList = [["addd", name],
+		self.assertEqual(self.transm.proceed(setCmd)[0], 1)
-				   ["set", name, "test"],
+		# Check after failed set that value is same as previous
-				   ["prout prout", "Stop"],
+		self.assertEqual(self.transm.proceed(getCmd), (0, initValue))
-				   ["fail2ban", "sucks"],
-				   ["set"],
-				   ["_/&%", "@*+%&"],
-				   [" quit"]]
 
-		outList = [1,
+	def jailAddDelTest(self, cmd, values, jail):
-				   1,
+		cmdAdd = "add" + cmd
-				   1,
+		cmdDel = "del" + cmd
-				   1,
-				   1,
-				   1,
-				   1]
 
-		cnt = 0
+		self.assertEqual(
-		for cmd in cmdList:
+			self.transm.proceed(["get", jail, cmd]), (0, []))
-			msg = self.__server.transm.proceed(cmd)
+		for n, value in enumerate(values):
-			self.assertEqual(msg[0], outList[cnt])
+			self.assertEqual(
-			cnt += 1
+				self.transm.proceed(["set", jail, cmdAdd, value]),
+				(0, values[:n+1]))
+			self.assertEqual(
+				self.transm.proceed(["get", jail, cmd]),
+				(0, values[:n+1]))
+		for n, value in enumerate(values):
+			self.assertEqual(
+				self.transm.proceed(["set", jail, cmdDel, value]),
+				(0, values[n+1:]))
+			self.assertEqual(
+				self.transm.proceed(["get", jail, cmd]),
+				(0, values[n+1:]))
 
-	def testJail(self):
+	def jailAddDelRegexTest(self, cmd, inValues, outValues, jail):
-		name = "TestCase"
+		cmdAdd = "add" + cmd
-		cmdList = [["add", name],
+		cmdDel = "del" + cmd
-				   ["set", name, "logpath", "testcases/files/testcase01.log"],
-				   ["set", name, "timeregex", "\S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}"],
-				   ["set", name, "timepattern", "%b %d %H:%M:%S"],
-				   ["set", name, "failregex", "Authentication failure"],
-				   ["start", name],
-				   ["stop", name],
-				   ["quit"]]
 
-		for cmd in cmdList:
+		if outValues is None:
-			self.__server.transm.proceed(cmd)
+			outValues = inValues
-			if cmd == ["start", name]:
-				time.sleep(2)
-				jail = self.__server.jails[name]
-				self.assertEqual(jail.getFilter().failManager.size(), 0)
-				self.assertEqual(jail.getAction().banManager.size(), 2)
 
+		self.assertEqual(
+			self.transm.proceed(["get", jail, cmd]), (0, []))
+		for n, value in enumerate(inValues):
+			self.assertEqual(
+				self.transm.proceed(["set", jail, cmdAdd, value]),
+				(0, outValues[:n+1]))
+			self.assertEqual(
+				self.transm.proceed(["get", jail, cmd]),
+				(0, outValues[:n+1]))
+		for n, value in enumerate(inValues):
+			self.assertEqual(
+				self.transm.proceed(["set", jail, cmdDel, 0]), # First item
+				(0, outValues[n+1:]))
+			self.assertEqual(
+				self.transm.proceed(["get", jail, cmd]),
+				(0, outValues[n+1:]))
+
+class Transmitter(TransmitterBase):
+
+	def setUp(self):
+		self.server = TestServer()
+		super(Transmitter, self).setUp()
+
+	def testStopServer(self):
+		self.assertEqual(self.transm.proceed(["stop"]), (0, None))
+
+	def testPing(self):
+		self.assertEqual(self.transm.proceed(["ping"]), (0, "pong"))
+
+	def testSleep(self):
+		t0 = time.time()
+		self.assertEqual(self.transm.proceed(["sleep", "1"]), (0, None))
+		t1 = time.time()
+		# Approx 1 second delay
+		self.assertAlmostEqual(t1 - t0, 1, places=2)
+
+	def testAddJail(self):
+		jail2 = "TestJail2"
+		jail3 = "TestJail3"
+		jail4 = "TestJail4"
+		self.assertEqual(
+			self.transm.proceed(["add", jail2, "polling"]), (0, jail2))
+		self.assertEqual(self.transm.proceed(["add", jail3]), (0, jail3))
+		self.assertEqual(
+			self.transm.proceed(["add", jail4, "invalid backend"])[0], 1)
+		self.assertEqual(
+			self.transm.proceed(["add", jail4, "auto"]), (0, jail4))
+		# Duplicate Jail
+		self.assertEqual(
+			self.transm.proceed(["add", self.jailName, "polling"])[0], 1)
+		# All name is reserved
+		self.assertEqual(
+			self.transm.proceed(["add", "all", "polling"])[0], 1)
+
+	def testStartStopJail(self):
+		self.assertEqual(
+			self.transm.proceed(["start", self.jailName]), (0, None))
+		time.sleep(1)
+		self.assertEqual(
+			self.transm.proceed(["stop", self.jailName]), (0, None))
+		self.assertRaises(
+			UnknownJailException, self.server.isAlive, self.jailName)
+
+	def testStartStopAllJail(self):
+		self.server.addJail("TestJail2", "auto")
+		self.assertEqual(
+			self.transm.proceed(["start", self.jailName]), (0, None))
+		self.assertEqual(
+			self.transm.proceed(["start", "TestJail2"]), (0, None))
+		# yoh: workaround for gh-146.  I still think that there is some
+		#      race condition and missing locking somewhere, but for now
+		#      giving it a small delay reliably helps to proceed with tests
+		time.sleep(0.1)
+		self.assertEqual(self.transm.proceed(["stop", "all"]), (0, None))
+		time.sleep(1)
+		self.assertRaises(
+			UnknownJailException, self.server.isAlive, self.jailName)
+		self.assertRaises(
+			UnknownJailException, self.server.isAlive, "TestJail2")
+
+	def testJailIdle(self):
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "idle", "on"]),
+			(0, True))
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "idle", "off"]),
+			(0, False))
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "idle", "CAT"])[0],
+			1)
+
+	def testJailFindTime(self):
+		self.setGetTest("findtime", "120", 120, jail=self.jailName)
+		self.setGetTest("findtime", "60", 60, jail=self.jailName)
+		self.setGetTest("findtime", "-60", -60, jail=self.jailName)
+		self.setGetTestNOK("findtime", "Dog", jail=self.jailName)
+
+	def testJailBanTime(self):
+		self.setGetTest("bantime", "600", 600, jail=self.jailName)
+		self.setGetTest("bantime", "50", 50, jail=self.jailName)
+		self.setGetTest("bantime", "-50", -50, jail=self.jailName)
+		self.setGetTestNOK("bantime", "Cat", jail=self.jailName)
+
+	def testJailUseDNS(self):
+		self.setGetTest("usedns", "yes", jail=self.jailName)
+		self.setGetTest("usedns", "warn", jail=self.jailName)
+		self.setGetTest("usedns", "no", jail=self.jailName)
+
+		# Safe default should be "no"
+		value = "Fish"
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "usedns", value]),
+			(0, "no"))
+
+	def testJailBanIP(self):
+		self.server.startJail(self.jailName) # Jail must be started
+
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "banip", "127.0.0.1"]),
+			(0, "127.0.0.1"))
+		time.sleep(1) # Give chance to ban
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "banip", "Badger"]),
+			(0, "Badger")) #NOTE: Is IP address validated? Is DNS Lookup done?
+		time.sleep(1) # Give chance to ban
+		# Unban IP
+		self.assertEqual(
+			self.transm.proceed(
+				["set", self.jailName, "unbanip", "127.0.0.1"]),
+			(0, "127.0.0.1"))
+		# Unban IP which isn't banned
+		self.assertEqual(
+			self.transm.proceed(
+				["set", self.jailName, "unbanip", "192.168.1.1"])[0],1)
+
+	def testJailMaxRetry(self):
+		self.setGetTest("maxretry", "5", 5, jail=self.jailName)
+		self.setGetTest("maxretry", "2", 2, jail=self.jailName)
+		self.setGetTest("maxretry", "-2", -2, jail=self.jailName)
+		self.setGetTestNOK("maxretry", "Duck", jail=self.jailName)
+
+	def testJailLogPath(self):
+		self.jailAddDelTest(
+			"logpath",
+			[
+				"testcases/files/testcase01.log",
+				"testcases/files/testcase02.log",
+				"testcases/files/testcase03.log",
+			],
+			self.jailName
+		)
+		# Try duplicates
+		value = "testcases/files/testcase04.log"
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "addlogpath", value]),
+			(0, [value]))
+		# Will silently ignore duplicate
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "addlogpath", value]),
+			(0, [value]))
+		self.assertEqual(
+			self.transm.proceed(["get", self.jailName, "logpath"]),
+			(0, [value]))
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "dellogpath", value]),
+			(0, []))
+
+		# Invalid file
+		value = "this_file_shouldn't_exist"
+		result = self.transm.proceed(
+			["set", self.jailName, "addlogpath", value])
+		self.assertTrue(isinstance(result[1], IOError))
+
+	def testJailIgnoreIP(self):
+		self.jailAddDelTest(
+			"ignoreip",
+			[
+				"127.0.0.1",
+				"192.168.1.1",
+				"8.8.8.8",
+			],
+			self.jailName
+		)
+
+		# Try duplicates
+		value = "127.0.0.1"
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "addignoreip", value]),
+			(0, [value]))
+		# Will allow duplicate
+		#NOTE: Should duplicates be allowed, or silent ignore like logpath?
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "addignoreip", value]),
+			(0, [value, value]))
+		self.assertEqual(
+			self.transm.proceed(["get", self.jailName, "ignoreip"]),
+			(0, [value, value]))
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "delignoreip", value]),
+			(0, [value]))
+
+	def testJailRegex(self):
+		self.jailAddDelRegexTest("failregex",
+			[
+				"user john at <HOST>",
+				"Admin user login from <HOST>",
+				"failed attempt from <HOST> again",
+			],
+			[
+				"user john at (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)",
+				"Admin user login from (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)",
+				"failed attempt from (?:::f{4,6}:)?(?P<host>[\w\-.^_]+) again",
+			],
+			self.jailName
+		)
+
+		self.assertEqual(
+			self.transm.proceed(
+				["set", self.jailName, "addfailregex", "No host regex"])[0],
+			1)
+		self.assertEqual(
+			self.transm.proceed(
+				["set", self.jailName, "addfailregex", 654])[0],
+			1)
+
+	def testJailIgnoreRegex(self):
+		self.jailAddDelRegexTest("ignoreregex",
+			[
+				"user john",
+				"Admin user login from <HOST>",
+				"Dont match me!",
+			],
+			[
+				"user john",
+				"Admin user login from (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)",
+				"Dont match me!",
+			],
+			self.jailName
+		)
+
+		self.assertEqual(
+			self.transm.proceed(
+				["set", self.jailName, "addignoreregex", "Invalid [regex"])[0],
+			1)
+		self.assertEqual(
+			self.transm.proceed(
+				["set", self.jailName, "addignoreregex", 50])[0],
+			1)
+
+	def testStatus(self):
+		jails = [self.jailName]
+		self.assertEqual(self.transm.proceed(["status"]),
+			(0, [('Number of jail', len(jails)), ('Jail list', ", ".join(jails))]))
+		self.server.addJail("TestJail2", "auto")
+		jails.append("TestJail2")
+		self.assertEqual(self.transm.proceed(["status"]),
+			(0, [('Number of jail', len(jails)), ('Jail list', ", ".join(jails))]))
+
+	def testJailStatus(self):
+		self.assertEqual(self.transm.proceed(["status", self.jailName]),
+			(0,
+				[
+					('filter', [
+						('Currently failed', 0),
+						('Total failed', 0),
+						('File list', [])]
+					),
+					('action', [
+						('Currently banned', 0),
+						('Total banned', 0),
+						('IP list', [])]
+					)
+				]
+			)
+		)
+
+	def testAction(self):
+		action = "TestCaseAction"
+		cmdList = [
+			"actionstart",
+			"actionstop",
+			"actioncheck",
+			"actionban",
+			"actionunban",
+		]
+		cmdValueList = [
+			"Action Start",
+			"Action Stop",
+			"Action Check",
+			"Action Ban",
+			"Action Unban",
+		]
+
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "addaction", action]),
+			(0, action))
+		self.assertEqual(
+			self.transm.proceed(["get", self.jailName, "addaction", action]),
+			(0, action))
+		for cmd, value in zip(cmdList, cmdValueList):
+			self.assertEqual(
+				self.transm.proceed(
+					["set", self.jailName, cmd, action, value]),
+				(0, value))
+		for cmd, value in zip(cmdList, cmdValueList):
+			self.assertEqual(
+				self.transm.proceed(["get", self.jailName, cmd, action]),
+				(0, value))
+		self.assertEqual(
+			self.transm.proceed(
+				["set", self.jailName, "setcinfo", action, "KEY", "VALUE"]),
+			(0, "VALUE"))
+		self.assertEqual(
+			self.transm.proceed(
+				["get", self.jailName, "cinfo", action, "KEY"]),
+			(0, "VALUE"))
+		self.assertEqual(
+			self.transm.proceed(
+				["get", self.jailName, "cinfo", action, "InvalidKey"])[0],
+			1)
+		self.assertEqual(
+			self.transm.proceed(
+				["set", self.jailName, "delcinfo", action, "KEY"]),
+			(0, None))
+		self.assertEqual(
+			self.transm.proceed(["set", self.jailName, "delaction", action]),
+			(0, None))
+		self.assertEqual(
+			self.transm.proceed(
+				["set", self.jailName, "delaction", "Doesn't exist"])[0],1)
+
+	def testNOK(self):
+		self.assertEqual(self.transm.proceed(["INVALID", "COMMAND"])[0],1)
+
+	def testSetNOK(self):
+		self.assertEqual(
+			self.transm.proceed(["set", "INVALID", "COMMAND"])[0],1)
+
+	def testGetNOK(self):
+		self.assertEqual(
+			self.transm.proceed(["get", "INVALID", "COMMAND"])[0],1)
+
+	def testStatusNOK(self):
+		self.assertEqual(
+			self.transm.proceed(["status", "INVALID", "COMMAND"])[0],1)
+
+class TransmitterLogging(TransmitterBase):
+
+	def setUp(self):
+		self.server = Server()
+		self.server.setLogTarget("/dev/null")
+		self.server.setLogLevel(0)
+		super(TransmitterLogging, self).setUp()
+
+	def testLogTarget(self):
+		logTargets = []
+		for _ in xrange(3):
+			tmpFile = tempfile.mkstemp("fail2ban", "transmitter")
+			logTargets.append(tmpFile[1])
+			os.close(tmpFile[0])
+		for logTarget in logTargets:
+			self.setGetTest("logtarget", logTarget)
+
+		# If path is invalid, do not change logtarget
+		value = "/this/path/should/not/exist"
+		self.setGetTestNOK("logtarget", value)
+
+		self.transm.proceed(["set", "/dev/null"])
+		for logTarget in logTargets:
+			os.remove(logTarget)
+
+		self.setGetTest("logtarget", "STDOUT")
+		self.setGetTest("logtarget", "STDERR")
+		self.setGetTest("logtarget", "SYSLOG")
+
+	def testLogLevel(self):
+		self.setGetTest("loglevel", "4", 4)
+		self.setGetTest("loglevel", "2", 2)
+		self.setGetTest("loglevel", "-1", -1)
+		self.setGetTest("loglevel", "0", 0)
+		self.setGetTestNOK("loglevel", "Bird")

testcases/sockettestcase.py

@@ -0,0 +1,82 @@
+# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
+# vi: set ft=python sts=4 ts=4 sw=4 noet :
+
+# This file is part of Fail2Ban.
+#
+# Fail2Ban is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# Fail2Ban is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Fail2Ban; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+
+# Author: Steven Hiscocks
+# 
+# $Revision$
+
+__author__ = "Steven Hiscocks"
+__version__ = "$Revision$"
+__date__ = "$Date$"
+__copyright__ = "Copyright (c) 2013 Steven Hiscocks"
+__license__ = "GPL"
+
+import unittest, time, tempfile, os, threading
+from server.asyncserver import AsyncServer, AsyncServerException
+from client.csocket import CSocket
+
+class Socket(unittest.TestCase):
+
+	def setUp(self):
+		"""Call before every test case."""
+		self.server = AsyncServer(self)
+		sock_fd, sock_name = tempfile.mkstemp('fail2ban.sock', 'socket')
+		os.close(sock_fd)
+		os.remove(sock_name)
+		self.sock_name = sock_name
+
+	def tearDown(self):
+		"""Call after every test case."""
+
+	@staticmethod
+	def proceed(message):
+		"""Test transmitter proceed method which just returns first arg"""
+		return message
+
+	def testSocket(self):
+		serverThread = threading.Thread(
+			target=self.server.start, args=(self.sock_name, False))
+		serverThread.daemon = True
+		serverThread.start()
+		time.sleep(1)
+
+		client = CSocket(self.sock_name)
+		testMessage = ["A", "test", "message"]
+		self.assertEqual(client.send(testMessage), testMessage)
+
+		self.server.stop()
+		serverThread.join(1)
+		self.assertFalse(os.path.exists(self.sock_name))
+
+	def testSocketForce(self):
+		open(self.sock_name, 'w').close() # Create sock file
+		# Try to start without force
+		self.assertRaises(
+			AsyncServerException, self.server.start, self.sock_name, False)
+
+		# Try agin with force set
+		serverThread = threading.Thread(
+			target=self.server.start, args=(self.sock_name, True))
+		serverThread.daemon = True
+		serverThread.start()
+		time.sleep(1)
+
+		self.server.stop()
+		serverThread.join(1)
+		self.assertFalse(os.path.exists(self.sock_name))

testcases/utils.py

@@ -0,0 +1,101 @@
+# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
+# vi: set ft=python sts=4 ts=4 sw=4 noet :
+
+# This file is part of Fail2Ban.
+#
+# Fail2Ban is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# Fail2Ban is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Fail2Ban; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+
+
+__author__ = "Yaroslav Halchenko"
+__copyright__ = "Copyright (c) 2013 Yaroslav Halchenko"
+__license__ = "GPL"
+
+import logging, os, re, traceback
+from os.path import basename, dirname
+
+#
+# Following "traceback" functions are adopted from PyMVPA distributed
+# under MIT/Expat and copyright by PyMVPA developers (i.e. me and
+# Michael).  Hereby I re-license derivative work on these pieces under GPL
+# to stay in line with the main Fail2Ban license
+#
+def mbasename(s):
+	"""Custom function to include directory name if filename is too common
+
+	Also strip .py at the end
+	"""
+	base = basename(s)
+	if base.endswith('.py'):
+		base = base[:-3]
+	if base in set(['base', '__init__']):
+		base = basename(dirname(s)) + '.' + base
+	return base
+
+class TraceBack(object):
+	"""Customized traceback to be included in debug messages
+	"""
+
+	def __init__(self, compress=False):
+		"""Initialize TrackBack metric
+
+		Parameters
+		----------
+		compress : bool
+		  if True then prefix common with previous invocation gets
+		  replaced with ...
+		"""
+		self.__prev = ""
+		self.__compress = compress
+
+	def __call__(self):
+		ftb = traceback.extract_stack(limit=100)[:-2]
+		entries = [[mbasename(x[0]), str(x[1])] for x in ftb]
+		entries = [ e for e in entries
+					if not e[0] in ['unittest', 'logging.__init__' ]]
+
+		# lets make it more consize
+		entries_out = [entries[0]]
+		for entry in entries[1:]:
+			if entry[0] == entries_out[-1][0]:
+				entries_out[-1][1] += ',%s' % entry[1]
+			else:
+				entries_out.append(entry)
+		sftb = '>'.join(['%s:%s' % (mbasename(x[0]),
+									x[1]) for x in entries_out])
+		if self.__compress:
+			# lets remove part which is common with previous invocation
+			prev_next = sftb
+			common_prefix = os.path.commonprefix((self.__prev, sftb))
+			common_prefix2 = re.sub('>[^>]*$', '', common_prefix)
+
+			if common_prefix2 != "":
+				sftb = '...' + sftb[len(common_prefix2):]
+			self.__prev = prev_next
+
+		return sftb
+
+class FormatterWithTraceBack(logging.Formatter):
+	"""Custom formatter which expands %(tb) and %(tbc) with tracebacks
+
+	TODO: might need locking in case of compressed tracebacks
+	"""
+	def __init__(self, fmt, *args, **kwargs):
+		logging.Formatter.__init__(self, fmt=fmt, *args, **kwargs)
+		compress = '%(tbc)s' in fmt
+		self._tb = TraceBack(compress=compress)
+
+	def format(self, record):
+		record.tbc = record.tb = self._tb()
+		return logging.Formatter.format(self, record)