Merge branch 'master' into 0.9

* master:
  ENH: "is None" instead of "== None" + tune ups in headers
  BF: log error only if there were missed config files that couldn't be read
  DOC: missing cinfo tags are ok. Log error for self referencing definitions
  DOC: s/defination/definition/g learn to spell
  Changelog entry for the previous commit and some untabify
  BF: pyinotify backend should also handle IN_MOVED_TO events
  ENH: remove stats of config files and use results of SafeConfigParserWithIncludes.read to facilitate meaningful error messages
  DOC: credits for gh-70 fix
  BF: ensure dates in email are in the C locale. Thanks iGeorgeX
  DOC: ChangeLog for recursive tag substition
  ENH: allow recursive tag substitution in action files.
  DOC: document <br> tag
  DOC: ChangeLog for named-refused entry
  ENH: Account for views in named filter. By Romain Riviere in gentoo bug #259458
  DOC: release documentation and distributor contacts
  DOC: changelog entry for enhanced ssh filter
  BF: Rename mentioning of README to README.md (Fixes #187)
  updated README.md to hyperlink, add travis and coversall
  Moving README into a markup README.md for github's goodnesses

Conflicts:
	DEVELOP
	README.md
	fail2ban/client/configreader.py
	fail2ban/server/datedetector.py

ChangeLog

@@ -54,6 +54,8 @@ Borreli, blotus:
    * [ab044b75] delay check for the existence of config directory until read.
    * [3b4084d4] fixing up for handling of TAI64N timestamps.
    * [154aa38e] do not shutdown logging until all jails stop.
+   * [f2156604] pyinotify -- monitor IN_MOVED_TO events. Closes gh-184.
+     Thanks to Jon Foster for report and troubleshooting.
   Orion Poplawski
    * [e4aedfdc00] pyinotify - use bitwise op on masks and do not try tracking
      newly created directories.
@@ -69,6 +71,8 @@ Borreli, blotus:
   Daniel Black
    * [f0610c01] Allow more that a one word command when changing and Action via
      the fail2ban-client. Closes gh-134.
+   * [945ad3d9] Fix dates on email actions to work in different locals. Closes
+     gh-70. Thanks to iGeorgeX for the idea.
   blotus
    * [96eb8986] ' and " should also be escaped in action tags Closes gh-109
 - New features:
@@ -114,10 +118,15 @@ Borreli, blotus:
    * [7cd6dab] Added help command to fail2ban-client.
    * [c8c7b0b,23bbc60] Better logging of log file read errors.
    * [3665e6d] Added code coverage to development process.
+   * [41b9f7b,32d10e9] More complete ssh filter rules to match openssh source.
+   * [1d9abd1] Action files can have tags in definition that refer to other
+     tags.
   Pascal Borreli
    * [a2b29b4] Fixed lots of typos in config files and documentation.
   hamilton5
    * [7ede1e8] Update dovecot filter config.
+  Romain Riviere
+   * [0ac8746] Enhance named-refused filter for views.
 
 Special Kudos also go to Fabian Wenk, Arturo 'Buanzo' Busleiman, Tom
 Hendrikx and other TBN heroes supporting users on fail2ban-users

DEVELOP

@@ -262,7 +262,39 @@ Takes care about executing start/check/ban/unban/stop commands
 Releasing
 =========
 
-# Ensure the version is correct in ./fail2ban/version.py
+# Check distribution patches and see if they can be included
+
+  * https://apps.fedoraproject.org/packages/fail2ban/sources
+  * http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/net-analyzer/fail2ban/
+  * http://svnweb.freebsd.org/ports/head/security/py-fail2ban/
+  * https://build.opensuse.org/package/show?package=fail2ban&project=openSUSE%3AFactory
+  * http://sophie.zarb.org/sources/fail2ban (Mageia)
+
+# Check distribution outstanding bugs
+
+  * https://github.com/fail2ban/fail2ban/issues?sort=updated&state=open
+  * http://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;package=fail2ban
+  * http://bugs.sabayon.org/buglist.cgi?quicksearch=net-analyzer%2Ffail2ban
+  * https://bugs.gentoo.org/buglist.cgi?query_format=advanced&short_desc=fail2ban&bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&short_desc_type=allwords
+  * https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&component=fail2ban&classification=Red%20Hat&classification=Fedora
+  * http://www.freebsd.org/cgi/query-pr-summary.cgi?text=fail2ban
+
+# Provide a release sample to distributors
+
+  * Debian: Yaroslav Halchenko <debian@onerussian.com>
+            http://packages.qa.debian.org/f/fail2ban.html
+  * FreeBSD: Christoph Theis theis@gmx.at>, Nick Hilliard <nick@foobar.org>
+            http://svnweb.freebsd.org/ports/head/security/py-fail2ban/Makefile?view=markup
+  * Fedora: Axel Thimm <Axel.Thimm@atrpms.net>
+            https://apps.fedoraproject.org/packages/fail2ban
+  * Gentoo: netmon@gentoo.org
+            http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/net-analyzer/fail2ban/metadata.xml?view=markup
+  * openSUSE: Stephan Kulow <coolo@suse.com>
+            https://build.opensuse.org/package/users?package=fail2ban&project=openSUSE%3AFactory
+
+# Wait for feedback from distributors
+
+# Ensure the version is correct in ./common/version.py
 
 # Add/finalize the corresponding entry in the ChangeLog
 
@@ -296,9 +328,10 @@ Releasing
 
 # Email users and development list of release
 
-TODO notifying distributors etc.
+# notify distributors
 
-Post Release:
+Post Release
+============
 
 Add the following to the top of the ChangeLog
 

MANIFEST

@@ -1,4 +1,4 @@
-README
+README.md
 ChangeLog
 TODO
 THANKS

README.md

@@ -2,10 +2,9 @@
                         / _|__ _(_) |_  ) |__  __ _ _ _  
                        |  _/ _` | | |/ /| '_ \/ _` | ' \ 
                        |_| \__,_|_|_/___|_.__/\__,_|_||_|
+                       v0.9.0a0                2013/05/02
 
-================================================================================
+## Fail2Ban: ban hosts that cause multiple authentication errors
-Fail2Ban (version 0.9.0a0)                                            20??/??/??
-================================================================================
 
 Fail2Ban scans log files like /var/log/pwdfail and bans IP that makes too many
 password failures. It updates firewall rules to reject the IP address. These
@@ -18,32 +17,29 @@ are available in fail2ban(1) manpage and on the website http://www.fail2ban.org
 Installation:
 -------------
 
+**It is possible that Fail2ban is already packaged for your distribution.  In
+this case, you should use it instead.**
+
 Required:
-   >=python-2.4 or >=python-3.0 (http://www.python.org)
+- [Python >= 2.4, including 3.x](http://www.python.org)
 
 Optional:
-   pyinotify:
+- [pyinotify >= 0.8.3](https://github.com/seb-m/pyinotify)
-      >=linux-2.6.13
+  - Linux >= 2.6.13
-      >=python-2.4
+- [gamin >= 0.0.21](http://www.gnome.org/~veillard/gamin)
-      >=pyinotify-0.8.3 (https://github.com/seb-m/pyinotify)
-   Gamin:
-      >=gamin-0.0.21 (http://www.gnome.org/~veillard/gamin)
 
 To install, just do:
 
-> tar xvfj fail2ban-0.8.8.tar.bz2
+    tar xvfj fail2ban-0.8.8.tar.bz2
-> cd fail2ban-0.8.8
+    cd fail2ban-0.8.8
-> python setup.py install
+    python setup.py install
 
 This will install Fail2Ban into /usr/share/fail2ban. The executable scripts are
-placed into /usr/bin.
+placed into /usr/bin, and configuration under /etc/fail2ban.
-
-It is possible that Fail2ban is already packaged for your distribution.  In
-this case, you should use it.
 
 Fail2Ban should be correctly installed now. Just type:
 
-> fail2ban-client -h
+    fail2ban-client -h
 
 to see if everything is alright. You should always use fail2ban-client and
 never call fail2ban-server directly.
@@ -57,29 +53,35 @@ available commands are described in the fail2ban-client(1) manpage.  Also see
 fail2ban(1) manpage for further references and find even more documentation on
 the website: http://www.fail2ban.org
 
+Code status:
+------------
+
+* [![tests status](https://secure.travis-ci.org/fail2ban/fail2ban.png)](https://travis-ci.org/fail2ban/fail2ban) travis-ci.org (master branch)
+
+* [![Coverage Status](https://coveralls.io/repos/fail2ban/fail2ban/badge.png?branch=master)](https://coveralls.io/r/fail2ban/fail2ban)
+
 Contact:
 --------
 
-Website: http://www.fail2ban.org
+### You need some new features, you found bugs?
-
+visit [Issues](https://github.com/fail2ban/fail2ban/issues)
-You need some new features, you found bugs?
-visit https://github.com/fail2ban/fail2ban/issues
 and if your issue is not yet known -- file a bug report.
 
-You would like to troubleshoot or discuss?
+### You would like to troubleshoot or discuss?
-join the mailing list
+join the [mailing list](https://lists.sourceforge.net/lists/listinfo/fail2ban-users)
-https://lists.sourceforge.net/lists/listinfo/fail2ban-users
 
-You just appreciate this program:
+### You would like to contribute (new filters/actions/code/documentation)?
-send kudos to the original author (Cyril Jaquier <cyril.jaquier@fail2ban.org>)
+send a pull request
-or better to the mailing list
+
-https://lists.sourceforge.net/lists/listinfo/fail2ban-users
+### You just appreciate this program:
+send kudos to the original author ([Cyril Jaquier](mailto: Cyril Jaquier <cyril.jaquier@fail2ban.org>)
+or better to the [mailing list](https://lists.sourceforge.net/lists/listinfo/fail2ban-users)
 since Fail2Ban is "community-driven" for years now.
 
 Thanks:
 -------
 
-See THANKS file.
+See [THANKS](https://github.com/fail2ban/fail2ban/blob/master/THANKS) file.
 
 License:
 --------

bin/fail2ban-client

@@ -342,9 +342,9 @@ class Fail2banClient:
 		# Set socket path
 		self.__configurator.readEarly()
 		conf = self.__configurator.getEarlyOptions()
-		if self.__conf["socket"] == None:
+		if self.__conf["socket"] is None:
 			self.__conf["socket"] = conf["socket"]
-		if self.__conf["pidfile"] == None:
+		if self.__conf["pidfile"] is None:
 			self.__conf["pidfile"] = conf["pidfile"]
 		logSys.info("Using socket file " + self.__conf["socket"])
 

config/action.d/sendmail-whois-lines.conf

@@ -12,7 +12,7 @@
 # Values:  CMD
 #
 actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
-              Date: `date -u +"%%a, %%d %%h %%Y %%T +0000"`
+              Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
               From: Fail2Ban <<sender>>
               To: <dest>\n
               Hi,\n
@@ -25,7 +25,7 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
 # Values:  CMD
 #
 actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped
-             Date: `date -u +"%%a, %%d %%h %%Y %%T +0000"`
+             Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
              From: Fail2Ban <<sender>>
              To: <dest>\n
              Hi,\n
@@ -46,7 +46,7 @@ actioncheck =
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
-            Date: `date -u +"%%a, %%d %%h %%Y %%T +0000"`
+            Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
             From: Fail2Ban <<sender>>
             To: <dest>\n
             Hi,\n

config/action.d/sendmail-whois.conf

@@ -12,7 +12,7 @@
 # Values:  CMD
 #
 actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
-              Date: `date -u +"%%a, %%d %%h %%Y %%T +0000"`
+              Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
               From: Fail2Ban <<sender>>
               To: <dest>\n
               Hi,\n
@@ -25,7 +25,7 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
 # Values:  CMD
 #
 actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped
-             Date: `date -u +"%%a, %%d %%h %%Y %%T +0000"`
+             Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
              From: Fail2Ban <<sender>>
              To: <dest>\n
              Hi,\n
@@ -46,7 +46,7 @@ actioncheck =
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
-            Date: `date -u +"%%a, %%d %%h %%Y %%T +0000"`
+            Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
             From: Fail2Ban <<sender>>
             To: <dest>\n
             Hi,\n

config/action.d/sendmail.conf

@@ -12,7 +12,7 @@
 # Values:  CMD
 #
 actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
-              Date: `date -u +"%%a, %%d %%h %%Y %%T +0000"`
+              Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
               From: Fail2Ban <<sender>>
               To: <dest>\n
               Hi,\n
@@ -25,7 +25,7 @@ actionstart = printf %%b "Subject: [Fail2Ban] <name>: started
 # Values:  CMD
 #
 actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped
-             Date: `date -u +"%%a, %%d %%h %%Y %%T +0000"`
+             Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
              From: Fail2Ban <<sender>>
              To: <dest>\n
              Hi,\n
@@ -46,7 +46,7 @@ actioncheck =
 # Values:  CMD
 #
 actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
-            Date: `date -u +"%%a, %%d %%h %%Y %%T +0000"`
+            Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
             From: Fail2Ban <<sender>>
             To: <dest>\n
             Hi,\n

config/filter.d/named-refused.conf

@@ -26,7 +26,7 @@ __line_prefix=(?:\s\S+ %(__daemon_combs_re)s\s+)?
 # Notes.: regex to match the password failures messages in the logfile.
 # Values: TEXT
 #
-failregex = %(__line_prefix)sclient <HOST>#.+: query(?: \(cache\))? '.*' denied\s*$
+failregex = %(__line_prefix)sclient <HOST>#\S+: (view (internal|external): )?query(?: \(cache\))? '.*' denied\s*$
 
 # Option:  ignoreregex
 # Notes.:  regex to ignore. If this regex matches, the line is ignored.

fail2ban/client/beautifier.py

@@ -56,10 +56,10 @@ class Beautifier:
 				msg = "Jail started"
 			elif inC[0] == "stop":
 				if len(inC) == 1:
-					if response == None:
+					if response is None:
 						msg = "Shutdown successful"
 				else:
-					if response == None:
+					if response is None:
 						msg = "Jail stopped"
 			elif inC[0] == "add":
 				msg = "Added jail " + response

fail2ban/client/configreader.py

@@ -52,9 +52,9 @@ class ConfigReader(SafeConfigParserWithIncludes):
 		return self._basedir
 	
 	def read(self, filename):
-		if not (os.path.exists(self._basedir) and os.access(self._basedir, os.R_OK | os.X_OK)):
+		if not os.path.exists(self._basedir):
-			raise ValueError("Base configuration directory %s either does not exist "
+			raise ValueError("Base configuration directory %s does not exist "
-							 "or is not accessible" % self._basedir)
+							  % self._basedir)
 		basename = os.path.join(self._basedir, filename)
 		logSys.debug("Reading configs for %s under %s "  % (basename, self._basedir))
 		config_files = [ basename + ".conf",
@@ -65,27 +65,20 @@ class ConfigReader(SafeConfigParserWithIncludes):
 
 		# possible further customizations under a .conf.d directory
 		config_dir = basename + '.d'
-		if os.path.exists(config_dir):
+		config_files += sorted(glob.glob('%s/*.conf' % config_dir))
-			if os.path.isdir(config_dir) and os.access(config_dir, os.X_OK | os.R_OK):
-				# files must carry .conf suffix as well
-				config_files += sorted(glob.glob('%s/*.conf' % config_dir))
-			else:
-				logSys.warning("%s exists but not a directory or not accessible"
-							 % config_dir)
 
-		# check if files are accessible, warn if any is not accessible
+		if len(config_files):
-		# and remove it from the list
-		config_files_accessible = []
-		for f in config_files:
-			if os.access(f, os.R_OK):
-				config_files_accessible.append(f)
-			else:
-				logSys.warning("%s exists but not accessible - skipping" % f)
-
-		if len(config_files_accessible):
 			# at least one config exists and accessible
-			SafeConfigParserWithIncludes.read(self, config_files_accessible)
+			logSys.debug("Reading config files: " + ', '.join(config_files))
-			return True
+			config_files_read = SafeConfigParserWithIncludes.read(self, config_files)
+			missed = [ cf for cf in config_files if cf not in config_files_read ]
+			if missed:
+				logSys.error("Could not read config files: " + ', '.join(missed))
+			if config_files_read:
+				return True
+			logSys.error("Found no accessible config files for %r under %s" %
+						 ( filename, self.getBaseDir() ))
+			return False
 		else:
 			logSys.error("Found no accessible config files for %r " % filename
 						 + (["under %s" % self.getBaseDir(),
@@ -113,7 +106,7 @@ class ConfigReader(SafeConfigParserWithIncludes):
 					v = self.getint(sec, option[1])
 				else:
 					v = self.get(sec, option[1])
-				if not pOptions == None and option[1] in pOptions:
+				if not pOptions is None and option[1] in pOptions:
 					continue
 				values[option[1]] = v
 			except NoSectionError, e:
@@ -121,7 +114,7 @@ class ConfigReader(SafeConfigParserWithIncludes):
 				logSys.error(e)
 				values[option[1]] = option[2]
 			except NoOptionError:
-				if not option[2] == None:
+				if not option[2] is None:
 					logSys.warning("'%s' not defined in '%s'. Using default one: %r"
 								% (option[1], sec, option[2]))
 					values[option[1]] = option[2]

fail2ban/server/action.py

@@ -17,18 +17,12 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-# Author: Cyril Jaquier
+__author__ = "Cyril Jaquier and Fail2Ban Contributors"
-# 
+__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2011-2012 Yaroslav Halchenko"
-# $Revision$
-
-__author__ = "Cyril Jaquier"
-__version__ = "$Revision$"
-__date__ = "$Date$"
-__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
 import logging, os
-import threading
+import threading, re
 #from subprocess import call
 
 # Gets the instance of the logger.
@@ -143,6 +137,10 @@ class Action:
 	# @return True if the command succeeded
 	
 	def execActionStart(self):
+		if self.__cInfo:
+			if not Action.substituteRecursiveTags(self.__cInfo):
+				logSys.error("Cinfo/definitions contain self referencing definitions and cannot be resolved")
+				return False
 		startCmd = Action.replaceTag(self.__actionStart, self.__cInfo)
 		return Action.executeCmd(startCmd)
 	
@@ -242,6 +240,38 @@ class Action:
 		stopCmd = Action.replaceTag(self.__actionStop, self.__cInfo)
 		return Action.executeCmd(stopCmd)
 
+	##
+	# Sort out tag definitions within other tags
+	#
+	# so:		becomes:
+	# a = 3		a = 3
+	# b = <a>_3	b = 3_3
+	# @param	tags, a dictionary
+	# @returns	tags altered or False if there is a recursive definition
+	#@staticmethod
+	def substituteRecursiveTags(tags):
+		t = re.compile(r'<([^ >]+)>')
+		for tag, value in tags.iteritems():
+			value = str(value)
+			m = t.search(value)
+			while m:
+				if m.group(1) == tag:
+					# recursive definitions are bad
+					return False
+				else:
+					if tags.has_key(m.group(1)):
+						value = value[0:m.start()] + tags[m.group(1)] + value[m.end():]
+						m = t.search(value, m.start())
+					else:
+						# Missing tags are ok so we just continue on searching.
+						# cInfo can contain aInfo elements like <HOST> and valid shell
+						# constructs like <STDIN>.
+						m = t.search(value, m.start() + 1)
+			tags[tag] = value
+		return tags
+	substituteRecursiveTags = staticmethod(substituteRecursiveTags)
+
+	#@staticmethod
 	def escapeTag(tag):
 		for c in '\\#&;`|*?~<>^()[]{}$\n\'"':
 			if c in tag:
@@ -304,7 +334,7 @@ class Action:
 				return False
 
 		# Replace tags
-		if not aInfo == None:
+		if not aInfo is None:
 			realCmd = Action.replaceTag(cmd, aInfo)
 		else:
 			realCmd = cmd

fail2ban/server/datedetector.py

@@ -17,13 +17,7 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-# Author: Cyril Jaquier
+__author__ = "Cyril Jaquier and Fail2Ban Contributors"
-# 
-# $Revision$
-
-__author__ = "Cyril Jaquier"
-__version__ = "$Revision$"
-__date__ = "$Date$"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
@@ -203,10 +197,7 @@ class DateDetector:
 
 	def getUnixTime(self, line):
 		date = self.getTime(line)
-		if date == None:
+		return date and time.mktime(tuple(date))
-			return None
-		else:
-			return time.mktime(tuple(date))
 
 	##
 	# Sort the template lists using the hits score. This method is not called

fail2ban/server/datetemplate.py

@@ -65,7 +65,7 @@ class DateTemplate:
 	
 	def matchDate(self, line):
 		dateMatch = self.__cRegex.search(line)
-		if not dateMatch == None:
+		if not dateMatch is None:
 			self.__hits += 1
 		return dateMatch
 	

fail2ban/server/failregex.py

@@ -17,13 +17,7 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-# Author: Cyril Jaquier
-# 
-# $Revision$
-
 __author__ = "Cyril Jaquier"
-__version__ = "$Revision$"
-__date__ = "$Date$"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
@@ -191,7 +185,7 @@ class FailRegex(Regex):
 	
 	def getHost(self):
 		host = self._matchCache.group("host")
-		if host == None:
+		if host is None:
 			# Gets a few information.
 			s = self._matchCache.string
 			r = self._matchCache.re

fail2ban/server/filter.py

@@ -17,14 +17,8 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-# Author: Cyril Jaquier
+__author__ = "Cyril Jaquier and Fail2Ban Contributors"
-#
+__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2011-2013 Yaroslav Halchenko"
-# $Revision$
-
-__author__ = "Cyril Jaquier"
-__version__ = "$Revision$"
-__date__ = "$Date$"
-__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
 from failmanager import FailManagerEmpty
@@ -384,7 +378,7 @@ class Filter(JailThread):
 					continue
 				# The failregex matched.
 				date = self.dateDetector.getUnixTime(timeLine)
-				if date == None:
+				if date is None:
 					logSys.debug("Found a match for %r but no valid date/time "
 								 "found for %r. Please file a detailed issue on"
 								 " https://github.com/fail2ban/fail2ban/issues "
@@ -521,7 +515,7 @@ class FileFilter(Filter):
 
 	def getFailures(self, filename):
 		container = self.getFileContainer(filename)
-		if container == None:
+		if container is None:
 			logSys.error("Unable to get failures in " + filename)
 			return False
 		# Try to open log file.
@@ -626,7 +620,7 @@ class FileContainer:
 		self.__handler.seek(self.__pos)
 
 	def readline(self):
-		if self.__handler == None:
+		if self.__handler is None:
 			return ""
 		line = self.__handler.readline()
 		try:
@@ -639,7 +633,7 @@ class FileContainer:
 		return line
 
 	def close(self):
-		if not self.__handler == None:
+		if not self.__handler is None:
 			# Saves the last position.
 			self.__pos = self.__handler.tell()
 			# Closes the file.

fail2ban/server/filterpyinotify.py

@@ -66,7 +66,7 @@ class FilterPyinotify(FileFilter):
 	def callback(self, event, origin=''):
 		logSys.debug("%sCallback for Event: %s", origin, event)
 		path = event.pathname
-		if event.mask & pyinotify.IN_CREATE:
+		if event.mask & ( pyinotify.IN_CREATE | pyinotify.IN_MOVED_TO ):
 			# skip directories altogether
 			if event.mask & pyinotify.IN_ISDIR:
 				logSys.debug("Ignoring creation of directory %s", path)
@@ -130,7 +130,7 @@ class FilterPyinotify(FileFilter):
 		if not (path_dir in self.__watches):
 			# we need to watch also  the directory for IN_CREATE
 			self.__watches.update(
-				self.__monitor.add_watch(path_dir, pyinotify.IN_CREATE))
+				self.__monitor.add_watch(path_dir, pyinotify.IN_CREATE | pyinotify.IN_MOVED_TO))
 			logSys.debug("Added monitor for the parent directory %s", path_dir)
 
 		self._addFileWatcher(path)

fail2ban/server/mytime.py

@@ -17,13 +17,7 @@
 # along with Fail2Ban; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
-# Author: Cyril Jaquier
-# 
-# $Revision$
-
 __author__ = "Cyril Jaquier"
-__version__ = "$Revision$"
-__date__ = "$Date$"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
@@ -61,7 +55,7 @@ class MyTime:
 	
 	#@staticmethod
 	def time():
-		if MyTime.myTime == None:
+		if MyTime.myTime is None:
 			return time.time()
 		else:
 			return MyTime.myTime
@@ -74,14 +68,14 @@ class MyTime:
 	
 	#@staticmethod
 	def gmtime():
-		if MyTime.myTime == None:
+		if MyTime.myTime is None:
 			return time.gmtime()
 		else:
 			return time.gmtime(MyTime.myTime)
 	gmtime = staticmethod(gmtime)
 
 	def localtime(x=None):
-		if MyTime.myTime == None or x is not None:
+		if MyTime.myTime is None or x is not None:
 			return time.localtime(x)
 		else:
 			return time.localtime(MyTime.myTime)

fail2ban/server/server.py

@@ -402,7 +402,7 @@ class Server:
 			hdlr.setFormatter(formatter)
 			logger.addHandler(hdlr)
 			# Does not display this message at startup.
-			if not self.__logTarget == None:
+			if not self.__logTarget is None:
 				logSys.info("Changed logging target to %s for Fail2ban v%s" %
 						(target, version.version))
 			# Sets the logging target.

fail2ban/tests/actiontestcase.py

@@ -62,6 +62,27 @@ class ExecuteAction(unittest.TestCase):
 	def _is_logged(self, s):
 		return s in self._log.getvalue()
 
+	def testSubstituteRecursiveTags(self):
+		aInfo = {
+			'HOST': "192.0.2.0",
+			'ABC': "123 <HOST>",
+			'xyz': "890 <ABC>",
+		}
+		# Recursion is bad
+		self.assertFalse(Action.substituteRecursiveTags({'A': '<A>'}))
+		self.assertFalse(Action.substituteRecursiveTags({'A': '<B>', 'B': '<A>'}))
+		self.assertFalse(Action.substituteRecursiveTags({'A': '<B>', 'B': '<C>', 'C': '<A>'}))
+		# missing tags are ok
+		self.assertEquals(Action.substituteRecursiveTags({'A': '<C>'}), {'A': '<C>'})
+		self.assertEquals(Action.substituteRecursiveTags({'A': '<C> <D> <X>','X':'fun'}), {'A': '<C> <D> fun', 'X':'fun'})
+		self.assertEquals(Action.substituteRecursiveTags({'A': '<C> <B>', 'B': 'cool'}), {'A': '<C> cool', 'B': 'cool'})
+		# rest is just cool
+		self.assertEquals(Action.substituteRecursiveTags(aInfo),
+								{ 'HOST': "192.0.2.0",
+									'ABC': '123 192.0.2.0',
+									'xyz': '890 123 192.0.2.0',
+								})
+
 	def testReplaceTag(self):
 		aInfo = {
 			'HOST': "192.0.2.0",

fail2ban/tests/files/logs/named-refused

@@ -3,3 +3,4 @@ Jul 24 14:16:56 raid5 named[3935]: client 62.123.164.113#32768: query 'ricreig.c
 Jul 24 14:17:13 raid5 named[3935]: client 148.160.29.6#33081: query (cache) 'geo-mueller.de/NS/IN' denied
 Jul 24 14:20:25 raid5 named[3935]: client 148.160.29.6#33081: query (cache) 'shivaree.de/NS/IN' denied
 Jul 24 14:23:36 raid5 named[3935]: client 148.160.29.6#33081: query (cache) 'mietberatung.de/NS/IN' denied
+Jul 24 14:23:36 raid5 named[3935]: client 62.109.4.89#9334: view external: query (cache) './NS/IN' denied

fail2ban/tests/filtertestcase.py

@@ -495,6 +495,40 @@ def get_monitor_failures_testcase(Filter_):
 			self.assertEqual(self.filter.failManager.getFailTotal(), 6)
 
 
+		def _test_move_into_file(self, interim_kill=False):
+			# if we move a new file into the location of an old (monitored) file
+			self.file1 = _copy_lines_between_files(GetFailures.FILENAME_01, self.name,
+												  n=100)
+			# make sure that it is monitored first
+			self.assert_correct_last_attempt(GetFailures.FAILURES_01)
+			self.assertEqual(self.filter.failManager.getFailTotal(), 3)
+
+			if interim_kill:
+				_killfile(None, self.name)
+				time.sleep(0.2)				  # let them know
+
+			# now create a new one to override old one
+			self.file = _copy_lines_between_files(GetFailures.FILENAME_01,
+												   self.name + '.new', n=100)
+			os.rename(self.name + '.new', self.name)
+			self.assert_correct_last_attempt(GetFailures.FAILURES_01)
+			self.assertEqual(self.filter.failManager.getFailTotal(), 6)
+
+			# and to make sure that it now monitored for changes
+			_copy_lines_between_files(GetFailures.FILENAME_01, self.name, n=100)
+			self.assert_correct_last_attempt(GetFailures.FAILURES_01)
+			self.assertEqual(self.filter.failManager.getFailTotal(), 9)
+
+
+		def test_move_into_file(self):
+			self._test_move_into_file(interim_kill=False)
+
+		def test_move_into_file_after_removed(self):
+			# exactly as above test + remove file explicitly
+			# to test against possible drop-out of the file from monitoring
+		    self._test_move_into_file(interim_kill=True)
+
+
 		def test_new_bogus_file(self):
 			# to make sure that watching whole directory does not effect
 			_copy_lines_between_files(GetFailures.FILENAME_01, self.name, n=100).close()

man/jail.conf.5

@@ -100,7 +100,7 @@ Commands specified in the [Definition] section are executed through a system she
 return 0, otherwise error would be logged.  Moreover if \fBactioncheck\fR exits with non-0 status, it is taken as indication that firewall status has changed and fail2ban needs to reinitialize itself (i.e. issue \fBactionstop\fR and \fBactionstart\fR commands).
 
 Tags are enclosed in <>.  All the elements of [Init] are tags that are replaced in all action commands.  Tags can be added by the
-\fBfail2ban-client\fR using the setctag command.
+\fBfail2ban-client\fR using the setctag command. \fB<br>\fR is a tag that is always a new line (\\n).
 
 More than a single command is allowed to be specified. Each command needs to be on a separate line and indented with whitespaces without blank lines. The following example defines
 two commands to be executed.

setup.cfg

@@ -5,6 +5,6 @@ formats=bztar
 release = 1
 packager = Yaroslav Halchenko <debian@onerussian.com>, Daniel Black <grooverdan@users.sourceforge.net>
 doc_files = DEVELOP
-            README
+            README.md
             THANKS
             doc/run-rootless.txt

setup.py

@@ -123,7 +123,7 @@ setup(
 							''
 						),
 						('/usr/share/doc/fail2ban',
-							['README', 'DEVELOP', 'doc/run-rootless.txt']
+							['README.md', 'DEVELOP', 'doc/run-rootless.txt']
 						)
 					],
 	**setup_extra