Merge branch 'master' into 0.9

* master: (26 commits)
  DOC: added a note that coverage script is python-coverage on Debian systems
  Fixed typos
  Added additional Transmitter tests, and some associated fixes
  TODO: test filters/examples files
  add corresponding ChangeLog entry
  do catch all exception
  ENH: typo + head -1 has been deprecated for 10+ years.
  ENH: add help command
  BF: allow more than single word for command action[start,stop,ban,unban,check] and for setcinfo too
  BF: general Exception catch was excessive. Only IOError and OSError are possible and has different meanings
  Add development documentation and framework for code coverage measurement
  FSF address changes missing from previous
  refresh generated manpages (since 0.8.2 state)
  Downgrade log rotation detection message to DEBUG level from INFO. Closes: gh-129
  BF: do not shutdown logging until all jails stop -- so move into Server.quit()
  BF: safeguard closing of log handlers + close in reverse order
  Added transmitter get cinfo option for action
  Fix for missing value in transmitter delaction
  Rewrite and enable server testcase for Transmitter
  ENH: adding more of diagnostic messages into -client while starting the daemon
  ...

.coveragerc

@@ -0,0 +1,4 @@
+
+[run]
+branch = True
+omit = /usr*

.gitignore

@@ -1,3 +1,6 @@
 *~
 build
+dist
 *.pyc
+htmlcov
+.coverage

DEVELOP

@@ -24,14 +24,97 @@ Request feature. You can find more details on the Fail2Ban wiki
 Testing
 =======
 
-Existing tests can be run by executing `fail2ban-testcases`.
+Existing tests can be run by executing `fail2ban-testcases`. This has options
+like --log-level that will probably be useful. `fail2ban-testcases --help` for
+full options.
+
+Test cases should cover all usual cases, all exception cases and all inside
+/ outside boundary conditions.
+
+Test cases should cover all branches. The coverage tool will help identify
+missing branches. Also see http://nedbatchelder.com/code/coverage/branch.html
+for more details.
+
+Install the package python-coverage to visualise your test coverage. Run the
+following (note: on Debian-based systems, the script is called
+`python-coverage`):
+
+coverage run fail2ban-testcases
+coverage html
+
+Then look at htmlcov/index.html and see how much coverage your test cases
+exert over the codebase. Full coverage is a good thing however it may not be
+complete. Try to ensure tests cover as many independant paths through the
+code.
+
+Manual Execution. To run in a development environment do:
+
+./fail2ban-client -c config/ -s /tmp/f2b.sock -i start
+
+some quick commands:
+
+status
+add test pyinotify
+status test
+set test addaction iptables
+set test actionban iptables echo <ip> <cidr> >> /tmp/ban
+set test actionunban iptables echo <ip> <cidr> >> /tmp/unban
+get test actionban iptables
+get test actionunban iptables
+set test banip 192.168.2.2
+status test
+
 
-Documentation about creating tests (when tests are required and some guidelines
-for creating good tests) will be added soon.
 
 Coding Standards
 ================ 
-Coming Soon.
+
+Style
+-----
+
+Please use tabs for now. Keep to 80 columns, at least for readable text.
+
+Tests
+-----
+
+Add tests. They should test all the code you add in a meaning way.
+
+Coverage
+--------
+
+Test coverage should always increase as you add code.
+
+You may use "# pragma: no cover" in the code for branches of code that support
+older versions on python. For all other uses of "pragma: no cover" or
+"pragma: no branch" document the reason why its not covered. "I haven't written
+a test case" isn't a sufficient reason.
+
+Documentation
+-------------
+
+Ensure this documentation is up to date after changes. Also ensure that the man
+pages still are accurate. Ensure that there is sufficient documentation for
+your new features to be used.
+
+Bugs
+----
+
+Remove them and don't add any more.
+
+Git
+---
+
+Use the following tags in your commit messages:
+
+'ENH:' for enhancements
+'BF:' for bug fixes
+'DOC:' for documentation fixes
+
+Adding Actions
+--------------
+
+If you add an action.d/*.conf file also add a example in config/jail.conf
+with enabled=false and maxretry=5 for ssh.
 
 
 Design
@@ -127,12 +210,14 @@ FileContainer
   .__pos
     Keeps the position pointer
 
+
+dnsutils.py
+~~~~~~~~~~~
+
 DNSUtils
 
   Utility class for DNS and IP handling
 
-  RF-Note: convert to functions within a separate submodule
-
 
 filter*.py
 ~~~~~~~~~~
@@ -156,3 +241,29 @@ action.py
 ~~~~~~~~~
 
 Takes care about executing start/check/ban/unban/stop commands
+
+
+Releasing
+=========
+
+Ensure the version is correct in ./common/version.py
+
+Add/finalize the corresponding entry in the ChangeLog
+
+# update man pages
+(cd man ; ./generate-man )
+
+git commit -m 'update man pages for release' man/*
+
+python setup.py check
+python setup.py sdist
+python setup.py bdist_rpm
+python setup.py upload
+
+Run the following and update the wiki with output:
+
+python -c 'import common.protocol; common.protocol.printWiki()'
+
+email users and development list of release
+
+TODO notifying distributors etc.

MANIFEST

@@ -3,6 +3,8 @@ ChangeLog
 TODO
 THANKS
 COPYING
+DEVELOP
+doc/run-rootless.txt
 fail2ban-client
 fail2ban-server
 fail2ban-testcases

README

@@ -91,5 +91,5 @@ WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
 PARTICULAR PURPOSE. See the GNU General Public License for more details.
 
 You should have received a copy of the GNU General Public License along with
-Fail2Ban; if not, write to the Free Software Foundation, Inc., 59 Temple Place,
+Fail2Ban; if not, write to the Free Software Foundation, Inc., 51 Franklin
-Suite 330, Boston, MA  02111-1307  USA
+Street, Fifth Floor, Boston, MA 02110, USA

TODO

@@ -13,6 +13,9 @@ Legend:
 # partially done
 * done
 
+- Run tests though all filters/examples files - (see sshd example file) as unit
+  test
+
 - Removed relative imports
 
 - Cleanup fail2ban-client and fail2ban-server. Move code to server/ and client/

client/fail2banreader.py

@@ -42,7 +42,8 @@ class Fail2banReader(ConfigReader):
 		ConfigReader.read(self, "fail2ban")
 	
 	def getEarlyOptions(self):
-		opts = [["string", "socket", "/tmp/fail2ban.sock"]]
+		opts = [["string", "socket", "/tmp/fail2ban.sock"],
+				["string", "pidfile", "/var/run/fail2ban/fail2ban.pid"]]
 		return ConfigReader.getOptions(self, "Definition", opts)
 	
 	def getOptions(self):

common/protocol.py

@@ -40,6 +40,7 @@ protocol = [
 ["stop", "stops all jails and terminate the server"], 
 ["status", "gets the current status of the server"], 
 ["ping", "tests if the server is alive"], 
+["help", "return this output"], 
 ['', "LOGGING", ""],
 ["set loglevel <LEVEL>", "sets logging level to <LEVEL>. 0 is minimal, 4 is debug"], 
 ["get loglevel", "gets the logging level"], 
@@ -92,6 +93,7 @@ protocol = [
 ["get <JAIL> actioncheck <ACT>", "gets the check command for the action <ACT> for <JAIL>"],
 ["get <JAIL> actionban <ACT>", "gets the ban command for the action <ACT> for <JAIL>"],
 ["get <JAIL> actionunban <ACT>", "gets the unban command for the action <ACT> for <JAIL>"],
+["get <JAIL> cinfo <ACT> <KEY>", "gets the value for <KEY> for the action <ACT> for <JAIL>"],
 ]
 
 ##

config/action.d/dshield.conf

@@ -116,7 +116,7 @@ actionunban = if [ -f <tmpfile>.first ]; then
 port = ???
 
 # Option:  userid
-# Notes.:  Your DSheild user ID. Should be provided either in the jail config or
+# Notes.:  Your DShield user ID. Should be provided either in the jail config or
 #          in a .local file.
 #          Register at https://secure.dshield.org/register.html
 # Values:  [ NUM ]  Default: 0
@@ -124,13 +124,13 @@ port = ???
 userid = 0
 
 # Option:  myip
-# Notes.:  TThe target IP for the attack (your public IP). Should be provided
+# Notes.:  The target IP for the attack (your public IP). Should be provided
 #          either in the jail config or in a .local file unless your PUBLIC IP
 #          is the first IP assigned to eth0
 # Values:  [ an IP address ]  Default: Tries to find the IP address of eth0,
 #          which in most cases will be a private IP, and therefore incorrect
 #
-myip = `ip -4 addr show dev eth0 | grep inet | head -1 | sed -r 's/.*inet ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/'`
+myip = `ip -4 addr show dev eth0 | grep inet | head -n 1 | sed -r 's/.*inet ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/'`
 
 # Option:  protocol
 # Notes.:  The protocol over which the attack is happening

config/action.d/iptables-allports.conf

@@ -53,7 +53,7 @@ actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables-ipset-proto4.conf

@@ -53,7 +53,7 @@ actionunban = ipset --test fail2ban-<name> <ip> && ipset --del fail2ban-<name> <
 
 [Init]
 
-# Defaut name of the ipset
+# Default name of the ipset
 #
 name = default
 

config/action.d/iptables-ipset-proto6.conf

@@ -53,7 +53,7 @@ actionunban = ipset del fail2ban-<name> <ip> -exist
 
 [Init]
 
-# Defaut name of the ipset
+# Default name of the ipset
 #
 name = default
 

config/action.d/iptables-multiport-log.conf

@@ -61,7 +61,7 @@ actionunban = iptables -D fail2ban-<name> -s <ip> -j fail2ban-<name>-log
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables-multiport.conf

@@ -51,7 +51,7 @@ actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables-new.conf

@@ -53,7 +53,7 @@ actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables-xt_recent-echo.conf

@@ -11,7 +11,7 @@
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-# Changing iptables rules requires root priviledges. If fail2ban is
+# Changing iptables rules requires root privileges. If fail2ban is
 # configured to run as root, firewall setup can be performed by
 # fail2ban automatically. However, if fail2ban is configured to run as
 # a normal user, the configuration must be done by some other means
@@ -65,7 +65,7 @@ actionunban = echo -<ip> > /proc/net/xt_recent/fail2ban-<name>
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/iptables.conf

@@ -51,7 +51,7 @@ actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/mail-whois-lines.conf

@@ -62,7 +62,7 @@ actionunban =
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/mail-whois.conf

@@ -59,7 +59,7 @@ actionunban =
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/mail.conf

@@ -57,7 +57,7 @@ actionunban =
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/mynetwatchman.conf

@@ -102,13 +102,13 @@ mnwlogin =
 mnwpass =
 
 # Option:  myip
-# Notes.:  TThe target IP for the attack (your public IP). Should be overridden
+# Notes.:  The target IP for the attack (your public IP). Should be overridden
 #          either in the jail config or in a .local file unless your PUBLIC IP
 #          is the first IP assigned to eth0
 # Values:  [ an IP address ]  Default: Tries to find the IP address of eth0,
 #          which in most cases will be a private IP, and therefore incorrect
 #
-myip = `ip -4 addr show dev eth0 | grep inet | head -1 | sed -r 's/.*inet ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/'`
+myip = `ip -4 addr show dev eth0 | grep inet | head -n 1 | sed -r 's/.*inet ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/'`
 
 # Option:  protocol
 # Notes.:  The protocol over which the attack is happening

config/action.d/route.conf

@@ -0,0 +1,25 @@
+# Fail2Ban configuration file
+#
+# Author: Michael Gebetsroither
+#
+# This is for blocking whole hosts through blackhole routes.
+#
+# PRO:
+#   - Works on all kernel versions and as no compatibility problems (back to debian lenny and WAY further).
+#   - It's FAST for very large numbers of blocked ips.
+#   - It's FAST because it Blocks traffic before it enters common iptables chains used for filtering.
+#   - It's per host, ideal as action against ssh password bruteforcing to block further attack attempts.
+#   - No additional software required beside iproute/iproute2
+#
+# CON:
+#   - Blocking is per IP and NOT per service, but ideal as action against ssh password bruteforcing hosts
+
+[Definition]
+actionban   = ip route add <type> <ip>
+actionunban = ip route del <type> <ip>
+
+# Type of blocking
+#
+# Type can be blackhole, unreachable and prohibit. Unreachable and prohibit correspond to the ICMP reject messages.
+
+type = blackhole

config/action.d/sendmail-buffered.conf

@@ -83,7 +83,7 @@ actionunban =
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/sendmail-whois-lines.conf

@@ -57,7 +57,7 @@ actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip>
             Here are more information about <ip>:\n
             `/usr/bin/whois <ip>`\n\n
             Lines containing IP:<ip> in <logpath>\n
-            `/bin/grep '\<<ip>\>' <logpath>`\n\n
+            `grep '\<<ip>\>' <logpath>`\n\n
             Regards,\n
             Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
 
@@ -73,7 +73,7 @@ actionunban =
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/sendmail-whois.conf

@@ -71,7 +71,7 @@ actionunban =
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/action.d/sendmail.conf

@@ -69,7 +69,7 @@ actionunban =
 
 [Init]
 
-# Defaut name of the chain
+# Default name of the chain
 #
 name = default
 

config/fail2ban.conf

@@ -40,3 +40,10 @@ logtarget = /var/log/fail2ban.log
 #
 socket = /var/run/fail2ban/fail2ban.sock
 
+# Option: pidfile
+# Notes.: Set the PID file. This is used to store the process ID of the
+#         fail2ban server.
+# Values: FILE  Default:  /var/run/fail2ban/fail2ban.sock
+#
+pidfile = /var/run/fail2ban/fail2ban.pid
+

config/filter.d/apache-badbots.conf

@@ -16,7 +16,7 @@ badbots = atSpider/1\.0|autoemailspider|China Local Browse 2\.6|ContentSmartz|Da
 # Option:  failregex
 # Notes.:  Regexp to catch known spambots and software alike. Please verify
 #          that it is your intent to block IPs which were driven by
-#          abovementioned bots.
+#          above mentioned bots.
 # Values:  TEXT
 #
 failregex = ^<HOST> -.*"(GET|POST).*HTTP.*"(?:%(badbots)s|%(badbotscustom)s)"$

config/jail.conf

@@ -101,6 +101,17 @@ action      = hostsdeny
 ignoreregex = for myuser from
 logpath     = /var/log/sshd.log
 
+# Here we use blackhole routes for not requiring any additional kernel support
+# to store large volumes of banned IPs
+
+[ssh-route]
+
+enabled = false
+filter = sshd
+action = route
+logpath = /var/log/sshd.log
+maxretry = 5
+
 # Here we use a combination of Netfilter/Iptables and IPsets
 # for storing large volumes of banned IPs
 #
@@ -230,7 +241,7 @@ logpath = /var/log/lighttpd/error.log
 maxretry = 2
 
 # Same as above for mod_auth
-# It catches wrong authentifications
+# It catches wrong authentications
 
 [lighttpd-auth]
 

doc/run-rootless.txt

@@ -1,7 +1,7 @@
-Fail2ban normally requires root priviledges to insert iptables rules
+Fail2ban normally requires root privileges to insert iptables rules
 through calls to /sbin/iptables and also to read the logfiles.
-Fail2ban can run as an unpriviledged user provided that those two
+Fail2ban can run as an unprivileged user provided that those two
-capabilites are preserved. The idea is to run fail2ban as a normal
+capabilities are preserved. The idea is to run fail2ban as a normal
 user (e.g. fail2ban) who belongs to a group which is allowed to read
 logfiles. The user should also be allowed to write to
 /proc/net/xt_recent/fail2ban-<name> (name is specified in the iptables
@@ -20,14 +20,14 @@ Another way to use xt_recent is by inserting the rules by writing to
 action. Files in /proc/net/xt_recent/ are protected by normal
 filesystem rules, so can be chown'ed and chmod'ed to be writable by a
 certain user. After the necessary iptables rules are inserted (which
-requires root priviledges), blacklisting can be perfomed by an
+requires root privileges), blacklisting can be performed by an
-unpriviledged user.
+unprivileged user.
 
 Using fail2ban with xt_recent allows smarter filtering than normal
 iptables rules with the xt_recent module can provide.
 
 The disadvantage is that fail2ban cannot perform the setup by itself,
-which would require the priviledge to call /sbin/iptables, and it must
+which would require the privilege to call /sbin/iptables, and it must
 be done through other means.
 
 The primary advantage is obvious: it's generally better to run
@@ -46,7 +46,7 @@ some user and thus allow delisting IPs by helper administrators
 without the ability to mess up other iptables rules.
 
 The xt_recent-echo jail can be used under the root user without
-further configuration. To run not as root, futher setup is necessary:
+further configuration. To run not as root, further setup is necessary:
 
 - Create user:
 

fail2ban-client

@@ -62,6 +62,7 @@ class Fail2banClient:
 		self.__conf["verbose"] = 1
 		self.__conf["interactive"] = False
 		self.__conf["socket"] = None
+		self.__conf["pidfile"] = None
 		
 	def dispVersion(self):
 		print "Fail2Ban v" + version
@@ -84,6 +85,7 @@ class Fail2banClient:
 		print "Options:"
 		print "    -c <DIR>                configuration directory"
 		print "    -s <FILE>               socket path"
+		print "    -p <FILE>               pidfile path"
 		print "    -d                      dump configuration. For debugging"
 		print "    -i                      interactive mode"
 		print "    -v                      increase verbosity"
@@ -119,6 +121,8 @@ class Fail2banClient:
 				self.__conf["conf"] = opt[1]
 			elif opt[0] == "-s":
 				self.__conf["socket"] = opt[1]
+			elif opt[0] == "-p":
+				self.__conf["pidfile"] = opt[1]
 			elif opt[0] == "-d":
 				self.__conf["dump"] = True
 			elif opt[0] == "-v":
@@ -181,8 +185,21 @@ class Fail2banClient:
 				# Do not continue if configuration is not 100% valid
 				if not ret:
 					return False
+				# verify that directory for the socket file exists
+				socket_dir = os.path.dirname(self.__conf["socket"])
+				if not os.path.exists(socket_dir):
+					logSys.error(
+						"There is no directory %s to contain the socket file %s."
+						% (socket_dir, self.__conf["socket"]))
+					return False
+				if not os.access(socket_dir, os.W_OK | os.X_OK):
+					logSys.error(
+						"Directory %s exists but not accessible for writing"
+						% (socket_dir,))
+					return False
 				# Start the server
 				self.__startServerAsync(self.__conf["socket"],
+										self.__conf["pidfile"],
 										self.__conf["force"])
 				try:
 					# Wait for the server to start
@@ -191,10 +208,10 @@ class Fail2banClient:
 					self.__processCmd(self.__stream, False)
 					return True
 				except ServerExecutionException:
-					logSys.error("Could not start server. Maybe an old " +
+					logSys.error("Could not start server. Maybe an old "
-								 "socket file is still present. Try to " +
+								 "socket file is still present. Try to "
-								 "remove " + self.__conf["socket"] + ". If " +
+								 "remove " + self.__conf["socket"] + ". If "
-								 "you used fail2ban-client to start the " +
+								 "you used fail2ban-client to start the "
 								 "server, adding the -x option will do it")
 					return False
 		elif len(cmd) == 1 and cmd[0] == "reload":
@@ -231,7 +248,7 @@ class Fail2banClient:
 	#
 	# Start the Fail2ban server in daemon mode.
 	
-	def __startServerAsync(self, socket, force = False):
+	def __startServerAsync(self, socket, pidfile, force = False):
 		# Forks the current process.
 		pid = os.fork()
 		if pid == 0:
@@ -242,22 +259,26 @@ class Fail2banClient:
 			# Set the socket path.
 			args.append("-s")
 			args.append(socket)
+			# Set the pidfile
+			args.append("-p")
+			args.append(pidfile)
 			# Force the execution if needed.
 			if force:
 				args.append("-x")
 			try:
 				# Use the current directory.
 				exe = os.path.abspath(os.path.join(sys.path[0], self.SERVER))
+				logSys.debug("Starting %r with args %r" % (exe, args))
 				os.execv(exe, args)
 			except OSError:
 				try:
 					# Use the PATH env.
+					logSys.warning("Initial start attempt failed.  Starting %r with the same args" % (self.SERVER,))
 					os.execvp(self.SERVER, args)
 				except OSError:
-					print "Could not find %s" % self.SERVER
+					logSys.error("Could not start %s" % self.SERVER)
 					os.exit(-1)
 	
-			
 	def __waitOnServer(self):
 		# Wait for the server to start
 		cnt = 0
@@ -276,7 +297,7 @@ class Fail2banClient:
 					delta = -1
 				elif pos < 2:
 					delta = 1
-			# The server has 30 secondes to start.
+			# The server has 30 seconds to start.
 			if cnt >= 300:
 				if self.__conf["verbose"] > 1:
 					sys.stdout.write('\n')
@@ -297,7 +318,7 @@ class Fail2banClient:
 		
 		# Reads the command line options.
 		try:
-			cmdOpts = 'hc:s:xdviqV'
+			cmdOpts = 'hc:s:p:xdviqV'
 			cmdLongOpts = ['help', 'version']
 			optList, args = getopt.getopt(self.__argv[1:], cmdOpts, cmdLongOpts)
 		except getopt.GetoptError:
@@ -328,9 +349,11 @@ class Fail2banClient:
 		
 		# Set socket path
 		self.__configurator.readEarly()
-		socket = self.__configurator.getEarlyOptions()
+		conf = self.__configurator.getEarlyOptions()
 		if self.__conf["socket"] == None:
-			self.__conf["socket"] = socket["socket"]
+			self.__conf["socket"] = conf["socket"]
+		if self.__conf["pidfile"] == None:
+			self.__conf["pidfile"] = conf["pidfile"]
 		logSys.info("Using socket file " + self.__conf["socket"])
 
 		if self.__conf["dump"]:
@@ -357,7 +380,9 @@ class Fail2banClient:
 						if cmd == "exit" or cmd == "quit":
 							# Exit
 							return True
-						if not cmd == "":
+						if cmd == "help":
+							self.dispUsage()
+						elif not cmd == "":
 							self.__processCommand(shlex.split(cmd))
 			except (EOFError, KeyboardInterrupt):
 				print
@@ -395,7 +420,7 @@ class Fail2banClient:
 class ServerExecutionException(Exception):
 	pass
 
-if __name__ == "__main__":
+if __name__ == "__main__": # pragma: no cover - can't test main
 	client = Fail2banClient()
 	# Exit with correct return value
 	if client.start(sys.argv):

fail2ban-server

@@ -54,6 +54,7 @@ class Fail2banServer:
 		self.__conf["background"] = True
 		self.__conf["force"] = False
 		self.__conf["socket"] = "/var/run/fail2ban/fail2ban.sock"
+		self.__conf["pidfile"] = "/var/run/fail2ban/fail2ban.pid"
 	
 	def dispVersion(self):
 		print "Fail2Ban v" + version
@@ -81,6 +82,7 @@ class Fail2banServer:
 		print "    -b                   start in background"
 		print "    -f                   start in foreground"
 		print "    -s <FILE>            socket path"
+		print "    -p <FILE>            pidfile path"
 		print "    -x                   force execution of the server (remove socket file)"
 		print "    -h, --help           display this help message"
 		print "    -V, --version        print the version"
@@ -97,6 +99,8 @@ class Fail2banServer:
 				self.__conf["background"] = False
 			if opt[0] == "-s":
 				self.__conf["socket"] = opt[1]
+			if opt[0] == "-p":
+				self.__conf["pidfile"] = opt[1]
 			if opt[0] == "-x":
 				self.__conf["force"] = True
 			if opt[0] in ["-h", "--help"]:
@@ -112,7 +116,7 @@ class Fail2banServer:
 		
 		# Reads the command line options.
 		try:
-			cmdOpts = 'bfs:xhV'
+			cmdOpts = 'bfs:p:xhV'
 			cmdLongOpts = ['help', 'version']
 			optList, args = getopt.getopt(self.__argv[1:], cmdOpts, cmdLongOpts)
 		except getopt.GetoptError:
@@ -123,7 +127,9 @@ class Fail2banServer:
 		
 		try:
 			self.__server = Server(self.__conf["background"])
-			self.__server.start(self.__conf["socket"], self.__conf["force"])
+			self.__server.start(self.__conf["socket"],
+								self.__conf["pidfile"],
+								self.__conf["force"])
 			return True
 		except Exception, e:
 			logSys.exception(e)

fail2ban-testcases

@@ -77,10 +77,10 @@ verbosity = {'debug': 3,
 			 'fatal': 0,
 			 None: 1}[opts.log_level]
 
-if opts.log_level is not None:
+if opts.log_level is not None: # pragma: no cover
 	# so we had explicit settings
 	logSys.setLevel(getattr(logging, opts.log_level.upper()))
-else:
+else: # pragma: no cover
 	# suppress the logging but it would leave unittests' progress dots
 	# ticking, unless like with '-l fatal' which would be silent
 	# unless error occurs
@@ -89,9 +89,9 @@ else:
 # Add the default logging handler
 stdout = logging.StreamHandler(sys.stdout)
 # Custom log format for the verbose tests runs
-if verbosity > 1:
+if verbosity > 1: # pragma: no cover
 	stdout.setFormatter(logging.Formatter(' %(asctime)-15s %(thread)s %(message)s'))
-else:
+else: # pragma: no cover
 	# just prefix with the space
 	stdout.setFormatter(logging.Formatter(' %(message)s'))
 logSys.addHandler(stdout)
@@ -99,7 +99,7 @@ logSys.addHandler(stdout)
 #
 # Let know the version
 #
-if not opts.log_level or opts.log_level != 'fatal':
+if not opts.log_level or opts.log_level != 'fatal': # pragma: no cover
 	print "Fail2ban %s test suite. Python %s. Please wait..." \
 	    % (version, str(sys.version).replace('\n', ''))
 
@@ -107,9 +107,9 @@ if not opts.log_level or opts.log_level != 'fatal':
 #
 # Gather the tests
 #
-if not len(regexps):
+if not len(regexps): # pragma: no cover
 	tests = unittest.TestSuite()
-else:
+else: # pragma: no cover
 	import re
 	class FilteredTestSuite(unittest.TestSuite):
 		_regexps = [re.compile(r) for r in regexps]
@@ -124,7 +124,7 @@ else:
 
 # Server
 #tests.addTest(unittest.makeSuite(servertestcase.StartStop))
-#tests.addTest(unittest.makeSuite(servertestcase.Transmitter))
+tests.addTest(unittest.makeSuite(servertestcase.Transmitter))
 tests.addTest(unittest.makeSuite(actiontestcase.ExecuteAction))
 # FailManager
 tests.addTest(unittest.makeSuite(failmanagertestcase.AddFailure))
@@ -160,13 +160,13 @@ filters = [FilterPoll]					  # always available
 try:
 	from server.filtergamin import FilterGamin
 	filters.append(FilterGamin)
-except Exception, e:
+except Exception, e: # pragma: no cover
 	print "I: Skipping gamin backend testing. Got exception '%s'" % e
 
 try:
 	from server.filterpyinotify import FilterPyinotify
 	filters.append(FilterPyinotify)
-except Exception, e:
+except Exception, e: # pragma: no cover
 	print "I: Skipping pyinotify backend testing. Got exception '%s'" % e
 
 for Filter_ in filters:
@@ -190,7 +190,7 @@ try:
 
 	tests_results = testRunner.run(tests)
 
-finally:
+finally: # pragma: no cover
 	# Just for the sake of it reset the TZ
 	# yoh: move all this into setup/teardown methods within tests
 	os.environ.pop('TZ')
@@ -198,5 +198,5 @@ finally:
 		os.environ['TZ'] = old_TZ
 	time.tzset()
 
-if not tests_results.wasSuccessful():
+if not tests_results.wasSuccessful(): # pragma: no cover
 	sys.exit(1)

files/cacti/README

@@ -49,5 +49,5 @@ details.
 
 You should have received a copy of the GNU General Public
 License along with Fail2Ban; if not, write to the Free
-Software Foundation, Inc., 59 Temple Place, Suite 330,
+Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
-Boston, MA  02111-1307  USA
+Boston, MA 02110, USA

man/fail2ban-client.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.36.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.40.10.
-.TH FAIL2BAN-CLIENT "1" "March 2008" "fail2ban-client v0.8.2" "User Commands"
+.TH FAIL2BAN-CLIENT "1" "March 2013" "fail2ban-client v0.8.8" "User Commands"
 .SH NAME
 fail2ban-client \- configure and control the server
 .SH SYNOPSIS
 .B fail2ban-client
 [\fIOPTIONS\fR] \fI<COMMAND>\fR
 .SH DESCRIPTION
-Fail2Ban v0.8.2 reads log file that contains password failure report
+Fail2Ban v0.8.8 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .SH OPTIONS
 .TP
@@ -16,6 +16,9 @@ configuration directory
 \fB\-s\fR <FILE>
 socket path
 .TP
+\fB\-p\fR <FILE>
+pidfile path
+.TP
 \fB\-d\fR
 dump configuration. For debugging
 .TP
@@ -110,7 +113,7 @@ adds <FILE> to the monitoring list
 of <JAIL>
 .TP
 \fBset <JAIL> dellogpath <FILE>\fR
-removes <FILE> to the monitoring
+removes <FILE> from the monitoring
 list of <JAIL>
 .TP
 \fBset <JAIL> addfailregex <REGEX>\fR
@@ -140,6 +143,15 @@ back for <JAIL>
 sets the number of seconds <TIME>
 a host will be banned for <JAIL>
 .TP
+\fBset <JAIL> usedns <VALUE>\fR
+sets the usedns mode for <JAIL>
+.TP
+\fBset <JAIL> banip <IP>\fR
+manually Ban <IP> for <JAIL>
+.TP
+\fBset <JAIL> unbanip <IP>\fR
+manually Unban <IP> in <JAIL>
+.TP
 \fBset <JAIL> maxretry <RETRY>\fR
 sets the number of failures
 <RETRY> before banning the host
@@ -191,14 +203,6 @@ files for <JAIL>
 gets the list of ignored IP
 addresses for <JAIL>
 .TP
-\fBget <JAIL> timeregex\fR
-gets the regular expression used
-for the time detection for <JAIL>
-.TP
-\fBget <JAIL> timepattern\fR
-gets the pattern used for the time
-detection for <JAIL>
-.TP
 \fBget <JAIL> failregex\fR
 gets the list of regular
 expressions which matches the
@@ -218,6 +222,9 @@ will look back for failures for
 gets the time a host is banned for
 <JAIL>
 .TP
+\fBget <JAIL> usedns\fR
+gets the usedns setting for <JAIL>
+.TP
 \fBget <JAIL> maxretry\fR
 gets the number of failures
 allowed for <JAIL>
@@ -245,15 +252,19 @@ action <ACT> for <JAIL>
 \fBget <JAIL> actionunban <ACT>\fR
 gets the unban command for the
 action <ACT> for <JAIL>
+.TP
+\fBget <JAIL> cinfo <ACT> <KEY>\fR
+gets the value for <KEY> for the
+action <ACT> for <JAIL>
 .SH FILES
 \fI/etc/fail2ban/*\fR
 .SH AUTHOR
 Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
 Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>.
 .SH "REPORTING BUGS"
-Report bugs on https://github.com/fail2ban/fail2ban/issues
+Report bugs to https://github.com/fail2ban/fail2ban/issues
 .SH COPYRIGHT
-Copyright \(co 2004-2008 Cyril Jaquier
+Copyright \(co 2004\-2008 Cyril Jaquier, 2008\- Fail2Ban Contributors
 .br
 Copyright of modifications held by their respective authors.
 Licensed under the GNU General Public License v2 (GPL).

man/fail2ban-regex.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.36.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.40.10.
-.TH FAIL2BAN-REGEX "1" "March 2008" "fail2ban-regex v0.8.2" "User Commands"
+.TH FAIL2BAN-REGEX "1" "March 2013" "fail2ban-regex v0.8.8" "User Commands"
 .SH NAME
 fail2ban-regex \- test Fail2ban "failregex" option
 .SH SYNOPSIS
 .B fail2ban-regex
 [\fIOPTIONS\fR] \fI<LOG> <REGEX> \fR[\fIIGNOREREGEX\fR]
 .SH DESCRIPTION
-Fail2Ban v0.8.2 reads log file that contains password failure report
+Fail2Ban v0.8.8 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .PP
 This tools can test regular expressions for "fail2ban".
@@ -17,6 +17,9 @@ display this help message
 .TP
 \fB\-V\fR, \fB\-\-version\fR
 print the version
+.TP
+\fB\-v\fR, \fB\-\-verbose\fR
+verbose output
 .SH LOG
 .TP
 \fBstring\fR
@@ -42,9 +45,9 @@ path to a filter file (filter.d/sshd.conf)
 Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
 Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>.
 .SH "REPORTING BUGS"
-Report bugs on https://github.com/fail2ban/fail2ban/issues
+Report bugs to https://github.com/fail2ban/fail2ban/issues
 .SH COPYRIGHT
-Copyright \(co 2004-2008 Cyril Jaquier
+Copyright \(co 2004\-2008 Cyril Jaquier
 .br
 Copyright of modifications held by their respective authors.
 Licensed under the GNU General Public License v2 (GPL).

man/fail2ban-server.1

@@ -1,12 +1,12 @@
-.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.36.
+.\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.40.10.
-.TH FAIL2BAN-SERVER "1" "March 2008" "fail2ban-server v0.8.2" "User Commands"
+.TH FAIL2BAN-SERVER "1" "March 2013" "fail2ban-server v0.8.8" "User Commands"
 .SH NAME
 fail2ban-server \- start the server
 .SH SYNOPSIS
 .B fail2ban-server
 [\fIOPTIONS\fR]
 .SH DESCRIPTION
-Fail2Ban v0.8.2 reads log file that contains password failure report
+Fail2Ban v0.8.8 reads log file that contains password failure report
 and bans the corresponding IP addresses using firewall rules.
 .PP
 Only use this command for debugging purpose. Start the server with
@@ -23,6 +23,9 @@ start in foreground
 \fB\-s\fR <FILE>
 socket path
 .TP
+\fB\-p\fR <FILE>
+pidfile path
+.TP
 \fB\-x\fR
 force execution of the server (remove socket file)
 .TP
@@ -35,9 +38,9 @@ print the version
 Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
 Many contributions by Yaroslav O. Halchenko <debian@onerussian.com>.
 .SH "REPORTING BUGS"
-Report bugs on https://github.com/fail2ban/fail2ban/issues
+Report bugs to https://github.com/fail2ban/fail2ban/issues
 .SH COPYRIGHT
-Copyright \(co 2004-2008 Cyril Jaquier
+Copyright \(co 2004\-2008 Cyril Jaquier, 2008\- Fail2Ban Contributors
 .br
 Copyright of modifications held by their respective authors.
 Licensed under the GNU General Public License v2 (GPL).

server/action.py

@@ -277,8 +277,8 @@ class Action:
 	# Executes a command with preliminary checks and substitutions.
 	#
 	# Before executing any commands, executes the "check" command first
-	# in order to check if prerequirements are met. If this check fails,
+	# in order to check if pre-requirements are met. If this check fails,
-	# it tries to restore a sane environnement before executing the real
+	# it tries to restore a sane environment before executing the real
 	# command.
 	# Replaces "aInfo" and "cInfo" in the query too.
 	#

server/actions.py

@@ -77,7 +77,8 @@ class Actions(JailThread):
 		for action in self.__actions:
 			if action.getName() == name:
 				self.__actions.remove(action)
-				break
+				return
+		raise KeyError("Invalid Action name: %s" % name)
 	
 	##
 	# Returns an action.
@@ -91,7 +92,7 @@ class Actions(JailThread):
 		for action in self.__actions:
 			if action.getName() == name:
 				return action
-		raise KeyError
+		raise KeyError("Invalid Action name")
 	
 	##
 	# Returns the last defined action.
@@ -131,7 +132,7 @@ class Actions(JailThread):
 			# Unban the IP.
 			self.__unBan(ticket)
 			return ip
-		return 'None'
+		raise ValueError("IP %s is not banned" % ip)
 
 	##
 	# Main loop.

server/asyncserver.py

@@ -142,7 +142,7 @@ class AsyncServer(asyncore.dispatcher):
 		if sys.version_info >= (2, 6): # if python 2.6 or greater...
 			logSys.debug("Detected Python 2.6 or greater. asyncore.loop() not using poll")
 			asyncore.loop(use_poll = False) # fixes the "Unexpected communication problem" issue on Python 2.6 and 3.0
-		else:
+		else: # pragma: no cover
 			logSys.debug("NOT Python 2.6/3.* - asyncore.loop() using poll")
 			asyncore.loop(use_poll = True)
 	

server/filter.py

@@ -44,7 +44,7 @@ logSys = logging.getLogger("fail2ban.filter")
 # Log reader class.
 #
 # This class reads a log file and detects login failures or anything else
-# that matches a given regular expression. This class is instanciated by
+# that matches a given regular expression. This class is instantiated by
 # a Jail object.
 
 class Filter(JailThread):
@@ -99,6 +99,7 @@ class Filter(JailThread):
 			self.__failRegex.append(regex)
 		except RegexException, e:
 			logSys.error(e)
+			raise e
 
 
 	def delFailRegex(self, index):
@@ -123,7 +124,7 @@ class Filter(JailThread):
 	# Add the regular expression which matches the failure.
 	#
 	# The regular expression can also match any other pattern than failures
-	# and thus can be used for many purporse.
+	# and thus can be used for many purpose.
 	# @param value the regular expression
 
 	def addIgnoreRegex(self, value):
@@ -132,6 +133,7 @@ class Filter(JailThread):
 			self.__ignoreRegex.append(regex)
 		except RegexException, e:
 			logSys.error(e)
+			raise e 
 
 	def delIgnoreRegex(self, index):
 		try:
@@ -234,7 +236,7 @@ class Filter(JailThread):
 	# file has been modified and looks for failures.
 	# @return True when the thread exits nicely
 
-	def run(self):
+	def run(self): # pragma: no cover
 		raise Exception("run() is abstract")
 
 	##
@@ -249,7 +251,7 @@ class Filter(JailThread):
 			self.failManager.addFailure(FailTicket(ip, unixTime))
 
 		# Perform the banning of the IP now.
-		try:
+		try: # pragma: no branch - exception is the only way out
 			while True:
 				ticket = self.failManager.toBan()
 				self.jail.putFailTicket(ticket)
@@ -401,7 +403,7 @@ class Filter(JailThread):
 								failList.append([ip, date])
 							# We matched a regex, it is enough to stop.
 							break
-					except RegexException, e:
+					except RegexException, e: # pragma: no cover - unsure if reachable
 						logSys.error(e)
 		return failList
 
@@ -442,7 +444,7 @@ class FileFilter(Filter):
 
 	def _addLogPath(self, path):
 		# nothing to do by default
-		# to be overriden by backends
+		# to be overridden by backends
 		pass
 
 
@@ -461,7 +463,7 @@ class FileFilter(Filter):
 
 	def _delLogPath(self, path):
 		# nothing to do by default
-		# to be overriden by backends
+		# to be overridden by backends
 		pass
 
 	##
@@ -505,10 +507,19 @@ class FileFilter(Filter):
 		# Try to open log file.
 		try:
 			container.open()
-		except Exception, e:
+		# see http://python.org/dev/peps/pep-3151/
+		except IOError, e:
 			logSys.error("Unable to open %s" % filename)
 			logSys.exception(e)
 			return False
+		except OSError, e: # pragma: no cover - requires race condition to tigger this
+			logSys.error("Error opening %s" % filename)
+			logSys.exception(e)
+			return False
+		except OSError, e: # pragma: no cover - Requires implemention error in FileContainer to generate
+			logSys.error("Internal errror in FileContainer open method - please report as a bug to https://github.com/fail2ban/fail2ban/issues")
+			logSys.exception(e)
+			return False
 
 		while True:
 			line = container.readline()
@@ -535,7 +546,7 @@ class FileFilter(Filter):
 try:
 	import hashlib
 	md5sum = hashlib.md5
-except ImportError:
+except ImportError: # pragma: no cover
 	# hashlib was introduced in Python 2.5.  For compatibility with those
 	# elderly Pythons, import from md5
 	import md5
@@ -578,7 +589,7 @@ class FileContainer:
 		stats = os.fstat(self.__handler.fileno())
 		# Compare hash and inode
 		if self.__hash != myHash or self.__ino != stats.st_ino:
-			logSys.info("Log rotation detected for %s" % self.__filename)
+			logSys.debug("Log rotation detected for %s" % self.__filename)
 			self.__hash = myHash
 			self.__ino = stats.st_ino
 			self.__pos = 0

server/filterpoll.py

@@ -39,7 +39,7 @@ logSys = logging.getLogger("fail2ban.filter")
 # Log reader class.
 #
 # This class reads a log file and detects login failures or anything else
-# that matches a given regular expression. This class is instanciated by
+# that matches a given regular expression. This class is instantiated by
 # a Jail object.
 
 class FilterPoll(FileFilter):

server/server.py

@@ -40,8 +40,6 @@ logSys = logging.getLogger("fail2ban.server")
 
 class Server:
 	
-	PID_FILE = "/var/run/fail2ban/fail2ban.pid"
-
 	def __init__(self, daemon = False):
 		self.__loggingLock = Lock()
 		self.__lock = RLock()
@@ -59,7 +57,7 @@ class Server:
 		logSys.debug("Caught signal %d. Exiting" % signum)
 		self.quit()
 	
-	def start(self, sock, force = False):
+	def start(self, sock, pidfile, force = False):
 		logSys.info("Starting Fail2ban v" + version.version)
 		
 		# Install signal handlers
@@ -79,8 +77,8 @@ class Server:
 		
 		# Creates a PID file.
 		try:
-			logSys.debug("Creating PID file %s" % Server.PID_FILE)
+			logSys.debug("Creating PID file %s" % pidfile)
-			pidFile = open(Server.PID_FILE, 'w')
+			pidFile = open(pidfile, 'w')
 			pidFile.write("%s\n" % os.getpid())
 			pidFile.close()
 		except IOError, e:
@@ -94,17 +92,11 @@ class Server:
 			logSys.error("Could not start server: %s", e)
 		# Removes the PID file.
 		try:
-			logSys.debug("Remove PID file %s" % Server.PID_FILE)
+			logSys.debug("Remove PID file %s" % pidfile)
-			os.remove(Server.PID_FILE)
+			os.remove(pidfile)
 		except OSError, e:
 			logSys.error("Unable to remove PID file: %s" % e)
 		logSys.info("Exiting Fail2ban")
-		# Shutdowns the logging.
-		try:
-			self.__loggingLock.acquire()
-			logging.shutdown()
-		finally:
-			self.__loggingLock.release()
 	
 	def quit(self):
 		# Stop communication first because if jail's unban action
@@ -114,9 +106,18 @@ class Server:
 		# are exiting)
 		# See https://github.com/fail2ban/fail2ban/issues/7
 		self.__asyncServer.stop()
+
 		# Now stop all the jails
 		self.stopAllJail()
 
+		# Only now shutdown the logging.
+		try:
+			self.__loggingLock.acquire()
+			logging.shutdown()
+		finally:
+			self.__loggingLock.release()
+
+	
 	def addJail(self, name, backend):
 		self.__jails.add(name, backend)
 		
@@ -375,11 +376,20 @@ class Server:
 					logSys.error("Unable to log to " + target)
 					logSys.info("Logging to previous target " + self.__logTarget)
 					return False
-			# Removes previous handlers
+			# Removes previous handlers -- in reverse order since removeHandler
-			for handler in logging.getLogger("fail2ban").handlers:
+			# alter the list in-place and that can confuses the iterable
-				# Closes the handler.
+			for handler in logging.getLogger("fail2ban").handlers[::-1]:
+				# Remove the handler.
 				logging.getLogger("fail2ban").removeHandler(handler)
+				# And try to close -- it might be closed already
+				try:
+					handler.flush()
 					handler.close()
+				except ValueError:
+					if sys.version_info >= (2,6):
+						raise
+					# is known to be thrown after logging was shutdown once
+					# with older Pythons -- seems to be safe to ignore there
 			# tell the handler to use this format
 			hdlr.setFormatter(formatter)
 			logging.getLogger("fail2ban").addHandler(hdlr)

server/transmitter.py

@@ -112,14 +112,18 @@ class Transmitter:
 			return self.__server.getLogLevel()
 		elif name == "logtarget":
 			value = command[1]
-			self.__server.setLogTarget(value)
+			if self.__server.setLogTarget(value):
 				return self.__server.getLogTarget()
+			else:
+				raise Exception("Failed to change log target")
 		# Jail
 		elif command[1] == "idle":
 			if command[2] == "on":
 				self.__server.setIdleJail(name, True)
 			elif command[2] == "off":
 				self.__server.setIdleJail(name, False)
+			else:
+				raise Exception("Invalid idle option, must be 'yes' or 'no'")
 			return self.__server.getIdleJail(name)
 		# Filter
 		elif command[1] == "addignoreip":
@@ -187,12 +191,13 @@ class Transmitter:
 			self.__server.addAction(name, value)
 			return self.__server.getLastAction(name).getName()
 		elif command[1] == "delaction":
+			value = command[2]
 			self.__server.delAction(name, value)
 			return None
 		elif command[1] == "setcinfo":
 			act = command[2]
 			key = command[3]
-			value = command[4]
+			value = " ".join(command[4:])
 			self.__server.setCInfo(name, act, key, value)
 			return self.__server.getCInfo(name, act, key)
 		elif command[1] == "delcinfo":
@@ -202,27 +207,27 @@ class Transmitter:
 			return None
 		elif command[1] == "actionstart":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionStart(name, act, value)
 			return self.__server.getActionStart(name, act)
 		elif command[1] == "actionstop":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionStop(name, act, value)
 			return self.__server.getActionStop(name, act)
 		elif command[1] == "actioncheck":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionCheck(name, act, value)
 			return self.__server.getActionCheck(name, act)
 		elif command[1] == "actionban":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionBan(name, act, value)
 			return self.__server.getActionBan(name, act)
 		elif command[1] == "actionunban":
 			act = command[2]
-			value = command[3]
+			value = " ".join(command[3:])
 			self.__server.setActionUnban(name, act, value)
 			return self.__server.getActionUnban(name, act)
 		raise Exception("Invalid command (no set action or not yet implemented)")
@@ -271,12 +276,16 @@ class Transmitter:
 		elif command[1] == "actionunban":
 			act = command[2]
 			return self.__server.getActionUnban(name, act)
+		elif command[1] == "cinfo":
+			act = command[2]
+			key = command[3]
+			return self.__server.getCInfo(name, act, key)
 		raise Exception("Invalid command (no get action or not yet implemented)")
 	
 	def status(self, command):
 		if len(command) == 0:
 			return self.__server.status()
-		else:
+		elif len(command) == 1:
 			name = command[0]
 			return self.__server.statusJail(name)
 		raise Exception("Invalid command (no status)")

testcases/filtertestcase.py

@@ -99,11 +99,11 @@ def _copy_lines_between_files(fin, fout, n=None, skip=0, mode='a', terminal_line
 
 	Returns open fout
 	"""
-	if sys.version_info[:2] <= (2,4):
+	if sys.version_info[:2] <= (2,4): # pragma: no cover
 		# on old Python st_mtime is int, so we should give at least 1 sec so
 		# polling filter could detect the change
 		time.sleep(1)
-	if isinstance(fin, str):
+	if isinstance(fin, str): # pragma: no branch - only used with str in test cases
 		fin = open(fin, 'r')
 	if isinstance(fout, str):
 		fout = open(fout, mode)
@@ -353,7 +353,7 @@ def get_monitor_failures_testcase(Filter_):
 			_killfile(self.file, self.name)
 			pass
 
-		def __str__(self):
+		def __str__(self): # pragma: no cover - will only show up if unexpected exception is thrown
 			return "MonitorFailures%s(%s)" \
 			  % (Filter_, hasattr(self, 'name') and self.name or 'tempfile')
 

testcases/servertestcase.py

@@ -27,8 +27,9 @@ __date__ = "$Date$"
 __copyright__ = "Copyright (c) 2004 Cyril Jaquier"
 __license__ = "GPL"
 
-import unittest, socket, time
+import unittest, socket, time, tempfile, os
 from server.server import Server
+from common.exceptions import UnknownJailException
 
 class StartStop(unittest.TestCase):
 
@@ -55,76 +56,430 @@ class Transmitter(unittest.TestCase):
 	def setUp(self):
 		"""Call before every test case."""
 		self.__server = Server()
+		self.__transm = self.__server._Server__transm
+		self.__server.setLogTarget("/dev/null")
 		self.__server.setLogLevel(0)
-		self.__server.start(False)
+		sock_fd, sock_name = tempfile.mkstemp('fail2ban.sock', 'transmitter')
+		os.close(sock_fd)
+		pidfile_fd, pidfile_name = tempfile.mkstemp(
+			'fail2ban.pid', 'transmitter')
+		os.close(pidfile_fd)
+		self.__server.start(sock_name, pidfile_name, force=False)
+		self.jailName = "TestJail1"
+		self.__server.addJail(self.jailName, "auto")
 
 	def tearDown(self):
 		"""Call after every test case."""
 		self.__server.quit()
 
-	def testSetActionOK(self):
+	def setGetTest(self, cmd, inValue, outValue=None, jail=None):
-		name = "TestCase"
+		setCmd = ["set", cmd, inValue]
-		cmdList = [["add", name],
+		getCmd = ["get", cmd]
-				   ["set", name, "actionstart", "Action Start"],
+		if jail is not None:
-				   ["set", name, "actionstop", "Action Stop"],
+			setCmd.insert(1, jail)
-				   ["set", name, "actioncheck", "Action Check"],
+			getCmd.insert(1, jail)
-				   ["set", name, "actionban", "Action Ban"],
+		if outValue is None:
-				   ["set", name, "actionunban", "Action Unban"],
+			outValue = inValue
-				   ["quit"]]
 
-		outList = [(0, name),
+		self.assertEqual(self.__transm.proceed(setCmd), (0, outValue))
-				   (0, 'Action Start'),
+		self.assertEqual(self.__transm.proceed(getCmd), (0, outValue))
-				   (0, 'Action Stop'),
-				   (0, 'Action Check'),
-				   (0, 'Action Ban'),
-				   (0, 'Action Unban'),
-				   (0, None)]
 
-		cnt = 0
+	def setGetTestNOK(self, cmd, inValue, jail=None):
-		for cmd in cmdList:
+		setCmd = ["set", cmd, inValue]
-			self.assertEqual(self.__server.transm.proceed(cmd), outList[cnt])
+		getCmd = ["get", cmd]
-			cnt += 1
+		if jail is not None:
+			setCmd.insert(1, jail)
+			getCmd.insert(1, jail)
 
-	def testSetActionNOK(self):
+		# Get initial value before trying invalid value
-		name = "TestCase"
+		initValue = self.__transm.proceed(getCmd)[1]
-		cmdList = [["addd", name],
+		self.assertEqual(self.__transm.proceed(setCmd)[0], 1)
-				   ["set", name, "test"],
+		# Check after failed set that value is same as previous
-				   ["prout prout", "Stop"],
+		self.assertEqual(self.__transm.proceed(getCmd), (0, initValue))
-				   ["fail2ban", "sucks"],
-				   ["set"],
-				   ["_/&%", "@*+%&"],
-				   [" quit"]]
 
-		outList = [1,
+	def jailAddDelTest(self, cmd, values, jail):
-				   1,
+		cmdAdd = "add" + cmd
-				   1,
+		cmdDel = "del" + cmd
-				   1,
-				   1,
-				   1,
-				   1]
 
-		cnt = 0
+		self.assertEqual(
-		for cmd in cmdList:
+			self.__transm.proceed(["get", jail, cmd]), (0, []))
-			msg = self.__server.transm.proceed(cmd)
+		for n, value in enumerate(values):
-			self.assertEqual(msg[0], outList[cnt])
+			self.assertEqual(
-			cnt += 1
+				self.__transm.proceed(["set", jail, cmdAdd, value]),
+				(0, values[:n+1]))
+			self.assertEqual(
+				self.__transm.proceed(["get", jail, cmd]),
+				(0, values[:n+1]))
+		for n, value in enumerate(values):
+			self.assertEqual(
+				self.__transm.proceed(["set", jail, cmdDel, value]),
+				(0, values[n+1:]))
+			self.assertEqual(
+				self.__transm.proceed(["get", jail, cmd]),
+				(0, values[n+1:]))
 
-	def testJail(self):
+	def jailAddDelRegexTest(self, cmd, inValues, outValues, jail):
-		name = "TestCase"
+		cmdAdd = "add" + cmd
-		cmdList = [["add", name],
+		cmdDel = "del" + cmd
-				   ["set", name, "logpath", "testcases/files/testcase01.log"],
-				   ["set", name, "timeregex", "\S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}"],
-				   ["set", name, "timepattern", "%b %d %H:%M:%S"],
-				   ["set", name, "failregex", "Authentication failure"],
-				   ["start", name],
-				   ["stop", name],
-				   ["quit"]]
 
-		for cmd in cmdList:
+		if outValues is None:
-			self.__server.transm.proceed(cmd)
+			outValues = inValues
-			if cmd == ["start", name]:
-				time.sleep(2)
-				jail = self.__server.jails[name]
-				self.assertEqual(jail.getFilter().failManager.size(), 0)
-				self.assertEqual(jail.getAction().banManager.size(), 2)
 
+		self.assertEqual(
+			self.__transm.proceed(["get", jail, cmd]), (0, []))
+		for n, value in enumerate(inValues):
+			self.assertEqual(
+				self.__transm.proceed(["set", jail, cmdAdd, value]),
+				(0, outValues[:n+1]))
+			self.assertEqual(
+				self.__transm.proceed(["get", jail, cmd]),
+				(0, outValues[:n+1]))
+		for n, value in enumerate(inValues):
+			self.assertEqual(
+				self.__transm.proceed(["set", jail, cmdDel, 0]), # First item
+				(0, outValues[n+1:]))
+			self.assertEqual(
+				self.__transm.proceed(["get", jail, cmd]),
+				(0, outValues[n+1:]))
+
+	def testStopServer(self):
+		self.assertEqual(self.__transm.proceed(["stop"]), (0, None))
+
+	def testPing(self):
+		self.assertEqual(self.__transm.proceed(["ping"]), (0, "pong"))
+
+	def testSleep(self):
+		t0 = time.time()
+		self.assertEqual(self.__transm.proceed(["sleep", "1"]), (0, None))
+		t1 = time.time()
+		# Approx 1 second delay
+		self.assertAlmostEqual(t1 - t0, 1, places=2)
+
+	def testLogTarget(self):
+		logTargets = []
+		for _ in xrange(3):
+			tmpFile = tempfile.mkstemp("fail2ban", "transmitter")
+			logTargets.append(tmpFile[1])
+			os.close(tmpFile[0])
+		for logTarget in logTargets:
+			self.setGetTest("logtarget", logTarget)
+
+		# If path is invalid, do not change logtarget
+		value = "/this/path/should/not/exist"
+		self.setGetTestNOK("logtarget", value)
+
+		self.__transm.proceed(["set", "/dev/null"])
+		for logTarget in logTargets:
+			os.remove(logTarget)
+
+	def testLogLevel(self):
+		self.setGetTest("loglevel", "4", 4)
+		self.setGetTest("loglevel", "2", 2)
+		self.setGetTest("loglevel", "-1", -1)
+		self.setGetTestNOK("loglevel", "Bird")
+
+	def testAddJail(self):
+		jail2 = "TestJail2"
+		jail3 = "TestJail3"
+		jail4 = "TestJail4"
+		self.assertEqual(
+			self.__transm.proceed(["add", jail2, "polling"]), (0, jail2))
+		self.assertEqual(self.__transm.proceed(["add", jail3]), (0, jail3))
+		self.assertEqual(
+			self.__transm.proceed(["add", jail4, "invalid backend"])[0], 1)
+		self.assertEqual(
+			self.__transm.proceed(["add", jail4, "auto"]), (0, jail4))
+		# Duplicate Jail
+		self.assertEqual(
+			self.__transm.proceed(["add", self.jailName, "polling"])[0], 1)
+		# All name is reserved
+		self.assertEqual(
+			self.__transm.proceed(["add", "all", "polling"])[0], 1)
+
+	def testStartStopJail(self):
+		self.assertEqual(
+			self.__transm.proceed(["start", self.jailName]), (0, None))
+		time.sleep(1)
+		self.assertEqual(
+			self.__transm.proceed(["stop", self.jailName]), (0, None))
+		self.assertRaises(
+			UnknownJailException, self.__server.isAlive, self.jailName)
+
+	def testStartStopAllJail(self):
+		self.__server.addJail("TestJail2", "auto")
+		self.assertEqual(
+			self.__transm.proceed(["start", self.jailName]), (0, None))
+		self.assertEqual(
+			self.__transm.proceed(["start", "TestJail2"]), (0, None))
+		self.assertEqual(self.__transm.proceed(["stop", "all"]), (0, None))
+		time.sleep(1)
+		self.assertRaises(
+			UnknownJailException, self.__server.isAlive, self.jailName)
+		self.assertRaises(
+			UnknownJailException, self.__server.isAlive, "TestJail2")
+
+	def testJailIdle(self):
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "idle", "on"]),
+			(0, True))
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "idle", "off"]),
+			(0, False))
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "idle", "CAT"])[0],
+			1)
+
+	def testJailFindTime(self):
+		self.setGetTest("findtime", "120", 120, jail=self.jailName)
+		self.setGetTest("findtime", "60", 60, jail=self.jailName)
+		self.setGetTest("findtime", "-60", -60, jail=self.jailName)
+		self.setGetTestNOK("findtime", "Dog", jail=self.jailName)
+
+	def testJailBanTime(self):
+		self.setGetTest("bantime", "600", 600, jail=self.jailName)
+		self.setGetTest("bantime", "50", 50, jail=self.jailName)
+		self.setGetTest("bantime", "-50", -50, jail=self.jailName)
+		self.setGetTestNOK("bantime", "Cat", jail=self.jailName)
+
+	def testJailUseDNS(self):
+		self.setGetTest("usedns", "yes", jail=self.jailName)
+		self.setGetTest("usedns", "warn", jail=self.jailName)
+		self.setGetTest("usedns", "no", jail=self.jailName)
+
+		# Safe default should be "no"
+		value = "Fish"
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "usedns", value]),
+			(0, "no"))
+
+	def testJailBanIP(self):
+		self.__server.startJail(self.jailName) # Jail must be started
+
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "banip", "127.0.0.1"]),
+			(0, "127.0.0.1"))
+		time.sleep(1) # Give chance to ban
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "banip", "Badger"]),
+			(0, "Badger")) #NOTE: Is IP address validated? Is DNS Lookup done?
+		time.sleep(1) # Give chance to ban
+		# Unban IP
+		self.assertEqual(
+			self.__transm.proceed(
+				["set", self.jailName, "unbanip", "127.0.0.1"]),
+			(0, "127.0.0.1"))
+		# Unban IP which isn't banned
+		self.assertEqual(
+			self.__transm.proceed(
+				["set", self.jailName, "unbanip", "192.168.1.1"])[0],1)
+
+	def testJailMaxRetry(self):
+		self.setGetTest("maxretry", "5", 5, jail=self.jailName)
+		self.setGetTest("maxretry", "2", 2, jail=self.jailName)
+		self.setGetTest("maxretry", "-2", -2, jail=self.jailName)
+		self.setGetTestNOK("maxretry", "Duck", jail=self.jailName)
+
+	def testJailLogPath(self):
+		self.jailAddDelTest(
+			"logpath",
+			[
+				"testcases/files/testcase01.log",
+				"testcases/files/testcase02.log",
+				"testcases/files/testcase03.log",
+			],
+			self.jailName
+		)
+		# Try duplicates
+		value = "testcases/files/testcase04.log"
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "addlogpath", value]),
+			(0, [value]))
+		# Will silently ignore duplicate
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "addlogpath", value]),
+			(0, [value]))
+		self.assertEqual(
+			self.__transm.proceed(["get", self.jailName, "logpath"]),
+			(0, [value]))
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "dellogpath", value]),
+			(0, []))
+
+		# Invalid file
+		value = "this_file_shouldn't_exist"
+		result = self.__transm.proceed(
+			["set", self.jailName, "addlogpath", value])
+		self.assertTrue(isinstance(result[1], IOError))
+
+	def testJailIgnoreIP(self):
+		self.jailAddDelTest(
+			"ignoreip",
+			[
+				"127.0.0.1",
+				"192.168.1.1",
+				"8.8.8.8",
+			],
+			self.jailName
+		)
+
+		# Try duplicates
+		value = "127.0.0.1"
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "addignoreip", value]),
+			(0, [value]))
+		# Will allow duplicate
+		#NOTE: Should duplicates be allowed, or silent ignore like logpath?
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "addignoreip", value]),
+			(0, [value, value]))
+		self.assertEqual(
+			self.__transm.proceed(["get", self.jailName, "ignoreip"]),
+			(0, [value, value]))
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "delignoreip", value]),
+			(0, [value]))
+
+	def testJailRegex(self):
+		self.jailAddDelRegexTest("failregex",
+			[
+				"user john at <HOST>",
+				"Admin user login from <HOST>",
+				"failed attempt from <HOST> again",
+			],
+			[
+				"user john at (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)",
+				"Admin user login from (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)",
+				"failed attempt from (?:::f{4,6}:)?(?P<host>[\w\-.^_]+) again",
+			],
+			self.jailName
+		)
+
+		self.assertEqual(
+			self.__transm.proceed(
+				["set", self.jailName, "addfailregex", "No host regex"])[0],
+			1)
+		self.assertEqual(
+			self.__transm.proceed(
+				["set", self.jailName, "addfailregex", 654])[0],
+			1)
+
+	def testJailIgnoreRegex(self):
+		self.jailAddDelRegexTest("ignoreregex",
+			[
+				"user john",
+				"Admin user login from <HOST>",
+				"Dont match me!",
+			],
+			[
+				"user john",
+				"Admin user login from (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)",
+				"Dont match me!",
+			],
+			self.jailName
+		)
+
+		self.assertEqual(
+			self.__transm.proceed(
+				["set", self.jailName, "addignoreregex", "Invalid [regex"])[0],
+			1)
+		self.assertEqual(
+			self.__transm.proceed(
+				["set", self.jailName, "addignoreregex", 50])[0],
+			1)
+
+	def testStatus(self):
+		jails = [self.jailName]
+		self.assertEqual(self.__transm.proceed(["status"]),
+			(0, [('Number of jail', len(jails)), ('Jail list', ", ".join(jails))]))
+		self.__server.addJail("TestJail2", "auto")
+		jails.append("TestJail2")
+		self.assertEqual(self.__transm.proceed(["status"]),
+			(0, [('Number of jail', len(jails)), ('Jail list', ", ".join(jails))]))
+
+	def testJailStatus(self):
+		self.assertEqual(self.__transm.proceed(["status", self.jailName]),
+			(0,
+				[
+					('filter', [
+						('Currently failed', 0),
+						('Total failed', 0),
+						('File list', [])]
+					),
+					('action', [
+						('Currently banned', 0),
+						('Total banned', 0),
+						('IP list', [])]
+					)
+				]
+			)
+		)
+
+	def testAction(self):
+		action = "TestCaseAction"
+		cmdList = [
+			"actionstart",
+			"actionstop",
+			"actioncheck",
+			"actionban",
+			"actionunban",
+		]
+		cmdValueList = [
+			"Action Start",
+			"Action Stop",
+			"Action Check",
+			"Action Ban",
+			"Action Unban",
+		]
+
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "addaction", action]),
+			(0, action))
+		self.assertEqual(
+			self.__transm.proceed(["get", self.jailName, "addaction", action]),
+			(0, action))
+		for cmd, value in zip(cmdList, cmdValueList):
+			self.assertEqual(
+				self.__transm.proceed(
+					["set", self.jailName, cmd, action, value]),
+				(0, value))
+		for cmd, value in zip(cmdList, cmdValueList):
+			self.assertEqual(
+				self.__transm.proceed(["get", self.jailName, cmd, action]),
+				(0, value))
+		self.assertEqual(
+			self.__transm.proceed(
+				["set", self.jailName, "setcinfo", action, "KEY", "VALUE"]),
+			(0, "VALUE"))
+		self.assertEqual(
+			self.__transm.proceed(
+				["get", self.jailName, "cinfo", action, "KEY"]),
+			(0, "VALUE"))
+		self.assertEqual(
+			self.__transm.proceed(
+				["get", self.jailName, "cinfo", action, "InvalidKey"])[0],
+			1)
+		self.assertEqual(
+			self.__transm.proceed(
+				["set", self.jailName, "delcinfo", action, "KEY"]),
+			(0, None))
+		self.assertEqual(
+			self.__transm.proceed(["set", self.jailName, "delaction", action]),
+			(0, None))
+		self.assertEqual(
+			self.__transm.proceed(
+				["set", self.jailName, "delaction", "Doesn't exist"])[0],1)
+
+	def testNOK(self):
+		self.assertEqual(self.__transm.proceed(["INVALID", "COMMAND"])[0],1)
+
+	def testSetNOK(self):
+		self.assertEqual(
+			self.__transm.proceed(["set", "INVALID", "COMMAND"])[0],1)
+
+	def testGetNOK(self):
+		self.assertEqual(
+			self.__transm.proceed(["get", "INVALID", "COMMAND"])[0],1)
+
+	def testStatusNOK(self):
+		self.assertEqual(
+			self.__transm.proceed(["status", "INVALID", "COMMAND"])[0],1)