Character detection heuristics for whois output via optional setting in mail-whois*.conf (Closes #1003)

when set by user,
 - detects character set of whois output (which is undefined by RFC 3912) via heuristics of the file command
 - converts whois data to UTF-8 character set with iconv
 - sends the whois output in UTF-8 character set to mail program
 - avoids that heirloom mailx creates binary attachment for input with unknown character set

ChangeLog

@@ -78,6 +78,11 @@ ver. 0.9.2 (2014/XX/XXX) - wanna-be-released
    * Absorbed DNSUtils.cidr into addr2bin in filter.py, added unittests
    * Added syslogsocket configuration to fail2ban.conf
    * Note in the jail.conf for the recidive jail to increase dbpurgeage (gh-964)
+   * Character detection heuristics for whois output via optional setting in mail-whois*.conf. When set by user,
+     - detects character set of whois output (which is undefined by RFC 3912) via heuristics of the file command
+     - converts whois data to UTF-8 character set with iconv
+     - sends the whois output in UTF-8 character set to mail program
+     - avoids that heirloom mailx creates binary attachment for input with unknown character set
 
 
 ver. 0.9.1 (2014/10/29) - better, faster, stronger

config/action.d/mail-whois-common.conf

@@ -0,0 +1,24 @@
+# Fail2Ban configuration file
+#
+# Common settings for mail actions
+#
+# Users can override the defaults in mail-whois-common.local
+
+[INCLUDES]
+
+# Load customizations if any available
+after = mail-whois-common.local
+
+[DEFAULT]
+#original character set of whois output will be sent to mail program
+_whois = whois <ip> || echo "missing whois program"
+
+# use heuristics to convert charset of whois output to a target
+# character set before sending it to a mail program
+_whois_target_charset = UTF-8
+_whois_convert_charset = whois <ip> |
+                         { c=$(cat) ; cs=$(echo -ne "$c" | file -b --mime-encoding -) ; echo -ne "$c" | iconv -f $cs -t %(_whois_target_charset)s//TRANSLIT - ; }
+
+# choose between _whois and _whois_convert_charset in mail-whois-common.local
+_whois_command = %(_whois)s
+#_whois_command = %(_whois_convert_charset)s

config/action.d/mail-whois-lines.conf

@@ -4,6 +4,10 @@
 # Modified-By: Yaroslav Halchenko to include grepping on IP over log files
 #
 
+[INCLUDES]
+
+before = mail-whois-common.conf
+
 [Definition]
 
 # Option:  actionstart
@@ -40,7 +44,7 @@ actionban = printf %%b "Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
             <failures> attempts against <name>.\n\n
             Here is more information about <ip>:\n
-            `whois <ip> || echo missing whois program`\n\n
+            `%(_whois_command)s`\n\n
             Lines containing IP:<ip> in <logpath>\n
             `grep -E '(^|[^0-9])<ip>([^0-9]|$)' <logpath>`\n\n
             Regards,\n

config/action.d/mail-whois.conf

@@ -4,6 +4,10 @@
 #
 #
 
+[INCLUDES]
+
+before = mail-whois-common.conf
+
 [Definition]
 
 # Option:  actionstart
@@ -40,7 +44,7 @@ actionban = printf %%b "Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
             <failures> attempts against <name>.\n\n
             Here is more information about <ip>:\n
-            `whois <ip> || echo missing whois program`\n
+            `%(_whois_command)s`\n
             Regards,\n
             Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>