From f73746d846587c3d450b18af60620b7f09471feb Mon Sep 17 00:00:00 2001 From: rhardy613 Date: Sun, 31 Jul 2016 13:50:52 -0400 Subject: [PATCH 01/15] Fix ASSP filter to work with current release of ASSP ASSP V1 development stopped at the end of 2014 and it is now deprecated. All users were urged to upgrade to ASSP V2 which is still actively developed. For some reason fail2ban 0.9.5 (and trunk) still have code which only understands ASSP V1 logs. This means the filter ignores brute force attacks against ASSP. --- config/filter.d/assp.conf | 22 +++++++++++----------- fail2ban/tests/files/logs/assp | 33 ++++++++------------------------- 2 files changed, 19 insertions(+), 36 deletions(-) diff --git a/config/filter.d/assp.conf b/config/filter.d/assp.conf index 2aa8958c..0bfba6dc 100644 --- a/config/filter.d/assp.conf +++ b/config/filter.d/assp.conf @@ -1,24 +1,24 @@ -# Fail2Ban filter for Anti-Spam SMTP Proxy Server also known as ASSP +# Fail2Ban filter for Anti-Spam SMTP Proxy Server (ASSP) Version 2.5.1 (or later) # -# Honmepage: http://www.magicvillage.de/~Fritz_Borgstedt/assp/0003D91C-8000001C/ -# ProjektSite: http://sourceforge.net/projects/assp/?source=directory +# Homepage: http://sourceforge.net/projects/assp/ +# ProjectSite: http://sourceforge.net/projects/assp/?source=directory # # [Definition] -__assp_actions = (?:dropping|refusing) - -failregex = ^(:? \[SSL-out\])? max sender authentication errors \(\d{,3}\) exceeded -- %(__assp_actions)s connection - after reply: \d{3} \d{1}\.\d{1}.\d{1} Error: authentication failed: \w+;$ - ^(?: \[SSL-out\])? SSL negotiation with client failed: SSL accept attempt failed with unknown error.*:unknown protocol;$ - ^ Blocking - too much AUTH errors \(\d{,3}\);$ +failregex = \<\S+@\S+\.\S+\> to: \S+@\S+\.\S+ relay attempt blocked for: \S+$ + \[SMTP Error\] 535 5\.7\.8 Error: authentication failed.*$ ignoreregex = # DEV Notes: # -# Examples: Apr-27-13 02:33:09 Blocking 217.194.197.97 - too much AUTH errors (41); -# Dec-29-12 17:10:31 [SSL-out] 200.247.87.82 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; -# Dec-30-12 04:01:47 [SSL-out] 81.82.232.66 max sender authentication errors (5) exceeded +# Examples: +# Jul-29-16 16:49:52 m1-25391-06124 [Worker_1] [TLS-out] [RelayAttempt] 0.0.0.0 to: user@example.org relay attempt blocked for: someone@example.org +# Jul-30-16 16:59:42 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 +# Jul-30-16 00:15:36 m1-52131-09651 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 +# Jul-31-16 06:45:59 [Worker_1] [TLS-in] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: # # Author: Enrico Labedzki (enrico.labedzki@deiwos.de) +# Updated: Robert Hardy (rhardy@webcon.ca) diff --git a/fail2ban/tests/files/logs/assp b/fail2ban/tests/files/logs/assp index 2c658eb9..71c28221 100644 --- a/fail2ban/tests/files/logs/assp +++ b/fail2ban/tests/files/logs/assp @@ -1,25 +1,8 @@ -# failJSON: { "time": "2013-04-07T07:08:36", "match": true , "host": "68.171.223.68" } -Apr-07-13 07:08:36 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; -# failJSON: { "time": "2013-04-07T07:08:36", "match": true , "host": "68.171.223.68" } -Apr-07-13 07:08:36 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; -# failJSON: { "time": "2013-04-07T07:10:37", "match": true , "host": "68.171.223.68" } -Apr-07-13 07:10:37 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; -# failJSON: { "time": "2013-04-07T07:12:37", "match": true , "host": "68.171.223.68" } -Apr-07-13 07:12:37 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; -# failJSON: { "time": "2013-04-07T07:14:36", "match": true , "host": "68.171.223.68" } -Apr-07-13 07:14:36 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; -# failJSON: { "time": "2013-04-27T02:25:09", "match": true , "host": "217.194.197.97" } -Apr-27-13 02:25:09 Blocking 217.194.197.97 - too much AUTH errors (8); -# failJSON: { "time": "2013-04-27T02:25:09", "match": true , "host": "217.194.197.97" } -Apr-27-13 02:25:09 Blocking 217.194.197.97 - too much AUTH errors (9); -# failJSON: { "time": "2013-04-27T02:25:09", "match": true , "host": "217.194.197.97" } -Apr-27-13 02:25:09 Blocking 217.194.197.97 - too much AUTH errors (10); -# failJSON: { "time": "2013-04-27T02:25:10", "match": true , "host": "217.194.197.97" } -Apr-27-13 02:25:10 [SSL-out] 217.194.197.97 max sender authentication errors (5) exceeded -- dropping connection - after reply: 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6; -# failJSON: { "time": "2013-04-27T02:25:10", "match": true , "host": "217.194.197.97" } -Apr-27-13 02:25:10 [SSL-out] 217.194.197.97 max sender authentication errors (5) exceeded -- dropping connection - after reply: 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6; -# failJSON: { "time": "2013-04-27T02:25:10", "match": true , "host": "217.194.197.97" } -Apr-27-13 02:25:10 [SSL-out] 217.194.197.97 max sender authentication errors (5) exceeded -- dropping connection - after reply: 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6; -# failJSON: { "time": "2013-04-27T02:25:11", "match": true , "host": "217.194.197.97" } -Apr-27-13 02:25:11 [SSL-out] 217.194.197.97 max sender authentication errors (5) exceeded -- dropping connection - after reply: 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6; - +# failJSON: { "time": "2016-07-29T16:49:52", "match": true , "host": "0.0.0.0" } +Jul-29-16 16:49:52 m1-25391-06124 [Worker_1] [TLS-out] [RelayAttempt] 0.0.0.0 to: user@example.org relay attempt blocked for: someone@example.org +# failJSON: { "time": "2016-07-30T17:07:25", "match": true , "host": "0.0.0.0" } +Jul-30-16 17:07:25 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 +# failJSON: { "time": "2016-07-30T17:11:05", "match": true , "host": "0.0.0.0" } +Jul-30-16 17:11:05 m1-13060-05386 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 +# failJSON: { "time": "2016-07-31T06:45:59", "match": true , "host": "0.0.0.0" } +Jul-31-16 06:45:59 [Worker_1] [TLS-in] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: From eb6e3c52ae10c1609bc6a585803883265c674b04 Mon Sep 17 00:00:00 2001 From: sebres Date: Mon, 1 Aug 2016 18:04:00 +0200 Subject: [PATCH 02/15] ChangeLog entries for the last fix (cherry pick from 0.10) --- ChangeLog | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/ChangeLog b/ChangeLog index 59b6b1af..3f1052a8 100644 --- a/ChangeLog +++ b/ChangeLog @@ -6,13 +6,31 @@ Fail2Ban: Changelog =================== -ver. 0.9.5 (2016/07/15) - old-not-obsolete +ver. 0.9.6 (2016/XX/XX) - wanna-be-released ----------- 0.9.x line is no longer heavily developed. If you are interested in new features (e.g. IPv6 support), please consider 0.10 branch and its releases. +### Fixes +* Misleading add resp. enable of (already available) jail in database, that + induced a subsequent error: last position of log file will be never retrieved (gh-795) +* Fixed a distribution related bug within testReadStockJailConfForceEnabled + (e.g. test-cases faults on Fedora, see gh-1353) + +### New Features + +### Enhancements +* Several test cases rewritten using new methods assertIn, assertNotIn +* New forward compatibility method assertRaisesRegexp (normally python >= 2.7). + Methods assertIn, assertNotIn, assertRaisesRegexp, assertLogged, assertNotLogged + are test covered now + + +ver. 0.9.5 (2016/07/15) - old-not-obsolete +----------- + ### Fixes * `filter.d/monit.conf` - Extended failregex with new monit "access denied" version (gh-1355) From c0994b0c6c2d4760f137ee3fdc1bf06c37a190dd Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Thu, 4 Aug 2016 10:23:05 -0400 Subject: [PATCH 03/15] DOC: minor typo (thanks John Bernard) Closes #1496 --- config/action.d/badips.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config/action.d/badips.conf b/config/action.d/badips.conf index 70b46546..6f9513f6 100644 --- a/config/action.d/badips.conf +++ b/config/action.d/badips.conf @@ -1,6 +1,6 @@ # Fail2ban reporting to badips.com # -# Note: This reports and IP only and does not actually ban traffic. Use +# Note: This reports an IP only and does not actually ban traffic. Use # another action in the same jail if you want bans to occur. # # Set the category to the appropriate value before use. From 890a3dcbb96d47fead173943a005d6b2a4c90dee Mon Sep 17 00:00:00 2001 From: rhardy613 Date: Fri, 5 Aug 2016 17:26:47 -0400 Subject: [PATCH 04/15] Fix ASSP filter to work with current release of ASSP ASSP V1 development stopped at the end of 2014 and it is now deprecated. All users were urged to upgrade to ASSP V2 which is still actively developed. For some reason fail2ban 0.9.5 (and trunk) still have code which only understands ASSP V1 logs. This means the filter ignores brute force attacks against ASSP. Now updated with anchored patterns tested against 6 months of log data. --- config/filter.d/assp.conf | 21 ++++++++++++++------- fail2ban/tests/files/logs/assp | 14 +++++++++++++- 2 files changed, 27 insertions(+), 8 deletions(-) diff --git a/config/filter.d/assp.conf b/config/filter.d/assp.conf index 0bfba6dc..e740ad63 100644 --- a/config/filter.d/assp.conf +++ b/config/filter.d/assp.conf @@ -7,18 +7,25 @@ [Definition] -failregex = \<\S+@\S+\.\S+\> to: \S+@\S+\.\S+ relay attempt blocked for: \S+$ - \[SMTP Error\] 535 5\.7\.8 Error: authentication failed.*$ +failregex = ^\s*(?:[m0-9\-]+\s+)*(?:\[\S+\]\s+)* (?:\<\S+@\S+\.\S+\> )*(?:to: \S+@\S+\.\S+ )*relay attempt blocked for(?: \(parsing\))*: \S+$ + ^\s*(?:[m0-9\-]+\s+)*(?:\[\S+\]\s+)* \[SMTP Error\] 535 5\.7\.8 Error: authentication failed:\s+(?:\S+|Connection lost to authentication server|Invalid authentication mechanism|Invalid base64 data in continued response)*$ ignoreregex = # DEV Notes: # -# Examples: -# Jul-29-16 16:49:52 m1-25391-06124 [Worker_1] [TLS-out] [RelayAttempt] 0.0.0.0 to: user@example.org relay attempt blocked for: someone@example.org -# Jul-30-16 16:59:42 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 -# Jul-30-16 00:15:36 m1-52131-09651 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 -# Jul-31-16 06:45:59 [Worker_1] [TLS-in] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: +# Examples matches: +# Jul-29-16 16:49:52 m1-25391-06124 [Worker_1] [TLS-out] [RelayAttempt] 0.0.0.0 to: user@example.org relay attempt blocked for: someone@example.org +# Jul-30-16 16:59:42 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 +# Jul-30-16 00:15:36 m1-52131-09651 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 +# Jul-31-16 06:45:59 [Worker_1] [TLS-in] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: +# Jan-05-16 08:38:49 m1-01129-09140 [Worker_1] [TLS-in] [TLS-out] [RelayAttempt] 0.0.0.0 relay attempt blocked for (parsing): +# Jun-12-16 16:43:37 m1-64217-12013 [Worker_1] [TLS-in] [TLS-out] [RelayAttempt] 0.0.0.0 to: user2@example.com relay attempt blocked for (parsing): +# Jan-22-16 22:25:51 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Invalid authentication mechanism +# Mar-19-16 13:42:20 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Invalid base64 data in continued response +# Jul-18-16 16:54:21 [Worker_2] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Connection lost to authentication server +# Jul-18-16 17:14:23 m1-76453-02949 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Connection lost to authentication server + # # Author: Enrico Labedzki (enrico.labedzki@deiwos.de) # Updated: Robert Hardy (rhardy@webcon.ca) diff --git a/fail2ban/tests/files/logs/assp b/fail2ban/tests/files/logs/assp index 71c28221..6e9c1c35 100644 --- a/fail2ban/tests/files/logs/assp +++ b/fail2ban/tests/files/logs/assp @@ -5,4 +5,16 @@ Jul-30-16 17:07:25 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: au # failJSON: { "time": "2016-07-30T17:11:05", "match": true , "host": "0.0.0.0" } Jul-30-16 17:11:05 m1-13060-05386 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 # failJSON: { "time": "2016-07-31T06:45:59", "match": true , "host": "0.0.0.0" } -Jul-31-16 06:45:59 [Worker_1] [TLS-in] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: +Jul-31-16 06:45:59 [Worker_1] [TLS-in] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: +# failJSON: { "time": "2016-01-05T08:38:49", "match": true , "host": "0.0.0.0" } +Jan-05-16 08:38:49 m1-01129-09140 [Worker_1] [TLS-in] [TLS-out] [RelayAttempt] 0.0.0.0 relay attempt blocked for (parsing): +# failJSON: { "time": "2016-06-12T16:43:37", "match": true , "host": "0.0.0.0" } +Jun-12-16 16:43:37 m1-64217-12013 [Worker_1] [TLS-in] [TLS-out] [RelayAttempt] 0.0.0.0 to: user2@example.com relay attempt blocked for (parsing): +# failJSON: { "time": "2016-01-22T22:25:51", "match": true , "host": "0.0.0.0" } +Jan-22-16 22:25:51 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Invalid authentication mechanism +# failJSON: { "time": "2016-03-19T13:42:20", "match": true , "host": "0.0.0.0" } +Mar-19-16 13:42:20 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Invalid base64 data in continued response +# failJSON: { "time": "2016-07-18T16:54:21", "match": true , "host": "0.0.0.0" } +Jul-18-16 16:54:21 [Worker_2] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Connection lost to authentication server +# failJSON: { "time": "2016-07-18T17:14:23", "match": true , "host": "0.0.0.0" } +Jul-18-16 17:14:23 m1-76453-02949 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Connection lost to authentication server From 66fe5a77cef8f4337a7a39b70c46b3afb49ecd22 Mon Sep 17 00:00:00 2001 From: rhardy613 Date: Fri, 5 Aug 2016 23:18:51 -0400 Subject: [PATCH 05/15] Fix ASSP filter to work with both ASSP V1 and V2 ASSP V1 development stopped at the end of 2014 and it is now deprecated. All users were urged to upgrade to ASSP V2 which is still actively developed. fail2ban 0.9.5 (and trunk) still have code which only understands ASSP V1 logs. This means the filter ignores brute force attacks against ASSP. This fix adds V2 support. --- config/filter.d/assp.conf | 21 +++++++++++++++------ fail2ban/tests/files/logs/assp | 24 ++++++++++++++++++++++++ 2 files changed, 39 insertions(+), 6 deletions(-) diff --git a/config/filter.d/assp.conf b/config/filter.d/assp.conf index e740ad63..e71313e7 100644 --- a/config/filter.d/assp.conf +++ b/config/filter.d/assp.conf @@ -1,5 +1,5 @@ -# Fail2Ban filter for Anti-Spam SMTP Proxy Server (ASSP) Version 2.5.1 (or later) -# +# Fail2Ban filter for Anti-Spam SMTP Proxy Server (ASSP) +# Filter works in theory for both ASSP V1 and V2. Recommended ASSP is V2.5.1 or later. # Homepage: http://sourceforge.net/projects/assp/ # ProjectSite: http://sourceforge.net/projects/assp/?source=directory # @@ -7,14 +7,23 @@ [Definition] -failregex = ^\s*(?:[m0-9\-]+\s+)*(?:\[\S+\]\s+)* (?:\<\S+@\S+\.\S+\> )*(?:to: \S+@\S+\.\S+ )*relay attempt blocked for(?: \(parsing\))*: \S+$ - ^\s*(?:[m0-9\-]+\s+)*(?:\[\S+\]\s+)* \[SMTP Error\] 535 5\.7\.8 Error: authentication failed:\s+(?:\S+|Connection lost to authentication server|Invalid authentication mechanism|Invalid base64 data in continued response)*$ +__assp_actions = (?:dropping|refusing) + +failregex = ^(:? \[SSL-out\])? max sender authentication errors \(\d{,3}\) exceeded -- %(__assp_actions)s connection - after reply: \d{3} \d{1}\.\d{1}.\d{1} Error: authentication failed: \w+;$ + ^(?: \[SSL-out\])? SSL negotiation with client failed: SSL accept attempt failed with unknown error.*:unknown protocol;$ + ^ Blocking - too much AUTH errors \(\d{,3}\);$ + ^\s*(?:[m0-9\-]+\s+)*(?:\[\S+\]\s+)* (?:\<\S+@\S+\.\S+\> )*(?:to: \S+@\S+\.\S+ )*relay attempt blocked for(?: \(parsing\))*: \S+$ + ^\s*(?:[m0-9\-]+\s+)*(?:\[\S+\]\s+)* \[SMTP Error\] 535 5\.7\.8 Error: authentication failed:\s+(?:\S+|Connection lost to authentication server|Invalid authentication mechanism|Invalid base64 data in continued response)*$ ignoreregex = # DEV Notes: +# V1 Examples matches: +# Apr-27-13 02:33:09 Blocking 217.194.197.97 - too much AUTH errors (41); +# Dec-29-12 17:10:31 [SSL-out] 200.247.87.82 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; +# Dec-30-12 04:01:47 [SSL-out] 81.82.232.66 max sender authentication errors (5) exceeded # -# Examples matches: +# V2 Examples matches: # Jul-29-16 16:49:52 m1-25391-06124 [Worker_1] [TLS-out] [RelayAttempt] 0.0.0.0 to: user@example.org relay attempt blocked for: someone@example.org # Jul-30-16 16:59:42 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 # Jul-30-16 00:15:36 m1-52131-09651 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6 @@ -28,4 +37,4 @@ ignoreregex = # # Author: Enrico Labedzki (enrico.labedzki@deiwos.de) -# Updated: Robert Hardy (rhardy@webcon.ca) +# V2 Filters: Robert Hardy (rhardy@webcon.ca) diff --git a/fail2ban/tests/files/logs/assp b/fail2ban/tests/files/logs/assp index 6e9c1c35..21b01f9f 100644 --- a/fail2ban/tests/files/logs/assp +++ b/fail2ban/tests/files/logs/assp @@ -1,3 +1,27 @@ +# failJSON: { "time": "2013-04-07T07:08:36", "match": true , "host": "68.171.223.68" } +Apr-07-13 07:08:36 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; +# failJSON: { "time": "2013-04-07T07:08:36", "match": true , "host": "68.171.223.68" } +Apr-07-13 07:08:36 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; +# failJSON: { "time": "2013-04-07T07:10:37", "match": true , "host": "68.171.223.68" } +Apr-07-13 07:10:37 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; +# failJSON: { "time": "2013-04-07T07:12:37", "match": true , "host": "68.171.223.68" } +Apr-07-13 07:12:37 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; +# failJSON: { "time": "2013-04-07T07:14:36", "match": true , "host": "68.171.223.68" } +Apr-07-13 07:14:36 [SSL-out] 68.171.223.68 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol; +# failJSON: { "time": "2013-04-27T02:25:09", "match": true , "host": "217.194.197.97" } +Apr-27-13 02:25:09 Blocking 217.194.197.97 - too much AUTH errors (8); +# failJSON: { "time": "2013-04-27T02:25:09", "match": true , "host": "217.194.197.97" } +Apr-27-13 02:25:09 Blocking 217.194.197.97 - too much AUTH errors (9); +# failJSON: { "time": "2013-04-27T02:25:09", "match": true , "host": "217.194.197.97" } +Apr-27-13 02:25:09 Blocking 217.194.197.97 - too much AUTH errors (10); +# failJSON: { "time": "2013-04-27T02:25:10", "match": true , "host": "217.194.197.97" } +Apr-27-13 02:25:10 [SSL-out] 217.194.197.97 max sender authentication errors (5) exceeded -- dropping connection - after reply: 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6; +# failJSON: { "time": "2013-04-27T02:25:10", "match": true , "host": "217.194.197.97" } +Apr-27-13 02:25:10 [SSL-out] 217.194.197.97 max sender authentication errors (5) exceeded -- dropping connection - after reply: 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6; +# failJSON: { "time": "2013-04-27T02:25:10", "match": true , "host": "217.194.197.97" } +Apr-27-13 02:25:10 [SSL-out] 217.194.197.97 max sender authentication errors (5) exceeded -- dropping connection - after reply: 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6; +# failJSON: { "time": "2013-04-27T02:25:11", "match": true , "host": "217.194.197.97" } +Apr-27-13 02:25:11 [SSL-out] 217.194.197.97 max sender authentication errors (5) exceeded -- dropping connection - after reply: 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6; # failJSON: { "time": "2016-07-29T16:49:52", "match": true , "host": "0.0.0.0" } Jul-29-16 16:49:52 m1-25391-06124 [Worker_1] [TLS-out] [RelayAttempt] 0.0.0.0 to: user@example.org relay attempt blocked for: someone@example.org # failJSON: { "time": "2016-07-30T17:07:25", "match": true , "host": "0.0.0.0" } From 8265e3f0f94e311345e162cd189b6fbbfc033891 Mon Sep 17 00:00:00 2001 From: rhardy613 Date: Fri, 5 Aug 2016 23:25:15 -0400 Subject: [PATCH 06/15] Fix comments For some reasons the comment changes weren't pickup in the last commit. This fixes it. --- config/filter.d/assp.conf | 3 +++ 1 file changed, 3 insertions(+) diff --git a/config/filter.d/assp.conf b/config/filter.d/assp.conf index e71313e7..896d7514 100644 --- a/config/filter.d/assp.conf +++ b/config/filter.d/assp.conf @@ -1,11 +1,14 @@ # Fail2Ban filter for Anti-Spam SMTP Proxy Server (ASSP) # Filter works in theory for both ASSP V1 and V2. Recommended ASSP is V2.5.1 or later. +# Support for ASSP V1 ended in 2014 so if you are still running ASSP V1 an immediate upgrade is recommended. +# # Homepage: http://sourceforge.net/projects/assp/ # ProjectSite: http://sourceforge.net/projects/assp/?source=directory # # [Definition] +# Note: First three failregex matches below are for ASSP V1 with the remaining being designed for V2. Deleting the V1 regex is recommended but I left it in for compatibilty reasons. __assp_actions = (?:dropping|refusing) From 89f8999fe51d52970f9357526a313d174758769e Mon Sep 17 00:00:00 2001 From: rhardy613 Date: Sat, 6 Aug 2016 01:07:04 -0400 Subject: [PATCH 07/15] Add changelog entry for ASSP filter changes Add changelog entry for ASSP filter changes --- ChangeLog | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ChangeLog b/ChangeLog index 59b6b1af..fa39326a 100644 --- a/ChangeLog +++ b/ChangeLog @@ -14,6 +14,8 @@ new features (e.g. IPv6 support), please consider 0.10 branch and its releases. ### Fixes +* `filter.d/assp.conf` + - Extended failregex and test cases to handle ASSP V1 and V2 (gh-1494) * `filter.d/monit.conf` - Extended failregex with new monit "access denied" version (gh-1355) - failregex of previous monit version merged as single expression From c52aaa8b78ad9b25ee02ac9fdeedf5e03bddd89c Mon Sep 17 00:00:00 2001 From: sebres Date: Mon, 8 Aug 2016 19:06:28 +0200 Subject: [PATCH 08/15] ASSP failregex minor fixes --- config/filter.d/assp.conf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/config/filter.d/assp.conf b/config/filter.d/assp.conf index 896d7514..278e25cb 100644 --- a/config/filter.d/assp.conf +++ b/config/filter.d/assp.conf @@ -15,8 +15,8 @@ __assp_actions = (?:dropping|refusing) failregex = ^(:? \[SSL-out\])? max sender authentication errors \(\d{,3}\) exceeded -- %(__assp_actions)s connection - after reply: \d{3} \d{1}\.\d{1}.\d{1} Error: authentication failed: \w+;$ ^(?: \[SSL-out\])? SSL negotiation with client failed: SSL accept attempt failed with unknown error.*:unknown protocol;$ ^ Blocking - too much AUTH errors \(\d{,3}\);$ - ^\s*(?:[m0-9\-]+\s+)*(?:\[\S+\]\s+)* (?:\<\S+@\S+\.\S+\> )*(?:to: \S+@\S+\.\S+ )*relay attempt blocked for(?: \(parsing\))*: \S+$ - ^\s*(?:[m0-9\-]+\s+)*(?:\[\S+\]\s+)* \[SMTP Error\] 535 5\.7\.8 Error: authentication failed:\s+(?:\S+|Connection lost to authentication server|Invalid authentication mechanism|Invalid base64 data in continued response)*$ + ^\s*(?:[\w\-]+\s+)*(?:\[\S+\]\s+)* (?:\<\S+@\S+\.\S+\> )*(?:to: \S+@\S+\.\S+ )*relay attempt blocked for(?: \(parsing\))?: \S+$ + ^\s*(?:[\w\-]+\s+)*(?:\[\S+\]\s+)* \[SMTP Error\] 535 5\.7\.8 Error: authentication failed:\s+(?:\S+|Connection lost to authentication server|Invalid authentication mechanism|Invalid base64 data in continued response)?$ ignoreregex = From 9ddbd642f725f3d5dfd94de7f08cc7526d83cd43 Mon Sep 17 00:00:00 2001 From: maksyms Date: Mon, 8 Aug 2016 22:07:55 +0100 Subject: [PATCH 09/15] Accept no space after "failed:" (#1501) yoh: Squashed to ease cherry-picking into 0.9 * accept no space after "failed:" fix issue #1497 * accept no space after "failed:" * Update postfix-sasl * Update postfix-sasl * Update postfix-sasl --- config/filter.d/postfix-sasl.conf | 2 +- fail2ban/tests/files/logs/postfix-sasl | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/config/filter.d/postfix-sasl.conf b/config/filter.d/postfix-sasl.conf index 4a6ceaaa..1a24ca94 100644 --- a/config/filter.d/postfix-sasl.conf +++ b/config/filter.d/postfix-sasl.conf @@ -9,7 +9,7 @@ before = common.conf _daemon = postfix(-\w+)?/(?:submission/|smtps/)?smtp[ds] -failregex = ^%(__prefix_line)swarning: [-._\w]+\[\]: SASL ((?i)LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(: [ A-Za-z0-9+/:]*={0,2})?\s*$ +failregex = ^%(__prefix_line)swarning: [-._\w]+\[\]: SASL ((?i)LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(:[ A-Za-z0-9+/:]*={0,2})?\s*$ ignoreregex = authentication failed: Connection lost to authentication server$ diff --git a/fail2ban/tests/files/logs/postfix-sasl b/fail2ban/tests/files/logs/postfix-sasl index 9fcb0f49..cdcb5121 100644 --- a/fail2ban/tests/files/logs/postfix-sasl +++ b/fail2ban/tests/files/logs/postfix-sasl @@ -26,3 +26,7 @@ Jan 29 08:11:45 mail postfix-incoming/smtpd[10752]: warning: unknown[1.1.1.1]: S # failJSON: { "time": "2005-04-12T02:24:11", "match": true , "host": "62.138.2.143" } Apr 12 02:24:11 xxx postfix/smtps/smtpd[42]: warning: astra4139.startdedicated.de[62.138.2.143]: SASL LOGIN authentication failed: UGFzc3dvcmQ6 + +# failJSON: { "time": "2005-08-03T15:30:49", "match": true , "host": "98.191.84.74" } +Aug 3 15:30:49 ksusha postfix/smtpd[17041]: warning: mail.foldsandwalker.com[98.191.84.74]: SASL Plain authentication failed: + From 123f4ceaee8d1f1b5c1896a6661a7ba066f29325 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Mon, 8 Aug 2016 17:11:07 -0400 Subject: [PATCH 10/15] Changelog for postfix-sasl fix --- ChangeLog | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ChangeLog b/ChangeLog index cbc7c8d5..76844b6f 100644 --- a/ChangeLog +++ b/ChangeLog @@ -20,6 +20,8 @@ releases. (e.g. test-cases faults on Fedora, see gh-1353) * `filter.d/assp.conf` - Extended failregex and test cases to handle ASSP V1 and V2 (gh-1494) +* `filter.d/postfix-sasl.conf` + - Allow for having no trailing space after 'failed:' (gh-1497) ### New Features From 9d70c49ea86b41a94cea8f5a2e035db0ff6871a5 Mon Sep 17 00:00:00 2001 From: Yaroslav Halchenko Date: Tue, 9 Aug 2016 06:49:40 -0400 Subject: [PATCH 11/15] BF: install doc files only under Linuxes and other GNU systems (Closes #1233) (#1503) --- .travis.yml | 2 ++ setup.py | 20 +++++++++++++++----- 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/.travis.yml b/.travis.yml index 50894a94..9ef607da 100644 --- a/.travis.yml +++ b/.travis.yml @@ -41,6 +41,8 @@ script: - if [[ "$F2B_PY_3" ]]; then coverage run bin/fail2ban-testcases; fi # Use $VENV_BIN (not python) or else sudo will always run the system's python (2.7) - sudo $VENV_BIN/pip install . + # Doc files should get installed on Travis under Linux + - test -e /usr/share/doc/fail2ban/FILTERS after_success: - coveralls - codecov diff --git a/setup.py b/setup.py index e3c499d2..2d3f4569 100755 --- a/setup.py +++ b/setup.py @@ -19,9 +19,11 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. __author__ = "Cyril Jaquier, Steven Hiscocks, Yaroslav Halchenko" -__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2008-2013 Fail2Ban Contributors" +__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2008-2016 Fail2Ban Contributors" __license__ = "GPL" +import platform + try: import setuptools from setuptools import setup @@ -85,6 +87,18 @@ if os.path.exists('/var/run'): # realpath is used to possibly resolve /var/run -> /run symlink data_files_extra += [(realpath('/var/run/fail2ban'), '')] +# Installing documentation files only under Linux or other GNU/ systems +# (e.g. GNU/kFreeBSD), since others might have protective mechanisms forbidding +# installation there (see e.g. #1233) +platform_system = platform.system().lower() +doc_files = ['README.md', 'DEVELOP', 'FILTERS', 'doc/run-rootless.txt'] +if platform_system in ('solaris', 'sunos'): + doc_files.append('README.Solaris') +if platform_system in ('linux', 'solaris', 'sunos') or platform_system.startswith('gnu'): + data_files_extra.append( + ('/usr/share/doc/fail2ban', doc_files) + ) + # Get version number, avoiding importing fail2ban. # This is due to tests not functioning for python3 as 2to3 takes place later exec(open(join("fail2ban", "version.py")).read()) @@ -148,10 +162,6 @@ setup( ('/var/lib/fail2ban', '' ), - ('/usr/share/doc/fail2ban', - ['README.md', 'README.Solaris', 'DEVELOP', 'FILTERS', - 'doc/run-rootless.txt'] - ) ] + data_files_extra, **setup_extra ) From 38d53a72fd2b55fae0c6a57e7ea10ffa3446b650 Mon Sep 17 00:00:00 2001 From: sebres Date: Thu, 11 Aug 2016 18:34:18 +0200 Subject: [PATCH 12/15] introduces new command "fail2ban-python", as automatically created symlink to python executable, where fail2ban currently installed (resp. its modules are located); fixed pythonic filters and test scripts (running via "fail2ban-python" now); fixed test case "testSetupInstallRoot" not for default python (also using direct call, out of virtualenv); # Conflicts: # config/filter.d/ignorecommands/apache-fakegooglebot # fail2ban/tests/files/config/apache-auth/digest.py # fail2ban/tests/files/ignorecommand.py # fail2ban/tests/misctestcase.py --- bin/fail2ban-testcases | 5 ++- .../ignorecommands/apache-fakegooglebot | 2 +- fail2ban/helpers.py | 18 +++++++++ .../tests/files/config/apache-auth/digest.py | 2 +- fail2ban/tests/files/ignorecommand.py | 2 +- fail2ban/tests/misctestcase.py | 21 ++++++++++ setup.py | 40 ++++++++++++++++++- 7 files changed, 85 insertions(+), 5 deletions(-) diff --git a/bin/fail2ban-testcases b/bin/fail2ban-testcases index dd6547a5..d02f482e 100755 --- a/bin/fail2ban-testcases +++ b/bin/fail2ban-testcases @@ -38,11 +38,14 @@ if os.path.exists("fail2ban/__init__.py"): from fail2ban.version import version from fail2ban.tests.utils import gatherTests -from fail2ban.helpers import FormatterWithTraceBack, getLogger +from fail2ban.helpers import updatePyExec, FormatterWithTraceBack, getLogger from fail2ban.server.mytime import MyTime from optparse import OptionParser, Option +# Update fail2ban-python env to current python version (where f2b-modules located/installed) +updatePyExec(os.path.dirname(__file__)) + def get_opt_parser(): # use module docstring for help output p = OptionParser( diff --git a/config/filter.d/ignorecommands/apache-fakegooglebot b/config/filter.d/ignorecommands/apache-fakegooglebot index 19fb5107..9e6f4459 100755 --- a/config/filter.d/ignorecommands/apache-fakegooglebot +++ b/config/filter.d/ignorecommands/apache-fakegooglebot @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/env fail2ban-python # Inspired by https://isc.sans.edu/forums/diary/When+Google+isnt+Google/15968/ # # Written in Python to reuse built-in Python batteries and not depend on diff --git a/fail2ban/helpers.py b/fail2ban/helpers.py index aef39835..8523f21b 100644 --- a/fail2ban/helpers.py +++ b/fail2ban/helpers.py @@ -113,6 +113,24 @@ class FormatterWithTraceBack(logging.Formatter): return logging.Formatter.format(self, record) +def updatePyExec(bindir, executable=None): + """Update fail2ban-python link to current python version (where f2b-modules located/installed) + """ + bindir = os.path.realpath(bindir) + if executable is None: + executable = sys.executable + pypath = os.path.join(bindir, 'fail2ban-python') + # if not exists or point to another version - update link: + isfile = os.path.isfile(pypath) + if not isfile or os.path.realpath(pypath) != os.path.realpath(executable): + if isfile: + os.unlink(pypath) + os.symlink(executable, pypath) + # extend current environment path (e.g. if fail2ban not yet installed): + if bindir not in os.environ["PATH"].split(os.pathsep): + os.environ["PATH"] = os.environ["PATH"] + os.pathsep + bindir; + + def getLogger(name): """Get logging.Logger instance with Fail2Ban logger name convention """ diff --git a/fail2ban/tests/files/config/apache-auth/digest.py b/fail2ban/tests/files/config/apache-auth/digest.py index 875ebffe..03588594 100755 --- a/fail2ban/tests/files/config/apache-auth/digest.py +++ b/fail2ban/tests/files/config/apache-auth/digest.py @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/env fail2ban-python import requests try: diff --git a/fail2ban/tests/files/ignorecommand.py b/fail2ban/tests/files/ignorecommand.py index dd6b5aab..7011b51b 100755 --- a/fail2ban/tests/files/ignorecommand.py +++ b/fail2ban/tests/files/ignorecommand.py @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/env fail2ban-python import sys if sys.argv[1] == "10.0.0.1": exit(0) diff --git a/fail2ban/tests/misctestcase.py b/fail2ban/tests/misctestcase.py index 5f7d853d..069ad659 100644 --- a/fail2ban/tests/misctestcase.py +++ b/fail2ban/tests/misctestcase.py @@ -70,6 +70,17 @@ class HelpersTest(unittest.TestCase): self.assertEqual(splitwords(' 1\n 2, 3'), ['1', '2', '3']) +def _getSysPythonVersion(): + import subprocess, locale + sysVerCmd = "fail2ban-python -c 'import sys; print(tuple(sys.version_info))'" + if sys.version_info >= (2,7): + sysVer = subprocess.check_output(sysVerCmd, shell=True) + else: + sysVer = subprocess.Popen(sysVerCmd, shell=True, stdout=subprocess.PIPE).stdout.read() + if sys.version_info >= (3,): + sysVer = sysVer.decode(locale.getpreferredencoding(), 'replace') + return str(sysVer).rstrip() + class SetupTest(unittest.TestCase): def setUp(self): @@ -79,6 +90,12 @@ class SetupTest(unittest.TestCase): raise unittest.SkipTest( "Seems to be running not out of source distribution" " -- cannot locate setup.py") + # compare current version of python installed resp. active one: + sysVer = _getSysPythonVersion() + if sysVer != str(tuple(sys.version_info)): + raise unittest.SkipTest( + "Seems to be running with python distribution %s" + " -- install can be tested only with system distribution %s" % (str(tuple(sys.version_info)), sysVer)) def testSetupInstallRoot(self): if not self.setup: @@ -125,6 +142,10 @@ class SetupTest(unittest.TestCase): 'etc/fail2ban/jail.conf'): self.assertTrue(os.path.exists(os.path.join(tmp, f)), msg="Can't find %s" % f) + self.assertEqual( + os.path.realpath(os.path.join(tmp, 'usr/local/bin/fail2ban-python')), + os.path.realpath(sys.executable)) + finally: # clean up shutil.rmtree(tmp) diff --git a/setup.py b/setup.py index 2d3f4569..aa9f7055 100755 --- a/setup.py +++ b/setup.py @@ -40,12 +40,46 @@ except ImportError: # python 2.x from distutils.command.build_py import build_py from distutils.command.build_scripts import build_scripts +# all versions +from distutils.command.install_scripts import install_scripts + import os from os.path import isfile, join, isdir, realpath import sys import warnings from glob import glob + +def updatePyExec(bindir, executable=None): + """Update fail2ban-python link to current python version (where f2b-modules located/installed) + """ + bindir = os.path.realpath(bindir) + if executable is None: + executable = sys.executable + pypath = os.path.join(bindir, 'fail2ban-python') + # if not exists or point to another version - update link: + isfile = os.path.isfile(pypath) + if not isfile or os.path.realpath(pypath) != os.path.realpath(executable): + if isfile: + os.unlink(pypath) + os.symlink(executable, pypath) + + +# Wrapper to install python binding (to current python version): +class install_scripts_f2b(install_scripts): + + def get_outputs(self): + outputs = install_scripts.get_outputs(self) + fn = None + for fn in outputs: + if os.path.basename(fn) == 'fail2ban-server': + break + bindir = os.path.dirname(fn) + print('creating fail2ban-python binding -> %s' % (bindir,)) + updatePyExec(bindir) + return outputs + + if setuptools and "test" in sys.argv: import logging logSys = logging.getLogger("fail2ban") @@ -113,12 +147,16 @@ setup( url = "http://www.fail2ban.org", license = "GPL", platforms = "Posix", - cmdclass = {'build_py': build_py, 'build_scripts': build_scripts}, + cmdclass = { + 'build_py': build_py, 'build_scripts': build_scripts, + 'install_scripts': install_scripts_f2b + }, scripts = [ 'bin/fail2ban-client', 'bin/fail2ban-server', 'bin/fail2ban-regex', 'bin/fail2ban-testcases', + # 'bin/fail2ban-python', -- link (binary), will be installed via install_scripts_f2b wrapper ], packages = [ 'fail2ban', From 6cdc1ce68522c137c7242ce8dbdb8bca1bb138e7 Mon Sep 17 00:00:00 2001 From: sebres Date: Thu, 11 Aug 2016 19:57:40 +0200 Subject: [PATCH 13/15] compatibility fix (virtualenv, running test cases in py3) # Conflicts: # MANIFEST --- MANIFEST | 1 + bin/fail2ban-testcases | 3 ++- fail2ban/helpers.py | 18 --------------- fail2ban/setup.py | 42 ++++++++++++++++++++++++++++++++++ fail2ban/tests/misctestcase.py | 33 ++++++++++++++++---------- setup.py | 18 ++++----------- 6 files changed, 70 insertions(+), 45 deletions(-) create mode 100644 fail2ban/setup.py diff --git a/MANIFEST b/MANIFEST index c27878e8..e0d7398c 100644 --- a/MANIFEST +++ b/MANIFEST @@ -194,6 +194,7 @@ fail2ban/server/server.py fail2ban/server/strptime.py fail2ban/server/ticket.py fail2ban/server/transmitter.py +fail2ban/setup.py fail2ban-testcases-all fail2ban-testcases-all-python3 fail2ban/tests/action_d/__init__.py diff --git a/bin/fail2ban-testcases b/bin/fail2ban-testcases index d02f482e..e0ff11d2 100755 --- a/bin/fail2ban-testcases +++ b/bin/fail2ban-testcases @@ -38,7 +38,8 @@ if os.path.exists("fail2ban/__init__.py"): from fail2ban.version import version from fail2ban.tests.utils import gatherTests -from fail2ban.helpers import updatePyExec, FormatterWithTraceBack, getLogger +from fail2ban.helpers import FormatterWithTraceBack, getLogger +from fail2ban.setup import updatePyExec from fail2ban.server.mytime import MyTime from optparse import OptionParser, Option diff --git a/fail2ban/helpers.py b/fail2ban/helpers.py index 8523f21b..aef39835 100644 --- a/fail2ban/helpers.py +++ b/fail2ban/helpers.py @@ -113,24 +113,6 @@ class FormatterWithTraceBack(logging.Formatter): return logging.Formatter.format(self, record) -def updatePyExec(bindir, executable=None): - """Update fail2ban-python link to current python version (where f2b-modules located/installed) - """ - bindir = os.path.realpath(bindir) - if executable is None: - executable = sys.executable - pypath = os.path.join(bindir, 'fail2ban-python') - # if not exists or point to another version - update link: - isfile = os.path.isfile(pypath) - if not isfile or os.path.realpath(pypath) != os.path.realpath(executable): - if isfile: - os.unlink(pypath) - os.symlink(executable, pypath) - # extend current environment path (e.g. if fail2ban not yet installed): - if bindir not in os.environ["PATH"].split(os.pathsep): - os.environ["PATH"] = os.environ["PATH"] + os.pathsep + bindir; - - def getLogger(name): """Get logging.Logger instance with Fail2Ban logger name convention """ diff --git a/fail2ban/setup.py b/fail2ban/setup.py new file mode 100644 index 00000000..87d50e66 --- /dev/null +++ b/fail2ban/setup.py @@ -0,0 +1,42 @@ +# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*- +# vi: set ft=python sts=4 ts=4 sw=4 noet : + +# This file is part of Fail2Ban. +# +# Fail2Ban is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Fail2Ban is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Fail2Ban; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +__author__ = "Serg G. Brester" +__license__ = "GPL" + +import os +import sys + + +def updatePyExec(bindir, executable=None): + """Update fail2ban-python link to current python version (where f2b-modules located/installed) + """ + bindir = os.path.realpath(bindir) + if executable is None: + executable = sys.executable + pypath = os.path.join(bindir, 'fail2ban-python') + # if not exists or point to another version - update link: + isfile = os.path.isfile(pypath) + if not isfile or os.path.realpath(pypath) != os.path.realpath(executable): + if isfile: + os.unlink(pypath) + os.symlink(executable, pypath) + # extend current environment path (e.g. if fail2ban not yet installed): + if bindir not in os.environ["PATH"].split(os.pathsep): + os.environ["PATH"] = os.environ["PATH"] + os.pathsep + bindir; diff --git a/fail2ban/tests/misctestcase.py b/fail2ban/tests/misctestcase.py index 069ad659..e2b4c0b6 100644 --- a/fail2ban/tests/misctestcase.py +++ b/fail2ban/tests/misctestcase.py @@ -70,16 +70,21 @@ class HelpersTest(unittest.TestCase): self.assertEqual(splitwords(' 1\n 2, 3'), ['1', '2', '3']) +if sys.version_info >= (2,7): + def _sh_call(cmd): + import subprocess, locale + ret = subprocess.check_output(cmd, shell=True) + if sys.version_info >= (3,): + ret = ret.decode(locale.getpreferredencoding(), 'replace') + return str(ret).rstrip() +else: + def _sh_call(cmd): + import subprocess + ret = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read() + return str(ret).rstrip() + def _getSysPythonVersion(): - import subprocess, locale - sysVerCmd = "fail2ban-python -c 'import sys; print(tuple(sys.version_info))'" - if sys.version_info >= (2,7): - sysVer = subprocess.check_output(sysVerCmd, shell=True) - else: - sysVer = subprocess.Popen(sysVerCmd, shell=True, stdout=subprocess.PIPE).stdout.read() - if sys.version_info >= (3,): - sysVer = sysVer.decode(locale.getpreferredencoding(), 'replace') - return str(sysVer).rstrip() + return _sh_call("fail2ban-python -c 'import sys; print(tuple(sys.version_info))'") class SetupTest(unittest.TestCase): @@ -142,9 +147,13 @@ class SetupTest(unittest.TestCase): 'etc/fail2ban/jail.conf'): self.assertTrue(os.path.exists(os.path.join(tmp, f)), msg="Can't find %s" % f) - self.assertEqual( - os.path.realpath(os.path.join(tmp, 'usr/local/bin/fail2ban-python')), - os.path.realpath(sys.executable)) + # Because the install (test) path in virtual-env differs from some development-env, + # it is not a `tmp + '/usr/local/bin/'`, so search for it: + installedPath = _sh_call('find ' + tmp+ ' -name fail2ban-python').split('\n') + self.assertTrue(len(installedPath) > 0) + for installedPath in installedPath: + self.assertEqual( + os.path.realpath(installedPath), os.path.realpath(sys.executable)) finally: # clean up diff --git a/setup.py b/setup.py index aa9f7055..a4a3f977 100755 --- a/setup.py +++ b/setup.py @@ -49,20 +49,7 @@ import sys import warnings from glob import glob - -def updatePyExec(bindir, executable=None): - """Update fail2ban-python link to current python version (where f2b-modules located/installed) - """ - bindir = os.path.realpath(bindir) - if executable is None: - executable = sys.executable - pypath = os.path.join(bindir, 'fail2ban-python') - # if not exists or point to another version - update link: - isfile = os.path.isfile(pypath) - if not isfile or os.path.realpath(pypath) != os.path.realpath(executable): - if isfile: - os.unlink(pypath) - os.symlink(executable, pypath) +from fail2ban.setup import updatePyExec # Wrapper to install python binding (to current python version): @@ -80,6 +67,9 @@ class install_scripts_f2b(install_scripts): return outputs +# Update fail2ban-python env to current python version (where f2b-modules located/installed) +updatePyExec(os.path.join(os.path.dirname(__file__), 'bin')) + if setuptools and "test" in sys.argv: import logging logSys = logging.getLogger("fail2ban") From db30b7ce06cd5da3fa0720a34116084026f0083d Mon Sep 17 00:00:00 2001 From: sebres Date: Fri, 12 Aug 2016 16:41:55 +0200 Subject: [PATCH 14/15] BF: prefer sys.argv[0] by retrieving of root resp. bin path: __file__ seems to be overwritten sometimes on some python versions (e.g. bug of 2.6 by running under cProfile, etc.) --- bin/fail2ban-testcases | 6 +++++- setup.py | 6 +++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/bin/fail2ban-testcases b/bin/fail2ban-testcases index e0ff11d2..a711e07c 100755 --- a/bin/fail2ban-testcases +++ b/bin/fail2ban-testcases @@ -45,7 +45,11 @@ from fail2ban.server.mytime import MyTime from optparse import OptionParser, Option # Update fail2ban-python env to current python version (where f2b-modules located/installed) -updatePyExec(os.path.dirname(__file__)) +bindir = os.path.dirname( + # __file__ seems to be overwritten sometimes on some python versions (e.g. bug of 2.6 by running under cProfile, etc.): + sys.argv[0] if os.path.basename(sys.argv[0]) == 'fail2ban-testcases' else __file__ +) +updatePyExec(bindir) def get_opt_parser(): # use module docstring for help output diff --git a/setup.py b/setup.py index a4a3f977..cbfc6e07 100755 --- a/setup.py +++ b/setup.py @@ -68,7 +68,11 @@ class install_scripts_f2b(install_scripts): # Update fail2ban-python env to current python version (where f2b-modules located/installed) -updatePyExec(os.path.join(os.path.dirname(__file__), 'bin')) +rootdir = os.path.realpath(os.path.dirname( + # __file__ seems to be overwritten sometimes on some python versions (e.g. bug of 2.6 by running under cProfile, etc.): + sys.argv[0] if os.path.basename(sys.argv[0]) == 'setup.py' else __file__ +)) +updatePyExec(os.path.join(rootdir, 'bin')) if setuptools and "test" in sys.argv: import logging From cb340db2209d9d2f518c8ff3f9bbea03c17be973 Mon Sep 17 00:00:00 2001 From: sebres Date: Fri, 12 Aug 2016 18:37:46 +0200 Subject: [PATCH 15/15] ChangeLog entry for gh-1508 --- ChangeLog | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/ChangeLog b/ChangeLog index 76844b6f..ecb8cc87 100644 --- a/ChangeLog +++ b/ChangeLog @@ -18,6 +18,13 @@ releases. induced a subsequent error: last position of log file will be never retrieved (gh-795) * Fixed a distribution related bug within testReadStockJailConfForceEnabled (e.g. test-cases faults on Fedora, see gh-1353) +* Fixed pythonic filters and test scripts (running via wrong python version, + uses "fail2ban-python" now); +* Fixed test case "testSetupInstallRoot" for not default python version (also + using direct call, out of virtualenv); +* `filter.d/ignorecommands/apache-fakegooglebot` + - Fixed error within apache-fakegooglebot, that will be called + with wrong python version (gh-1506) * `filter.d/assp.conf` - Extended failregex and test cases to handle ASSP V1 and V2 (gh-1494) * `filter.d/postfix-sasl.conf` @@ -26,6 +33,18 @@ releases. ### New Features ### Enhancements +* Introduces new command "fail2ban-python", as automatically created symlink to + python executable, where fail2ban currently installed (resp. its modules are located): + - allows to use the same version, fail2ban currently running, e.g. in + external scripts just via replace python with fail2ban-python: + ```diff + -#!/usr/bin/env python + +#!/usr/bin/env fail2ban-python + ``` + - always the same pickle protocol + - the same (and also guaranteed available) fail2ban modules + - simplified stand-alone install, resp. stand-alone installation possibility + via setup (like gh-1487) is getting closer * Several test cases rewritten using new methods assertIn, assertNotIn * New forward compatibility method assertRaisesRegexp (normally python >= 2.7). Methods assertIn, assertNotIn, assertRaisesRegexp, assertLogged, assertNotLogged