github
/
fail2ban
mirror of https://github.com/fail2ban/fail2ban
1
0
Fork 0
Code Issues Releases Wiki Activity

MRG: from master again 2014-01-01

pull/547/head
Daniel Black 2014-01-01 19:28:38 +11:00
parent 04438cd1a1 71869dd6e1
commit 391b5fc883
111 changed files with 3112 additions and 1024 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -7,3 +7,4 @@ htmlcov
*.orig
*.rej
*.bak
__pycache__

1
.travis.yml
View File

@ -6,6 +6,7 @@ python:
- "2.7"
- "3.2"
- "3.3"
- "pypy"
before_install:
- sudo apt-get update -qq
install:

90
ChangeLog
View File

@ -4,51 +4,66 @@
|_| \__,_|_|_/___|_.__/\__,_|_||_|
================================================================================
Fail2Ban (version 0.9.0a1) 2013/??/??
Fail2Ban (version 0.9.0a2) 2014/??/??
================================================================================
ver. 0.9.0 (2013/??/??) - alpha
ver. 0.9.0 (2014/??/??) - alpha
----------
Carries all fixes in 0.8.9 and new features and enhancements. Nearly
all development is thanks to Steven Hiscocks (THANKS!) with only
code-review and minor additions from Yaroslav Halchenko.
Carries all fixes, features and enhancements from 0.8.12 with major changes.
Nearly all development is thanks to Steven Hiscocks (THANKS!), merging,
testcases and timezone support from Daniel Black, and code-review and minor
additions from Yaroslav Halchenko.
The minimum supported python version is now 2.6. If you have python-2.4 or 2.5
you can use the 0.8.12 version of fail2ban.
Major changes have occured since version 0.8.12. Please test your
configuration before relying on it.
- Refactoring (IMPORTANT -- Please review your setup and configuration):
Yaroslav Halchenko
* [..bddbf1e] jail.conf was heavily refactored and now is similar
to how it looked on Debian systems:
- default action could be configured once for all jails
- jails definitions only provide customizations (port, logpath)
- no need to specify 'filter' if name matches jail name
Steven Hiscocks
* [..5aef036] Core functionality moved into fail2ban/ module.
Closes gh-26
* Added fail2ban persistent database
- default location at /var/lib/fail2ban/fail2ban.sqlite3
- allows active bans to be reinstated on restart
- log files read from last position after restart
* Added systemd journal backend
- Dependency on python-systemd
- New "journalmatch" option added to filter configs files
- New "systemd-journal" option added to fail2ban-regex
* Added python3 support
* Support %z (Timezone offset) and %f (sub-seconds) support for
datedetector. Enhanced existing date/time have been updated patterns to
support these. ISO8601 now defaults to localtime unless specified otherwise.
Some filters have been change as required to capture these elements in the
right timezone correctly.
- New features:
Steven Hiscocks
* [..c7ae460] Multiline failregex. Close gh-54
* [8af32ed] Guacamole filter and support for Apache Tomcat date
format
* [..4869186] Python3 support
* [..b6059f4] 'timeout' option for actions Close gh-60 and Debian bug
#410077. Also it would now capture and include stdout and stderr
into logging messages in case of error or at DEBUG loglevel.
Daniel Black and TESTOVIK
* Multiline filter for sendmail-spam. Close gh-418
* Added action xarf-login-attack to report formatted attack messages
according to the XARF standard (v0.2). Close gh-105
* Support PyPy
- Enhancements
Steven Hiscocks
* Multiline filter for sendmail-spam. Close gh-418
* Multiline regex for Disconnecting: Too many authentication failures for
root [preauth]\nConnection closed by 6X.XXX.XXX.XXX [preauth]
* Replacing use of deprecated API (.warning, .assertEqual, etc)
* [..a648cc2] Filters can have options now too
* [..e019ab7] Multiple instances of the same action are allowed in the
same jail -- use actname option to disambiguate.
Daniel Black
* Support %z (Timezone offset) and %f (sub-seconds) support for
datedetector. Enhanced existing date/time have been updated patterns to
support these. ISO8601 now defaults to localtime unless specified otherwise.
Some filters have been change as required to capture these elements in the
right timezone correctly.
ver. 0.8.12 (2013/12/XX) - things-can-only-get-better
-----------
@ -56,15 +71,46 @@ ver. 0.8.12 (2013/12/XX) - things-can-only-get-better
- IMPORTANT incompatible changes:
- Fixes:
- Rename firewall-cmd-direct-new to firewall-cmd-new to fit within jail name
name length. As per gh-395
- allow for ",milliseconds" in the custom date format of proftpd.log
- allow for ", referer ..." in apache-* filter for apache error logs.
- allow for spaces at the beginning of kernel messages. Closes gh-448
- recidive jail to block all protocols. Closes gh-440. Thanks Ioan Indreias
- smtps not a IANA standard and has been removed from Arch. Replaced with
465. Thanks Stefan. Closes gh-447
- mysqld-syslog-iptables rule was too long. Part of gh-447.
- add 'flushlogs' command to allow logrotation without clobbering logtarget
settings. Closes gh-458, Debian bug #697333, Redhat bug #891798.
- complain action - ensure where not matching other IPs in log sample.
Closes gh-467
- Fix firewall-cmd actioncheck - patch from Adam Tkac. Redhat Bug #979622
- Fix apache-common for apache-2.4 log file format. Thanks Mark White.
Closes gh-516
- Asynchat changed to use push method which verifys whether all data was
send. This ensures that all data is sent before closing the connection.
- Enhancements:
- added firewallcmd-ipset action
- long names on jails documented based on iptables limit of 30 less
len("fail2ban-").
- remove indentation of name and loglevel while logging to SYSLOG to
resolve syslog(-ng) parsing problems. Closes Debian bug #730202.
- added squid filter. Thanks Roman Gelfand.
- updated check_fail2ban to return performance data for all jails.
- filter apache-noscript now includes php cgi scripts.
Thanks dani. Closes gh-503
- added ufw action. Thanks Guilhem Lettron. lp-#701522
- exim-spam filter to match spamassassin log entry for option SAdevnull.
Thanks Ivo Truxa. Closes gh-533
- filter.d/nsd.conf -- also amended Unix date template to match nsd format
- loglines now also report "[PID]" after the name portion
- New Features:
Daniel Black
* filter.d/solid-pop3d -- added thanks to Jacques Lav!gnotte on mailinglist.
- Enhancements:
- Added filter for solid-pop3d -- thanks to Jacques Lav!gnotte on mailinglist.
- Added filter for apache-modsecurity
- Added filter for openwebmail thanks Ivo Truxa. Closes gh-543
ver. 0.8.11 (2013/11/13) - loves-unittests-and-tight-DoS-free-filter-regexes

464
DEVELOP
View File

@ -34,465 +34,7 @@ When submitting pull requests on GitHub we ask you to:
* Include a change to the relevant section of the ChangeLog; and
* Include yourself in THANKS if not already there.
Filters
=======
Filters are tricky. They need to:
* work with a variety of the versions of the software that generates the logs;
* work with the range of logging configuration options available in the
software;
* work with multiple operating systems;
* not make assumptions about the log format in excess of the software
(e.g. do not assume a username doesn't contain spaces and use \S+ unless
you've checked the source code);
* account for how future versions of the software will log messages
(e.g. guess what would happen to the log message if different authentication
types are added);
* not be susceptible to DoS vulnerabilities (see Filter Security below); and
* match intended log lines only.
Please follow the steps from Filter Test Cases to Developing Filter Regular
Expressions and submit a GitHub pull request (PR) afterwards. If you get stuck,
you can push your unfinished changes and still submit a PR -- describe
what you have done, what is the hurdle, and we'll attempt to help (PR
will be automagically updated with future commits you would push to
complete it).
Filter test cases
-----------------
Purpose:
Start by finding the log messages that the application generates related to
some form of authentication failure. If you are adding to an existing filter
think about whether the log messages are of a similar importance and purpose
to the existing filter. If you were a user of Fail2Ban, and did a package
update of Fail2Ban that started matching new log messages, would anything
unexpected happen? Would the bantime/findtime for the jail be appropriate for
the new log messages? If it doesn't, perhaps it needs to be in a separate
filter definition, for example like exim filter aims at authentication failures
and exim-spam at log messages related to spam.
Even if it is a new filter you may consider separating the log messages into
different filters based on purpose.
Cause:
Are some of the log lines a result of the same action? For example, is a PAM
failure log message, followed by an application specific failure message the
result of the same user/script action? If you add regular expressions for
both you would end up with two failures for a single action.
Therefore, select the most appropriate log message and document the other log
message) with a test case not to match it and a description as to why you chose
one over another.
With the selected log lines consider what action has caused those log
messages and whether they could have been generated by accident? Could
the log message be occurring due to the first step towards the application
asking for authentication? Could the log messages occur often? If some of
these are true make a note of this in the jail.conf example that you provide.
Samples:
It is important to include log file samples so any future change in the regular
expression will still work with the log lines you have identified.
The sample log messages are provided in a file under testcases/files/logs/
named identically as the corresponding filter (but without .conf extension).
Each log line should be preceded by a line with failJSON metadata (so the logs
lines are tested in the test suite) directly above the log line. If there is
any specific information about the log message, such as version or an
application configuration option that is needed for the message to occur,
include this in a comment (line beginning with #) above the failJSON metadata.
Log samples should include only one, definitely not more than 3, examples of
log messages of the same form. If log messages are different in different
versions of the application log messages that show this are encouraged.
Also attempt to inject an IP into the application (e.g. by specifying
it as a username) so that Fail2Ban possibly detects the IP
from user input rather than the true origin. See the Filter Security section
and the top example in testcases/files/logs/apache-auth as to how to do this.
One you have discovered that this is possible, correct the regex so it doesn't
match and provide this as a test case with "match": false (see failJSON below).
If the mechanism to create the log message isn't obvious provide a
configuration and/or sample scripts testcases/files/config/{filtername} and
reference these in the comments above the log line.
FailJSON metadata:
A failJSON metadata is a comment immediately above the log message. It will
look like:
# failJSON: { "time": "2013-06-10T10:10:59", "match": true , "host": "93.184.216.119" }
Time should match the time of the log message. It is in a specific format of
Year-Month-Day'T'Hour:minute:Second. If your log message does not include a
year, like the example below, the year should be listed as 2005, if before Sun
Aug 14 10am UTC, and 2004 if afterwards. Here is an example failJSON
line preceding a sample log line:
# failJSON: { "time": "2005-03-24T15:25:51", "match": true , "host": "198.51.100.87" }
Mar 24 15:25:51 buffalo1 dropbear[4092]: bad password attempt for 'root' from 198.51.100.87:5543
The "host" in failJSON should contain the IP or domain that should be blocked.
For long lines that you do not want to be matched (e.g. from log injection
attacks) and any log lines to be excluded (see "Cause" section above), set
"match": false in the failJSON and describe the reason in the comment above.
After developing regexes, the following command will test all failJSON metadata
against the log lines in all sample log files
./fail2ban-testcases testSampleRegex
Developing Filter Regular Expressions
-------------------------------------
Date/Time:
At the moment, Fail2Ban depends on log lines to have time stamps. That is why
before starting to develop failregex, check if your log line format known to
Fail2Ban. Copy the time component from the log line and append an IP address to
test with following command:
./fail2ban-regex "2013-09-19 02:46:12 1.2.3.4" "<HOST>"
Output of such command should contain something like:
Date template hits:
|- [# of hits] date format
| [1] Year-Month-Day Hour:Minute:Second
Ensure that the template description matches time/date elements in your log line
time stamp. If there is no matched format then date template needs to be added
to server/datedetector.py. Ensure that a new template is added in the order
that more specific matches occur first and that there is no confusion between a
Day and a Month.
Filter file:
The filter is specified in a config/filter.d/{filtername}.conf file. Filter file
can have sections INCLUDES (optional) and Definition as follows:
[INCLUDES]
before = common.conf
after = filtername.local
[Definition]
failregex = ....
ignoreregex = ....
This is also documented in the man page jail.conf (section 5). Other definitions
can be added to make failregex's more readable and maintainable to be used
through string Interpolations (see http://docs.python.org/2.7/library/configparser.html)
General rules:
Use "before" if you need to include a common set of rules, like syslog or if
there is a common set of regexes for multiple filters.
Use "after" if you wish to allow the user to overwrite a set of customisations
of the current filter. This file doesn't need to exist.
Try to avoid using ignoreregex mainly for performance reasons. The case when you
would use it is if in trying to avoid using it, you end up with an unreadable
failregex.
Syslog:
If your application logs to syslog you can take advantage of log line prefix
definitions present in common.conf. So as a base use:
[INCLUDES]
before = common.conf
[Definition]
_daemon = app
failregex = ^%(__prefix_line)s
In this example common.conf defines __prefix_line which also contains the
_daemon name (in syslog terms the service) you have just specified. _daemon
can also be a regex.
For example, to capture following line _daemon should be set to "dovecot"
Dec 12 11:19:11 dunnart dovecot: pop3-login: Aborted login (tried to use disabled plaintext auth): rip=190.210.136.21, lip=113.212.99.193
and then ^%(__prefix_line)s would match "Dec 12 11:19:11 dunnart dovecot:
". Note it matches the trailing space(s) as well.
Substitutions (AKA string interpolations):
We have used string interpolations in above examples. They are useful for
making the regexes more readable, reuse generic patterns in multiple failregex
lines, and also to refer definition of regex parts to specific filters or even
to the user. General principle is that value of a _name variable replaces
occurrences of %(_name)s within the same section or anywhere in the config file
if defined in [DEFAULT] section.
Regular Expressions:
Regular expressions (failregex, ignoreregex) assume that the date/time has been
removed from the log line (this is just how fail2ban works internally ATM).
If the format is like '<date...> error 1.2.3.4 is evil' then you need to match
the < at the start so regex should be similar to '^<> <HOST> is evil$' using
<HOST> where the IP/domain name appears in the log line.
The following general rules apply to regular expressions:
* ensure regexes start with a ^ and are as restrictive as possible. E.g. do not
use .* if \d+ is sufficient;
* use functionality of Python regexes defined in the standard Python re library
http://docs.python.org/2/library/re.html;
* make regular expressions readable (as much as possible). E.g.
(?:...) represents a non-capturing regex but (...) is more readable, thus
preferred.
If you have only a basic knowledge of regular repressions we advise to read
http://docs.python.org/2/library/re.html first. It doesn't take long and would
remind you e.g. which characters you need to escape and which you don't.
Developing/testing a regex:
You can develop a regex in a file or using command line depending on your
preference. You can also use samples you have already created in the test cases
or test them one at a time.
The general tool for testing Fail2Ban regexes is fail2ban-regex. To see how to
use it run:
./fail2ban-regex --help
Take note of -l heavydebug / -l debug and -v as they might be very useful.
TIP: Take a look at the source code of the application you are developing
failregex for. You may see optional or extra log messages, or parts there
of, that need to form part of your regex. It may also reveal how some
parts are constrained and different formats depending on configuration or
less common usages.
TIP: For looking through source code - http://sourcecodebrowser.com/ . It has
call graphs and can browse different versions.
TIP: Some applications log spaces at the end. If you are not sure add \s*$ as
the end part of the regex.
If your regex is not matching, http://www.debuggex.com/?flavor=python can help
to tune it. fail2ban-regex -D ... will present Debuggex URLs for the regexs
and sample log files that you pass into it.
In general use when using regex debuggers for generating fail2ban filters:
* use regex from the ./fail2ban-regex output (to ensure all substitutions are
done)
* replace <HOST> with (?&.ipv4)
* make sure that regex type set to Python
* for the test data put your log output with the date/time removed
When you have fixed the regex put it back into your filter file.
Please spread the good word about Debuggex - Serge Toarca is kindly continuing
its free availability to Open Source developers.
Finishing up:
If you've added a new filter, add a new entry in config/jail.conf. The theory
here is that a user will create a jail.local with [filtername]\nenable=true to
enable your jail.
So more specifically in the [filter] section in jail.conf:
* ensure that you have "enabled = false" (users will enable as needed);
* use "filter =" set to your filter name;
* use a typical action to disable ports associated with the application;
* set "logpath" to the usual location of application log file;
* if the default findtime or bantime isn't appropriate to the filter, specify
more appropriate choices (possibly with a brief comment line).
Submit github pull request (See "Pull Requests" above) for
github.com/fail2ban/fail2ban containing your great work.
Filter Security
---------------
Poor filter regular expressions are susceptible to DoS attacks.
When a remote user has the ability to introduce text that would match filter's
failregex, while matching inserted text to the <HOST> part, they have the
ability to deny any host they choose.
So the <HOST> part must be anchored on text generated by the application, and
not the user, to an extent sufficient to prevent user inserting the entire text
matching this or any other failregex.
Ideally filter regex should anchor at the beginning and at the end of log line.
However as more applications log at the beginning than the end, anchoring the
beginning is more important. If the log file used by the application is shared
with other applications, like system logs, ensure the other application that use
that log file do not log user generated text at the beginning of the line, or,
if they do, ensure the regexes of the filter are sufficient to mitigate the risk
of insertion.
Examples of poor filters
------------------------
1. Too restrictive
We find a log message:
Apr-07-13 07:08:36 Invalid command fial2ban from 1.2.3.4
We make a failregex
^Invalid command \S+ from <HOST>
Now think evil. The user does the command 'blah from 1.2.3.44'
The program diligently logs:
Apr-07-13 07:08:36 Invalid command blah from 1.2.3.44 from 1.2.3.4
And fail2ban matches 1.2.3.44 as the IP that it ban. A DoS attack was successful.
The fix here is that the command can be anything so .* is appropriate.
^Invalid command .* from <HOST>
Here the .* will match until the end of the string. Then realise it has more to
match, i.e. "from <HOST>" and go back until it find this. Then it will ban
1.2.3.4 correctly. Since the <HOST> is always at the end, end the regex with a $.
^Invalid command .* from <HOST>$
Note if we'd just had the expression:
^Invalid command \S+ from <HOST>$
Then provided the user put a space in their command they would have never been
banned.
2. Unanchored regex can match other user injected data
From the Apache vulnerability CVE-2013-2178
( original ref: https://vndh.net/note:fail2ban-089-denial-service ).
An example bad regex for Apache:
failregex = [[]client <HOST>[]] user .* not found
Since the user can do a get request on:
GET /[client%20192.168.0.1]%20user%20root%20not%20found HTTP/1.0
Host: remote.site
Now the log line will be:
[Sat Jun 01 02:17:42 2013] [error] [client 192.168.33.1] File does not exist: /srv/http/site/[client 192.168.0.1] user root not found
As this log line doesn't match other expressions hence it matches the above
regex and blocks 192.168.33.1 as a denial of service from the HTTP requester.
3. Over greedy pattern matching
From: https://github.com/fail2ban/fail2ban/pull/426
An example ssh log (simplified)
Sep 29 17:15:02 spaceman sshd[12946]: Failed password for user from 127.0.0.1 port 20000 ssh1: ruser remoteuser
As we assume username can include anything including spaces its prudent to put
.* here. The remote user can also exist as anything so lets not make assumptions again.
failregex = ^%(__prefix_line)sFailed \S+ for .* from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$
So this works. The problem is if the .* after remote user is injected by the
user to be 'from 1.2.3.4'. The resultant log line is.
Sep 29 17:15:02 spaceman sshd[12946]: Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4
Testing with:
fail2ban-regex -v 'Sep 29 17:15:02 Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4' '^ Failed \S+ for .* from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$'
TIP: I've removed the bit that matches __prefix_line from the regex and log.
Shows:
1) [1] ^ Failed \S+ for .* from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$
1.2.3.4 Sun Sep 29 17:15:02 2013
It should of matched 127.0.0.1. So the first greedy part of the greedy regex
matched until the end of the string. The was no "from <HOST>" so the regex
engine worked backwards from the end of the string until this was matched.
The result was that 1.2.3.4 was matched, injected by the user, and the wrong IP
was banned.
The solution here is to make the first .* non-greedy with .*?. Here it matches
as little as required and the fail2ban-regex tool shows the output:
fail2ban-regex -v 'Sep 29 17:15:02 Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4' '^ Failed \S+ for .*? from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$'
1) [1] ^ Failed \S+ for .*? from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$
127.0.0.1 Sun Sep 29 17:15:02 2013
So the general case here is a log line that contains:
(fixed_data_1)<HOST>(fixed_data_2)(user_injectable_data)
Where the regex that matches fixed_data_1 is gready and matches the entire
string, before moving backwards and user_injectable_data can match the entire
string.
Another case:
ref: https://www.debuggex.com/r/CtAbeKMa2sDBEfA2/0
A webserver logs the following without URL escaping:
[error] 2865#0: *66647 user "xyz" was not found in "/file", client: 1.2.3.1, server: www.host.com, request: "GET ", client: 3.2.1.1, server: fake.com, request: "GET exploited HTTP/3.3", host: "injected.host", host: "www.myhost.com"
regex:
failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (?:password mismatch|was not found in ".*"), client: <HOST>, server: \S+, request: "\S+ .+ HTTP/\d+\.\d+", host: "\S+"
The .* matches to the end of the string. Finds that it can't continue to match
", client ... so it moves from the back and find that the user injected web URL:
", client: 3.2.1.1, server: fake.com, request: "GET exploited HTTP/3.3", host: "injected.host
In this case there is a fixed host: "www.myhost.com" at the end so the solution
is to anchor the regex at the end with a $.
If this wasn't the case then first .* needed to be made so it didn't capture
beyond <HOST>.
4. Application generates two identical log messages with different meanings
If the application generates the following two messages under different
circumstances:
client <IP>: authentication failed
client <USER>: authentication failed
Then it's obvious that a regex of "^client <HOST>: authentication
failed$" will still cause problems if the user can trigger the second
log message with a <USER> of 123.1.1.1.
Here there's nothing to do except request/change the application so it logs
messages differently.
If you are developing filters see the FILTERS file for documentation.
Code Testing
============
@ -743,10 +285,14 @@ Releasing
* https://github.com/fail2ban/fail2ban/issues?sort=updated&state=open
* http://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;package=fail2ban
* https://bugs.launchpad.net/ubuntu/+source/fail2ban
* http://bugs.sabayon.org/buglist.cgi?quicksearch=net-analyzer%2Ffail2ban
* https://bugs.archlinux.org/?project=5&cat%5B%5D=33&string=fail2ban
* https://bugs.gentoo.org/buglist.cgi?query_format=advanced&short_desc=fail2ban&bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&short_desc_type=allwords
* https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&component=fail2ban&classification=Red%20Hat&classification=Fedora
* http://www.freebsd.org/cgi/query-pr-summary.cgi?text=fail2ban
* https://bugs.mageia.org/buglist.cgi?quicksearch=fail2ban
* https://build.opensuse.org/package/requests/openSUSE:Factory/fail2ban
# Make sure the tests pass

469
FILTERS Normal file
View File

@ -0,0 +1,469 @@
__ _ _ ___ _
/ _|__ _(_) |_ ) |__ __ _ _ _
| _/ _` | | |/ /| '_ \/ _` | ' \
|_| \__,_|_|_/___|_.__/\__,_|_||_|
================================================================================
Developing Filters
================================================================================
Filters
=======
Filters are tricky. They need to:
* work with a variety of the versions of the software that generates the logs;
* work with the range of logging configuration options available in the
software;
* work with multiple operating systems;
* not make assumptions about the log format in excess of the software
(e.g. do not assume a username doesn't contain spaces and use \S+ unless
you've checked the source code);
* account for how future versions of the software will log messages
(e.g. guess what would happen to the log message if different authentication
types are added);
* not be susceptible to DoS vulnerabilities (see Filter Security below); and
* match intended log lines only.
Please follow the steps from Filter Test Cases to Developing Filter Regular
Expressions and submit a GitHub pull request (PR) afterwards. If you get stuck,
you can push your unfinished changes and still submit a PR -- describe
what you have done, what is the hurdle, and we'll attempt to help (PR
will be automagically updated with future commits you would push to
complete it).
Filter test cases
-----------------
Purpose:
Start by finding the log messages that the application generates related to
some form of authentication failure. If you are adding to an existing filter
think about whether the log messages are of a similar importance and purpose
to the existing filter. If you were a user of Fail2Ban, and did a package
update of Fail2Ban that started matching new log messages, would anything
unexpected happen? Would the bantime/findtime for the jail be appropriate for
the new log messages? If it doesn't, perhaps it needs to be in a separate
filter definition, for example like exim filter aims at authentication failures
and exim-spam at log messages related to spam.
Even if it is a new filter you may consider separating the log messages into
different filters based on purpose.
Cause:
Are some of the log lines a result of the same action? For example, is a PAM
failure log message, followed by an application specific failure message the
result of the same user/script action? If you add regular expressions for
both you would end up with two failures for a single action.
Therefore, select the most appropriate log message and document the other log
message) with a test case not to match it and a description as to why you chose
one over another.
With the selected log lines consider what action has caused those log
messages and whether they could have been generated by accident? Could
the log message be occurring due to the first step towards the application
asking for authentication? Could the log messages occur often? If some of
these are true make a note of this in the jail.conf example that you provide.
Samples:
It is important to include log file samples so any future change in the regular
expression will still work with the log lines you have identified.
The sample log messages are provided in a file under testcases/files/logs/
named identically as the corresponding filter (but without .conf extension).
Each log line should be preceded by a line with failJSON metadata (so the logs
lines are tested in the test suite) directly above the log line. If there is
any specific information about the log message, such as version or an
application configuration option that is needed for the message to occur,
include this in a comment (line beginning with #) above the failJSON metadata.
Log samples should include only one, definitely not more than 3, examples of
log messages of the same form. If log messages are different in different
versions of the application log messages that show this are encouraged.
Also attempt to inject an IP into the application (e.g. by specifying
it as a username) so that Fail2Ban possibly detects the IP
from user input rather than the true origin. See the Filter Security section
and the top example in testcases/files/logs/apache-auth as to how to do this.
One you have discovered that this is possible, correct the regex so it doesn't
match and provide this as a test case with "match": false (see failJSON below).
If the mechanism to create the log message isn't obvious provide a
configuration and/or sample scripts testcases/files/config/{filtername} and
reference these in the comments above the log line.
FailJSON metadata:
A failJSON metadata is a comment immediately above the log message. It will
look like:
# failJSON: { "time": "2013-06-10T10:10:59", "match": true , "host": "93.184.216.119" }
Time should match the time of the log message. It is in a specific format of
Year-Month-Day'T'Hour:minute:Second. If your log message does not include a
year, like the example below, the year should be listed as 2005, if before Sun
Aug 14 10am UTC, and 2004 if afterwards. Here is an example failJSON
line preceding a sample log line:
# failJSON: { "time": "2005-03-24T15:25:51", "match": true , "host": "198.51.100.87" }
Mar 24 15:25:51 buffalo1 dropbear[4092]: bad password attempt for 'root' from 198.51.100.87:5543
The "host" in failJSON should contain the IP or domain that should be blocked.
For long lines that you do not want to be matched (e.g. from log injection
attacks) and any log lines to be excluded (see "Cause" section above), set
"match": false in the failJSON and describe the reason in the comment above.
After developing regexes, the following command will test all failJSON metadata
against the log lines in all sample log files
./fail2ban-testcases testSampleRegex
Developing Filter Regular Expressions
-------------------------------------
Date/Time:
At the moment, Fail2Ban depends on log lines to have time stamps. That is why
before starting to develop failregex, check if your log line format known to
Fail2Ban. Copy the time component from the log line and append an IP address to
test with following command:
./fail2ban-regex "2013-09-19 02:46:12 1.2.3.4" "<HOST>"
Output of such command should contain something like:
Date template hits:
|- [# of hits] date format
| [1] Year-Month-Day Hour:Minute:Second
Ensure that the template description matches time/date elements in your log line
time stamp. If there is no matched format then date template needs to be added
to server/datedetector.py. Ensure that a new template is added in the order
that more specific matches occur first and that there is no confusion between a
Day and a Month.
Filter file:
The filter is specified in a config/filter.d/{filtername}.conf file. Filter file
can have sections INCLUDES (optional) and Definition as follows:
[INCLUDES]
before = common.conf
after = filtername.local
[Definition]
failregex = ....
ignoreregex = ....
This is also documented in the man page jail.conf (section 5). Other definitions
can be added to make failregex's more readable and maintainable to be used
through string Interpolations (see http://docs.python.org/2.7/library/configparser.html)
General rules:
Use "before" if you need to include a common set of rules, like syslog or if
there is a common set of regexes for multiple filters.
Use "after" if you wish to allow the user to overwrite a set of customisations
of the current filter. This file doesn't need to exist.
Try to avoid using ignoreregex mainly for performance reasons. The case when you
would use it is if in trying to avoid using it, you end up with an unreadable
failregex.
Syslog:
If your application logs to syslog you can take advantage of log line prefix
definitions present in common.conf. So as a base use:
[INCLUDES]
before = common.conf
[Definition]
_daemon = app
failregex = ^%(__prefix_line)s
In this example common.conf defines __prefix_line which also contains the
_daemon name (in syslog terms the service) you have just specified. _daemon
can also be a regex.
For example, to capture following line _daemon should be set to "dovecot"
Dec 12 11:19:11 dunnart dovecot: pop3-login: Aborted login (tried to use disabled plaintext auth): rip=190.210.136.21, lip=113.212.99.193
and then ^%(__prefix_line)s would match "Dec 12 11:19:11 dunnart dovecot:
". Note it matches the trailing space(s) as well.
Substitutions (AKA string interpolations):
We have used string interpolations in above examples. They are useful for
making the regexes more readable, reuse generic patterns in multiple failregex
lines, and also to refer definition of regex parts to specific filters or even
to the user. General principle is that value of a _name variable replaces
occurrences of %(_name)s within the same section or anywhere in the config file
if defined in [DEFAULT] section.
Regular Expressions:
Regular expressions (failregex, ignoreregex) assume that the date/time has been
removed from the log line (this is just how fail2ban works internally ATM).
If the format is like '<date...> error 1.2.3.4 is evil' then you need to match
the < at the start so regex should be similar to '^<> <HOST> is evil$' using
<HOST> where the IP/domain name appears in the log line.
The following general rules apply to regular expressions:
* ensure regexes start with a ^ and are as restrictive as possible. E.g. do not
use .* if \d+ is sufficient;
* use functionality of Python regexes defined in the standard Python re library
http://docs.python.org/2/library/re.html;
* make regular expressions readable (as much as possible). E.g.
(?:...) represents a non-capturing regex but (...) is more readable, thus
preferred.
If you have only a basic knowledge of regular repressions we advise to read
http://docs.python.org/2/library/re.html first. It doesn't take long and would
remind you e.g. which characters you need to escape and which you don't.
Developing/testing a regex:
You can develop a regex in a file or using command line depending on your
preference. You can also use samples you have already created in the test cases
or test them one at a time.
The general tool for testing Fail2Ban regexes is fail2ban-regex. To see how to
use it run:
./fail2ban-regex --help
Take note of -l heavydebug / -l debug and -v as they might be very useful.
TIP: Take a look at the source code of the application you are developing
failregex for. You may see optional or extra log messages, or parts there
of, that need to form part of your regex. It may also reveal how some
parts are constrained and different formats depending on configuration or
less common usages.
TIP: For looking through source code - http://sourcecodebrowser.com/ . It has
call graphs and can browse different versions.
TIP: Some applications log spaces at the end. If you are not sure add \s*$ as
the end part of the regex.
If your regex is not matching, http://www.debuggex.com/?flavor=python can help
to tune it. fail2ban-regex -D ... will present Debuggex URLs for the regexs
and sample log files that you pass into it.
In general use when using regex debuggers for generating fail2ban filters:
* use regex from the ./fail2ban-regex output (to ensure all substitutions are
done)
* replace <HOST> with (?&.ipv4)
* make sure that regex type set to Python
* for the test data put your log output with the date/time removed
When you have fixed the regex put it back into your filter file.
Please spread the good word about Debuggex - Serge Toarca is kindly continuing
its free availability to Open Source developers.
Finishing up:
If you've added a new filter, add a new entry in config/jail.conf. The theory
here is that a user will create a jail.local with [filtername]\nenable=true to
enable your jail.
So more specifically in the [filter] section in jail.conf:
* ensure that you have "enabled = false" (users will enable as needed);
* use "filter =" set to your filter name;
* use a typical action to disable ports associated with the application;
* set "logpath" to the usual location of application log file;
* if the default findtime or bantime isn't appropriate to the filter, specify
more appropriate choices (possibly with a brief comment line).
Submit github pull request (See "Pull Requests" above) for
github.com/fail2ban/fail2ban containing your great work.
Filter Security
---------------
Poor filter regular expressions are susceptible to DoS attacks.
When a remote user has the ability to introduce text that would match filter's
failregex, while matching inserted text to the <HOST> part, they have the
ability to deny any host they choose.
So the <HOST> part must be anchored on text generated by the application, and
not the user, to an extent sufficient to prevent user inserting the entire text
matching this or any other failregex.
Ideally filter regex should anchor at the beginning and at the end of log line.
However as more applications log at the beginning than the end, anchoring the
beginning is more important. If the log file used by the application is shared
with other applications, like system logs, ensure the other application that use
that log file do not log user generated text at the beginning of the line, or,
if they do, ensure the regexes of the filter are sufficient to mitigate the risk
of insertion.
Examples of poor filters
------------------------
1. Too restrictive
We find a log message:
Apr-07-13 07:08:36 Invalid command fial2ban from 1.2.3.4
We make a failregex
^Invalid command \S+ from <HOST>
Now think evil. The user does the command 'blah from 1.2.3.44'
The program diligently logs:
Apr-07-13 07:08:36 Invalid command blah from 1.2.3.44 from 1.2.3.4
And fail2ban matches 1.2.3.44 as the IP that it ban. A DoS attack was successful.
The fix here is that the command can be anything so .* is appropriate.
^Invalid command .* from <HOST>
Here the .* will match until the end of the string. Then realise it has more to
match, i.e. "from <HOST>" and go back until it find this. Then it will ban
1.2.3.4 correctly. Since the <HOST> is always at the end, end the regex with a $.
^Invalid command .* from <HOST>$
Note if we'd just had the expression:
^Invalid command \S+ from <HOST>$
Then provided the user put a space in their command they would have never been
banned.
2. Unanchored regex can match other user injected data
From the Apache vulnerability CVE-2013-2178
( original ref: https://vndh.net/note:fail2ban-089-denial-service ).
An example bad regex for Apache:
failregex = [[]client <HOST>[]] user .* not found
Since the user can do a get request on:
GET /[client%20192.168.0.1]%20user%20root%20not%20found HTTP/1.0
Host: remote.site
Now the log line will be:
[Sat Jun 01 02:17:42 2013] [error] [client 192.168.33.1] File does not exist: /srv/http/site/[client 192.168.0.1] user root not found
As this log line doesn't match other expressions hence it matches the above
regex and blocks 192.168.33.1 as a denial of service from the HTTP requester.
3. Over greedy pattern matching
From: https://github.com/fail2ban/fail2ban/pull/426
An example ssh log (simplified)
Sep 29 17:15:02 spaceman sshd[12946]: Failed password for user from 127.0.0.1 port 20000 ssh1: ruser remoteuser
As we assume username can include anything including spaces its prudent to put
.* here. The remote user can also exist as anything so lets not make assumptions again.
failregex = ^%(__prefix_line)sFailed \S+ for .* from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$
So this works. The problem is if the .* after remote user is injected by the
user to be 'from 1.2.3.4'. The resultant log line is.
Sep 29 17:15:02 spaceman sshd[12946]: Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4
Testing with:
fail2ban-regex -v 'Sep 29 17:15:02 Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4' '^ Failed \S+ for .* from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$'
TIP: I've removed the bit that matches __prefix_line from the regex and log.
Shows:
1) [1] ^ Failed \S+ for .* from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$
1.2.3.4 Sun Sep 29 17:15:02 2013
It should of matched 127.0.0.1. So the first greedy part of the greedy regex
matched until the end of the string. The was no "from <HOST>" so the regex
engine worked backwards from the end of the string until this was matched.
The result was that 1.2.3.4 was matched, injected by the user, and the wrong IP
was banned.
The solution here is to make the first .* non-greedy with .*?. Here it matches
as little as required and the fail2ban-regex tool shows the output:
fail2ban-regex -v 'Sep 29 17:15:02 Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4' '^ Failed \S+ for .*? from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$'
1) [1] ^ Failed \S+ for .*? from <HOST>( port \d*)?( ssh\d+)?(: ruser .*)?$
127.0.0.1 Sun Sep 29 17:15:02 2013
So the general case here is a log line that contains:
(fixed_data_1)<HOST>(fixed_data_2)(user_injectable_data)
Where the regex that matches fixed_data_1 is gready and matches the entire
string, before moving backwards and user_injectable_data can match the entire
string.
Another case:
ref: https://www.debuggex.com/r/CtAbeKMa2sDBEfA2/0
A webserver logs the following without URL escaping:
[error] 2865#0: *66647 user "xyz" was not found in "/file", client: 1.2.3.1, server: www.host.com, request: "GET ", client: 3.2.1.1, server: fake.com, request: "GET exploited HTTP/3.3", host: "injected.host", host: "www.myhost.com"
regex:
failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (?:password mismatch|was not found in ".*"), client: <HOST>, server: \S+, request: "\S+ .+ HTTP/\d+\.\d+", host: "\S+"
The .* matches to the end of the string. Finds that it can't continue to match
", client ... so it moves from the back and find that the user injected web URL:
", client: 3.2.1.1, server: fake.com, request: "GET exploited HTTP/3.3", host: "injected.host
In this case there is a fixed host: "www.myhost.com" at the end so the solution
is to anchor the regex at the end with a $.
If this wasn't the case then first .* needed to be made so it didn't capture
beyond <HOST>.
4. Application generates two identical log messages with different meanings
If the application generates the following two messages under different
circumstances:
client <IP>: authentication failed
client <USER>: authentication failed
Then it's obvious that a regex of "^client <HOST>: authentication
failed$" will still cause problems if the user can trigger the second
log message with a <USER> of 123.1.1.1.
Here there's nothing to do except request/change the application so it logs
messages differently.

62
MANIFEST
View File

@ -5,8 +5,9 @@ TODO
THANKS
COPYING
DEVELOP
doc/run-rootless.txt
FILTERS
fail2ban-2to3
fail2ban-testcases-all
fail2ban-testcases-all-python3
bin/fail2ban-client
bin/fail2ban-server
@ -24,6 +25,7 @@ fail2ban/client/__init__.py
fail2ban/client/configurator.py
fail2ban/client/csocket.py
fail2ban/server/asyncserver.py
fail2ban/server/database.py
fail2ban/server/filter.py
fail2ban/server/filterpyinotify.py
fail2ban/server/filtergamin.py
@ -59,22 +61,30 @@ fail2ban/tests/servertestcase.py
fail2ban/tests/sockettestcase.py
fail2ban/tests/utils.py
fail2ban/tests/misctestcase.py
fail2ban/tests/config/apache-auth/digest/.htaccess
fail2ban/tests/config/apache-auth/digest/.htpasswd
fail2ban/tests/config/apache-auth/digest_time/.htaccess
fail2ban/tests/config/apache-auth/digest_time/.htpasswd
fail2ban/tests/config/apache-auth/basic/authz_owner/.htaccess
fail2ban/tests/config/apache-auth/basic/authz_owner/cant_get_me.html
fail2ban/tests/config/apache-auth/basic/authz_owner/.htpasswd
fail2ban/tests/config/apache-auth/basic/file/.htaccess
fail2ban/tests/config/apache-auth/basic/file/.htpasswd
fail2ban/tests/config/apache-auth/digest.py
fail2ban/tests/config/apache-auth/digest_wrongrelm/.htaccess
fail2ban/tests/config/apache-auth/digest_wrongrelm/.htpasswd
fail2ban/tests/config/apache-auth/digest_anon/.htaccess
fail2ban/tests/config/apache-auth/digest_anon/.htpasswd
fail2ban/tests/config/apache-auth/README
fail2ban/tests/config/apache-auth/noentry/.htaccess
fail2ban/tests/config/jail.conf
fail2ban/tests/config/fail2ban.conf
fail2ban/tests/config/filter.d/simple.conf
fail2ban/tests/config/action.d/brokenaction.conf
fail2ban/tests/files/config/apache-auth/digest/.htaccess
fail2ban/tests/files/config/apache-auth/digest/.htpasswd
fail2ban/tests/files/config/apache-auth/digest_time/.htaccess
fail2ban/tests/files/config/apache-auth/digest_time/.htpasswd
fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htaccess
fail2ban/tests/files/config/apache-auth/basic/authz_owner/cant_get_me.html
fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htpasswd
fail2ban/tests/files/config/apache-auth/basic/file/.htaccess
fail2ban/tests/files/config/apache-auth/basic/file/.htpasswd
fail2ban/tests/files/config/apache-auth/digest.py
fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htaccess
fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htpasswd
fail2ban/tests/files/config/apache-auth/digest_anon/.htaccess
fail2ban/tests/files/config/apache-auth/digest_anon/.htpasswd
fail2ban/tests/files/config/apache-auth/README
fail2ban/tests/files/config/apache-auth/noentry/.htaccess
fail2ban/tests/files/database_v1.db
fail2ban/tests/files/ignorecommand.py
fail2ban/tests/files/filter.d/testcase-common.conf
fail2ban/tests/files/filter.d/testcase01.conf
fail2ban/tests/files/testcase01.log
fail2ban/tests/files/testcase02.log
fail2ban/tests/files/testcase03.log
@ -92,6 +102,7 @@ fail2ban/tests/files/logs/dovecot
fail2ban/tests/files/logs/exim
fail2ban/tests/files/logs/nginx-http-auth
fail2ban/tests/files/logs/lighttpd-auth
fail2ban/tests/files/logs/openwebmail
fail2ban/tests/files/logs/named-refused
fail2ban/tests/files/logs/pam-generic
fail2ban/tests/files/logs/postfix
@ -144,6 +155,7 @@ setup.py
setup.cfg
kill-server
config/jail.conf
config/fail2ban.conf
config/filter.d/common.conf
config/filter.d/apache-auth.conf
config/filter.d/apache-badbots.conf
@ -158,13 +170,17 @@ config/filter.d/exim.conf
config/filter.d/gssftpd.conf
config/filter.d/suhosin.conf
config/filter.d/named-refused.conf
config/filter.d/openwebmail.conf
config/filter.d/pam-generic.conf
config/filter.d/php-url-fopen.conf
config/filter.d/postfix-sasl.conf
config/filter.d/pam-generic.conf
config/filter.d/php-url-fopen.conf
config/filter.d/postfix-sasl.conf
config/filter.d/postfix.conf
config/filter.d/proftpd.conf
config/filter.d/pure-ftpd.conf
config/filter.d/qmail.conf
config/filter.d/pam-generic.conf
config/filter.d/php-url-fopen.conf
config/filter.d/postfix-sasl.conf
config/filter.d/sieve.conf
config/filter.d/solid-pop3d.conf
config/filter.d/sshd.conf
@ -201,7 +217,8 @@ config/action.d/osx-ipfw.conf
config/action.d/sendmail-common.conf
config/action.d/bsd-ipfw.conf
config/action.d/dummy.conf
config/action.d/firewall-cmd-direct-new.conf
config/action.d/firewallcmd-new.conf
config/action.d/firewallcmd-ipset.conf
config/action.d/iptables-ipset-proto6-allports.conf
config/action.d/iptables-blocktype.conf
config/action.d/iptables-ipset-proto4.conf
@ -229,7 +246,8 @@ config/action.d/sendmail-buffered.conf
config/action.d/sendmail-whois.conf
config/action.d/sendmail-whois-lines.conf
config/action.d/shorewall.conf
config/fail2ban.conf
config/action.d/xarf-login-attack.conf
config/action.d/ufw.conf
doc/run-rootless.txt
man/fail2ban-client.1
man/fail2ban.1

8
README.md
View File

@ -2,7 +2,7 @@
/ _|__ _(_) |_ ) |__ __ _ _ _
| _/ _` | | |/ /| '_ \/ _` | ' \
|_| \__,_|_|_/___|_.__/\__,_|_||_|
v0.9.0a0 2013/??/??
v0.9.0a2 2014/??/??
## Fail2Ban: ban hosts that cause multiple authentication errors
@ -21,7 +21,7 @@ Installation:
this case, you should use it instead.**
Required:
- [Python2 >= 2.4 or Python3 >= 3.2](http://www.python.org)
- [Python2 >= 2.6 or Python3 >= 3.2](http://www.python.org) or [PyPy](http://pypy.org)
Optional:
- [pyinotify >= 0.8.3](https://github.com/seb-m/pyinotify)
@ -31,8 +31,8 @@ Optional:
To install, just do:
tar xvfj fail2ban-0.8.11.tar.bz2
cd fail2ban-0.8.11
tar xvfj fail2ban-0.9.0.tar.bz2
cd fail2ban-0.9.0
python setup.py install
This will install Fail2Ban into /usr/share/fail2ban. The executable scripts are

13
THANKS
View File

@ -6,13 +6,17 @@ the project. If you have been left off, please let us know
(preferably send a pull request on github with the "fix") and you will
be added
Adam Tkac
Adrien Clerc
ache
ag4ve (Shawn)
Alasdair D. Campbell
Amir Caspi
Andrey G. Grozin
Andy Fragen
Arturo 'Buanzo' Busleiman
Axel Thimm
Bas van den Dikkenberg
Beau Raines
Bill Heaton
Carlos Alberto Lopez Perez
@ -22,6 +26,7 @@ Christoph Haas
Christos Psonis
Cyril Jaquier
Daniel B. Cid
Daniel B.
Daniel Black
David Nutter
Eric Gerbier
@ -30,10 +35,14 @@ ftoppi
François Boulogne
Frédéric
Georgiy Mernov
Guilhem Lettron
Guillaume Delvit
Hanno 'Rince' Wagner
Iain Lea
Ivo Truxa
John Thoe
Jacques Lav!gnotte
Ioan Indreias
Jonathan Kamens
Jonathan Lanning
Jonathan Underwood
@ -43,10 +52,12 @@ Justin Shore
Kévin Drapel
kjohnsonecl
kojiro
Lee Clemens
Manuel Arostegui Ramirez
Marcel Dopita
Mark Edgington
Mark McKinstry
Mark White
Markus Hoffmann
Marvin Rouge
mEDI
@ -60,10 +71,12 @@ RealRancor
René Berber
Robert Edeker
Rolf Fokkens
Roman Gelfand
Russell Odom
Sebastian Arcus
Sireyessire
silviogarbes
Stefan Tatschner
Stephen Gildea
Steven Hiscocks
TESTOVIK

75
bin/fail2ban-regex
View File

@ -98,7 +98,7 @@ def get_opt_parser():
LOG:
string a string representing a log line
filename path to a log file (/var/log/auth.log)
"systemd-journal" search systemd journal (systemd-python required)
"systemd-journal" search systemd journal (systemd-python required)
REGEX:
string a string representing a 'failregex'
@ -239,7 +239,9 @@ class Fail2banRegex(object):
if not self._datepattern_set:
self._filter.setDatePattern(pattern)
self._datepattern_set = True
print "Use datepattern : %s" % self._filter.getDatePattern()[1]
if pattern is not None:
print "Use datepattern : %s" % (
self._filter.getDatePattern()[1], )
def setMaxLines(self, v):
if not self._maxlines_set:
@ -309,7 +311,7 @@ class Fail2banRegex(object):
def testIgnoreRegex(self, line):
found = False
try:
ret = self._filter.ignoreLine(line)
ret = self._filter.ignoreLine([(line, "", "")])
if ret is not None:
found = True
regex = self._ignoreregex[ret].inc()
@ -318,11 +320,11 @@ class Fail2banRegex(object):
return False
return found
def testRegex(self, line):
def testRegex(self, line, date=None):
orgLineBuffer = self._filter._Filter__lineBuffer
fullBuffer = len(orgLineBuffer) >= self._filter.getMaxLines()
try:
line, ret = self._filter.processLine(line, checkAllRegex=True)
line, ret = self._filter.processLine(line, date, checkAllRegex=True)
for match in ret:
# Append True/False flag depending if line was matched by
# more than one regex
@ -338,35 +340,32 @@ class Fail2banRegex(object):
return False
for bufLine in orgLineBuffer[int(fullBuffer):]:
if bufLine not in self._filter._Filter__lineBuffer:
if self.removeMissedLine(bufLine):
try:
self._line_stats.missed_lines.pop(
self._line_stats.missed_lines.index("".join(bufLine)))
self._line_stats.missed_lines_timeextracted.pop(
self._line_stats.missed_lines_timeextracted.index(
"".join(bufLine[::2])))
except ValueError:
pass
else:
self._line_stats.matched += 1
return line, ret
def removeMissedLine(self, line):
"""Remove `line` from missed lines, by comparing without time match"""
for n, missed_line in \
enumerate(reversed(self._line_stats.missed_lines)):
timeMatch = self._filter.dateDetector.matchTime(
missed_line, incHits=False)
if timeMatch:
logLine = (missed_line[:timeMatch.start()] +
missed_line[timeMatch.end():])
else:
logLine = missed_line
if logLine.rstrip("\r\n") == line:
self._line_stats.missed_lines.pop(
len(self._line_stats.missed_lines) - n - 1)
return True
return False
def process(self, test_lines):
for line_no, line in enumerate(test_lines):
if line.startswith('#') or not line.strip():
# skip comment and empty lines
continue
is_ignored = fail2banRegex.testIgnoreRegex(line)
line_datetimestripped, ret = fail2banRegex.testRegex(line)
if isinstance(line, tuple):
line_datetimestripped, ret = fail2banRegex.testRegex(
line[0], line[1])
line = "".join(line[0])
else:
line = line.rstrip('\r\n')
if line.startswith('#') or not line:
# skip comment and empty lines
continue
line_datetimestripped, ret = fail2banRegex.testRegex(line)
is_ignored = fail2banRegex.testIgnoreRegex(line_datetimestripped)
if is_ignored:
self._line_stats.ignored_lines.append(line)
@ -381,7 +380,7 @@ class Fail2banRegex(object):
self._line_stats.missed_lines_timeextracted.append(line_datetimestripped)
self._line_stats.tested += 1
if line_no % 10 == 0:
if line_no % 10 == 0 and self._filter.dateDetector is not None:
self._filter.dateDetector.sortTemplate()
@ -436,7 +435,7 @@ class Fail2banRegex(object):
" %s %s%s" % (
ip[1],
timeString,
ip[3] and " (multiple regex matched)" or ""))
ip[-1] and " (multiple regex matched)" or ""))
print "\n%s: %d total" % (title, total)
pprint_list(out, " #) [# of hits] regular expression")
@ -447,12 +446,14 @@ class Fail2banRegex(object):
_ = print_failregexes("Ignoreregex", self._ignoreregex)
print "\nDate template hits:"
out = []
for template in self._filter.dateDetector.getTemplates():
if self._verbose or template.getHits():
out.append("[%d] %s" % (template.getHits(), template.getName()))
pprint_list(out, "[# of hits] date format")
if self._filter.dateDetector is not None:
print "\nDate template hits:"
out = []
for template in self._filter.dateDetector.getTemplates():
if self._verbose or template.getHits():
out.append("[%d] %s" % (
template.getHits(), template.getName()))
pprint_list(out, "[# of hits] date format")
print "\nLines: %s" % self._line_stats
@ -531,7 +532,7 @@ if __name__ == "__main__":
sys.exit(-1)
myjournal = journal.Reader(converters={'__CURSOR': lambda x: x})
journalmatch = fail2banRegex._journalmatch
fail2banRegex.setDatePattern("ISO8601")
fail2banRegex.setDatePattern(None)
if journalmatch:
try:
for element in journalmatch:

4
bin/fail2ban-testcases
View File

@ -48,7 +48,7 @@ def get_opt_parser():
p.add_options([
Option('-l', "--log-level", type="choice",
dest="log_level",
choices=('heavydebug', 'debug', 'info', 'warn', 'error', 'fatal'),
choices=('heavydebug', 'debug', 'info', 'warning', 'error', 'fatal'),
default=None,
help="Log level for the logger to use during running tests"),
Option('-n', "--no-network", action="store_true",
@ -72,7 +72,7 @@ parser = get_opt_parser()
logSys = logging.getLogger("fail2ban")
# Numerical level of verbosity corresponding to a log "level"
verbosity = {'heavydebug': 3,
verbosity = {'heavydebug': 4,
'debug': 3,
'info': 2,
'warning': 1,

7
config/action.d/apf.conf
View File

@ -41,3 +41,10 @@ actionban = apf --deny <ip> "banned by Fail2Ban <name>"
# Values: CMD
#
actionunban = apf --remove <ip>
[Init]
# Name used in APF configuration
#
name = default

86
config/action.d/blocklist_de.conf Normal file
View File

@ -0,0 +1,86 @@
# Fail2Ban configuration file
#
# Author: Steven Hiscocks
#
#
# Action to report IP address to blocklist.de
# Blocklist.de must be signed up to at www.blocklist.de
# Once registered, one or more servers can be added.
# This action requires the server 'email address' and the assoicate apikey.
#
# From blocklist.de:
# www.blocklist.de is a free and voluntary service provided by a
# Fraud/Abuse-specialist, whose servers are often attacked on SSH-,
# Mail-Login-, FTP-, Webserver- and other services.
# The mission is to report all attacks to the abuse deparments of the
# infected PCs/servers to ensure that the responsible provider can inform
# the customer about the infection and disable them
#
# IMPORTANT:
#
# Reporting an IP of abuse is a serious complaint. Make sure that it is
# serious. Fail2ban developers and network owners recommend you only use this
# action for:
# * The recidive where the IP has been banned multiple times
# * Where maxretry has been set quite high, beyond the normal user typing
# password incorrectly.
# * For filters that have a low likelyhood of receiving human errors
#
[Definition]
# Option: actionstart
# Notes.: command executed once at the start of Fail2Ban.
# Values: CMD
#
actionstart =
# Option: actionstop
# Notes.: command executed once at the end of Fail2Ban
# Values: CMD
#
actionstop =
# Option: actioncheck
# Notes.: command executed once before each actionban command
# Values: CMD
#
actioncheck =
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionban = curl --fail --data-urlencode 'server=<email>' --data 'apikey=<apikey>' --data 'service=<service>' --data 'ip=<ip>' --data-urlencode 'logs=<matches>' --data 'format=text' --user-agent "fail2ban v0.8.12" "https://www.blocklist.de/en/httpreports.html"
# Option: actionunban
# Notes.: command executed when unbanning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionunban =
[Init]
# Option: email
# Notes server email address, as per blocklise.de account
# Values: STRING Default: None
#
#email =
# Option: apikey
# Notes your user blocklist.de user account apikey
# Values: STRING Default: None
#
#apikey =
# Option: service
# Notes service name you are reporting on, typically aligns with filter name
# see http://www.blocklist.de/en/httpreports.html for full list
# Values: STRING Default: None
#
#service =

2
config/action.d/complain.conf
View File

@ -58,7 +58,7 @@ actioncheck =
actionban = ADDRESSES=`whois <ip> | perl -e 'while (<STDIN>) { next if /^changed|@(ripe|apnic)\.net/io; $m += (/abuse|trouble:|report|spam|security/io?3:0); if (/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)/io) { while (s/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)//io) { if ($m) { $a{lc($1)}=$m } else { $b{lc($1)}=$m } } $m=0 } else { $m && --$m } } if (%%a) {print join(",",keys(%%a))} else {print join(",",keys(%%b))}'`
IP=<ip>
if [ ! -z "$ADDRESSES" ]; then
(printf %%b "<message>\n"; date '+Note: Local timezone is %%z (%%Z)'; grep '<ip>' <logpath>) | <mailcmd> "Abuse from <ip>" <mailargs> $ADDRESSES
(printf %%b "<message>\n"; date '+Note: Local timezone is %%z (%%Z)'; grep -E '(^|[^0-9])<ip>([^0-9]|$)' <logpath>) | <mailcmd> "Abuse from <ip>" <mailargs> $ADDRESSES
fi
# Option: actionunban

69
config/action.d/firewallcmd-ipset.conf Normal file
View File

@ -0,0 +1,69 @@
# Fail2Ban action file for firewall-cmd/ipset
#
# This requires:
# ipset (package: ipset)
# firewall-cmd (package: firewalld)
#
# This is for ipset protocol 6 (and hopefully later) (ipset v6.14).
# Use ipset -V to see the protocol and version.
#
# IPset was a feature introduced in the linux kernel 2.6.39 and 3.0.0 kernels.
#
# If you are running on an older kernel you make need to patch in external
# modules.
[INCLUDES]
before = iptables-blocktype.conf
[Definition]
actionstart = ipset create fail2ban-<name> hash:ip timeout <bantime>
firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j <blocktype>
actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j <blocktype>
ipset flush fail2ban-<name>
ipset destroy fail2ban-<name>
actioncheck = firewall-cmd --direct --get-chains ipv4 filter | grep -q '^fail2ban-<name>$'
actionban = ipset add fail2ban-<name> <ip> timeout <bantime> -exist
actionunban = ipset del fail2ban-<name> <ip> -exist
[Init]
# Default name of the chain
#
name = default
# Option: port
# Notes.: specifies port to monitor
# Values: [ NUM | STRING ]
#
port = ssh
# Option: protocol
# Notes.: internally used by config reader for interpolations.
# Values: [ tcp | udp | icmp | all ]
#
protocol = tcp
# Option: chain
# Notes specifies the iptables chain to which the fail2ban rules should be
# added
# Values: [ STRING ]
#
chain = INPUT_direct
# Option: bantime
# Notes: specifies the bantime in seconds (handled internally rather than by fail2ban)
# Values: [ NUM ] Default: 600
bantime = 600
# DEV NOTES:
#
# Author: Edgar Hoch and Daniel Black
# firewallcmd-new / iptables-ipset-proto6 combined for maximium goodness

30
config/action.d/firewall-cmd-direct-new.conf → config/action.d/firewallcmd-new.conf
View File

@ -1,9 +1,5 @@
# Fail2Ban configuration file
#
# Author: Edgar Hoch
# Copied from iptables-new.conf and modified for use with firewalld by Edgar Hoch.
# It uses "firewall-cmd" instead of "iptables".
#
# Because of the --remove-rules in stop this action requires firewalld-0.3.8+
[INCLUDES]
@ -20,7 +16,7 @@ actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -m state
firewall-cmd --direct --remove-rules ipv4 filter fail2ban-<name>
firewall-cmd --direct --remove-chain ipv4 filter fail2ban-<name>
actioncheck = firewall-cmd --direct --get-chains ipv4 filter | grep -q 'fail2ban-<name>[ \t]'
actioncheck = firewall-cmd --direct --get-chains ipv4 filter | grep -q '^fail2ban-<name>$'
actionban = firewall-cmd --direct --add-rule ipv4 filter fail2ban-<name> 0 -s <ip> -j <blocktype>
@ -50,3 +46,27 @@ protocol = tcp
# Values: [ STRING ]
#
chain = INPUT_direct
# DEV NOTES:
#
# Author: Edgar Hoch
# Copied from iptables-new.conf and modified for use with firewalld by Edgar Hoch.
# It uses "firewall-cmd" instead of "iptables".
#
# Output:
#
# $ firewall-cmd --direct --add-chain ipv4 filter fail2ban-name
# success
# $ firewall-cmd --direct --add-rule ipv4 filter fail2ban-name 1000 -j RETURN
# success
# $ sudo firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 0 -m state --state NEW -p tcp --dport 22 -j fail2ban-name
# success
# $ firewall-cmd --direct --get-chains ipv4 filter
# fail2ban-name
# $ firewall-cmd --direct --get-chains ipv4 filter | od -h
# 0000000 6166 6c69 6232 6e61 6e2d 6d61 0a65
# $ firewall-cmd --direct --get-chains ipv4 filter | grep -Eq 'fail2ban-name( |$)' ; echo $?
# 0
# $ firewall-cmd -V
# 0.3.8

2
config/action.d/ipfw.conf
View File

@ -43,7 +43,7 @@ actionban = ipfw add <blocktype> tcp from <ip> to <localhost> <port>
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionunban = ipfw delete `ipfw list | grep -i <ip> | awk '{print $1;}'`
actionunban = ipfw delete `ipfw list | grep -i "[^0-9]<ip>[^0-9]" | awk '{print $1;}'`
[Init]

6
config/action.d/mail-whois-lines.conf
View File

@ -39,10 +39,10 @@ actioncheck =
actionban = printf %%b "Hi,\n
The IP <ip> has just been banned by Fail2Ban after
<failures> attempts against <name>.\n\n
Here are more information about <ip>:\n
`whois <ip>`\n\n
Here is more information about <ip>:\n
`whois <ip> || echo missing whois program`\n\n
Lines containing IP:<ip> in <logpath>\n
`grep '\<<ip>\>' <logpath>`\n\n
`grep '[^0-9]<ip>[^0-9]' <logpath>`\n\n
Regards,\n
Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>

4
config/action.d/mail-whois.conf
View File

@ -39,8 +39,8 @@ actioncheck =
actionban = printf %%b "Hi,\n
The IP <ip> has just been banned by Fail2Ban after
<failures> attempts against <name>.\n\n
Here are more information about <ip>:\n
`whois <ip>`\n
Here is more information about <ip>:\n
`whois <ip> || echo missing whois program`\n
Regards,\n
Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>

50
config/action.d/sendmail-common.conf
View File

@ -8,6 +8,56 @@
after = sendmail-common.local
[Definition]
# Option: actionstart
# Notes.: command executed once at the start of Fail2Ban.
# Values: CMD
#
actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The jail <name> has been started successfully.\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actionstop
# Notes.: command executed once at the end of Fail2Ban
# Values: CMD
#
actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The jail <name> has been stopped.\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actioncheck
# Notes.: command executed once before each actionban command
# Values: CMD
#
actioncheck =
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionban =
# Option: actionunban
# Notes.: command executed when unbanning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionunban =
[Init]
# Recipient mail address

37
config/action.d/sendmail-whois-ipjailmatches.conf Normal file
View File

@ -0,0 +1,37 @@
# Fail2Ban configuration file
#
# Author: Cyril Jaquier
#
#
[INCLUDES]
before = sendmail-common.conf
[Definition]
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The IP <ip> has just been banned by Fail2Ban after
<failures> attempts against <name>.\n\n
Here are more information about <ip>:\n
`/usr/bin/whois <ip>`\n\n
Matches for <name> with <ipjailfailures> failures IP:<ip>\n
<ipjailmatches>\n\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
[Init]
# Default name of the chain
#
name = default

37
config/action.d/sendmail-whois-ipmatches.conf Normal file
View File

@ -0,0 +1,37 @@
# Fail2Ban configuration file
#
# Author: Cyril Jaquier
#
#
[INCLUDES]
before = sendmail-common.conf
[Definition]
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The IP <ip> has just been banned by Fail2Ban after
<failures> attempts against <name>.\n\n
Here are more information about <ip>:\n
`/usr/bin/whois <ip>`\n\n
Matches with <ipfailures> failures IP:<ip>\n
<ipmatches>\n\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
[Init]
# Default name of the chain
#
name = default

46
config/action.d/sendmail-whois-lines.conf
View File

@ -10,38 +10,6 @@ before = sendmail-common.conf
[Definition]
# Option: actionstart
# Notes.: command executed once at the start of Fail2Ban.
# Values: CMD
#
actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The jail <name> has been started successfully.\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actionstop
# Notes.: command executed once at the end of Fail2Ban
# Values: CMD
#
actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The jail <name> has been stopped.\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actioncheck
# Notes.: command executed once before each actionban command
# Values: CMD
#
actioncheck =
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
@ -55,21 +23,13 @@ actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
Hi,\n
The IP <ip> has just been banned by Fail2Ban after
<failures> attempts against <name>.\n\n
Here are more information about <ip>:\n
`/usr/bin/whois <ip>`\n\n
Here is more information about <ip>:\n
`/usr/bin/whois <ip> || echo missing whois program`\n\n
Lines containing IP:<ip> in <logpath>\n
`grep '\<<ip>\>' <logpath>`\n\n
`grep '[^0-9]<ip>[^0-9]' <logpath>`\n\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actionunban
# Notes.: command executed when unbanning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionunban =
[Init]
# Default name of the chain

37
config/action.d/sendmail-whois-matches.conf Normal file
View File

@ -0,0 +1,37 @@
# Fail2Ban configuration file
#
# Author: Cyril Jaquier
#
#
[INCLUDES]
before = sendmail-common.conf
[Definition]
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The IP <ip> has just been banned by Fail2Ban after
<failures> attempts against <name>.\n\n
Here are more information about <ip>:\n
`/usr/bin/whois <ip>`\n\n
Matches:\n
<matches>\n\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
[Init]
# Default name of the chain
#
name = default

44
config/action.d/sendmail-whois.conf
View File

@ -10,38 +10,6 @@ before = sendmail-common.conf
[Definition]
# Option: actionstart
# Notes.: command executed once at the start of Fail2Ban.
# Values: CMD
#
actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The jail <name> has been started successfully.\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actionstop
# Notes.: command executed once at the end of Fail2Ban
# Values: CMD
#
actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The jail <name> has been stopped.\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actioncheck
# Notes.: command executed once before each actionban command
# Values: CMD
#
actioncheck =
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
@ -55,19 +23,11 @@ actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
Hi,\n
The IP <ip> has just been banned by Fail2Ban after
<failures> attempts against <name>.\n\n
Here are more information about <ip>:\n
`/usr/bin/whois <ip>`\n
Here is more information about <ip>:\n
`/usr/bin/whois <ip> || echo missing whois program`\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actionunban
# Notes.: command executed when unbanning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionunban =
[Init]
# Default name of the chain

40
config/action.d/sendmail.conf
View File

@ -10,38 +10,6 @@ before = sendmail-common.conf
[Definition]
# Option: actionstart
# Notes.: command executed once at the start of Fail2Ban.
# Values: CMD
#
actionstart = printf %%b "Subject: [Fail2Ban] <name>: started on `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The jail <name> has been started successfully.\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actionstop
# Notes.: command executed once at the end of Fail2Ban
# Values: CMD
#
actionstop = printf %%b "Subject: [Fail2Ban] <name>: stopped on `uname -n`
Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"`
From: <sendername> <<sender>>
To: <dest>\n
Hi,\n
The jail <name> has been stopped.\n
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actioncheck
# Notes.: command executed once before each actionban command
# Values: CMD
#
actioncheck =
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the
# command is executed with Fail2Ban user rights.
@ -58,14 +26,6 @@ actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
Regards,\n
Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>
# Option: actionunban
# Notes.: command executed when unbanning an IP. Take care that the
# command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionunban =
[Init]
# Default name of the chain

40
config/action.d/ufw.conf Normal file
View File

@ -0,0 +1,40 @@
# Fail2Ban action configuration file for ufw
#
# You are required to run "ufw enable" before this will have an effect.
#
# The insert position should be approprate to block the required traffic.
# A number after an allow rule to the application won't be much use.
[Definition]
actionstart =
actionstop =
actioncheck =
actionban = [ -n "<application>" ] && app="app <application>" ; ufw insert <insertpos> <blocktype> from <ip> to <destination> $app
actionunban = [ -n "<application>" ] && app="app <application>" ; ufw delete <blocktype> from <ip> to <destination> $app
[Init]
# Option: insertpos
# Notes.: The postition number in the firewall list to insert the block rule
insertpos = 1
# Option: blocktype
# Notes.: reject or deny
blocktype = reject
# Option: destination
# Notes.: The destination address to block in the ufw rule
destination = any
# Option: application
# Notes.: application from sudo ufw app list
application =
# DEV NOTES:
#
# Author: Guilhem Lettron
# Enhancements: Daniel Black

125
config/action.d/xarf-login-attack.conf Normal file
View File

@ -0,0 +1,125 @@
# Fail2Ban action for sending xarf Login-Attack messages to IP owner
#
# IMPORTANT:
#
# Emailing a IP owner of abuse is a serious complain. Make sure that it is
# serious. Fail2ban developers and network owners recommend you only use this
# action for:
# * The recidive where the IP has been banned multiple times
# * Where maxretry has been set quite high, beyond the normal user typing
# password incorrectly.
# * For filters that have a low likelyhood of receiving human errors
#
# DEPENDANCIES:
#
# This requires the dig command from bind-utils
#
# This uses the https://abusix.com/contactdb.html to lookup abuse contacts.
#
# XARF is a specification for sending a formatted response
# for non-messaging based abuse including:
#
# Login-Attack, Malware-Attack, Fraud (Phishing, etc.), Info DNSBL
#
# For details see:
# https://github.com/abusix/xarf-specification
# http://www.x-arf.org/schemata.html
#
# Author: Daniel Black
# Based on complain written by Russell Odom <russ@gloomytrousers.co.uk>
#
#
[Definition]
actionstart =
actionstop =
actioncheck =
actionban = oifs=${IFS}; IFS=.;SEP_IP=( <ip> ); set -- ${SEP_IP} ;ADDRESSES=$(dig +short -t txt -q $4.$3.$2.$1.abuse-contacts.abusix.org); IFS=${oifs}
IP=<ip>
FROM=<sender>
SERVICE=<service>
FAILURES=<failures>
MATCHES='<matches>'
REPORTID=<time>@`uname -n`
TLP=<tlp>
PORT=<port>
DATE=`LC_TIME=C date -u --date=@<time> +"%%a, %%d %%h %%Y %%T +0000"`
if [ ! -z "$ADDRESSES" ]; then
(printf -- %%b "<header>\n<message>\n<report>\n${MATCHES}\n";
date '+Note: Local timezone is %%z (%%Z)';
printf -- %%b "<ipmatches>\n\n<footer>") | <mailcmd> <mailargs> ${ADDRESSES//,/\" \"}
fi
actionunban =
[Init]
# Option: header
# Notes: This is really a fixed value
header = Subject: abuse report about $IP - $DATE\nAuto-Submitted: auto-generated\nX-XARF: PLAIN\nContent-Transfer-Encoding: 7bit\nContent-Type: multipart/mixed; charset=utf8;\n boundary=Abuse-bfbb0f920793ac03cb8634bde14d8a1e;\n\n--Abuse-bfbb0f920793ac03cb8634bde14d8a1e\nMIME-Version: 1.0\nContent-Transfer-Encoding: 7bit\nContent-Type: text/plain; charset=utf-8;\n
# Option: footer
# Notes: This is really a fixed value and needs to match the report and header
# mime delimiters
footer = \n\n--Abuse-bfbb0f920793ac03cb8634bde14d8a1e--
# Option: report
# Notes: Intended to be fixed
report = --Abuse-bfbb0f920793ac03cb8634bde14d8a1e\nMIME-Version: 1.0\nContent-Transfer-Encoding: 7bit\nContent-Type: text/plain; charset=utf-8; name=\"report.txt\";\n\n---\nReported-From: $FROM\nCategory: abuse\nReport-ID: $REPORTID\nReport-Type: login-attack\nService: $SERVICE\nVersion: 0.2\nUser-Agent: Fail2ban v0.9\nDate: $DATE\nSource-Type: ip-address\nSource: $IP\nPort: $PORT\nSchema-URL: http://www.x-arf.org/schema/abuse_login-attack_0.1.2.json\nAttachment: text/plain\nOccurances: $FAILURES\nTLP: $TLP\n\n\n--Abuse-bfbb0f920793ac03cb8634bde14d8a1e\nMIME-Version: 1.0\nContent-Transfer-Encoding: 7bit\nContent-Type: text/plain; charset=utf8; name=\"logfile.log\";
# Option: Message
# Notes: This can be modified by the users
message = Dear Sir/Madam,\n\nWe have detected abuse from the IP address $IP, which according to abusix.com is on your network. We would appreciate if you would investigate and take action as appropriate.\n\nLog lines are given below, but please ask if you require any further information.\n\n(If you are not the correct person to contact about this please accept our apologies - your e-mail address was extracted from the whois record by an automated process.)\n\n This mail was generated by Fail2Ban in a X-ARF format! You can find more information about x-arf at http://www.x-arf.org/specification.html.\n\nThe recipient address of this report was provided by the Abuse Contact DB by abusix.com. abusix.com does not maintain the content of the database. All information which we pass out, derives from the RIR databases and is processed for ease of use. If you want to change or report non working abuse contacts please contact the appropriate RIR. If you have any further question, contact abusix.com directly via email (info@abusix.com). Information about the Abuse Contact Database can be found here: https://abusix.com/global-reporting/abuse-contact-db\nabusix.com is neither responsible nor liable for the content or accuracy of this message.\n
# Option: loglines
# Notes.: The number of log lines to search for the IP for the report
loglines = 9000
# Option: mailcmd
# Notes.: Your system mail command. It is passed the recipient
# Values: CMD
#
mailcmd = /usr/sbin/sendmail
# Option: mailargs
# Notes.: Additional arguments to mail command. e.g. for standard Unix mail:
# CC reports to another address:
# -c me@example.com
# Appear to come from a different address - the '--' indicates
# arguments to be passed to Sendmail:
# -- -f me@example.com
# Values: [ STRING ]
#
mailargs = -f <sender>
# Option: tlp
# Notes.: Traffic light protocol defining the sharing of this information.
# http://www.trusted-introducer.org/ISTLPv11.pdf
# green is share to those involved in network security but it is not
# to be released to the public.
tlp = green
# ALL of the following parameters should be set so the report contains
# meaningful information
# Option: service
# Notes.: This is the service type that was attacked. e.g. ssh, pop3
service = unspecified
# Option: logpath
# Notes: Path to the log files which contain relevant lines for the abuser IP
# Values: Filename(s) space separated and can contain wildcards (these are
# greped for the IP so make sure these aren't too long
logpath = /dev/null
# Option: sender
# Notes.: This is the sender that is included in the XARF report
sender = fail2ban@`uname -n`
# Option: port
# Notes.: This is the port number that received the login-attack
port = 0

12
config/fail2ban.conf
View File

@ -47,3 +47,15 @@ socket = /var/run/fail2ban/fail2ban.sock
#
pidfile = /var/run/fail2ban/fail2ban.pid
# Options: dbfile
# Notes.: Set the file for the fail2ban persistent data to be stored.
# A value of ":memory:" means database is only stored in memory
# and data is lost once fail2ban is stops.
# A value of "None" disables the database.
# Values: [ None :memory: FILE ] Default: /var/lib/fail2ban/fail2ban.sqlite3
dbfile = /var/lib/fail2ban/fail2ban.sqlite3
# Options: dbpurgeage
# Notes.: Sets age at which bans should be purged from the database
# Values: [ SECONDS ] Default: 86400 (24hours)
dbpurgeage = 86400

3
config/filter.d/apache-common.conf
View File

@ -8,12 +8,13 @@ after = apache-common.local
[DEFAULT]
_apache_error_client = \[\] \[(error|\S+:\S+)\]( \[pid \d+:\S+ \d+\])? \[client <HOST>(:\d{1,5})?\]
_apache_error_client = \[\] \[(:?error|\S+:\S+)\]( \[pid \d+(:\S+ \d+)?\])? \[client <HOST>(:\d{1,5})?\]
# Common prefix for [error] apache messages which also would include <HOST>
# Depending on the version it could be
# 2.2: [Sat Jun 01 11:23:08 2013] [error] [client 1.2.3.4]
# 2.4: [Thu Jun 27 11:55:44.569531 2013] [core:info] [pid 4101:tid 2992634688] [client 1.2.3.4:46652]
# 2.4 (perfork): [Mon Dec 23 07:49:01.981912 2013] [:error] [pid 3790] [client 204.232.202.107:46301] script '/var/www/timthumb.php' not found or unable to
#
# Reference: https://github.com/fail2ban/fail2ban/issues/268
#

18
config/filter.d/apache-modsecurity.conf Normal file
View File

@ -0,0 +1,18 @@
# Fail2Ban apache-modsec filter
#
[INCLUDES]
# Read common prefixes. If any customizations available -- read them from
# apache-common.local
before = apache-common.conf
[Definition]
failregex = ^%(_apache_error_client)s ModSecurity: (\[.*?\] )*Access denied with code [45]\d\d.*$
ignoreregex =
# https://github.com/SpiderLabs/ModSecurity/wiki/ModSecurity-2-Data-Formats
# Author: Daniel Black

4
config/filter.d/apache-noscript.conf
View File

@ -9,8 +9,8 @@ before = apache-common.conf
[Definition]
failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): /\S*(\.php|\.asp|\.exe|\.pl)(, referer: \S+)?\s*$
^%(_apache_error_client)s script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat(, referer: \S+)?\s*$
failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): /\S*(php([45]|[.-]cgi)?|\.asp|\.exe|\.pl)(, referer: \S+)?\s*$
^%(_apache_error_client)s script '/\S*(php([45]|[.-]cgi)?|\.asp|\.exe|\.pl)\S*' not found or unable to stat(, referer: \S+)?\s*$
ignoreregex =

2
config/filter.d/common.conf
View File

@ -34,7 +34,7 @@ __daemon_combs_re = (?:%(__pid_re)s?:\s+%(__daemon_re)s|%(__daemon_re)s%(__pid_r
# Some messages have a kernel prefix with a timestamp
# EXAMPLES: kernel: [769570.846956]
__kernel_prefix = kernel: \[\d+\.\d+\]
__kernel_prefix = kernel: \[ *\d+\.\d+\]
__hostname = \S+

2
config/filter.d/exim-spam.conf
View File

@ -1,5 +1,6 @@
# Fail2Ban filter for exim the spam rejection messages
#
## For the SA: Action: silently tossed message... to be logged exim's SAdevnull option needs to be used.
[INCLUDES]
@ -12,6 +13,7 @@ before = exim-common.conf
failregex = ^%(pid)s \S+ F=(<>|\S+@\S+) %(host_info)srejected by local_scan\(\): .{0,256}$
^%(pid)s %(host_info)sF=(<>|[^@]+@\S+) rejected RCPT [^@]+@\S+: .*dnsbl.*\s*$
^%(pid)s \S+ %(host_info)sF=(<>|[^@]+@\S+) rejected after DATA: This message contains a virus \(\S+\)\.\s*$
^%(pid)s \S+ SA: Action: silently tossed message: score=\d+\.\d+ required=\d+\.\d+ trigger=\d+\.\d+ \(scanned in \d+/\d+ secs \| Message-Id: \S+\)\. From \S+ \(host=(\S+ )?\[<HOST>\]\) for \S+$
ignoreregex =

26
config/filter.d/nsd.conf Normal file
View File

@ -0,0 +1,26 @@
# Fail2Ban configuration file
#
# Author: Bas van den Dikkenberg
#
#
[INCLUDES]
# Read common prefixes. If any customizations available -- read them from
# common.local
before = common.conf
[Definition]
_daemon = nsd
# Option: failregex
# Notes.: regex to match the password failures messages in the logfile. The
# host must be matched by a group named "host". The tag "<HOST>" can
# be used for standard IP/hostname matching and is only an alias for
# (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
# Values: TEXT
failregex = ^\[\]%(__prefix_line)sinfo: ratelimit block .* query <HOST> TYPE255$
^\[\]%(__prefix_line)sinfo: .* <HOST> refused, no acl matches\.$

15
config/filter.d/openwebmail.conf Normal file
View File

@ -0,0 +1,15 @@
# Fail2Ban filter for Openwebmail
# banning hosts with authentication errors in /var/log/openwebmail.log
# OpenWebMail http://openwebmail.org
#
[Definition]
failregex = ^ - \[\d+\] \(<HOST>\) (?P<USER>\S+) - login error - (no such user - loginname=(?P=USER)|auth_unix.pl, ret -4, Password incorrect)$
^ - \[\d+\] \(<HOST>\) (?P<USER>\S+) - userinfo error - auth_unix.pl, ret -4, User (?P=USER) doesn't exist$
ignoreregex =
# DEV Notes:
#
# Author: Ivo Truxa (c) 2013 truXoft.com

2
config/filter.d/proftpd.conf
View File

@ -1,5 +1,7 @@
# Fail2Ban fitler for the Proftpd FTP daemon
#
# Set "UseReverseDNS off" in proftpd.conf to avoid the need for DNS.
# See: http://www.proftpd.org/docs/howto/DNS.html
[INCLUDES]

6
config/filter.d/recidive.conf
View File

@ -21,16 +21,16 @@ before = common.conf
[Definition]
_daemon = fail2ban\.actions
_daemon = fail2ban\.server\.actions
# The name of the jail that this filter is used for. In jail.conf, name the
# jail using this filter 'recidive', or change this line!
_jailname = recidive
failregex = ^(%(__prefix_line)s|,\d{3} fail2ban.actions:\s+)WARNING\s+\[(?!%(_jailname)s\])(?:.*)\]\s+Ban\s+<HOST>\s*$
failregex = ^(%(__prefix_line)s| %(_daemon)s%(__pid_re)s?:\s+)WARNING\s+\[(?!%(_jailname)s\])(?:.*)\]\s+Ban\s+<HOST>\s*$
[Init]
journalmatch = _SYSTEMD_UNIT=fail2ban.service
journalmatch = _SYSTEMD_UNIT=fail2ban.service PRIORITY=4
# Author: Tom Hendrikx, modifications by Amir Caspi

13
config/filter.d/squid.conf Normal file
View File

@ -0,0 +1,13 @@
# Fail2Ban filter for Squid attempted proxy bypasses
#
#
[Definition]
failregex = ^\s+\d\s<HOST>\s+[A-Z_]+_DENIED/403 .*$
^\s+\d\s<HOST>\s+NONE/405 .*$
# Author: Daniel Black

1
config/filter.d/sshd.conf
View File

@ -23,6 +23,7 @@ failregex = ^%(__prefix_line)s(?:error: PAM: )?[aA]uthentication (?:failure|erro
^%(__prefix_line)sUser .+ from <HOST> not allowed because a group is listed in DenyGroups\s*$
^%(__prefix_line)sUser .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*$
^(?P<__prefix>%(__prefix_line)s)User .+ not allowed because account is locked<SKIPLINES>(?P=__prefix)(?:error: )?Received disconnect from <HOST>: 11: Bye Bye \[preauth\]$
^(?P<__prefix>%(__prefix_line)s)Disconnecting: Too many authentication failures for .+? \[preauth\]<SKIPLINES>(?P=__prefix)(?:error: )?Connection closed by <HOST> \[preauth\]$
ignoreregex =

41
config/jail.conf
View File

@ -44,6 +44,12 @@
# defined using space separator.
ignoreip = 127.0.0.1/8
# External command that will take an tagged arguments to ignore, e.g. <ip>,
# and return true if the IP is to be ignored. False otherwise.
#
# ignorecommand = /path/to/command <ip>
ignorecommand =
# "bantime" is the number of seconds that a host is banned.
bantime = 600
@ -109,9 +115,12 @@ filter = %(__name__)s
# Some options used for actions
# Destination email address used solely for the interpolations in
# jail.{conf,local} configuration files.
# jail.{conf,local,d/*} configuration files.
destemail = root@localhost
# Sender email address used solely for some actions
sender = root@localhost
# E-mail action. Since 0.8.1 Fail2Ban uses sendmail MTA for the
# mailing. Change mta configuration parameter to mail if you want to
# revert to conventional 'mail'.
@ -148,6 +157,24 @@ action_mw = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protoc
action_mwl = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
%(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]
# See the IMPORTANT note in action.d/xarf-login-attack for when to use this action
#
# ban & send a xarf e-mail to abuse contact of IP address and include relevant log lines
# to the destemail.
action_xarf = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
xarf-login-attack[service=%(__name__)s, sender="%(sender)s", logpath=%(logpath)s, port="%(port)s"]
# Report block via blocklist.de fail2ban reporting service API
#
# See the IMPORTANT note in action.d/blocklist_de.conf for when to
# use this action. Create a file jail.d/blocklist_de.local containing
# [Init]
# blocklist_de_apikey = {api key from registration]
#
action_blocklist_de = blocklist_de[email="%(sender)s", service=%(filter)s, apikey="%(blocklist_de_apikey)s"]
# Choose default action. To change, just override value of 'action' with the
# interpolation to the chosen action shortcut (e.g. action_mw, action_mwl, etc) in jail.local
# globally (section [DEFAULT]) or per specific section
@ -321,6 +348,13 @@ logpath = /var/log/apache*/*error.log
maxretry = 2
[apache-modsecurity]
port = http,https
logpath = /var/log/apache*/*error.log
maxretry = 2
[nginx-http-auth]
ports = http,https
@ -356,6 +390,11 @@ port = http,https
logpath = /var/log/roundcube/userlogins
[openwebmail]
port = http,https
logpath = /var/log/openwebmail.log`
[sogo-auth]
# Monitor SOGo groupware server
# without proxy this would be:

16
fail2ban/client/beautifier.py
View File

@ -63,6 +63,8 @@ class Beautifier:
msg = "Jail stopped"
elif inC[0] == "add":
msg = "Added jail " + response
elif inC[0] == "flushlogs":
msg = "logs: " + response
elif inC[0:1] == ['status']:
if len(inC) > 1:
# Create IP list
@ -102,6 +104,18 @@ class Beautifier:
msg = msg + "DEBUG"
else:
msg = msg + `response`
elif inC[1] == "dbfile":
if response is None:
msg = "Database currently disabled"
else:
msg = "Current database file is:\n"
msg = msg + "`- " + response
elif inC[1] == "dbpurgeage":
if response is None:
msg = "Database currently disabled"
else:
msg = "Current database purge age is:\n"
msg = msg + "`- %iseconds" % response
elif inC[2] in ("logpath", "addlogpath", "dellogpath"):
if len(response) == 0:
msg = "No file is currently monitored"
@ -122,7 +136,7 @@ class Beautifier:
elif inC[2] == "datepattern":
msg = "Current date pattern set to: "
if response is None:
msg = msg + "Default Detectors"
msg = msg + "Not set/required"
elif response[0] is None:
msg = msg + "%s" % response[1]
else:

1
fail2ban/client/configparserinc.py
View File

@ -100,6 +100,7 @@ after = 1.conf
def __init__(self, *args, **kwargs):
kwargs = kwargs.copy()
kwargs['interpolation'] = BasicInterpolationWithName()
kwargs['inline_comment_prefixes'] = ";"
super(SafeConfigParserWithIncludes, self).__init__(
*args, **kwargs)

6
fail2ban/client/configreader.py
View File

@ -113,6 +113,7 @@ class ConfigReader(SafeConfigParserWithIncludes):
# No "Definition" section or wrong basedir
logSys.error(e)
values[option[1]] = option[2]
# TODO: validate error handling here.
except NoOptionError:
if not option[2] is None:
logSys.warning("'%s' not defined in '%s'. Using default one: %r"
@ -137,12 +138,13 @@ class DefinitionInitConfigReader(ConfigReader):
def __init__(self, file_, jailName, initOpts, **kwargs):
ConfigReader.__init__(self, **kwargs)
self._file = file_
self._jailName = jailName
self.setFile(file_)
self.setJailName(jailName)
self._initOpts = initOpts
def setFile(self, fileName):
self._file = fileName
self._initOpts = {}
def getFile(self):
return self._file

8
fail2ban/client/fail2banreader.py
View File

@ -45,7 +45,9 @@ class Fail2banReader(ConfigReader):
def getOptions(self):
opts = [["int", "loglevel", 1],
["string", "logtarget", "STDERR"]]
["string", "logtarget", "STDERR"],
["string", "dbfile", "/var/lib/fail2ban/fail2ban.sqlite3"],
["int", "dbpurgeage", 86400]]
self.__opts = ConfigReader.getOptions(self, "Definition", opts)
def convert(self):
@ -55,5 +57,9 @@ class Fail2banReader(ConfigReader):
stream.append(["set", "loglevel", self.__opts[opt]])
elif opt == "logtarget":
stream.append(["set", "logtarget", self.__opts[opt]])
elif opt == "dbfile":
stream.append(["set", "dbfile", self.__opts[opt]])
elif opt == "dbpurgeage":
stream.append(["set", "dbpurgeage", self.__opts[opt]])
return stream

53
fail2ban/client/jailreader.py
View File

@ -62,7 +62,7 @@ class JailReader(ConfigReader):
return out
def isEnabled(self):
return self.__force_enable or self.__opts["enabled"]
return self.__force_enable or ( self.__opts and self.__opts["enabled"] )
@staticmethod
def _glob(path):
@ -72,12 +72,10 @@ class JailReader(ConfigReader):
"""
pathList = []
for p in glob.glob(path):
if not os.path.exists(p):
logSys.warning("File %s doesn't even exist, thus cannot be monitored" % p)
elif not os.path.lexists(p):
logSys.warning("File %s is a dangling link, thus cannot be monitored" % p)
else:
if os.path.exists(p):
pathList.append(p)
else:
logSys.warning("File %s is a dangling link, thus cannot be monitored" % p)
return pathList
def getOptions(self):
@ -91,24 +89,31 @@ class JailReader(ConfigReader):
["string", "usedns", "warn"],
["string", "failregex", None],
["string", "ignoreregex", None],
["string", "ignorecommand", None],
["string", "ignoreip", None],
["string", "filter", ""],
["string", "action", ""]]
self.__opts = ConfigReader.getOptions(self, self.__name, opts)
if not self.__opts:
return False
if self.isEnabled():
# Read filter
filterName, filterOpt = JailReader.extractOptions(
self.__opts["filter"])
self.__filter = FilterReader(
filterName, self.__name, filterOpt, basedir=self.getBaseDir())
ret = self.__filter.read()
if ret:
self.__filter.getOptions(self.__opts)
if self.__opts["filter"]:
filterName, filterOpt = JailReader.extractOptions(
self.__opts["filter"])
self.__filter = FilterReader(
filterName, self.__name, filterOpt, basedir=self.getBaseDir())
ret = self.__filter.read()
if ret:
self.__filter.getOptions(self.__opts)
else:
logSys.error("Unable to read the filter")
return False
else:
logSys.error("Unable to read the filter")
return False
self.__filter = None
logSys.warn("No filter set for jail %s" % self.__name)
# Read action
for act in self.__opts["action"].split('\n'):
try:
@ -147,12 +152,15 @@ class JailReader(ConfigReader):
self.__opts.get('backend', None) != "systemd":
found_files = 0
for path in self.__opts[opt].split("\n"):
path = path.rsplit(" ", 1)
path, tail = path if len(path) > 1 else (path[0], "false")
pathList = JailReader._glob(path)
if len(pathList) == 0:
logSys.error("No file(s) found for glob %s" % path)
for p in pathList:
found_files += 1
stream.append(["set", self.__name, "addlogpath", p])
stream.append(
["set", self.__name, "addlogpath", p, tail])
if not (found_files or allow_no_files):
raise ValueError(
"Have not found any log file for %s jail" % self.__name)
@ -175,12 +183,15 @@ class JailReader(ConfigReader):
stream.append(["set", self.__name, "usedns", self.__opts[opt]])
elif opt == "failregex":
stream.append(["set", self.__name, "addfailregex", self.__opts[opt]])
elif opt == "ignorecommand":
stream.append(["set", self.__name, "ignorecommand", self.__opts[opt]])
elif opt == "ignoreregex":
for regex in self.__opts[opt].split('\n'):
# Do not send a command if the rule is empty.
if regex != '':
stream.append(["set", self.__name, "addignoreregex", regex])
stream.extend(self.__filter.convert())
if self.__filter:
stream.extend(self.__filter.convert())
for action in self.__actions:
stream.extend(action.convert())
stream.insert(0, ["add", self.__name, backend])
@ -188,7 +199,11 @@ class JailReader(ConfigReader):
#@staticmethod
def extractOptions(option):
option_name, optstr = JailReader.optionCRE.match(option).groups()
match = JailReader.optionCRE.match(option)
if not match:
# TODO propper error handling
return None, None
option_name, optstr = match.groups()
option_opts = dict()
if optstr:
for optmatch in JailReader.optionExtractRE.finditer(optstr):

5
fail2ban/client/jailsreader.py
View File

@ -60,6 +60,7 @@ class JailsReader(ConfigReader):
sections = [ section ]
# Get the options of all jails.
parse_status = True
for sec in sections:
jail = JailReader(sec, basedir=self.getBaseDir(),
force_enable=self.__force_enable)
@ -71,8 +72,8 @@ class JailsReader(ConfigReader):
self.__jails.append(jail)
else:
logSys.error("Errors in jail %r. Skipping..." % sec)
return False
return True
parse_status = False
return parse_status
def convert(self, allow_no_files=False):
"""Convert read before __opts and jails to the commands stream

10
fail2ban/protocol.py
View File

@ -43,6 +43,12 @@ protocol = [
["get loglevel", "gets the logging level"],
["set logtarget <TARGET>", "sets logging target to <TARGET>. Can be STDOUT, STDERR, SYSLOG or a file"],
["get logtarget", "gets logging target"],
["flushlogs", "flushes the logtarget if a file and reopens it. For log rotation."],
['', "DATABASE", ""],
["set dbfile <FILE>", "set the location of fail2ban persistent datastore. Set to \"None\" to disable"],
["get dbfile", "get the location of fail2ban persistent datastore"],
["set dbpurgeage <SECONDS>", "sets the max age in <SECONDS> that history of bans will be kept"],
["get dbpurgeage", "gets the max age in seconds that history of bans will be kept"],
['', "JAIL CONTROL", ""],
["add <JAIL> <BACKEND>", "creates <JAIL> using <BACKEND>"],
["start <JAIL>", "starts the jail <JAIL>"],
@ -52,13 +58,14 @@ protocol = [
["set <JAIL> idle on|off", "sets the idle state of <JAIL>"],
["set <JAIL> addignoreip <IP>", "adds <IP> to the ignore list of <JAIL>"],
["set <JAIL> delignoreip <IP>", "removes <IP> from the ignore list of <JAIL>"],
["set <JAIL> addlogpath <FILE>", "adds <FILE> to the monitoring list of <JAIL>"],
["set <JAIL> addlogpath <FILE> ['tail']", "adds <FILE> to the monitoring list of <JAIL>, optionally starting at the 'tail' of the file (default 'head')."],
["set <JAIL> dellogpath <FILE>", "removes <FILE> from the monitoring list of <JAIL>"],
["set <JAIL> logencoding <ENCODING>", "sets the <ENCODING> of the log files for <JAIL>"],
["set <JAIL> addjournalmatch <MATCH>", "adds <MATCH> to the journal filter of <JAIL>"],
["set <JAIL> deljournalmatch <MATCH>", "removes <MATCH> from the journal filter of <JAIL>"],
["set <JAIL> addfailregex <REGEX>", "adds the regular expression <REGEX> which must match failures for <JAIL>"],
["set <JAIL> delfailregex <INDEX>", "removes the regular expression at <INDEX> for failregex"],
["set <JAIL> ignorecommand <VALUE>", "sets ignorecommand of <JAIL>"],
["set <JAIL> addignoreregex <REGEX>", "adds the regular expression <REGEX> which should match pattern to exclude for <JAIL>"],
["set <JAIL> delignoreregex <INDEX>", "removes the regular expression at <INDEX> for ignoreregex"],
["set <JAIL> findtime <TIME>", "sets the number of seconds <TIME> for which the filter will look back for <JAIL>"],
@ -84,6 +91,7 @@ protocol = [
["get <JAIL> logencoding <ENCODING>", "gets the <ENCODING> of the log files for <JAIL>"],
["get <JAIL> journalmatch", "gets the journal filter match for <JAIL>"],
["get <JAIL> ignoreip", "gets the list of ignored IP addresses for <JAIL>"],
["get <JAIL> ignorecommand", "gets ignorecommand of <JAIL>"],
["get <JAIL> failregex", "gets the list of regular expressions which matches the failures for <JAIL>"],
["get <JAIL> ignoreregex", "gets the list of regular expressions which matches patterns to ignore for <JAIL>"],
["get <JAIL> findtime", "gets the time for which the filter will look back for failures for <JAIL>"],

17
fail2ban/server/action.py
View File

@ -295,7 +295,7 @@ class Action:
#@staticmethod
def escapeTag(tag):
for c in '\\#&;`|*?~<>^()[]{}$\n\'"':
for c in '\\#&;`|*?~<>^()[]{}$\'"':
if c in tag:
tag = tag.replace(c, '\\' + c)
return tag
@ -314,12 +314,15 @@ class Action:
"""
string = query
for tag, value in aInfo.iteritems():
value = str(value) # assure string
if tag == 'matches':
# That one needs to be escaped since its content is
# out of our control
value = Action.escapeTag(value)
string = string.replace('<' + tag + '>', value)
if "<%s>" % tag in query:
if callable(value):
value = value()
value = str(value) # assure string
if tag.endswith('matches'):
# That one needs to be escaped since its content is
# out of our control
value = Action.escapeTag(value)
string = string.replace('<' + tag + '>', value)
# New line
string = string.replace("<br>", '\n')
return string

15
fail2ban/server/actions.py
View File

@ -183,7 +183,20 @@ class Actions(JailThread):
aInfo["ip"] = bTicket.getIP()
aInfo["failures"] = bTicket.getAttempt()
aInfo["time"] = bTicket.getTime()
aInfo["matches"] = "".join(bTicket.getMatches())
aInfo["matches"] = "\n".join(bTicket.getMatches())
if self.jail.getDatabase() is not None:
aInfo["ipmatches"] = lambda: "\n".join(
self.jail.getDatabase().getBansMerged(
ip=bTicket.getIP()).getMatches())
aInfo["ipjailmatches"] = lambda: "\n".join(
self.jail.getDatabase().getBansMerged(
ip=bTicket.getIP(), jail=self.jail).getMatches())
aInfo["ipfailures"] = lambda: "\n".join(
self.jail.getDatabase().getBansMerged(
ip=bTicket.getIP()).getAttempt())
aInfo["ipjailfailures"] = lambda: "\n".join(
self.jail.getDatabase().getBansMerged(
ip=bTicket.getIP(), jail=self.jail).getAttempt())
if self.__banManager.addBanTicket(bTicket):
logSys.warning("[%s] Ban %s" % (self.jail.getName(), aInfo["ip"]))
for action in self.__actions:

2
fail2ban/server/asyncserver.py
View File

@ -76,7 +76,7 @@ class RequestHandler(asynchat.async_chat):
# Serializes the response.
message = dumps(message, HIGHEST_PROTOCOL)
# Sends the response to the client.
self.send(message + RequestHandler.END_STRING)
self.push(message + RequestHandler.END_STRING)
# Closes the channel.
self.close_when_done()

307
fail2ban/server/database.py Normal file
View File

@ -0,0 +1,307 @@
# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
# vi: set ft=python sts=4 ts=4 sw=4 noet :
# This file is part of Fail2Ban.
#
# Fail2Ban is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# Fail2Ban is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Fail2Ban; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
__author__ = "Steven Hiscocks"
__copyright__ = "Copyright (c) 2013 Steven Hiscocks"
__license__ = "GPL"
import logging
import sys
import shutil, time
import sqlite3
import json
import locale
from functools import wraps
from fail2ban.server.mytime import MyTime
from fail2ban.server.ticket import FailTicket
# Gets the instance of the logger.
logSys = logging.getLogger(__name__)
if sys.version_info >= (3,):
sqlite3.register_adapter(
dict,
lambda x: json.dumps(x, ensure_ascii=False).encode(
locale.getpreferredencoding(), 'replace'))
sqlite3.register_converter(
"JSON",
lambda x: json.loads(x.decode(
locale.getpreferredencoding(), 'replace')))
else:
sqlite3.register_adapter(dict, json.dumps)
sqlite3.register_converter("JSON", json.loads)
def commitandrollback(f):
@wraps(f)
def wrapper(self, *args, **kwargs):
with self._db: # Auto commit and rollback on exception
return f(self, self._db.cursor(), *args, **kwargs)
return wrapper
class Fail2BanDb(object):
__version__ = 2
# Note all _TABLE_* strings must end in ';' for py26 compatibility
_TABLE_fail2banDb = "CREATE TABLE fail2banDb(version INTEGER);"
_TABLE_jails = "CREATE TABLE jails(" \
"name TEXT NOT NULL UNIQUE, " \
"enabled INTEGER NOT NULL DEFAULT 1" \
");" \
"CREATE INDEX jails_name ON jails(name);"
_TABLE_logs = "CREATE TABLE logs(" \
"jail TEXT NOT NULL, " \
"path TEXT, " \
"firstlinemd5 TEXT, " \
"lastfilepos INTEGER DEFAULT 0, " \
"FOREIGN KEY(jail) REFERENCES jails(name) ON DELETE CASCADE, " \
"UNIQUE(jail, path)," \
"UNIQUE(jail, path, firstlinemd5)" \
");" \
"CREATE INDEX logs_path ON logs(path);" \
"CREATE INDEX logs_jail_path ON logs(jail, path);"
#TODO: systemd journal features \
#"journalmatch TEXT, " \
#"journlcursor TEXT, " \
#"lastfiletime INTEGER DEFAULT 0, " # is this easily available \
_TABLE_bans = "CREATE TABLE bans(" \
"jail TEXT NOT NULL, " \
"ip TEXT, " \
"timeofban INTEGER NOT NULL, " \
"data JSON, " \
"FOREIGN KEY(jail) REFERENCES jails(name) " \
");" \
"CREATE INDEX bans_jail_timeofban_ip ON bans(jail, timeofban);" \
"CREATE INDEX bans_jail_ip ON bans(jail, ip);" \
"CREATE INDEX bans_ip ON bans(ip);" \
def __init__(self, filename, purgeAge=24*60*60):
try:
self._db = sqlite3.connect(
filename, check_same_thread=False,
detect_types=sqlite3.PARSE_DECLTYPES)
self._dbFilename = filename
self._purgeAge = purgeAge
self._bansMergedCache = {}
logSys.info(
"Connected to fail2ban persistent database '%s'", filename)
except sqlite3.OperationalError, e:
logSys.error(
"Error connecting to fail2ban persistent database '%s': %s",
filename, e.args[0])
raise
cur = self._db.cursor()
cur.execute("PRAGMA foreign_keys = ON;")
try:
cur.execute("SELECT version FROM fail2banDb LIMIT 1")
except sqlite3.OperationalError:
logSys.warning("New database created. Version '%i'",
self.createDb())
else:
version = cur.fetchone()[0]
if version < Fail2BanDb.__version__:
newversion = self.updateDb(version)
if newversion == Fail2BanDb.__version__:
logSys.warning( "Database updated from '%i' to '%i'",
version, newversion)
else:
logSys.error( "Database update failed to acheive version '%i'"
": updated from '%i' to '%i'",
Fail2BanDb.__version__, version, newversion)
raise Exception('Failed to fully update')
finally:
cur.close()
def getFilename(self):
return self._dbFilename
def getPurgeAge(self):
return self._purgeAge
def setPurgeAge(self, value):
self._purgeAge = int(value)
@commitandrollback
def createDb(self, cur):
# Version info
cur.executescript(Fail2BanDb._TABLE_fail2banDb)
cur.execute("INSERT INTO fail2banDb(version) VALUES(?)",
(Fail2BanDb.__version__, ))
# Jails
cur.executescript(Fail2BanDb._TABLE_jails)
# Logs
cur.executescript(Fail2BanDb._TABLE_logs)
# Bans
cur.executescript(Fail2BanDb._TABLE_bans)
cur.execute("SELECT version FROM fail2banDb LIMIT 1")
return cur.fetchone()[0]
@commitandrollback
def updateDb(self, cur, version):
self.dbBackupFilename = self._dbFilename + '.' + time.strftime('%Y%m%d-%H%M%S', MyTime.gmtime())
shutil.copyfile(self._dbFilename, self.dbBackupFilename)
if version > Fail2BanDb.__version__:
raise NotImplementedError(
"Attempt to travel to future version of database ...how did you get here??")
if version < 2:
cur.executescript("BEGIN TRANSACTION;"
"CREATE TEMPORARY TABLE logs_temp AS SELECT * FROM logs;"
"DROP TABLE logs;"
"%s;"
"INSERT INTO logs SELECT * from logs_temp;"
"DROP TABLE logs_temp;"
"UPDATE fail2banDb SET version = 2;"
"COMMIT;" % Fail2BanDb._TABLE_logs)
cur.execute("SELECT version FROM fail2banDb LIMIT 1")
return cur.fetchone()[0]
@commitandrollback
def addJail(self, cur, jail):
cur.execute(
"INSERT OR REPLACE INTO jails(name, enabled) VALUES(?, 1)",
(jail.getName(),))
def delJail(self, jail):
return self.delJailName(jail.getName())
@commitandrollback
def delJailName(self, cur, name):
# Will be deleted by purge as appropriate
cur.execute(
"UPDATE jails SET enabled=0 WHERE name=?", (name, ))
@commitandrollback
def delAllJails(self, cur):
# Will be deleted by purge as appropriate
cur.execute("UPDATE jails SET enabled=0")
@commitandrollback
def getJailNames(self, cur):
cur.execute("SELECT name FROM jails")
return set(row[0] for row in cur.fetchmany())
@commitandrollback
def addLog(self, cur, jail, container):
lastLinePos = None
cur.execute(
"SELECT firstlinemd5, lastfilepos FROM logs "
"WHERE jail=? AND path=?",
(jail.getName(), container.getFileName()))
try:
firstLineMD5, lastLinePos = cur.fetchone()
except TypeError:
firstLineMD5 = False
cur.execute(
"INSERT OR REPLACE INTO logs(jail, path, firstlinemd5, lastfilepos) "
"VALUES(?, ?, ?, ?)",
(jail.getName(), container.getFileName(),
container.getHash(), container.getPos()))
if container.getHash() != firstLineMD5:
lastLinePos = None
return lastLinePos
@commitandrollback
def getLogPaths(self, cur, jail=None):
query = "SELECT path FROM logs"
queryArgs = []
if jail is not None:
query += " WHERE jail=?"
queryArgs.append(jail.getName())
cur.execute(query, queryArgs)
return set(row[0] for row in cur.fetchmany())
@commitandrollback
def updateLog(self, cur, *args, **kwargs):
self._updateLog(cur, *args, **kwargs)
def _updateLog(self, cur, jail, container):
cur.execute(
"UPDATE logs SET firstlinemd5=?, lastfilepos=? "
"WHERE jail=? AND path=?",
(container.getHash(), container.getPos(),
jail.getName(), container.getFileName()))
@commitandrollback
def addBan(self, cur, jail, ticket):
self._bansMergedCache = {}
#TODO: Implement data parts once arbitrary match keys completed
cur.execute(
"INSERT INTO bans(jail, ip, timeofban, data) VALUES(?, ?, ?, ?)",
(jail.getName(), ticket.getIP(), ticket.getTime(),
{"matches": ticket.getMatches(),
"failures": ticket.getAttempt()}))
@commitandrollback
def _getBans(self, cur, jail=None, bantime=None, ip=None):
query = "SELECT ip, timeofban, data FROM bans WHERE 1"
queryArgs = []
if jail is not None:
query += " AND jail=?"
queryArgs.append(jail.getName())
if bantime is not None:
query += " AND timeofban > ?"
queryArgs.append(MyTime.time() - bantime)
if ip is not None:
query += " AND ip=?"
queryArgs.append(ip)
query += " ORDER BY timeofban"
return cur.execute(query, queryArgs)
def getBans(self, **kwargs):
tickets = []
for ip, timeofban, data in self._getBans(**kwargs):
#TODO: Implement data parts once arbitrary match keys completed
tickets.append(FailTicket(ip, timeofban, data['matches']))
tickets[-1].setAttempt(data['failures'])
return tickets
def getBansMerged(self, ip, jail=None, **kwargs):
cacheKey = ip if jail is None else "%s|%s" % (ip, jail.getName())
if cacheKey in self._bansMergedCache:
return self._bansMergedCache[cacheKey]
matches = []
failures = 0
for ip, timeofban, data in self._getBans(ip=ip, jail=jail, **kwargs):
#TODO: Implement data parts once arbitrary match keys completed
matches.extend(data['matches'])
failures += data['failures']
ticket = FailTicket(ip, timeofban, matches)
ticket.setAttempt(failures)
self._bansMergedCache[cacheKey] = ticket
return ticket
@commitandrollback
def purge(self, cur):
self._bansMergedCache = {}
cur.execute(
"DELETE FROM bans WHERE timeofban < ?",
(MyTime.time() - self._purgeAge, ))
cur.execute(
"DELETE FROM jails WHERE enabled = 0 "
"AND NOT EXISTS(SELECT * FROM bans WHERE jail = jails.name)")

2
fail2ban/server/datedetector.py
View File

@ -140,7 +140,7 @@ class DateDetector:
date = template.getDate(line)
if date is None:
continue
logSys.debug("Got time %i for \"%r\" using template %s" % (date[0], date[1].group(), template.getName()))
logSys.debug("Got time %f for \"%r\" using template %s" % (date[0], date[1].group(), template.getName()))
return date
except ValueError:
pass

5
fail2ban/server/datetemplate.py
View File

@ -82,7 +82,7 @@ class DateEpoch(DateTemplate):
def __init__(self):
DateTemplate.__init__(self)
self.setRegex("(?:^|(?P<selinux>(?<=audit\()))\d{10}(?:\.\d{3,6})?(?(selinux)(?=:\d+\)))")
self.setRegex("(?:^|(?P<square>(?<=^\[))|(?P<selinux>(?<=audit\()))\d{10}(?:\.\d{3,6})?(?(selinux)(?=:\d+\))(?(square)(?=\])))")
def getDate(self, line):
dateMatch = self.matchDate(line)
@ -208,7 +208,8 @@ class DateStrptime(DateTemplate):
# If it is Jan 1st, it is either really Jan 1st or there
# is neither month nor day in the log.
# NOTE: Possibly makes week/year day incorrect
date = date.replace(month=MyTime.gmtime()[1], day=1)
date = date.replace(
month=MyTime.gmtime()[1], day=MyTime.gmtime()[2])
if date.tzinfo:
return ( calendar.timegm(date.utctimetuple()), dateMatch )

68
fail2ban/server/failregex.py
View File

@ -21,7 +21,7 @@ __author__ = "Cyril Jaquier"
__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
__license__ = "GPL"
import re, sre_constants
import re, sre_constants, sys
##
# Regular expression class.
@ -72,10 +72,11 @@ class Regex:
# Sets an internal cache (match object) in order to avoid searching for
# the pattern again. This method must be called before calling any other
# method of this object.
# @param value the line
# @param a list of tupples. The tupples are ( prematch, datematch, postdatematch )
def search(self, value):
self._matchCache = self._regexObj.search(value)
def search(self, tupleLines):
self._matchCache = self._regexObj.search(
"\n".join("".join(value[::2]) for value in tupleLines) + "\n")
if self.hasMatched():
# Find start of the first line where the match was found
try:
@ -89,8 +90,26 @@ class Regex:
"\n", self._matchCache.end() - 1) + 1
except ValueError:
self._matchLineEnd = len(self._matchCache.string)
##
lineCount1 = self._matchCache.string.count(
"\n", 0, self._matchLineStart)
lineCount2 = self._matchCache.string.count(
"\n", 0, self._matchLineEnd)
self._matchedTupleLines = tupleLines[lineCount1:lineCount2]
self._unmatchedTupleLines = tupleLines[:lineCount1]
n = 0
for skippedLine in self.getSkippedLines():
for m, matchedTupleLine in enumerate(
self._matchedTupleLines[n:]):
if "".join(matchedTupleLine[::2]) == skippedLine:
self._unmatchedTupleLines.append(
self._matchedTupleLines.pop(n+m))
n += m
break
self._unmatchedTupleLines.extend(tupleLines[lineCount2:])
# Checks if the previous call to search() matched.
#
# @return True if a match was found, False otherwise
@ -114,10 +133,16 @@ class Regex:
n = 0
while True:
try:
skippedLines += self._matchCache.group("skiplines%i" % n)
if self._matchCache.group("skiplines%i" % n) is not None:
skippedLines += self._matchCache.group("skiplines%i" % n)
n += 1
except IndexError:
break
# KeyError is because of PyPy issue1665 affecting pypy <= 2.2.1
except KeyError:
if 'PyPy' not in sys.version: # pragma: no cover - not sure this is even reachable
raise
break
return skippedLines.splitlines(False)
##
@ -125,15 +150,18 @@ class Regex:
#
# This returns unmatched lines including captured by the <SKIPLINES> tag.
# @return list of unmatched lines
def getUnmatchedTupleLines(self):
if not self.hasMatched():
return []
else:
return self._unmatchedTupleLines
def getUnmatchedLines(self):
if not self.hasMatched():
return []
unmatchedLines = (
self._matchCache.string[:self._matchLineStart].splitlines(False)
+ self.getSkippedLines()
+ self._matchCache.string[self._matchLineEnd:].splitlines(False))
return unmatchedLines
else:
return ["".join(line) for line in self._unmatchedTupleLines]
##
# Returns matched lines.
@ -141,14 +169,18 @@ class Regex:
# This returns matched lines by excluding those captured
# by the <SKIPLINES> tag.
# @return list of matched lines
def getMatchedTupleLines(self):
if not self.hasMatched():
return []
else:
return self._matchedTupleLines
def getMatchedLines(self):
if not self.hasMatched():
return []
matchedLines = self._matchCache.string[
self._matchLineStart:self._matchLineEnd].splitlines(False)
return [line for line in matchedLines
if line not in self.getSkippedLines()]
else:
return ["".join(line) for line in self._matchedTupleLines]
##
# Exception dedicated to the class Regex.

200
fail2ban/server/filter.py
View File

@ -21,8 +21,6 @@ __author__ = "Cyril Jaquier and Fail2Ban Contributors"
__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2011-2013 Yaroslav Halchenko"
__license__ = "GPL"
import sys
from failmanager import FailManagerEmpty
from failmanager import FailManager
from ticket import FailTicket
@ -31,6 +29,7 @@ from datedetector import DateDetector
from datetemplate import DatePatternRegex, DateISO8601, DateEpoch, DateTai64n
from mytime import MyTime
from failregex import FailRegex, Regex, RegexException
from action import Action
import logging, re, os, fcntl, time, sys, locale, codecs
@ -75,6 +74,8 @@ class Filter(JailThread):
## Store last time stamp, applicable for multi-line
self.__lastTimeText = ""
self.__lastDate = None
## External command
self.__ignoreCommand = False
self.dateDetector = DateDetector()
self.dateDetector.addDefaultTemplate()
@ -199,8 +200,10 @@ class Filter(JailThread):
# @param pattern the date template pattern
def setDatePattern(self, pattern):
dateDetector = DateDetector()
if pattern.upper() == "ISO8601":
if pattern is None:
self.dateDetector = None
return
elif pattern.upper() == "ISO8601":
template = DateISO8601()
template.setName("ISO8601")
elif pattern.upper() == "EPOCH":
@ -215,8 +218,8 @@ class Filter(JailThread):
template.setPattern(pattern[1:], anchor=True)
else:
template.setPattern(pattern, anchor=False)
dateDetector.appendTemplate(template)
self.dateDetector = dateDetector
self.dateDetector = DateDetector()
self.dateDetector.appendTemplate(template)
logSys.info("Date pattern set to `%r`: `%s`" %
(pattern, template.getName()))
logSys.debug("Date pattern regex for %r: %s" %
@ -228,17 +231,18 @@ class Filter(JailThread):
# @return pattern of the date template pattern
def getDatePattern(self):
templates = self.dateDetector.getTemplates()
if len(templates) > 1:
return None # Default Detectors in use
elif len(templates) == 1:
if hasattr(templates[0], "getPattern"):
pattern = templates[0].getPattern()
if templates[0].getRegex()[0] == "^":
pattern = "^" + pattern
else:
pattern = None
return pattern, templates[0].getName()
if self.dateDetector is not None:
templates = self.dateDetector.getTemplates()
if len(templates) > 1:
return None, "Default Detectors"
elif len(templates) == 1:
if hasattr(templates[0], "getPattern"):
pattern = templates[0].getPattern()
if templates[0].getRegex()[0] == "^":
pattern = "^" + pattern
else:
pattern = None
return pattern, templates[0].getName()
##
# Set the maximum retry value.
@ -286,6 +290,20 @@ class Filter(JailThread):
def run(self): # pragma: no cover
raise Exception("run() is abstract")
##
# Set external command, for ignoredips
#
def setIgnoreCommand(self, command):
self.__ignoreCommand = command
##
# Get external command, for ignoredips
#
def getIgnoreCommand(self):
return self.__ignoreCommand
##
# Ban an IP - http://blogs.buanzo.com.ar/2009/04/fail2ban-patch-ban-ip-address-manually.html
# Arturo 'Buanzo' Busleiman <buanzo@buanzo.com.ar>
@ -293,6 +311,9 @@ class Filter(JailThread):
# to enable banip fail2ban-client BAN command
def addBannedIP(self, ip):
if self.inIgnoreIPList(ip):
logSys.warning('Requested to manually ban an ignored IP %s. User knows best. Proceeding to ban it.' % ip)
unixTime = MyTime.time()
for i in xrange(self.failManager.getMaxRetry()):
self.failManager.addFailure(FailTicket(ip, unixTime))
@ -355,36 +376,45 @@ class Filter(JailThread):
continue
if a == b:
return True
if self.__ignoreCommand:
command = Action.replaceTag(self.__ignoreCommand, { 'ip': ip } )
logSys.debug('ignore command: ' + command)
return Action.executeCmd(command)
return False
def processLine(self, line, returnRawHost=False, checkAllRegex=False):
def processLine(self, line, date=None, returnRawHost=False,
checkAllRegex=False):
"""Split the time portion from log msg and return findFailures on them
"""
l = line.rstrip('\r\n')
logSys.log(7, "Working on line %r", line)
timeMatch = self.dateDetector.matchTime(l)
if timeMatch:
# Lets split into time part and log part of the line
timeText = timeMatch.group()
# Lets leave the beginning in as well, so if there is no
# anchore at the beginning of the time regexp, we don't
# at least allow injection. Should be harmless otherwise
logLine = l[:timeMatch.start()] + l[timeMatch.end():]
if date:
tupleLine = line
else:
timeText = None
logLine = l
l = line.rstrip('\r\n')
logSys.log(7, "Working on line %r", line)
return logLine, self.findFailure(timeText, logLine, returnRawHost, checkAllRegex)
timeMatch = self.dateDetector.matchTime(l)
if timeMatch:
tupleLine = (
l[:timeMatch.start()],
l[timeMatch.start():timeMatch.end()],
l[timeMatch.end():])
else:
tupleLine = (l, "", "")
def processLineAndAdd(self, line):
return "".join(tupleLine[::2]), self.findFailure(
tupleLine, date, returnRawHost, checkAllRegex)
def processLineAndAdd(self, line, date=None):
"""Processes the line for failures and populates failManager
"""
for element in self.processLine(line)[1]:
for element in self.processLine(line, date)[1]:
failregex = element[0]
ip = element[1]
unixTime = element[2]
lines = element[3]
logSys.debug("Processing line with time:%s and ip:%s"
% (unixTime, ip))
if unixTime < MyTime.time() - self.getFindTime():
@ -396,7 +426,7 @@ class Filter(JailThread):
continue
logSys.debug("Found %s" % ip)
## print "D: Adding a ticket for %s" % ((ip, unixTime, [line]),)
self.failManager.addFailure(FailTicket(ip, unixTime, [line]))
self.failManager.addFailure(FailTicket(ip, unixTime, lines))
##
# Returns true if the line should be ignored.
@ -405,9 +435,9 @@ class Filter(JailThread):
# @param line: the line
# @return: a boolean
def ignoreLine(self, line):
def ignoreLine(self, tupleLines):
for ignoreRegexIndex, ignoreRegex in enumerate(self.__ignoreRegex):
ignoreRegex.search(line)
ignoreRegex.search(tupleLines)
if ignoreRegex.hasMatched():
return ignoreRegexIndex
return None
@ -419,18 +449,22 @@ class Filter(JailThread):
# to find the logging time.
# @return a dict with IP and timestamp.
def findFailure(self, timeText, logLine,
returnRawHost=False, checkAllRegex=False):
def findFailure(self, tupleLine, date=None, returnRawHost=False,
checkAllRegex=False):
failList = list()
# Checks if we must ignore this line.
if self.ignoreLine(logLine) is not None:
if self.ignoreLine([tupleLine[::2]]) is not None:
# The ignoreregex matched. Return.
logSys.log(7, "Matched ignoreregex and was \"%s\" ignored", logLine)
logSys.log(7, "Matched ignoreregex and was \"%s\" ignored",
"".join(tupleLine[::2]))
return failList
if timeText:
timeText = tupleLine[1]
if date:
self.__lastTimeText = timeText
self.__lastDate = date
elif timeText:
dateTimeMatch = self.dateDetector.getTime(timeText)
@ -446,49 +480,53 @@ class Filter(JailThread):
self.__lastTimeText = timeText
self.__lastDate = date
else:
timeText = self.__lastTimeText or logLine
timeText = self.__lastTimeText or "".join(tupleLine[::2])
date = self.__lastDate
self.__lineBuffer = (self.__lineBuffer + [logLine])[-self.__lineBufferSize:]
logLine = "\n".join(self.__lineBuffer) + "\n"
self.__lineBuffer = (
self.__lineBuffer + [tupleLine])[-self.__lineBufferSize:]
# Iterates over all the regular expressions.
for failRegexIndex, failRegex in enumerate(self.__failRegex):
failRegex.search(logLine)
failRegex.search(self.__lineBuffer)
if failRegex.hasMatched():
# Checks if we must ignore this match.
if self.ignoreLine(
"\n".join(failRegex.getMatchedLines()) + "\n") \
is not None:
# The ignoreregex matched. Remove ignored match.
self.__lineBuffer = failRegex.getUnmatchedLines()
logSys.log(7, "Matched ignoreregex and was ignored")
continue
# The failregex matched.
logSys.log(7, "Matched %s", failRegex)
# Checks if we must ignore this match.
if self.ignoreLine(failRegex.getMatchedTupleLines()) \
is not None:
# The ignoreregex matched. Remove ignored match.
self.__lineBuffer = failRegex.getUnmatchedTupleLines()
logSys.log(7, "Matched ignoreregex and was ignored")
if not checkAllRegex:
break
else:
continue
if date is None:
logSys.debug("Found a match for %r but no valid date/time "
"found for %r. Please try setting a custom "
"date pattern (see man page jail.conf(5)). "
"If format is complex, please "
"file a detailed issue on"
" https://github.com/fail2ban/fail2ban/issues "
"in order to get support for this format."
% (logLine, timeText))
logSys.debug(
"Found a match for %r but no valid date/time "
"found for %r. Please try setting a custom "
"date pattern (see man page jail.conf(5)). "
"If format is complex, please "
"file a detailed issue on"
" https://github.com/fail2ban/fail2ban/issues "
"in order to get support for this format."
% ("\n".join(failRegex.getMatchedLines()), timeText))
else:
self.__lineBuffer = failRegex.getUnmatchedLines()
self.__lineBuffer = failRegex.getUnmatchedTupleLines()
try:
host = failRegex.getHost()
if returnRawHost:
failList.append([failRegexIndex, host, date])
failList.append([failRegexIndex, host, date,
failRegex.getMatchedLines()])
if not checkAllRegex:
break
else:
ipMatch = DNSUtils.textToIp(host, self.__useDns)
if ipMatch:
for ip in ipMatch:
failList.append([failRegexIndex, ip, date])
failList.append([failRegexIndex, ip, date,
failRegex.getMatchedLines()])
if not checkAllRegex:
break
except RegexException, e: # pragma: no cover - unsure if reachable
@ -527,6 +565,11 @@ class FileFilter(Filter):
logSys.error(path + " already exists")
else:
container = FileContainer(path, self.getLogEncoding(), tail)
db = self.jail.getDatabase()
if db is not None:
lastpos = db.addLog(self.jail, container)
if lastpos and not tail:
container.setPos(lastpos)
self.__logPath.append(container)
logSys.info("Added logfile = %s" % path)
self._addLogPath(path) # backend specific
@ -546,11 +589,14 @@ class FileFilter(Filter):
for log in self.__logPath:
if log.getFileName() == path:
self.__logPath.remove(log)
db = self.jail.getDatabase()
if db is not None:
db.updateLog(self.jail, log)
logSys.info("Removed logfile = %s" % path)
self._delLogPath(path)
return
def _delLogPath(self, path):
def _delLogPath(self, path): # pragma: no cover - overwritten function
# nothing to do by default
# to be overridden by backends
pass
@ -644,6 +690,9 @@ class FileFilter(Filter):
break
self.processLineAndAdd(line)
container.close()
db = self.jail.getDatabase()
if db is not None:
db.updateLog(self.jail, container)
return True
def status(self):
@ -682,7 +731,7 @@ class FileContainer:
try:
firstLine = handler.readline()
# Computes the MD5 of the first line.
self.__hash = md5sum(firstLine).digest()
self.__hash = md5sum(firstLine).hexdigest()
# Start at the beginning of file if tail mode is off.
if tail:
handler.seek(0, 2)
@ -702,6 +751,15 @@ class FileContainer:
def getEncoding(self):
return self.__encoding
def getHash(self):
return self.__hash
def getPos(self):
return self.__pos
def setPos(self, value):
self.__pos = value
def open(self):
self.__handler = open(self.__filename, 'rb')
# Set the file descriptor to be FD_CLOEXEC
@ -717,7 +775,7 @@ class FileContainer:
return False
firstLine = self.__handler.readline()
# Computes the MD5 of the first line.
myHash = md5sum(firstLine).digest()
myHash = md5sum(firstLine).hexdigest()
## print "D: fn=%s hashes=%s/%s inos=%s/%s pos=%s rotate=%s" % (
## self.__filename, self.__hash, myHash, stats.st_ino, self.__ino, self.__pos,
## self.__hash != myHash or self.__ino != stats.st_ino)
@ -790,7 +848,7 @@ class DNSUtils:
Thanks to Kevin Drapel.
"""
try:
return socket.gethostbyname_ex(dns)[2]
return set(socket.gethostbyname_ex(dns)[2])
except socket.error, e:
logSys.warn("Unable to find a corresponding IP address for %s: %s"
% (dns, e))

4
fail2ban/server/filterpyinotify.py
View File

@ -115,9 +115,6 @@ class FilterPyinotify(FileFilter):
wd = self.__monitor.add_watch(path, pyinotify.IN_MODIFY)
self.__watches.update(wd)
logSys.debug("Added file watcher for %s", path)
# process the file since we did get even
self._process_file(path)
def _delFileWatcher(self, path):
wdInt = self.__watches[path]
@ -143,6 +140,7 @@ class FilterPyinotify(FileFilter):
logSys.debug("Added monitor for the parent directory %s", path_dir)
self._addFileWatcher(path)
self._process_file(path)
##

24
fail2ban/server/filtersystemd.py
View File

@ -22,7 +22,7 @@ __author__ = "Steven Hiscocks"
__copyright__ = "Copyright (c) 2013 Steven Hiscocks"
__license__ = "GPL"
import logging, datetime
import logging, datetime, time
from distutils.version import LooseVersion
from systemd import journal
@ -57,7 +57,7 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
# Initialise systemd-journal connection
self.__journal = journal.Reader(converters={'__CURSOR': lambda x: x})
self.__matches = []
self.setDatePattern("ISO8601")
self.setDatePattern(None)
logSys.debug("Created FilterSystemd")
@ -162,8 +162,7 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
@staticmethod
def formatJournalEntry(logentry):
logelements = [logentry.get('_SOURCE_REALTIME_TIMESTAMP',
logentry.get('__REALTIME_TIMESTAMP')).isoformat()]
logelements = [""]
if logentry.get('_HOSTNAME'):
logelements.append(logentry['_HOSTNAME'])
if logentry.get('SYSLOG_IDENTIFIER'):
@ -188,18 +187,22 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
logelements.append(logentry.get('MESSAGE', ''))
try:
logline = u" ".join(logelements) + u"\n"
logline = u" ".join(logelements)
except UnicodeDecodeError:
# Python 2, so treat as string
logline = " ".join([str(logline) for logline in logelements]) + "\n"
logline = " ".join([str(logline) for logline in logelements])
except TypeError:
# Python 3, one or more elements bytes
logSys.warning("Error decoding log elements from journal: %s" %
repr(logelements))
logline = self._joinStrAndBytes(logelements) + "\n"
logline = self._joinStrAndBytes(logelements)
logSys.debug("Read systemd journal entry: %s" % repr(logline))
return logline
date = logentry.get('_SOURCE_REALTIME_TIMESTAMP',
logentry.get('__REALTIME_TIMESTAMP'))
logSys.debug("Read systemd journal entry: %r" %
"".join([date.isoformat(), logline]))
return (('', date.isoformat(), logline),
time.mktime(date.timetuple()) + date.microsecond/1.0E6)
##
# Main loop.
@ -232,7 +235,7 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
continue
if logentry:
self.processLineAndAdd(
self.formatJournalEntry(logentry))
*self.formatJournalEntry(logentry))
self.__modified = True
else:
break
@ -243,7 +246,6 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover
self.jail.putFailTicket(ticket)
except FailManagerEmpty:
self.failManager.cleanup(MyTime.time())
self.dateDetector.sortTemplate()
self.__modified = False
self.__journal.wait(self.getSleepTime())
logSys.debug((self.jail is not None and self.jail.getName()

2
fail2ban/server/iso8601.py
View File

@ -41,7 +41,7 @@ __all__ = ["parse_date", "ParseError"]
# Adapted from http://delete.me.uk/2005/03/iso8601.html
ISO8601_REGEX_RAW = "(?P<year>[0-9]{4})-(?P<month>[0-9]{1,2})-(?P<day>[0-9]{1,2})" \
"T(?P<hour>[0-9]{2}):(?P<minute>[0-9]{2})(:(?P<second>[0-9]{2})(\.(?P<fraction>[0-9]+))?)?" \
"(?P<timezone>Z|(([-+])([0-9]{2}):([0-9]{2})))?"
"(?P<timezone>Z|[-+][0-9]{2}(:?[0-9]{2})?)?"
ISO8601_REGEX = re.compile(ISO8601_REGEX_RAW)
TIMEZONE_REGEX = re.compile("(?P<prefix>[+-])(?P<hours>[0-9]{2}):?(?P<minutes>[0-9]{2})?")

19
fail2ban/server/jail.py
View File

@ -37,7 +37,8 @@ class Jail:
# list had .index until 2.6
_BACKENDS = ['pyinotify', 'gamin', 'polling', 'systemd']
def __init__(self, name, backend = "auto"):
def __init__(self, name, backend = "auto", db=None):
self.__db = db
self.setName(name)
self.__queue = Queue.Queue()
self.__filter = None
@ -109,15 +110,20 @@ class Jail:
self.__filter = FilterSystemd(self)
def setName(self, name):
# 20 based on iptable chain name limit of 30 less len('fail2ban-')
if len(name) >= 20:
logSys.warning("Jail name %r might be too long and some commands "
"might not function correctly. Please shorten"
logSys.warning("Jail name %r might be too long and some commands"
" (e.g. iptables) might not function correctly."
" Please shorten"
% name)
self.__name = name
def getName(self):
return self.__name
def getDatabase(self):
return self.__db
def getFilter(self):
return self.__filter
@ -126,6 +132,8 @@ class Jail:
def putFailTicket(self, ticket):
self.__queue.put(ticket)
if self.__db is not None:
self.__db.addBan(self, ticket)
def getFailTicket(self):
try:
@ -136,6 +144,11 @@ class Jail:
def start(self):
self.__filter.start()
self.__action.start()
# Restore any previous valid bans from the database
if self.__db is not None:
for ticket in self.__db.getBans(
jail=self, bantime=self.__action.getBanTime()):
self.__queue.put(ticket)
logSys.info("Jail '%s' started" % self.__name)
def stop(self):

4
fail2ban/server/jails.py
View File

@ -50,13 +50,13 @@ class Jails:
# @param name The name of the jail
# @param backend The backend to use
def add(self, name, backend):
def add(self, name, backend, db=None):
try:
self.__lock.acquire()
if self.__jails.has_key(name):
raise DuplicateJailException(name)
else:
self.__jails[name] = Jail(name, backend)
self.__jails[name] = Jail(name, backend, db)
finally:
self.__lock.release()

70
fail2ban/server/server.py
View File

@ -30,6 +30,7 @@ from filter import FileFilter, JournalFilter
from transmitter import Transmitter
from asyncserver import AsyncServer
from asyncserver import AsyncServerException
from database import Fail2BanDb
from fail2ban import version
import logging, logging.handlers, sys, os, signal
@ -42,6 +43,7 @@ class Server:
self.__loggingLock = Lock()
self.__lock = RLock()
self.__jails = Jails()
self.__db = None
self.__daemon = daemon
self.__transm = Transmitter(self)
self.__asyncServer = AsyncServer(self.__transm)
@ -117,10 +119,14 @@ class Server:
def addJail(self, name, backend):
self.__jails.add(name, backend)
self.__jails.add(name, backend, self.__db)
if self.__db is not None:
self.__db.addJail(self.__jails.get(name))
def delJail(self, name):
self.__jails.remove(name)
if self.__db is not None:
self.__db.delJailName(name)
def startJail(self, name):
try:
@ -169,10 +175,10 @@ class Server:
def getIgnoreIP(self, name):
return self.__jails.getFilter(name).getIgnoreIP()
def addLogPath(self, name, fileName):
def addLogPath(self, name, fileName, tail=False):
filter_ = self.__jails.getFilter(name)
if isinstance(filter_, FileFilter):
filter_.addLogPath(fileName)
filter_.addLogPath(fileName, tail)
def delLogPath(self, name, fileName):
filter_ = self.__jails.getFilter(name)
@ -228,6 +234,12 @@ class Server:
def getDatePattern(self, name):
return self.__jails.getFilter(name).getDatePattern()
def setIgnoreCommand(self, name, value):
self.__jails.getFilter(name).setIgnoreCommand(value)
def getIgnoreCommand(self, name):
return self.__jails.getFilter(name).getIgnoreCommand()
def addFailRegex(self, name, value):
self.__jails.getFilter(name).addFailRegex(value)
@ -403,13 +415,12 @@ class Server:
try:
self.__loggingLock.acquire()
# set a format which is simpler for console use
formatter = logging.Formatter("%(asctime)s %(name)-16s: %(levelname)-6s %(message)s")
formatter = logging.Formatter("%(asctime)s %(name)-16s[%(process)d]: %(levelname)-7s %(message)s")
if target == "SYSLOG":
# Syslog daemons already add date to the message.
formatter = logging.Formatter("%(name)-16s: %(levelname)-6s %(message)s")
formatter = logging.Formatter("%(name)s[%(process)d]: %(levelname)s %(message)s")
facility = logging.handlers.SysLogHandler.LOG_DAEMON
hdlr = logging.handlers.SysLogHandler("/dev/log",
facility = facility)
hdlr = logging.handlers.SysLogHandler("/dev/log", facility=facility)
elif target == "STDOUT":
hdlr = logging.StreamHandler(sys.stdout)
elif target == "STDERR":
@ -418,7 +429,7 @@ class Server:
# Target should be a file
try:
open(target, "a").close()
hdlr = logging.FileHandler(target)
hdlr = logging.handlers.RotatingFileHandler(target)
except IOError:
logSys.error("Unable to log to " + target)
logSys.info("Logging to previous target " + self.__logTarget)
@ -460,6 +471,37 @@ class Server:
finally:
self.__loggingLock.release()
def flushLogs(self):
if self.__logTarget not in ['STDERR', 'STDOUT', 'SYSLOG']:
for handler in logging.getLogger(__name__).parent.parent.handlers:
try:
handler.doRollover()
logSys.info("rollover performed on %s" % self.__logTarget)
except AttributeError:
handler.flush()
logSys.info("flush performed on %s" % self.__logTarget)
return "rolled over"
else:
for handler in logging.getLogger(__name__).parent.parent.handlers:
handler.flush()
logSys.info("flush performed on %s" % self.__logTarget)
return "flushed"
def setDatabase(self, filename):
if self.__jails.size() == 0:
if filename.lower() == "none":
self.__db = None
else:
self.__db = Fail2BanDb(filename)
self.__db.delAllJails()
else:
raise RuntimeError(
"Cannot change database when there are jails present")
def getDatabase(self):
return self.__db
def __createDaemon(self): # pragma: no cover
""" Detach a process from the controlling terminal and run it in the
background as a daemon.
@ -467,6 +509,14 @@ class Server:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/278731
"""
# When the first child terminates, all processes in the second child
# are sent a SIGHUP, so it's ignored.
# We need to set this in the parent process, so it gets inherited by the
# child process, and this makes sure that it is effect even if the parent
# terminates quickly.
signal.signal(signal.SIGHUP, signal.SIG_IGN)
try:
# Fork a child process so the parent can exit. This will return control
# to the command line or shell. This is required so that the new process
@ -489,10 +539,6 @@ class Server:
# leader.
os.setsid()
# When the first child terminates, all processes in the second child
# are sent a SIGHUP, so it's ignored.
signal.signal(signal.SIGHUP, signal.SIG_IGN)
try:
# Fork a second child to prevent zombies. Since the first child is
# a session leader without a controlling terminal, it's possible for

17
fail2ban/server/ticket.py
View File

@ -46,9 +46,20 @@ class Ticket:
self.__matches = matches or []
def __str__(self):
return "%s: ip=%s time=%s #attempts=%d" % \
(self.__class__.__name__.split('.')[-1], self.__ip, self.__time, self.__attempt)
return "%s: ip=%s time=%s #attempts=%d matches=%r" % \
(self.__class__.__name__.split('.')[-1], self.__ip, self.__time, self.__attempt, self.__matches)
def __repr__(self):
return str(self)
def __eq__(self, other):
try:
return self.__ip == other.__ip and \
round(self.__time,2) == round(other.__time,2) and \
self.__attempt == other.__attempt and \
self.__matches == other.__matches
except AttributeError:
return False
def setIP(self, value):
if isinstance(value, basestring):

49
fail2ban/server/transmitter.py
View File

@ -92,6 +92,8 @@ class Transmitter:
value = command[1]
time.sleep(int(value))
return None
elif command[0] == "flushlogs":
return self.__server.flushLogs()
elif command[0] == "set":
return self.__commandSet(command[1:])
elif command[0] == "get":
@ -113,6 +115,21 @@ class Transmitter:
return self.__server.getLogTarget()
else:
raise Exception("Failed to change log target")
#Database
elif name == "dbfile":
self.__server.setDatabase(command[1])
db = self.__server.getDatabase()
if db is None:
return None
else:
return db.getFilename()
elif name == "dbpurgeage":
db = self.__server.getDatabase()
if db is None:
return None
else:
db.setPurgeAge(command[1])
return db.getPurgeAge()
# Jail
elif command[1] == "idle":
if command[2] == "on":
@ -131,10 +148,21 @@ class Transmitter:
value = command[2]
self.__server.delIgnoreIP(name, value)
return self.__server.getIgnoreIP(name)
elif command[1] == "ignorecommand":
value = command[2]
self.__server.setIgnoreCommand(name, value)
return self.__server.getIgnoreCommand(name)
elif command[1] == "addlogpath":
value = command[2:]
for path in value:
self.__server.addLogPath(name, path)
value = command[2]
tail = False
if len(command) == 4:
if command[3].lower() == "tail":
tail = True
elif command[3].lower() != "head":
raise ValueError("File option must be 'head' or 'tail'")
elif len(command) > 4:
raise ValueError("Only one file can be added at a time")
self.__server.addLogPath(name, value, tail)
return self.__server.getLogPath(name)
elif command[1] == "dellogpath":
value = command[2]
@ -257,6 +285,19 @@ class Transmitter:
return self.__server.getLogLevel()
elif name == "logtarget":
return self.__server.getLogTarget()
#Database
elif name == "dbfile":
db = self.__server.getDatabase()
if db is None:
return None
else:
return db.getFilename()
elif name == "dbpurgeage":
db = self.__server.getDatabase()
if db is None:
return None
else:
return db.getPurgeAge()
# Filter
elif command[1] == "logpath":
return self.__server.getLogPath(name)
@ -266,6 +307,8 @@ class Transmitter:
return self.__server.getJournalMatch(name)
elif command[1] == "ignoreip":
return self.__server.getIgnoreIP(name)
elif command[1] == "ignorecommand":
return self.__server.getIgnoreCommand(name)
elif command[1] == "failregex":
return self.__server.getFailRegex(name)
elif command[1] == "ignoreregex":

79
fail2ban/tests/actionstestcase.py Normal file
View File

@ -0,0 +1,79 @@
# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
# vi: set ft=python sts=4 ts=4 sw=4 noet :
# This file is part of Fail2Ban.
#
# Fail2Ban is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# Fail2Ban is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Fail2Ban; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
# Author: Daniel Black
#
__author__ = "Daniel Black"
__copyright__ = "Copyright (c) 2013 Daniel Black"
__license__ = "GPL"
import unittest, time
import sys, os, tempfile
from fail2ban.server.actions import Actions
from dummyjail import DummyJail
class ExecuteActions(unittest.TestCase):
def setUp(self):
"""Call before every test case."""
self.__jail = DummyJail()
self.__actions = Actions(self.__jail)
self.__tmpfile, self.__tmpfilename = tempfile.mkstemp()
def tearDown(self):
os.remove(self.__tmpfilename)
def defaultActions(self):
self.__actions.addAction('ip')
self.__ip = self.__actions.getAction('ip')
self.__ip.setActionStart('echo ip start 64 >> "%s"' % self.__tmpfilename )
self.__ip.setActionBan('echo ip ban <ip> >> "%s"' % self.__tmpfilename )
self.__ip.setActionUnban('echo ip unban <ip> >> "%s"' % self.__tmpfilename )
self.__ip.setActionCheck('echo ip check <ip> >> "%s"' % self.__tmpfilename )
self.__ip.setActionStop('echo ip stop >> "%s"' % self.__tmpfilename )
def testActionsManipulation(self):
self.__actions.addAction('test')
self.assertTrue(self.__actions.getAction('test'))
self.assertTrue(self.__actions.getLastAction())
self.assertRaises(KeyError,self.__actions.getAction,*['nonexistant action'])
self.__actions.addAction('test1')
self.__actions.delAction('test')
self.__actions.delAction('test1')
self.assertRaises(KeyError, self.__actions.getAction, *['test'])
self.assertRaises(IndexError,self.__actions.getLastAction)
self.__actions.setBanTime(127)
self.assertEqual(self.__actions.getBanTime(),127)
self.assertRaises(ValueError, self.__actions.removeBannedIP, '127.0.0.1')
def testActionsOutput(self):
self.defaultActions()
self.__actions.start()
with open(self.__tmpfilename) as f:
time.sleep(3)
self.assertEqual(f.read(),"ip start 64\n")
self.__actions.stop()
self.__actions.join()
self.assertEqual(self.__actions.status(),[("Currently banned", 0 ),
("Total banned", 0 ), ("IP list", [] )])

52
fail2ban/tests/actiontestcase.py
View File

@ -24,41 +24,25 @@ __author__ = "Cyril Jaquier"
__copyright__ = "Copyright (c) 2004 Cyril Jaquier"
__license__ = "GPL"
import unittest, time
import time
import logging, sys
from StringIO import StringIO
from fail2ban.server.action import Action
class ExecuteAction(unittest.TestCase):
from fail2ban.tests.utils import LogCaptureTestCase
class ExecuteAction(LogCaptureTestCase):
def setUp(self):
"""Call before every test case."""
self.__action = Action("Test")
# For extended testing of what gets output into logging
# system, we will redirect it to a string
logSys = logging.getLogger("fail2ban")
# Keep old settings
self._old_level = logSys.level
self._old_handlers = logSys.handlers
# Let's log everything into a string
self._log = StringIO()
logSys.handlers = [logging.StreamHandler(self._log)]
logSys.setLevel(getattr(logging, 'DEBUG'))
LogCaptureTestCase.setUp(self)
def tearDown(self):
"""Call after every test case."""
# print "O: >>%s<<" % self._log.getvalue()
logSys = logging.getLogger("fail2ban")
logSys.handlers = self._old_handlers
logSys.level = self._old_level
LogCaptureTestCase.tearDown(self)
self.__action.execActionStop()
def _is_logged(self, s):
return s in self._log.getvalue()
def testNameChange(self):
self.assertEqual(self.__action.getName(), "Test")
self.__action.setName("Tricky Test")
@ -102,8 +86,28 @@ class ExecuteAction(unittest.TestCase):
"Text 890 text 123 ABC")
self.assertEqual(
self.__action.replaceTag("<matches>",
{'matches': "some >char< should \< be[ escap}ed&"}),
r"some \>char\< should \\\< be\[ escap\}ed\&")
{'matches': "some >char< should \< be[ escap}ed&\n"}),
"some \\>char\\< should \\\\\\< be\\[ escap\\}ed\\&\n")
self.assertEqual(
self.__action.replaceTag("<ipmatches>",
{'ipmatches': "some >char< should \< be[ escap}ed&\n"}),
"some \\>char\\< should \\\\\\< be\\[ escap\\}ed\\&\n")
self.assertEqual(
self.__action.replaceTag("<ipjailmatches>",
{'ipjailmatches': "some >char< should \< be[ escap}ed&\n"}),
"some \\>char\\< should \\\\\\< be\\[ escap\\}ed\\&\n")
# Callable
self.assertEqual(
self.__action.replaceTag("09 <callable> 11",
{'callable': lambda: str(10)}),
"09 10 11")
# As tag not present, therefore callable should not be called
# Will raise ValueError if it is
self.assertEqual(
self.__action.replaceTag("abc",
{'callable': lambda: int("a")}), "abc")
def testExecuteActionBan(self):
self.__action.setActionStart("touch /tmp/fail2ban.test")

172
fail2ban/tests/clientreadertestcase.py
View File

@ -29,12 +29,16 @@ from fail2ban.client.filterreader import FilterReader
from fail2ban.client.jailsreader import JailsReader
from fail2ban.client.actionreader import ActionReader
from fail2ban.client.configurator import Configurator
from fail2ban.tests.utils import LogCaptureTestCase
TEST_FILES_DIR = os.path.join(os.path.dirname(__file__), "files")
if os.path.exists('config/fail2ban.conf'):
if os.path.exists(os.path.join('config','fail2ban.conf')):
CONFIG_DIR='config'
else:
CONFIG_DIR='/etc/fail2ban'
CONFIG_DIR=os.path.join('etc','fail2ban')
IMPERFECT_CONFIG = os.path.join('fail2ban', 'tests','config')
class ConfigReaderTest(unittest.TestCase):
@ -79,7 +83,14 @@ option = %s
self._write('d.conf', 0)
self.assertEqual(self._getoption('d'), 0)
os.chmod(f, 0)
self.assertFalse(self.c.read('d')) # should not be readable BUT present
# fragile test and known to fail e.g. under Cygwin where permissions
# seems to be not enforced, thus condition
if not os.access(f, os.R_OK):
self.assertFalse(self.c.read('d')) # should not be readable BUT present
else:
# SkipTest introduced only in 2.7 thus can't yet use generally
# raise unittest.SkipTest("Skipping on %s -- access rights are not enforced" % platform)
pass
def testOptionalDotDDir(self):
@ -131,11 +142,49 @@ z = 3%(__name__)s
self.assertEqual(self.c.get('section', 'zz'), 'thesection') # __name__ works even 'delayed'
self.assertEqual(self.c.get('section2', 'z'), '3section2') # and differs per section ;)
class JailReaderTest(unittest.TestCase):
def testComments(self):
self.assertFalse(self.c.read('g')) # nothing is there yet
self._write("g.conf", value=None, content="""
[DEFAULT]
# A comment
b = a
c = d ;in line comment
""")
self.assertTrue(self.c.read('g'))
self.assertEqual(self.c.get('DEFAULT', 'b'), 'a')
self.assertEqual(self.c.get('DEFAULT', 'c'), 'd')
class JailReaderTest(LogCaptureTestCase):
def testIncorrectJail(self):
jail = JailReader('XXXABSENTXXX', basedir=CONFIG_DIR)
self.assertRaises(ValueError, jail.read)
def testJailActionEmpty(self):
jail = JailReader('emptyaction', basedir=IMPERFECT_CONFIG)
self.assertTrue(jail.read())
self.assertTrue(jail.getOptions())
self.assertTrue(jail.isEnabled())
self.assertTrue(self._is_logged('No filter set for jail emptyaction'))
self.assertTrue(self._is_logged('No actions were defined for emptyaction'))
def testJailActionFilterMissing(self):
jail = JailReader('missingbitsjail', basedir=IMPERFECT_CONFIG)
self.assertTrue(jail.read())
self.assertFalse(jail.getOptions())
self.assertTrue(jail.isEnabled())
self.assertTrue(self._is_logged("Found no accessible config files for 'filter.d/catchallthebadies' under %s" % IMPERFECT_CONFIG))
self.assertTrue(self._is_logged('Unable to read the filter'))
def TODOtestJailActionBrokenDef(self):
jail = JailReader('brokenactiondef', basedir=IMPERFECT_CONFIG)
self.assertTrue(jail.read())
self.assertFalse(jail.getOptions())
self.assertTrue(jail.isEnabled())
self.printLog()
self.assertTrue(self._is_logged('Error in action definition joho[foo'))
self.assertTrue(self._is_logged('Caught exception: While reading action joho[foo we should have got 1 or 2 groups. Got: 0'))
def testStockSSHJail(self):
jail = JailReader('sshd', basedir=CONFIG_DIR) # we are running tests from root project dir atm
@ -143,7 +192,9 @@ class JailReaderTest(unittest.TestCase):
self.assertTrue(jail.getOptions())
self.assertFalse(jail.isEnabled())
self.assertEqual(jail.getName(), 'sshd')
jail.setName('ssh-funky-blocker')
self.assertEqual(jail.getName(), 'ssh-funky-blocker')
def testSplitOption(self):
# Simple example
option = "mail-whois[name=SSH]"
@ -151,6 +202,19 @@ class JailReaderTest(unittest.TestCase):
result = JailReader.extractOptions(option)
self.assertEqual(expected, result)
self.assertEqual(('mail.who_is', {}), JailReader.extractOptions("mail.who_is"))
self.assertEqual(('mail.who_is', {'a':'cat', 'b':'dog'}), JailReader.extractOptions("mail.who_is[a=cat,b=dog]"))
self.assertEqual(('mail--ho_is', {}), JailReader.extractOptions("mail--ho_is"))
self.assertEqual(('mail--ho_is', {}), JailReader.extractOptions("mail--ho_is['s']"))
#self.printLog()
#self.assertTrue(self._is_logged("Invalid argument ['s'] in ''s''"))
self.assertEqual(('mail', {'a': ','}), JailReader.extractOptions("mail[a=',']"))
#self.assertRaises(ValueError, JailReader.extractOptions ,'mail-how[')
# Empty option
option = "abc[]"
expected = ('abc', {})
@ -175,6 +239,27 @@ class JailReaderTest(unittest.TestCase):
result = JailReader.extractOptions(option)
self.assertEqual(expected, result)
def testGlob(self):
d = tempfile.mkdtemp(prefix="f2b-temp")
# Generate few files
# regular file
f1 = os.path.join(d, 'f1')
open(f1, 'w').close()
# dangling link
f2 = os.path.join(d, 'f2')
os.symlink('nonexisting',f2)
# must be only f1
self.assertEqual(JailReader._glob(os.path.join(d, '*')), [f1])
# since f2 is dangling -- empty list
self.assertEqual(JailReader._glob(f2), [])
self.assertTrue(self._is_logged('File %s is a dangling link, thus cannot be monitored' % f2))
self.assertEqual(JailReader._glob(os.path.join(d, 'nonexisting')), [])
os.remove(f1)
os.remove(f2)
os.rmdir(d)
class FilterReaderTest(unittest.TestCase):
def testConvert(self):
@ -223,13 +308,74 @@ class FilterReaderTest(unittest.TestCase):
output[-1][-1] = "5"
self.assertEqual(sorted(filterReader.convert()), sorted(output))
class JailsReaderTest(unittest.TestCase):
class JailsReaderTest(LogCaptureTestCase):
def testProvidingBadBasedir(self):
if not os.path.exists('/XXX'):
reader = JailsReader(basedir='/XXX')
self.assertRaises(ValueError, reader.read)
def testReadTestJailConf(self):
jails = JailsReader(basedir=IMPERFECT_CONFIG)
self.assertTrue(jails.read())
self.assertFalse(jails.getOptions())
self.assertRaises(ValueError, jails.convert)
comm_commands = jails.convert(allow_no_files=True)
self.maxDiff = None
self.assertEqual(sorted(comm_commands),
sorted([['add', 'emptyaction', 'auto'],
['set', 'emptyaction', 'usedns', 'warn'],
['set', 'emptyaction', 'maxretry', 3],
['set', 'emptyaction', 'findtime', 600],
['set', 'emptyaction', 'logencoding', 'auto'],
['set', 'emptyaction', 'bantime', 600],
['add', 'special', 'auto'],
['set', 'special', 'usedns', 'warn'],
['set', 'special', 'maxretry', 3],
['set', 'special', 'addfailregex', '<IP>'],
['set', 'special', 'findtime', 600],
['set', 'special', 'logencoding', 'auto'],
['set', 'special', 'bantime', 600],
['add', 'missinglogfiles', 'auto'],
['set', 'missinglogfiles', 'usedns', 'warn'],
['set', 'missinglogfiles', 'maxretry', 3],
['set', 'missinglogfiles', 'findtime', 600],
['set', 'missinglogfiles', 'logencoding', 'auto'],
['set', 'missinglogfiles', 'bantime', 600],
['set', 'missinglogfiles', 'addfailregex', '<IP>'],
['add', 'brokenaction', 'auto'],
['set', 'brokenaction', 'usedns', 'warn'],
['set', 'brokenaction', 'maxretry', 3],
['set', 'brokenaction', 'findtime', 600],
['set', 'brokenaction', 'logencoding', 'auto'],
['set', 'brokenaction', 'bantime', 600],
['set', 'brokenaction', 'addfailregex', '<IP>'],
['set', 'brokenaction', 'addaction', 'brokenaction'],
['set',
'brokenaction',
'actionban',
'brokenaction',
'hit with big stick <ip>'],
['set', 'brokenaction', 'actionstop', 'brokenaction', ''],
['set', 'brokenaction', 'actionstart', 'brokenaction', ''],
['set', 'brokenaction', 'actionunban', 'brokenaction', ''],
['set', 'brokenaction', 'actioncheck', 'brokenaction', ''],
['add', 'parse_to_end_of_jail.conf', 'auto'],
['set', 'parse_to_end_of_jail.conf', 'usedns', 'warn'],
['set', 'parse_to_end_of_jail.conf', 'maxretry', 3],
['set', 'parse_to_end_of_jail.conf', 'findtime', 600],
['set', 'parse_to_end_of_jail.conf', 'logencoding', 'auto'],
['set', 'parse_to_end_of_jail.conf', 'bantime', 600],
['set', 'parse_to_end_of_jail.conf', 'addfailregex', '<IP>'],
['start', 'emptyaction'],
['start', 'special'],
['start', 'missinglogfiles'],
['start', 'brokenaction'],
['start', 'parse_to_end_of_jail.conf'],]))
self.assertTrue(self._is_logged("Errors in jail 'missingbitsjail'. Skipping..."))
self.assertTrue(self._is_logged("No file(s) found for glob /weapons/of/mass/destruction"))
def testReadStockJailConf(self):
jails = JailsReader(basedir=CONFIG_DIR) # we are running tests from root project dir atm
self.assertTrue(jails.read()) # opens fine
@ -239,6 +385,15 @@ class JailsReaderTest(unittest.TestCase):
# commands to communicate to the server
self.assertEqual(comm_commands, [])
# TODO: make sure this is handled well
## We should not "read" some bogus jail
#old_comm_commands = comm_commands[:] # make a copy
#self.assertRaises(ValueError, jails.getOptions, "BOGUS")
#self.printLog()
#self.assertTrue(self._is_logged("No section: 'BOGUS'"))
## and there should be no side-effects
#self.assertEqual(jails.convert(), old_comm_commands)
allFilters = set()
# All jails must have filter and action set
@ -360,7 +515,10 @@ class JailsReaderTest(unittest.TestCase):
# and there is logging information left to be passed into the
# server
self.assertEqual(sorted(commands),
[['set', 'loglevel', 3],
[['set', 'dbfile',
'/var/lib/fail2ban/fail2ban.sqlite3'],
['set', 'dbpurgeage', 86400],
['set', 'loglevel', 3],
['set', 'logtarget', '/var/log/fail2ban.log']])
# and if we force change configurator's fail2ban's baseDir

4
fail2ban/tests/config/action.d/brokenaction.conf Normal file
View File

@ -0,0 +1,4 @@
[Definition]
actionban = hit with big stick <ip>

5
fail2ban/tests/config/fail2ban.conf Normal file
View File

@ -0,0 +1,5 @@
[Definition]
# 3 = INFO
loglevel = 3

4
fail2ban/tests/config/filter.d/simple.conf Normal file
View File

@ -0,0 +1,4 @@
[Definition]
failregex = <IP>

33
fail2ban/tests/config/jail.conf Normal file
View File

@ -0,0 +1,33 @@
[DEFAULT]
filter = simple
logpath = /non/exist
[emptyaction]
enabled = true
filter =
action =
[special]
failregex = <IP>
ignoreregex =
ignoreip =
[missinglogfiles]
logpath = /weapons/of/mass/destruction
[brokenactiondef]
enabled = true
action = joho[foo
[brokenaction]
enabled = true
action = brokenaction
[missingbitsjail]
filter = catchallthebadies
action = thefunkychickendance
[parse_to_end_of_jail.conf]
enabled = true
action =

219
fail2ban/tests/databasetestcase.py Normal file
View File

@ -0,0 +1,219 @@
# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
# vi: set ft=python sts=4 ts=4 sw=4 noet :
# This file is part of Fail2Ban.
#
# Fail2Ban is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# Fail2Ban is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Fail2Ban; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
# Fail2Ban developers
__copyright__ = "Copyright (c) 2013 Steven Hiscocks"
__license__ = "GPL"
import os
import unittest
import tempfile
import sqlite3
import shutil
from fail2ban.server.database import Fail2BanDb
from fail2ban.server.filter import FileContainer
from fail2ban.server.mytime import MyTime
from fail2ban.server.ticket import FailTicket
from fail2ban.tests.dummyjail import DummyJail
class DatabaseTest(unittest.TestCase):
def setUp(self):
"""Call before every test case."""
_, self.dbFilename = tempfile.mkstemp(".db", "fail2ban_")
self.db = Fail2BanDb(self.dbFilename)
def tearDown(self):
"""Call after every test case."""
# Cleanup
os.remove(self.dbFilename)
def testGetFilename(self):
self.assertEqual(self.dbFilename, self.db.getFilename())
def testCreateInvalidPath(self):
self.assertRaises(
sqlite3.OperationalError,
Fail2BanDb,
"/this/path/should/not/exist")
def testCreateAndReconnect(self):
self.testAddJail()
# Reconnect...
self.db = Fail2BanDb(self.dbFilename)
# and check jail of same name still present
self.assertTrue(
self.jail.getName() in self.db.getJailNames(),
"Jail not retained in Db after disconnect reconnect.")
def testUpdateDb(self):
shutil.copyfile('fail2ban/tests/files/database_v1.db', self.dbFilename)
self.db = Fail2BanDb(self.dbFilename)
self.assertEqual(self.db.getJailNames(), set(['DummyJail #29162448 with 0 tickets']))
self.assertEqual(self.db.getLogPaths(), set(['/tmp/Fail2BanDb_pUlZJh.log']))
ticket = FailTicket("127.0.0.1", 1388009242.26, [u"abc\n"])
self.assertEqual(self.db.getBans()[0], ticket)
self.assertEqual(self.db.updateDb(Fail2BanDb.__version__), Fail2BanDb.__version__)
self.assertRaises(NotImplementedError, self.db.updateDb, Fail2BanDb.__version__ + 1)
os.remove(self.db.dbBackupFilename)
def testAddJail(self):
self.jail = DummyJail()
self.db.addJail(self.jail)
self.assertTrue(
self.jail.getName() in self.db.getJailNames(),
"Jail not added to database")
def testAddLog(self):
self.testAddJail() # Jail required
_, filename = tempfile.mkstemp(".log", "Fail2BanDb_")
self.fileContainer = FileContainer(filename, "utf-8")
self.db.addLog(self.jail, self.fileContainer)
self.assertTrue(filename in self.db.getLogPaths(self.jail))
os.remove(filename)
def testUpdateLog(self):
self.testAddLog() # Add log file
# Write some text
filename = self.fileContainer.getFileName()
file_ = open(filename, "w")
file_.write("Some text to write which will change md5sum\n")
file_.close()
self.fileContainer.open()
self.fileContainer.readline()
self.fileContainer.close()
# Capture position which should be after line just written
lastPos = self.fileContainer.getPos()
self.assertTrue(lastPos > 0)
self.db.updateLog(self.jail, self.fileContainer)
# New FileContainer for file
self.fileContainer = FileContainer(filename, "utf-8")
self.assertEqual(self.fileContainer.getPos(), 0)
# Database should return previous position in file
self.assertEqual(
self.db.addLog(self.jail, self.fileContainer), lastPos)
# Change md5sum
file_ = open(filename, "w") # Truncate
file_.write("Some different text to change md5sum\n")
file_.close()
self.fileContainer = FileContainer(filename, "utf-8")
self.assertEqual(self.fileContainer.getPos(), 0)
# Database should be aware of md5sum change, such doesn't return
# last position in file
self.assertEqual(
self.db.addLog(self.jail, self.fileContainer), None)
os.remove(filename)
def testAddBan(self):
self.testAddJail()
ticket = FailTicket("127.0.0.1", 0, ["abc\n"])
self.db.addBan(self.jail, ticket)
self.assertEquals(len(self.db.getBans(jail=self.jail)), 1)
self.assertTrue(
isinstance(self.db.getBans(jail=self.jail)[0], FailTicket))
def testGetBansWithTime(self):
self.testAddJail()
ticket = FailTicket("127.0.0.1", MyTime.time() - 40, ["abc\n"])
self.db.addBan(self.jail, ticket)
self.assertEquals(len(self.db.getBans(jail=self.jail,bantime=50)), 1)
self.assertEquals(len(self.db.getBans(jail=self.jail,bantime=20)), 0)
def testGetBansMerged(self):
self.testAddJail()
jail2 = DummyJail()
self.db.addJail(jail2)
ticket = FailTicket("127.0.0.1", 10, ["abc\n"])
ticket.setAttempt(10)
self.db.addBan(self.jail, ticket)
ticket = FailTicket("127.0.0.1", 20, ["123\n"])
ticket.setAttempt(20)
self.db.addBan(self.jail, ticket)
ticket = FailTicket("127.0.0.2", 30, ["ABC\n"])
ticket.setAttempt(30)
self.db.addBan(self.jail, ticket)
ticket = FailTicket("127.0.0.1", 40, ["ABC\n"])
ticket.setAttempt(40)
self.db.addBan(jail2, ticket)
# All for IP 127.0.0.1
ticket = self.db.getBansMerged("127.0.0.1")
self.assertEqual(ticket.getIP(), "127.0.0.1")
self.assertEqual(ticket.getAttempt(), 70)
self.assertEqual(ticket.getMatches(), ["abc\n", "123\n", "ABC\n"])
# All for IP 127.0.0.1 for single jail
ticket = self.db.getBansMerged("127.0.0.1", jail=self.jail)
self.assertEqual(ticket.getIP(), "127.0.0.1")
self.assertEqual(ticket.getAttempt(), 30)
self.assertEqual(ticket.getMatches(), ["abc\n", "123\n"])
# Should cache result if no extra bans added
self.assertEqual(
id(ticket),
id(self.db.getBansMerged("127.0.0.1", jail=self.jail)))
newTicket = FailTicket("127.0.0.1", 40, ["ABC\n"])
ticket.setAttempt(40)
self.db.addBan(self.jail, newTicket)
# Added ticket, so cache should have been cleared
self.assertNotEqual(
id(ticket),
id(self.db.getBansMerged("127.0.0.1", jail=self.jail)))
def testPurge(self):
self.testAddJail() # Add jail
self.db.purge() # Jail enabled by default so shouldn't be purged
self.assertEqual(len(self.db.getJailNames()), 1)
self.db.delJail(self.jail)
self.db.purge() # Should remove jail
self.assertEqual(len(self.db.getJailNames()), 0)
self.testAddBan()
self.db.delJail(self.jail)
self.db.purge() # Purge should remove all bans
self.assertEqual(len(self.db.getJailNames()), 0)
self.assertEqual(len(self.db.getBans(jail=self.jail)), 0)
# Should leave jail
self.testAddJail()
self.db.addBan(
self.jail, FailTicket("127.0.0.1", MyTime.time(), ["abc\n"]))
self.db.delJail(self.jail)
self.db.purge() # Should leave jail as ban present
self.assertEqual(len(self.db.getJailNames()), 1)
self.assertEqual(len(self.db.getBans(jail=self.jail)), 1)

10
fail2ban/tests/dummyjail.py
View File

@ -23,6 +23,7 @@ __copyright__ = "Copyright (c) 2012 Yaroslav Halchenko"
__license__ = "GPL"
from threading import Lock
class DummyJail(object):
"""A simple 'jail' to suck in all the tickets generated by Filter's
"""
@ -54,6 +55,15 @@ class DummyJail(object):
finally:
self.lock.release()
def setIdle(self, value):
pass
def getIdle(self):
pass
def getName(self):
return "DummyJail #%s with %d tickets" % (id(self), len(self))
def getDatabase(self):
return None

9
fail2ban/tests/failmanagertestcase.py
View File

@ -93,16 +93,21 @@ class AddFailure(unittest.TestCase):
# finish with rudimentary tests of the ticket
# verify consistent str
ticket_str = str(ticket)
ticket_repr = repr(ticket)
self.assertEqual(
ticket_str,
'FailTicket: ip=193.168.0.128 time=1167605999.0 #attempts=5')
'FailTicket: ip=193.168.0.128 time=1167605999.0 #attempts=5 matches=[]')
self.assertEqual(
ticket_repr,
'FailTicket: ip=193.168.0.128 time=1167605999.0 #attempts=5 matches=[]')
self.assertFalse(ticket == False)
# and some get/set-ers otherwise not tested
ticket.setTime(1000002000.0)
self.assertEqual(ticket.getTime(), 1000002000.0)
# and str() adjusted correspondingly
self.assertEqual(
str(ticket),
'FailTicket: ip=193.168.0.128 time=1000002000.0 #attempts=5')
'FailTicket: ip=193.168.0.128 time=1000002000.0 #attempts=5 matches=[]')
def testbanNOK(self):
self.__failManager.setMaxRetry(10)

0
fail2ban/tests/config/apache-auth/README → fail2ban/tests/files/config/apache-auth/README
View File

0
fail2ban/tests/config/apache-auth/basic/authz_owner/.htaccess → fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htaccess
View File

0
fail2ban/tests/config/apache-auth/basic/authz_owner/.htpasswd → fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htpasswd
View File

0
fail2ban/tests/config/apache-auth/basic/authz_owner/cant_get_me.html → fail2ban/tests/files/config/apache-auth/basic/authz_owner/cant_get_me.html
View File

0
fail2ban/tests/config/apache-auth/basic/file/.htaccess → fail2ban/tests/files/config/apache-auth/basic/file/.htaccess
View File

0
fail2ban/tests/config/apache-auth/basic/file/.htpasswd → fail2ban/tests/files/config/apache-auth/basic/file/.htpasswd
View File

0
fail2ban/tests/config/apache-auth/digest.py → fail2ban/tests/files/config/apache-auth/digest.py
View File

0
fail2ban/tests/config/apache-auth/digest/.htaccess → fail2ban/tests/files/config/apache-auth/digest/.htaccess
View File

0
fail2ban/tests/config/apache-auth/digest/.htpasswd → fail2ban/tests/files/config/apache-auth/digest/.htpasswd
View File

0
fail2ban/tests/config/apache-auth/digest_anon/.htaccess → fail2ban/tests/files/config/apache-auth/digest_anon/.htaccess
View File

0
fail2ban/tests/config/apache-auth/digest_anon/.htpasswd → fail2ban/tests/files/config/apache-auth/digest_anon/.htpasswd
View File

0
fail2ban/tests/config/apache-auth/digest_time/.htaccess → fail2ban/tests/files/config/apache-auth/digest_time/.htaccess
View File

0
fail2ban/tests/config/apache-auth/digest_time/.htpasswd → fail2ban/tests/files/config/apache-auth/digest_time/.htpasswd
View File

0
fail2ban/tests/config/apache-auth/digest_wrongrelm/.htaccess → fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htaccess
View File

0
fail2ban/tests/config/apache-auth/digest_wrongrelm/.htpasswd → fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htpasswd
View File

0
fail2ban/tests/config/apache-auth/noentry/.htaccess → fail2ban/tests/files/config/apache-auth/noentry/.htaccess
View File

BIN
fail2ban/tests/files/database_v1.db Normal file
View File

Binary file not shown.

5
fail2ban/tests/files/ignorecommand.py Executable file
View File

@ -0,0 +1,5 @@
#!/usr/bin/python
import sys
if sys.argv[1] == "10.0.0.1":
exit(0)
exit(1)

5
fail2ban/tests/files/logs/apache-modsecurity Normal file
View File

@ -0,0 +1,5 @@
# failJSON: { "time": "2013-12-23T13:12:31", "match": true , "host": "173.255.225.101" }
[Mon Dec 23 13:12:31 2013] [error] [client 173.255.225.101] ModSecurity: [file "/etc/httpd/modsecurity.d/activated_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "47"] [id "960015"] [rev "1"] [msg "Request Missing an Accept Header"] [severity "NOTICE"] [ver "OWASP_CRS/2.2.8"] [maturity "9"] [accuracy "9"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/MISSING_HEADER_ACCEPT"] [tag "WASCTC/WASC-21"][tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] Access denied with code 403 (phase 2). Operator EQ matched 0 at REQUEST_HEADERS. [hostname "www.mysite.net"] [uri "/"] [unique_id "Urf@f12qgHIAACrFOlgAAABA"]
# failJSON: { "time": "2013-12-28T09:18:05", "match": true , "host": "32.65.254.69" }
[Sat Dec 28 09:18:05 2013] [error] [client 32.65.254.69] ModSecurity: [file "/etc/httpd/modsecurity.d/10_asl_rules.conf"] [line "635"] [id "340069"] [rev "4"] [msg "Atomicorp.com UNSUPPORTED DELAYED Rules: Web vulnerability scanner"] [severity "CRITICAL"] Access denied with code 403 (phase 2). Pattern match "(?:nessus(?:_is_probing_you_|test)|^/w00tw00t\\\\.at\\\\.)" at REQUEST_URI. [hostname "192.81.249.191"] [uri "/w00tw00t.at.blackhats.romanian.anti-sec:)"] [unique_id "4Q6RdsBR@b4AAA65LRUAAAAA"]

14
fail2ban/tests/files/logs/apache-noscript
View File

@ -2,3 +2,17 @@
[Sun Jun 09 07:57:47 2013] [error] [client 192.0.43.10] script '/usr/lib/cgi-bin/gitweb.cgiwp-login.php' not found or unable to stat
# failJSON: { "time": "2008-07-22T06:48:30", "match": true , "host": "198.51.100.86" }
[Tue Jul 22 06:48:30 2008] [error] [client 198.51.100.86] File does not exist: /home/southern/public_html/azenv.php
# failJSON: { "time": "2008-07-22T06:48:30", "match": true , "host": "198.51.100.86" }
[Tue Jul 22 06:48:30 2008] [error] [client 198.51.100.86] script not found or unable to stat: /home/e-smith/files/ibays/Primary/cgi-bin/php
# failJSON: { "time": "2008-07-22T06:48:30", "match": true , "host": "198.51.100.86" }
[Tue Jul 22 06:48:30 2008] [error] [client 198.51.100.86] script not found or unable to stat: /home/e-smith/files/ibays/Primary/cgi-bin/php5
# failJSON: { "time": "2008-07-22T06:48:30", "match": true , "host": "198.51.100.86" }
[Tue Jul 22 06:48:30 2008] [error] [client 198.51.100.86] script not found or unable to stat: /home/e-smith/files/ibays/Primary/cgi-bin/php-cgi
# failJSON: { "time": "2008-07-22T06:48:30", "match": true , "host": "198.51.100.86" }
[Tue Jul 22 06:48:30 2008] [error] [client 198.51.100.86] script not found or unable to stat: /home/e-smith/files/ibays/Primary/cgi-bin/php.cgi
# failJSON: { "time": "2008-07-22T06:48:30", "match": true , "host": "198.51.100.86" }
[Tue Jul 22 06:48:30 2008] [error] [client 198.51.100.86] script not found or unable to stat: /home/e-smith/files/ibays/Primary/cgi-bin/php4
# apache 2.4
# failJSON: { "time": "2013-12-23T07:49:01", "match": true , "host": "204.232.202.107" }
[Mon Dec 23 07:49:01.981912 2013] [:error] [pid 3790] [client 204.232.202.107:46301] script '/var/www/timthumb.php' not found or unable to stat

7
fail2ban/tests/files/logs/exim-spam
View File

@ -14,4 +14,11 @@
2013-06-15 11:20:36 [2516] 1Unmew-0000ea-SE H=egeftech.static.otenet.gr [83.235.177.148]:32706 I=[1.2.3.4]:25 F=auguriesvbd40@google.com rejected after DATA: This message contains a virus (Sanesecurity.Junk.39934.UNOFFICIAL).
# failJSON: { "time": "2013-06-16T02:50:43", "match": true , "host": "111.67.203.114" }
2013-06-16 02:50:43 H=dbs.marsukov.com [111.67.203.114] F=<trudofspiori@mail.ru> rejected RCPT <info@nanomedtech.ua>: rejected because 111.67.203.114 is in a black list at dnsbl.sorbs.net\nCurrently Sending Spam See: http://www.sorbs.net/lookup.shtml?111.67.203.114
# https://github.com/fail2ban/fail2ban/issues/533
# failJSON: { "time": "2013-12-29T15:34:12", "match": true , "host": "188.76.45.72" }
2013-12-29 15:34:12 1VxHRO-000NiI-Ly SA: Action: silently tossed message: score=31.0 required=5.0 trigger=30.0 (scanned in 6/6 secs | Message-Id: etPan.09bd0c40.c3d5f675.fdf7@server.local). From <Flossiedpd@jazztel.es> (host=72.45.76.188.dynamic.jazztel.es [188.76.45.72]) for me@my.com
# https://github.com/fail2ban/fail2ban/issues/533
# failJSON: { "time": "2013-12-29T15:39:11", "match": true , "host": "178.123.108.196" }
2013-12-29 15:39:11 1VxHWD-000NuW-83 SA: Action: silently tossed message: score=35.8 required=5.0 trigger=30.0 (scanned in 6/6 secs | Message-Id: 1VxHWD-000NuW-83). From <> (host=NULL [178.123.108.196]) for me@my.com

4
fail2ban/tests/files/logs/nsd Normal file
View File

@ -0,0 +1,4 @@
# failJSON: { "time": "2013-12-17T14:58:14", "match": true , "host": "192.0.2.105" }
[1387288694] nsd[7745]: info: ratelimit block example.com. type any target 192.0.2.0/24 query 192.0.2.105 TYPE255
# failJSON: { "time": "2013-12-18T07:42:15", "match": true , "host": "192.0.2.115" }
[1387348935] nsd[23600]: info: axfr for zone domain.nl. from client 192.0.2.115 refused, no acl matches.

6
fail2ban/tests/files/logs/openwebmail Normal file
View File

@ -0,0 +1,6 @@
# failJSON: { "time": "2013-12-28T19:03:53", "match": true , "host": "178.123.108.196" }
Sat Dec 28 19:03:53 2013 - [72926] (178.123.108.196) gsdfg - userinfo error - auth_unix.pl, ret -4, User gsdfg doesn't exist
# failJSON: { "time": "2013-12-28T19:04:03", "match": true , "host": "178.123.108.196" }
Sat Dec 28 19:04:03 2013 - [72926] (178.123.108.196) gsdfg - login error - no such user - loginname=gsdfg
# failJSON: { "time": "2013-12-28T19:05:38", "match": true , "host": "178.123.108.196" }
Sat Dec 28 19:05:38 2013 - [73540] (178.123.108.196) myname - login error - auth_unix.pl, ret -4, Password incorrect

10
fail2ban/tests/files/logs/recidive
View File

@ -1,10 +1,12 @@
# failJSON: { "time": "2006-02-13T15:52:30", "match": true , "host": "1.2.3.4" }
2006-02-13 15:52:30,388 fail2ban.actions: WARNING [sendmail] Ban 1.2.3.4
2006-02-13 15:52:30,388 fail2ban.server.actions: WARNING [sendmail] Ban 1.2.3.4
# failJSON: { "time": "2006-02-13T15:52:30", "match": true , "host": "1.2.3.4", "desc": "Extended with [PID]" }
2006-02-13 15:52:30,388 fail2ban.server.actions[123]: WARNING [sendmail] Ban 1.2.3.4
# failJSON: { "match": false }
2006-02-13 16:07:31,183 fail2ban.actions: WARNING [sendmail] Unban 1.2.3.4
2006-02-13 16:07:31,183 fail2ban.server.actions: WARNING [sendmail] Unban 1.2.3.4
# failJSON: { "match": false }
2006-02-13 15:52:30,388 fail2ban.actions: WARNING [recidive] Ban 1.2.3.4
2006-02-13 15:52:30,388 fail2ban.server.actions: WARNING [recidive] Ban 1.2.3.4
# syslog example
# failJSON: { "time": "2004-09-16T00:44:55", "match": true , "host": "10.0.0.7" }
Sep 16 00:44:55 spaceman fail2ban.actions: WARNING [jail] Ban 10.0.0.7
Sep 16 00:44:55 spaceman fail2ban.server.actions: WARNING [jail] Ban 10.0.0.7

13
fail2ban/tests/files/logs/squid Normal file
View File

@ -0,0 +1,13 @@
# Logs thanks to Roman Gelfand
#
# failJSON: { "time": "2013-12-08T23:55:23.000", "match": true , "host": "91.188.124.227" }
1386543323.000 4 91.188.124.227 TCP_DENIED/403 4099 GET http://www.proxy-listen.de/azenv.php - HIER_NONE/- text/html
# failJSON: { "time": "2013-12-08T23:58:20", "match": true , "host": "175.44.0.184" }
1386543500.000 5 175.44.0.184 NONE/405 3364 CONNECT error:method-not-allowed - HIER_NONE/- text/html
# failJSON: { "time": "2013-12-09T00:08:04.000", "match": true , "host": "198.74.125.200" }
1386544084.000 3 198.74.125.200 TCP_DENIED/403 3722 GET http://www2t.biglobe.ne.jp/~take52/test/env.cgi - HIER_NONE/- text/html
# failJSON: { "time": "2013-12-09T00:09:06.000", "match": true , "host": "175.42.91.151" }
1386544146.000 1 175.42.91.151 TCP_DENIED/403 3745 GET http://pkfsp.ru/wp-content/uploads/proxyc/engine.php - HIER_NONE/- text/html

12
fail2ban/tests/files/logs/sshd
View File

@ -117,3 +117,15 @@ Sep 29 17:15:02 spaceman sshd[12946]: Failed password for user from 127.0.0.1 po
# failJSON: { "time": "2004-11-11T08:04:51", "match": true , "host": "127.0.0.1", "desc": "Injecting on username ssh 'from 10.10.1.1'@localhost" }
Nov 11 08:04:51 redbamboo sshd[2737]: Failed password for invalid user from 10.10.1.1 from 127.0.0.1 port 58946 ssh2
# failJSON: { "match": false }
Nov 23 21:50:19 sshd[8148]: Disconnecting: Too many authentication failures for root [preauth]
# failJSON: { "time": "2004-11-23T21:50:37", "match": true , "host": "61.0.0.1", "desc": "Multiline match for preauth failures" }
Nov 23 21:50:37 sshd[8148]: Connection closed by 61.0.0.1 [preauth]
# failJSON: { "match": false }
Nov 23 21:50:19 sshd[9148]: Disconnecting: Too many authentication failures for root [preauth]
# failJSON: { "match": false , "desc": "Pids don't match" }
Nov 23 21:50:37 sshd[7148]: Connection closed by 61.0.0.1 [preauth]

159
fail2ban/tests/filtertestcase.py
View File

@ -36,19 +36,16 @@ except ImportError:
from fail2ban.server.jail import Jail
from fail2ban.server.filterpoll import FilterPoll
from fail2ban.server.filter import FileFilter, DNSUtils
from fail2ban.server.filter import Filter, FileFilter, DNSUtils
from fail2ban.server.failmanager import FailManager
from fail2ban.server.failmanager import FailManagerEmpty
from fail2ban.server.mytime import MyTime
from fail2ban.tests.utils import setUpMyTime, tearDownMyTime
from fail2ban.tests.utils import mtimesleep, LogCaptureTestCase
TEST_FILES_DIR = os.path.join(os.path.dirname(__file__), "files")
#
# Useful helpers
#
from utils import mtimesleep
from fail2ban.tests.dummyjail import DummyJail
# yoh: per Steven Hiscocks's insight while troubleshooting
# https://github.com/fail2ban/fail2ban/issues/103#issuecomment-15542836
@ -192,44 +189,111 @@ def _copy_lines_to_journal(in_, fields={},n=None, skip=0, terminal_line=""): # p
# Actual tests
#
class IgnoreIP(unittest.TestCase):
class BasicFilter(unittest.TestCase):
def setUp(self):
self.filter = Filter('name')
def testGetSetUseDNS(self):
# default is warn
self.assertEqual(self.filter.getUseDns(), 'warn')
self.filter.setUseDns(True)
self.assertEqual(self.filter.getUseDns(), 'yes')
self.filter.setUseDns(False)
self.assertEqual(self.filter.getUseDns(), 'no')
def testGetSetDatePattern(self):
self.assertEqual(self.filter.getDatePattern(),
(None, "Default Detectors"))
self.filter.setDatePattern("^%Y-%m-%d-%H%M%S.%f %z")
self.assertEqual(self.filter.getDatePattern(),
("^%Y-%m-%d-%H%M%S.%f %z",
"Year-Month-Day-24hourMinuteSecond.Microseconds Zone offset"))
class IgnoreIP(LogCaptureTestCase):
def setUp(self):
"""Call before every test case."""
self.filter = FileFilter(None)
def tearDown(self):
"""Call after every test case."""
LogCaptureTestCase.setUp(self)
self.jail = DummyJail()
self.filter = FileFilter(self.jail)
def testIgnoreIPOK(self):
ipList = "127.0.0.1", "192.168.0.1", "255.255.255.255", "99.99.99.99"
for ip in ipList:
self.filter.addIgnoreIP(ip)
self.assertTrue(self.filter.inIgnoreIPList(ip))
# Test DNS
self.filter.addIgnoreIP("www.epfl.ch")
self.assertTrue(self.filter.inIgnoreIPList("128.178.50.12"))
def testIgnoreIPNOK(self):
ipList = "", "999.999.999.999", "abcdef", "192.168.0."
for ip in ipList:
self.filter.addIgnoreIP(ip)
self.assertFalse(self.filter.inIgnoreIPList(ip))
def testIgnoreIPCIDR(self):
self.filter.addIgnoreIP('192.168.1.0/25')
self.assertTrue(self.filter.inIgnoreIPList('192.168.1.0'))
self.assertTrue(self.filter.inIgnoreIPList('192.168.1.1'))
self.assertTrue(self.filter.inIgnoreIPList('192.168.1.127'))
self.assertFalse(self.filter.inIgnoreIPList('192.168.1.128'))
self.assertFalse(self.filter.inIgnoreIPList('192.168.1.255'))
self.assertFalse(self.filter.inIgnoreIPList('192.168.0.255'))
def testIgnoreInProcessLine(self):
setUpMyTime()
self.filter.addIgnoreIP('192.168.1.0/25')
self.filter.addFailRegex('<HOST>')
self.filter.processLineAndAdd('1387203300.222 192.168.1.32')
self.assertTrue(self._is_logged('Ignore 192.168.1.32'))
tearDownMyTime()
def testIgnoreAddBannedIP(self):
self.filter.addIgnoreIP('192.168.1.0/25')
self.filter.addBannedIP('192.168.1.32')
self.assertFalse(self._is_logged('Ignore 192.168.1.32'))
self.assertTrue(self._is_logged('Requested to manually ban an ignored IP 192.168.1.32. User knows best. Proceeding to ban it.'))
def testIgnoreCommand(self):
self.filter.setIgnoreCommand("fail2ban/tests/files/ignorecommand.py <ip>")
self.assertTrue(self.filter.inIgnoreIPList("10.0.0.1"))
self.assertFalse(self.filter.inIgnoreIPList("10.0.0.0"))
class IgnoreIPDNS(IgnoreIP):
def testIgnoreIPDNSOK(self):
self.filter.addIgnoreIP("www.epfl.ch")
self.assertTrue(self.filter.inIgnoreIPList("128.178.50.12"))
def testIgnoreIPDNSNOK(self):
# Test DNS
self.filter.addIgnoreIP("www.epfl.ch")
self.assertFalse(self.filter.inIgnoreIPList("127.177.50.10"))
self.assertFalse(self.filter.inIgnoreIPList("128.178.50.11"))
self.assertFalse(self.filter.inIgnoreIPList("128.178.50.13"))
class LogFile(LogCaptureTestCase):
class LogFile(unittest.TestCase):
MISSING = 'testcases/missingLogFile'
def setUp(self):
LogCaptureTestCase.setUp(self)
def tearDown(self):
LogCaptureTestCase.tearDown(self)
def testMissingLogFiles(self):
self.filter = FilterPoll(None)
self.assertRaises(IOError, self.filter.addLogPath, LogFile.MISSING)
class LogFileFilterPoll(unittest.TestCase):
FILENAME = os.path.join(TEST_FILES_DIR, "testcase01.log")
def setUp(self):
"""Call before every test case."""
self.filter = FilterPoll(None)
self.filter.addLogPath(LogFile.FILENAME)
self.filter = FilterPoll(DummyJail())
self.filter.addLogPath(LogFileFilterPoll.FILENAME)
def tearDown(self):
"""Call after every test case."""
@ -239,25 +303,28 @@ class LogFile(unittest.TestCase):
# self.filter.openLogFile(LogFile.FILENAME)
def testIsModified(self):
self.assertTrue(self.filter.isModified(LogFile.FILENAME))
self.assertTrue(self.filter.isModified(LogFileFilterPoll.FILENAME))
self.assertFalse(self.filter.isModified(LogFileFilterPoll.FILENAME))
class LogFileMonitor(unittest.TestCase):
class LogFileMonitor(LogCaptureTestCase):
"""Few more tests for FilterPoll API
"""
def setUp(self):
"""Call before every test case."""
setUpMyTime()
LogCaptureTestCase.setUp(self)
self.filter = self.name = 'NA'
_, self.name = tempfile.mkstemp('fail2ban', 'monitorfailures')
self.file = open(self.name, 'a')
self.filter = FilterPoll(None)
self.filter = FilterPoll(DummyJail())
self.filter.addLogPath(self.name)
self.filter.setActive(True)
self.filter.addFailRegex("(?:(?:Authentication failure|Failed [-/\w+]+) for(?: [iI](?:llegal|nvalid) user)?|[Ii](?:llegal|nvalid) user|ROOT LOGIN REFUSED) .*(?: from|FROM) <HOST>")
def tearDown(self):
tearDownMyTime()
LogCaptureTestCase.tearDown(self)
_killfile(self.file, self.name)
pass
@ -275,6 +342,21 @@ class LogFileMonitor(unittest.TestCase):
# shorter wait time for not modified status
return not self.isModified(0.4)
def testNoLogFile(self):
os.chmod(self.name, 0)
self.filter.getFailures(self.name)
self.assertTrue(self._is_logged('Unable to open %s' % self.name))
def testRemovingFailRegex(self):
self.filter.delFailRegex(0)
self.assertFalse(self._is_logged('Cannot remove regular expression. Index 0 is not valid'))
self.filter.delFailRegex(0)
self.assertTrue(self._is_logged('Cannot remove regular expression. Index 0 is not valid'))
def testRemovingIgnoreRegex(self):
self.filter.delIgnoreRegex(0)
self.assertTrue(self._is_logged('Cannot remove regular expression. Index 0 is not valid'))
def testNewChangeViaIsModified(self):
# it is a brand new one -- so first we think it is modified
self.assertTrue(self.isModified())
@ -357,7 +439,6 @@ class LogFileMonitor(unittest.TestCase):
from threading import Lock
from dummyjail import DummyJail
def get_monitor_failures_testcase(Filter_):
"""Generator of TestCase's for different filters/backends
@ -710,12 +791,13 @@ class GetFailures(unittest.TestCase):
# so that they could be reused by other tests
FAILURES_01 = ('193.168.0.128', 3, 1124017199.0,
[u'Aug 14 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 193.168.0.128\n']*3)
[u'Aug 14 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 193.168.0.128']*3)
def setUp(self):
"""Call before every test case."""
setUpMyTime()
self.filter = FileFilter(None)
self.jail = DummyJail()
self.filter = FileFilter(self.jail)
self.filter.setActive(True)
# TODO Test this
#self.filter.setTimeRegex("\S{3}\s{1,2}\d{1,2} \d{2}:\d{2}:\d{2}")
@ -725,7 +807,13 @@ class GetFailures(unittest.TestCase):
"""Call after every test case."""
tearDownMyTime()
def testTail(self):
self.filter.addLogPath(GetFailures.FILENAME_01, tail=True)
self.assertEqual(self.filter.getLogPath()[-1].getPos(), 1653)
self.filter.getLogPath()[-1].close()
self.assertEqual(self.filter.getLogPath()[-1].readline(), "")
self.filter.delLogPath(GetFailures.FILENAME_01)
self.assertEqual(self.filter.getLogPath(),[])
def testGetFailures01(self, filename=None, failures=None):
filename = filename or GetFailures.FILENAME_01
@ -747,16 +835,13 @@ class GetFailures(unittest.TestCase):
fout.close()
# now see if we should be getting the "same" failures
self.testGetFailures01(filename=fname,
failures=GetFailures.FAILURES_01[:3] +
([x.rstrip('\n') + '\r\n' for x in
GetFailures.FAILURES_01[-1]],))
self.testGetFailures01(filename=fname)
_killfile(fout, fname)
def testGetFailures02(self):
output = ('141.3.81.106', 4, 1124017139.0,
[u'Aug 14 11:%d:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:141.3.81.106 port 51332 ssh2\n'
[u'Aug 14 11:%d:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:141.3.81.106 port 51332 ssh2'
% m for m in 53, 54, 57, 58])
self.filter.addLogPath(GetFailures.FILENAME_02)
@ -789,11 +874,11 @@ class GetFailures(unittest.TestCase):
def testGetFailuresUseDNS(self):
# We should still catch failures with usedns = no ;-)
output_yes = ('93.184.216.119', 2, 1124017139.0,
[u'Aug 14 11:54:59 i60p295 sshd[12365]: Failed publickey for roehl from example.com port 51332 ssh2\n',
u'Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:93.184.216.119 port 51332 ssh2\n'])
[u'Aug 14 11:54:59 i60p295 sshd[12365]: Failed publickey for roehl from example.com port 51332 ssh2',
u'Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:93.184.216.119 port 51332 ssh2'])
output_no = ('93.184.216.119', 1, 1124017139.0,
[u'Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:93.184.216.119 port 51332 ssh2\n'])
[u'Aug 14 11:58:59 i60p295 sshd[12365]: Failed publickey for roehl from ::ffff:93.184.216.119 port 51332 ssh2'])
# Actually no exception would be raised -- it will be just set to 'no'
#self.assertRaises(ValueError,
@ -802,7 +887,8 @@ class GetFailures(unittest.TestCase):
for useDns, output in (('yes', output_yes),
('no', output_no),
('warn', output_yes)):
filter_ = FileFilter(None, useDns=useDns)
jail = DummyJail()
filter_ = FileFilter(jail, useDns=useDns)
filter_.setActive(True)
filter_.failManager.setMaxRetry(1) # we might have just few failures
@ -916,5 +1002,6 @@ class JailTests(unittest.TestCase):
def testSetBackend_gh83(self):
# smoke test
jail = Jail('test', backend='polling') # Must not fail to initiate
# Must not fail to initiate
jail = Jail('test', backend='polling')

Some files were not shown because too many files have changed in this diff Show More