github
/
fail2ban
mirror of https://github.com/fail2ban/fail2ban
1
0
Fork 0
Code Issues Releases Wiki Activity

DOC: used spellchecker finally. Debuggex needs promotion more than donations

pull/361/head
Daniel Black 2013-09-19 19:18:04 +10:00
parent 3fac971a5a
commit 2562b2f1bc
1 changed files with 36 additions and 35 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

71
DEVELOP
View File

@ -26,7 +26,7 @@ Pull Requests
When submitting pull requests on GitHub we ask you to: When submitting pull requests on GitHub we ask you to:
* Clearly describe the problem you're solving; * Clearly describe the problem you're solving;
* Don't introduce regressions that will make it hard for systems adminstrators * Don't introduce regressions that will make it hard for systems administrators
to update; to update;
* If adding a major feature rebase your changes on master and get to a single commit; * If adding a major feature rebase your changes on master and get to a single commit;
* Include test cases (see below); * Include test cases (see below);
@ -44,12 +44,12 @@ Filters are tricky. They need to:
* work in multiple operating systems; * work in multiple operating systems;
* not make assumptions about the log format in excess of the software; * not make assumptions about the log format in excess of the software;
* make assumptions as to how future versions of the software will log messages; * make assumptions as to how future versions of the software will log messages;
* not be susceptable to DoS vulernabilities; and * not be susceptible to DoS vulnerabilities; and
* match intended log lines only. * match intended log lines only.
Please follow the steps from Filter Test Cases to Developing Filter Regular Please follow the steps from Filter Test Cases to Developing Filter Regular
Expressions and submit a github pull request afterward. If you get stuck, Expressions and submit a GitHub pull request afterwards. If you get stuck,
create a github issue with what you have done and we'll attempt to help. create a GitHub issue with what you have done and we'll attempt to help.
Filter test cases Filter test cases
----------------- -----------------
@ -58,13 +58,13 @@ Purpose:
Start by finding the log messages that the application generates related to Start by finding the log messages that the application generates related to
some form of authentication failure. If you are adding to an existing filter some form of authentication failure. If you are adding to an existing filter
think about wheither the log messages are of a simlar importance and purpose think about whether the log messages are of a similar importance and purpose
to the existing filter. If you where a user of fail2ban, and did a package to the existing filter. If you where a user of fail2ban, and did a package
update of fail2ban that started matching the new log messages, would anything update of fail2ban that started matching the new log messages, would anything
unexpected happen? Would the bantime/findtime for the jail be approprate for unexpected happen? Would the bantime/findtime for the jail be appropriate for
the new log messages. If it doesn't perhaps it needs to be in a separate the new log messages. If it doesn't perhaps it needs to be in a separate
filter defination, for example like exim is authentication failures and filter definition, for example like exim is authentication failures and
exim-spam contains log messages replated to spam. exim-spam contains log messages related to spam.
Even if its a new filter you may consider separating the log messages into Even if its a new filter you may consider separating the log messages into
different filters based on purpose. different filters based on purpose.
@ -75,12 +75,12 @@ Are some of the log lines a result of the same action? For example is a PAM
failure log message, followed by an application specific failure message the failure log message, followed by an application specific failure message the
result of the same user/script action. The result is if you add regular result of the same user/script action. The result is if you add regular
expressions for both you'll end up with two failures for a single action. expressions for both you'll end up with two failures for a single action.
Select the most approprate log message and document the other log message with Select the most appropriate log message and document the other log message with
a test case not to match it and a description as to why you chose one over a test case not to match it and a description as to why you chose one over
another. another.
With the log lines selected consider what occured to generate those log With the log lines selected consider what occurred to generate those log
messages and wheither they could of been generated by accidental means. Could messages and whether they could of been generated by accidental means. Could
the log message occur always as this is the first step towards the application the log message occur always as this is the first step towards the application
asking for authentication? Could the log messages occur often? If some of asking for authentication? Could the log messages occur often? If some of
these are true make a note of this in the jail.conf example that you provide. these are true make a note of this in the jail.conf example that you provide.
@ -97,7 +97,7 @@ any specific information about the log message, such as version or an
application configuration option that is needed for the message to occur, application configuration option that is needed for the message to occur,
include this in a comment (line beginning with #) above the failJSON metadata. include this in a comment (line beginning with #) above the failJSON metadata.
Log samples should include only one, definately not more than 3, examples of Log samples should include only one, definitely not more than 3, examples of
log messages of the same form. If log messages are different in different log messages of the same form. If log messages are different in different
versions of the application log messages that show this is encouraged. versions of the application log messages that show this is encouraged.
@ -122,7 +122,7 @@ Mar 24 15:25:51 buffalo1 dropbear[4092]: bad password attempt for 'root' from 19
The host will contain the IP or domain that should be blocked. The host will contain the IP or domain that should be blocked.
For long lines that you don't want matched, like log injection vulerabilities For long lines that you don't want matched, like log injection vulnerabilities
and log lines excluded (see "Cause" section above), a "match": false in the and log lines excluded (see "Cause" section above), a "match": false in the
failJSON and the reason why in the comment above. failJSON and the reason why in the comment above.
@ -157,7 +157,7 @@ is the date or month.
Filter file: Filter file:
The filter file is in config/filter.d/{filtername}.conf. The format of the The filter file is in config/filter.d/{filtername}.conf. The format of the
filter file has two sections INCLUDES and Defination as follows: filter file has two sections INCLUDES and Definition as follows:
[INCLUDES] [INCLUDES]
@ -172,7 +172,7 @@ failregex = ....
ignoreregex = .... ignoreregex = ....
This is also documented in the man pages as jail.conf (section 5). Other This is also documented in the man pages as jail.conf (section 5). Other
definations can be added to make failregex's more readable and maintainable. definitions can be added to make failregex's more readable and maintainable.
General rules: General rules:
@ -180,7 +180,7 @@ General rules:
Use "before" if you need to include a common set of rules, like syslog or if Use "before" if you need to include a common set of rules, like syslog or if
there's a common set of regexs for multiple filters. there's a common set of regexs for multiple filters.
Use "after" if you wish to allow the user to overwrite a set of customisations Use "after" if you wish to allow the user to overwrite a set of customisation's
of the current filter. This file doesn't need to exist. of the current filter. This file doesn't need to exist.
Try to avoid using ignoreregex mainly for performance reasons. The case when Try to avoid using ignoreregex mainly for performance reasons. The case when
@ -214,10 +214,10 @@ So now ^%(__prefix_line)s matches "Dec 12 11:19:11 dunnart dovecot: ". Note it
matches the trailing space. Putting a space after ^%(__prefix_line)s in the matches the trailing space. Putting a space after ^%(__prefix_line)s in the
regex will probably not match. regex will probably not match.
Substitions: Substitutions:
Substitions are what the syslog uses. The regex bits of %(_name)s substitute Substation's are what the syslog uses. The regex bits of %(_name)s substitute
the _name defination into the regex. They are useful for making the regexes the _name definition into the regex. They are useful for making the regexes
more readable and also defining regex parts that occur in multiple log lines. more readable and also defining regex parts that occur in multiple log lines.
Regular Expressions: Regular Expressions:
@ -256,13 +256,13 @@ Take note of -l heavydebug / -l debug and -v as they will be most useful.
TIP: Take a look at the source code of the application. You may see optional or TIP: Take a look at the source code of the application. You may see optional or
extra log messages, or parts there of, that need to form part of your regex. extra log messages, or parts there of, that need to form part of your regex.
It may also show how some parts are contrained and different formats It may also show how some parts are con trained and different formats
depending on configuration or less common usages. depending on configuration or less common usages.
TIP: Some applications log spaces at the end. If you're not sure add \s*$ as the TIP: Some applications log spaces at the end. If you're not sure add \s*$ as the
end part of the regex. end part of the regex.
If your regex isn't matching take a look at http://www.debuggex.com/. If your regex isn't matching take a look at http://www.debuggex.com/?flavor=python
Using the regex from the ./fail2ban-regex output (to ensure all substitutions Using the regex from the ./fail2ban-regex output (to ensure all substitutions
are done) and with <HOST> replaced with (?&.ipv4). Set the regex type to are done) and with <HOST> replaced with (?&.ipv4). Set the regex type to
@ -272,7 +272,8 @@ For the test data put your log output with the time removed.
When you've fixed the regex put it back into your filter file. When you've fixed the regex put it back into your filter file.
Please give a donation to stoarca for debuggex. Its a great tool isn't it. Please spread the good word about debuggex - Serge Toarca is kindly continuing
its free availability to Open Source developers.
Finishing up: Finishing up:
@ -284,8 +285,8 @@ So more specifically in the [filter] section in jail.conf:
* use "filter =" set to your filter name. * use "filter =" set to your filter name.
* use a action to disable ports associated with the application * use a action to disable ports associated with the application
* set "logpath" to a usual location for the log file for the application. * set "logpath" to a usual location for the log file for the application.
* If the default findtime or bantime isn't approprate to the filter set a value * If the default findtime or bantime isn't appropriate to the filter set a value
that is more approprate. that is more appropriate.
Send the fail2ban a git pull request (See "Pull Requests" above) containing Send the fail2ban a git pull request (See "Pull Requests" above) containing
your great work. your great work.
@ -293,7 +294,7 @@ your great work.
Filter Security Filter Security
--------------- ---------------
Poor filter regular expressions are suseptable to DoS attacks. Poor filter regular expressions are susceptible to DoS attacks.
When a remote user has the ability to introduce text that will match the When a remote user has the ability to introduce text that will match the
filter regex, such that the inserted text matches the <HOST> part, they have the filter regex, such that the inserted text matches the <HOST> part, they have the
@ -303,7 +304,7 @@ So the <HOST> part must be anchored on text generated by the application, and no
the user, to a sufficient extent that the user cannot insert the entire text. the user, to a sufficient extent that the user cannot insert the entire text.
Ideally filter regex should anchor to the beginning and end of the log line Ideally filter regex should anchor to the beginning and end of the log line
however as more applications log at the beginning than the end, achoring the however as more applications log at the beginning than the end, anchoring the
beginning is more important. If the log file used by the application is shared beginning is more important. If the log file used by the application is shared
with other applications, like system logs, ensure the other application that with other applications, like system logs, ensure the other application that
use that log file do not log user generated text at the beginning of the line, use that log file do not log user generated text at the beginning of the line,
@ -326,13 +327,13 @@ We make a failregex
Now think evil. The user does the command 'blah from 1.2.3.44' Now think evil. The user does the command 'blah from 1.2.3.44'
The program diliently logs: The program diligently logs:
Apr-07-13 07:08:36 Invalid command blah from 1.2.3.44 from 1.2.3.4 Apr-07-13 07:08:36 Invalid command blah from 1.2.3.44 from 1.2.3.4
And fail2ban matches 1.2.3.44 as the IP that it ban. A DoS attack was successful. And fail2ban matches 1.2.3.44 as the IP that it ban. A DoS attack was successful.
The fix here is that the command can be anything so .* is approprate. The fix here is that the command can be anything so .* is appropriate.
^Invalid command .* from <HOST> ^Invalid command .* from <HOST>
@ -351,10 +352,10 @@ banned.
2. Filter regex can match other user injected data 2. Filter regex can match other user injected data
From the apache vulnerability CVE-2013-2178 From the Apache vulnerability CVE-2013-2178
( original ref: https://vndh.net/note:fail2ban-089-denial-service ). ( original ref: https://vndh.net/note:fail2ban-089-denial-service ).
An example bad regex for apache: An example bad regex for Apache:
failregex = [[]client <HOST>[]] user .* not found failregex = [[]client <HOST>[]] user .* not found
@ -370,10 +371,10 @@ Now the log line will be:
As this log line doesn't match other expressions hence it matches the above As this log line doesn't match other expressions hence it matches the above
regex and blocks 192.168.33.1 as a denial of service from the HTTP requester. regex and blocks 192.168.33.1 as a denial of service from the HTTP requester.
3. Applicaiton generates two identical log messages with different meanings 3. Application generates two identical log messages with different meanings
If the application generates the following two messages under different If the application generates the following two messages under different
circmstances: circumstances:
client <IP>: authentication failed client <IP>: authentication failed
client <USER>: authentication failed client <USER>: authentication failed
@ -409,7 +410,7 @@ coverage run fail2ban-testcases
coverage html coverage html
Then look at htmlcov/index.html and see how much coverage your test cases Then look at htmlcov/index.html and see how much coverage your test cases
exert over the codebase. Full coverage is a good thing however it may not be exert over the code base. Full coverage is a good thing however it may not be
complete. Try to ensure tests cover as many independent paths through the complete. Try to ensure tests cover as many independent paths through the
code. code.
@ -500,7 +501,7 @@ Design
Fail2Ban was initially developed with Python 2.3 (IIRC). It should Fail2Ban was initially developed with Python 2.3 (IIRC). It should
still be compatible with Python 2.4 and such compatibility assurance still be compatible with Python 2.4 and such compatibility assurance
makes code ... old-fashioned in many places (RF-Note). In 0.7 the makes code ... old-fashioned in many places (RF-Note). In 0.7 the
design went through major refactoring into client/server, design went through major re-factoring into client/server,
a-thread-per-jail design which made it a bit difficult to follow. a-thread-per-jail design which made it a bit difficult to follow.
Below you can find a sketchy description of the main components of the Below you can find a sketchy description of the main components of the
system to orient yourself better. system to orient yourself better.
@ -611,7 +612,7 @@ one way or another provide
except FailManagerEmpty: except FailManagerEmpty:
self.failManager.cleanup(MyTime.time()) self.failManager.cleanup(MyTime.time())
thus channeling "ban tickets" from their failManager to the thus channelling "ban tickets" from their failManager to the
corresponding jail. corresponding jail.
action.py action.py