github
/
fail2ban
mirror of https://github.com/fail2ban/fail2ban
1
0
Fork 0
Code Issues Releases Wiki Activity

DOC: used spellchecker finally. Debuggex needs promotion more than donations

pull/361/head
Daniel Black 2013-09-19 19:18:04 +10:00
parent 3fac971a5a
commit 2562b2f1bc
1 changed files with 36 additions and 35 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

71
DEVELOP
View File

@ -26,7 +26,7 @@ Pull Requests
When submitting pull requests on GitHub we ask you to:
* Clearly describe the problem you're solving;
* Don't introduce regressions that will make it hard for systems adminstrators
* Don't introduce regressions that will make it hard for systems administrators
to update;
* If adding a major feature rebase your changes on master and get to a single commit;
* Include test cases (see below);
@ -44,12 +44,12 @@ Filters are tricky. They need to:
* work in multiple operating systems;
* not make assumptions about the log format in excess of the software;
* make assumptions as to how future versions of the software will log messages;
* not be susceptable to DoS vulernabilities; and
* not be susceptible to DoS vulnerabilities; and
* match intended log lines only.
Please follow the steps from Filter Test Cases to Developing Filter Regular
Expressions and submit a github pull request afterward. If you get stuck,
create a github issue with what you have done and we'll attempt to help.
Expressions and submit a GitHub pull request afterwards. If you get stuck,
create a GitHub issue with what you have done and we'll attempt to help.
Filter test cases
-----------------
@ -58,13 +58,13 @@ Purpose:
Start by finding the log messages that the application generates related to
some form of authentication failure. If you are adding to an existing filter
think about wheither the log messages are of a simlar importance and purpose
think about whether the log messages are of a similar importance and purpose
to the existing filter. If you where a user of fail2ban, and did a package
update of fail2ban that started matching the new log messages, would anything
unexpected happen? Would the bantime/findtime for the jail be approprate for
unexpected happen? Would the bantime/findtime for the jail be appropriate for
the new log messages. If it doesn't perhaps it needs to be in a separate
filter defination, for example like exim is authentication failures and
exim-spam contains log messages replated to spam.
filter definition, for example like exim is authentication failures and
exim-spam contains log messages related to spam.
Even if its a new filter you may consider separating the log messages into
different filters based on purpose.
@ -75,12 +75,12 @@ Are some of the log lines a result of the same action? For example is a PAM
failure log message, followed by an application specific failure message the
result of the same user/script action. The result is if you add regular
expressions for both you'll end up with two failures for a single action.
Select the most approprate log message and document the other log message with
Select the most appropriate log message and document the other log message with
a test case not to match it and a description as to why you chose one over
another.
With the log lines selected consider what occured to generate those log
messages and wheither they could of been generated by accidental means. Could
With the log lines selected consider what occurred to generate those log
messages and whether they could of been generated by accidental means. Could
the log message occur always as this is the first step towards the application
asking for authentication? Could the log messages occur often? If some of
these are true make a note of this in the jail.conf example that you provide.
@ -97,7 +97,7 @@ any specific information about the log message, such as version or an
application configuration option that is needed for the message to occur,
include this in a comment (line beginning with #) above the failJSON metadata.
Log samples should include only one, definately not more than 3, examples of
Log samples should include only one, definitely not more than 3, examples of
log messages of the same form. If log messages are different in different
versions of the application log messages that show this is encouraged.
@ -122,7 +122,7 @@ Mar 24 15:25:51 buffalo1 dropbear[4092]: bad password attempt for 'root' from 19
The host will contain the IP or domain that should be blocked.
For long lines that you don't want matched, like log injection vulerabilities
For long lines that you don't want matched, like log injection vulnerabilities
and log lines excluded (see "Cause" section above), a "match": false in the
failJSON and the reason why in the comment above.
@ -157,7 +157,7 @@ is the date or month.
Filter file:
The filter file is in config/filter.d/{filtername}.conf. The format of the
filter file has two sections INCLUDES and Defination as follows:
filter file has two sections INCLUDES and Definition as follows:
[INCLUDES]
@ -172,7 +172,7 @@ failregex = ....
ignoreregex = ....
This is also documented in the man pages as jail.conf (section 5). Other
definations can be added to make failregex's more readable and maintainable.
definitions can be added to make failregex's more readable and maintainable.
General rules:
@ -180,7 +180,7 @@ General rules:
Use "before" if you need to include a common set of rules, like syslog or if
there's a common set of regexs for multiple filters.
Use "after" if you wish to allow the user to overwrite a set of customisations
Use "after" if you wish to allow the user to overwrite a set of customisation's
of the current filter. This file doesn't need to exist.
Try to avoid using ignoreregex mainly for performance reasons. The case when
@ -214,10 +214,10 @@ So now ^%(__prefix_line)s matches "Dec 12 11:19:11 dunnart dovecot: ". Note it
matches the trailing space. Putting a space after ^%(__prefix_line)s in the
regex will probably not match.
Substitions:
Substitutions:
Substitions are what the syslog uses. The regex bits of %(_name)s substitute
the _name defination into the regex. They are useful for making the regexes
Substation's are what the syslog uses. The regex bits of %(_name)s substitute
the _name definition into the regex. They are useful for making the regexes
more readable and also defining regex parts that occur in multiple log lines.
Regular Expressions:
@ -256,13 +256,13 @@ Take note of -l heavydebug / -l debug and -v as they will be most useful.
TIP: Take a look at the source code of the application. You may see optional or
extra log messages, or parts there of, that need to form part of your regex.
It may also show how some parts are contrained and different formats
It may also show how some parts are con trained and different formats
depending on configuration or less common usages.
TIP: Some applications log spaces at the end. If you're not sure add \s*$ as the
end part of the regex.
If your regex isn't matching take a look at http://www.debuggex.com/.
If your regex isn't matching take a look at http://www.debuggex.com/?flavor=python
Using the regex from the ./fail2ban-regex output (to ensure all substitutions
are done) and with <HOST> replaced with (?&.ipv4). Set the regex type to
@ -272,7 +272,8 @@ For the test data put your log output with the time removed.
When you've fixed the regex put it back into your filter file.
Please give a donation to stoarca for debuggex. Its a great tool isn't it.
Please spread the good word about debuggex - Serge Toarca is kindly continuing
its free availability to Open Source developers.
Finishing up:
@ -284,8 +285,8 @@ So more specifically in the [filter] section in jail.conf:
* use "filter =" set to your filter name.
* use a action to disable ports associated with the application
* set "logpath" to a usual location for the log file for the application.
* If the default findtime or bantime isn't approprate to the filter set a value
that is more approprate.
* If the default findtime or bantime isn't appropriate to the filter set a value
that is more appropriate.
Send the fail2ban a git pull request (See "Pull Requests" above) containing
your great work.
@ -293,7 +294,7 @@ your great work.
Filter Security
---------------
Poor filter regular expressions are suseptable to DoS attacks.
Poor filter regular expressions are susceptible to DoS attacks.
When a remote user has the ability to introduce text that will match the
filter regex, such that the inserted text matches the <HOST> part, they have the
@ -303,7 +304,7 @@ So the <HOST> part must be anchored on text generated by the application, and no
the user, to a sufficient extent that the user cannot insert the entire text.
Ideally filter regex should anchor to the beginning and end of the log line
however as more applications log at the beginning than the end, achoring the
however as more applications log at the beginning than the end, anchoring the
beginning is more important. If the log file used by the application is shared
with other applications, like system logs, ensure the other application that
use that log file do not log user generated text at the beginning of the line,
@ -326,13 +327,13 @@ We make a failregex
Now think evil. The user does the command 'blah from 1.2.3.44'
The program diliently logs:
The program diligently logs:
Apr-07-13 07:08:36 Invalid command blah from 1.2.3.44 from 1.2.3.4
And fail2ban matches 1.2.3.44 as the IP that it ban. A DoS attack was successful.
The fix here is that the command can be anything so .* is approprate.
The fix here is that the command can be anything so .* is appropriate.
^Invalid command .* from <HOST>
@ -351,10 +352,10 @@ banned.
2. Filter regex can match other user injected data
From the apache vulnerability CVE-2013-2178
From the Apache vulnerability CVE-2013-2178
( original ref: https://vndh.net/note:fail2ban-089-denial-service ).
An example bad regex for apache:
An example bad regex for Apache:
failregex = [[]client <HOST>[]] user .* not found
@ -370,10 +371,10 @@ Now the log line will be:
As this log line doesn't match other expressions hence it matches the above
regex and blocks 192.168.33.1 as a denial of service from the HTTP requester.
3. Applicaiton generates two identical log messages with different meanings
3. Application generates two identical log messages with different meanings
If the application generates the following two messages under different
circmstances:
circumstances:
client <IP>: authentication failed
client <USER>: authentication failed
@ -409,7 +410,7 @@ coverage run fail2ban-testcases
coverage html
Then look at htmlcov/index.html and see how much coverage your test cases
exert over the codebase. Full coverage is a good thing however it may not be
exert over the code base. Full coverage is a good thing however it may not be
complete. Try to ensure tests cover as many independent paths through the
code.
@ -500,7 +501,7 @@ Design
Fail2Ban was initially developed with Python 2.3 (IIRC). It should
still be compatible with Python 2.4 and such compatibility assurance
makes code ... old-fashioned in many places (RF-Note). In 0.7 the
design went through major refactoring into client/server,
design went through major re-factoring into client/server,
a-thread-per-jail design which made it a bit difficult to follow.
Below you can find a sketchy description of the main components of the
system to orient yourself better.
@ -611,7 +612,7 @@ one way or another provide
except FailManagerEmpty:
self.failManager.cleanup(MyTime.time())
thus channeling "ban tickets" from their failManager to the
thus channelling "ban tickets" from their failManager to the
corresponding jail.
action.py