mirror of https://github.com/fail2ban/fail2ban
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
425 lines
13 KiB
425 lines
13 KiB
__ _ _ ___ _ |
|
/ _|__ _(_) |_ ) |__ __ _ _ _ |
|
| _/ _` | | |/ /| '_ \/ _` | ' \ |
|
|_| \__,_|_|_/___|_.__/\__,_|_||_| |
|
|
|
================================================================================ |
|
How to develop for Fail2Ban |
|
================================================================================ |
|
|
|
Fail2Ban uses GIT (http://git-scm.com/) distributed source control. This gives |
|
each developer their own complete copy of the entire repository. Developers can |
|
add and switch branches and commit changes when ever they want and then ask a |
|
maintainer to merge their changes. |
|
|
|
Fail2Ban uses GitHub (https://github.com/fail2ban/fail2ban) to manage access to |
|
the Git repository. GitHub provides free hosting for open-source projects as |
|
well as a web-based Git repository browser and an issue tracker. |
|
|
|
If you are familiar with Python and you have a bug fix or a feature that you |
|
would like to add to Fail2Ban, the best way to do so it to use the GitHub Pull |
|
Request feature. You can find more details on the Fail2Ban wiki |
|
(http://www.fail2ban.org/wiki/index.php/Get_Involved) |
|
|
|
Pull Requests |
|
============= |
|
|
|
When submitting pull requests on GitHub we ask you to: |
|
* Clearly describe the problem you're solving; |
|
* Don't introduce regressions that will make it hard for systems administrators |
|
to update; |
|
* If adding a major feature rebase your changes on master and get to a single commit; |
|
* Include test cases (see below); |
|
* Include sample logs (if relevant); |
|
* Include a change to the relevant section of the ChangeLog; and |
|
* Include yourself in THANKS if not already there. |
|
|
|
If you are developing filters see the FILTERS file for documentation. |
|
|
|
Code Testing |
|
============ |
|
|
|
Existing tests can be run by executing `fail2ban-testcases`. This has options |
|
like --log-level that will probably be useful. `fail2ban-testcases --help` for |
|
full options. |
|
|
|
Test cases should cover all usual cases, all exception cases and all inside |
|
/ outside boundary conditions. |
|
|
|
Test cases should cover all branches. The coverage tool will help identify |
|
missing branches. Also see http://nedbatchelder.com/code/coverage/branch.html |
|
for more details. |
|
|
|
Install the package python-coverage to visualise your test coverage. Run the |
|
following (note: on Debian-based systems, the script is called |
|
`python-coverage`): |
|
|
|
coverage run fail2ban-testcases |
|
coverage html |
|
|
|
Then look at htmlcov/index.html and see how much coverage your test cases |
|
exert over the code base. Full coverage is a good thing however it may not be |
|
complete. Try to ensure tests cover as many independent paths through the |
|
code. |
|
|
|
Manual Execution. To run in a development environment do: |
|
|
|
./fail2ban-client -c config/ -s /tmp/f2b.sock -i start |
|
|
|
some quick commands: |
|
|
|
status |
|
add test pyinotify |
|
status test |
|
set test addaction iptables |
|
set test actionban iptables echo <ip> <cidr> >> /tmp/ban |
|
set test actionunban iptables echo <ip> <cidr> >> /tmp/unban |
|
get test actionban iptables |
|
get test actionunban iptables |
|
set test banip 192.168.2.2 |
|
status test |
|
|
|
|
|
|
|
Coding Standards |
|
================ |
|
|
|
Style |
|
----- |
|
|
|
Please use tabs for now. Keep to 80 columns, at least for readable text. |
|
|
|
Tests |
|
----- |
|
|
|
Add tests. They should test all the code you add in a meaning way. |
|
|
|
Coverage |
|
-------- |
|
|
|
Test coverage should always increase as you add code. |
|
|
|
You may use "# pragma: no cover" in the code for branches of code that support |
|
older versions on python. For all other uses of "pragma: no cover" or |
|
"pragma: no branch" document the reason why its not covered. "I haven't written |
|
a test case" isn't a sufficient reason. |
|
|
|
Documentation |
|
------------- |
|
|
|
Ensure this documentation is up to date after changes. Also ensure that the man |
|
pages still are accurate. Ensure that there is sufficient documentation for |
|
your new features to be used. |
|
|
|
Bugs |
|
---- |
|
|
|
Remove them and don't add any more. |
|
|
|
Git |
|
--- |
|
|
|
Use the following tags in your commit messages: |
|
|
|
'BF:' for bug fixes |
|
'DOC:' for documentation fixes |
|
'ENH:' for enhancements |
|
'TST:' for commits concerning tests only (thus not touching the main code-base) |
|
|
|
Multiple tags could be joined with +, e.g. "BF+TST:". |
|
|
|
Use the text "closes #333"/"resolves #333 "/"fixes #333" where 333 represents |
|
an issue that is closed. Other text and details in link below. |
|
See: https://help.github.com/articles/closing-issues-via-commit-messages |
|
|
|
If merge resulted in conflicts, clarify what changes were done to |
|
corresponding files in the 'Conflicts:' section of the merge commit |
|
message. See e.g. https://github.com/fail2ban/fail2ban/commit/f5a8a8ac |
|
|
|
Adding Actions |
|
-------------- |
|
|
|
If you add an action.d/*.conf file also add a example in config/jail.conf |
|
with enabled=false and maxretry=5 for ssh. |
|
|
|
|
|
Design |
|
====== |
|
|
|
Fail2Ban was initially developed with Python 2.3 (IIRC). It should |
|
still be compatible with Python 2.4 and such compatibility assurance |
|
makes code ... old-fashioned in many places (RF-Note). In 0.7 the |
|
design went through major re-factoring into client/server, |
|
a-thread-per-jail design which made it a bit difficult to follow. |
|
Below you can find a sketchy description of the main components of the |
|
system to orient yourself better. |
|
|
|
server/ |
|
------ |
|
|
|
Core classes hierarchy (feel welcome to draw a better/more complete |
|
one):: |
|
|
|
-> inheritance |
|
+ delegation |
|
* storage of multiple instances |
|
|
|
RF-Note just a note which might be useful to address while doing RF |
|
|
|
JailThread -> Filter -> FileFilter -> {FilterPoll, FilterPyinotify, ...} |
|
| * FileContainer |
|
+ FailManager |
|
+ DateDetector |
|
+ Jail (provided in __init__) which contains this Filter |
|
(used for passing tickets from FailManager to Jail's __queue) |
|
Server |
|
+ Jails |
|
* Jail |
|
+ Filter (in __filter) |
|
* tickets (in __queue) |
|
+ Actions (in __action) |
|
* Action |
|
+ BanManager |
|
|
|
|
|
failmanager.py |
|
~~~~~~~~~~~~~~ |
|
|
|
FailManager |
|
|
|
Keeps track of failures, recorded as 'tickets'. All operations are |
|
done via acquiring a lock |
|
|
|
FailManagerEmpty(Exception) |
|
|
|
raised by FailManager.toBan after reaching the list of tickets |
|
(RF-Note: asks to become a generator ;) ) |
|
|
|
|
|
filter.py |
|
~~~~~~~~~~ |
|
|
|
Filter(JailThread) |
|
|
|
Wraps (non-threaded) FailManager (and proxies to it quite a bit), |
|
and provides all primary logic for processing new lines, what IPs to |
|
ignore, etc |
|
|
|
.failManager [FailManager] |
|
.dateDetector [DateDetector] |
|
.__failRegex [list] |
|
.__ignoreRegex [list] |
|
Contains regular expressions for failures and ignores |
|
.__findTime [numeric] |
|
Used in `processLineAndAdd` to skip old lines |
|
|
|
FileFilter(Filter): |
|
|
|
Files-aware Filter |
|
|
|
.__logPath [list] |
|
keeps the tracked files (added 1-by-1 using addLogPath) |
|
stored as FileContainer's |
|
.getFailures |
|
actually just returns |
|
True |
|
if managed to open and get lines (until empty) |
|
False |
|
if failed to open or absent container matching the filename |
|
|
|
FileContainer |
|
|
|
Adapter for a file to deal with log rotation. |
|
|
|
.open,.close,.readline |
|
RF-Note: readline returns "" with handler absent... shouldn't it be None? |
|
.__pos |
|
Keeps the position pointer |
|
|
|
|
|
dnsutils.py |
|
~~~~~~~~~~~ |
|
|
|
DNSUtils |
|
|
|
Utility class for DNS and IP handling |
|
|
|
|
|
filter*.py |
|
~~~~~~~~~~ |
|
|
|
Implementations of FileFilter's for specific backends. Derived |
|
classes should provide an implementation of `run` and usually |
|
override `addLogPath`, `delLogPath` methods. In run() method they all |
|
one way or another provide |
|
|
|
try: |
|
while True: |
|
ticket = self.failManager.toBan() |
|
self.jail.putFailTicket(ticket) |
|
except FailManagerEmpty: |
|
self.failManager.cleanup(MyTime.time()) |
|
|
|
thus channelling "ban tickets" from their failManager to the |
|
corresponding jail. |
|
|
|
action.py |
|
~~~~~~~~~ |
|
|
|
Takes care about executing start/check/ban/unban/stop commands |
|
|
|
|
|
Releasing |
|
========= |
|
|
|
# Check distribution patches and see if they can be included |
|
|
|
* https://apps.fedoraproject.org/packages/fail2ban/sources |
|
* http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/net-analyzer/fail2ban/ |
|
* http://svnweb.freebsd.org/ports/head/security/py-fail2ban/ |
|
* https://build.opensuse.org/package/show?package=fail2ban&project=openSUSE%3AFactory |
|
* http://sophie.zarb.org/sources/fail2ban (Mageia) |
|
* https://trac.macports.org/browser/trunk/dports/security/fail2ban |
|
|
|
# Check distribution outstanding bugs |
|
|
|
* https://github.com/fail2ban/fail2ban/issues?sort=updated&state=open |
|
* http://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;package=fail2ban |
|
* https://bugs.launchpad.net/ubuntu/+source/fail2ban |
|
* http://bugs.sabayon.org/buglist.cgi?quicksearch=net-analyzer%2Ffail2ban |
|
* https://bugs.archlinux.org/?project=5&cat%5B%5D=33&string=fail2ban |
|
* https://bugs.gentoo.org/buglist.cgi?query_format=advanced&short_desc=fail2ban&bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&short_desc_type=allwords |
|
* https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&component=fail2ban&classification=Red%20Hat&classification=Fedora |
|
* http://www.freebsd.org/cgi/query-pr-summary.cgi?text=fail2ban |
|
* https://bugs.mageia.org/buglist.cgi?quicksearch=fail2ban |
|
* https://build.opensuse.org/package/requests/openSUSE:Factory/fail2ban |
|
|
|
# Make sure the tests pass |
|
|
|
./fail2ban-testcases-all |
|
|
|
# Ensure the version is correct |
|
|
|
in: |
|
* ./common/version.py |
|
* top of ChangeLog |
|
* README.md |
|
|
|
# Ensure the MANIFEST is complete |
|
|
|
Run: |
|
|
|
python setup.py sdist |
|
|
|
Look for errors like: |
|
'testcases/files/logs/mysqld.log' not a regular file -- skipping |
|
|
|
Which indicates that testcases/files/logs/mysqld.log has been moved or is a directory |
|
|
|
tar -C /tmp -jxf dist/fail2ban-0.8.15.tar.bz2 |
|
|
|
# clean up current direcory |
|
|
|
diff -rul --exclude \*.pyc . /tmp/fail2ban-0.8.15/ |
|
|
|
# Only differences should be files that you don't want distributed. |
|
|
|
# Ensure the tests work from the tarball |
|
|
|
cd /tmp/fail2ban-0.8.15/ && ./fail2ban-testcases-all |
|
|
|
# Add/finalize the corresponding entry in the ChangeLog |
|
|
|
To generate a list of committers use e.g. |
|
|
|
git shortlog -sn 0.8.14.. | sed -e 's,^[ 0-9\t]*,,g' | tr '\n' '\|' | sed -e 's:|:, :g' |
|
|
|
Ensure the top of the ChangeLog has the right version and current date. |
|
|
|
Ensure the top entry of the ChangeLog has the right version and current date. |
|
|
|
# Update man pages |
|
|
|
(cd man ; ./generate-man ) |
|
git commit -m 'DOC/ENH: update man pages for release' man/* |
|
|
|
# Prepare source and rpm binary distributions |
|
|
|
python setup.py sdist |
|
python setup.py bdist_rpm |
|
python setup.py upload |
|
|
|
# Provide a release sample to distributors |
|
|
|
* Arch Linux: |
|
https://www.archlinux.org/packages/community/any/fail2ban/ |
|
* Debian: Yaroslav Halchenko <debian@onerussian.com> |
|
http://packages.qa.debian.org/f/fail2ban.html |
|
* FreeBSD: Christoph Theis theis@gmx.at>, Nick Hilliard <nick@foobar.org> |
|
http://svnweb.freebsd.org/ports/head/security/py-fail2ban/Makefile?view=markup |
|
http://www.freebsd.org/cgi/query-pr-summary.cgi?text=fail2ban |
|
* Fedora: Axel Thimm <Axel.Thimm@atrpms.net> |
|
https://apps.fedoraproject.org/packages/fail2ban |
|
http://pkgs.fedoraproject.org/cgit/fail2ban.git |
|
https://admin.fedoraproject.org/pkgdb/acls/bugs/fail2ban |
|
* Gentoo: netmon@gentoo.org |
|
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/net-analyzer/fail2ban/metadata.xml?view=markup |
|
https://bugs.gentoo.org/buglist.cgi?quicksearch=fail2ban |
|
* openSUSE: Stephan Kulow <coolo@suse.com> |
|
https://build.opensuse.org/package/show/openSUSE:Factory/fail2ban |
|
* Mac Ports: @Malbrouck on github (gh-49) |
|
https://trac.macports.org/browser/trunk/dports/security/fail2ban/Portfile |
|
* Mageia: |
|
https://bugs.mageia.org/buglist.cgi?quicksearch=fail2ban |
|
An potentially to the fail2ban-users directory. |
|
|
|
# Wait for feedback from distributors |
|
|
|
# Prepare a release notice https://github.com/fail2ban/fail2ban/releases/new |
|
|
|
Upload the source/binaries from the dist directory and tag the release using the URL |
|
|
|
# Upload source/binaries to sourceforge http://sourceforge.net/projects/fail2ban/ |
|
|
|
# Run the following and update the wiki with output: |
|
python -c 'import common.protocol; common.protocol.printWiki()' |
|
|
|
page: http://www.fail2ban.org/wiki/index.php/Commands |
|
|
|
* Update: |
|
http://www.fail2ban.org/wiki/index.php?title=Template:Fail2ban_Versions&action=edit |
|
|
|
http://www.fail2ban.org/wiki/index.php?title=Template:Fail2ban_News&action=edit |
|
move old bits to: |
|
http://www.fail2ban.org/wiki/index.php?title=Template:Fail2ban_OldNews&action=edit |
|
|
|
http://www.fail2ban.org/wiki/index.php/ChangeLog |
|
http://www.fail2ban.org/wiki/index.php/Requirements (Check requirement) |
|
http://www.fail2ban.org/wiki/index.php/Features |
|
|
|
* See if any filters are upgraded: |
|
http://www.fail2ban.org/wiki/index.php/Special:AllPages |
|
|
|
# Email users and development list of release |
|
|
|
# notify distributors |
|
|
|
Post Release |
|
============ |
|
|
|
Add the following to the top of the ChangeLog |
|
|
|
ver. 0.8.15 (2014/XX/XXX) - wanna-be-released |
|
----------- |
|
|
|
- Fixes: |
|
|
|
- New Features: |
|
|
|
- Enhancements: |
|
|
|
Alter the git shortlog command in the previous section to refer to the just |
|
released version. |
|
|
|
and adjust common/version.py to carry .dev suffix to signal |
|
a version under development.
|
|
|