PyFilter for security peace of mind developing

group

User experience focused

PyFilter aims to be as customizable as possible, allowing the use of Sqlite or Redis to be used. PyFilter does a lot of the work by being able to sync bans across an unlimited amount of servers.

security

Security

PyFilter helps secure your servers by filtering out all of the requests that are not legitimate to your server, and blocks them if too many are sent. It works by reading log files and checking if a failed request has came from the same IP address within a user configurable amount of time and adding rules to the firewall if too many attempts have been captured.

settings

Extremely easy to work with

PyFilter is extremely easy to work with, taking around 5 minutes from install to run time. PyFilter uses json for its config to ensure readability is as high as it could be.

Installation

Downloading PyFilter through git.

git clone https://github.com/Jason2605/pyFilter.git

Optional: install.sh will setup a service for pyFilter, and you can start/stop it by using sudo systemctl start/stop pyFilter and get the status of the pyFilter service using sudo systemctl status pyFilter. To run this make sure you give permission to the install.sh file sudo chmod +x install.sh.

Installing Redis - Optional

The redis python module is required, if you use redis. To download use the following command.

pip3 install redis

Installing the Redis server.

sudo apt-get update
sudo apt-get install build-essential
sudo apt-get install tcl8.5
wget http://download.redis.io/releases/redis-stable.tar.gz
tar xzf redis-stable.tar.gz
cd redis-stable
make
make test
sudo make install
cd utils
sudo ./install_server.sh

Starting and stopping Redis.

                
                sudo service redis_6379 start
                sudo service redis_6379 stop
                
            

Configuration

Copy the default config file and name it config.json, below is the default, or it can be found here.

{
    "logging": {
        "directory": "Logs",
        "active": true
    },
    "sqlite": {
        "database": "PyFilter.db"
    },
    "settings": {
        "reload_iptables": true,
        "rules": {
            "ssh": {
                "time_format": "%b %d %H:%M:%S",
                "log_file": "/var/log/auth.log",
                "regex_patterns": [
                    "([a-zA-Z]{3}\\s+\\d{1,2} \\d{1,2}:\\d{1,2}:\\d{1,2}).* Invalid user .* from (.*) port (.*)",
                    "([a-zA-Z]{3}\\s+\\d{1,2} \\d{1,2}:\\d{1,2}:\\d{1,2}).* Failed password for .* from (.*) port (.*)",
                    "([a-zA-Z]{3}\\s+\\d{1,2} \\d{1,2}:\\d{1,2}:\\d{1,2}).* Did not receive identification string from (.*) port (.*)",
                    "([a-zA-Z]{3}\\s+\\d{1,2} \\d{1,2}:\\d{1,2}:\\d{1,2}).* Received disconnect from (.*) port (.*):\\d{0,4}: .*",
                    "([a-zA-Z]{3}\\s+\\d{1,2} \\d{1,2}:\\d{1,2}:\\d{1,2}).* Unable to negotiate with (.*) port .*",
                    [
                        "([a-zA-Z]{3}\\s+\\d{1,2} \\d{1,2}:\\d{1,2}:\\d{1,2}).* error: maximum authentication attempts exceeded for .* from (.*) port .*",
                        true
                    ]
                ]
            },
            "apache": {
                "urls": [
                    "login",
                    "admin"
                ],
                "http_status_blocks": [
                    200
                ],
                "time_format": "%d/%b/%Y:%H:%M:%S",
                "log_file": "",
                "regex_patterns": [
                    [
                        "(\\d{{1,3}}\\.\\d{{1,3}}\\.\\d{{1,3}}\\.\\d{{1,3}}) .* \\[(.*)\\] \"POST /({}) HTTP/1.1\" (\\d{{0,3}})",
                        "urls"
                    ]
                ]
            },
            "nginx": {
                "urls": [
                    "",
                    "admin"
                ],
                "http_status_blocks": [
                    405
                ],
                "time_format": "%d/%b/%Y:%H:%M:%S",
                "log_file": "",
                "regex_patterns": [
                    [
                        "(\\d{{1,3}}\\.\\d{{1,3}}\\.\\d{{1,3}}\\.\\d{{1,3}}) .* \\[(.*)\\] \"POST /({}) HTTP/1.1\" (\\d{{0,3}})",
                        "urls"
                    ]
                ]
            },
            "mysql": {
                "time_format": "%Y-%m-%d  %H:%M:%S",
                "log_file": "",
                "regex_patterns": [
                    "(\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}) .* Access denied for user '.*'@'(.*)' .*"
                ]
            }
        },
        "deny_type": "DROP",
        "ignored_ips": [
            "127.0.0.1",
            "192.168.0.10",
            "192.168.0.18",
            "77.98.39.61",
            "46.101.47.22"
        ],
        "run_once": false,
        "check_time": 600,
        "request_time": 10,
        "database": "redis",
        "failed_attempts": 5
    },
    "redis": {
        "password": "knight11",
        "database": 0,
        "sync_bans": {
            "check_time": 600,
            "name": "Pi-Home",
            "active": true
        },
        "host": "46.101.47.22"
    }
}
Setting up more rules

To setup more rules just add another section to the rules list, like this snippet below for example to add mariadb monitoring.

                
                      "mariadb": {
                        "log_file": "/var/log/x",
                        "regex_patterns": [
                          "(\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}) .* Access denied for user '.*'@'(.*)' .*"
                        ],
                        "time_format": "%Y-%m-%d  %H:%M:%S"
                      }
                
            
Database

By default PyFilter uses sqlite to store banned IP addresses, if you wish to switch to redis it is as simple as changing.

                "database": "sqlite"
            

to the following in your config file.

                "database": "redis"
            
Reload iptables

By default iptables is not persistent over restarts, so this setting will reload the table with the saved bans so far on launch and update the rules.

Log files

This is the path to your log files, can be a full path or relative from where PyFilter is run.

Regex patterns

Regex patterns have to match an IP address and a timestamp, preferably matching the timestamp first. If you have a regex pattern you wish to instantly ban on, wrap the pattern with [] and add , true. For example

                
                    ["([a-zA-Z]{3}\\s+\\d{1,2} \\d{1,2}:\\d{1,2}:\\d{1,2}).* error: maximum authentication attempts exceeded for .* from (.*) port .*", true]
                
            
Time format

The time format needs to match the log format to form a datetime object. For example 2017-10-30 13:37:11 will match with %Y-%m-%d %H:%M:%S.

                
                    %Y -> Year
                    %m -> Month (Month as number form e.g 10 = October)
                    %b -> Month (Month as abbreviated name e.g Oct)
                    %B -> Month (Month as full name e.g October)
                    %d -> Day
                    %H -> Hour
                    %S -> Second
                    %d -> Day within a month (e.g 1 for 1st of the month)
                
            
Full list
Ignored IP addresses

This is quite explanatory, if match has been found but the IP address is within this list, it will be ignored so that IP address will not get blacklisted. You can add more IPs "ignored_ips": ["127.0.0.1", "123.456.789.1"].

Request time

Request time, is the time in seconds the responses have to be sent, so for example "request_time": 5 if two requests are sent within 5 seconds of each other, that will add an attempt to that IP address, if that happens 5 times they will be blacklisted and added to the firewall rules.

Deny type

Deny type is the way iptables will deal with the incoming packets, DENY is recommended however you may also REJECT them.

Failed attempts

Failed attempts is the number of matches that IP address needs to get trying to connect each rule for it to get blacklisted, for example "failed_attempts": 5 5 failed attempts on an SSH connection will get it banned, however 3 on SSH and 2 on MySQL will not get it banned, they are separate.

Cross server ban syncing


Redis

Host is the ip address of where the redis server is located. The "database" option is the database you want the banned IP addresses to be stored in, by default within redis the options are 0 to 15. If you have a password for your redis server change "password": null to "password": "your password".

Cross server ban syncing allows IP addresses to be banned across multiple servers if this is enabled. For example if IP address X was banned on server Y, and server Z has ban syncing enabled it will blacklist that IP even if that IP has not met the required failed attempts on that server.

Active

Enables or disables cross server ban syncing.

Name

This is the name of the server, this has to be different for each server running pyFilter or the bans will not get synced properly. This name can be anything as long as it is unique, for example "name": "VPS-Lon-1".

Check time

The amount of time in seconds the redis server will be polled to check for new bans, and sync them.

Running PyFilter

If you ran install.sh, PyFilter would have started and been enabled as a service, in the case you haven't there is another bash file called run.sh which can be executed to start PyFilter.

                
                    $ ./run.sh
                
            

If you get a permission error on running the bash script, make it executable with the following command.

                
                    $ sudo chmod +x run.sh