SuStorID is an advanced Intrusion Detection System (IDS) for web services, based on machine learning. Its name comes from the term “Su Stori”, which in Sardinian language
means “The Falcon”. It’s version is experimental, but demonstrates a number of interesting features, that can be readily exploited to detect and act against web attacks:
- Autonomous Learning. Given a sample of web requests towards your web server, it can learn autonomously the profile of legitimate traffic, exploiting a number of machine learning algorithms.
- Anomaly-based Approach. SuStorID detect known as well as unknown (never-before-seen) web attacks by highlighting anomalous traffic.
- Multi-model Architecture. A multi-model architecture is employed to characterize different categories of anomalies and provide a human-readable representation of anomalous events.
- Real-time Counteractions. Depending on the set of raised anomalies and their reliability, a different counteraction can be taken.
- Easy integration with modsecurity. SuStorID can be coupled with modsecurity, the well known web application firewall, to gather training data and provide for real-time counteractions. So, SuStorID is a host-based Intrusion Detection System, and by means of modsecurity can access internal web server’s data (i.e. request and response message fields) exactly as Apache does.
- Inspection of Encrypted traffic. Since SuStorID can directly access Apache’s internal data, it can monitor both HTTP and HTTPS traffic.
- User-friendly Interface. The IDS interface is very important to investigate security issues, so SuStorID itself has been implemented as a web application with a user-friendly interface. SuStorID is written in Python exploiting the amazing Django framework, and some machine learning algorithms have been implemented in C language.
Last, but not least, SuStorID is free software, released under the GNU General Public License version 3!
- Make sure that Python v.2.6 or higher (and python numpy) and Django v. 1.3 are installed on your system. On Ubuntu:
$ apt-get install python django python-numpy
Download and extract SuStorID’s source code from the official repository.
- Open a Python Interpreter and run:
from random import choice
''.join([choice('abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)') for i in range(50)])
paste the result for your SECRET_KEY in settings.py (initial value is empty).
Inside the main directory of the project digit
$ python manage.py syncdb
This will create all tables needed by SuStorID. Specify a username and a password for administration: these credentials will be necessary to manage SuStorID and visualize detection results. By default, SuStorID uses the SQLite database, however you may use any database supported by Django to enhance performances, or employ a client-server db infrastructure (e.g. through Postgres).
$ python manage.py runserver
This will run the default Django server on the loopback interface (IP 127.0.0.1 and TCP port 8000). As any Django application, settings can be modified on settings.py. You may also change the web server to anyone supported by Django (e.g. Apache), to increase reliability and performances.
Optimization of Machine Learning algorithms
Installation Modsecurity for SuStorID
- gathering training data (i.e., web request samples)
- providing real-time counteractions against suspicious web traffic
NOTE: This guide assumes Ubuntu OS 10.04 (some steps may be valid, some other may change depending on the OS).
Download modsecurity for SuStorID source code from the official repository.
Install the required packages to build this code:
$ sudo apt-get install libc6-dev libc6 apache2 apache2-threaded-dev libxml2 libxml2-dev gcc libpcre3-dev
- Within the main directory of the project digit
This command will highlight any other missing dependency: correct it by installing the related headers/libraries.
Get into the Apache2 folder, and digit
$ sudo make install
If you don’t encounter errors, the dynamic loadable library /usr/lib/apache2/modules/mod_security2.so should be installed. It is necessary to load the mod_unique_id module of apache (thus we create a symbolic link inside the mods-enabled folder)
$ sudo ln -s /etc/apache2/mods-available/unique_id.load /etc/apache2/mods-enabled/
Inside the sustorid_config directory of this project, digit
$ sudo cp security2.load /etc/apache2/mods-available/
$ sudo ln -s /etc/apache2/mods-available/security2.load /etc/apache2/mods-enabled/
$ sudo mkdir /etc/apache2/modsecurity_conf
$ sudo cp modsecurity_conf/mod_security.conf /etc/apache2/modsecurity_conf/
Make sure that the apache2 user (e.g. www-data) can read such files (check access permissions for such files).
- Add the following line to /etc/apache2/apache2.conf
This tells Apache to load the SuStorID configuration /etc/apache2/modsecurity_conf/mod_security.conf. Read the documentation in the Configuration of modsecurity for SuStorID section to correctly configure modsecurity for SuStorID. The default configuration enables the logging module only, that is, without active Intrusion Detection/Protection: the IDS has not been trained yet and needs some training data.
- Restart Apache2
$ sudo /etc/init.d/apache2 restart
As soon as Apache restarts modsecurity for SuStorID will be activated.
Configuration of modsecurity for SuStorID
modsecurity for SuStorID comes with three additional options (with respect to modsecurity options):
- IDServer: you should specify (space-separated) IP address and TCP port where SuStorID is listening to.
- IntrusionDetection: set to On to enable Intrusion Detection by SuStorID (in conjunction with modsecurity rules). Please note that this only works if SecRuleEngine On, because this tells modsecurity to actively counteract against detected threats.
- PostRequests: set to On to enable request forwarding, that is, to automatically send requests and responses to SuStorID (it is necessary to get sample requests for learning).
That’s it. Enjoy!
Get Help and Send Suggestions
Have you got any problem? Did you find any bug? Would you like to send suggestions? Send an email to igino <dot> corona <at> diee.unica.it .
Are you a researcher working on web security? Feel free to contact igino <dot> corona <at> diee.unica.it to join the SuStorID development team. There are a number of improvements and additional features that we plan to develop in the future, towards a safer World Wide Web.
This project has been supported by a grant from Regione Autonoma della Sardegna
awarded to Igino Corona
, PO Sardegna FSE 2007-2013, L.R.7/2007 “Promotion of the scientific research and technological innovation in Sardinia”.