| 1 |
$Id: README 724 2012-02-09 14:18:30Z bw $ |
| 2 |
|
| 3 |
fwlogwatch is a security tool written in C by Boris Wesslowski originally for |
| 4 |
RUS-CERT. It is a packet filter/firewall/IDS log analyzer with support for a |
| 5 |
lot of log formats and has many analysis options. It also features realtime |
| 6 |
response capabilities and an interactive web interface. |
| 7 |
|
| 8 |
It is available at http://fwlogwatch.inside-security.de/ |
| 9 |
|
| 10 |
|
| 11 |
FEATURES |
| 12 |
- General features: |
| 13 |
- Can detect and process log entries in the following formats: |
| 14 |
- Linux ipchains |
| 15 |
- Linux netfilter/iptables |
| 16 |
- Solaris/BSD/IRIX/HP-UX ipfilter |
| 17 |
- BSD ipfw |
| 18 |
- Cisco IOS |
| 19 |
- Cisco PIX/FWSM/ASA |
| 20 |
- NetScreen |
| 21 |
- Elsa Lancom router |
| 22 |
- Snort IDS |
| 23 |
- Entries can be parsed from single, multiple and combined log files, the |
| 24 |
parsers to be used can be selected. |
| 25 |
- Gzip-compressed logs are supported transparently. |
| 26 |
- Can separate recent from old entries and detects timewarps in log |
| 27 |
files. |
| 28 |
- Can recognize 'last message repeated' entries concerning the firewall. |
| 29 |
- Integrated resolver for protocols, services and host names. |
| 30 |
- Can do lookups in the whois database. |
| 31 |
- Own DNS and whois information cache and GNU adns support for faster |
| 32 |
lookups. |
| 33 |
- Hosts, networks, ports, chains and branches (targets) can be selected |
| 34 |
or excluded as needed. |
| 35 |
- Support for internationalization (available in english, german, |
| 36 |
portuguese, simplified and traditional chinese, swedish and japanese). |
| 37 |
- Supports IPv6 (currently only the netfilter parser, dns cache and web |
| 38 |
interface make use of it). |
| 39 |
- Log summary mode: |
| 40 |
- A lot of options to find and display relevant patterns in connection |
| 41 |
attempts. |
| 42 |
- Intelligent selection of certain fields (e.g. the host name column is |
| 43 |
omitted and the host mentioned in the header of the summary if the log |
| 44 |
is from a single host, the same happens with chains, targets and |
| 45 |
interfaces). |
| 46 |
- Output as plain text or HTML (W3C XHTML 1.1 with inline or linked CSS) |
| 47 |
with limit and sort options. |
| 48 |
- Can send summaries by email. |
| 49 |
- Realtime response mode: |
| 50 |
- The program detaches and stays in background as a daemon. |
| 51 |
- For ipchains setups detection of necessary rules with logging turned on |
| 52 |
can be configured. |
| 53 |
- Can catch up reading existing entries to provide up-to-date state |
| 54 |
information from program start on. |
| 55 |
- Response can be a notification (in form of a log file entry, an email, |
| 56 |
a remote winpopup message or whatever you can put into a shell script), |
| 57 |
or a customizable firewall modification. |
| 58 |
- The included response script adds a new chain for fwlogwatch to |
| 59 |
ipchains or netfilter setups and attackers are blocked with new |
| 60 |
firewall rules. |
| 61 |
- Supports trusted hosts (anti-spoofing). |
| 62 |
- The current status of the program can be followed and controlled |
| 63 |
through a web interface. |
| 64 |
|
| 65 |
The commented configuration file supports and explains all options and will |
| 66 |
get you started quickly. Please read the man page for details on the |
| 67 |
command line options. |
| 68 |
|
| 69 |
|
| 70 |
PARSER NOTES |
| 71 |
- Cisco PIX/ASA support focuses on denied packets, knows some permitted |
| 72 |
packet log entries and ignores all other kinds of log entries. It |
| 73 |
expects log entries as from a syslog host. If your PIX/ASA uses names |
| 74 |
or objects in the log you can use the script asa-hosts.sh from the |
| 75 |
contrib directory to extract them from a saved PIX/ASA configuration |
| 76 |
and format them as a hosts file, which you can use to initialize the |
| 77 |
dns cache. If fwlogwatch is not able to resolve the names it will |
| 78 |
discard the corresponding log entries. |
| 79 |
- The ipfilter parser does not support logs with resolved service names |
| 80 |
and tcp/udp entries without ports. |
| 81 |
- The Snort parser does not analyze portscan entries. |
| 82 |
- The NetScreen parser does not recognize packet-filter-unrelated entries |
| 83 |
and some icmp types which the NetScreen does not seem to recognize |
| 84 |
itself... |
| 85 |
|
| 86 |
|
| 87 |
INSTALLATION |
| 88 |
- General |
| 89 |
The Makefile assumes you use Linux, to compile on Mac OS X, Solaris, |
| 90 |
OpenBSD, FreeBSD or IRIX look for the corresponding lines at the top of |
| 91 |
the Makefile. fwlogwatch may also be compiled and run on Windows with |
| 92 |
help of cygwin or mingw. You may want to have a look at main.h if you |
| 93 |
want to change some default values. |
| 94 |
|
| 95 |
flex is required to build fwlogwatch. Besides that a simple 'make' should |
| 96 |
be enough to obtain a working binary. If your make (like the one on |
| 97 |
OpenBSD) thinks flex can only produce files called lex.yy.c type make |
| 98 |
several times, after all parsers are generated linking will work. |
| 99 |
|
| 100 |
If you use the realtime response mode you will need to install fwlogwatch |
| 101 |
with superuser permissions for certain configurations. If all you need is |
| 102 |
read access to the system's standard log file (e.g. /var/log/messages) |
| 103 |
you can use group permissions. You can also bind a nonprivileged port for |
| 104 |
the status server. 'make install' will install the binary (in |
| 105 |
/usr/local/sbin) and the man page, 'make install-config' will also |
| 106 |
install a sample configuration file in /etc. |
| 107 |
|
| 108 |
- Zlib/Gettext/GNU adns |
| 109 |
If you define HAVE_ZLIB, HAVE_GETTEXT and HAVE_ADNS fwlogwatch will be |
| 110 |
compiled with zlib, gettext and adns support. The output of fwlogwatch -V |
| 111 |
will contain an overview of the compiler options used. |
| 112 |
|
| 113 |
You will need the zlib compression library version 1.0.9 or newer and |
| 114 |
it's header files to be able to use zlib support. |
| 115 |
|
| 116 |
To enable the internationalization support you will need to have the |
| 117 |
gettext package (sometimes with it's surroundings libtool, GNU m4, |
| 118 |
autoconf and automake) installed. |
| 119 |
|
| 120 |
The GNU adns library and header files must be installed to build fwlogwatch |
| 121 |
with support for faster, asynchronous DNS lookups. |
| 122 |
|
| 123 |
- Linux |
| 124 |
For ipchains you will need at least kernel 2.2.10 (which you should have |
| 125 |
updated for security reasons anyway), kernel versions before 2.2.10 don't |
| 126 |
log enough. |
| 127 |
|
| 128 |
|
| 129 |
BASICS (with examples for iptables) |
| 130 |
- You should use a whitelist (meaning your firewall only allows connections |
| 131 |
that are explicitly permitted and should be as specific as possible), so |
| 132 |
create a packet filter with ACCEPT rules for all connections you need. |
| 133 |
|
| 134 |
- Your default policies can be DROP if you want your firewall to block all |
| 135 |
connections in case all rules are deleted: |
| 136 |
|
| 137 |
iptables -P INPUT DROP |
| 138 |
iptables -P FORWARD DROP |
| 139 |
|
| 140 |
Otherwise you should add a DROP rule at the end of all chains: |
| 141 |
|
| 142 |
iptables -A INPUT -j DROP |
| 143 |
iptables -A FORWARD -j DROP |
| 144 |
|
| 145 |
- Before those DROP rules (or the end of chains with DROP policy) you |
| 146 |
should have a LOG rule with some descriptive text like the context and |
| 147 |
chain name: |
| 148 |
|
| 149 |
iptables -A INPUT -j LOG --log-prefix "fw input drop " |
| 150 |
iptables -A FORWARD -j LOG --log-prefix "fw forward drop " |
| 151 |
|
| 152 |
So the last 2 lines in your INPUT, FORWARD and custom chains should be |
| 153 |
|
| 154 |
pkts bytes target prot opt in out source destination |
| 155 |
0 0 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 LOG flags 0 level 4 prefix `fw forward drop ` |
| 156 |
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 |
| 157 |
|
| 158 |
- If you have a persistent log spammer drop his packets by inserting a DROP |
| 159 |
rule before the logging rule above (or use the block response mode). |
| 160 |
|
| 161 |
|
| 162 |
CONFIGURATION AND EXAMPLES |
| 163 |
- You should make a configuration file for each function you want, look at |
| 164 |
the included sample file, it should be easy to adapt to your needs. |
| 165 |
|
| 166 |
- Command line example: |
| 167 |
(The options are in the order they are mentioned in the text) |
| 168 |
If you want very verbose generation of a HTML summary in the file |
| 169 |
'log.html' of all packet filter entries at most one day old representing |
| 170 |
at least two connection attempts with output including start and end |
| 171 |
timestamps, time intervals, resolved IP addresses and service names and |
| 172 |
with connections separated by protocol, source and destination ports and |
| 173 |
TCP options using the files that match the expression "messages*.gz" as |
| 174 |
input you would use the command |
| 175 |
|
| 176 |
fwlogwatch -v -v -w -o log.html -l 1d -m 2 -t -e -z -n -N -p -s -d -y messages*.gz |
| 177 |
|
| 178 |
- If you want to use fwlogwatch as a CGI, e.g. to have a quick look at |
| 179 |
what happened in the last hour when you get a notification: |
| 180 |
Copy the file fwlogsummary_small.cgi from the contrib directory to a |
| 181 |
place where your web server can execute it (fwlogwatch must be reachable |
| 182 |
and have enough permissions to read the log file). |
| 183 |
|
| 184 |
- A script for generation of 8 general summaries (fwlogsummary.cgi) is also |
| 185 |
included. You can use it as above or if you comment a few lines also on |
| 186 |
the command line or from cron. It will use the output directory |
| 187 |
/var/www/html/fwlogwatch as default. Look at the index.html file to |
| 188 |
select the level of detail you want. |
| 189 |
|
| 190 |
- Contrib also contains a web frontend written in PHP (fwlogwatch.php) to |
| 191 |
apply fwlogwatch to selected files and test different options. Change the |
| 192 |
header of the script to adapt it to your system (e.g. prefix of the log |
| 193 |
file names, location of fwlogwatch). |
| 194 |
|
| 195 |
- Sample init files (for Red Hat and openSUSE Linux systems) to start |
| 196 |
fwlogwatch in realtime response mode at system start are also included. |
| 197 |
|
| 198 |
- You might want to replace your /etc/services file by the one supplied |
| 199 |
with nmap (http://www.insecure.org/nmap/), a lot more services will be |
| 200 |
recognized, you can also add the ICMP types (don't assign one to type 0 |
| 201 |
since it is the ipchains default). The same applies to /etc/protocols, |
| 202 |
you can use RFC 1700 (Assigned Numbers) to extend your version. |
| 203 |
|
| 204 |
|
| 205 |
FEEDBACK |
| 206 |
If you find a bug or have an idea for a new feature please send an email |
| 207 |
to Boris Wesslowski <[email protected]>. |
| 208 |
|
| 209 |
If you want to report a parser problem or submit unrecognized entries |
| 210 |
please use the unrecognized entry submission page: |
| 211 |
http://fwlogwatch.inside-security.de/unrecognized.php |