Interface Monitoring / graphing Linux Firewall

[compucoder]

Hi,

I took an old machine and made it into a perimiter firewall/gateway for the company network. I am trying to setup cacti to monitor and graph the network activity going through the 2 NIC's. 1 NIC is connected to our CiSCO router and the other goes to the internal network and servers (through a 24 port switch).

I want to see the bandwidth in/out through the firewall and a nice added bonus would be able to see a breakdown on IP/Host of our internal users and what they contribute to the overall bandwidth. I notice that /var/log/syslog seems to track all the network activity... I think - I didn't set this up so I assume it's automatic. I configured the interfaces and firewall rules using Shorewall.

I am not sure if I even need SNMP as cacti/rrdtool are running on the same machine and would have access to logs like syslog.

I have been trying everything I could think of to graph this with cacti and it never works... To be fair, I know very little about graphing and rrdtool in general so I just may be doing everything wrong.

Can someone point me in the right direction on graphing the activity going through the NIC's on my Linux firewall?

I appreciate any help you all can offer.

Thx.

[gandalf]

Typically, you would create net throughput graphs based on snmp data grabbing. See fmangeants nice HowTo on snmp setup here: http://forums.cacti.net/viewtopic.php?t=15353

Breakdown by IP@/Host is not possible using pure snmp. This would require some "flow analysis". Code for integrating this into cacti is available by means of plugins (e.g. ntop/netflow/flowtools plugin); see plugin section of this forum.

Reinhard

[compucoder]

Thanks for the Link!

Since I won't need to do anything with Syslog can you tell me how to eliminate it from recording all network activity? I didn't actually set it to do this, I assume it is the default, problem is it takes more than 1Gb of drive space each day and I am only testing on one client :S I can see it growing to 10GB or more when I install the firewall on our network.

Thanks.

[gandalf]

That's not a cacti specific question.
Quick approach: Change syslog.conf settings on your syslog server. You may find sth like

Code: Select all

# some text
local7.*                                                /your/syslog/file

Where the exact number of the log facility (local7) may vary.
Better approach: Stop syslog writing on all target systems. This is target dependant and I do not have a general approach to that.
Best approach: Don't stop syslog writing but change config on those systems to only report errors or the like. (Part of this may be realized by setting the above filter to accept only error messages or above. See man syslog.conf)
Reinhard

[compucoder]

Thanks for all the help. Using all the info you provided I was able to get basic bandwidth graphs working! I think I'll try that Netflow plugin when I get better at all this.

Btw, I do see something strange though - I am polling my 2 NIC's and my NIC that is connected to the internet (eth0) is showing very little bandwidth. I use IP Masquerading to move all internet requests between eth1 and eth0. I see the right stats for eth1 but only a few K for eth0 - shouldn't eth0 show virtually the same if not more than eth0 as everything is technically going through it?

Thanks for any added info on this topic!

[compucoder]

Oh, In case you need to see logs, I've included the last poll logs. There are errors which I don't quite understand. All the graphs I have made seem to be working... whether correctly, well that's the real question... i.e. eth0

10/10/2006 09:50:03 PM - CMDPHP: Poller[0] ERROR: There are no RRA's assigned to local_data_id: 17.
10/10/2006 09:50:03 PM - CMDPHP: Poller[0] ERROR: There are no RRA's assigned to local_data_id: 17.
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (57,'traffic_out','2006-10-10 21:50:02','5233532')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (57,'traffic_in','2006-10-10 21:50:02','93136965')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (54,'traffic_out','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (54,'traffic_in','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (53,'traffic_out','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (53,'traffic_in','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (52,'traffic_out','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (52,'traffic_in','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (51,'traffic_in','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (51,'traffic_out','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (44,'cpu_user','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (43,'cpu_system','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (46,'load_15min','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (48,'mem_buffers','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (47,'load_5min','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (49,'mem_cache','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (50,'mem_free','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (45,'load_1min','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (42,'cpu_nice','2006-10-10 21:50:02','U')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ASSERT: '457601<457601' failed. Recaching host '192.168.2.1', data query #1
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (55,'proc','2006-10-10 21:50:02','81')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (33,'','2006-10-10 21:50:02','1min:0.02 5min:0.03 10min:0.00')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (20,'hdd_free','2006-10-10 21:50:02','33402120')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (20,'hdd_used','2006-10-10 21:50:02','1600044')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (18,'hdd_free','2006-10-10 21:50:02','33402120')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (18,'hdd_used','2006-10-10 21:50:02','1600044')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (17,'hdd_free','2006-10-10 21:50:02','33402120')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (7,'proc','2006-10-10 21:50:02','79')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (17,'hdd_used','2006-10-10 21:50:02','1600044')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (4,'mem_swap','2006-10-10 21:50:02','1614492')"
10/10/2006 09:50:02 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "insert into poller_output (local_data_id,rrd_name,time,output) values (3,'mem_buffers','2006-10-10 21:50:02','501008')"

[gandalf]

The first two errors look very weird; I've never seen the up to now. This looks like a Data Source with no rra file assigned. This should not happen. It may indicate a broken mysql table (try repair them, mysql cmd is "repait table xxx", you may use phpmyadmin for ease of use or sth similar).
The last ones may indicate a problem solved by existing patches. Did you apply all patches published on main cacti site?
If problem persists, please have a look at my signatures link on "NaN Debugging". This should either help solving or at least getting more info about your problem.
Happy cactiing
Reinhard

[compucoder]

Thanks lvm,

I will try the repair table - as far as patches... I installed cacti right from the ubuntu server apt tool - so whatever it installs is what I have.

I also see a lot of errors about poller taking longer than 292 seconds and failing - I am only running interface graphing on my 2 nics - seems to be quite a long time just for 2 graphs.

I tried cactid and set it up according to the docs but that gave me no graph data so I went back to cmd.php

I'll have a look into the patches though.

Thx

[gandalf]

AFAIK, there are some posts on Ubuntu install. They might help you as well
Reinhard