Netfilter Conntrack Status (shows all current connections)

Templates, scripts for templates, scripts and requests for templates.

Moderators: Developers, Moderators

User avatar
red_neon
Posts: 24
Joined: Sun Mar 21, 2010 11:50 pm
Location: Moscow, Russia

Netfilter Conntrack Status (shows all current connections)

Post by red_neon »

Netfilter Conntrack Status v0.4 (2014.01.04)
nf_conntrack_status_v0.4-preview.png
nf_conntrack_status_v0.4-preview.png (279.16 KiB) Viewed 17855 times
=======================================================================
DESCRIPTION


* conntrack.pl
* v0.4 (2014.01.04)
* by red_neon (red_neon [at] dcpp [dot] ru)
* Netfilter Conntrack Status [ tcp / udp / icmp / igmp / other ]

* Shows all current connections on the linux gateway\server
* it uses the netfilter conntrack module:
* http://www.frozentux.net/iptables-tutor ... ACKENTRIES

* Linux kernel version must be >= 2.6.18
=======================================================================
CHANGES


Netfilter Conntrack Status 0.4 - 2013.01.04:

1. added: method getting of stats via conntrack-tools (conntrack) by default
2. added: experimentally, realtime-mode to use with realtime plugin (only via conntrack-tools)
3. added: ipv6 support (graph shows sum of connections ipv4+ipv6)
4. added: tcp-flags: SYN_SENT2 (Linux kernel >= 2.6.31), NONE and Unknown (connections that not entered in previous. see tcp-flags info)
5. added: localhost filter for ignore src|dst=127.0.0.1|::1 connections
6. changed: now in crontab conntrack.pl must run with root privileges, because it need for conntrack-tools
7. changed: removed the need for commands in rc.local, added loading of modules nf_conntrack* via /etc/modules
8. chanded: on all graphs: "GPRINT Type" changed to some modified "Normal" (for good reading >1k connections), "Minimum" has been replaced on "Average"
9. changed: all templates have been renamed to "Netfilter Conntrack Status"
10. chanded: on the graph "Netfilter Conntrack Status - TCP" have been changed some colours


How to update previous version 0.3:

1. remove from /etc/rc.local:

/bin/chmod +r /proc/net/nf_conntrack
/bin/chmod +r /proc/net/ip_conntrack

2. in /etc/crontab
replace old string with one new (see INSTALLATION)

Next, as in instruction below.
=======================================================================
INSTALLATION


[REMOTE-SERVER SIDE]

[1] conntrack script
Copy conntrack.pl
from: conntrack_status_v0.4/remote_server/
to: /etc/srv_scripts/ (create if not exists)
set rights for execute:
$ chmod +x /etc/srv_scripts/conntrack.pl

[2] iptables (Linux kernel version must be >= 2.6.18)
$ modprobe nf_conntrack
$ modprobe nf_conntrack_ipv4
if you use ipv6:
$ modprobe nf_conntrack_ipv6

Load modules on boot:
$ echo "nf_conntrack" >> /etc/modules
$ echo "nf_conntrack_ipv4" >> /etc/modules
if you use ipv6:
$ echo "nf_conntrack_ipv6" >> /etc/modules

[3] note
On highload gateways reading of /proc/net/nf_conntrack
takes a lot of time and possibly can cause freeze.
To solve this problem - install conntrack-tools
(http://conntrack-tools.netfilter.org/)
To see how much connections now (safely):
$ cat /proc/sys/net/netfilter/nf_conntrack_count

[3a] stats via conntrack-tools
On Debian-based system:
$ apt-get install conntrack
RPM:
$ yum install conntrack-tools

Check, that "$_mode = 1" in this script below.
Ok. Go to pat.[4]

[3b] stats via /proc/net/nf_conntrack (NOT RECOMMENDED)
Use this method on server with < 10k connections at the same time.
Set "$_mode = 0" in this script below.

[4] first run
checking:
$ /usr/bin/perl /etc/srv_scripts/conntrack.pl
if is all ok - you will see stats.
$ cat /tmp/conntrack.stat

[5] snmpd
Put into snmpd.conf string with exec:
$ nano /etc/snmp/snmpd.conf
exec conntrack /bin/cat /tmp/conntrack.stat

restart snmpd:
$ /etc/init.d/snmpd restart

[6] cron
$ nano /etc/crontab
*/5 * * * * root /bin/sleep 290; /usr/bin/perl /etc/srv_scripts/conntrack.pl cron >/dev/null 2>&1

Script will get connections status every 5 mins, then it will get data and save them into temporary file in /tmp/ directory.
You can change directory and name of temporary file in script-settings below ($_tempfile),
do not forget to change string with exec the snmpd.conf

[CACTI-SERVER SIDE]

[7] cacti script
Copy snmp_execscripts.php
from: conntrack_status_v0.4/cacti_dir/scripts/
to: cacti/scripts/

[8] cacti templates
Import templates from conntrack_status_v0.4/cacti_web_templates/ in Cacti:


[EXPERIMENTALLY]

Show realtime statistics (Cacti-plugin: Realtime Graph Viewer) (only with conntrack-tools):

a) install "sudo"
$ apt-get install sudo
b) give root privileges for run this script from snmp user
$ visudo
snmp ALL = NOPASSWD:/etc/srv_scripts/conntrack.pl
c) edit snmpd.conf (replace old string, see pat.[5])
$ nano /etc/snmp/snmpd.conf
exec conntrack /usr/bin/sudo /etc/srv_scripts/conntrack.pl realtime
d) restart snmpd:
$ /etc/init.d/snmpd restart
e) remove task-string in /etc/crontab (see pat.[6])

=======================================================================

Last update: 2015.04.07
reason: attached new snmp_execscripts.php for new versions of Net-SNMP with "extend" only option.
Replace snmp_execscripts.php on new and change "exec" to "extend" in snmpd.conf



red_neon
Attachments
snmp_execscripts.php.zip
For new versions of Net-SNMP with "extend" only option
(1.32 KiB) Downloaded 429 times
nf_conntrack_status_v0.3.zip
v0.3, old
(263.38 KiB) Downloaded 343 times
nf_conntrack_status_v0.4.zip
v0.4, last
(230.46 KiB) Downloaded 936 times
Last edited by red_neon on Mon Apr 06, 2015 4:38 pm, edited 6 times in total.
User avatar
BSOD2600
Cacti Moderator
Posts: 12171
Joined: Sat May 08, 2004 12:44 pm
Location: USA

Post by BSOD2600 »

Nice lookin' graphs. Please consider posting them to the repository: http://docs.cacti.net/templates
User avatar
Hyperlord
Cacti User
Posts: 211
Joined: Tue Feb 10, 2009 3:24 am
Location: Frankfurt (a.M.), Germany

Post by Hyperlord »

Hi,

nice, will try that today/later!
Thanks for sharing and instructions!!

brgds
Om Tat Sat
User avatar
Hyperlord
Cacti User
Posts: 211
Joined: Tue Feb 10, 2009 3:24 am
Location: Frankfurt (a.M.), Germany

Post by Hyperlord »

Hi,

wow, works like a charm! I needed to do some things different (the cron part because I applied that on some linux firewall distri) but the results are working well ;)

Would you mind explaining the difference between nf_conntrack and ip_conntrack?

brgds
Om Tat Sat
User avatar
red_neon
Posts: 24
Joined: Sun Mar 21, 2010 11:50 pm
Location: Moscow, Russia

Post by red_neon »

Hello, Hyperlord!

Sample lines:

nf_conntrack:
ipv4 2 tcp 6 14395 ESTABLISHED src=1.2.3.4 dst=4.3.2.1 sport=49928 dport=80 packets=556 bytes=24203 src=4.3.2.1 dst=1.2.3.4 sport=80 dport=49928 packets=567 bytes=84160 [ASSURED] mark=0 secmark=0 use=1


ip_conntrack:
tcp 6 14399 ESTABLISHED src=1.2.3.4 dst=4.3.2.1 sport=49928 dport=80 packets=564 bytes=24523 src=4.3.2.1 dst=1.2.3.4 sport=80 dport=49928 packets=575 bytes=85338 [ASSURED] mark=0 secmark=0 use=1
"Imagination is more important than Knowledge."
User avatar
Hyperlord
Cacti User
Posts: 211
Joined: Tue Feb 10, 2009 3:24 am
Location: Frankfurt (a.M.), Germany

Post by Hyperlord »

Hi red_neon,

I was researching yesterday, found out that nf is the "enhanced" ip conntrack by netfilter. I've changed to nf, will see what happened when returning home tomorrow ;) My first look (with ip conntrac) seems quite similar to the RFC1213 stuff (http://docs.cacti.net/usertemplate:data ... _mib:stats) ... I'll investigate because those values are retrieved by SNMP directly so I don't need to actually "install" something.

brgds
Om Tat Sat
mxxcon
Posts: 31
Joined: Tue May 19, 2009 5:20 pm

Post by mxxcon »

Am i wrong or are you graphing everything stacked? Shouldn't all these graphs overlap rather than stack?

For example if you look at your screenshot of "Conntrack Status TCP". Y axis makes it looks like at peak "TIME_WAIT" had over 2000 connections, when in reality its only 280.

You'd stack graphs if you wanted to count how many total sockets you have, but it makes it useless when you are trying to look at obsolete numbers of each state.
I think that data should be graphed like in this thread http://forums.cacti.net/about31396.html
paulinster
Posts: 1
Joined: Tue Aug 17, 2010 2:27 pm

Post by paulinster »

This look to be great. I would like to add that to our cacti server. I have been trying to get it working but no success so far. Look like it doesn't get the data ... looking at the rra directory I see that the new rrd file are present but arent' getting update as other rrd file of the server.

Code: Select all

-rw-r--r--  1 bb bb  24M Aug 17 15:00 firewall_-_fwny-01_tcp_finwait_4563.rrd
-rw-r--r--  1 bb bb  43M Aug 17 15:00 firewall_-_fwny-01_igmp_unreplied_4562.rrd
-rw-r--r--  1 bb bb 5.9M Aug 17 15:00 firewall_-_fwny-01_icmp_unreplied_4561.rrd
-rw-r--r--  1 bb bb 7.8M Aug 17 15:05 firewall_-_fwny-01_udp_half_assured_4564.rrd
-rw-r--r--  1 bb bb  93K Aug 17 15:30 firewall_fwny01_traffic_in_3584.rrd
-rw-r--r--  1 bb bb  93K Aug 17 15:30 firewall_fwny01_traffic_in_3583.rrd
-rw-r--r--  1 bb bb  93K Aug 17 15:30 firewall_fwny01_traffic_in_3582.rrd
-rw-r--r--  1 bb bb  93K Aug 17 15:30 firewall_fwny01_traffic_in_3581.rrd
-rw-r--r--  1 bb bb  93K Aug 17 15:30 firewall_fwny01_traffic_in_3580.rrd
-rw-r--r--  1 bb bb  47K Aug 17 15:30 firewall_fwny01_mem_free_3579.rrd
-rw-r--r--  1 bb bb  47K Aug 17 15:30 firewall_fwny01_mem_cache_3578.rrd
-rw-r--r--  1 bb bb  47K Aug 17 15:30 firewall_fwny01_mem_buffers_3577.rrd
-rw-r--r--  1 bb bb  47K Aug 17 15:30 firewall_fwny01_load_5min_3576.rrd
-rw-r--r--  1 bb bb  47K Aug 17 15:30 firewall_fwny01_load_1min_3574.rrd
-rw-r--r--  1 bb bb  47K Aug 17 15:30 firewall_fwny01_load_15min_3575.rrd
-rw-r--r--  1 bb bb  47K Aug 17 15:30 firewall_fwny01_cpu_user_3573.rrd
-rw-r--r--  1 bb bb  47K Aug 17 15:30 firewall_fwny01_cpu_system_3572.rrd
-rw-r--r--  1 bb bb  47K Aug 17 15:30 firewall_fwny01_cpu_nice_3571.rrd
Anyone could see reason why they aren't getting updated. Cacti version is cacti-0.8.7g
densky
Posts: 2
Joined: Thu Mar 06, 2008 3:55 am

Post by densky »

on last post

[root@monitor scripts]# ls -la
...
-rwxr-x--- 1 root root 2746 Sep 15 16:43 snmp_execscripts.php

should be

-rwxr-xr--
or
cacti cacti
TvL2386
Posts: 13
Joined: Wed Nov 25, 2009 5:55 am

Re: Conntrack Status(shows all connections on the linux-gate

Post by TvL2386 »

Thanks for your Conntrack template!
I'm a happy user!

Image
User avatar
tmircea
Posts: 1
Joined: Sat Dec 28, 2013 1:24 pm

Re: Conntrack Status(shows all connections on the linux-gate

Post by tmircea »

For conntrack tables with 100k+ entries:
- install conntrack-tools
# chmod +s /usr/sbin/conntrack
- on conntrack.pl replace:

if (open($_output,'<','/proc/net/nf_conntrack')) {$_ftype=1}
elsif (open($_output,'<','/proc/net/ip_conntrack')) {$_ftype=2}

with:

if (open($_output,'/usr/sbin/conntrack -L|')) {$_ftype=2}
elsif (open($_output,'<','/proc/net/nf_conntrack')) {$_ftype=1}

Execution time (time perl ./conntrack.pl) on a router with 180k conntrack entries:
- with 'conntrack -L'

real 0m2.294s
user 0m1.848s
sys 0m1.260s

- without 'conntrack -L'

real 3m41.241s
user 0m0.500s
sys 2m1.560s
User avatar
red_neon
Posts: 24
Joined: Sun Mar 21, 2010 11:50 pm
Location: Moscow, Russia

Re: Conntrack Status(shows all connections on the linux-gate

Post by red_neon »

tmircea wrote: For conntrack tables with 100k+ entries:
- install conntrack-tools
Thanks for reminding me, been wanting to do this!
"Imagination is more important than Knowledge."
User avatar
red_neon
Posts: 24
Joined: Sun Mar 21, 2010 11:50 pm
Location: Moscow, Russia

Re: Netfilter Conntrack Status (shows all current connection

Post by red_neon »

First post has been updated!

New version!

CHANGES


Netfilter Conntrack Status 0.4 - 2013.01.04:

1. added: method getting of stats via conntrack-tools (conntrack) by default
2. added: experimentally, realtime-mode to use with realtime plugin (only via conntrack-tools)
3. added: ipv6 support (graph shows sum of connections ipv4+ipv6)
4. added: tcp-flags: SYN_SENT2 (Linux kernel >= 2.6.31), NONE and Unknown (connections that not entered in previous. see tcp-flags info)
5. added: localhost filter for ignore src|dst=127.0.0.1|::1 connections
6. changed: now in crontab conntrack.pl must run with root privileges, because it need for conntrack-tools
7. changed: removed the need for commands in rc.local, added loading of modules nf_conntrack* via /etc/modules
8. chanded: on all graphs: "GPRINT Type" changed to some modified "Normal" (for good reading >1k connections), "Minimum" has been replaced on "Average"
9. changed: all templates have been renamed to "Netfilter Conntrack Status"
10. chanded: on the graph "Netfilter Conntrack Status - TCP" have been changed some colours


How to update previous version 0.3:

1. remove from /etc/rc.local:

/bin/chmod +r /proc/net/nf_conntrack
/bin/chmod +r /proc/net/ip_conntrack

2. in /etc/crontab
replace old string with one new (see INSTALLATION)

Next, as in instruction.
"Imagination is more important than Knowledge."
Chrysalis
Cacti User
Posts: 70
Joined: Fri Sep 19, 2008 10:14 am
Location: UK

Re: Netfilter Conntrack Status (shows all current connection

Post by Chrysalis »

this isnt compatible with if_conntrack?
also on centos 5 yum install contrack-tools is no such package, had to look for custom rpm.

with conntrack-tools installed on centos however still doesnt work.

Code: Select all

# ./conntrack.pl
conntrack v1.0.0 (conntrack-tools): 71851 flow entries have been shown.
conntrack v1.0.0 (conntrack-tools): Operation failed: Address family not supported by protocol
Died at ./conntrack.pl line 294.
However conntrack -L works fine.

any ideas?

fix is to disable ipv6 on conntrack support, seems the script doesnt fallback to ipv4 only on a ipv4 only server.
lines 226 and 227 comment out.
User avatar
red_neon
Posts: 24
Joined: Sun Mar 21, 2010 11:50 pm
Location: Moscow, Russia

Re: Netfilter Conntrack Status (shows all current connection

Post by red_neon »

Chrysalis wrote:this isnt compatible with if_conntrack?
if_conntrack? No, maybe you mean ip_ or nf_?
Chrysalis wrote:

Code: Select all

# ./conntrack.pl
conntrack v1.0.0 (conntrack-tools): 71851 flow entries have been shown.
conntrack v1.0.0 (conntrack-tools): Operation failed: Address family not supported by protocol
Died at ./conntrack.pl line 294.
fix is to disable ipv6 on conntrack support, seems the script doesnt fallback to ipv4 only on a ipv4 only server.
lines 226 and 227 comment out.
I think your linux-kernel without IPv6 support. Then yes, simply comment out the lines 226,227.
I will correct this in next version, thanks.
"Imagination is more important than Knowledge."
Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests