monitoring wide network activity... nead help...

[Sirijus]

Hi,

i want to draw network activity graphs for real wide network.

i have more then 100 C class networks.
using iptables i gathering traffic for each class into proc file system like this:

/proc/net/ipt_account/all_10.0.0

Code: Select all

ip = 10.0.0.0 bytes_src = 268418884856 packets_src = 314154820 bytes_dest = 117268842086 packets_dest = 267543132 time = 0
ip = 10.0.0.1 bytes_src = 23974267 packets_src = 25166 bytes_dest = 3836600 packets_dest = 23424 time = 103
ip = 10.0.0.2 bytes_src = 0 packets_src = 0 bytes_dest = 694105 packets_dest = 1497 time = 47
ip = 10.0.0.3 bytes_src = 1090788 packets_src = 2509 bytes_dest = 956473 packets_dest = 3310 time = 103
ip = 10.0.0.4 bytes_src = 60 packets_src = 1 bytes_dest = 566117 packets_dest = 1216 time = 103
...
...
...
ip = 10.0.0.254 bytes_src = 237770 packets_src = 564 bytes_dest = 432256 packets_dest = 993 time = 76
ip = 10.0.0.255 bytes_src = 0 packets_src = 0 bytes_dest = 450094 packets_dest = 1028 time = 132

and

/proc/net/ipt_account/p2p_10_0_0

Code: Select all

ip = 10.0.0.0 bytes_src = 247995259676 packets_src = 229683574 bytes_dest = 52048287474 packets_dest = 170078858 ti
me = 0
ip = 10.0.0.1 bytes_src = 0 packets_src = 0 bytes_dest = 0 packets_dest = 0 time = 142561
ip = 10.0.0.2 bytes_src = 0 packets_src = 0 bytes_dest = 0 packets_dest = 0 time = 142561
ip = 10.0.0.3 bytes_src = 0 packets_src = 0 bytes_dest = 0 packets_dest = 0 time = 142561
...
...
...
ip = 10.0.0.253 bytes_src = 0 packets_src = 0 bytes_dest = 74449 packets_dest = 704 time = 44
ip = 10.0.0.254 bytes_src = 0 packets_src = 0 bytes_dest = 0 packets_dest = 0 time = 142561
ip = 10.0.0.255 bytes_src = 0 packets_src = 0 bytes_dest = 0 packets_dest = 0 time = 142561

so i nead read data from thous files and put in one rrd file.

using script i can manipulate with data and give it to cacti like this:

./account_multi.php 10_0_0

result can be:

Code: Select all

allFrom0:270242745536 allTo0:84932551882 p2pFrom0:248253526984 p2pTo0:52287684287 allFrom1:24024546 allTo1:3841236 p2pFrom1:0 p2pTo1:0 allFrom2:0 allTo2:696055 p2pFrom2:0 p2pTo2:0 allFrom3:1090788 allTo3:958423 p2pFrom3:0 p2pTo3:0 allFrom4:60 allTo4:568067 p2pFrom4:0 p2pTo4:0 allFrom5:0 allTo5:577918 p2pFrom5:0 p2pTo5:0 allFrom6:12925704 allTo6:1200417 p2pFrom6:0 p2pTo6:0 ... ... ... allFrom255:0 allTo255:0 p2pFrom255:0 p2pTo255:0

now i have the problem with Data input metods. have i by hands create 1020 data output fields ? or i can aquire data in any other way ?
and lets suppose i create so much fields, what to do with data template then ? have i create again 1020 data source items ?

is there any way to automate this proccess ?

thanks for your help...

[gandalf]

...
now i have the problem with Data input metods. have i by hands create 1020 data output fields ? or i can aquire data in any other way ?
and lets suppose i create so much fields, what to do with data template then ? have i create again 1020 data source items ?

is there any way to automate this proccess ?

thanks for your help...

This depends largely on the ability to create an index. E.g. if you collect data of same kind for each of your "targets" and there is an index that corresponds to that target (e.g. target name will do, no need for pure numeric indices), that's fine. You'll create a Data Template for the data retrieved from/about each of those targets and cacti will do the work. See last two links of my signature for some insight and ask, if it's not clear enough
Reinhard

[Sirijus]

Hi,

actualy here and is the problem.

if i will create like suggested in your example it will create 1 rrd file per 1 IP address...

this can be a problem, becouse now i have 160 C classes - so it means 256 x 160 = 40960 data sources... and 40960 queries each 5 min... this will bring to much load on system.

if i can get all data for 256 hosts per query i will have just 160 data sources and 160 queries each 5 minutes.

maybe it is posibble to insert info about data directly into the SQL table ? i just dont understand from where i must generate hash...

[gandalf]

maybe it is posibble to insert info about data directly into the SQL table ? i just dont understand from where i must generate hash...

Retrieved data is stored in rrd's only, not MySQL
Reinhard