Graph with NaN value and datasource not as expected

[antokapo]

Hi

I'm experiencing a crazy behaviour on my cact installation :

General Information
Date Wed, 16 Oct 2019 12:48:20 +0200
Cacti Version 0.8.8b
Cacti OS unix
SNMP Version NET-SNMP version: 5.4.3
RRDTool Version RRDTool 1.4.x
Hosts 1428
Graphs 43500
Data Sources Script/Command: 5
SNMP: 1950
SNMP Query: 124602
Script Query: 3
Script Query - Script Server: 378
Total: 126938
Poller Information
Interval 300
Type SPINE 0.8.8b Copyright 2002-2013 by The Cacti Group
Items Action[0]: 246712
Action[1]: 11
Action[2]: 582
Total: 247305
Concurrent Processes 4
Max Threads 5
PHP Servers 4
Script Timeout 180
Max OID 15
Last Run Statistics Time:111.7686 Method:spine Processes:4 Threads:5 Hosts:1427 HostsPerProcess:357 DataSources:247305 RRDsProcessed:0
PHP Information
PHP Version 5.3.10-1ubuntu3.24
PHP OS Linux
PHP uname Linux netmonitor 3.13.0-96-generic #143~precise1-Ubuntu SMP Tue Aug 30 14:38:06 UTC 2016 x86_64
PHP SNMP Installed
max_execution_time 300
memory_limit 2560M

I created a new graph , but it doesn't populate, all NaN values in the rrd , i suspect that database got corrupted because of many inconstistencies found by the script repair_database.php ,
and also the data source that has been generated is different from the other ones related to the same data template.

I can give further details if you need them.

One thing i've done is to change the data template associated with the graph, maybe this is messing all up.

Thanks for your help

Antonello

[antokapo]

Hi

I'm experiencing a crazy behaviour on my cact installation :

General Information
Date Wed, 16 Oct 2019 12:48:20 +0200
Cacti Version 0.8.8b
Cacti OS unix
SNMP Version NET-SNMP version: 5.4.3
RRDTool Version RRDTool 1.4.x
Hosts 1428
Graphs 43500
Data Sources Script/Command: 5
SNMP: 1950
SNMP Query: 124602
Script Query: 3
Script Query - Script Server: 378
Total: 126938
Poller Information
Interval 300
Type SPINE 0.8.8b Copyright 2002-2013 by The Cacti Group
Items Action[0]: 246712
Action[1]: 11
Action[2]: 582
Total: 247305
Concurrent Processes 4
Max Threads 5
PHP Servers 4
Script Timeout 180
Max OID 15
Last Run Statistics Time:111.7686 Method:spine Processes:4 Threads:5 Hosts:1427 HostsPerProcess:357 DataSources:247305 RRDsProcessed:0
PHP Information
PHP Version 5.3.10-1ubuntu3.24
PHP OS Linux
PHP uname Linux netmonitor 3.13.0-96-generic #143~precise1-Ubuntu SMP Tue Aug 30 14:38:06 UTC 2016 x86_64
PHP SNMP Installed
max_execution_time 300
memory_limit 2560M

I created a new graph , but it doesn't populate, all NaN values in the rrd , i suspect that database got corrupted because of many inconstistencies found by the script repair_database.php ,
and also the data source that has been generated is different from the other ones related to the same data template.

I can give further details if you need them.

One thing i've done is to change the data template associated with the graph, maybe this is messing all up.

Thanks for your help

Antonello

Even if i simply save the data_template , data_source changes ... that drives me crazy .. probably it's better to upgrade this CACTI instance

[antokapo]

I found a possibile problem when querying the remote Cisco router:

anledda@netmonitor:~$ sudo snmpwalk -v 2c -c CiscovieW xx.xxx.xxx.xxx .1.3.6.1.2.1.31.1.1.1.6.53
IF-MIB::ifHCInOctets.53 = No Such Instance currently exists at this OID
anledda@netmonitor:~$ sudo snmpwalk -v 2c -c CiscovieW xxx.xxx.xxx.xxx .1.3.6.1.2.1.31.1.1.1.10.53
IF-MIB::ifHCOutOctets.53 = No Such Instance currently exists at this OID
anledda@netmonitor:~$

But if i do a check on the router i get :

ifName : GigabitEthernet0/2/0/38 ifIndex : 53

as excerpt of the output of " sh snmp interface " command ...

[antokapo]

and finally i had to restart the mibd_interface process on the router to get it working!!

ONE MAN SHOW

Bye

Antonello

[netniV]

Well done on diagnosing that, I doubt I would have found it as I wasn't even aware you could just restart processes on a router.

[antokapo]

To tell you the truth ,
after i set the poller in DEBUG mode,
I noticed this row :

SPINE: Poller[0] Host[1419] TH[1] DS[362819] SNMP: v2: xxx.xxx.xxx.xxx, dsname: traffic_in, oid: .1.3.6.1.2.1.31.1.1.1.6.53, value: U

and it reminded me to a similar problem we had in the past with the CISCO ASR9K family ,
as a member of the network department i could restart any process

Regards

Antonello

[netniV]

Lucky to have that prior knowledge. I was more meaning that I didn't realise you could restart a process within a router without fully rebooting. I just would never image that they'd allow the end user to do that as I've not had a device allowing it.

[antokapo]

Hi

Please excuse me, i misunderstood what you wrote.

CISCO IOS XR allow you to have more granularity on processes control, as you can do in UNIX platforms, you can even run a shell on those enterprise level devices.

My team in in charge for interface and statistics graph on many network devices, so we often face that kind of problem

Regards

Antonello