No graph for newly added devices

[Fox5]

I have been using Cacti for few years now with no problem. I have a few dozen devices being monitored. Lately cacti has stopped updating one graph, and if I add any new device(s) I get an empty graph for it! I followed http://docs.cacti.net/manual:087:4_help ... #debugging to the end, but still no go. The poller is working, the .rrd files are all updated every 5 minutes for all devices including the newly added one, and the one with graph problem! No error in the log files regarding my problem. I can do a manual snmpwalk, and snmpget, but my problem persists.

Any advise is greatly appreciated.

[gandalf]

As you already followed the debugging guide, what was the result of
- manually updating the rrd file
- fetching last data from rrd file
R.

[Fox5]

File gets updated no problem. Manually, or every 5 minutes. I do however get this:
/usr/bin/php -q /usr/share/cacti/cmd.php 158 158
PHP Warning: Unknown(): Invalid library (maybe not a PHP library) 'rrdtool.so' in Unknown on line 0
But, I get this even when I run the above on working graphs too.

I ended up removing this node from data source and added back hoping this was a glitch and will go away, but now running
# rrdtool fetch rout02_8600_traffic_in_3425.rrd AVERAGE gives nothing but
traffic_in traffic_out
1357856400: nan nan
1357856700: nan nan
1357857000: nan nan

# rrdtool info rout02_8600_traffic_in_3425.rrd
filename = "rout02_8600_traffic_in_3425.rrd"
rrd_version = "0003"
step = 300
last_update = 1357942157
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 600
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 1.0000000000e+09
ds[traffic_in].last_ds = "184383092958"
ds[traffic_in].value = 0.0000000000e+00
ds[traffic_in].unknown_sec = 257
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 600
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 1.0000000000e+09
ds[traffic_out].last_ds = "57122154468"
ds[traffic_out].value = 0.0000000000e+00
ds[traffic_out].unknown_sec = 257
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0

rrdtool tune rout02_8600_traffic_in_3425.rrd --maximum traffic_in:4.0000000000e+09
rrdtool tune rout02_8600_traffic_in_3425.rrd --maximum traffic_out:4.0000000000e+09

made absolutely no difference!

Thanks...

[gandalf]

File gets updated no problem. Manually, or every 5 minutes. I do however get this:
/usr/bin/php -q /usr/share/cacti/cmd.php 158 158
PHP Warning: Unknown(): Invalid library (maybe not a PHP library) 'rrdtool.so' in Unknown on line 0
But, I get this even when I run the above on working graphs too.

Nevertheless, I see no sense in going further unless this issue is solved. Did you manually enter sth into the php config? Is there anything about rrd to be found either in /etc/php.ini or /etc/php.d?
R.

[Fox5]

Cacti was setup by someone else.
# grep -i -r rrd /etc/php.d/*
/etc/php.d/rrdtool.ini:; Enable rrdtool extension module
/etc/php.d/rrdtool.ini:extension=rrdtool.so

I commented out and restarted http. The PHP warning is gone, but still no graph!


For the records;
# rpm -q cacti
cacti-0.8.7e-1.el4.noarch
# rpm -q rrdtool
rrdtool-1.2.27-3.el4.i386


Thanks...

[gandalf]

Please visit Graph Management and select that graph.
Is it empty or is it broken?

Please switch to DEBUG mode for that graph, then.
Copy it to command line (or a shel script) and run it, piping output to e.g. foo.png.
Any error message shown?

R.

[Fox5]

> Please visit Graph Management and select that graph.
> Is it empty or is it broken?
It is not broken. It is empty.

> Please switch to DEBUG mode for that graph, then.
> Copy it to command line (or a shel script) and run it, piping output to e.g. foo.png.
> Any error message shown?

No error message.
+ Running data query [1].
+ Found type = '3' [snmp query].
+ Found data query XML file at '/usr/share/cacti/resource/snmp_queries/interface.xml'
+ XML file parsed ok.
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.2.1.2.2.1.1'
+ Index found at OID: '.1.3.6.1.2.1.2.2.1.1.64' value: '64'
+ Index found at OID: '.1.3.6.1.2.1.2.2.1.1.65' value: '65'
+ Index found at OID: '.1.3.6.1.2.1.2.2.1.1.66' value: '66'
+ Index found at OID: '.1.3.6.1.2.1.2.2.1.1.67' value: '67'
+ Index found at OID: '.1.3.6.1.2.1.2.2.1.1.68' value: '68'
.........................
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.1'
+ Found item [ifIndex='64'] index: 64 [from value]
+ Found item [ifIndex='65'] index: 65 [from value]

rrdtool info xyz.rrd gives:
rrd_version = "0003"
step = 300
last_update = 1358288047
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 600
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 1.0000000000e+09
ds[traffic_in].last_ds = "85089476016855"
ds[traffic_in].value = 0.0000000000e+00
ds[traffic_in].unknown_sec = 0
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 600
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 1.0000000000e+09
ds[traffic_out].last_ds = "64289409453504"
ds[traffic_out].value = 0.0000000000e+00
ds[traffic_out].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700

However, rrdtool fetch xyz.rrd AVERAGE is showing bunch of zeros for traffic in/out:
1358201100: nan nan
1358201400: 0.0000000000e+00 0.0000000000e+00
1358201700: 0.0000000000e+00 0.0000000000e+00
1358202000: 0.0000000000e+00 0.0000000000e+00
1358202300: 0.0000000000e+00 0.0000000000e+00
1358202600: 0.0000000000e+00 0.0000000000e+00
1358202900: 0.0000000000e+00 0.0000000000e+00
1358203200: 0.0000000000e+00 0.0000000000e+00
..............
1358287200: nan nan
1358287500: nan nan


This is bizarre!!!

[gandalf]

So we seem to have an issue with rrdtool update.
Please see that section of 2nd link of my sig
R.

[Fox5]

I reinstalled cacti using the rpm on a different server, but got the same result for the same node! This problem is unique to one node only!
Data Query Debug Information
+ Running data query [1].
+ Found type = '3' [SNMP Query].
+ Found data query XML file at '/usr/share/cacti/resource/snmp_queries/interface.xml'
+ XML file parsed ok.
+ Executing SNMP get for num of indexes @ '.1.3.6.1.2.1.2.1.0' Index Count: 16
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.2.1.2.2.1.1' Index Count: 16
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.64' value: '64'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.65' value: '65'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.66' value: '66'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.67' value: '67'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.68' value: '68'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.69' value: '69'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.70' value: '70'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.71' value: '71'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.128' value: '128'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.129' value: '129'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.130' value: '130'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.131' value: '131'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.132' value: '132'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.133' value: '133'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.134' value: '134'
+ Index found at OID: '1.3.6.1.2.1.2.2.1.1.135' value: '135'
+ Located input field 'ifIndex' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.1'
+ Found item [ifIndex='64'] index: 64 [from value]
+ Found item [ifIndex='65'] index: 65 [from value]
+ Found item [ifIndex='66'] index: 66 [from value]
+ Found item [ifIndex='67'] index: 67 [from value]
+ Found item [ifIndex='68'] index: 68 [from value]
+ Found item [ifIndex='69'] index: 69 [from value]
+ Found item [ifIndex='70'] index: 70 [from value]
+ Found item [ifIndex='71'] index: 71 [from value]
+ Found item [ifIndex='128'] index: 128 [from value]
+ Found item [ifIndex='129'] index: 129 [from value]
+ Found item [ifIndex='130'] index: 130 [from value]
+ Found item [ifIndex='131'] index: 131 [from value]
+ Found item [ifIndex='132'] index: 132 [from value]
+ Found item [ifIndex='133'] index: 133 [from value]
+ Found item [ifIndex='134'] index: 134 [from value]
+ Found item [ifIndex='135'] index: 135 [from value]
+ Located input field 'ifOperStatus' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.8'
+ Found item [ifOperStatus='Up'] index: 64 [from value]
+ Found item [ifOperStatus='Up'] index: 65 [from value]
+ Found item [ifOperStatus='Up'] index: 66 [from value]
+ Found item [ifOperStatus='Up'] index: 67 [from value]
+ Found item [ifOperStatus='Up'] index: 68 [from value]
+ Found item [ifOperStatus='Up'] index: 69 [from value]
+ Found item [ifOperStatus='Up'] index: 70 [from value]
+ Found item [ifOperStatus='Down'] index: 71 [from value]
+ Found item [ifOperStatus='Up'] index: 128 [from value]
+ Found item [ifOperStatus='Up'] index: 129 [from value]
+ Found item [ifOperStatus='Up'] index: 130 [from value]
+ Found item [ifOperStatus='Up'] index: 131 [from value]
+ Found item [ifOperStatus='Up'] index: 132 [from value]
+ Found item [ifOperStatus='Up'] index: 133 [from value]
+ Found item [ifOperStatus='Up'] index: 134 [from value]
+ Found item [ifOperStatus='Up'] index: 135 [from value]
+ Located input field 'ifDescr' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.2'
.......

[Fox5]

A question, what is the procedure for migrating cacti to a new server that has a newer version of cacti running with its own database? How would you merge them yet at the same time keep the .rrd files happy?

Thanks...

[paulgevers]

A question, what is the procedure for migrating cacti to a new server that has a newer version of cacti running with its own database? How would you merge them yet at the same time keep the .rrd files happy?

Thanks...

You can get some ideas here. Mainly: just copy the mysql database over, and tell cacti on request to update the database for the newer version. This works if the old cacti version is 0.8.7a or higher. If the architecture of both machines is the same you could also just copy the rdd files over instead of using the dump script.