Some graphs show -NAN values and some show the correct value

[sokaku]

hey there, pls someone help me, it´s driving me crazy; this is my enviroment :

Ubuntu 12.04.1 LTS
Cacti Version 0.8.7i
RDDTool 1.4.x
SNMP Version 2c
NET-SNMP 5.x

Mostly of my graphs are showing correct values but some of them are showing -NAN values. The strange thing is that focussing in a single host, i can see correct values in some graphs (like net stats) and -NAN values in others (like cpu stats)
I´ve alredy checked for the correct perms in /var/lib/cacti/rra, and tried remove and readd the hosts ; nothing changes.....
Now i did a restore from a previous backup of /var/lib/cacti/rra and everything is the same but hosts where i was seeing some graphs with -NAN values and some with correct values; now im seeing -NAN values in there.

pls help!
regards!

[gandalf]

Please see 2nd link of my sig for debugging hints
R.

[sokaku]

Hey there gandalf!
thanks for your reply, i have checked everything but there is no clues, for example :

the host "sdprdbss" is showing NAN values in all graphs but NET graphs, what are showing right.

Check Cacti Log File

root@shinjuku:/var/log/cacti# grep SPINE cacti.log
root@shinjuku:/var/log/cacti# grep -i SPINE cacti.log

Check Basic Data Gathering

cacti@shinjuku:/usr/share/cacti/site/scripts$ id
uid=1001(cacti) gid=1001(cacti) grupos=1001(cacti),33(www-data)
cacti@shinjuku:/usr/share/cacti/site/scripts$ ./ping.pl sdprdbss
0.200

root@shinjuku:/etc/snmp# snmpget -v 2c -c cacti sdprdbss iso.3.6.1.2.1.6.13.1.3.0.0.0.0.51162.0.0.0.0.0
iso.3.6.1.2.1.6.13.1.3.0.0.0.0.51162.0.0.0.0.0 = INTEGER: 51162


Check Cacti's Poller

using cmd.php almost works
using spine everything is -NAN

root@shinjuku:#php -q /usr/share/cacti/site/cmd.php 165
-1

Check RRD File Update

from poller cache:

sdprdbss - Lan traffic - 192.168.199.1 SNMP Version: 2, Community: cacti, OID: .1.3.6.1.2.1.2.2.1.20.8
RRD: /var/lib/cacti/rra/165/5375.rrd
sdprdbss - Lan traffic - 192.168.199.1 SNMP Version: 2, Community: cacti, OID: .1.3.6.1.2.1.2.2.1.16.8
RRD: /var/lib/cacti/rra/165/5375.rrd
sdprdbss - Lan traffic - 192.168.199.1 SNMP Version: 2, Community: cacti, OID: .1.3.6.1.2.1.2.2.1.10.8
RRD: /var/lib/cacti/rra/165/5375.rrd
sdprdbss - Lan traffic - 192.168.199.1 SNMP Version: 2, Community: cacti, OID: .1.3.6.1.2.1.2.2.1.14.8
RRD: /var/lib/cacti/rra/165/5375.rrd
sdprdbss - Phys Memory SNMP Version: 2, Community: cacti, OID: .1.3.6.1.4.1.11.2.3.1.1.8.0
RRD: /var/lib/cacti/rra/165/5363.rrd
sdprdbss - Processes SNMP Version: 2, Community: cacti, OID: .1.3.6.1.4.1.11.2.3.1.4.1.0
RRD: /var/lib/cacti/rra/165/5367.rrd
sdprdbss - SystemUptime SNMP Version: 2, Community: cacti, OID: .1.3.6.1.4.1.11.2.3.1.1.1.0
RRD: /var/lib/cacti/rra/165/5366.rrd
sdprdbss - Users SNMP Version: 2, Community: cacti, OID: .1.3.6.1.4.1.11.2.3.1.1.2.0
RRD: /var/lib/cacti/rra/165/5368.rrd


Check RRD File Ownership

root@shinjuku:# id cacti
uid=1001(cacti) gid=1001(cacti) grupos=1001(cacti),33(www-data)
root@shinjuku:# id www-data
uid=33(www-data) gid=33(www-data) grupos=33(www-data),1001(cacti)

root@shinjuku:# ls -l /var/lib/cacti/rra | more
total 661552
drwxrwxr-x 2 www-data www-data 4096 sep 21 11:30 101
drwxrwxr-x 2 www-data www-data 4096 jun 25 15:00 102
drwxrwxr-x 2 www-data www-data 4096 jun 25 15:00 105
-rwxrwxr-x 1 www-data www-data 111392 jun 6 10:59 10_7_113_141_hdd_used_1689.rrd
-rwxrwxr-x 1 www-data www-data 111392 jun 6 10:59 10_7_113_141_hdd_used_1690.rrd
-rwxrwxr-x 1 www-data www-data 111392 jun 6 10:59 10_7_113_141_hdd_used_1691.rrd
-rwxrwxr-x 1 www-data www-data 111392 jun 6 10:59 10_7_113_141_hdd_used_1692.rrd

chown cacti:cacti *.rrd --> this does not fix the trouble

Check RRD File Numbers

root@shinjuku:# rrdtool fetch /var/lib/cacti/rra/165/5357.rrd AVERAGE | tail -20
1348468800: -nan
1348469100: -nan
1348469400: -nan
1348469700: -nan
1348470000: -nan
1348470300: -nan
1348470600: -nan
1348470900: -nan
1348471200: -nan
1348471500: -nan
1348471800: -nan
1348472100: -nan
1348472400: -nan
1348472700: -nan
1348473000: -nan
1348473300: -nan
1348473600: -nan
1348473900: -nan
1348474200: -nan
1348474500: -nan


root@shinjuku:# rrdtool info /var/lib/cacti/rra/165/5357.rrd|egrep "min|max"
ds[hpux_SysCPU].minimal_heartbeat = 600
ds[hpux_SysCPU].min = 0,0000000000e+00
ds[hpux_SysCPU].max = NaN

root@shinjuku:# rrdtool info /var/lib/cacti/rra/165/5357.rrd|grep ds
ds[hpux_SysCPU].index = 0
ds[hpux_SysCPU].type = "COUNTER"
ds[hpux_SysCPU].minimal_heartbeat = 600
ds[hpux_SysCPU].min = 0,0000000000e+00
ds[hpux_SysCPU].max = NaN
ds[hpux_SysCPU].last_ds = "U"
ds[hpux_SysCPU].value = NaN
ds[hpux_SysCPU].unknown_sec = 2

Miscellaneous

root@shinjuku:/# cat /etc/php5/cgi/php.ini | grep memory_limit
memory_limit = 128M
root@shinjuku:/# cat /etc/php5/cli/php.ini | grep memory_limit
memory_limit = 128M
root@shinjuku:/# cat /etc/php5/apache2/php.ini | grep memory_limit
memory_limit = 128M

cacti.png (111.55 KiB) Viewed 2252 times

[paulgevers]

Not 100% sure if this helps, but on Debian and Ubuntu for cmd.php the user is www-data. I am pretty sure that you should use the same user for spine.

[sokaku]

Hey paulgevers, thanks for reply!

my first step was check perm and users to solve the trouble, so i keep 2 copies of "/var/lib/cacti/rra" one belongs to user "cacti" and one belongs to user "ww-data"; anyway users are in same groups to avoid errors; this doesn´t work ; i have no more options m8:

root@shinjuku:~# id cacti
uid=1001(cacti) gid=1001(cacti) grupos=1001(cacti),33(www-data)
root@shinjuku:~# id www-data
uid=33(www-data) gid=33(www-data) grupos=33(www-data),1001(cacti)


Regards!

[gandalf]

Let's focus on a single failing graph for now, e.g. the "users" graph for the "sdprdbss" host.
Please see (e.g. from System Utilities -> View Poller Cache), which command is used for that. Find that in the verbose log and the related result as well.
R.

[sokaku]

Hey there, guys it doesn´t make sense, last night we got a power failure in the basement, where my hosts are; all hosts went down but my cacti server couse it is located on 1º floor; after that when the comunication among the servers came back i started to get correct values for all hosts. Omg i guess a reboot makes miracles sometimes. Thanks so much for your help guys, and apologies to waste your time.

Regards!