[solved] RRD files are there but NAN values

[meha81]

Hello ,
Please help me debug the below issue where I have empty graphs with no data being created ( even though I get RRD files in the rra directory) for a simple perl script (for printing random numbers between 1 and 10) which I have configured via the Input type of Script/Command of Data Input Methods to the Cacti (Please note I am not using SNMP) and when I run the rrd info command ./rrdtool info ../rra/random_number_11.rrd I get NAN for the below values and also see the errors in the Cacti log file - Please note that my poller table does not remain populated for a very long time - even though it gives a warning in the logs that it is not empty but it empties itself quickly . I have the UPDATE RRD files option checked in the (Output field of the Data Input method in the cacti)

03/04/2010 01:26:00 PM - CMDPHP: Poller[0] Host[0] DS[13] WARNING: Result from CMD not valid. Partial Result: U
03/04/2010 01:26:01 PM - CMDPHP: Poller[0] ERROR: Detected RRDtool Crash attempting to perform write

03/04/2010 01:30:00 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 19, Data Sources: random_number(DS[11]), random_number(DS[11]), random_number(DS[11]), random_number(DS[11]), random_number(DS[11]), random_number(DS[11]), rand_num(DS[12]), rand_num(DS[12]), rand_num(DS[12]), rand_num(DS[12]), rand_num(DS[12]), rand_num(DS[12]), rand_num(DS[12]), random_number(DS[13]), random_number(DS[13]), random_number(DS[13]), random_number(DS[13]), random_number(DS[13]), random_number(DS[13])

ds[random_number].value = NaN
rra[1].cdp_prep[0].value = NaN
rra[1].cdp_prep[0].unknown_datapoints = 0
......
rra[9].cdp_prep[0].value = NaN
rra[9].cdp_prep[0].unknown_datapoints = 221

MY SET UP IS BELOW :
I have installed cacti 0.8.7e on my windows machine with rrdtool 1.2.30 (version for win32) C:\xampplite\htdocs\cacti\rrdtool is the path for rrdtool.EXE

1. I have apache , mysql , php and have changed the settings accordingly as specified in the cacti manual
2. My poller is running via a scheduled task set to the cron interval of 5 minutes with poll interval set to 1 minute
and cron interval set to 5 minutes in the settings of cacti
3.My RRD files are also created C:\xampplite\htdocs\cacti\rra

But MY GRAPHS have NO DATA and when I run the
./rrdtool info ../rra/random_number_11.rrd command i get NAN as explained above for certain values . I suspect that is causing the graph to be empty ??

$ ./rrdtool info ../rra/random_number_11.rrd
filename = "../rra/random_number_11.rrd"
rrd_version = "0003"
step = 300
last_update = 1267727340
ds[random_number].type = "GAUGE"
ds[random_number].minimal_heartbeat = 600
ds[random_number].min = 0.0000000000e+000
ds[random_number].max = 1.0000000000e+002
ds[random_number].last_ds = "U"
ds[random_number].value = NaN
ds[random_number].unknown_sec = 240
rra[0].cf = "AVERAGE"
rra[0].rows = 500
rra[0].cur_row = 492
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-001
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 600
rra[1].cur_row = 133
rra[1].pdp_per_row = 1
rra[1].xff = 5.0000000000e-001
rra[1].cdp_prep[0].value = NaN
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 700
rra[2].cur_row = 501
rra[2].pdp_per_row = 6
rra[2].xff = 5.0000000000e-001
rra[2].cdp_prep[0].value = NaN
rra[2].cdp_prep[0].unknown_datapoints = 5
rra[3].cf = "AVERAGE"
rra[3].rows = 775
rra[3].cur_row = 218
rra[3].pdp_per_row = 24
rra[3].xff = 5.0000000000e-001
rra[3].cdp_prep[0].value = NaN
rra[3].cdp_prep[0].unknown_datapoints = 5
rra[4].cf = "AVERAGE"
rra[4].rows = 797
rra[4].cur_row = 13
rra[4].pdp_per_row = 288
rra[4].xff = 5.0000000000e-001
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 221
rra[5].cf = "MAX"
rra[5].rows = 500
rra[5].cur_row = 213
rra[5].pdp_per_row = 1
rra[5].xff = 5.0000000000e-001
rra[5].cdp_prep[0].value = NaN
rra[5].cdp_prep[0].unknown_datapoints = 0
rra[6].cf = "MAX"
rra[6].rows = 600
rra[6].cur_row = 559
rra[6].pdp_per_row = 1
rra[6].xff = 5.0000000000e-001
rra[6].cdp_prep[0].value = NaN
rra[6].cdp_prep[0].unknown_datapoints = 0
rra[7].cf = "MAX"
rra[7].rows = 700
rra[7].cur_row = 174
rra[7].pdp_per_row = 6
rra[7].xff = 5.0000000000e-001
rra[7].cdp_prep[0].value = NaN
rra[7].cdp_prep[0].unknown_datapoints = 5
rra[8].cf = "MAX"
rra[8].rows = 775
rra[8].cur_row = 106
rra[8].pdp_per_row = 24
rra[8].xff = 5.0000000000e-001
rra[8].cdp_prep[0].value = NaN
rra[8].cdp_prep[0].unknown_datapoints = 5
rra[9].cf = "MAX"
rra[9].rows = 797
rra[9].cur_row = 532
rra[9].pdp_per_row = 288
rra[9].xff = 5.0000000000e-001
rra[9].cdp_prep[0].value = NaN
rra[9].cdp_prep[0].unknown_datapoints = 221


Please help me debug this issue .

I would sincerely appreciate your help.
Thanks much,
Meha.

[BSOD2600]

1) Unless you've done all the tweaking to your cacti installation for 1 min polling, please change the polling interval back to 5 min.

2) With the cacti logging level set to debug, do you see your script run and return data?

3) The rrdtool crash is concerning. I along with lots of other users are also using the 1.2.30 win32 build and there are no reports of it crashing. Wonder what is the deal with yours...

4) "Poller Output Table not Empty" is a sign of the wrong poller interval and/or the poller running too long and over itself.

[meha81]

Hello,
Thanks so much for getting back to me with the above tips .
After your suggestions , I did the below steps :

1. Changed the poller interval to 5 minutes ( now my cron interval and poller interval in cacti settings are both set to 5 minutes)

2. About the rrd tool crashing error that I was getting previously in the logs - I noticed that in the cacti settings I had the default setting selected rrd 1.0.x but I am actually using rrd 1.2.30 so I changed the cacti setting of the RRD Utility tool to rrd 1.2.x

3. I rebuilt my Poller cache "( not sure how to check if it cleared everything ) I went to my localhost cacti and clicked on system utilities and hit the tab for Rebuild Poller cache and saw the DataSource names and details and then hit the clear tab

4. To test it from scratch I had deleted my rrd files from the rra directory

5. I stopped my windows scheduled task and restarted it - the scheduled task is scheduled to run every 5 minutes


6. I have changed the cacti logging level to "Debug" and below is the snippet from the cacti logs

RESULT = rrd files get created but still have NAN values - Graphs are still empty and From the cacti logs i do see the script creating the RRD files but after that I get the crashing error

Please see below snippet from the cacti log:

CACTI2RRD: C:/xampplite/htdocs/cacti/rrdtool/rrdtool.exe create C:/xampplite/htdocs/cacti/rra/random_number_11.rrd --step 300 DS:random_number:GAUGE:600:0:100 RRA:AVERAGE:0.5:1:500 RRA:AVERAGE:0.5:1:600 RRA:AVERAGE:0.5:6:700 RRA:AVERAGE:0.5:24:775 RRA:AVERAGE:0.5:288:797 RRA:MAX:0.5:1:500 RRA:MAX:0.5:1:600 RRA:MAX:0.5:6:700 RRA:MAX:0.5:24:775 RRA:MAX:0.5:288:797


ERROR: Detected RRDtool Crash attempting to perform write

SYSTEM STATS: Time:1.7865 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:3 RRDsProcessed:3

CMDPHP: Poller[0] Host[0] DS[11] WARNING: Result from CMD not valid. Partial Result: U

03/05/2010 06:02:01 PM - CMDPHP: Poller[0] Host[0] DS[11] CMD: C:\xampplite\htdocs\cacti\scripts\TestPerlwithCacti.pl, output: U



Kindly advise . Your help is sincerely appreciated .

thanks a lot,
Meha.

[BSOD2600]

Start simple and then get more complicated.

1) Make sure the at the very least, the included cacti templates work against some device -- snmp processor usage, disk space, interface, etc. This will help validate that your install is functional.

2) once a built in cacti template works, then you can continue developing your own script/templates. As you can see from the debug log, your script isn't returning any data: TestPerlwithCacti.pl, output: U.

The Perl script will be executed with the Cacti Scheduled task user account. It must have read/execute permissions on the perl binary\folder for starters.

[meha81]

Hello Cacti Moderator,
Thanks a lot ... the perl script worked - your last hint of permissions led me to see my data input method again. And I saw that I had provided only the path to the random number generator perl script ... had not provided the "perl" command in the beginning of the path in the data input method . When I corrected this My RRD files got generated again ( i had deleted the old ones to test from scratch) and I see graphs getting plotted . Thanks very much.
My next challenge is now to make a shell script ( which is in turn executing a java program internally) plug with cacti and since i have installed cacti on windows machine and use cygwin for shell scripting I hope cacti will be able to work with cygwin .
I will get back to you in case of any questions.
Your prompt response is sincerely appreciated.

Thanks very much,
Meha.

[meha81]

Hello Cacti Moderator,
Thanks for all the help .. Now I am trying to make a multiple output script run . I have a Java Program which I am executing via a shell script whose output is in the form

name1:value1 name2:value2 name3:value3 name4:value4

And my Data Input Method in Cacti settings is given below:
sh C:/xampplite/htdocs/cacti/scripts/OrphanCategoriesCacti.sh

Cacti does execute the Script as you can see from the log snippet below but fails to create the RRD file for multiple output fields . RRD files get created and work fine for other 3 data input methods ( rrd 11 , rrd 12 , rrd 13 ) which have only 1 output field defined in the Data Input Method but fails to create rrd 14 .

Another thing I noticed is that Poller Output table is not empty for this item , even when I truncate the table the data for this DS[14] it reappears again.

03/07/2010 09:28:00 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 1, Data Sources: DS[14]

03/07/2010 09:28:04 PM - CMDPHP: Poller[0] Host[0] DS[14] CMD: sh C:/xampplite/htdocs/cacti/scripts/OrphanCategoriesCacti.sh, output: a:3 b:1 c:4 d:3 e:15

-------------------More snippet from the log-----------------------------
03/07/2010 09:28:00 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 1, Data Sources: (DS[14])
03/07/2010 09:28:00 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Process [CMD: C:/xampplite/php/php.exe, ARGS: -q "c:/xampplite/htdocs/cacti/cmd.php" 0 0]
03/07/2010 09:28:00 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Process [CMD: C:/xampplite/php/php.exe, ARGS: -q "c:/xampplite/htdocs/cacti/cmd.php" 2 2]
03/07/2010 09:28:00 PM - CMDPHP: Poller[0] Host[0] DS[11] CMD: perl C:\xampplite\htdocs\cacti\scripts\TestPerlwithCacti.pl, output: 8
03/07/2010 09:28:00 PM - CMDPHP: Poller[0] Host[0] DS[12] CMD: perl C:\xampplite\htdocs\cacti\scripts\TestPerlwithCacti.pl, output: 6
03/07/2010 09:28:00 PM - CMDPHP: Poller[0] Host[0] DS[13] CMD: perl C:\xampplite\htdocs\cacti\scripts\TestPerlwithCacti.pl, output: 5
03/07/2010 09:28:00 PM - POLLER: Poller[0] NOTE: There are no items in your poller for this polling cycle!
03/07/2010 09:28:01 PM - POLLER: Poller[0] CACTI2RRD: C:/xampplite/htdocs/cacti/rrdtool/rrdtool.exe update C:\xampplite\htdocs\cacti\rra\random_number_11.rrd --template random_number 1268015280:8
03/07/2010 09:28:01 PM - POLLER: Poller[0] CACTI2RRD: C:/xampplite/htdocs/cacti/rrdtool/rrdtool.exe update C:\xampplite\htdocs\cacti\rra\rand_num_12.rrd --template rand_num 1268015280:6
03/07/2010 09:28:01 PM - POLLER: Poller[0] CACTI2RRD: C:/xampplite/htdocs/cacti/rrdtool/rrdtool.exe update C:\xampplite\htdocs\cacti\rra\random_number_13.rrd --template random_number 1268015280:5
03/07/2010 09:28:04 PM - CMDPHP: Poller[0] Host[0] DS[14] CMD: sh C:/xampplite/htdocs/cacti/scripts/OrphanCategoriesCacti.sh, output: a:3 b:1 c:4 d:3 e:15
03/07/2010 09:28:04 PM - CMDPHP: Poller[0] Time: 3.8218 s, Theads: N/A, Hosts: 0
03/07/2010 09:28:04 PM - SYSTEM STATS: Time:4.3395 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:4 RRDsProcessed:3

[BSOD2600]

Are there 5 fields in the Data Template for that data input method?

Are all 5 of those fields enabled for updating in your data input method? The checkbox to the far right...

Actually, as I look over the cacti log output, it only appears the TestPerlwithCacti.pl scripts' data is getting updated in the rrd files. OrphanCategoriesCacti.sh is not at all -- with a mutli-field output like that you should see cacti 'finding' and then parsing each of the fields which will then get mapped to the correct associated Data Template fields.