SOLVED: data collection through python script

[kmart]

Hi all,

Cacti version 1.2.2 running on ubuntu server 18.04

I have a python script with multiple outputs which looks like this:

Code: Select all

icTotal:19 icAvgTime:40.31 icQueuedTotal:2 icAvgQueueTime:0.84 icMaxQueueTime:1 gcTotal:18 gcAvgTime:10.74 gcQueuedTotal:0 gcAvgQueueTime:0.00 gcMaxQueueTime:0 totalCalls:37 totalAvgTime:25.92

I can run it from the command line no probem. script takes an ip address and a second parameter like so:
python3 /opt/cacti/scripts/bsCallStats.py 192.168.10.38 5

So I've created a data input method for this script, data source templates, and now I'm trying to graph two of those outputs but I'm just getting NaN... There's nothing in the logs that indicates what is happening.

Anyone have any advice on how to troubleshoot this? With a lack of indication in the cacti logs i don't know where to start. screenshot attached that may be relevant.

Thanks

Data input method for script

data input method.png (33.27 KiB) Viewed 3293 times

Data input method for script

data input method.png (33.27 KiB) Viewed 3293 times

Data input method for script

data input method.png (33.27 KiB) Viewed 3293 times

[kmart]

well i enabled debug logging and i can see that the script runs with the proper output. The script server parses the output properly and appears to put it in the rrd according to the log but my graph still has no data.

[camerabob]

Look at the file date on the rrd file to see if it is really being updated.

[kmart]

This is the logging output. Script is executing and it seems like the output is being parsed properly:

Code: Select all

2019/08/02 00:38:36 - POLLER: Poller[Main Poller] Parsed MULTI output field 'icTotal:1' [map icTotal->icTotal]
2019/08/02 00:38:35 - SPINE: Poller[Main Poller] Device[BS06-Hillcrest] HT[1] DS[BS06-Hillcrest_indvCalls] Graphs[BS06-Hillcrest - Calls By Type] SCRIPT: /usr/bin/python3 /opt/cacti/scripts/bsCallStats.py '192.168.10.38' '5', output: icTotal:1 icAvgTime:12.94 icQueuedTotal:0 icAvgQueueTime:0.00 icMaxQueueTime:0 gcTotal:12 gcAvgTime:15.66 gcQueuedTotal:0 gcAvgQueueTime:0.00 gcMaxQueueTime:0 totalCalls:13 totalAvgTime:15.45 

I also see the rrdtool update with the parsed data... But the rrd only contains NaN's

I believe I've found what could be the problem but i can't seem to fix it.

filename = "bs06-hillcrest_ictotal_2315.rrd"
rrd_version = "0003"
step = 300
last_update = 1564718608
header_size = 2040
ds[icTotal].index = 0
ds[icTotal].type = "GAUGE"
ds[icTotal].minimal_heartbeat = 120
ds[icTotal].min = NaN
ds[icTotal].max = NaN
ds[icTotal].last_ds = "11"
ds[icTotal].value = NaN
ds[icTotal].unknown_sec = 208
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].cur_row = 309
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[1].cur_row = 236
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 0.0000000000e+00
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 775
rra[2].cur_row = 611
rra[2].pdp_per_row = 24
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 0.0000000000e+00
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[3].cur_row = 109
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 0.0000000000e+00
rra[3].cdp_prep[0].unknown_datapoints = 48
rra[4].cf = "MAX"
rra[4].rows = 600
rra[4].cur_row = 424
rra[4].pdp_per_row = 1
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[5].cf = "MAX"
rra[5].rows = 700
rra[5].cur_row = 11
rra[5].pdp_per_row = 6
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = -inf
rra[5].cdp_prep[0].unknown_datapoints = 0
rra[6].cf = "MAX"
rra[6].rows = 775
rra[6].cur_row = 195
rra[6].pdp_per_row = 24
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = -inf
rra[6].cdp_prep[0].unknown_datapoints = 0
rra[7].cf = "MAX"
rra[7].rows = 797
rra[7].cur_row = 336
rra[7].pdp_per_row = 288
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = -inf
rra[7].cdp_prep[0].unknown_datapoints = 48

the data source min and max are set to NaN. When i setup the template min and max were set to U, I'm not sure why it's currently NaN.

But when i try to use:

Code: Select all

rrdtool tune bs06-hillcrest_ictotal_2315.rrd --maximum ds[icTotal].max:U

I get:

Code: Select all

ERROR: Invalid arguments for maximum ds value

Any ideas on
1) why the rrd would be made with no min/max specified despite being set to U in the template.
2) I can't change it to U

Thanks

[kmart]

well my syntax was wrong using rrdtune... Once i figured it out i realized that U for .max .min values makes them NaN so that's not the issue.

I'm at a loss. I even see in the log that cacti is trying to update the rrd with the value extracted from the script output

Code: Select all

2019/08/02 01:08:29 - POLLER: Poller[Main Poller] CACTI2RRD: /usr/bin/rrdtool update /opt/cacti/rra/bs06-hillcrest_ictotal_2317.rrd --skip-past-updates --template icTotal 1564722508:3
2019/08/02 01:03:32 - POLLER: Poller[Main Poller] CACTI2RRD: /usr/bin/rrdtool update /opt/cacti/rra/bs06-hillcrest_ictotal_2317.rrd --skip-past-updates --template icTotal 1564722211:1 

If i run that from cli rrdtool accepts it but the rrd just has NaN. You can see in the info output that the last value seems to make it so what gives?

[kmart]

Throw enough crap at the wall and eventually something will stick...

Turns out it was the minimal_heartbeat setting in the rrd that was the problem.

I've been dealing with odd issues here and there since upgrading from 0.8.8 to 1.2.2. One of them seems to be that my 5 minute polling data profile has a heartbeat setting of 2 minutes (120 seconds) after the upgrade.

[netniV]

I've raised a feature request on github to report this, probably in the troubleshooting -> datasources section. Though a daily or weekly check for that kind of issue would probably not go amiss.