Hello All,
I thought I had gone through all of the recommended steps here: http://docs.cacti.net/manual:087:4_help ... #debugging and will summarize below.
Poller records in cacti.log appear to be correct
Code: Select all
<SNIP> = al/bin/rrdtool update /opt/local/share/httpd/htdocs/cacti/rra/rrd_file.rrd --template players:Slots:CA:
05/13/2013 04:05:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368461101:714:1730:255:184:227:48
05/13/2013 04:10:06 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368461400:710:1620:249:184:229:48
05/13/2013 04:15:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368461700:794:1730:264:237:226:67
05/13/2013 04:20:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368462001:710:1680:234:209:198:69
05/13/2013 04:25:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368462300:758:1730:236:223:222:77
05/13/2013 04:30:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368462600:771:1730:253:241:203:74
05/13/2013 04:35:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368462901:777:1730:259:243:195:80
05/13/2013 04:40:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368463201:778:1730:258:238:203:79
05/13/2013 04:45:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368463500:786:1730:260:236:208:82
05/13/2013 04:50:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368463801:781:1730:263:234:194:90
05/13/2013 04:55:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368464100:769:1730:261:237:178:93
05/13/2013 05:00:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368464401:768:1730:271:244:148:105
05/13/2013 05:05:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368464701:807:1730:284:252:164:107
05/13/2013 05:10:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368465000:851:1730:286:244:207:114
05/13/2013 05:15:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368465300:851:1730:290:228:232:101
05/13/2013 05:20:03 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368465600:858:1730:288:231:241:98
05/13/2013 05:25:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368465901:860:1730:269:242:244:105
05/13/2013 05:30:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368466200:867:1730:276:230:244:117
05/13/2013 05:35:03 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368466500:826:1670:223:237:244:122
05/13/2013 05:40:03 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368466800:869:1730:252:247:248:122
05/13/2013 05:45:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368467100:840:1680:253:224:245:118
05/13/2013 05:50:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368467400:859:1680:262:231:245:121
05/13/2013 05:55:03 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368467700:854:1680:263:214:245:132
05/13/2013 06:00:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368468000:844:1680:266:209:241:128
05/13/2013 06:05:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368468301:824:1680:249:216:225:134
05/13/2013 06:10:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368468600:787:1620:208:209:245:125
Code: Select all
bash-4.1$ php ./cacti.php
Players:838 Slots:1620 CA:287 EU:178 UK:212 US:161
Code: Select all
<SNIP> = extra stuff
<!-- 2013-05-13 16:40:00 UTC / 1368463200 --> <row><v>7.7800000000e+02</v> .. <SNIP> .. <v>2.5800000000e+02</v></row>
<!-- 2013-05-13 16:45:00 UTC / 1368463500 --> <row><v>7.8597333333e+02</v> .. <SNIP> .. <v>2.5999333333e+02</v></row>
<!-- 2013-05-13 16:50:00 UTC / 1368463800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2013-05-13 16:55:00 UTC / 1368464100 --> <row><v>7.6900000000e+02</v> .. <SNIP> .. <v>2.6100000000e+02</v></row>
<!-- 2013-05-13 17:00:00 UTC / 1368464400 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2013-05-13 17:05:00 UTC / 1368464700 --> <row><v>8.0700000000e+02</v> .. <SNIP> .. <v>2.8400000000e+02</v></row>
<!-- 2013-05-13 17:10:00 UTC / 1368465000 --> <row><v>8.5085333333e+02</v> .. <SNIP> .. <v>2.8599333333e+02</v></row>
<!-- 2013-05-13 17:15:00 UTC / 1368465300 --> <row><v>8.5100000000e+02</v> .. <SNIP> .. <v>2.9000000000e+02</v></row>
<!-- 2013-05-13 17:20:00 UTC / 1368465600 --> <row><v>8.5800000000e+02</v> .. <SNIP> .. <v>2.8800000000e+02</v></row>
<!-- 2013-05-13 17:25:00 UTC / 1368465900 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2013-05-13 17:30:00 UTC / 1368466200 --> <row><v>8.6700000000e+02</v> .. <SNIP> .. <v>2.7600000000e+02</v></row>
<!-- 2013-05-13 17:35:00 UTC / 1368466500 --> <row><v>8.2600000000e+02</v> .. <SNIP> .. <v>2.2300000000e+02</v></row>
<!-- 2013-05-13 17:40:00 UTC / 1368466800 --> <row><v>8.6900000000e+02</v> .. <SNIP> .. <v>2.5200000000e+02</v></row>
<!-- 2013-05-13 17:45:00 UTC / 1368467100 --> <row><v>8.4000000000e+02</v> .. <SNIP> .. <v>2.5300000000e+02</v></row>
<!-- 2013-05-13 17:50:00 UTC / 1368467400 --> <row><v>8.5900000000e+02</v> .. <SNIP> .. <v>2.6200000000e+02</v></row>
<!-- 2013-05-13 17:55:00 UTC / 1368467700 --> <row><v>8.5400000000e+02</v> .. <SNIP> .. <v>2.6300000000e+02</v></row>
<!-- 2013-05-13 18:00:00 UTC / 1368468000 --> <row><v>8.4400000000e+02</v> .. <SNIP> .. <v>2.6600000000e+02</v></row>
<!-- 2013-05-13 18:05:00 UTC / 1368468300 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
<!-- 2013-05-13 18:10:00 UTC / 1368468600 --> <row><v>7.8700000000e+02</v> .. <SNIP> .. <v>2.0800000000e+02</v></row>
<!-- 2013-05-13 18:15:00 UTC / 1368468900 --> <row><v>8.3500000000e+02</v> .. <SNIP> .. <v>2.5300000000e+02</v></row>
I don't think that it would help to post graphs, however I am generating two graphs from a single RRD. They have gaps in the same locations.<!-- 2013-05-13 16:50:00 UTC / 1368463800 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
05/13/2013 04:50:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368463801:781263:234:194:90
<!-- 2013-05-13 17:00:00 UTC / 1368464400 --> <row><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v><v>NaN</v></row>
05/13/2013 05:00:04 PM - POLLER: Poller[0] CACTI2RRD: /opt/loc … <SNIP> ...EU:UK:US 1368464401:768
etc
263:234:194:90I can't seem to let go of the coincidence of the timestamps differing by one for each NAN. this is consistent across the entire log file.
From what I can tell the custom script is run, collects the information, the information is registered in cacti.log, cacti.log shows that it is updating the RRD, however the RRD has NaNs.
Code: Select all
ds[players].index = 0
ds[players].type = "GAUGE"
ds[players].minimal_heartbeat = 300
ds[players].min = 0.0000000000e+00
ds[players].max = 2.0000000000e+03
ds[players].last_ds = "1004"
ds[players].value = 0.0000000000e+00
ds[players].unknown_sec = 0
ds[Slots].index = 1
ds[Slots].type = "GAUGE"
ds[Slots].minimal_heartbeat = 300
ds[Slots].min = 0.0000000000e+00
ds[Slots].max = 2.0000000000e+03
ds[Slots].last_ds = "1680"
ds[Slots].value = 0.0000000000e+00
ds[Slots].unknown_sec = 0
ds[EU].index = 2
ds[EU].type = "GAUGE"
ds[EU].minimal_heartbeat = 300
ds[EU].min = 0.0000000000e+00
ds[EU].max = 1.0000000000e+03
ds[EU].last_ds = "212"
ds[EU].value = 0.0000000000e+00
ds[EU].unknown_sec = 0
ds[US].index = 3
ds[US].type = "GAUGE"
ds[US].minimal_heartbeat = 300
ds[US].min = 0.0000000000e+00
ds[US].max = 1.0000000000e+03
ds[US].last_ds = "229"
ds[US].value = 0.0000000000e+00
ds[US].unknown_sec = 0
ds[UK].index = 4
ds[UK].type = "GAUGE"
ds[UK].minimal_heartbeat = 300
ds[UK].min = 0.0000000000e+00
ds[UK].max = 1.0000000000e+03
ds[UK].last_ds = "227"
ds[UK].value = 0.0000000000e+00
ds[UK].unknown_sec = 0
ds[CA].index = 5
ds[CA].type = "GAUGE"
ds[CA].minimal_heartbeat = 300
ds[CA].min = 0.0000000000e+00
ds[CA].max = 1.0000000000e+03
ds[CA].last_ds = "336"
ds[CA].value = 0.0000000000e+00
ds[CA].unknown_sec = 0
So after all of this, as I pasted the above, I realized that my minimal_heartbeat was set to 300, not 600. This should have been obvious to me with my many years of trouble-free use of rrdtool and cacti.
I am leaving this up here to preserve the fix for future befuddled users.
Code: Select all
# rrdtool tune rra/rrd_file.rrd -h UK:600
# rrdtool tune rra/rrd_file.rrd -h CA:600
# rrdtool tune rra/rrd_file.rrd -h US:600
# rrdtool tune rra/rrd_file.rrd -h EU:600
# rrdtool tune rra/rrd_file.rrd -h players:600
# rrdtool tune rra/rrd_file.rrd -h Slots:600
Code: Select all
ds[players].index = 0
ds[players].type = "GAUGE"
ds[players].minimal_heartbeat = 600
ds[players].min = 0.0000000000e+00
ds[players].max = 2.0000000000e+03
ds[players].last_ds = "987"
ds[players].value = NaN
ds[players].unknown_sec = 1
ds[Slots].index = 1
ds[Slots].type = "GAUGE"
ds[Slots].minimal_heartbeat = 600
ds[Slots].min = 0.0000000000e+00
ds[Slots].max = 2.0000000000e+03
ds[Slots].last_ds = "1680"
ds[Slots].value = NaN
ds[Slots].unknown_sec = 1
ds[EU].index = 2
ds[EU].type = "GAUGE"
ds[EU].minimal_heartbeat = 600
ds[EU].min = 0.0000000000e+00
ds[EU].max = 1.0000000000e+03
ds[EU].last_ds = "203"
ds[EU].value = NaN
ds[EU].unknown_sec = 1
ds[US].index = 3
ds[US].type = "GAUGE"
ds[US].minimal_heartbeat = 600
ds[US].min = 0.0000000000e+00
ds[US].max = 1.0000000000e+03
ds[US].last_ds = "253"
ds[US].value = NaN
ds[US].unknown_sec = 1
ds[UK].index = 4
ds[UK].type = "GAUGE"
ds[UK].minimal_heartbeat = 600
ds[UK].min = 0.0000000000e+00
ds[UK].max = 1.0000000000e+03
ds[UK].last_ds = "203"
ds[UK].value = NaN
ds[UK].unknown_sec = 1
ds[CA].index = 5
ds[CA].type = "GAUGE"
ds[CA].minimal_heartbeat = 600
ds[CA].min = 0.0000000000e+00
ds[CA].max = 1.0000000000e+03
ds[CA].last_ds = "328"
ds[CA].value = NaN
ds[CA].unknown_sec = 1