how does cacti determine the maximum value for a COUNTER32?

[jlg]

hi folks, i have a snmpd that gives a value like this:

UCD-DISKIO-MIB::diskIONRead.24 = Counter32: 45797520384

note: 45,797,520,384 is greater than 32 bits can store.

currently, the corresponding rra has no data and i'm assuming it's because of the overflow. can this be fixed by changing 32 -> 64?

UCD-DISKIO-MIB::diskIONRead.24 = Counter64: 45797520384

or any other suggestions why the rra has no data?

thanks,
jlg


ps: for anyone interested, this is from net-snmp 5.1.2 with ucd-snmp/diskio support. it's built on redhat linux where sizeof(long) == 8 so it does not internally overflow the snmpd daemon. also, the corresponding linux data sources (/proc/diskstats) return long values for sectors which then get converted to bytes in the deamon multiplying by 512. so this returned value will overflow 32bits very quickly...well before the source linux counter overflows.

[BSOD2600]

Try stepping through this http://docs.cacti.net/node/283.

I'd also set that to a 64bit counter since thats apparently truly what it is.

[gandalf]

hi folks, i have a snmpd that gives a value like this:

UCD-DISKIO-MIB::diskIONRead.24 = Counter32: 45797520384

note: 45,797,520,384 is greater than 32 bits can store.

currently, the corresponding rra has no data and i'm assuming it's because of the overflow. can this be fixed by changing 32 -> 64?

UCD-DISKIO-MIB::diskIONRead.24 = Counter64: 45797520384

or any other suggestions why the rra has no data?

thanks,
jlg


ps: for anyone interested, this is from net-snmp 5.1.2 with ucd-snmp/diskio support. it's built on redhat linux where sizeof(long) == 8 so it does not internally overflow the snmpd daemon. also, the corresponding linux data sources (/proc/diskstats) return long values for sectors which then get converted to bytes in the deamon multiplying by 512. so this returned value will overflow 32bits very quickly...well before the source linux counter overflows.

You may need to tweak the data sources MAXIMUM values
- in the Data Template
- in all existing rrd files using "rrdtool tune --maximum"
Reinhard

[jlg]

You may need to tweak the data sources MAXIMUM values
- in the Data Template
- in all existing rrd files using "rrdtool tune --maximum"
Reinhard

hi reinhard, i confirmed that the MAXIMUM values should not be a problem. in the Data Template, "Maximum Value" == 0. and from the rrd file, we have:

Code: Select all

ds[BytesWritten].type = "COUNTER"
ds[BytesWritten].minimal_heartbeat = 600
ds[BytesWritten].min = 0.0000000000e+00
ds[BytesWritten].max = NaN
ds[BytesWritten].last_ds = "0"
ds[BytesWritten].value = 0.0000000000e+00
ds[BytesWritten].unknown_sec = 0

jlg

[jlg]

further analysis:

i can see from the dump, that i the rrd file is being updated. this is a snippet from a dump:

Code: Select all

<!-- 2006-12-27 23:25:00 PST / 1167290700 --> <row><v> 0.0000000000e+00 </v><v> 0.0000000000e+00 </v></row>
<!-- 2006-12-27 23:30:00 PST / 1167291000 --> <row><v> 1.7009587514e+01 </v><v> 0.0000000000e+00 </v></row>
<!-- 2006-12-27 23:35:00 PST / 1167291300 --> <row><v> 1.4221414128e+07 </v><v> 0.0000000000e+00 </v></row>
<!-- 2006-12-27 23:40:00 PST / 1167291600 --> <row><v> 9.5126515526e+04 </v><v> 0.0000000000e+00 </v></row>
<!-- 2006-12-27 23:45:00 PST / 1167291900 --> <row><v> 0.0000000000e+00 </v><v> 0.0000000000e+00 </v></row>

i wrote a cron job to capture the raw snmpget data which results in these data points:

Code: Select all

Wed Dec 27 23:20:01 PST 2006
   8    5 sda5 1427338 20534140 93768689 750149456
UCD-DISKIO-MIB::diskIONWritten.23 = Counter32: 384,076,726,272  

Wed Dec 27 23:25:01 PST 2006
   8    5 sda5 1427345 20534196 93770122 750160920
UCD-DISKIO-MIB::diskIONWritten.23 = Counter32: 384,082,391,040

Wed Dec 27 23:30:01 PST 2006
   8    5 sda5 1427350 20534236 93771377 750170960
UCD-DISKIO-MIB::diskIONWritten.23 = Counter32: 384,087,658,496

Wed Dec 27 23:35:01 PST 2006
   8    5 sda5 1427354 20534268 93772634 750181016
UCD-DISKIO-MIB::diskIONWritten.23 = Counter32: 384,092,762,112

Wed Dec 27 23:40:01 PST 2006
   8    5 sda5 1427359 20534308 93774060 750192424
UCD-DISKIO-MIB::diskIONWritten.23 = Counter32: 384,098,639,872

Wed Dec 27 23:45:01 PST 2006
   8    5 sda5 1427363 20534340 93775337 750202640
UCD-DISKIO-MIB::diskIONWritten.23 = Counter32: 384,103,833,600

the "sda5" line is from /proc/diskstats directly. the last value (ie, 750149456 in the 1st data point above) is the number of sectors written. the daemon multiplies this by 512 to get the final result: 384,076,726,272. (actually, this is slightly off by 400 sectors due to this being 2 separate reads of /proc/diskstats.) also, i added the commas for readability.


so notice that the rrd dump has a value of "0" for the period ending 23:25 even though the underlying counter went from 384,076,726,272 to 384,082,391,040. again, these values overflow 32 bits. however, the next period ending 23:30 also overflows 32 bits...yet there is a value of 17 bytes per second (1.7009587514e+01) which is not "0".

so again, since i cannot find anywhere in cacti where i can specify COUNTER32 vs COUNTER64 (the data template only lets me specify COUNTER), i'm guessing that cacti is getting the 32bit vs 64bit from the snmp data. i was hoping someone might know for a fact.

regardless, i'll try to get the daemon to say COUNTER64 or else have the data truncated to 32bits.

thanks,
jlg

[gandalf]

RRDTool does not care for COUNTER32/64. But the mysql update may perhaps fail due to size limitations. Please check this. To do so, capture the mysql insert statement fpr poller_output and run this manually to check for errors
Reinhard

[jlg]

RRDTool does not care for COUNTER32/64. But the mysql update may perhaps fail due to size limitations. Please check this. To do so, capture the mysql insert statement fpr poller_output and run this manually to check for errors
Reinhard

hi Reinhard, do you have a link that describes how to capture the mysql update? i searched for "capture poller output" and did not find any obvious relevant thread. and i did select * from poller_output and found the table to be empty.

i would be glad to help debug this...especially since any of the COUNTER64 snmp OIDs can theoretically give values much higher than 45797520384. a size limitation here would definitely mean that any COUNTER64 OID would be ungraphable.

thanks,
jlg

[gandalf]

Please read http://docs.cacti.net/node/283
Reinhard