I'm facing a strange issue. Hope you would be able to help me.
I have many devices from different vendors monitored by a cacti instance.
Its version now is 1.0.4 (current on EPEL repo), but the issue started before upgrade from 0.8.8h.
Well, the problem is that one interface graph from a Juniper device gets huge spikes.
Interface is 1G and spikes are over that (even almost 2G) which is not possible of course.
It just started one day and I can't get over it. Firmware upgrade and cacti upgrade didn't help.
Graph looks like this:
And here are some info:
Code: Select all
$ rrdtool info /usr/share/cacti/rra/home_traffic_in_178.rrd
filename = "/usr/share/cacti/rra/home_traffic_in_178.rrd"
rrd_version = "0003"
step = 300
last_update = 1490461505
header_size = 2912
ds[traffic_in].index = 0
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 600
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 1.0000000000e+09
ds[traffic_in].last_ds = "13965310746092"
ds[traffic_in].value = 1.9607700997e+05
ds[traffic_in].unknown_sec = 0
ds[traffic_out].index = 1
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 600
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 1.0000000000e+09
ds[traffic_out].last_ds = "14903740407"
ds[traffic_out].value = 1.6769983389e+04
ds[traffic_out].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].cur_row = 547
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[1].cur_row = 406
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 3.9564737057e+04
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[1].cdp_prep[1].value = 3.3573851278e+03
rra[1].cdp_prep[1].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 775
rra[2].cur_row = 395
rra[2].pdp_per_row = 24
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 5.2538968253e+08
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[2].cdp_prep[1].value = 4.5391716235e+04
rra[2].cdp_prep[1].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[3].cur_row = 323
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 3.5281730304e+09
rra[3].cdp_prep[0].unknown_datapoints = 0
rra[3].cdp_prep[1].value = 6.4514119845e+06
rra[3].cdp_prep[1].unknown_datapoints = 0
rra[4].cf = "MAX"
rra[4].rows = 600
rra[4].cur_row = 563
rra[4].pdp_per_row = 1
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[4].cdp_prep[1].value = NaN
rra[4].cdp_prep[1].unknown_datapoints = 0
rra[5].cf = "MAX"
rra[5].rows = 700
rra[5].cur_row = 381
rra[5].pdp_per_row = 6
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = 3.9564737057e+04
rra[5].cdp_prep[0].unknown_datapoints = 0
rra[5].cdp_prep[1].value = 3.3573851278e+03
rra[5].cdp_prep[1].unknown_datapoints = 0
rra[6].cf = "MAX"
rra[6].rows = 775
rra[6].cur_row = 403
rra[6].pdp_per_row = 24
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = 2.5823713796e+08
rra[6].cdp_prep[0].unknown_datapoints = 0
rra[6].cdp_prep[1].value = 4.9757264444e+03
rra[6].cdp_prep[1].unknown_datapoints = 0
rra[7].cf = "MAX"
rra[7].rows = 797
rra[7].cur_row = 430
rra[7].pdp_per_row = 288
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = 2.5823713796e+08
rra[7].cdp_prep[0].unknown_datapoints = 0
rra[7].cdp_prep[1].value = 4.6186131804e+05
rra[7].cdp_prep[1].unknown_datapoints = 0
Code: Select all
1490450400: 7.0202643889e+03 1.5334180000e+03
1490450700: 6.8609833889e+03 1.6165467778e+03
1490451000: 6.6286807778e+03 1.5610220000e+03
1490451300: 2.5816976864e+08 1.5650064071e+03
1490451600: 3.5075793019e+06 1.5871864532e+03
1490451900: 5.7331677219e+03 1.5498839731e+03
1490452200: 7.5436987991e+03 1.6331070654e+03
1490452500: 7.6024328342e+03 1.5708110457e+03
1490452800: 6.9486104889e+03 1.5729626222e+03
1490453100: 6.7613938769e+03 1.5272167443e+03
1490453400: 6.9509365565e+03 1.6331769112e+03
1490453700: 7.6367291979e+03 1.5372353410e+03
1490454000: 7.2592671360e+03 1.5731140269e+03
1490454300: 6.9824493883e+03 1.5697194655e+03
1490454600: 7.3597241667e+03 1.5804372222e+03
1490454900: 6.8334299444e+03 1.5282476667e+03
1490455200: 7.3085343333e+03 1.6148406111e+03
1490455500: 6.7349803333e+03 1.5467328333e+03
1490455800: 7.2231488333e+03 1.5734636111e+03
1490456100: 6.4095462778e+03 1.5701246111e+03
1490456400: 2.5742901659e+08 1.6121073333e+03
1490456700: 4.3875944834e+06 2.5299699133e+05
1490457000: 2.0134049833e+04 5.6408828833e+04
1490457300: 1.2051571188e+05 2.4622759930e+05
1490457600: 1.7450038750e+05 9.0570346902e+03
1490457900: 2.5785085011e+08 3.4017467350e+03
1490458200: 4.4050885086e+06 2.7511295556e+03
1490458500: 7.7132377278e+04 4.9757264444e+03
1490458800: 1.0938497517e+05 4.7375096111e+03
1490459100: 4.0979896778e+04 2.8908860556e+03
1490459400: 4.4685819833e+04 3.1051066667e+03
1490459700: 9.6206846111e+03 2.1584849444e+03
1490460000: 2.5823713796e+08 4.2481816111e+03
1490460300: 4.4308790722e+06 3.1750338889e+03
1490460600: 3.8479904056e+04 3.8727655000e+03
1490460900: 4.0871269167e+04 3.1180148889e+03
1490461200: 6.5007214854e+04 3.5997452053e+03
1490461500: 3.9564737057e+04 3.3573851278e+03
Any ideas on that?