Nodes showing offline intermittently

[SSami]

Hi,

We are facing an issue with our Cacti that it keeps showing some nodes offline (gray arrows) even though the devices are marked as up and we are able to ping them. we have tried increasing polling interval , increasing ping timeout value for these nodes, removing those nodes and adding them again , clearing the statistics for them and eventually restarting the service but with no avail.

These nodes sometimes appear online (green arrow) and then it shows again after some time offline.

Any suggestions?

[Rno]

I have the same issue but only with snmp, I got error that the device is not responding to snmp but ping is ok !!
And snmp timeout is over 10 seconds !! But that won't solve the problem

[Howie]

Grey arrows in weathermap?

What does your scale look like?

Weathermap doesn't show anything as up or down - it just takes a number and turns it into a colour. If the scale (which does the translation) has gaps, or is incorrect for your interpretation of the data, then you might get grey arrows. The most common problem like this is where people have a scale like

Code: Select all

SCALE myscale 0 1   255 0 0
SCALE myscale 1 2   0 255 0

or

Code: Select all

SCALE myscale 1 1   255 0 0
SCALE myscale 2 2   0 255 0

and then get a value of 1.5. If you are storing things like interface status in rrd files, then you sometimes get values in between 1 and 2, even if you think that you can only get 1 or 2 back from the device, depending on how you have set up the rrd parameters. It will produce an average if those aren't correct.

The easiest fix is:

Code: Select all

SCALE myscale 0.5 1.5   255 0 0
SCALE myscale 1.5 2.5   0 255 0

Now there are no gaps. The actual state might still be wrong though - if the average is 1.7 because the state has only recently changed from 2 to 1, the map will show an incorrect state. So the real answer is to fix the rrd parameters if this is what the problem is.

[SSami]

Hi Howie,

Thanks for your reply, much appreciated. Basically we are monitoring traffic on our devices. if any of them stopped passing traffic,the link (or arrow) would turn from green to grey on the weathermap and would mark the device as down in the devices list.

The issue we are facing that even though device is marked up and its passing traffic and we are able to reach it from cacti , its still showing grey link as if its not passing traffic.

I have attached sample of the logs we are getting for one of the devices we are facing this issue with.

PS: we have tried rebuilding the rrd file manually but with no avail.

Also kindly can you please help in the location of where to check the scale from? As I'm not aware of.

Thanks,

[Howie]

The scale is defined in the top of the weathermap map config file.

If you are getting 'no valid data' though, it's probably not that.

At the time that you get that error in your logs, please check the last-modified date on that specific file.