Monitoring devices for changes

[adrianmarsh]

One trick I used to do with MRTG, was to re-create the config file (with cfgmaker, filtering by interface type) y way of a .bat file and a schedule task. I did this so that if a new virtual ATM interface was added, then it would automatically be included in the graphing results.

I can't see a way of doing this in Cacti yet, hence the feature request. It would be really good to mark a device as "watches" and setup an auto-add filter.

For now, I'll just have to have the team add graphs manually whenever we create new interfaces....

Unless someone else knows otherwise??

[Exo7]

Here is an idea :

You could setup your routers so they send an snmp trap if their configurations change. When you receive such a trap, launch a script than connect to the router, retrieve the configuration and store it in a cvs / svn repository.
I have done this with the help of subversion, netsnmp, snmptt and a perl script using Net::Telnet. I later discovered an open source project which is exactly for this kind of work : Rancid.

If a change is detected, you could also run a shell script that update Cacti, like you have for MRTG.
The downside is : there is no command line tool I am aware of that make you able to add datasources and graph to Cacti cleanly. And I think it is a much needed feature, perhaps planned for the next 0.9 release.

[adrianmarsh]

Thanks,

Wish I'd seen Rancid before I wrote my own... I'm not quite sure how to link it all into Cacti though. I'll take a look at some point though.

[TheWitness]

Mechanizing Cacti is not too difficult if you understand the mechanics. The are you want to review is host.php. The automation functions are not in quite yet. Need also to create a rule engine for determining what interfaces and what states needed to be achieved by each in order to decide when to create.

Here is a question though. What happens when you delete an interface, or turn down a port? What do you do with those graphs?

TheWitness

[Linegod]

I've been trying to wrap my head around this for quite a while. It might be possible to meet in the middle, with a 'configuration changed' indicator.

By creating a cronjob (which could run nightly/weekly so as not to overload the box) based on host.php?action=query_reload that does not update the graph tables, but stores the information in a separate table, it could be compared against what is currently being graphed, and a status of 'Changed' could appear in host.php in the Devices view. This initially leaves it up to the user to react or not to the change - and could then be templated somewhere down the road.

This of course breaks right away, since there is no way to determine which graph template (bits, bytes, 64bit, packets) is being used without a major change to the current graph templates

This is why I rely on syslog, snmp-traps, external audit scripts and good change management

[adrianmarsh]

TheWitness,

To answer your question about removed interfaces: I used to create the rrdfiles by interface name. For my foundries that was easy, as it names by slot_port_virtualid, so the rrd files matched. If we removed an interface, that rrd file and graph simply stops getting updated, and on the index page it simply disappears. If we re-add that interfac later on then the graph re-appears and the rrd file continues to get updated..

I've never programmed in php before, so I'll take a look at host.php, but I won't hold my breath for figuring it out.

I've now halted running MRTG and Catci at the same time, in favour of Cacti, as I realised it could corrupt the results (2 systems polling) (?? - not sure)

[TheWitness]

I received a note from a student in India who may take this on as a student project. We'll see.

TheWitness

[adrianmarsh]

I took another look at this again this week. I think i've solved my major problem by changing the index refresh method to "index count changed". This at least should stop the polling errors (that stop the device polling properly) on interfaces that haven't changed, but it won't pick up on new interfaces automatically (etc).

Its also had the effect if increasing my poll-run-time from 30s to 60-70s, and that was only by adding 3 Foundry BigIrons...

[TheWitness]

I never did receive any feedback from that student in India. So be it. Cactid should help with slow devices, but right now I am concerned about some outstanding stability issues with the version that helps.

TheWitness

[Hiney]

Perhaps this is a policy solution and not a technological one (OMG, i'm sounding like my boss).

Someone mentioned it before, good change mangement. Have it as part of your change management to alter cacti (add, delete or alter) when something is changed (interface, server nic, whole server, etc).

Use a private tool or rancid to alert for changes by all means, but leave the cacti change to being done by hand.

Question, does the code for checking the amount of drives, or interfaces (or any iterative query) allow for not adding an item that is already in the database, or will duplicates be seen. EG: If an extra interface is added to a router, what will be seen when you go to "*Create Graphs for this Host" in the router's device configuration ? Will only that new interface be available to be added ?
I ask as this might be a bug, or it could already working this way.


H

[adrianmarsh]

The only issue with "change management" is that then you'd have to show people how to work with Cacti... entailing an understanding of SNMP, how the MIBs relate to the graphs, the graphs to templates and the rest of it..

Whereas all they are interested in, is plugging the cable into the switch.

My original point here was that way back in '05 when I started with MRTG, I found that I could automated the process quite easily, and it seemed to be missing from Cacti - it would still be useful I think..

[Hiney]

I wonder then if we can find the code for that runs the queries and run that from a cron job ?

H