Automatic discovery of items (like zabbix)

[Pentium100]

Hi, is there a way to make cacti automatically discover items in a host and create the appropriate graphs?

Longer version:
There are a few hypervisors running a lot of VMs. The VMs are created, deleted and moved between the hosts. I want to monitor the cpu usage etc of each VM (run a script, get data), however, with unmodified cacti, I would have to manually add each VM and create the graphs.
It looks like the autom8 plugin could be persuaded to create the graphs automatically on each reindex. However, it looks like it cannot remove graphs and I would have to remove graphs (in case the VM is moved to another host or deleted), but after some time, not instantly).

Zabbix can do this - query a script that returns a list of VMs, then add each item and create graphs. If a VM is no longer detected, delete the items etc after 30 days. However, cacti graphs are so much better

Anyone has done this with cacti?

[phalek]

EDIT: argh, i should have read the whole post ( you mentioned autom8 already )

[Pentium100]

You could use Autom8 and create some rules that does this.
http://docs.cacti.net/plugin:autom8

Reading the manual it seemed like Autom8 can only create graphs, not delete them. Will try to use it tomorrow.

[phalek]

The CLI scripts should be able to delete graphs, but then you will have some mechanism to be able to identify the ones that need to be deleted. I'd say that may require some additional scripting.

I know the hosts have a "down since" field, but I'm not aware of graphs, so you probably would have to analyze the RRD files for NANs and delete them based on these

[gandalf]

However, it looks like it cannot remove graphs and I would have to remove graphs (in case the VM is moved to another host or deleted), but after some time, not instantly).

If a VM is no longer detected, delete the items etc after 30 days. However, cacti graphs are so much better

You're quite right. AUTOM8 cannot delete right now. And that's on purpose - not because I can't code that.
The reason is, that I don't know a general "rule" that would be able to decide, that a data source has to be deleted. There are a lot of (corner) cases, where a "good" rule may fail.
E.g. how do you detect, that a data source is no longer "used"? When the "index" is no longer available? After which time? Configurable (per data source)? What is, when the index comes back?
What is, when the data source is no longer available because the host is down (e.g. due to a planned outage)?

A most flexible method would be to allow the user to configure sth like a "delete trigger"; e.g. a script that returns e.g. false when a data source has to be deleted. But that includes a lot of configuration labor.

And up to now, I was avoiding users who loose all their data due to a misconfiguration or misunderstanding.

R.

[Pentium100]

The reason is, that I don't know a general "rule" that would be able to decide, that a data source has to be deleted. There are a lot of (corner) cases, where a "good" rule may fail.
E.g. how do you detect, that a data source is no longer "used"? When the "index" is no longer available? After which time? Configurable (per data source)? What is, when the index comes back?
What is, when the data source is no longer available because the host is down (e.g. due to a planned outage)?

The way zabbix works is like this:
The discovery rule runs once in a while if the host is up. Any new items that are found get added, any items that are no longer found get marked as "not found" and deleted after x days (configurable per rule). If the item comes back up after a couple of months (longer than the configured time) the data source gets created again.

While the removal of items is not that useful for switch interfaces (rarely you can add more ports to a switch), it is useful for partitions (plug, unplug hard drives), and what I want to use it for - VMs in a cloud. They get created, deleted and moved between hosts. The data sources should be deleted when the VM is no longer running in that host (if it moved to another host, create a new DS - it would too much coding to move the data source).

[gandalf]

The way zabbix works is like this:
The discovery rule runs once in a while if the host is up. Any new items that are found get added, any items that are no longer found get marked as "not found"

The correct trigger for this would be a reindex event in Cacti. This is, where AUTOM8 is triggered today.

and deleted after x days (configurable per rule).

Hmm. So we would have to mark data sources as "eligible for deletion" and add some timestamp to that (e.g. based on some rules that may be related to some attributes of a data source). Of course, it is preferred to view this on the data source interface and create a filter to view those objects marked for deletion.

Still we might argue, whether the poller then should have an automatic call to _really_ delete data sources or if we only provide e.g. a script which the user him/herself has to put into crontab - just to blame him/her in case of trouble

This way, the INDEX is the object we would pay attention to. This is of course easier than scanning each rrd file for valid data or for absence of this.

R.

[Pentium100]

I installed Autom8, but there is a problem with rule creation - If I click "Add" for rule items - eligible hosts, it opens a blank frame.
The link, however uses urlencoding -
plugins/autom8/autom8_graph_rules.php?action=item_edit&id=3&rule_type=1

If I change those & to "&" it works normally.

[marthin]

Hmm. So we would have to mark data sources as "eligible for deletion" and add some timestamp to that (e.g. based on some rules that may be related to some attributes of a data source). Of course, it is preferred to view this on the data source interface and create a filter to view those objects marked for deletion.

Still we might argue, whether the poller then should have an automatic call to _really_ delete data sources or if we only provide e.g. a script which the user him/herself has to put into crontab - just to blame him/her in case of trouble

This way, the INDEX is the object we would pay attention to. This is of course easier than scanning each rrd file for valid data or for absence of this.

R.

I'm curious Gandalf, is something like in planning for autom8? Would be great feature if we can vote for it somewhere.

rgds