cacti, collectd and externally updated remote rrds

Anything that you think should be in Cacti.

Moderators: Developers, Moderators

Post Reply
User avatar
psyber
Cacti User
Posts: 84
Joined: Tue Aug 02, 2005 7:18 pm

cacti, collectd and externally updated remote rrds

Post by psyber »

I'm really behind the curve on collectd but after reading the docs I'm excited.
My very first thoughts were on how to display that data in Cacti. Then I found this post: http://forums.cacti.net/about28169.html
(Which BTW I would love to see something along these lines supported in cacti itself maybe a collectd plugin AND some way to better create ds and graphs for externally updated rrds)

I know there was discussion on one of the developer forums about designing a distributed cacti, and I make extensive use of externally updated rrds. I've played with rrd server some and now I'm wondering if I could make use of externally updated rrds that exist on remote servers. From a scalability standpoint where the number of data sources is your limiting factor, spreading the DSs across multiple servers makes a lot of sense. I think the rrd file location could be hacked in such a way if you passed it hostname:13900/path/to/rrd/ cacti could pass its graph commands through the socket to the remote host and have the img returned for display.

So the model I'm building in my head is collectd --->collectd server running an rrdserver <-----cacti connecting to the rrd server and passsing a graph command--->image from collectd server back to cacti for display...
This is all completely untested, I haven't even looked at the code yet, and quite possibly the network latency may kill the end user experience which would make this all a moot point. But I'm quite curious if anyone out there has tried something like this and what their experiences might be.

Thoughts? Comments? Lost my mind?
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Post by gandalf »

We are thinking about a "mini Cacti", including spine, reading a writing to a central MySQL. That comes close to your structure. But there are currently no plans to support different pollers.
R.
heybigben
Posts: 9
Joined: Tue Nov 24, 2009 12:16 am

cacti and collectd

Post by heybigben »

I also really like the idea of combining collectd and cacti. I like collectd because it includes a push model where we can have have a large number of compute hosts all reporting data over tcp channel to a central server where the data is logged to flat files via csv or directly to rrd. Also the config file is really easy to customize. I really like cacti for the tree views, and multiple methods of viewing data (list, preview, etc.) and the web access controls and realms.

So I had a slightly different approach to combine the two programs together. Use cacti to poll the collectd daemon for data via the collectd cli (or direct socket in future release) or have cacti use a custom data input method to grab the data from the cvs file. Then just have a script run in cron which queries collectd and determines the hostnames and then auto populates cacti with the device entries (if there missing). There probably a way to do this better as a cacti plugin but I haven't figured that out yet. By using the cacti poller to get data from collectd, it avoids the external update method - which doesn't seem to work with realtime or thold.
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

Quietly stating that "ganglia" is the right information propagation methodology, but very quietly.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
heybigben
Posts: 9
Joined: Tue Nov 24, 2009 12:16 am

Post by heybigben »

Both ganglia and collectd are great tools for cluster monitoring. I prefer collectd since it has more input and output options (i.e. cvs, rrd, etc.). But I would still really like to use cacti for the frontend and let those other tools gather the data. Much easier then snmp.
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

I would say plugin. The real issue that the former solves is the whole concept of aggregation/association. Stay well.
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests