Patch: "Structured RRD Path"

Post by **browniebraun** » Tue Jul 25, 2006 8:46 am

Hi CACTI community!

One disadvantage of the Cacti file structure is the flat file directory RRA, especially in great enviroments. For example we have 30.000 rrd's in one folder.
To overcome this, we decided to structure the directory in host_id/local_data_id.
A second issue was the naming of the rrd's with devicename_data_source_local_data_id when a hostname changed. The only needed identifier is the local_data_id. For human readability we plan a script to reverse lookup local_data_id to devicename.

Old structure of RRA folder:

/RRA/
......../labrt01_traffic_in_194.rrd

New structure of RRA folder:

/RRA/
..../$host_id/
................../$local_data_id.rrd

The patch is designed to give you the chance to switch between the old, flat file organization and a structured form with subfolders.
The screenshot displays the modified path setting checkbox.
The script (extended_RRA.php) make it possible to migrate a existing Cacti installation with all rrd files and database entries.
It's designed to run only from command line.

It's my first shot with PHP, so please be patient with me, and give me as much feedback as you can.

Please notice:
USE THIS PATCH AT YOUR OWN RISK and
PLEASE
READ THE README.TXT BEFORE

Best regards
BrownieBraun

Post by **rony** » Tue Jul 25, 2006 9:15 am

Um...

Post by **gandalf** » Tue Jul 25, 2006 12:47 pm

Hey, not that bad ...
Lately, there were some requests for excatls this feature. Hope, they'll find this bit
Reinhard

Post by **TheWitness** » Tue Jul 25, 2006 9:12 pm

We had planned to forstall this feature till 0.9. But it's been on the request list for it seams like decades. Maybe we should do something about it.

TheWitness

Post by **TheWitness** » Tue Jul 25, 2006 9:14 pm

Oh, BTB, this is a core function, therefore, I would have difficulty supporting this unless it was a part of the "base" product. It is one clear feature that could make it difficult to support future upgrades for those who choose to implement it.

So, if you are reading this, implement at your own risk. I would hold off for a bit.

TheWitness

fmangeant · Post by **fmangeant** » Wed Jul 26, 2006 9:06 am

TheWitness wrote:We had planned to forstall this feature till 0.9. But it's been on the request list for it seams like decades. Maybe we should do something about it.

Not decades

, but I've submitted a bug nearly 2 years ago : http://bugs.cacti.net/view.php?id=284

Believe me or not, I've changed <path_rra> for all of my ~ 10k data sources by hand...
And I still love Cacti !

Post by **browniebraun** » Wed Jul 26, 2006 10:44 am

Windows formatted readme file:

zheka · Post by **zheka** » Fri Jul 28, 2006 11:56 am

Hi there,

Recently bumped into this post. Great and awesome feature. I needed it too. It works with me, BUT I am always very picky. Is it possible to have the name of the host instead of the host ID in the path ../name/graph_xxx.rrd?
Structured RRA increments host id numbers for every added host. After adding more than dozen hosts you will forget which folder is which.

Eugene

Post by **TheWitness** » Fri Jul 28, 2006 1:22 pm

When we roll this into production, we will offer a drop down that let's you pick your method. Then, all hosts/graphs created after that point will be locked in. If you change in the middle though I do not propose modifying RRA file names from the user interface.

Ian has been against doing these types of things from the web interface due to security issues. Aka, read access is ok from the web site, but write is not.

This is one of the reasons we came up with the poller_commands table. It's intention is to perform various tasks after the poller has completed.

Regards,

TheWitness

zheka · Post by **zheka** » Fri Jul 28, 2006 1:32 pm

Thanks, it is the answer from the real PRO indeed.
I will try to experiment with it anyway. I need it in production too!
Are there any hardware requirements for Cacti host that will graph and monitor about 50 hosts? I think my poor AMD 500 with 512 MB of RAM is not strong enough.

Post by **browniebraun** » Mon Jul 31, 2006 6:52 am

Hi CACTI community!

Last days I preferred to keep silent and to watch the trend of this discussion.

In my opinion it would be great to integrate a function in install.php,
so you can make a decision which RRD structure you want to use after you have installed cacti.
I agree with TheWitness's idea to lock it after you have picked your method. As you can
read it in the readme file I can not provide a mixed structure.

Using the poller_commands table would close the little security issue i have opened with this patch.

zheka: Please notice. In a great enviroment it's possible to have routers with the same name but different domains.
So if you change the name of the folder from $host_id to its name, you will lose the singleness which is needed.
The name of a router must not be forever. So what would you do if its name has been changed?
By using this structure ($host_id\local_data_id) in cacti you will only have to edit the description of the device under
"devices". Thats all!

A reverse lookup would be much more better in this case.
Here i'll see an synergy with "rrdcleaner" plugin.

Best regards
BrownieBraun

estair · Post by **estair** » Tue Aug 22, 2006 1:17 pm

This is functional, and makes the device:file naming MUCH more elegant to work with behind the scenes (calling rrdtool from other apps). Is there any intent to include this work in Cacti? Is there any reason not to? IMO, this makes more sense than naming the RRD after sysName, and (potentially) having to change it. The ID can be pulled from the database easily, and will always* match the filename.

I for one definitely would like to see this in the next production release.

/eli

Post by **rony** » Tue Aug 22, 2006 1:24 pm

estair wrote:This is functional, and makes the device:file naming MUCH more elegant to work with behind the scenes (calling rrdtool from other apps). Is there any intent to include this work in Cacti? Is there any reason not to? IMO, this makes more sense than naming the RRD after sysName, and (potentially) having to change it. The ID can be pulled from the database easily, and will always* match the filename.

I for one definitely would like to see this in the next production release.

/eli

One issue, who says the device is SNMP enabled, or even has sysName set.

estair · Post by **estair** » Tue Aug 22, 2006 1:35 pm

I considered clarifying, but was too distracted... so thanks

I don't necessarily mean "SNMP sysName", moreso that the host->description in the database itself is less valid as a unique identifier than the 'id' row.

/eli

Post by **rony** » Tue Aug 22, 2006 1:37 pm

id is the only unique name that doesn't change with user input.

So, that being said, it will probably be used as the host directory name.

Cacti

Patch: "Structured RRD Path"

Patch: "Structured RRD Path"

Inclusion of this patch in 0.86i?

Re: Inclusion of this patch in 0.86i?

Terminology correction :)

Who is online