Speeding up cmd.php

[bulek]

Just in case spine (is this idea still alive?) will not get into 0.8 I have two ideas how to offload cmd.php a bit.

1. I noticed that cmp.php makes a lot of "joins" on cacti tables to identify fields of launched scripts, then to put gathered data inside and generate rrd files. This at least in my case slows down cmd.php considerably. My idea is redesign cacti to use separate table per Data Input defined. Such table with name of Data Input would have columns defined in the same way as input and output fields of Data Input. For example:

Code: Select all

mysql> describe Get_SNMP_Network_Data;
+-------------+-------------+------+-----+---------+-------+
| Field       | Type        | Null | Key | Default | Extra |
+-------------+-------------+------+-----+---------+-------+
| ID          | smallint(6) |      | PRI | 0       |       |
| address_ip  | varchar(50) | YES  |     | 0       |       |
| community   | varchar(50) | YES  |     | 0       |       |
| ifc_num     | smallint(6) | YES  |     | 0       |       |
| in          | int(11)     | YES  |     | 0       |       |
| out         | int(11)     | YES  |     | 0       |       |
+-------------+-------------+------+-----+---------+-------+

This way cacti quite easy can find parameter names and write collected data to proper columns.


2. I noticed that cmd.exe launches rrdtool per every DS. What about using pipelining feature of rrdtool? I mean cmd.php could popen() on rrdtool at the beginning then feed opened pipe with update requests and pclose() at the end. I'm not sure how much speed gain would be in this case.

- bulek

[raX]

You make some very valid points here. The good news is that our c-based poller should make it into 0.8, as seen here.

Of course I am going to still have to take this into consideration because:

1) Executing rrdtool commands and the speed of MySQL queries is even a larger bottleneck with such a multi-threaded application.

2) If all goes well, I still plan on including cmd.php with 0.8 just in case users can't get the poller working correctly or don't want to use it for some reason. Win32 users may be stuck with this option for a bit longer until we can come up with a win32 binary.

I must admit that I haven't thought of creating separate tables for each data input source, but it really does make sense logically. I will take this suggestion into consideration, but first I will have to sit down and think about how much less SQL will be required to make this happen.

Thanks for the interesting idea.

-Ian