Automation of data source and graph creation

[Burvil]

Update: clarified data source creation part.

I always follow the steps on the initial login page to create graphs, and they work. However, I think there's some things I'm missing.

* Create devices for network
=====================
This is simple enough.


* Create graphs for your new devices
=====================
A couple things here.

1) After creating the host, there are no data sources for that host, and thus nothing to graph. Shouldn't these automatically be created? I usually have to:

- Create the data source for each graph (i.e. on the left hand menu, choose Management -> Data Sources -> and type in the appropriate information),
- select debug mode, then
- cut and paste the rrdtool output into the command line, and run it.

Otherwise, the rrd file simply isn't there in /var/www/html/cacti/rra.

This is the case whether I'm just using the standard templates that come with cacti, or ones from this forum. Why aren't the datasources getting created?

2) If a graph template is associated with a given host, why not automatically create the graph when the device is created? For example, if I create a device that's a linux host, why can't cacti just automatically create the graphs for load, etc. for me, instead of me doing it myself? I mean, I usually just pick the defaults when creating the graph anyway.

* View your new graphs
=====================
Again, why do I have to manually assign graphs in a given host to the default tree?

Sorry if it seems like I'm complaining. But when you have a couple hundred systems and have a million other things on your plate, doing stuff like this really adds up, and it makes me not want to do use cacti.

That said, I do think the output cacti creates out of the box is a lot better than a lot of monitoring apps out there, even such commercial apps like HP Openview or Sun Management Center. I have limited experience with the former, but I took a class at Sun on the latter, and the graphs don't compare. They just take time to put together; time that I don't have.

[billygoat32]

I have found that I have to manually create the graphs also. I installed Cacti using the directions posted here.

http://www.debianhelp.co.uk/cactifromsource.htm

I am running Debian testing PHP 4.4.2-1.1 mysql 5.0.24a and have applied all of the patches to Cacti. Any info with this problem is appreciated.

[Balint]

Hi!

After spending 2 days with cacti, I second billygoat's opinion. I would also like to add that after changing a Host Template it should update all existing devices assigned with the particular template.


Greets,
B

[cigamit]

Um.. maybe I'm just being dense at 1 in the morning, but why not assign a Host Template to the device, and then click the big lettered "Create Graphs for this Host" link at the top of the device window?

Also, I have a "bulk" adder that auto assignes host templates to a device based on the Sysdescription, then it creates all graphs based upon that host template, and then auto-inserts it into a defined place on the tree. Its not out yet (I want lots more options), but it should be released in the next month.

[Burvil]

why not assign a Host Template to the device, and then click the big lettered "Create Graphs for this Host" link at the top of the device window?

Yup, I tried that. If I do that, the graphs give an error message in debug mode, saying there's no data source. Creating the data source manually fixes the problem in some cases, i.e. creates a valid graph, like load average, %CPU used, etc., but not others, like interface traffic.

[cigamit]

Sorry, if I'm asking some simple questions, but without more info to go on, its difficult to diagnose. I assume you waiting until after 1 polling, since during the polling is when everything gets created. If its not creating the datasource during polling, then that is a whole different issue. I quick dump of your debug log should show us what is happening then.

[billygoat32]

This is how I got it to not work.

I Clicked devices, selected 'bh1' clicked "create graphs for host " selected the graphs I wanted. They were not created. I go to the Manage graphs and turn on debug and this is what i see.

Code: Select all

 RRDTool Command:

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="BH1 - Volume Information - VOL1" \
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="bytes" \
--slope-mode \
DEF:a="/usr/share/cacti/site/rra/bh1_vol_total_22.rrd":vol_total:AVERAGE \
DEF:b="/usr/share/cacti/site/rra/bh1_vol_total_22.rrd":vol_free:AVERAGE \
DEF:c="/usr/share/cacti/site/rra/bh1_vol_total_22.rrd":vol_freeable:AVERAGE \
CDEF:cdefa=a,1024,* \
CDEF:cdefe=b,1024,* \
CDEF:cdefi=c,1024,* \
AREA:cdefa#00004D:"Total Space"  \
GPRINT:cdefa:LAST:"   Current\:%8.2lf %s"  \
GPRINT:cdefa:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefa:MAX:"Maximum\:%8.2lf %s\n"  \
AREA:cdefe#FFF200:"Free Space"  \
GPRINT:cdefe:LAST:"    Current\:%8.2lf %s"  \
GPRINT:cdefe:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefe:MAX:"Maximum\:%8.2lf %s\n"  \
LINE2:cdefi#7EE600:"Freeable Space"  \
GPRINT:cdefi:LAST:"Current\:%8.2lf %s"  \
GPRINT:cdefi:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefi:MAX:"Maximum\:%8.2lf %s\n" 

RRDTool Says:

ERROR: opening '/usr/share/cacti/site/rra/bh1_vol_total_22.rrd': No such file or directory

If I run this from the command prompt the graph is created. That is the issue we are talking about.

[cigamit]

Yes, I understand that part, but I was asking is whether they are created after 5-10 minutes if you do nothing after telling Cacti to create them. Since the rrd file, is not created when you say "Create Graphs" instead it is created after the first polling of that graph by the poller.

[Burvil]

Sorry, if I'm asking some simple questions, but without more info to go on, its difficult to diagnose.

No problem. I appreciate your help.

I assume you waiting until after 1 polling, since during the polling is when everything gets created.

.
Correct. In fact, here's a separate, but related post, of mine that shows more detailed information. I grepped for information from the debug log.
http://forums.cacti.net/viewtopic.php?t ... 7d4c696cdb.
That's with the Interface traffic, which doesn't seem to be updating at all, even if I create the rrd by hand. I think that must be corrupted somehow, as when I was trying to put together the example below to iillustrate the problem, I was able to create traffic graphs, albeit they were the ones that came with the HP-UX template, not the ones that came with cacti.

OK, so I'll go through an example.

1) I confirm I have 'DEBUG - statistics, errors..." set in Configuration -> Settings -> Poller logging level.

I create a host, using this HP-UX template -
I am able to get connectivity, i.e. after creating, I see

Code: Select all

 hostname-here (my.ip.address.here)
SNMP Information
System: HP-UX hostname-here B.11.11 U 9000/800 2225551502
Uptime: 618520143
Hostname: hostname-here 
Location:
Contact: 

So I know that SNMP is working on this host, and cacti recognizes that.

2) I monitor the log for any activity regarding this host:

Code: Select all

[cactiuser@burvil log]$ date; tail -f cacti.log  | grep hostname-here_
Fri Sep 15 21:52:44 BST 2006

I wait for twenty minutes, but see nothing. Then, when I manually go in and add a data source for this host, I get

Code: Select all

09/15/2006 10:11:28 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "update data_template_data set data_source_path='<path_rra>/hostname-here_hpux_idlecpu_67.rrd' where local_data_id=67"
09/15/2006 10:11:28 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "insert into poller_item (local_data_id,host_id,action,hostname,   snmp_community,snmp_version,snmp_timeout,snmp_username,snmp_password,snmp_port,rrd_name,rrd_path,   rrd_num,arg1,arg2,arg3) values (67,10,0,'my.ip.address.here',   'public','1','500',   '','','161',   'hpux_IdleCPU','/var/www/html/cacti/rra/hostname-here_hpux_idlecpu_67.rrd',   '1','.1.3.6.1.4.1.11.2.3.1.1.15.0','','')"
09/15/2006 10:11:28 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "insert into poller_item (local_data_id,host_id,action,hostname,   snmp_community,snmp_version,snmp_timeout,snmp_username,snmp_password,snmp_port,rrd_name,rrd_path,   rrd_num,arg1,arg2,arg3) values (67,10,0,'my.ip.address.here',   'public','1','500',   '','','161',   'hpux_IdleCPU','/var/www/html/cacti/rra/hostname-here_hpux_idlecpu_67.rrd',   '1','.1.3.6.1.4.1.11.2.3.1.1.15.0','','')"
09/15/2006 10:11:46 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "insert into poller_item (local_data_id,host_id,action,hostname,   snmp_community,snmp_version,snmp_timeout,snmp_username,snmp_password,snmp_port,rrd_name,rrd_path,   rrd_num,arg1,arg2,arg3) values (67,10,0,'my.ip.address.here',   'public','1','500',   '','','161',   'hpux_IdleCPU','/var/www/html/cacti/rra/hostname-here_hpux_idlecpu_67.rrd',   '1','.1.3.6.1.4.1.11.2.3.1.1.15.0','','')"

and OK, I see my rrd file:

Code: Select all

[cactiuser@burvil log]$ ls -l /var/www/html/cacti/rra/hostname-here_hpux_idlecpu_67.rrd
-rw-r--r--  1 cactiuser cactiuser 95324 Sep 15 22:15 /var/www/html/cacti/rra/hostname-here_hpux_idlecpu_67.rrd

and it does get updated, as excerpted here from the log:

Code: Select all

09/15/2006 10:15:04 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool create /var/www/html/cacti/rra/hostname-here_hpux_idlecpu_67.rrd   --step 300   DS:hpux_IdleCPU:COUNTER:600:0:U  RRA:AVERAGE:0.5:1:600  RRA:AVERAGE:0.5:6:700  RRA:AVERAGE:0.5:24:775  RRA:AVERAGE:0.5:288:797  RRA:MIN:0.5:1:600  RRA:MIN:0.5:6:700  RRA:MIN:0.5:24:775  RRA:MIN:0.5:288:797  RRA:MAX:0.5:1:600  RRA:MAX:0.5:6:700  RRA:MAX:0.5:24:775  RRA:MAX:0.5:288:797  RRA:LAST:0.5:1:600  RRA:LAST:0.5:6:700  RRA:LAST:0.5:24:775  RRA:LAST:0.5:288:797 
09/15/2006 10:15:04 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/hostname-here_hpux_idlecpu_67.rrd --template hpux_IdleCPU N:454407822
09/15/2006 10:20:05 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/hostname-here_hpux_idlecpu_67.rrd --template hpux_IdleCPU 1158355202:454419191
09/15/2006 10:25:05 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/hostname-here_hpux_idlecpu_67.rrd --template hpux_IdleCPU 1158355502:454429622

So, perhaps I didn't actually have to run it by hand; I guess it would have gotten created if I waited, or I created the data source when it just happened to be polling.

Well, I just realized I skipped the step of clicking 'Create Graphs for this host'. I went back to the device, and clicked it, and the graphs are in fact being created now. I guess I just had to wait.