RRDtool not graphing, RRDupdate ok

[wasted247]

Hi All. I have a weird problem.

I have a cacti install consisting of the following components:

- windows 2003 server R2 x64
- cacti 8.7.e
- IIS 6 webserver
- php 5.2.13
- mysql 5.1.45
- rrdtool 1.3.8
- net-snmp 5.5.0-2

my test host (the cacti server itself) is responding to snmp requests, generating rrd files and polling correctly. But the graphs are not being generated. When turning on graph debugging i get the following response:

Code: Select all

RRDTool Command:

c:/rrdtool/rrdtool.exe graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="cacti01.domeintje.local - Traffic - 192.168.1.234 (Intel(R) PRO/10)" \
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="bits per second" \
--slope-mode \
--font TITLE:12: \
--font AXIS:8: \
--font LEGEND:10: \
--font UNIT:8: \
DEF:a="C\:/htdocs/rra/cacti01_domeintje_local_traffic_in_12.rrd":traffic_in:AVERAGE \
DEF:b="C\:/htdocs/rra/cacti01_domeintje_local_traffic_in_12.rrd":traffic_out:AVERAGE \
CDEF:cdefa=a,8,* \
CDEF:cdefe=b,8,* \
AREA:cdefa#00CF00FF:"Inbound"  \
GPRINT:cdefa:LAST:" Current\:%8.2lf %s"  \
GPRINT:cdefa:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefa:MAX:"Maximum\:%8.2lf %s\n"  \
LINE1:cdefe#002A97FF:"Outbound"  \
GPRINT:cdefe:LAST:"Current\:%8.2lf %s"  \
GPRINT:cdefe:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefe:MAX:"Maximum\:%8.2lf %s\n"  \
 \
HRULE:11042.61#FF0000FF:"95th Percentile"  \
COMMENT:"(0.01 mbit in+out)\n" 

RRDTool Says:

As you can see RRDtool is not responding to the graph command. When i enter the command from a command prompt, it generates the png flawlessly. With the data up to date, proving that the rrd files are being updated by RRDtool.

I have tried several things to fix this:

- Check permissions on c:\htdocs\ (for IISUSRS, CACTIUSR).
- Check permissions on c:\rrdtool\ (for IISUSRS, CACTIUSR).
- Check permissions on c:\php\ (for IISUSRS, CACTIUSR).
- Check permissions on c:\net-snmp\ (for IISUSRS, CACTIUSR).
- Check permissions on c:\windows\system32\cmd.exe (for IISUSRS, CACTIUSR).
- check execute permissions in iis 6, these are on.
- run iis 6 site and apppool under a local user 'CACTIUSR', no change.
- Cacti logging in debug mode, no obvious errors.
- PHP error logging, no obvious errors.
- PATH variables, php, rrtool and net-snmp paths are present.

Since the rrd files are being updated i dont suspect a permission problem. But i honestly dont know where to look. Anyone have any ideas?

Below a snippet of the rrdtool.exe update command:

Code: Select all

03/24/2010 11:12:01 AM - POLLER: Poller[0] CACTI2RRD: c:/rrdtool/rrdtool.exe update C:\htdocs\rra\cacti01_domeintje_local_cpu_13.rrd --template cpu 1269425521:0
03/24/2010 11:12:01 AM - POLLER: Poller[0] CACTI2RRD: c:/rrdtool/rrdtool.exe update C:\htdocs\rra\cacti01_domeintje_local_traffic_in_12.rrd --template traffic_in:traffic_out 1269425521:23481198:6137151
03/24/2010 11:12:01 AM - POLLER: Poller[0] CACTI2RRD: c:/rrdtool/rrdtool.exe update C:\htdocs\rra\cacti01_domeintje_local_cpu_14.rrd --template cpu 1269425521:0

[BSOD2600]

Are you sure that c:\rrdtool\rrdtool.exe has read/execute permissions for your IUSR_ and IIS_WPG accounts? That is how IIS 6 (default, unless you changed things) will invoke the graph creation (via cmd.exe of course).

Viewing all graphs via graph management doesn't work either, I assume?

[wasted247]

Absolutely! Double, double, double checked

Also the rrdtool.exe update command is being executed. That's the annoying thing in this case. everything is configured like its supposed to be. Although i just realized that the update command is being invoked by the poller scheduled task right?

Went through the install guide for windows like 4 times by now.

I have the following users in the permissions:

- IIS_WPG
- IUSR_CACTI01
- IWAM_CACTI01
- CACTIUSR

These are configured in multiple places, which include the rrdtool folder, and c:\windows\system32\cmd.exe.

At first i was working with the default IIS6 accounts, later on i configured a specific account, to see if it mattered. It didn't. Put the default account back.

EDIT: To make the problem more interesting:

I installed process monitor to see what was happening, cmd.exe is actually being executed, as well as rrdtool.exe, the response never arrives at the webserver.

The process being created from w3wp.exe:

Code: Select all

cmd.exe /c c:/rrdtool/rrdtool.exe graph -  --imgformat=PNG  --start=-86400  --end=-300  --title="cacti01.je-itc.local - CPU Utilization - CPU0"  --rigid  --base=1000  --height=120  --width=500  --alt-autoscale-max  --lower-limit=0  --vertical-label="percent"  --slope-mode  --font TITLE:12:  --font AXIS:8:  --font LEGEND:10:  --font UNIT:8:  DEF:a="C\:/htdocs/rra/cacti01_je-itc_local_cpu_13.rrd":cpu:AVERAGE  AREA:a#FF0000FF:"CPU Utilization"   GPRINT:a:LAST:"Current\:%8.0lf"   GPRINT:a:AVERAGE:"Average\:%8.0lf"   GPRINT:a:MAX:"Maximum\:%8.0lf\n"

Followed by:

Code: Select all

c:/rrdtool/rrdtool.exe graph -  --imgformat=PNG  --start=-86400  --end=-300  --title="cacti01.je-itc.local - CPU Utilization - CPU0"  --rigid  --base=1000  --height=120  --width=500  --alt-autoscale-max  --lower-limit=0  --vertical-label="percent"  --slope-mode  --font TITLE:12:  --font AXIS:8:  --font LEGEND:10:  --font UNIT:8:  DEF:a="C\:/htdocs/rra/cacti01_je-itc_local_cpu_13.rrd":cpu:AVERAGE  AREA:a#FF0000FF:"CPU Utilization"   GPRINT:a:LAST:"Current\:%8.0lf"   GPRINT:a:AVERAGE:"Average\:%8.0lf"   GPRINT:a:MAX:"Maximum\:%8.0lf\n"

Then a whole bunch of ReadFile SUCCES commands on:
C:\htdocs\rra\cacti01_je-itc_local_cpu_13.rrd

Then a thread exit, procces exit for rrdtool.exe, followed by the same for cmd.exe.

[BSOD2600]

Although i just realized that the update command is being invoked by the poller scheduled task right?

Correct. Thats what can make this whole process even more complicated, because different user accounts are used for polling and graphing.

The "CACTIUSR" account is for the scheduled task?

I installed process monitor to see what was happening, cmd.exe is actually being executed, as well as rrdtool.exe, the response never arrives at the webserver.

Nice a user who actually knows how to do his own in-depth investigation. How refreshing on this forum.

1) What do you mean it never arrives at the web server?
2) Any 'denied' error messages in procmon during attempted graphing?
3) you have rrdtool 1.3 set in cacti's settings?
4) for testing, go back to a rrdtool 1.2.x binary and see if that changes anything.

[wasted247]

The CACTIUSR was for testing purposes, because the IUSR_CACTI01 didn't seems to work. The scheduled task was running under a local admin account, in a desperate attempt even ran the webserver under that account, still nothing.

1) With 'the response never reaches the webserver' i meant that rrdtool is generating something, but the result is not being picked up by the webserver, thus not being displayed. Don't know how to explain it any better . In procmon i see both cmd and rrdtool getting to work, but nothing seems to happen in the web interface.

2) No error messages, unfortunately, only SUCCESS messages.

3) Yep.

4) Already tried an older rrdtool, nothing either.

Fortunately, this is a medium sized ESX farm. I built a second cacti machine, this time using your excellent installer i found later (wish i found that earlier, before my separate component install ).

When using IIS as the HTTP server i got some CGI errors, and the cacti pages wouldn't run. Personally i prefer Apache anyways. The choice for IIS was mainly because our other webservers run IIS.

Retried with the Apache webserver, and wouldn't you know it, works like a charm! I only had to reset the admin password through the mysql cli. Now searching for the appropriate templates for the machines and services we are running. Already got a bunch, haven't tried them yet.

Thank you for your help, and of course the installer, which did it all right at once!