graph export slowdown

[phpgold]

Hi all,

I'm hoping that someone out there can help me out. I'm currently experiencing a major slowdown when I enable the graph export function under cacti. The settings are set to Classical, Tree Presentation exporting to a local directory on the cacti box. We are currently using rrdtool 1.2.

When I run cacti without the graph export everything works fine and the entire process takes 15 seconds using cmd.php as the poller.

When I enable the graph export it takes an entire 5 minutes and barely get's done before the next poll begins.

Now granted it is running on a P800 system. But that should be enough, right...? Can someone tell me if this is normal?

If it is normal then can someone tell how to size the system for a paricular amount of graphs. If I want to deploy this system with graph exports using around 1000 graphs what kind of system should I be looking at?

Any help is greatly appreciated.......

Thanks...

Neil

[BSOD2600]

1000 graphs for a P3 800? Yea... you're really taxing it there. What version of cacti / rrdtool are you running exactly?

I'd think you'd need at least a 2ghz or more to export that many graphs.

[phpgold]

1000 graphs is probably going to be the amount of graphs in the end but right now there are only 150 graphs.

I am currently using rrdtool-1.2.12-1.1 from the dag repository.

Later...
Neil

[coolsoftware]

I have the same issue - 1.7Ghz machine with Gig of RAM - lots of disk space...multiple disk drives - exports are written to drive with this specific purpose - ONLY cacti, apache and ssh run on this machine...FreeBSD latest stable.

running snmp cactid - idle time 0.0 most of the time....graphs exporting about 125 - however, once we went "over" about 110 graphs , all became unstable in the graph export area....part of the problem is that the index.html exported seems to be re-written for every graph when "each" graph is exported. A strange poor practice, resulting booging things down. At teh time of it's writting, I bet it worked well for the "test" and exported a whole dozen graphs.

So can you export 1000 graphs with cacti?...no way...i have had 2 very good php programmers look at this -- also post here and no one has a responce other than more cpu power.

So if cpu power is the need...to export 1000 graphs -- well, based upon you and I currently exporting 100+ with the current broken results...one would need a 10+ Ghz machine.

My guess is that there is no one exporting over 125 graphs using rrdtools and cacti.

[kwabbernoot]

This to give an idea,

System: HP DL360 G3
CPU: Intel Xeon 3.06Ghz
RAM: 1GB
Storage Controller: Smart Array 5i/532

Linux version: Slackware 10.2.0
MySQL version: 4.1.14
Cacti version: 0.8.6h
PHP version: 5.0.4

Cacti Poller settings:
Poller Type: cactid
Max Threads: 50
PHP Script Servers: 4

Graph Export Stats:
EXPORT STATS: ExportTime:396.7878 TotalGraphs:5555

Graph Export function begins to fail and I will need to launch using an external script.

Regards,
Kwabbernoot

[coolsoftware]

Your data post got me thinking...each graph export appears to take 30 seconds on my machine 1.7ghz gig ram lots of plenty of disks & space.

396.xxx is this value in minutes?....currently my script time out setting was 20 mins increased it to 60.....my export graph setting interval was 1800 seconds - increased it to 7200...prior to the changes, i looked at the time stamp on the exported graphs, only 50 export (out of 100+) and 20 mins script time limit is exceeded - my thread setting was 8 - increased it to 50 - increased concurrent poller from 4 to 8

now rebooting and will check it later.

thanks
bob evans

[kwabbernoot]

Your data post got me thinking...each graph export appears to take 30 seconds on my machine 1.7ghz gig ram lots of plenty of disks & space.

396.xxx is this value in minutes?....currently my script time out setting was 20 mins increased it to 60.....my export graph setting interval was 1800 seconds - increased it to 7200...prior to the changes, i looked at the time stamp on the exported graphs, only 50 export (out of 100+) and 20 mins script time limit is exceeded - my thread setting was 8 - increased it to 50 - increased concurrent poller from 4 to 8

now rebooting and will check it later.

thanks
bob evans

No, the value is in seconds.
I started playing with the threads and PHP script session and managed to decrease from ExportTime:483.1442 for 5480 grpahs to 396.7878 seconds for 5555 Graphs. I had to add some extra devices.

I only export once an hour and my poller settings are like this:
Script and Script Server timeout: 25 seconds

My poller is "cactid" this one is a lot faster than cmd.php

Regards,
Kwabbernoot

[coolsoftware]

my export graph job would stop, when it wasnt finished and the next export task was to begin. i set the interval to export every 2 hours or so - now i get all the graphs exported. i think this setting was the key - the export task takes to long....30 seconds per graph (about 5 images and an html for each graph).

this appears to have solved my immediate issue...but a 1.7ghz machine with latest freebsd, gig of ram, disks carefully organized to not use the same for the os, cacti and the export....the export needs to be tuned....the machine between exports runs fine with left over cpu idle times....tuning the export to "only export the daily graph instead of all of them everytime" - not rewriting the index.html evertime a new graph is exported....stuff like that will make this mroe robust.

Thanks for the input i appreciate it
bob evans

[petero]

Is there any one who can write little addon so we can export like only graphs we need.Like main link or so.
Just a hint

Peter