Cacti in large environment fails.

[Adze15021979]

Hi guys,

I have a huge problem getting cacti to run for more than a few hours. As a desperate measure i started this thread. Please, please help me, cause we use cacti in our company network management, which at this point doesn't work. (i recommended cacti and now everybody is making fun of me...)

We have a dual P3 600MHz server with 1GB mem + 3x 9,1 GB SCSI RAID5. On it runs RedHat EL 4 (fresh install). We use cati version 0.8.6e + cactid 0.8.6e. We currently are monitoring 18000+ data sources on 86 hosts (cisco switches).

All was fun and games until last monday. I was adding about 20 new switches to monitor when I noticed that graphs aren't updated anymore. Also new graphs aren't made. After many hours troubleshooting and forum reading I found a sollution to get it working again (clear poller_output & poller_item in MySQL and regenerate cache), but just for a short while.

I attached a output from the cacti.log file. I can't see anything strange in it.

I already tried downgrading rrdtool from v1.2 to v1.0 but to no affail. I also disabled selinux.

Packages installed:
net-snmp-libs-5.1.2-11
net-snmp-utils-5.1.2-11
net-snmp-5.1.2-11
mysql-4.1.10a-2
rrdtool-1.0.49-2.2.el4.rf
php-4.3.9-3.6
httpd-2.0.52-12

Please help me

[Adze15021979]

Update:

When i try to run poller.php by hand i get the following:

Code: Select all

[root@vnolx001 rra]# /usr/bin/php /var/www/html/cacti/poller.php
Allowed memory size of 8388608 bytes exhausted (tried to allocate 10 bytes)

When i clear poller_output in MySQL, the poller script runs for several times and then exits with above memory size error message.

[fmangeant]

Hi

try to increase the memory allocated to the PHP scripts (in php.ini) :

Code: Select all

memory_limit = 64M      ; Maximum amount of memory a script may consume

On my system, the default value was 8 Mb...

[Adze15021979]

I also get this error when i run "/usr/bin/php /var/www/html/cacti/poller.php":

Code: Select all

read_config_store open failure on /var/net-snmp/cactid.conf
read_config_store open failure on /var/net-snmp/cactid.conf
read_config_store open failure on /var/net-snmp/cactid.conf

The poller script does continue however. Is it just cosmetic ?

[fmangeant]

You can IMHO ignore this one, I get the same kind of error when running as non root :

Code: Select all

# php -v
PHP 4.3.11 (cli) (built: Jun 13 2005 08:00:35)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies

$ php -v
PHP 4.3.11 (cli) (built: Jun 13 2005 08:00:35)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies
No log handling enabled - turning on stderr logging
Cannot rename //var/lib/net-snmp/snmpapp.conf to //var/lib/net-snmp/snmpapp.0.conf
Cannot unlink //var/lib/net-snmp/snmpapp.conf
read_config_store open failure on //var/lib/net-snmp/snmpapp.conf
read_config_store open failure on //var/lib/net-snmp/snmpapp.conf
read_config_store open failure on //var/lib/net-snmp/snmpapp.conf

[Adze15021979]

Hi

try to increase the memory allocated to the PHP scripts (in php.ini) :

Code: Select all

memory_limit = 64M      ; Maximum amount of memory a script may consume

On my system, the default value was 8 Mb...

I altered the time and memory settings in php.ini and cacti runs longer then it ever did the past 3 days... You are my hero

[rony]

If polling does not complete in the 5 minute window, you might want to consider a faster box.

But, what so you have your Settings->Poller->Poller Execution Parameters, Maximum Concurrent Poller Processes and Maximum Threads per Process set to?

You should be able to get away with 4 Maximum Conncurrent Poller Processes and 6 for Maximum Threads per Process. But take a baseline first and see if it really does help your polling time to alter these settings.

[Adze15021979]

But, what so you have your Settings->Poller->Poller Execution Parameters, Maximum Concurrent Poller Processes and Maximum Threads per Process set to?

1 process and 4 threads. I use cactid and it completes well under 5 minutes. Most of the time within 1 minute...

Altering php.ini did the trick solving my problem, at least so far...

[rony]

At least do 2 threads and 4 processes. You have 2 CPU's sitting there.

The only reason I would say don't, is if you are doing other things on the box. IE, network monitoring/alerting, big brother/nagios type stuff.

[Shouclack]

But, what so you have your Settings->Poller->Poller Execution Parameters, Maximum Concurrent Poller Processes and Maximum Threads per Process set to?

1 process and 4 threads. I use cactid and it completes well under 5 minutes. Most of the time within 1 minute...

Altering php.ini did the trick solving my problem, at least so far...

I have the same issue, Could you post what setting you altered and to which value? Thx!