Cactid waiting on pollers - no other debug info ?

[BelgianViking]

I am succesfully running Cacti with the cmd.php poller process. I'm trying to switch to Cactid (0.8.6f) and checked all installation information. When I run it manually I get the following output only:
C:\wwwroot\cacti>php poller.php
10/19/2005 01:07:25 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Proces
s [CMD: c:/cactid/cactid.exe, ARGS: 0 0]
10/19/2005 01:07:25 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Proces
s [CMD: c:/cactid/cactid.exe, ARGS: 2 2]
10/19/2005 01:07:25 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Proces
s [CMD: c:/cactid/cactid.exe, ARGS: 3 3]
10/19/2005 01:07:25 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Proces
s [CMD: c:/cactid/cactid.exe, ARGS: 4 4]
10/19/2005 01:07:25 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Proces
s [CMD: c:/cactid/cactid.exe, ARGS: 5 5]
10/19/2005 01:07:25 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Proces
s [CMD: c:/cactid/cactid.exe, ARGS: 6 6]
10/19/2005 01:07:25 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Proces
s [CMD: c:/cactid/cactid.exe, ARGS: 7 7]
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
Waiting on 7/7 pollers.
^C

And it will wait 'indefinitely'.
There nothing more than the same DEBUG messages in the Cacti log (set to the highest debug level). Would anyone have a suggestion on where I should start looking ?

Thanks a lot.

[BSOD2600]

possibly permissions. make sure the scheduled task user has read/execute rights on cactid.exe. I forget, but you might need to add IIS_WPG user to cactid.exe too.

What happens when you manually run cactid.exe?

[BelgianViking]

I run the Sheduled Task under the Network Administrator account. This one should have every possible right. I've added IIS_WPG for read & execute rights on the c:\cactid directory. The result is still the same. The only output is the "spawning ...", which I see in the DOS box, as well as in the Cacti log. Then there's the "waiting ..." messages in the DOS box. I see some other DOS boxes getting started and than dissappear again. I 'm under the impression that a message appaers in each of these boxes before they dissapear, but I can't grasp them.

[TheWitness]

What happens when you just run cactid from the cactid directory in DEBUG mode?

TheWitness

[BelgianViking]

Aha, that seems clear. This must be the message popping up in the DOS boxes.
I guess you mean to set DEBUG in the Poller logging level ?

This is the output:
C:\cactid>cactid
CACTID: Using cactid config file [cactid.conf]
CACTID: MYSQL: Connection Failed: #HY000Host 'netmon.be.bvdep.net' is not allowed to connect to this MySQL server

My cactid.conf file is this :
DB_Host 172.28.2.148
DB_Database cacti
DB_User cactiuser
DB_Pass cacti

It seems a security problem, but I have no clue how to handle this.
Thanks for your suggestion.

[BelgianViking]

Could it be I need to GRANT some more permissions ? Or that the local host from which connections are allowed is stored as 'netmon' instead of the full classified name ?

[BelgianViking]

Since it is clearly about privileges, I googled a bit and tried these commands:

mysql> GRANT ALL PRIVILEGES ON cacti.* TO 'cactiuser'@'localhost' IDENTIFIED BY
'cacti' WITH GRANT OPTION;
Query OK, 0 rows affected (0.01 sec)

mysql> GRANT ALL PRIVILEGES ON cacti.* TO 'cactiuser'@'%' IDENTIFIED BY 'cacti'
WITH GRANT OPTION;
Query OK, 0 rows affected (0.02 sec)

Than flushing of course. Note that the commands in the BSOD intstall manual are a bit different !

Now running Cactid in the cactid directory gives me this output:

C:\cactid>cactid
CACTID: Using cactid config file [cactid.conf]
CACTID: Version 0.8.6f starting
No log handling enabled - turning on stderr logging
NOTIFICATION-LOG-MIB: Module not found
NET-SNMP-AGENT-MIB: Module not found
Creating directory: /var/net-snmp
CACTID: Execution Time: 20.5820 s, Threads: 50, Hosts: 7

Swapping to the Cactid.php instead of the cmd.php gives me the same output as above in each dos boc popping up, and than gets the results all right.
Execution time however is longer than with cmd.php (9 versus 6 seconds).This indicates something is not completely right ! Maybe this is related to the MIB module errors ?
Thanks for helping me out !

[BelgianViking]

I added a debug log. Right after running cactid.cmd, I ran cmd.php.

[BelgianViking]

I've found these mib files in the c:\net-snmp\share\snmp\mibs directory and copied them to the c:\php\extras\mibs directory and the error has gone now. However, execution time for cactid.php is still 12 seconds compared to the 6 seconds for cmd.php.
How can I tackle this ?

[BelgianViking]

I decreased the number of concurrent poller processes from 50 to 8. I'm down to 5.5 seconds now. Is there any rules to tune these settings ?

[TheWitness]

Yes: concurrent processes no higher than 2 times the number of processes, threads 25 or below. In fact, in my production environment, I have 1 less cactid process than the number or processors and about 20 threads and it seems to be the sweat spot.

Performance is realy a factor of your polling environment: snmp vs. script vs. script server. Also, the type of scripts are important, if they run longer, then you might want more concurrent processes.

I guess to have a good understanding of how to tune your environment, you should know a little bit about how the poller operates. One of my projects once work calms down. Until then, just use my recommendations above.

TheWitness