PHP Process hanging

adrianmarsh · Post by **adrianmarsh** » Thu Aug 18, 2005 5:25 am

All,

After getting the graphs to work, the system now looks good. But there are several points within the Cacti menus where I can cause the whole system to hang...

Heres what I do:

I go to Devices, select my PC (actually the www server itself), and the try to add an "Associated Data Query" - selecting SNMP Get Processor info. I click "Add" and then the system hangs. IE spins its icon, and after 60s I get a cgi timeout message.

On the server, using Process Explorer, I see that inetinfo.exe has spawned php.exe, which in turn spawns cmd.exe, which in turn spawns another php.exe process.

Until the timeout (or if I kill the last php.exe process manually on the server), all other Cacti windows hang as well. Other PHP-based tools on the server (WebCalender) work fine, even whilst Cacti has hung.

I tried switching on the debug, but nothing comes out in the log..

Obviously if I were to release this out "as is" now, the server would constantly hang...

Edit: I'm still testing, but I'm beginning to see a pattern that this only seems to happen when the Target is a PC.

Edit: I thought this might have been the SNMP agent on the client, so tried net-snmp as well, but got the same result.

Post by **rony** » Thu Aug 18, 2005 12:48 pm

What versions of:

Cacti
PHP
IIS
SNMP

adrianmarsh · Post by **adrianmarsh** » Thu Aug 18, 2005 1:14 pm

Hi Tony,

php:4.4.0
cacti:0.8.6f
iis:5
snmp: (windows: what comes with 2000AS?): net-snmp: 5.2.1.1, SNMP: v1

Definitly only happens on PCs (Cisco/Foundry machines are ok).

Is there any debug I can switch on?

adrianmarsh · Post by **adrianmarsh** » Sun Aug 21, 2005 3:56 am

This still hangs. I've tried against two PCs now, one running microsoft SNMP and the other net-snmp (www server itself).

From a 3rd machine I can snmp-walk to both of the other two

Post by **TheWitness** » Sun Aug 21, 2005 8:16 am

I believe that this is a permissions issue with IIS and it's ability to access cmd.exe or some other directory. Please goto one of your working graphs under graph management and see if you can edit the graph. When editing the graph hit the "Graph Debug" link in the upper right hand corner. Does you system hang then?

TheWitness

adrianmarsh · Post by **adrianmarsh** » Sun Aug 21, 2005 8:30 am

I tried switching on graph debug with a cisco CPU graph:

- On the www server, I saw php and cmd.exe all startup and then quit
- on the www page I saw the rrdtool Says : "OK"

Was this what you were expecting?

I've been playing/setting up graphs/templates for Solaris/Cisco/Avaya kit all day with no problems. only when I start to add Windows PCs...

v. strange..

I thought I'd start again... so cleaned out the Computers, and then just went through the "add" device 'wizard' again.. Sure enough after clicking Add I see inetinfo -> php -> cmd -> php processes start and stay running.

Are there any manual php commands I can run on the server to simulate clicking the Add button, maybe I'd get some output that way?

It does look as though some of the setup works because after manually killing the last php process, the computer is then listed in Devices

Post by **TheWitness** » Sun Aug 21, 2005 8:42 am

I think you must have some database issue causing this, or you have found a bug in PHP 4.4. You should possibly back off to an earlier version of PHP, say 4.3.11 and try again.

From my perspective, I find the logging available in Apache better. Another idea would be to deploy apache just for testing and then review it's error log for some important messages. You could do this by running apache on port 81 and keep IIS running on port 80.

I would like to spend some more time with you debugging, but unfortunately, I have to do some work around the house at the moment.

Regards,

TheWitness

adrianmarsh · Post by **adrianmarsh** » Sun Aug 21, 2005 9:16 am

No problem... same here (or at least "I should go and do that housework..")...

I'll look into backing out php next week, or going the apache route if need be. Odd that its only the PCs being affected though.

What makes you think it might be a db issue?

Post by **TheWitness** » Sun Aug 21, 2005 10:27 am

So, you say it's a browser issue. Hmm. Then, please state the OS and browser versions affected. Just got done cutting the grass.

TheWitness

adrianmarsh · Post by **adrianmarsh** » Sun Aug 21, 2005 11:29 am

Well.. I don't remember saying that it was a browser issue, but to test the idea I used firefox to create the PCs and found it worked (normal browser is IE 6)... Then I realised that I hadn't specified the host template. Then I tried the same thing in IE (omitting the host template), and it worked... so now I'm looking into what component of the Windows host template causes it to crash out....

Trying each one at a time I find:

Karlnet - Wireless Bridge Statistics OK
Netware - Get Available Volumes OK
Netware - Get Processor Information OK
*SNMP - Get Mounted Partitions (CRASHES) - removes OK
*SNMP - Get Processor Information (CRASHES) - removes OK
SNMP - Interface Statistics OK
ucd/net - Get Monitored Partitions OK
Unix - Get Mounted Partitions OK

Looking at the Windows template I see that it includes the two that cause the crash. Obviously I'm not applying the whole of the above to the PC for real, just doing this for testing now..

If I select SNMP - Get Processor Info and allow it to crash out (kill the last php) and then go back into the device I see it listed as Success. If I then do a verbose output (and again it crashes) I get :

+ Running data query [9].
+ Found type = '6 '[script query].
+ Found data query XML file at 'C:\inetpub\wwwroot\cacti/resource/script_server/host_cpu.xml'
+ XML file parsed ok.
+ Executing script for list of indexes 'c:\php\php.exe -q C:\inetpub\wwwroot\cacti\scripts\ss_host_cpu.php swocmpq04 public 1 161 500 index'
+ Executing script query 'c:\php\php.exe -q C:\inetpub\wwwroot\cacti\scripts\ss_host_cpu.php swocmpq04 public 1 161 500 query index'
+ Found data query XML file at 'C:\inetpub\wwwroot\cacti/resource/script_server/host_cpu.xml'
+ Found data query XML file at 'C:\inetpub\wwwroot\cacti/resource/script_server/host_cpu.xml'
+ Found data query XML file at 'C:\inetpub\wwwroot\cacti/resource/script_server/host_cpu.xml'

Would you expect anything different? Would the XML file message repeat ? If its looping then we'd get the result I'm seeing of PHP never ending.

Can I try the host_cpu script manually in any way?

adrianmarsh · Post by **adrianmarsh** » Sun Aug 21, 2005 11:39 am

I also just tried it with php4.3.11 and got the same result (now put 4.4.0 back in place)

adrianmarsh · Post by **adrianmarsh** » Sun Aug 21, 2005 11:48 am

Also, adding Graph Templates :

Host MIB - Logged in Users
Host MIB - Processes

Doesn't cause the issue... so at lest I've narrowed it down.. and can use the network interfaces ok. I guess this means its either an sql issue or a php script issue then ?

Attached is a full snmpwalk from swocmpq04 in case theres something of interest there.

Post by **TheWitness** » Sun Aug 21, 2005 11:55 am

Do you have 0.8.6f patches applied. Also, I am on-line now if you can share.

TheWitness

Post by **TheWitness** » Sun Aug 21, 2005 12:00 pm

Also, can you copy the following files from the distribution to your working Cacti directory:

resources/script_server/host_disk.xml
resources/script_server/host_cpu.xml

scripts/ss_host_disk.php
scripts/ss_host_cpu.php

This may simply fix the problem.

TheWitness

adrianmarsh · Post by **adrianmarsh** » Sun Aug 21, 2005 12:15 pm

I only installed cacti last week, from a base 0.8.6f install - are there patches to this?

I've re-copied those files from cacti-0.8.6f.zip (no diff, same time/dates on the files)

Still no change though

PHP Process hanging

PHP Process hanging

Who is online