Spine Poller POPEN timeout never disappear

Post support questions that relate to the Windows 2003/2000/XP operating systems.

Moderators: Developers, Moderators

Post Reply
guitou
Posts: 49
Joined: Wed Feb 27, 2008 4:13 am

Spine Poller POPEN timeout never disappear

Post by guitou »

Hi,

i've recently upgrade my cacti version from 0.8.6j to 0.8.7e on windows 2003 server. I've passed the poller from cmd.php to spine and it's OK. When a host is detected as DOWN the poller return me a POPEN timeout (it's normal, i know) but on the server the poller windows where the timeout occure stay open. Even more the system stats time (duration) increase about 40 seconds when a host is detected as down. So imagine when more than two host go down, the poller became out of sync because the poller pass more than 300 seconds to finish.
Any ideas ?

Thanks for your support

Cacti Version - 0.8.7e
Plugin Architecture - 2.4
Poller Type - Cactid v
Server Info - Windows NT 5.2
Web Server - Apache/2.2.11 (Win32) PHP/5.2.9
PHP - 5.2.9
PHP Extensions - bcmath, calendar, com_dotnet, ctype, session, filter, ftp, hash, iconv, json, odbc, pcre, Reflection, date, libxml, standard, tokenizer, zlib, SimpleXML, dom, SPL, wddx, xml, xmlreader, xmlwriter, apache2handler, ldap, mysql, snmp, sockets
MySQL - 5.0.83-community-nt
RRDTool - 1.2.30
SNMP - snmpwalk [OPTIONS] AGENT [OID]
Plugins
  • Syslog Monitoring (syslog - v0.5.2)
    Router Configs (routerconfigs - v0.1)
    Device Tracking (mactrack - v1.1)
    Host Info (hostinfo - v0.2)
    Global Plugin Settings (settings - v0.6)
Attachments
cactiplus windows with popen timeout never close herslef
cactiplus windows with popen timeout never close herslef
popen_not_disappear.jpg (23.63 KiB) Viewed 8217 times
guitou
Posts: 49
Joined: Wed Feb 27, 2008 4:13 am

Nobody want to help me ?? !

Post by guitou »

Hi, i still don't understand the POPEN timeout message error. The status (UP or DOWN) of the device is not a consequence of this error message so i wonder why it happens.
Thanks for your support
User avatar
BSOD2600
Cacti Moderator
Posts: 12171
Joined: Sat May 08, 2004 12:44 pm
Location: USA

Post by BSOD2600 »

Not sure, going to require the spine developer to chime in.

What happens when you run spine with the verbose debug mode enabled?
guitou
Posts: 49
Joined: Wed Feb 27, 2008 4:13 am

Post by guitou »

Hi,

it's very difficult to debug this problème because is it not permanent. I've install cacti on a windows 2003 server R2 with the last package cacti-0.8.7e-1.931x86.exe
I' don't have apply any patch so is there some to install (pre-patched files)
Also, i'm running cacti 0.8.7e with PIA 2.4 and i wonder if it is possible to downgrade Spine from 0.8.7e to 0.8.7c.

Finally, what's does it mean POPEN timeout ?

Thanks
Attachments
cacti-log.JPG
cacti-log.JPG (236.62 KiB) Viewed 8111 times
User avatar
BSOD2600
Cacti Moderator
Posts: 12171
Joined: Sat May 08, 2004 12:44 pm
Location: USA

Post by BSOD2600 »

Popen is more of a linux thing that works on windows... "The popen() function executes the command specified by the string command. It creates a pipe between the calling program and the executed command, and returns a pointer to a stream that can be used to either read from or write to the pipe. "

Yes, you could downgrade spine versions if you wanted.
guitou
Posts: 49
Joined: Wed Feb 27, 2008 4:13 am

Post by guitou »

I have tried to downgrade spine to 0.8.7d and c, this don't solve the POPEN problem. This action have increase a lot of different new problems like perl script Empty result with spine poller. At 11:20 i've tried to passe the poller on cmd.php and the poller seems to not finish well. After all I finally change the poller to spine with 0.8.7e binaries.

I've seen a new patch name "Cross-Site Scripting Fixes" publish at 2009/11/21. What does it solve ? Did I have to apply this and where can i fin the pre-patched file associated ?

Cheers
Attachments
ScreenShot001.jpg
ScreenShot001.jpg (212.31 KiB) Viewed 7792 times
User avatar
BSOD2600
Cacti Moderator
Posts: 12171
Joined: Sat May 08, 2004 12:44 pm
Location: USA

Post by BSOD2600 »

guitou wrote:I've seen a new patch name "Cross-Site Scripting Fixes" publish at 2009/11/21. What does it solve ? Did I have to apply this and where can i fin the pre-patched file associated ?
Details are possibly in the bug tracker and/or can be extrapolated from the lines of code it patches.

Prepatched files: http://www.cacti.net/downloads/patches/ ... e-patched/
guitou
Posts: 49
Joined: Wed Feb 27, 2008 4:13 am

Post by guitou »

Thanks,

I'll try this asap.

Also, I post you the windows error when Spine crash ERROR: Spine Timed Out While Processing Hosts Internal or ERROR: The POPEN timed out
Did the Spine Dev have return to you something ...?

Finally, the cacti server i an VMware virtual serveur with quad core and 2GB memory, is there known bug on VM plateform ..?
Attachments
Spine error windows debug.jpg
Spine error windows debug.jpg (36.65 KiB) Viewed 7761 times
User avatar
BSOD2600
Cacti Moderator
Posts: 12171
Joined: Sat May 08, 2004 12:44 pm
Location: USA

Post by BSOD2600 »

Naw, doubt its a vmware issue since cacti/spine aren't aware of it. I do see the crash was likely caused by the cygwin dll. Search your system for cygwin1.dll. Make sure you only have ONE COPY and that it resides in your spine folder. If you have other applications which depend on cygwin functionality, then install it, put it in the system path and then remove all cygwin dlls from the spine folder.

TheWitness is the spine author who can better troubleshoot how/why its broken.
guitou
Posts: 49
Joined: Wed Feb 27, 2008 4:13 am

Post by guitou »

Hi,

after several tuning test of poller option, I've finally discover that my problem is link with the number of threads value I put in poller option, strange... When i passe this value from 1 to 2, 4, 8 or more, POPEN timeout occurs more often.

Also i have search for duplicate cygwin1.dll file. I just find one in the spine folder.

Finally, i have reinstall my cacti server with the latest installer 1.94 on a new stand alone plateform and the problem still exist
guitou
Posts: 49
Joined: Wed Feb 27, 2008 4:13 am

Post by guitou »

Hi BSOD2600,

I'm looking for answer on my POPEN timout problem. When i put value 1 in maximum threads per process field this error disappear and sometime i've got this message"SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal". If I increase the maximum thread to 2, 4 or more in the poller option the POPEN timout message reappear more often.

Best Regards
User avatar
TheWitness
Developer
Posts: 17061
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

This is why I dislike Windows, for spine and forking anyway. I never know if it's a Cygwin issue or a spine issue.

Windows has issues (at the API level) when using native ANSI C applications combined with forking. This is most evident when killing child processes that timeout.

The Nifty Popen is designed to kill the child if it times out. That's why its the default. The native Popen provides no mechanism to trap a timeout without an alarm. I guess that's what I have to implement. Not today I suspect though.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest