Spine empty result on ws_apachestats.pl

[derekcat]

Our monitoring server is a little bit overloaded, I'd like to switch to spine to take some of the pressure off [as it uses about 1/5th the CPU, according to ucd/net - CPU Usage w/iowait for this server.

Code: Select all

10/28/2014 05:20:50 PM - SPINE: Poller[0] Host[436] ERROR: Empty result [127.0.0.1]: '/usr/bin/perl /usr/share/cacti-0.8.8b/site/scripts/ws_apachestats.pl 127.0.0.1'

Of course, if you run it manually on the CLI:
root@servername:/usr/share/cacti/site/log# /usr/bin/perl /usr/share/cacti-0.8.8b/site/scripts/ws_apachestats.pl 127.0.0.1
apache_total_hits:53511 apache_total_kbytes:102518 apache_busy_workers:1 apache_idle_workers:9 thread_O:246 threadC:0 threadD:0 threadG:0 threadI:0 threadK:0 threadL:0 threadR:0 threadS:0 threadW:1 thread_W:9 apache_cpuload:.00510802

Everything works beautifully via cmd.php, but this machine is an old dual-dual core AMD Opteron 1212 with 8GB RAM running both cacti and Nagios, Ubuntu 12.04 LTS (bleh). (/proc/cpuinfo says that it's running at 1ghz)

Currently about 1400 graphs in Cacti, with 92 hosts. Nagios has 95 hosts and 232 services.

[derekcat]

Our monitoring server is a little bit overloaded, I'd like to switch to spine to take some of the pressure off [as it uses about 1/5th the CPU, according to ucd/net - CPU Usage w/iowait for this server.

*~15% CPU with spine, ~50% CPU using cmd.php

Also, the apache stats checks work fine when using my test VM (CentOS 7/Cacti 0.8.8b/spine 0.8.8b)

Perhaps something in the database is bad? I've been told we are using an import of some ancient cacti that was running on an old PowerMac G5. Apparently the implementation was more or less directly moved to Ubuntu, and the upgrades/conversions were run, leaving everything in a half broken state (mostly all working now, save for some barcoding). I just did the 0.8.7i > 0.8.8b update the other day, per cacti documentation, but it didn't help.

Also, I was under the impression that I should have a settings page for plugins after running the 0.8.8b upgrade, but it's missing from the console page on the real server.

All of this may become a moot point in the future, as we're building an Ubuntu OpenStack setup, and once that's alive, I may just be rebuilding cacti cleanly there.

[derekcat]

No ideas guys?

The CPU usage numbers I gave were based on Cacti monitoring itself, and hence were terribly inaccurate for a second by second view of what was happening.

The way I had it setup for cmd.php, it would slam the CPU the entire time of the run. I turned it down to only 1 thread and 1 process, but that only got the CPU down to ~90%, and pushes the cycle time all the way to 270-90 seconds. (These tests were via htop)

Spine does better, but I still have no idea why it doesn't get results for ws_apachestats.pl
>_< So I can't use it at the moment.

Something has to be corrupt in the config or database, right? But where, I have no idea. (poking around with phpmyadmin was useless, because I'm not familiar with the internals of SQL databases, nor Cacti's use of such.

As a funny test, I fired up a VM, allocated only 1 core, and ran passmark.com's benchmarking tool. The CPU tested a couple hundred points higher than our actual Cacti/Nagios server's CPU's listed benchmark! (Using Virtualbox on a Mid 2012 Mac Mini, Windows 7 Pro VM)

Obviously, when a 1.83Ghz Core2Duo from 2007 is faster, our best solution is clear..