Polling time increased dramatically

[princea]

All,

On one of my old Cacti servers - been running for about 15 months now with no issues, I have noticed in the logs today:-

SYSTEM STATS: Time:1186.9415 Method:cmd.php Processes:5 Threads:N/A Hosts:259 HostsPerProcess:52 DataSources:17500 RRDsProcessed:9296

This is the first time I have seen the server take so long?? Now my weathermaps are also logging issues along the line of:-

** on config line 224 had no valid data, according to weatherMapDataSource_rrd

I have no idea why all of a sudden this is occuring? Below is my setup - any help suggestions would be really helpfull:

Cacti Version - 0.8.6h
Plugin Architecture - 1.0
Poller Type - CMD.php
Server Info - Linux 2.6.9-42.0.3.EL
Web Server - Apache/2.0.52 (CentOS)
PHP - 4.3.9
PHP Extensions - yp, xml, wddx, tokenizer, sysvshm, sysvsem, standard, sockets, shmop, session, pspell, posix, pcre, overload, mime_magic, iconv, gmp, gettext, ftp, exif, dio, dbx, dba, curl, ctype, calendar, bz2, bcmath, zlib, openssl, apache2handler, gd, mysql, snmp, eAccelerator
MySQL - 4.1.20
RRDTool - 1.2.11
SNMP - 5.1.2
Plugins

• PHP Network Weathermap (weathermap - v0.92)
  Global Plugin Settings (settings - v0.2)
  Thresholds (thold - v0.3.0)
  Network Tools (tools - v0.2)
  Update Checker (update - v0.3)
  Host Info (hostinfo - v0.1)

[TheWitness]

Likely a network conjestion issue or a down network device or two. This is why we have Cactid. It is much more resilient to network/device issues.

The latest version of Cactid is 0.8.6k which is only available via the announcement's forum at the moment.

TheWitness

[princea]

Thanks Witness,

which leads me neatly into my next question - I have tried to use the Cactid poller. But when ever I change it in >settings>poller>poller type and change from cmd.php to cactid - nothing happens.

Graphs stop populating, THOLD & Weathermap stops??? It's not until I change it back to cmd.php everything starts working again. Is there a sequence of events I am missing??

[TheWitness]

Well, first you must compile and install it. Once that is done, you have to setup the cactid.conf file. Once that's done, you need to test it:

./cactid -V 3

Once you have confirmed that it is collecting data, then you change the poller type from the Web Interface.

TheWitness

[princea]

Witness,

the cactid.conf file is configured with the correct info - checked against my config.php file. When I run ./cactid -V 3 I get the below output?

[root@eunetcat1 cactid]# ./cactid -V 3
CACTID 0.8.6h Copyright 2002-2006 by The Cacti Group

Usage: cactid [options] [firstid lastid]

Options:

-h/--help Show this brief help listing
-f/--first=X Start polling with host X
-l/--last=X End polling with host X
-p/--poller=X Poller ID = X
-C/--conf=F Read Cactid configuration from file F
-O/--option=S:V Override DB settings 'set' with value 'V'
-R/--readonly This Cactid run is readonly with respect to the database
-S/--stdout Logging is performed to the standard output
-V/--verbosity=V Set logging verbosity to <V>
--snmponly Only do SNMP polling: no script stuff

Either both of --first/--last must be provided, or neither can be,
and in their absense, all hosts are processed.

Without the --conf parameter, cactid searches for its cactid.conf
file in the usual places.

Verbosity is one of NONE/LOW/MEDIUM/HIGH/DEBUG or 1..5

Runtime options are read from the 'settings' table in the Cacti
database, but they can be overridden with the --option=S:V
parameter.

Cactid is distributed under the Terms of the GNU Lessor
General Public License Version 2.1. (http://www.gnu.org/licenses/lgpl.txt)
For more information, see http://www.cacti.net

[TheWitness]

Way old version. Check the Announcements forum for 0.8.6k. Released about a week ago.

TheWitness

[princea]

Witness,

Thanks for that - that is all now squared away and working. Just for my information only more than anything - what are the most optimal settings for the cactid process, currently I have:-

Maximum Concurrent Poller Processes: 5
Maximum Threads per Process: 5
Number of PHP Script Servers: 2
Script and Script Server Timeout Value: 25
The Maximum SNMP OID's Per SNMP Get Request: 20

As poller log info:-
** SYSTEM STATS: Time:226.3539 Method:cactid Processes:5 Threads:5 Hosts:250 HostsPerProcess:50 DataSources:17179 RRDsProcessed:8777

[TheWitness]

My suspicion is this. Do the following:

Code: Select all

du -k /var/www/html/cacti/rra

Tell me also how much memory on the box?

TheWitness

[princea]

Witness,

output of command:-

2693688 /var/www/html/rra


[**@** ~]# cat /proc/meminfo
MemTotal: 1035724 kB
MemFree: 148380 kB
Buffers: 28496 kB
Cached: 599816 kB
SwapCached: 1708 kB
Active: 386416 kB
Inactive: 470968 kB
HighTotal: 131048 kB
HighFree: 252 kB
LowTotal: 904676 kB
LowFree: 148128 kB
SwapTotal: 779144 kB
SwapFree: 776084 kB
Dirty: 39908 kB
Writeback: 0 kB
Mapped: 255772 kB
Slab: 19008 kB
CommitLimit: 1297004 kB
Committed_AS: 606572 kB
PageTables: 1852 kB
VmallocTotal: 106488 kB
VmallocUsed: 1668 kB
VmallocChunk: 104296 kB
HugePages_Total: 0
HugePages_Free: 0
Hugepagesize: 4096 kB
[**@** ~]#

[TheWitness]

Once your box has 4GBytes of Memory, you will be in much better shape.

TheWintess