"Holes" in my graphs

[wvankuyk]

The more hosts I have been adding the more I see this....[See Attached] I dont know why this is happening is there any way I can "smooth" over the gaps?

General Information:

Date Tue, 06 Jan 2009 13:49:50 -0500
Cacti Version 0.8.7b
Cacti OS win32
SNMP Version net-snmp
RRDTool Version RRDTool 1.2.x
Hosts 17
Graphs 280
Data Sources Script/Command: 108
Script Query: 172
Total: 280

Poller Information:

Interval 300
Type cmd.php
Items Action[1]: 1285
Total: 1285
Concurrent Processes 20
Max Threads 25
PHP Servers 5
Script Timeout 25
Max OID 10
Last Run Statistics Time:136.4219 Method:cmd.php Processes:20 Threads:N/A Hosts:18 HostsPerProcess:1 DataSources:1285 RRDsProcessed:280

[gandalf]

The more hosts I have been adding the more I see this....[See Attached] I dont know why this is happening is there any way I can "smooth" over the gaps?

It is possible to do so by using specific rrdtool create parameters. BUT THIS IS NOT RECOMMENDED.
Better idea is to tackle to root cause of the dropouts.
See some basic help at the 2nd link of my sig.
YOu are running quite few hosts currently, but mostly on a script basis. I suggest upgrading to spine to allow for better usage of system resources. But the very first thing to pay attention to is the scripts timeout conditions.
We recently ran into this very badly. So please make sure, that script have enough time BUT make them abend if necessary to avoid problems with other hosts waiting in the process queue and never getting polled due to infinte script runtime
Reinhard

[wvankuyk]

YOu are running quite few hosts currently, but mostly on a script basis. I suggest upgrading to spine to allow for better usage of system resources. But the very first thing to pay attention to is the scripts timeout conditions.

Well all of my counters/Data is gathered via vbscript using WMI with the exception of one data point, I have found that using WMI counters and cooking the results myself work better then SNMP provided counters, but there is another reason.... my security department has issue with SNMP and as a rule we do not run it here, maybe there is an alternative?

WMI sometimes takes a REALLY long time to timeout and I dont know of any timeout/termination process that works when a process runs away. I have expanded the number of processes from 2 to 20 hoping that will help. I have also added "ping" logic to the script where it will ensure the host is alive before it timesout trying to attach itself.

I tried spine but I got nasty error messages.....

Code: Select all

01/06/2009 03:59:48 PM - SPINE: Poller[0] FATAL: Spine Encountered a Segmentation Fault (Spine thread)

[pacman]

We had this issue before. I was a database issue. Make sure that your database is running properly and check the writes.

[wvankuyk]

Any tests or indicators I can look for to verify database is functioning?

Every now and then I get a U from the script, and I dont know why that is happening either. Even with debug logging I dont see a reason..... In MRTG if you get a 0 it will carry over until the next "valid data" could cacti do this as well? If I force the script to output a 0 if no valid data?

[pacman]

Any tests or indicators I can look for to verify database is functioning?

Every now and then I get a U from the script, and I dont know why that is happening either. Even with debug logging I dont see a reason..... In MRTG if you get a 0 it will carry over until the next "valid data" could cacti do this as well? If I force the script to output a 0 if no valid data?

In my case, the problem was that there was a write limit to the databases thus, the more devices you add the more writes you needed and it got to a point that not all the devices were able to write their data and we ended up with gaps. I would suggest to look at the database and make sure there is not a write limit.

[wvankuyk]

[quote="pacman]I would suggest to look at the database and make sure there is not a write limit.[/quote]
Im new to the world of mysql how could I find this out?

[pacman]

[quote="pacman]I would suggest to look at the database and make sure there is not a write limit.

Im new to the world of mysql how could I find this out?[/quote][/quote]

http://www.mysqlperformanceblog.com/200 ... on-basics/

Google search is your friend.

[k-y]

Is there a general rule of thumb for when one should convert from using MyISAM to InnoDB?

[gandalf]

What magnitude of data sources are we talking about? More than 100k?
Reinhard

[mcdmagix]

Hey guys,

I'm new to Cacti and monitoring in general... So far I've managed to start creating graphs and get a general feel for how Cacti works..

However I'm getting the same gaps / holes in my graphs as the original poster...

I read someone suggested looking at my mySQL db for performance issues.. hrmm I thought all the data was written to rrd files not a mysql database? I also only have like 6 devices I'm monitoring, so it's hardly a big install at this stage.

Cacti is running on a CentOS 5.2 32bit install which is running on a VMware ESX server. The VM has 2x CPU's and 2GB or RAM allocated, the hard drives are running off a speedy SAN. So I can't really fault it as a hardware issue..

Any advice on where I could start looking to resolve this problem?

Thank you in advance.

[mcdmagix]

Hello again..

I downloaded a pre-built Cacti virtual appliance from rPath and it doesn't appear to suffer the same "gaps" as the Cacti machine I built myself...

http://www.rpath.org/project/cacti-appliance/

I must admit I didn't compile Cacti myself and instead downloaded an RPM... There might be something funny in my original environment vs pre-built virtual appliance.