Lots of cuts in graph!!!

[charan89]

hi all..

Can someone help me.. i have almost 200host in my cacti and i find for few clients the graph cuts a lot... but actually there is no cut...

What should i do..?? The problem is not for all the host only for few..

Another problem is for a single host cacti doesnt record at all... Please help..

Thanks in advance...

[idle]

Same here.
Started at 21th.

[winni]

Welcome to the club. We've been having problems like this for years now. It gets a bit better when you use five minute polling instead of one minute polling, but there will still be white gaps and the "bar codes" don't entirely disappear.

I don't know what the problem is or how to fix it. And when I see how many times people post similar issues here on the forum - and don't get an answer that would fix the problem - I'm getting the feeling that this is something that one simply has to live with when you use Cacti.

What would interest me at this point is if running Cacti on, let's say, Windows Server would improve the situation or make it even worse. We're still running Cacti on two Ubuntu 12.04 LTS x64 servers here, and I'd like to rule out the operating system as a probable cause.

So is anybody running Cacti in a rather large, global SATELLITE BASED network and does NOT have gaps or "bar code" graphs? And if so, what hard- and software are you using?

[idle]

Welcome to the club. We've been having problems like this for years now.

Actually I'm using cacti for decade now, and never faced this one.

[winni]

Actually I'm using cacti for decade now, and never faced this one.

That is reassuring to hear!

May I ask how many hosts/devices you are monitoring and how many data sources do you query?
Do you use one minute or five minute polling?
What operating do you use? CentOS? Ubuntu? Windows? Any other?
Do you use Apache or nginx? Or IIS?

How many CPU cores and RAM does your server have?
Do you use SSDs or hard disks?

Any of this information might be very helpful for others to also get proper graphs. For reference, we're monitoring 247 hosts with 2251 graphs on one of our servers, which is an Octo-Core Xeon machine 16 GB RAM and an OCZ 256GB SSD. It's connected via One-Gig-Ethernet directly to one of our backbone switches. It runs on one minute polling and, according to the logs, Spine finishes a polling cycle in 21 seconds. Cacti's web interface is awfully slow on that server and although the graphs don't look as bad the one's that the original poster published here, there are white gaps every 9.5 hours that last for three minutes:



As you can see, there are not only white gaps, but also spikes that should not be there.

We also occasionally have "bar code"-style graphs as the OP, but this can usually be fixed by deleting and re-creating a host.

Our second Cacti server only uses a Quad Core Xeon CPU with only 4 GB RAM and an HDD instead of an SSD, but it uses five minute polling and interestingly enough does not show the gaps. But it also shows drops in the graph that shouldn't be there:



Frankly speaking, I no longer know what screws I can turn to fix this thing. I'd really appreciate any more detailed information about your working setup, maybe it will help us to fix our system.

[winni]

Just as an update, all of a sudden three of our switches began to produce "bar code" graphs last night:

[Lowb]

Im in the same basket, having cuts in the last graphs added to my system

[phalek]

I honestly urge everyone running a cacti system to install the Cacti poller template:

http://docs.cacti.net/usertemplate:host:cacti:poller

Gaps can have multiple causes, including:
- High load on a polled System ( i have that when there's backups runing on linux systems -> SNMP has a very low prio on these )
- Poller time exceeds the 300/60 seconds max runtime ( The load is distributed so some graphs may just see this causally, some other will always be hitting this )

The later one can be tracked down with that template. And some scripts causing a high runtme can actually be heavily optimized by caching mechanisms.

E.g. why getting data points from a system one-by-one instead of going in, getting everything at once and then just process the data locally ?

[winni]

Hi phalek,

Thanks for the suggestions - I've installed the poller template on both servers and will keep an eye on the result over the next hours.

E.g. why getting data points from a system one-by-one instead of going in, getting everything at once and then just process the data locally ?

Because, unfortunately, a bunch of the devices that we're monitoring, e.g. certain satellite modems, will only return one value at a time and don't respond to bulk requests.

However, I'm not convinced that the poller itself is the cause of the problems - according to the log files, both machines finish the polling cycle in less than a minute.

I'm beginning to suspect that our Juniper firewall cluster might cause some of the "bar code" style graphs, but at this point it's more of a gut feeling than something that I can actually prove.

In any case, that still doesn't fix the other problem that both of our servers have - the ultra slow response times of the web interface. The installation on the SSD server initially was very fast, but then rapidly degraded over the weeks.

[winni]

What is the unit behind "Export Duration"? Seconds? Milliseconds? The server currently shows an export duration of 792.55, but I don't know what this number is supposed to tell me.

[phalek]

According to the PHP code, this would be seconds.

list($micro,$seconds) = split(" ", microtime());
$start = $seconds + $micro;
list($micro,$seconds) = split(" ", microtime());
$end = $seconds + $micro;

ExportDuration = round($end - $start,4):

[phalek]

Is this a pure SSD installation ?

Did you start a "top" command and look at it through the poller run ?

Is your web-interface always slow, or just when the poller runs ?

[winni]

Oh, okay. So exporting 18.06k graphs in almost 12 minutes is something I should be concerned about, is that right?

[phalek]

not if your web-interface is responding fine and you don't have gaps ...

or this is being done just once a day or at least hourly.

[Tolaris]

Winni, are you by chance using Boost? I have found Boost actually reduces web interface performance when used with SSDs. The extra caching doesn't help much when you have fast write cycles.