[SOLVED] Some graphs missing data

[9Nails]

This might be a case of not enough memory, but I'm not keen to the steps on how to best determine that. What I'm seeing is that after a reboot, my Cacti server slowly loses memory over time and stays below 100 MB free. Perhaps, as a consequence, my graphs are losing data. It seems at best some graphs can only keep 5 hours of data, while others enjoy a fairly long history. Especially including the Cacti Server localhost.

I've looked at increasing RAM from 512 to 2 GB, to no avail.

I've checked that the file permissions are the same. Permissions for the images below are:

(Working graph)
drwxr-xr-x 2 apache apache 4096 Jul 24 13:45 9
-rw-r--r-- 1 root root 3081032 Jul 27 07:57 917.rrd

(Failing graph)
drwxr-xr-x 2 apache apache 4096 Jul 27 01:03 22
-rw-r--r-- 1 root root 3081032 Jul 27 07:58 163.rrd

I'm not noticing any errors in my logs, just a lot of repeating this similiar line:
07/27/2009 07:59:05 AM - SYSTEM STATS: Time:3.5829 Method:spine Processes:1 Threads:1 Hosts:79 HostsPerProcess:79 DataSources:715 RRDsProcessed:397

I'm not sure where else to check. Any suggestions?

My system Specifics:
Cacti Version - 0.8.7e
Plugin Architecture - 2.5
Poller Type - Cactid v
Server Info - Linux 2.6.9-78.0.22.plus.c4smp
Web Server - Apache/2.0.63 (CentOS)
PHP - 5.1.6
PHP Extensions - libxml, xml, wddx, tokenizer, sysvshm, sysvsem, sysvmsg, standard, SimpleXML, sockets, SPL, shmop, session, Reflection, pspell, posix, mime_magic, iconv, hash, gmp, gettext, ftp, exif, date, curl, ctype, calendar, bz2, zlib, pcre, openssl, apache2handler, gd, ldap, mysql, mysqli, PDO, pdo_mysql, pdo_sqlite, snmp, eAccelerator
MySQL - 5.0.68
RRDTool - 1.2.23
SNMP - 5.1.2
Plugins

• Global Plugin Settings (settings - v0.5)
  Large Site Performane Booster (boost - v2.4)
  Network Tools (tools - v0.3)
  Syslog Monitoring (syslog - v0.5.2)
  Device Tracking (mactrack - v1.1)
  Login Page Mod (loginmod - v1.0)
  Update Checker (update - v0.4)
  FlowView (flowview - v0.6)
  Host Info (hostinfo - v0.2)
  Error Images (errorimage - v0.1)
  PHP Network Weathermap (weathermap - v0.96a)
  docs
  Cacti-ReportIt (reportit - v0.6.1)
  Router Configs (routerconfigs - v0.2)
  WMI Query (wmi - v0.1)

[BSOD2600]

General cacti debugging http://docs.cacti.net/manual:087:4_help ... #debugging

As for memory loss... I'm not a linux person, but there must be commands/programs you can run to see what is consuming all the memory.

[9Nails]

Thanks for the general debugging steps BSOD2600. Though they didn't perfectly match my situation, they did help me get some understanding. I've looked for errors, and nothing ever stood out. I even ran the logs in debug and got 3 MB of data in 3 minutes. But no errors in that file. My situation is frustrating because everything is working, except I'm losing historical data from my graphs. I'm not sure if there's a clean-up process that removes old data from the graphs.

What I've done for test is tuned PHP to allow more memory (Up from 32 to 128 MB) and tweak Spine to allow more threads and script servers to run. Spine is set to Poll every 1 minute with 1 thread process. My thinking on this is that it's just a resource issue and Spine isn't getting enough of the resources to get done in time.

[9Nails]

What I've done for test is tuned PHP to allow more memory (Up from 32 to 128 MB) and tweak Spine to allow more threads and script servers to run. Spine is set to Poll every 1 minute with 1 thread process. My thinking on this is that it's just a resource issue and Spine isn't getting enough of the resources to get done in time.

I only had about 11 graphs that weren't working properly. After the changes made above, and double checking my Cacti setup I was able to find success.

I've deleted all the affected graphs and all associated graph data, leaving the devices only. Then I recreated the graphs for the devices and monitored the changes. After 1 day, I was able to determine that the graphs are now working properly.

The memory issues with Linux appear to be correct. As it seems, after 1AM, Linux will shift more memory into a cache to create a disk cache of sorts. This is supposed to save on disk drive life and improve performance. Drive writes are sent out to the disk periodically so I shouldn't lose any data, and get a better working system. That's the theory any way.