RRD Files Stopped Updating for No Reason

Post general support questions here that do not specifically fall into the Linux or Windows categories.

Moderators: Developers, Moderators

eshine
Posts: 30
Joined: Fri Feb 25, 2005 11:38 am
Location: Sao Paulo - Brazil

Post by eshine »

Hi Larry,

Maybe could be thold... I could try upgrade to 0.86f without thold and see if the problem is fixed. :-?

I've checked the poller entries and in fact there are missing entries. Then I've truncated table poller_item manually (successfully) and repopulate it using web "clear poller cache" command. But the poller_item entries keep the traffic lines I want out.
I don't know why repopulate() don't get the traffic row... see:

---
mysql> select * from data_local where host_id="192";
+------+------------------+---------+---------------+------------+
| id | data_template_id | host_id | snmp_query_id | snmp_index |
+------+------------------+---------+---------------+------------+
| 6009 | 111 | 192 | 0 | |
| 2911 | 18 | 192 | 0 | |
| 4528 | 101 | 192 | 1 | 1 |
| 6010 | 110 | 192 | 1 | 1 |
+------+------------------+---------+---------------+------------+

##fail##:
mysql> select name_cache,local_data_id from data_template_data where local_data_id = "6010";
+----------------------------+---------------+
| name_cache | local_data_id |
+----------------------------+---------------+
| Ap-Amesp_Butanta - Traffic | 6010 |
+----------------------------+---------------+

mysql> select * from poller_item where local_data_id = "6010";
Empty set (0.00 sec)

##works##:
mysql> select name_cache,local_data_id from data_template_data where local_data_id = "6009";
+-----------------------------+---------------+
| name_cache | local_data_id |
+-----------------------------+---------------+
| Ap-Amesp_Butanta - Rx Power | 6009 |
+-----------------------------+---------------+

mysql> select local_data_id,host_id from poller_item where local_data_id = "6009";
+---------------+---------+
| local_data_id | host_id |
+---------------+---------+
| 6009 | 192 |
+---------------+---------+
---

How can I debug this?

Many thanks for your support.
rgds,
Edgar Shine
CactiJohn
Posts: 1
Joined: Thu Sep 29, 2005 1:29 pm
Location: Pensacola, FL
Contact:

Possible information on this problem

Post by CactiJohn »

Hello,
I have had the same problem as others.. I am not going into much detail about specific steps and/or versions.

Summary:
My cacti was working.. updated Cacti a week ago..
Everything fine.
Updated rrdtool and switched from cmd.php to cactid.
At this time, ALL graphs ceased.

Read many forum entries.. tried multiple things including the new rrd.php.
Finally backed off on my rrdtool version and restored the original rrd.php and switched my polling back to cmd.php.

Only a few graphs show up now (5%).

Possible enlightening Info that I found out.
All graphs that are not working insist that they do not have data-source 'traffic_in' in their rrd file.

I have done an 'rrdtool info' on those rrd file and found that the DS 'traffic_in' is there.

** However, the difference that I found on Working-graphs and NON-Working-graphs is 'Case' difference.
i.e.
where cacti-datasource-debug states that 'traffic_in' does not exist,
the 'rrdtool info ###.rrd' shows 'Traffic_In'
Notice the Proper Caps.

on graphs that do work.. the cacti-datasource-debug AND the 'rrdtool info' both show 'traffic_in' for the DS.. (lowercase).

So,
I still have DS's that can't be found for some reason.
If anyone has any information on this.. I would appreciate greatly.

John.
raX
Lead Developer
Posts: 2243
Joined: Sat Oct 13, 2001 7:00 pm
Location: Carlisle, PA
Contact:

Post by raX »

It looks like there might be a traffic template floating around out there that has a renamed RRD data source name. First, I would go through your traffic-related data templates and make sure that data source name field reads "traffic_in" or "traffic_out". Secondly, I wrote a quick script that will mass-rename RRD data sources for you.

Code: Select all

#!/bin/bash

if (( $# < 3 ))
then
  echo "Usage: rrdrename.sh OLD_RRD NEW_RRD FILENAME";
  exit
fi

RRD_OLD_NAME=$1
RRD_NEW_NAME=$2

until [ -z "$3" ]  # Until all parameters used up...
do
  echo "+ rrdtool tune $3 --data-source-rename $RRD_OLD_NAME:$RRD_NEW_NAME"
  rrdtool tune $3 --data-source-rename $RRD_OLD_NAME:$RRD_NEW_NAME
  shift
done
To use, do something like:

Code: Select all

./rrdrename.sh Traffic_In traffic_in *traffic*.rrd
./rrdrename.sh Traffic_Out traffic_out *traffic*.rrd
Be sure to backup your RRD files first!

-Ian
eshine
Posts: 30
Joined: Fri Feb 25, 2005 11:38 am
Location: Sao Paulo - Brazil

Post by eshine »

Larry,

I've reinstalled Cacti (0.8.6e) without THold and the problem with the clear poller cache function remains the same.

The problem seems be related to the xml scripts I have, when I use the original interface.xml, it works fine (with 1700 poller itens), when I use my customized xml script it points to 3400 poller itens (verified using mysql line command) and poller view function doesn't work anymore.
rgds,
Edgar Shine
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

EShine,

PM me your IM information.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
eshine
Posts: 30
Joined: Fri Feb 25, 2005 11:38 am
Location: Sao Paulo - Brazil

Post by eshine »

Hi TheWitness,

I upgraded my env to Cacti 0.8.6g today. Think is more reasonable work on updated versions instead of digging problems on old ones.

Cacti behavior is the same in this new version. Since the problem seem be related to the XML script, I tried the following:
1) Create a new data query, "Aperto Device", linked to a XLM called aperto_interface.xml
2) Change the host template "Aperto SU" to point this data query
3) Remove and re-create a device "Ap-Amesp_Itatiaia"
4) Create Graphs for it
5) Replace the original interface.xml script to have a functional "clear poller cache"
6) Do a clear poller cache and see if the new data is being cached.

Well... it didn't work. The traffic is not being polled. And also not cached in the poller item.

Additional info, maybe it's relevant..
- /usr/local/cacti/rra is a symlink to /usr/local/cactidb/rra. cactiuser:cacti
is the owner/group for both dir cacti/cactidb
- upgrade was done in the following sequence: 0.8.6e -> 0.8.6f -> 0.8.6g

Thanks for your support.
Attachments
data_query.jpg
data_query.jpg (89.55 KiB) Viewed 4748 times
This image show that we have just two data sources being cached
This image show that we have just two data sources being cached
poller_cache_itatiaia.jpg (32.57 KiB) Viewed 4748 times
aperto_interface.xml
xml file
(10.47 KiB) Downloaded 161 times
rgds,
Edgar Shine
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

What are the '#' signs in the XML script:
#
- <ifName>
#
<name>Name (IF-MIB)</name>
#
<method>walk</method>
#
<source>value</source>
#
<direction>input</direction>
#
<oid>.1.3.6.1.2.1.31.1.1.1.1</oid>
#
</ifName>
#
- <ifAlias>
#
<name>Alias (IF-MIB)</name>
#
<method>walk</method>
#
<source>value</source>
#
<direction>input</direction>
#
<oid>.1.3.6.1.2.1.31.1.1.1.18</oid>
#
</ifAlias>
I'll need more information to dig in deeper.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

Also,

Please explain the following in more detail:
5) Replace the original interface.xml script to have a functional "clear poller cache"
TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
eshine
Posts: 30
Joined: Fri Feb 25, 2005 11:38 am
Location: Sao Paulo - Brazil

Post by eshine »

Sorry, wrong file... in fact, the real cacti xml script file is just a `grep -v "#"` of the attached xml file.

The "original interface.xml" is the interface.xml that is shipped in the cacti tgz pack. When I use it, the clear poller cache function in "console -> system utilities" seems to work (at least return me a poller view screen). When I use a customized interface.xml, with some aditional lines in interface config, the clear poller cache don't return me a screen.

I'm attaching the modified interface.xml and the system utilities screens.

Thanks for your support.
Attachments
This is the screen that Cacti presents me if I use the modified interface.xml script (attached in this msg)
This is the screen that Cacti presents me if I use the modified interface.xml script (attached in this msg)
after_click.jpg (44.63 KiB) Viewed 4730 times
interface.xml
the interface.xml modified script
(18.02 KiB) Downloaded 859 times
when I use the interface.xml which is shipped in Cacti tgz package, it shows me this screen doing a clear poller cache. Nota that I'm using the same Cacti installation for all tests.
when I use the interface.xml which is shipped in Cacti tgz package, it shows me this screen doing a clear poller cache. Nota that I'm using the same Cacti installation for all tests.
poller_cache.jpg (127.08 KiB) Viewed 4730 times
rgds,
Edgar Shine
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

Please do not modify the original Interface.xml file. Doing so will cause problems for others if you ever publish a template or ever import a template.

So, create a new one and then create new Data and Graph templates based upon it. I will now look at your XML file for the moment.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

In addition, when clearing your poller cache, review your apache log directory for the contents of the error.log file and post what it returns.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
axelilly
Posts: 27
Joined: Thu Aug 04, 2005 12:23 pm

Fixed yet?

Post by axelilly »

I applied all patches for 0.8.6f and this has not corrected the problems with not being able to add any new graphs to cacti. Has this issue been corrected in 0.8.6g? I don't want to upgrade yet if this still hasn't been fixed. 0.8.6f still seems really flaky. For instance, so graphs just stop updating out of the blue and log just shows that RRD suddenly won't accept the values anymore. Is cacti essentially broken right now? What's going on here?
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

Make sure PHP maximum memory is greater than 8mbytes. Then "truncate" the poller_output table.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
jrichardson
Cacti User
Posts: 66
Joined: Tue Mar 22, 2005 10:11 am

Is this fixed?

Post by jrichardson »

I'm running 0.8.6c. And I see this happen every 10 days or so. Really annoying. Is this fixed in a current verion? If so, at what version was it fixed so I can recommend upgrading/testing.

What exactly caused this? I don't understand how clearing the poller cache table is 'fixing' everything. Is it because I'm polling thousands of interfaces? Is this fixed with the C-based poller?
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

jrichardson,

This is likely caused by a memory problem for most users. When you say every X day's. How do you fix it? Please be more specific.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest