SOLVED : Some of my graphs don't contain any data anymore...

Post general support questions here that do not specifically fall into the Linux or Windows categories.

Moderators: Developers, Moderators

Post Reply
FredX17
Posts: 8
Joined: Wed Mar 29, 2006 6:35 am

SOLVED : Some of my graphs don't contain any data anymore...

Post by FredX17 »

Hi,

Since a few days, some of my graphs don't contain any data anymore (the others are posted normally), although RRD's files are updated every 5 min. This problem appeared suddenly, right in the middle of the night (nobody could modify something...).

When I launch manually the poller.php (under root), I get the following error messages :

[root@lnx1 cacti]# No log handling enabled - turning on stderr logging
Cannot find module (NONE): At line 0 in (none)
No log handling enabled - turning on stderr logging
Cannot find module (NONE): At line 0 in (none)

And this is my cacti.log :
03/29/2006 01:55:22 PM - SYSTEM STATS: Time:20.0327 Method:cactid Processes:1 Threads:2 Hosts:109 HostsPerProcess:109 DataSources:2167 RRDsProcessed:906
03/29/2006 02:00:21 PM - SYSTEM STATS: Time:18.9916 Method:cactid Processes:1 Threads:2 Hosts:109 HostsPerProcess:109 DataSources:2167 RRDsProcessed:906
03/29/2006 02:05:26 PM - SYSTEM STATS: Time:23.9990 Method:cactid Processes:1 Threads:2 Hosts:109 HostsPerProcess:109 DataSources:2167 RRDsProcessed:906
03/29/2006 02:10:22 PM - SYSTEM STATS: Time:20.0331 Method:cactid Processes:1 Threads:2 Hosts:109 HostsPerProcess:109 DataSources:2167 RRDsProcessed:906
03/29/2006 02:15:22 PM - SYSTEM STATS: Time:20.0116 Method:cactid Processes:1 Threads:2 Hosts:109 HostsPerProcess:109 DataSources:2167 RRDsProcessed:906

I tried several things (restart the machine, clear the poller cache, change the number of threads), but no improvements...

Is anyone have an idea ?

Thanks in advance,

Fred
Last edited by FredX17 on Mon Apr 24, 2006 3:12 am, edited 1 time in total.
User avatar
BSOD2600
Cacti Moderator
Posts: 12171
Joined: Sat May 08, 2004 12:44 pm
Location: USA

Post by BSOD2600 »

You using cacti 0.8.6g (or older)? I recall there was a patch to fix that error "Cannot find module (NONE):".

As for no graphs, the data is in the rrd file. Turn on graph debugging and see if there any errors. Post the rrdtool graph command when it's broken.
FredX17
Posts: 8
Joined: Wed Mar 29, 2006 6:35 am

Post by FredX17 »

I'm using cacti 0.8.6h and this is the graph debugging :

RRDTool Command:

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="SYNER-LAON1 - Trafic - Fa0/0.2 (vers PE)" \
--rigid \
--base=1000 \
--height=120 \
--width=600 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="bits/sec" \
DEF:a="/var/www/html/cacti/rra/synerlaon1_traffic_in_1200.rrd":traffic_in:AVERAGE \
DEF:b="/var/www/html/cacti/rra/synerlaon1_traffic_in_1200.rrd":traffic_out:AVERAGE \
CDEF:cdefa=a,8,* \
CDEF:cdefe=b,8,* \
AREA:cdefa#00CF00:"Entrant" \
GPRINT:cdefa:LAST:" Actuel \:%8.2lf %s" \
GPRINT:cdefa:AVERAGE:" Moyen \:%8.2lf %s" \
GPRINT:cdefa:MAX:" Maxi \:%8.2lf %s\n" \
LINE2:cdefe#002A97:"Sortant" \
GPRINT:cdefe:LAST:" Actuel \:%8.2lf %s" \
GPRINT:cdefe:AVERAGE:" Moyen \:%8.2lf %s" \
GPRINT:cdefe:MAX:" Maxi \:%8.2lf %s"

RRDTool Says:

OK
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Post by gandalf »

1. Switch Settings->Logging Level to DEBUG for one polling cycle. Try to find the OID/script that's worrying you in log/cacti.log. You should find the value returend from the target (if any). Proceed to step 2 if this works
2. You've got data from the target, fine. Now find the "rrdtool update" statement some lines below for the related rrd file. If this is there, fine. Please proceed to step 3.
3. cacti "tried" to update the rrd file. But lack of permission may fail this request. Please copy the whole "rrdtool update" statement to cli when running as cactiuser (the user, that runs the poller from crontab). You may need to adjust the timestamp (replace by U denoting Now). If this works, proceed to step 4.
4. Updating rrd file works, fine. But a given MAXIMUM value may chop it off. So please check the MAXIMUM value in two places: First: the related data source of the used Data Template. Second: use rrdtool info on the rrd file in question at pay attention to the first 20 lines to find the current MAXIMUM values. When changing the data source, you must manually rrdtool tune all existing rrd files!
5. Final check: rrdtool fetch <rrd file> AVERAGE will show you all values of the last day. Please check the last few lines for the last updates. If there are meaningful readings, well done. If not, rrd file may be ok but graphing may fail (this will require some more sophisticated procedures, starting with the whole rrdtool graph statement to e found when clicking the wrench next to the graph)

hth
Reinhard
FredX17
Posts: 8
Joined: Wed Mar 29, 2006 6:35 am

Post by FredX17 »

Hi,

This is the cacti.log, with logging level to DEBUG :

04/20/2006 12:00:03 PM - CACTID: Poller[0] Host[6] DS[740] SNMP: v2: xx.xx.xx.xx, dsname: 5min_cpu, oid: .1.3.6.1.4.1.9.2.1.58.0, value: 5
04/20/2006 12:00:03 PM - PHPSVR: Poller[0] ERROR: Input Expected, Script Server Terminating
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "insert into poller_time (poller_id, start_time, end_time) values (0, NOW(), NOW())"
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "replace into settings (name,value) values ('date',NOW())"
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_id,end_time from poller_time where poller_id = 0"
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_output.output, poller_output.time, poller_output.local_data_id, poller_item.rrd_path, poller_item.rrd_name, poller_item.rrd_num from (poller_output,poller_item) where (poller_output.local_data_id=poller_item.local_data_id and poller_output.rrd_name=poller_item.rrd_name)"
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select data_template_rrd.data_source_name, data_input_fields.data_name from (data_template_rrd,data_input_fields) where data_template_rrd.data_input_field_id=data_input_fields.id and data_template_rrd.local_data_id=5"
04/20/2006 12:00:03 PM - POLLER: Poller[0] Parsed MULTI output field '1min:0.35' [map 1min->load_1min]
04/20/2006 12:00:03 PM - POLLER: Poller[0] Parsed MULTI output field '5min:0.18' [map 5min->load_5min]
04/20/2006 12:00:03 PM - POLLER: Poller[0] Parsed MULTI output field '10min:0.15' [map 10min->load_15min]
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select data_template_rrd.data_source_name, data_input_fields.data_name from (data_template_rrd,data_input_fields) where data_template_rrd.data_input_field_id=data_input_fields.id and data_template_rrd.local_data_id=203"
04/20/2006 12:00:03 PM - POLLER: Poller[0] Parsed MULTI output field '1min:0.35' [map 1min->load_1min]
04/20/2006 12:00:03 PM - POLLER: Poller[0] Parsed MULTI output field '5min:0.18' [map 5min->load_5min]
04/20/2006 12:00:03 PM - POLLER: Poller[0] Parsed MULTI output field '10min:0.15' [map 10min->load_15min]
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "delete from poller_output where local_data_id='12' and rrd_name='traffic_in' and time='2006-04-20 12:00:02'"
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "delete from poller_output where local_data_id='13' and rrd_name='traffic_in' and time='2006-04-20 12:00:02'"
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "delete from poller_output where local_data_id='14' and rrd_name='traffic_in' and time='2006-04-20 12:00:02'"
04/20/2006 12:00:03 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "delete from poller_output where local_data_id='15' and rrd_name='traffic_in' and time='2006-04-20 12:00:02'"
etc...

Is this the cause of my problem ?
Last edited by FredX17 on Fri Apr 21, 2006 6:19 am, edited 1 time in total.
FredX17
Posts: 8
Joined: Wed Mar 29, 2006 6:35 am

Post by FredX17 »

In the cacti.log, I don't see the rrdtool update for the files concerned... but I see the others (which are OK) :

04/20/2006 04:25:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/laon_cisco_3750_traffic_in_12.rrd --template traffic_in:traffic_out 1145543102:1608424941:3399681984
04/20/2006 04:25:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/laon_cisco_3750_traffic_in_13.rrd --template traffic_in:traffic_out 1145543102:178194113:2311557569
04/20/2006 04:25:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/laon_cisco_3750_traffic_in_14.rrd --template traffic_in:traffic_out 1145543102:382076182:2484369152
04/20/2006 04:25:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/laon_cisco_3750_traffic_in_15.rrd --template traffic_in:traffic_out 1145543102:4068859189:1802687263

This is the output of the rrdtool info command on a file which is OK :

-bash-3.00$ rrdtool info synftl1_traffic_in_220.rrd
filename = "synftl1_traffic_in_220.rrd"
rrd_version = "0001"
step = 300
last_update = 1145544601
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 600
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 1.0000000000e+08
ds[traffic_in].last_ds = "2009712185"
ds[traffic_in].value = 1.6924804767e+06
ds[traffic_in].unknown_sec = 0
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 600
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 1.0000000000e+08
ds[traffic_out].last_ds = "1119430242"
ds[traffic_out].value = 1.7240240567e+06
ds[traffic_out].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 6.3937259232e+06
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[1].cdp_prep[1].value = 8.5748432870e+06
rra[1].cdp_prep[1].unknown_datapoints = 0

... and on a file which have the problem :

-bash-3.00$ rrdtool info synerlaxou2_traffic_in_1221.rrd
filename = "synerlaxou2_traffic_in_1221.rrd"
rrd_version = "0001"
step = 300
last_update = 1145544301
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 600
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 1.0000000000e+08
ds[traffic_in].last_ds = "968935840"
ds[traffic_in].value = 0.0000000000e+00
ds[traffic_in].unknown_sec = 1
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 600
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 1.0000000000e+08
ds[traffic_out].last_ds = "2239671471"
ds[traffic_out].value = 0.0000000000e+00
ds[traffic_out].unknown_sec = 1
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = NaN
rra[1].cdp_prep[0].unknown_datapoints = 3
rra[1].cdp_prep[1].value = NaN
rra[1].cdp_prep[1].unknown_datapoints = 3

Any idea ?
Last edited by FredX17 on Fri Apr 21, 2006 6:16 am, edited 2 times in total.
FredX17
Posts: 8
Joined: Wed Mar 29, 2006 6:35 am

Post by FredX17 »

And the output of the poller.php command launched with the cactiuser, in DEBUG level :
...
04/20/2006 05:32:37 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/sondes2950lb_traffic_in_78.rrd --template traffic_in:traffic_out 1145547156:155586275:175721765
OK u:0.00 s:0.01 r:0.13
04/20/2006 05:32:37 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/sondes2950lb_traffic_in_79.rrd --template traffic_in:traffic_out 1145547156:2210628668:2582402874
OK u:0.00 s:0.01 r:0.13
04/20/2006 05:32:37 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/sondes2950lb_traffic_in_75.rrd --template traffic_in:traffic_out 1145547156:235708090:487754696
OK u:0.00 s:0.01 r:0.13
04/20/2006 05:32:37 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/sondes2950lb_traffic_in_76.rrd --template traffic_in:traffic_out 1145547156:450033538:173182879
OK u:0.00 s:0.01 r:0.13
04/20/2006 05:32:37 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/sondes2950lb_traffic_in_77.rrd --template traffic_in:traffic_out 1145547156:771017806:151555408
OK u:0.00 s:0.01 r:0.13
Waiting on 1/1 pollers.
Waiting on 1/1 pollers.
Waiting on 1/1 pollers.
Waiting on 1/1 pollers.
Waiting on 1/1 pollers.
...
FredX17
Posts: 8
Joined: Wed Mar 29, 2006 6:35 am

Post by FredX17 »

I'm wondering whether there is not a problem of memory limit in php... I saw in others posts that we could increase the max value (in php.ini ???) : can somebody explain to me how to proceed ?

Thanks in advance,

Fred
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Post by gandalf »

For the last question, see /etc/php.ini or the like:

Code: Select all

;;;;;;;;;;;;;;;;;;;
; Resource Limits ;
;;;;;;;;;;;;;;;;;;;

max_execution_time = 30     ; Maximum execution time of each script, in seconds
max_input_time = 60     ; Maximum amount of time each script may spend parsing request data
;memory_limit = 8M      ; Maximum amount of memory a script may consume (8MB)
memory_limit = 64M      ; Maximum amount of memory a script may consume (64MB recommended for cacti)
Please tell us more about the failing Graphs (What Data Input Methods/Data Queries, Script_Server used, ...)
Reinhard
FredX17
Posts: 8
Joined: Wed Mar 29, 2006 6:35 am

Post by FredX17 »

Pb solved... I used the patch rrd.php provided by TheWitness (force the RRDtool to not use pipes : http://forums.cacti.net/viewtopic.php?t ... p&start=30) and now, all the graphs contain data normally.

Thank you lvm & TheWitness,

Fred
ulerkeket
Posts: 2
Joined: Tue Sep 23, 2008 8:30 pm

Post by ulerkeket »

gandalf wrote: 4. Updating rrd file works, fine. But a given MAXIMUM value may chop it off. So please check the MAXIMUM value in two places: First: the related data source of the used Data Template. Second: use rrdtool info on the rrd file in question at pay attention to the first 20 lines to find the current MAXIMUM values. When changing the data source, you must manually rrdtool tune all existing rrd files!
hth
Reinhard
Thank you for the pointer. One of my graph is not showing because the result was above rrd max value. It's normal now after manually updating the rrd.
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest