SQL Exec: "truncate table poller_output"

jsingleton71 · Post by **jsingleton71** » Thu Jul 12, 2007 12:21 pm

After adding a new device (Cisco 3560) and creating new graphs for this device, I do not see the graphic for the graph. All other device graphs look ok - but new device graphs will not display properly. I have run all of the poller related scripts, checked ownership and permissions, nothing seems to work. We are well under the 10000 data source bug, so that is not the issue.

We are running cacti-0.8.6j with cmd.php on FreeBSD 4, php4, apache 1.3, rrdtool 1.0.41 -- cacti is patched to support the php4 snmp calls.

I have a some questions about the SQL Exec entry in cacti.log where is 'truncates table poller_output'

Is this an error or a process?
What is it doing exactly?
Could this be the cause of my problem?

Here is the tail of my last poller run:

# tail log/cacti.log
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_224.rrd --template traffic_in:traffic_out 1184260201:227565377:702154027
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_225.rrd --template traffic_out:traffic_in 1184260201:3232840:0
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_226.rrd --template traffic_in:traffic_out 1184260201:564391289:641701871
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_227.rrd --template traffic_out:traffic_in 1184260201:2133107715:4124034983
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_228.rrd --template traffic_out:traffic_in 1184260201:3818025678:385213496
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_229.rrd --template traffic_out:traffic_in 1184260201:4776098:1920591
07/12/2007 05:10:02 PM - SYSTEM STATS: Time: 1.2066 s, Method: cmd.php, Processes: 1, Threads: N/A, Hosts: 6, Hosts/Process: 6
07/12/2007 05:10:02 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_command.action, poller_command.command from poller_command where poller_command.poller_id=0"
07/12/2007 05:10:02 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "update settings set value='8' where name='path_html_export_ctr'"
07/12/2007 05:10:02 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "truncate table poller_output"
ux-utl01-atl-02# tail log/poller.log
OK u:0.00 s:0.01 r:1.12
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_226.rrd --template traffic_in:traffic_out 1184260201:564391289:641701871
OK u:0.00 s:0.01 r:1.12
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_227.rrd --template traffic_out:traffic_in 1184260201:2133107715:4124034983
OK u:0.00 s:0.01 r:1.12
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_228.rrd --template traffic_out:traffic_in 1184260201:3818025678:385213496
OK u:0.00 s:0.01 r:1.12
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_229.rrd --template traffic_out:traffic_in 1184260201:4776098:1920591
OK u:0.00 s:0.01 r:1.12
07/12/2007 05:10:02 PM - SYSTEM STATS: Time: 1.2066 s, Method: cmd.php, Processes: 1, Threads: N/A, Hosts: 6, Hosts/Process: 6

Here is the tail of my last cacti.log:

# tail log/cacti.log
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_224.rrd --template traffic_in:traffic_out 1184260201:227565377:702154027
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_225.rrd --template traffic_out:traffic_in 1184260201:3232840:0
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_226.rrd --template traffic_in:traffic_out 1184260201:564391289:641701871
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_227.rrd --template traffic_out:traffic_in 1184260201:2133107715:4124034983
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_228.rrd --template traffic_out:traffic_in 1184260201:3818025678:385213496
07/12/2007 05:10:02 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update /usr/support/htdocs/cacti/rra/swcsc01atl02_traffic_in_229.rrd --template traffic_out:traffic_in 1184260201:4776098:1920591
07/12/2007 05:10:02 PM - SYSTEM STATS: Time: 1.2066 s, Method: cmd.php, Processes: 1, Threads: N/A, Hosts: 6, Hosts/Process: 6
07/12/2007 05:10:02 PM - CMDPHP: Poller[0] DEBUG: SQL Assoc: "select poller_command.action, poller_command.command from poller_command where poller_command.poller_id=0"
07/12/2007 05:10:02 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "update settings set value='8' where name='path_html_export_ctr'"
07/12/2007 05:10:02 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "truncate table poller_output"

Can someone help me out with this one?

Post by **rony** » Thu Jul 12, 2007 12:29 pm

It's normal and not the cause of your problem.

What is probably wrong is an issue with the rra settings or the data template associated with the graphs that you have connected to the new host.

jsingleton71 · Post by **jsingleton71** » Thu Jul 12, 2007 12:44 pm

rony wrote:It's normal and not the cause of your problem.

What is probably wrong is an issue with the rra settings or the data template associated with the graphs that you have connected to the new host.

Ok. But if I created the graphs in the exact same fashion as the others that are working, then I am not sure how to troubleshoot this?

I simply added a new device, then got the snmp info showing that it can communicate with my device. I do not do anything special, simple SNMP OID Traffic graphs is all we use. Nothing is changed for description or datasources.

The rrd files are being created and are owned by the cactiuser.

Any ideas?

jsingleton71 · Post by **jsingleton71** » Thu Jul 12, 2007 12:55 pm

I took a look at the rrdtool command that cacti is passing and saw this at the top of the command-line:

Notice: Undefined offset: 1 in /usr/support/htdocs/cacti/lib/rrd.php on line 1131

Could this be causing the problem?

jsingleton71 · Post by **jsingleton71** » Thu Jul 12, 2007 1:27 pm

Seems that the below code may be where the issue is. This server is running old server software - FreeBSD 4.x, Apache 1.3.x, MySQL 4, and rrdtool 1.0.x

I spoke with my staff php guru and he believes that the explode() function from lib/rrd.php is not returning the correct value for date() and thus is not putting any data inside of the rrd.

Anyone brave enough to take a whack at this?

This code snippet begin around Line: 1125 to the EOF:

Code: Select all

                                $value_array = explode(":", $graph_item["value"]);
 
                                if ($value_array[0] < 0) {
                                        $value = date("U") - (-3600 * $value_array[0]) - 60 * $value_array[1];
                                }else{
                                        $value = date("U", mktime($value_array[0],$value_array[1],0));
                                }
  
                                $txt_graph_items .= $graph_item_types{$graph_item["graph_type_id"]} . ":" . $value . $graph_item_color_cod
e . ":\"" . $graph_variables["text_format"][$graph_item_id] . $hardreturn[$graph_item_id] . "\" ";
                        }
                }else{
                        $need_rrd_nl = FALSE;
                }
 
                $i++;

                if (($i < sizeof($graph_items)) && ($need_rrd_nl)) {
                        $txt_graph_items .= RRD_NL;
                }
        }
        }
        
        /* either print out the source or pass the source onto rrdtool to get us a nice PNG */
        if (isset($graph_data_array["print_source"])) {
                print "<PRE>" . read_config_option("path_rrdtool") . " graph $graph_opts$graph_defs$txt_graph_items</PRE>";
        }else{
                if (isset($graph_data_array["export"])) {
                        rrdtool_execute("graph $graph_opts$graph_defs$txt_graph_items", false, RRDTOOL_OUTPUT_NULL, $rrd_struc);
                        return 0;
                }else{
                        if (isset($graph_data_array["output_flag"])) {
                                $output_flag = $graph_data_array["output_flag"];
                        }else{
                                $output_flag = RRDTOOL_OUTPUT_GRAPH_DATA;
                        }

                        return rrdtool_execute("graph $graph_opts$graph_defs$txt_graph_items", false, $output_flag, $rrd_struc);
                }
        }
}

?>

jsingleton71 · Post by **jsingleton71** » Thu Jul 12, 2007 1:46 pm

Adding an @ in front of the date() function on line 1131 of lib/rrd.php fixed the problem

Line 1131 before:

Code: Select all

$value = date("U", mktime($value_array[0],$value_array[1],0));

Line 1131 after:

Code: Select all

$value = @date("U", mktime($value_array[0],$value_array[1],0));

Suppressing the Notice: error seems to have done the trick.

Post by **rony** » Thu Jul 12, 2007 2:21 pm

There is nothing wrong with explode.....

What php version, full version are you running? We highly recommend version 4.3.4 or higher.

If you current graphs are working, they there is nothing wrong with explode.

If I can take a look at your installation (web access), I can probably tell you what is wrong.

jsingleton71 · Post by **jsingleton71** » Fri Jul 13, 2007 10:13 am

-bash-2.05b$ php -v
PHP 4.4.2 (cli) (built: May 22 2006 20:19:17)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies

and just because explode works on some and not for others, does not mean there isn't a problem with explode(). In fact, I have received the same information from 3 other php guru's that explode() in php4 is flaky (not buggy).

Since you claim explode is not flaky, then you will have to prove to me otherwise. Because using the @ in front of date() on line 1131 of lib/rrd.php to suppress the NOTICE: error did fix the problem. And guess what, line 1131 is part of the explode function that you claim is not broken.

Of course, all of my information is 2nd hand, so I can't argue, I can only state. From a network systems point of view, with quite a bit of logical thinking experience, I say, that it is something to do with either the explode() function itself, or the data that is collected using the explode() function.

Since I already proved that suppressing the NOTICE error fixes my issue, I think this is a problem for the developers to troubleshoot and fix.

Post by **rony** » Fri Jul 13, 2007 10:24 am

I would still like to either login to your system (web) or get a copy of your database.

I want to figure out why you where seeing this on a new device, but not existing.

FYI, while there maybe issues with explode, I heard way to many "php gurus" claim it's broken or flaky and to be honesty with you, it's getting old quick. I'm not ruling it out, but I'm not taking it as the reason.

I want to figure this out and you seem to have a good explain of what can be considered a bug (php or Cacti).

jsingleton71 · Post by **jsingleton71** » Fri Jul 13, 2007 10:46 am

Well I cannot give you access to our server or the database, as this is a secured environment.

One thing I do not understand, is, if you've heard this same thing about explode() from other php guru's, then how come a single developer can argue. Are these guru's not really guru's? I can assure you that the guru's I spoke with are long time friends of mine and of whom I have worked side by side with many times. So I tend to believe those whom I have known, vs those who just claim to be.

Also, from what I understand, explode() does not actually do what the documentation says it does, especially when the data returned is null. Feel free to correct me on this, but again this info comes 2nd hand.

Bottom line here is, you cannot argue with what works. I put the @ in front of date() and suppressed that error, and now Graphs work. So the question is: was that NOTICE error somehow getting into the binary data of the graph? If so, then why doesn't it do it for the already working graphs? NOTE: these already working graphs were created under 0.86h, but same version of php/apache/mysql.

--
BTW, I am not intending to sound harsh. I just get wordy when a decent argument is being had. Its healthy for the brain to have discussions like this!

Post by **rony** » Fri Jul 13, 2007 2:04 pm

I don't mind having a discussion, and even if it seemed heated, I will not hold it against you.

But, and I quote this on purpose, "PHP Guru" is a title I hear being thrown around a bit much. I have been working with PHP for many years, lost count, since version 3. I do recall issues with explode in the past, but from my research, the version that you are running does not have any issues. Unless someone can prove to me that explode still has problems in the version of PHP that you are running or any other version of PHP that Cacti supports (4.3.4+), I will continue to argue that there is nothing wrong with it. It just seems like the scape goat this year concerning PHP issues.

Finally, I have no problem admitting I'm wrong when I'm wrong. I just need more proof than some "PHP Guru" says so.

Post by **rony** » Fri Jul 13, 2007 2:12 pm

Um....

Poking at the code I do not think there is an issue with explode, but probably the data being passed into it.

I will have to play with this.

jsingleton71 · Post by **jsingleton71** » Fri Jul 13, 2007 3:14 pm

Which is the other possibility that one of the "php programmers" I have spoke with. The data returned from the explode() must not be expected or is returning more (the Error?).

So this is why I was wondering if that NOTICE error message was somehow getting captured in $value and then inserted into the graph image. Just a theory.

Post by **TheWitness** » Sat Jul 14, 2007 9:00 pm

This post is almost identical to one posted some time ago and the problem is the same. You are using a VRULE and the second value in the value array does not exist.

Therefore, explode naturally complains. It is not a problem with PHP, but with what is passed to the VRULE function. So, we may have a bug there.

What I would like you to do is go into the Graph Management page, view what would normally be the graph from there and turn on Graph Debug.

Post the debug information here so I can research this.

Thanks,

TheWitness

Post by **TheWitness** » Sat Jul 14, 2007 9:17 pm

I think that the VRULE function is expecting your time in HH:MM format. This is why you have the "explode" on the value.

However, our documentation is poor here. I think it should accept a timestamp as well. Les see if this replacement snippet works.

Original code:

Code: Select all

				$value_array = explode(":", $graph_item["value"]);

				if ($value_array[0] < 0) {
					$value = date("U") - (-3600 * $value_array[0]) - 60 * $value_array[1];
				}else{
					$value = date("U", mktime($value_array[0],$value_array[1],0));
				}

New code:

Code: Select all

				if (substr_count($graph_item["value"], ":")) {
					$value_array = explode(":", $graph_item["value"]);

					if ($value_array[0] < 0) {
						$value = date("U") - (-3600 * $value_array[0]) - 60 * $value_array[1];
					}else{
						$value = date("U", mktime($value_array[0],$value_array[1],0));
					}
				}else if (is_numeric($graph_item["value"])) {
					$value = $graph_item["value"];
				}

Now, with this code, the VRULE will accept an absolute Unix timestamp.

TheWitness

SQL Exec: "truncate table poller_output"

SQL Exec: "truncate table poller_output"

Who is online