Graphs not updating

[csumb]

Hi,
(this is under RHEL 5.5)
I upgraded from 0.87e to 0.87g (was using spine previously), started seeing mysql errors in cacti.log and graphs were no longer gathering data. (tested using mysql cmd line client and could connect as the cacti user to the mysql server (a diff machine), I changed to using cmd.php to troubleshoot. That made things slightly happier (most of the graphs) are updating but are really spotty (instead of being smooth they are really jumpy attached a screenshot. I tried killing cacti and running myisamchk against the db and its clean and ran check table against the tables it was complaining about and that didn't help. I attached a snippet from the logs and a screenshot of what the graphs are looking like (these are graphs that fed by shell scripts and SNMP v3 that are affected). Poller is set to run every 5 minutes. I have concurrent poller processes set to 4 (machine is a 2 proc * 2 core opteron)

log snippet:

12/03/2010 04:39:40 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT count(*) FROM poller_time WHERE end_time='0000-00-00 00:00:00'"
12/03/2010 04:39:40 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"TRUNCATE TABLE poller_time'
12/03/2010 04:39:40 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"SELECT local_data_id, rrd_name FROM poller_output"
12/03/2010 04:39:40 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('date',NOW())'
12/03/2010 04:39:40 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT count(*) FROM poller_time WHERE poller_id=0 AND end_time>'0000-00-00 00:00:00'"
12/03/2010 04:39:40 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"select poller_output.output, poller_output.time, poller_output.local_data_id, poller_item.rrd_path, poller_item.rrd_name, poller_item.rrd_num from (poller_output,poller_item) where (poller_output.local_data_id=poller_item.local_data_id and poller_output.rrd_name=poller_item.rrd_name) "
12/03/2010 04:39:40 PM - SYSTEM STATS: Time:0.0042 Method:cmd.php Processes:4 Threads:N/A Hosts:0 HostsPerProcess:0 DataSources:853 RRDsProcessed:0
12/03/2010 04:39:40 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('stats_poller','Time:0.0042 Method:cmd.php Processes:4 Threads:N/A Hosts:0 HostsPerProcess:0 DataSources:853 RRDsProcessed:0')'
12/03/2010 04:39:40 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT COUNT(*) FROM poller_command"
12/03/2010 04:39:40 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('stats_recache','RecacheTime:0.0 HostsRecached:0')'
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"SELECT id FROM host WHERE disabled = '' ORDER BY id"
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('path_webroot','/Volumes/external/home/httpd/cacti')'
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT count(*) FROM poller_time WHERE end_time='0000-00-00 00:00:00'"
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"TRUNCATE TABLE poller_time'
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"SELECT local_data_id, rrd_name FROM poller_output"
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('date',NOW())'
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT count(*) FROM poller_time WHERE poller_id=0 AND end_time>'0000-00-00 00:00:00'"
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"select poller_output.output, poller_output.time, poller_output.local_data_id, poller_item.rrd_path, poller_item.rrd_name, poller_item.rrd_num from (poller_output,poller_item) where (poller_output.local_data_id=poller_item.local_data_id and poller_output.rrd_name=poller_item.rrd_name) "
12/03/2010 04:40:10 PM - SYSTEM STATS: Time:0.0038 Method:cmd.php Processes:4 Threads:N/A Hosts:0 HostsPerProcess:0 DataSources:853 RRDsProcessed:0
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('stats_poller','Time:0.0038 Method:cmd.php Processes:4 Threads:N/A Hosts:0 HostsPerProcess:0 DataSources:853 RRDsProcessed:0')'
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT COUNT(*) FROM poller_command"
12/03/2010 04:40:10 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('stats_recache','RecacheTime:0.0 HostsRecached:0')'
12/03/2010 04:40:40 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"SELECT id FROM host WHERE disabled = '' ORDER BY id"
12/03/2010 04:40:40 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('path_webroot','/Volumes/external/home/httpd/cacti')'
12/03/2010 04:40:40 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT count(*) FROM poller_time WHERE end_time='0000-00-00 00:00:00'"

[gandalf]

We had a similar issue that was solved by current SVN/0.8.7/lib/database.php
R.

[csumb]

We had a similar issue that was solved by current SVN/0.8.7/lib/database.php
R.

downloaded will update in a few hours when I can tell if its back to normal or not

[csumb]

grabbed database.php off svn but still seeing the same behavior

same kinda stuff but a little cleaner in the logs

12/05/2010 12:10:56 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1062', SQL:"insert into poller_output (local_data_id, rrd_name, time, output) values (4959, 'nfs_lookup', '2010-12-05 00:10:54', '200905436')'
12/05/2010 12:10:56 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1062', SQL:"insert into poller_output (local_data_id, rrd_name, time, output) values (4970, 'nfs_setattr', '2010-12-05 00:10:54', '75741121')'
12/05/2010 12:10:56 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1062', SQL:"insert into poller_output (local_data_id, rrd_name, time, output) values (4971, 'nfs_symlink', '2010-12-05 00:10:54', '0')'
12/05/2010 12:10:56 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1062', SQL:"insert into poller_output (local_data_id, rrd_name, time, output) values (4972, 'nfs_write', '2010-12-05 00:10:54', '748440409')'
12/05/2010 12:10:56 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1062', SQL:"insert into poller_output (local_data_id, rrd_name, time, output) values (4973, 'mount', '2010-12-05 00:10:56', '0.000')'
12/05/2010 12:10:56 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1062', SQL:"insert into poller_output (local_data_id, rrd_name, time, output) values (4974, 'number', '2010-12-05 00:10:56', '0')'

[gandalf]

And? Did you google for that error? It's a duplicate key issue!
This will happen, when two pollers are running in parallel. See e.g. 2nd link of my sig for explanation and hints
R.

[fering]

Hi,

having same issue after upgrade from 0.8.7e to 0.8.7g
how exactly can i get this database.php file please.


12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('stats_recache','RecacheTime:0.0 HostsRecached:0')'
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT COUNT(*) FROM poller_command"
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('stats_poller','Time:0.0027 Method:cmd.php Processes:40 Threads:N/A Hosts:0 HostsPerProcess:0 DataSources:837 RRDsProcessed:0')'
12/29/2010 06:23:02 PM - SYSTEM STATS: Time:0.0027 Method:cmd.php Processes:40 Threads:N/A Hosts:0 HostsPerProcess:0 DataSources:837 RRDsProcessed:0
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"select poller_output.output, poller_output.time, poller_output.local_data_id, poller_item.rrd_path, poller_item.rrd_name, poller_item.rrd_num from (poller_output,poller_item) where (poller_output.local_data_id=poller_item.local_data_id and poller_output.rrd_name=poller_item.rrd_name) "
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT count(*) FROM poller_time WHERE poller_id=0 AND end_time>'0000-00-00 00:00:00'"
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('date',NOW())'
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"SELECT local_data_id, rrd_name FROM poller_output"
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"TRUNCATE TABLE poller_time'
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT count(*) FROM poller_time WHERE end_time='0000-00-00 00:00:00'"
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('path_webroot','/var/www/html/cacti')'
12/29/2010 06:23:02 PM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"SELECT id FROM host WHERE disabled = '' ORDER BY id"


Thanks,
Danny

[gandalf]

http://svn.cacti.net/
R.

[ftorres]

Having same issue here.
Try all in the manual and still have critical errors. I'm using spine and don't know why is reporting CMDPHP.
We need URGENT support in this...! Any company or person that can give support?

04/05/2011 09:21:56 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"show tables from `cacti`"
04/05/2011 09:21:56 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'0', SQL:"SELECT 1 AS id, ph.name, ph.file, ph.function FROM plugin_hooks AS ph LEFT JOIN plugin_config AS pc ON pc.directory=ph.name WHERE ph.status = 1 AND hook = 'poller_bottom' AND ph.name IN ('settings', 'boost', 'dsstats') UNION SELECT pc.id, ph.name, ph.file, ph.function FROM plugin_hooks AS ph LEFT JOIN plugin_config AS pc ON pc.directory=ph.name WHERE ph.status = 1 AND hook = 'poller_bottom' AND ph.name NOT IN ('settings', 'boost', 'dsstats') ORDER BY id ASC"
04/05/2011 09:21:56 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('stats_recache','RecacheTime:0.0 HostsRecached:0')'
04/05/2011 09:21:56 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'0', SQL:"SELECT COUNT(*) FROM poller_command"
04/05/2011 09:21:56 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'0', SQL:"REPLACE INTO settings (name,value) VALUES ('stats_poller','Time:0.0213 Method:spine Processes:1 Threads:10 Hosts:0 HostsPerProcess:0 DataSources:12944 RRDsProcessed:0')'

[gandalf]

Did you try the fix I posted above?
R.

[ftorres]

Yes, with no susccess...

[gandalf]

Please provide URL to the exact code you have downloaded.
R.

[ftorres]

I just solve the issue making a massive upgrade to CACTI DB and also to the program.
Went from first cacti version to current.

[dexx]

I was getting the same issue. I downloaded the database.php from the SVN as recommended. Now the poller errors are no longer appearing in the Cacti log. However the graphs are still not updating. Do i need to flush it in some way?

[gandalf]

No. It may happen, that your error is quite different.
So please open a new thread and post your findings after reading 2nd link of my sig.
R.