Cacti Graph whit "hole"

[LK00X]

Hi all as you can see by attached image, Cacti 1.1.36 + Spine 1.1.36 on Centos 7 generate graphs but there are some random hole.
Is it possible to solve?

Thank-you in advance.

[netniV]

Check your logs for the SYSTEM STATS: lines, see if around the time of these gaps, the polling times were increased. Also, check for any errors which relate to the device/graph in question.

[LK00X]

Thank-you for answer.
Please, can you explain better what I have to do/verify?

Thank-you.

[ok1mgj]

just take look into logfile for line like:
07/30/2018 02:09:13 PM - SYSTEM STATS: Time:11.8184 Method:spine Processes:6 Threads:32 Hosts:667 HostsPerProcess:112 DataSources:170064 RRDsProcessed:69415

There can be some timeouts etc.

[LK00X]

Dear ok1mgj, thank-you for answer.
Below what I see near a "hole"
This sound strange because device is always working and function.

2018/07/30 14:38:04 - SYSTEM STATS: Time:2.4207 Method:spine Processes:1 Threads:5 Hosts:3 HostsPerProcess:3 DataSources:52 RRDsProcessed:43
2018/07/30 14:38:02 - SPINE: Poller[Main Poller] Device[4] Device[Cisco ASA 5510] Hostname[172.16.0.249] NOTICE: HOST EVENT: Device Returned from DOWN State
2018/07/30 14:37:06 - SYSTEM SYSLOG STATS:Time:0.23 Deletes:0 Incoming:19 Removes:0 XFers:19 Alerts:0 Alarms:0 Reports:0
2018/07/30 14:37:05 - SYSTEM MONITOR STATS: Time:0.4 Reboots:0 DownDevices:0 Notifications:0 Purges:0
2018/07/30 14:37:05 - SYSTEM WEBSEER STATS: Total Time:0.065, Service Checks:1, Servers:0
2018/07/30 14:37:05 - SYSTEM DSSTATS STATS: Type:HOURLY, Time:0.0254
2018/07/30 14:37:05 - SYSTEM THOLD STATS: Time:0.0268 Tholds:0 TotalDevices:3 DownDevices:0 NewDownDevices:
2018/07/30 14:37:04 - SYSTEM STATS: Time:2.4660 Method:spine Processes:1 Threads:5 Hosts:3 HostsPerProcess:3 DataSources:52 RRDsProcessed:43
2018/07/30 14:36:07 - SYSTEM SYSLOG STATS:Time:0.23 Deletes:0 Incoming:19 Removes:0 XFers:19 Alerts:0 Alarms:0 Reports:0
2018/07/30 14:36:07 - SYSTEM MONITOR STATS: Time:0.47 Reboots:0 DownDevices:0 Notifications:0 Purges:0
2018/07/30 14:36:07 - SYSTEM WEBSEER STATS: Total Time:0.09, Service Checks:1, Servers:0
2018/07/30 14:36:06 - SYSTEM THOLD STATS: Time:0.0147 Tholds:0 TotalDevices:3 DownDevices:0 NewDownDevices:
2018/07/30 14:36:06 - SYSTEM DSSTATS STATS: Type:HOURLY, Time:0.0336
2018/07/30 14:36:05 - SYSTEM STATS: Time:3.5264 Method:spine Processes:1 Threads:5 Hosts:3 HostsPerProcess:3 DataSources:52 RRDsProcessed:43
2018/07/30 14:35:06 - SYSTEM SYSLOG STATS:Time:0.27 Deletes:0 Incoming:19 Removes:0 XFers:19 Alerts:0 Alarms:0 Reports:0
2018/07/30 14:35:06 - SYSTEM MONITOR STATS: Time:0.38 Reboots:0 DownDevices:0 Notifications:0 Purges:0
2018/07/30 14:35:06 - SYSTEM WEBSEER STATS: Total Time:0.109, Service Checks:1, Servers:0
2018/07/30 14:35:05 - SYSTEM DSSTATS STATS: Type:HOURLY, Time:0.0303
2018/07/30 14:35:05 - SYSTEM THOLD STATS: Time:0.0065 Tholds:0 TotalDevices:3 DownDevices:1 NewDownDevices:
2018/07/30 14:35:04 - SYSTEM STATS: Time:3.4445 Method:spine Processes:1 Threads:5 Hosts:3 HostsPerProcess:3 DataSources:52 RRDsProcessed:29
2018/07/30 14:35:03 - SPINE: Poller[Main Poller] Device[4] Device[Cisco ASA 5510] Hostname[172.16.0.249] ERROR: HOST EVENT: Device is DOWN Message: Device responded to SNMP, ICMP: Ping timed out
2018/07/30 14:34:07 - SYSTEM SYSLOG STATS:Time:0.26 Deletes:0 Incoming:19 Removes:0 XFers:19 Alerts:0 Alarms:0 Reports:0

[ok1mgj]

Hi,
this looks like some troubles for ICMP packets.
I think it`s standard trouble on networking devices, just don`t use ICMP for detecting device reachability.

Try to switch Downed Device Detection to SNMP Uptime (it is under device configuration), it works better.

[LK00X]

Hi ok1mgj,
I changed the configuration as you requested.
I'm monitoring if now it works better.
I let you know as soon as possible.

Thank-you for your support.

[LK00X]

It seems better but I find again one little "hole":

2018/07/31 00:05:00 - CMDPHP PHP ERROR WARNING Backtrace: (/poller.php: 632 CactiErrorHandler)(/lib/functions.php: 4586 cacti_debug_backtrace)
2018/07/31 00:05:00 - ERROR PHP WARNING: A non-numeric value encountered in file: /var/www/html/cacti-1.1.36/poller.php on line: 632
2018/07/31 00:04:29 - SPINE: Poller[Main Poller] ERROR: SQL Failed! Error:'2013', Message:'Lost connection to MySQL server during query', SQL Fragment:'INSERT INTO poller_time (poller_id, pid, start_time, end_time) VALUES (1, 13450, NOW(), '0000-00-00 00:00:00')'
2018/07/31 00:04:29 - PHPSVR ERROR: Input Expected, Script Server Terminating
2018/07/31 00:04:29 - PHPSVR ERROR: Input Expected, Script Server Terminating
2018/07/31 00:04:29 - SPINE: Poller[Main Poller] FATAL: MySQL Error:'2006', Message:'MySQL server has gone away' (Spine thread)
2018/07/31 00:04:29 - SPINE: Poller[Main Poller] ERROR: SQL Failed! Error:'2013', Message:'Lost connection to MySQL server during query', SQL Fragment:'INSERT INTO poller_time (poller_id, pid, start_time, end_time) VALUES (1, 13443, NOW(), '0000-00-00 00:00:00')'
2018/07/31 00:04:12 - CMDPHP PHP ERROR WARNING Backtrace: (/poller.php: 632 CactiErrorHandler)(/lib/functions.php: 4586 cacti_debug_backtrace)
2018/07/31 00:04:12 - ERROR PHP WARNING: A non-numeric value encountered in file: /var/www/html/cacti-1.1.36/poller.php on line: 632
2018/07/31 00:04:00 - SYSTEM STATS: Time:57.7429 Method:spine Processes:1 Threads:5 Hosts:3 HostsPerProcess:3 DataSources:52 RRDsProcessed:0
2018/07/31 00:04:00 - POLLER: Poller[Main Poller] Maximum runtime of 58 seconds exceeded. Exiting.
2018/07/31 00:03:37 - SYSTEM STATS: Time:93.1411 Method:spine Processes:1 Threads:5 Hosts:3 HostsPerProcess:3 DataSources:52 RRDsProcessed:0
2018/07/31 00:03:14 - SPINE: Poller[Main Poller] FATAL: MySQL Error:'2013', Message:'Lost connection to MySQL server during query' (Spine parent)
2018/07/31 00:03:14 - SPINE: Poller[Main Poller] FATAL: MySQL Error:'2013', Message:'Lost connection to MySQL server during query' (Spine parent)
2018/07/31 00:03:00 - POLLER: Poller[Main Poller] Maximum runtime of 58 seconds exceeded. Exiting.
2018/07/31 00:02:35 - SPINE: Poller[Main Poller] FATAL: MySQL Error:'2013', Message:'Lost connection to MySQL server during query' (Spine parent)
2018/07/31 00:02:35 - SPINE: Poller[Main Poller] FATAL: MySQL Error:'2013', Message:'Lost connection to MySQL server during query' (Spine parent)
2018/07/31 00:01:44 - SYSTEM MONITOR STATS: Time:3.93 Reboots:0 DownDevices:0 Notifications:0 Purges:0
2018/07/31 00:01:39 - SYSTEM DSSTATS STATS: Type:HOURLY, Time:1.7052
2018/07/31 00:01:38 - SYSTEM WEBSEER STATS: Total Time:0.071, Service Checks:1, Servers:0
2018/07/31 00:01:38 - SYSTEM THOLD STATS: Time:0.3981 Tholds:0 TotalDevices:3 DownDevices:0 NewDownDevices:
2018/07/31 00:01:25 - SYSTEM SYSLOG: Removing old partition 'dd20180724'
2018/07/31 00:01:20 - SYSTEM STATS: Time:17.4770 Method:spine Processes:1 Threads:5 Hosts:3 HostsPerProcess:3 DataSources:52 RRDsProcessed:43
2018/07/31 00:01:14 - SYSTEM DSSTATS STATS: Type:MAJOR, Time:0.8252
2018/07/31 00:01:07 - SYSTEM DSSTATS STATS: Type:HOURLY, Time:2.9493
2018/07/31 00:01:07 - SYSTEM LOGMAINT STATS: Time:3.8804
2018/07/31 00:01:07 - MAINT Cacti Log Rotation - Purging Log : cacti.log-20180729
2018/07/31 00:01:07 - SYSTEM MONITOR STATS: Time:3.52 Reboots:0 DownDevices:0 Notifications:0 Purges:0
2018/07/31 00:01:07 - MAINT Cacti Log Rotation - Purging all logs before 20180730
2018/07/31 00:01:07 - MAINT Cacti Log Rotation - Created Log cacti.log-20180730
Previous

[ok1mgj]

Hi,

this looks like some troubles with sql server - please check sql logfile. There should be some errors.

[LK00X]

Hi,
I update Cacti and Spine from 1.1.36 to 1.1.38 and now it seems it works fine.
Will see in the next days...

Thank-you