Repeated poller messages in cacti logs
Moderators: Developers, Moderators
-
- Cacti User
- Posts: 79
- Joined: Mon Jun 22, 2009 12:58 am
- Location: India
Repeated poller messages in cacti logs
Hi,
recently i am observing few repeated messages in my log file with respect to the poller.
and few of my device graph is also no updating properly. i am seeing lot of blanks in between. But this is happening for only 2 devices. Rest all the devices graphs are updating properly.
any clue what could be wrong.
Here is the sample log.
03/19/2015 07:35:02 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 2630, Data Sources: BytesIn(DS[8251]), BytesOut(DS[8251]), BytesIn(DS[8252]), BytesOut(DS[8252]), BytesIn(DS[8253]), BytesOut(DS[8253]), BytesIn(DS[8254]), BytesOut(DS[8254]), BytesIn(DS[8255]), BytesOut(DS[8255]), BytesIn(DS[8256]), BytesOut(DS[8256]), BytesIn(DS[8257]), BytesOut(DS[8257]), BytesIn(DS[8258]), BytesOut(DS[8258]), BytesIn(DS[8259]), BytesOut(DS[8259]), BytesIn(DS[8260]), BytesOut(DS[8260]), Additional Issues Remain. Only showing first 20
03/19/2015 07:30:37 PM - RECACHE STATS: RecacheTime:22.8142 HostsRecached:3
03/19/2015 07:30:37 PM - RECACHE STATS: RecacheTime:22.9245 HostsRecached:3
03/19/2015 07:30:37 PM - RECACHE STATS: RecacheTime:22.9151 HostsRecached:3
03/19/2015 07:30:37 PM - RECACHE STATS: RecacheTime:22.9233 HostsRecached:3
03/19/2015 07:30:30 PM - PCOMMAND: Poller[0] Host[432] WARNING: Recache Event Detected for Host
03/19/2015 07:30:30 PM - PCOMMAND: Poller[0] Host[432] WARNING: Recache Event Detected for Host
03/19/2015 07:30:30 PM - PCOMMAND: Poller[0] Host[432] WARNING: Recache Event Detected for Host
03/19/2015 07:30:30 PM - PCOMMAND: Poller[0] Host[432] WARNING: Recache Event Detected for Host
03/19/2015 07:30:22 PM - PCOMMAND: Poller[0] Host[308] WARNING: Recache Event Detected for Host
03/19/2015 07:30:22 PM - PCOMMAND: Poller[0] Host[308] WARNING: Recache Event Detected for Host
03/19/2015 07:30:22 PM - PCOMMAND: Poller[0] Host[308] WARNING: Recache Event Detected for Host
03/19/2015 07:30:22 PM - PCOMMAND: Poller[0] Host[308] WARNING: Recache Event Detected for Host
03/19/2015 07:30:15 PM - PCOMMAND: Poller[0] Host[305] WARNING: Recache Event Detected for Host
03/19/2015 07:30:14 PM - PCOMMAND: Poller[0] Host[305] WARNING: Recache Event Detected for Host
03/19/2015 07:30:14 PM - PCOMMAND: Poller[0] Host[305] WARNING: Recache Event Detected for Host
03/19/2015 07:30:14 PM - PCOMMAND: Poller[0] Host[305] WARNING: Recache Event Detected for Host
03/19/2015 07:30:12 PM - SYSTEM THOLD STATS: Time:0.0362 Tholds:1 DownHosts:0
03/19/2015 07:30:12 PM - SYSTEM THOLD STATS: Time:0.0369 Tholds:1 DownHosts:0
03/19/2015 07:30:12 PM - SYSTEM THOLD STATS: Time:0.0356 Tholds:1 DownHosts:0
03/19/2015 07:30:12 PM - SYSTEM THOLD STATS: Time:0.0363 Tholds:1 DownHosts:0
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5683 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7317
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5649 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7341
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5652 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7304
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5619 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7330
Thanks
Soma
Cacti-0.8.8a|OS-windows-2008|NET-SNMP- 5.6.1.1|RRDTool-1.4.x|PHP-5.3.17|SPINE-0.8.8a|Hosts-336|Graphs-7500|
recently i am observing few repeated messages in my log file with respect to the poller.
and few of my device graph is also no updating properly. i am seeing lot of blanks in between. But this is happening for only 2 devices. Rest all the devices graphs are updating properly.
any clue what could be wrong.
Here is the sample log.
03/19/2015 07:35:02 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 2630, Data Sources: BytesIn(DS[8251]), BytesOut(DS[8251]), BytesIn(DS[8252]), BytesOut(DS[8252]), BytesIn(DS[8253]), BytesOut(DS[8253]), BytesIn(DS[8254]), BytesOut(DS[8254]), BytesIn(DS[8255]), BytesOut(DS[8255]), BytesIn(DS[8256]), BytesOut(DS[8256]), BytesIn(DS[8257]), BytesOut(DS[8257]), BytesIn(DS[8258]), BytesOut(DS[8258]), BytesIn(DS[8259]), BytesOut(DS[8259]), BytesIn(DS[8260]), BytesOut(DS[8260]), Additional Issues Remain. Only showing first 20
03/19/2015 07:30:37 PM - RECACHE STATS: RecacheTime:22.8142 HostsRecached:3
03/19/2015 07:30:37 PM - RECACHE STATS: RecacheTime:22.9245 HostsRecached:3
03/19/2015 07:30:37 PM - RECACHE STATS: RecacheTime:22.9151 HostsRecached:3
03/19/2015 07:30:37 PM - RECACHE STATS: RecacheTime:22.9233 HostsRecached:3
03/19/2015 07:30:30 PM - PCOMMAND: Poller[0] Host[432] WARNING: Recache Event Detected for Host
03/19/2015 07:30:30 PM - PCOMMAND: Poller[0] Host[432] WARNING: Recache Event Detected for Host
03/19/2015 07:30:30 PM - PCOMMAND: Poller[0] Host[432] WARNING: Recache Event Detected for Host
03/19/2015 07:30:30 PM - PCOMMAND: Poller[0] Host[432] WARNING: Recache Event Detected for Host
03/19/2015 07:30:22 PM - PCOMMAND: Poller[0] Host[308] WARNING: Recache Event Detected for Host
03/19/2015 07:30:22 PM - PCOMMAND: Poller[0] Host[308] WARNING: Recache Event Detected for Host
03/19/2015 07:30:22 PM - PCOMMAND: Poller[0] Host[308] WARNING: Recache Event Detected for Host
03/19/2015 07:30:22 PM - PCOMMAND: Poller[0] Host[308] WARNING: Recache Event Detected for Host
03/19/2015 07:30:15 PM - PCOMMAND: Poller[0] Host[305] WARNING: Recache Event Detected for Host
03/19/2015 07:30:14 PM - PCOMMAND: Poller[0] Host[305] WARNING: Recache Event Detected for Host
03/19/2015 07:30:14 PM - PCOMMAND: Poller[0] Host[305] WARNING: Recache Event Detected for Host
03/19/2015 07:30:14 PM - PCOMMAND: Poller[0] Host[305] WARNING: Recache Event Detected for Host
03/19/2015 07:30:12 PM - SYSTEM THOLD STATS: Time:0.0362 Tholds:1 DownHosts:0
03/19/2015 07:30:12 PM - SYSTEM THOLD STATS: Time:0.0369 Tholds:1 DownHosts:0
03/19/2015 07:30:12 PM - SYSTEM THOLD STATS: Time:0.0356 Tholds:1 DownHosts:0
03/19/2015 07:30:12 PM - SYSTEM THOLD STATS: Time:0.0363 Tholds:1 DownHosts:0
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5683 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7317
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5649 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7341
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5652 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7304
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5619 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7330
Thanks
Soma
Cacti-0.8.8a|OS-windows-2008|NET-SNMP- 5.6.1.1|RRDTool-1.4.x|PHP-5.3.17|SPINE-0.8.8a|Hosts-336|Graphs-7500|
Re: Repeated poller messages in cacti logs
I assume You have 5min pooling
The issue migh be too low memory limit for php runing pooler scrip (recomended memory is 512mb if im right)
You should find indication of this problem in webserver logs something Like Trying to allocate
Here is a page with some explanations
http://www.cacti.net/downloads/docs/htm ... empty.html
The issue migh be too low memory limit for php runing pooler scrip (recomended memory is 512mb if im right)
You should find indication of this problem in webserver logs something Like Trying to allocate
Here is a page with some explanations
http://www.cacti.net/downloads/docs/htm ... empty.html
-
- Cacti User
- Posts: 79
- Joined: Mon Jun 22, 2009 12:58 am
- Location: India
Re: Repeated poller messages in cacti logs
Hi,
here my problem is my poller output table is not empty and i am getting the same poller message 4 times.
Regards
Soma
here my problem is my poller output table is not empty and i am getting the same poller message 4 times.
Regards
Soma
Re: Repeated poller messages in cacti logs
This is your problem.
It looks like your poller is running 4 different times every 5 minutes.
Code: Select all
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5683 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7317
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5649 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7341
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5652 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7304
03/19/2015 07:30:12 PM - SYSTEM STATS: Time:10.5619 Method:spine Processes:3 Threads:25 Hosts:377 HostsPerProcess:126 DataSources:14820 RRDsProcessed:7330
-
- Cacti User
- Posts: 79
- Joined: Mon Jun 22, 2009 12:58 am
- Location: India
Re: Repeated poller messages in cacti logs
Hi cigamit,
yes, i observed this. But unable to understand why it is happening. But few incidents it is appearing twice or thrice. The below is the sample log.
03/25/2015 01:30:29 PM - RECACHE STATS: RecacheTime:19.7835 HostsRecached:1
03/25/2015 01:30:29 PM - RECACHE STATS: RecacheTime:19.7642 HostsRecached:1
03/25/2015 01:30:07 PM - SYSTEM THOLD STATS: Time:0.0280 Tholds:1 DownHosts:0
03/25/2015 01:30:07 PM - SYSTEM THOLD STATS: Time:0.0275 Tholds:1 DownHosts:0
03/25/2015 01:30:07 PM - SYSTEM STATS: Time:5.2064 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:4522
03/25/2015 01:30:07 PM - SYSTEM STATS: Time:5.2067 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:4422
03/25/2015 01:25:14 PM - RECACHE STATS: RecacheTime:5.5221 HostsRecached:1
03/25/2015 01:25:14 PM - RECACHE STATS: RecacheTime:5.8574 HostsRecached:1
03/25/2015 01:25:14 PM - RECACHE STATS: RecacheTime:5.5157 HostsRecached:1
03/25/2015 01:25:07 PM - SYSTEM THOLD STATS: Time:0.0229 Tholds:1 DownHosts:0
03/25/2015 01:25:07 PM - SYSTEM THOLD STATS: Time:0.0225 Tholds:1 DownHosts:0
03/25/2015 01:25:07 PM - SYSTEM STATS: Time:5.3475 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6115
03/25/2015 01:25:07 PM - SYSTEM STATS: Time:5.3477 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6121
03/25/2015 01:25:07 PM - SYSTEM THOLD STATS: Time:0.0209 Tholds:0 DownHosts:0
03/25/2015 01:25:06 PM - SYSTEM STATS: Time:5.0125 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6011
03/25/2015 01:20:31 PM - RECACHE STATS: RecacheTime:21.9048 HostsRecached:1
03/25/2015 01:20:31 PM - RECACHE STATS: RecacheTime:21.8553 HostsRecached:1
03/25/2015 01:20:07 PM - SYSTEM THOLD STATS: Time:0.0339 Tholds:1 DownHosts:0
03/25/2015 01:20:07 PM - SYSTEM THOLD STATS: Time:0.0335 Tholds:1 DownHosts:0
03/25/2015 01:20:07 PM - SYSTEM STATS: Time:5.1757 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6065
03/25/2015 01:20:07 PM - SYSTEM STATS: Time:5.1760 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6065
03/25/2015 01:15:07 PM - SYSTEM THOLD STATS: Time:0.0208 Tholds:0 DownHosts:0
03/25/2015 01:15:07 PM - SYSTEM THOLD STATS: Time:0.0208 Tholds:0 DownHosts:0
03/25/2015 01:15:07 PM - SYSTEM THOLD STATS: Time:0.0194 Tholds:0 DownHosts:0
03/25/2015 01:15:07 PM - SYSTEM STATS: Time:5.1048 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5789
03/25/2015 01:15:07 PM - SYSTEM STATS: Time:5.1107 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5333
03/25/2015 01:15:07 PM - SYSTEM STATS: Time:5.1058 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5333
03/25/2015 01:11:05 PM - RECACHE STATS: RecacheTime:55.4682 HostsRecached:1
03/25/2015 01:11:05 PM - RECACHE STATS: RecacheTime:55.4348 HostsRecached:1
03/25/2015 01:10:07 PM - SYSTEM THOLD STATS: Time:0.0900 Tholds:2 DownHosts:0
03/25/2015 01:10:07 PM - SYSTEM THOLD STATS: Time:0.1228 Tholds:2 DownHosts:0
03/25/2015 01:10:07 PM - SYSTEM STATS: Time:5.5039 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5916
When ever the poller completes the task with in 5 seconds one of the host having blank graphs, but in the debug i'm able to see the device responding for the poller.
i have attached the debug log for that specific device. can you or some one go through that and guide me is there anything wrong with respect to that host.
Note - rest all the other host were polled and graphed properly.
Thanks
Soma
yes, i observed this. But unable to understand why it is happening. But few incidents it is appearing twice or thrice. The below is the sample log.
03/25/2015 01:30:29 PM - RECACHE STATS: RecacheTime:19.7835 HostsRecached:1
03/25/2015 01:30:29 PM - RECACHE STATS: RecacheTime:19.7642 HostsRecached:1
03/25/2015 01:30:07 PM - SYSTEM THOLD STATS: Time:0.0280 Tholds:1 DownHosts:0
03/25/2015 01:30:07 PM - SYSTEM THOLD STATS: Time:0.0275 Tholds:1 DownHosts:0
03/25/2015 01:30:07 PM - SYSTEM STATS: Time:5.2064 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:4522
03/25/2015 01:30:07 PM - SYSTEM STATS: Time:5.2067 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:4422
03/25/2015 01:25:14 PM - RECACHE STATS: RecacheTime:5.5221 HostsRecached:1
03/25/2015 01:25:14 PM - RECACHE STATS: RecacheTime:5.8574 HostsRecached:1
03/25/2015 01:25:14 PM - RECACHE STATS: RecacheTime:5.5157 HostsRecached:1
03/25/2015 01:25:07 PM - SYSTEM THOLD STATS: Time:0.0229 Tholds:1 DownHosts:0
03/25/2015 01:25:07 PM - SYSTEM THOLD STATS: Time:0.0225 Tholds:1 DownHosts:0
03/25/2015 01:25:07 PM - SYSTEM STATS: Time:5.3475 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6115
03/25/2015 01:25:07 PM - SYSTEM STATS: Time:5.3477 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6121
03/25/2015 01:25:07 PM - SYSTEM THOLD STATS: Time:0.0209 Tholds:0 DownHosts:0
03/25/2015 01:25:06 PM - SYSTEM STATS: Time:5.0125 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6011
03/25/2015 01:20:31 PM - RECACHE STATS: RecacheTime:21.9048 HostsRecached:1
03/25/2015 01:20:31 PM - RECACHE STATS: RecacheTime:21.8553 HostsRecached:1
03/25/2015 01:20:07 PM - SYSTEM THOLD STATS: Time:0.0339 Tholds:1 DownHosts:0
03/25/2015 01:20:07 PM - SYSTEM THOLD STATS: Time:0.0335 Tholds:1 DownHosts:0
03/25/2015 01:20:07 PM - SYSTEM STATS: Time:5.1757 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6065
03/25/2015 01:20:07 PM - SYSTEM STATS: Time:5.1760 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6065
03/25/2015 01:15:07 PM - SYSTEM THOLD STATS: Time:0.0208 Tholds:0 DownHosts:0
03/25/2015 01:15:07 PM - SYSTEM THOLD STATS: Time:0.0208 Tholds:0 DownHosts:0
03/25/2015 01:15:07 PM - SYSTEM THOLD STATS: Time:0.0194 Tholds:0 DownHosts:0
03/25/2015 01:15:07 PM - SYSTEM STATS: Time:5.1048 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5789
03/25/2015 01:15:07 PM - SYSTEM STATS: Time:5.1107 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5333
03/25/2015 01:15:07 PM - SYSTEM STATS: Time:5.1058 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5333
03/25/2015 01:11:05 PM - RECACHE STATS: RecacheTime:55.4682 HostsRecached:1
03/25/2015 01:11:05 PM - RECACHE STATS: RecacheTime:55.4348 HostsRecached:1
03/25/2015 01:10:07 PM - SYSTEM THOLD STATS: Time:0.0900 Tholds:2 DownHosts:0
03/25/2015 01:10:07 PM - SYSTEM THOLD STATS: Time:0.1228 Tholds:2 DownHosts:0
03/25/2015 01:10:07 PM - SYSTEM STATS: Time:5.5039 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5916
When ever the poller completes the task with in 5 seconds one of the host having blank graphs, but in the debug i'm able to see the device responding for the poller.
i have attached the debug log for that specific device. can you or some one go through that and guide me is there anything wrong with respect to that host.
Note - rest all the other host were polled and graphed properly.
Thanks
Soma
- Attachments
-
- Debug log.txt
- (644.43 KiB) Downloaded 174 times
-
- Cacti User
- Posts: 79
- Joined: Mon Jun 22, 2009 12:58 am
- Location: India
Re: Repeated poller messages in cacti logs
somasundaram wrote:Hi cigamit,
yes, i observed this. But unable to understand why it is happening. But few incidents it is appearing twice or thrice. The below is the sample log.
03/25/2015 01:30:29 PM - RECACHE STATS: RecacheTime:19.7835 HostsRecached:1
03/25/2015 01:30:29 PM - RECACHE STATS: RecacheTime:19.7642 HostsRecached:1
03/25/2015 01:30:07 PM - SYSTEM THOLD STATS: Time:0.0280 Tholds:1 DownHosts:0
03/25/2015 01:30:07 PM - SYSTEM THOLD STATS: Time:0.0275 Tholds:1 DownHosts:0
03/25/2015 01:30:07 PM - SYSTEM STATS: Time:5.2064 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:4522
03/25/2015 01:30:07 PM - SYSTEM STATS: Time:5.2067 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:4422
03/25/2015 01:25:14 PM - RECACHE STATS: RecacheTime:5.5221 HostsRecached:1
03/25/2015 01:25:14 PM - RECACHE STATS: RecacheTime:5.8574 HostsRecached:1
03/25/2015 01:25:14 PM - RECACHE STATS: RecacheTime:5.5157 HostsRecached:1
03/25/2015 01:25:07 PM - SYSTEM THOLD STATS: Time:0.0229 Tholds:1 DownHosts:0
03/25/2015 01:25:07 PM - SYSTEM THOLD STATS: Time:0.0225 Tholds:1 DownHosts:0
03/25/2015 01:25:07 PM - SYSTEM STATS: Time:5.3475 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6115
03/25/2015 01:25:07 PM - SYSTEM STATS: Time:5.3477 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6121
03/25/2015 01:25:07 PM - SYSTEM THOLD STATS: Time:0.0209 Tholds:0 DownHosts:0
03/25/2015 01:25:06 PM - SYSTEM STATS: Time:5.0125 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6011
03/25/2015 01:20:31 PM - RECACHE STATS: RecacheTime:21.9048 HostsRecached:1
03/25/2015 01:20:31 PM - RECACHE STATS: RecacheTime:21.8553 HostsRecached:1
03/25/2015 01:20:07 PM - SYSTEM THOLD STATS: Time:0.0339 Tholds:1 DownHosts:0
03/25/2015 01:20:07 PM - SYSTEM THOLD STATS: Time:0.0335 Tholds:1 DownHosts:0
03/25/2015 01:20:07 PM - SYSTEM STATS: Time:5.1757 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6065
03/25/2015 01:20:07 PM - SYSTEM STATS: Time:5.1760 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:6065
03/25/2015 01:15:07 PM - SYSTEM THOLD STATS: Time:0.0208 Tholds:0 DownHosts:0
03/25/2015 01:15:07 PM - SYSTEM THOLD STATS: Time:0.0208 Tholds:0 DownHosts:0
03/25/2015 01:15:07 PM - SYSTEM THOLD STATS: Time:0.0194 Tholds:0 DownHosts:0
03/25/2015 01:15:07 PM - SYSTEM STATS: Time:5.1048 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5789
03/25/2015 01:15:07 PM - SYSTEM STATS: Time:5.1107 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5333
03/25/2015 01:15:07 PM - SYSTEM STATS: Time:5.1058 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5333
03/25/2015 01:11:05 PM - RECACHE STATS: RecacheTime:55.4682 HostsRecached:1
03/25/2015 01:11:05 PM - RECACHE STATS: RecacheTime:55.4348 HostsRecached:1
03/25/2015 01:10:07 PM - SYSTEM THOLD STATS: Time:0.0900 Tholds:2 DownHosts:0
03/25/2015 01:10:07 PM - SYSTEM THOLD STATS: Time:0.1228 Tholds:2 DownHosts:0
03/25/2015 01:10:07 PM - SYSTEM STATS: Time:5.5039 Method:spine Processes:3 Threads:15 Hosts:377 HostsPerProcess:126 DataSources:14217 RRDsProcessed:5916
When ever the poller completes the task with in 5 seconds one of the host having blank graphs, but in the debug i'm able to see the device responding for the poller.
i have attached the debug log for that specific device. can you or some one go through that and guide me is there anything wrong with respect to that host.
Note - rest all the other host were polled and graphed properly.
Thanks
Soma
Hi, Just for a testing i have made the no of polling process into 1, but even then multiple poller messages are appearing intermittently. At the same time in the task scheduler logs there are 3 instance with 3 different ID's were executed during the same time. so it means the one instance for default cacti and the other instance for the script server, but unable to find out the reason for the third instance. (Attached the log of task scheduler)
03/25/2015 03:50:17 PM - SYSTEM STATS: Time:15.0773 Method:spine Processes:1 Threads:15 Hosts:377 HostsPerProcess:377 DataSources:14219 RRDsProcessed:7121
03/25/2015 03:45:19 PM - SYSTEM THOLD STATS: Time:0.0256 Tholds:2 DownHosts:0
03/25/2015 03:45:19 PM - SYSTEM THOLD STATS: Time:0.0253 Tholds:2 DownHosts:0
03/25/2015 03:45:19 PM - SYSTEM STATS: Time:17.1646 Method:spine Processes:1 Threads:15 Hosts:377 HostsPerProcess:377 DataSources:14219 RRDsProcessed:8394
03/25/2015 03:45:19 PM - SYSTEM STATS: Time:17.1637 Method:spine Processes:1 Threads:15 Hosts:377 HostsPerProcess:377 DataSources:14219 RRDsProcessed:8463
03/25/2015 03:40:16 PM - SYSTEM THOLD STATS: Time:0.0285 Tholds:2 DownHosts:0
03/25/2015 03:40:16 PM - SYSTEM STATS: Time:14.6031 Method:spine Processes:1 Threads:15 Hosts:377 HostsPerProcess:377 DataSources:14219 RRDsProcessed:7121
also there is one more strange thing observed, today i enabled debug to get the detailed log. But even after disabling the poller logging level into LOW still there are few debug messages are appearing int he cacti poller logs. I have attached the file. no clue what is happening with my box.
Thanks
Soma
- Attachments
-
- OS-Scheduled-Task-Poller.txt
- (4.63 KiB) Downloaded 159 times
-
- logs after disabling debug.txt
- (38.27 KiB) Downloaded 141 times
Re: Repeated poller messages in cacti logs
You still need to check your scheduled tasks and see if there are multiple schedules set for the poller to run, if so, remove all but 1.
-
- Cacti User
- Posts: 79
- Joined: Mon Jun 22, 2009 12:58 am
- Location: India
Re: Repeated poller messages in cacti logs
Hi,
i have checked there is only one schedule is running in the scheduler.
Scheduler Trigger
Scheduler Action
Really do not know what could be triggering the second poller action.
Also the apache service running as 2 services, if i kill one will it inturupt the polling, any idea?
Thanks
Soma
i have checked there is only one schedule is running in the scheduler.
Scheduler Trigger
Scheduler Action
Really do not know what could be triggering the second poller action.
Also the apache service running as 2 services, if i kill one will it inturupt the polling, any idea?
Thanks
Soma
Re: Repeated poller messages in cacti logs
sysinternals process explorer will show you if/what each apache instance is associated to a service.somasundaram wrote:Also the apache service running as 2 services, if i kill one will it interrupt the polling, any idea?
| Scripts: Monitor processes | RFC1213 MIB | DOCSIS Stats | Dell PowerEdge | Speedfan | APC UPS | DOCSIS CMTS | 3ware | Motorola Canopy |
| Guides: Windows Install | [HOWTO] Debug Windows NTFS permission problems |
| Tools: Windows All-in-one Installer |
Re: Repeated poller messages in cacti logs
The Apache process shouldn't be touched. It will spawn multiple processes as needed, and it has nothing to do with the poller, which is going to be ran directly by PHP. When it runs, you should see the PHP process for it in the task manager. See if multiple are showing up.
-
- Cacti User
- Posts: 79
- Joined: Mon Jun 22, 2009 12:58 am
- Location: India
Re: Repeated poller messages in cacti logs
cigamit wrote:The Apache process shouldn't be touched. It will spawn multiple processes as needed, and it has nothing to do with the poller, which is going to be ran directly by PHP. When it runs, you should see the PHP process for it in the task manager. See if multiple are showing up.
Yep at times i am seeing only two PHP process in the task manager, one instance for default cacti and the other instance for the script server.
But few times i am seeing multiple instance of PHP noticed in task manager and the cacti logs were also matching with multiple entries. But how to identify why only few incidents the poller is running more than one time. any other clue or suggestions pl.
thanks
Soma
-
- Cacti User
- Posts: 79
- Joined: Mon Jun 22, 2009 12:58 am
- Location: India
Re: Repeated poller messages in cacti logs
BSOD2600 wrote:sysinternals process explorer will show you if/what each apache instance is associated to a service.somasundaram wrote:Also the apache service running as 2 services, if i kill one will it interrupt the polling, any idea?
yep, in process explorer it is showing as a tree. Only one instance is running as a service.
So there is no need of doubting on Apache now....
-
- Cacti User
- Posts: 79
- Joined: Mon Jun 22, 2009 12:58 am
- Location: India
Re: Repeated poller messages in cacti logs
This is the snap of task manager when PHP was running two times. At the same time the logs were also generated twice.somasundaram wrote:cigamit wrote:The Apache process shouldn't be touched. It will spawn multiple processes as needed, and it has nothing to do with the poller, which is going to be ran directly by PHP. When it runs, you should see the PHP process for it in the task manager. See if multiple are showing up.
Yep at times i am seeing only two PHP process in the task manager, one instance for default cacti and the other instance for the script server.
But few times i am seeing multiple instance of PHP noticed in task manager and the cacti logs were also matching with multiple entries. But how to identify why only few incidents the poller is running more than one time. any other clue or suggestions pl.
thanks
Soma
One this i have noticed that when the poller instance is executed single time there were 7K RRD's were updated. whereas when poller executed twice there are 9K+9K=18K RRD's were updated. Is there any reason behind this or how to identify this which all the RRD's are included during the 2 instance poller update.
Task Manager log Cacti Logs
03/26/2015 01:30:19 PM - SYSTEM THOLD STATS: Time:0.0258 Tholds:2 DownHosts:0
03/26/2015 01:30:19 PM - SYSTEM THOLD STATS: Time:0.0285 Tholds:2 DownHosts:0
03/26/2015 01:30:19 PM - SYSTEM STATS: Time:17.6028 Method:spine Processes:1 Threads:15 Hosts:377 HostsPerProcess:377 DataSources:14221 RRDsProcessed:9131
03/26/2015 01:30:19 PM - SYSTEM STATS: Time:17.5900 Method:spine Processes:1 Threads:15 Hosts:377 HostsPerProcess:377 DataSources:14221 RRDsProcessed:9128
Re: Repeated poller messages in cacti logs
sysinternals process explorer will show/log far more info than taskmanger; i'd highly recommend utilizing. sysinternals process monitor would also show the process tree/history for all processes for a monitoring session. that would answer the duplicate question.
for starters, are there two instances in task scheduler?
for starters, are there two instances in task scheduler?
| Scripts: Monitor processes | RFC1213 MIB | DOCSIS Stats | Dell PowerEdge | Speedfan | APC UPS | DOCSIS CMTS | 3ware | Motorola Canopy |
| Guides: Windows Install | [HOWTO] Debug Windows NTFS permission problems |
| Tools: Windows All-in-one Installer |
-
- Cacti User
- Posts: 79
- Joined: Mon Jun 22, 2009 12:58 am
- Location: India
Re: Repeated poller messages in cacti logs
Hi BSOD,BSOD2600 wrote:sysinternals process explorer will show/log far more info than taskmanger; i'd highly recommend utilizing. sysinternals process monitor would also show the process tree/history for all processes for a monitoring session. that would answer the duplicate question.
for starters, are there two instances in task scheduler?
There is only one scheduler in task scheduler. The multiple instance are seen intermittently not always. every 15 or 20 minutes once this is been appearing for to polling cycles, then after disappearing.
Also one this which i have noticed is, when the multiple instance are seen in the task manager both the ID's are having same process id. Does it means there the single task configured in the scheduler is triggering multiple instances.
I have attached my technical support info and my poller settings here. can you have a look on this pl.
Thanks.
- Attachments
-
- Technical Support.pdf
- (852.66 KiB) Downloaded 236 times
-
- poller-settings.jpg (320.23 KiB) Viewed 3399 times
Who is online
Users browsing this forum: No registered users and 4 guests