Script Server Problems

sllywhtboy · Post by **sllywhtboy** » Sun Jul 09, 2006 2:12 am

evening all,

i've been trying to get this to run for a day or so without any success. ubuntu dapper (current) linux, 2 667mhz cpu's, 1gb ram. cacti Version 0.8.6h. graphs for all kinds of other things (linux, microsoft, netware, and printers) all work great.

i put the PHP files in the OP in the scripts directory, i imported the template. when i run the poller from the command line or with the script server on the command line, it works. it pulls the data from the table.

command line wrote:/usr/share/cacti/site/scripts# php ss_poller.php ss_poller
Time:292.9616 Method:cmd.php Processes:5 Threads:N/A Hosts:115 HostsPerProcess:23 DataSources:1378 RRDsProcessed:787

script_server wrote:/usr/share/cacti/site# php script_server.php
PHP Script Server has Started - Parent is cmd
/usr/share/cacti/site/scripts/ss_poller.php ss_poller

Time:292.9616 Method:cmd.php Processes:5 Threads:N/A Hosts:115 HostsPerProcess:23 DataSources:1378 RRDsProcessed:787
quit
PHP Script Server Shutdown request received, exiting

i know, it runs slow. leave me alone.

i checked the debug in the graph editor.

debug mode from the graph editor wrote:RRDTool Command:

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="Local - Poller Statistics" \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="seconds" \
--slope-mode \
DEF:a="/usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd":Time:AVERAGE \
DEF:b="/usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd":Processes:AVERAGE \
DEF:c="/usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd":Threads:AVERAGE \
DEF:d="/usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd":Hosts:AVERAGE \
DEF:e="/usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd":HostsPerProcess:AVERAGE \
DEF:f="/usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd":DataSources:AVERAGE \
DEF:g="/usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd":RRDsProcessed:AVERAGE \
LINE1:a#FF00FF:"Poller Runtime" \
GPRINTLAST:" Current\:" \
GPRINTAVERAGE:"Average\:" \
GPRINTMAX:"Max\:\n" \
LINE1:b#FFD660:"Processes" \
GPRINTLAST:" Current\:\n" \
LINE1:c#FFAB00:"Threads" \
GPRINT:c:LAST:" Current\:\n" \
LINE1:d#FF0000:"Hosts" \
GPRINT:d:LAST:" Current\:\n" \
LINE1:e#DE0056:"Hosts per Process" \
GPRINT:e:LAST:"Current\:\n" \
LINE1:f#0000FF:"DataSources" \
GPRINT:f:LAST:" Current\:\n" \
LINE1:g#00CF00:"RRDs" \
GPRINT:g:LAST:" Current\:\n"

RRDTool Says:

ERROR: opening '/usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd': No such file or directory

i tried creating the "missing file" in question and setting RW permissions thinking that cacti/rrd would just "update" the file.
touch /usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd
chown www-data /usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd

i got this error so i just deleted it again since that obviously doesn't work.

graph debug mode wrote:RRDTool Says:

ERROR: reading the cookie off /usr/share/cacti/site/rra/aht_nttc_localhost_rrdsprocessed_964.rrd faild.

cacti logfile pertaining to the ss_poller?

snippet from the cacti.log wrote:/var/log/cacti# tail -n 9000 cacti.log | grep -i "ss_poller"
07/09/2006 02:30:07 AM - CMDPHP: Poller[0] Host[1] DS[917] SERVER: /usr/share/cacti/site/scripts/ss_poller.php ss_poller, output: U
07/09/2006 02:30:07 AM - CMDPHP: Poller[0] Host[1] DS[963] SERVER: /usr/share/cacti/site/scripts/ss_poller.php ss_poller, output: Time:109.9393 Method:cmd.php Processes:5 Threads:N/A Hosts:115 HostsPerProcess:23 DataSources:1379 RRDsProcessed:790
07/09/2006 02:35:06 AM - CMDPHP: Poller[0] Host[1] DS[917] SERVER: /usr/share/cacti/site/scripts/ss_poller.php ss_poller, output: U
07/09/2006 02:35:06 AM - CMDPHP: Poller[0] Host[1] DS[963] SERVER: /usr/share/cacti/site/scripts/ss_poller.php ss_poller, output: Time:292.6977 Method:cmd.php Processes:5 Threads:N/A Hosts:115 HostsPerProcess:23 DataSources:1379 RRDsProcessed:987
07/09/2006 02:40:06 AM - CMDPHP: Poller[0] Host[1] DS[917] SERVER: /usr/share/cacti/site/scripts/ss_poller.php ss_poller, output: U
07/09/2006 02:40:06 AM - CMDPHP: Poller[0] Host[1] DS[963] SERVER: /usr/share/cacti/site/scripts/ss_poller.php ss_poller, output: Time:292.5045 Method:cmd.php Processes:5 Threads:N/A Hosts:115 HostsPerProcess:23 DataSources:1379 RRDsProcessed:794
07/09/2006 02:45:06 AM - CMDPHP: Poller[0] Host[1] DS[917] SERVER: /usr/share/cacti/site/scripts/ss_poller.php ss_poller, output: U

any suggestions for resolving this would be great.

Post by **gandalf** » Thu Jul 20, 2006 6:59 am

This looks very much like the "Duplicate Poller" issue. Please check ALL your crontabs, e.g.
- /etc/crontab
- /etc/cron.d/cacti
- crontabs of root and cacti(user)
for poller entries. Only one should survive
Reinhard

sllywhtboy · Post by **sllywhtboy** » Thu Jul 20, 2006 1:17 pm

lvm wrote:This looks very much like the "Duplicate Poller" issue. Please check ALL your crontabs, e.g.
- /etc/crontab
- /etc/cron.d/cacti
- crontabs of root and cacti(user)
for poller entries. Only one should survive
Reinhard

i didn't see any other entries for cacti's poller in my crons. i only have /etc/cron.d/cacti which tells it to run every 5 minutes and that file has one entry in it.

Post by **gandalf** » Fri Jul 21, 2006 5:34 am

Please look for the host status in cacti.log when running in DEBUG mode (this are the very first line belonging to that specific Host[..] entry)
Reinhard

sllywhtboy · Post by **sllywhtboy** » Fri Jul 21, 2006 12:05 pm

lvm wrote:Please look for the host status in cacti.log when running in DEBUG mode (this are the very first line belonging to that specific Host[..] entry)
Reinhard

i enabled debug in the web interface and included the lined related to 'localhost' where ss_poller is running against.

cacti.log debug mode wrote:tail -n 9000 /var/log/cacti/cacti.log | grep -i "host\[1\]"
07/21/2006 12:55:03 PM - CMDPHP: Poller[0] Host[1] SNMP: Host responded to SNMP
07/21/2006 12:55:03 PM - CMDPHP: Poller[0] Host[1] RECACHE: Processing 2 items in the auto reindex cache for '127.0.0.1'.
07/21/2006 12:55:03 PM - CMDPHP: Poller[0] Host[1] DS[3] CMD: perl /usr/share/cacti/site/scripts/linux_memory.pl MemFree:, output: 226964
07/21/2006 12:55:03 PM - CMDPHP: Poller[0] Host[1] DS[4] CMD: perl /usr/share/cacti/site/scripts/linux_memory.pl SwapFree:, output: 971276
07/21/2006 12:55:03 PM - CMDPHP: Poller[0] Host[1] DS[5] CMD: perl /usr/share/cacti/site/scripts/loadavg_multi.pl, output: 1min:0.54 5min:0.85 10min:0.93
07/21/2006 12:55:03 PM - CMDPHP: Poller[0] Host[1] DS[6] CMD: perl /usr/share/cacti/site/scripts/unix_users.pl, output: 1
07/21/2006 12:55:03 PM - CMDPHP: Poller[0] Host[1] DS[7] CMD: perl /usr/share/cacti/site/scripts/unix_processes.pl, output: 82
07/21/2006 12:55:03 PM - CMDPHP: Poller[0] Host[1] DS[1053] WARNING: Result from SERVER not valid. Partial Result:
07/21/2006 12:55:03 PM - CMDPHP: Poller[0] Host[1] DS[1053] SERVER: /usr/share/cacti/site/scripts/ss_poller.php ss_poller, output: U
07/21/2006 12:55:04 PM - CMDPHP: Poller[0] Host[1] DS[902] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get total 2, output: 1059532800
07/21/2006 12:55:04 PM - CMDPHP: Poller[0] Host[1] DS[902] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get used 2, output: 842645504
07/21/2006 12:55:05 PM - CMDPHP: Poller[0] Host[1] DS[45] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get used 5, output: 0
07/21/2006 12:55:05 PM - CMDPHP: Poller[0] Host[1] DS[47] SNMP: v1: 127.0.0.1, dsname: traffic_in, oid: .1.3.6.1.2.1.2.2.1.10.2, output: 129255151
07/21/2006 12:55:05 PM - CMDPHP: Poller[0] Host[1] DS[47] SNMP: v1: 127.0.0.1, dsname: traffic_out, oid: .1.3.6.1.2.1.2.2.1.16.2, output: 136819565
07/21/2006 12:55:05 PM - CMDPHP: Poller[0] Host[1] DS[539] SNMP: v1: 127.0.0.1, dsname: traffic_in, oid: .1.3.6.1.2.1.2.2.1.10.2, output: 129255151
07/21/2006 12:55:05 PM - CMDPHP: Poller[0] Host[1] DS[539] SNMP: v1: 127.0.0.1, dsname: traffic_out, oid: .1.3.6.1.2.1.2.2.1.16.2, output: 136819565
07/21/2006 12:55:05 PM - CMDPHP: Poller[0] Host[1] DS[45] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get total 5, output: 0
07/21/2006 12:55:06 PM - CMDPHP: Poller[0] Host[1] DS[903] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get used 3, output: 630784
07/21/2006 12:55:07 PM - CMDPHP: Poller[0] Host[1] DS[904] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get used 4, output: 1779585024
07/21/2006 12:55:07 PM - CMDPHP: Poller[0] Host[1] DS[905] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get used 7, output: 21226213376
07/21/2006 12:55:08 PM - CMDPHP: Poller[0] Host[1] DS[905] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get total 7, output: 34845564928
07/21/2006 12:55:08 PM - CMDPHP: Poller[0] Host[1] DS[904] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get total 4, output: 17803001856
07/21/2006 12:55:09 PM - CMDPHP: Poller[0] Host[1] DS[903] CMD: /usr/bin/php4 -q /usr/share/cacti/site/scripts/query_host_partitions.php 127.0.0.1 public 1 get total 3, output: 995217408
07/21/2006 12:55:09 PM - CMDPHP: Poller[0] Host[1] DS[1030] SNMP: v1: 127.0.0.1, dsname: traffic_out, oid: .1.3.6.1.2.1.2.2.1.16.1, output: 13364394
07/21/2006 12:55:09 PM - CMDPHP: Poller[0] Host[1] DS[1030] SNMP: v1: 127.0.0.1, dsname: traffic_in, oid: .1.3.6.1.2.1.2.2.1.10.1, output: 13364394

Post by **gandalf** » Sat Jul 22, 2006 7:46 am

Split off original thread as it's merely a script server problem
Reinhard

Post by **gandalf** » Sat Jul 22, 2006 7:55 am

Do you use the stuff from http://forums.cacti.net/viewtopic.php?t=9272? Where do you place the script?
Reinhard

sllywhtboy · Post by **sllywhtboy** » Sat Jul 22, 2006 3:28 pm

lvm wrote:Do you use the stuff from http://forums.cacti.net/viewtopic.php?t=9272? Where do you place the script?
Reinhard

yeah, i used the stuff from the other thread. i put the script in w/ the rest of the scripts. my distribution places them in /usr/share/cacti/site/scripts/

Post by **gandalf** » Sun Jul 23, 2006 6:44 am

Sorry, currently I'm stuck. The script is quite simple (not far from a one-liner), so there's hardly anything to debug. You checked already cli and script_server operation to no avail. Do you use other script server scripts (NOT data queries)?
And please send a screenshot of the Script's definition within cacti (Data Input Method)
Reinhard

Post by **TheWitness** » Sun Jul 23, 2006 7:13 am

Could be a connection limit issue. Check your MySQL configuration and increase the total number of connections available to users and then restart MySQL.

TheWitness

sllywhtboy · Post by **sllywhtboy** » Mon Jul 24, 2006 12:18 pm

TheWitness wrote:Could be a connection limit issue. Check your MySQL configuration and increase the total number of connections available to users and then restart MySQL.

TheWitness

it's default at 100. mysql administrator monitor doesn't show connections near the limit.

if i am browsing through tons of graphs, then i would see a lot of connections (20 peak in the attachment for example) and an intermittent poller issue. however, this happens all the time. unless my understanding is off, which could be the case

the stats in the graph are typical/normal.

sllywhtboy · Post by **sllywhtboy** » Tue Jul 25, 2006 3:34 pm

i switched the ss_poller form script_server to script and it works fine (had to add php -q to the front)

so it really is a script server issue. what's next to check into?

sllywhtboy · Post by **sllywhtboy** » Tue Jul 25, 2006 3:44 pm

lvm wrote:Sorry, currently I'm stuck. The script is quite simple (not far from a one-liner), so there's hardly anything to debug. You checked already cli and script_server operation to no avail. Do you use other script server scripts (NOT data queries)?
And please send a screenshot of the Script's definition within cacti (Data Input Method)
Reinhard

just about everything is a script/query, the rest are snmp.

i dunno how i missed this, but here you go. i have since changed the data-input method to script, it works.

Post by **TheWitness** » Tue Jul 25, 2006 8:52 pm

It's interesting that the script server is locking up. My question is is what are your php settings set to? I am interested in how much memory you are allowing and what version of PHP are you running? This has been quite stable for a while now.. It could be that another script server is hosing the script server. Cactid handles those situations better, but I would only use the beta that was just released.

TheWitness

sllywhtboy · Post by **sllywhtboy** » Wed Jul 26, 2006 10:32 am

TheWitness wrote:It's interesting that the script server is locking up. My question is is what are your php settings set to? I am interested in how much memory you are allowing and what version of PHP are you running? This has been quite stable for a while now.. It could be that another script server is hosing the script server. Cactid handles those situations better, but I would only use the beta that was just released.

TheWitness

here's a anonymized phpinfo();

Script Server Problems

Who is online