cactid problems

Morgan · Post by **Morgan** » Fri Mar 19, 2004 5:57 am

I'm going to try to be as detailed as possible here.

Machine:
SunOS 5.6 Generic_105181-23 sun4u sparc SUNW,Ultra-80

i'm having a few problems with cactid. I'm running an 0.8.6 version of cactid as suggested by Rax, as the patches do not apply properly on solaris.

I've put everything i have in there that's relevant
the rrd.log from cactid is showing only what i pasted below, the rest is all normal. the only errors listed are snmp errors when one of my devices is down.

:/usr/local/cacti/rra# ls | grep -ic rrd
576

the processes sometime seem to get stuck, and mainly with scripts. Yes it is the same behaviour as the patch is suposed to fix, and i am seeing gaps in my script graphs.

Any thoughts as to how to fix it as a 0.8.6 version of cactid doesnt seem to have done the trick.

Thanks a lot,
Morgan

Code: Select all

:/usr/local/cacti/log# ps -ef | grep cacti
    root 15199 15198  0 06:10:00 ?        0:01 ./cactid
    root 14927   191  0 06:00:00 ?        0:00 sh -c cd /usr/local/cactid/bin; ./cactid > /dev/null 2>&1
    root 15236 15199  0 06:10:10 ?        0:00 sh -c /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/snmp_multi_stats.pl 
    root 15253 15199  0 06:10:11 ?        0:00 sh -c /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/snmp_multi_stats.pl 
    root 14928 14927  0 06:00:00 ?        0:01 ./cactid
    root 15198   191  0 06:10:00 ?        0:00 sh -c cd /usr/local/cactid/bin; ./cactid > /dev/null 2>&1
    root 15262 15199  0 06:10:12 ?        0:00 sh -c /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/cpu_dns_stats.pl 10.1.
    root 15269 15268  0 06:10:13 ?        0:00 /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/smtp_messages.pl 10.3.
    root 15011 15010  0 06:00:14 ?        0:00 /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/smtp_volume.pl 10.3.
    root 15237 15236  0 06:10:10 ?        0:00 /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/snmp_multi_stats.pl community 1
    root 15254 15253  0 06:10:11 ?        0:00 /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/snmp_multi_stats.pl community
    root 15263 15262  0 06:10:12 ?        0:00 /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/cpu_dns_stats.pl 10.1.
    root 15010 14928  0 06:00:14 ?        0:00 sh -c /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/smtp_volume.pl 10.3.
    root 15268 15199  0 06:10:13 ?        0:00 sh -c /usr/local/bin/perl /usr/local/cacti-0.8.5/scripts/smtp_messages.pl 10.3.

Code: Select all

[4] SNMP v2: router, dsname: traffic_out, oid: .1.3.6.1.2.1.2.2.1.16.6, value: 2239213440
RRDCMD: update '/usr/local/cacti-0.8.5/rra/routernameremoved_traffic_in_12.rrd' --template traffic_in:trafficOK u:0.00 s:0.00
r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.00 r:2.45
OK u:0.00 s:0.01 r:2.45
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
RRDtool 1.0.46  Copyright 1997-2001 by Tobias Oetiker <tobi@oetiker.ch>

Usage: rrdtool [options] command command_options

Valid commands: create, update, graph, dump, restore,
                last, info, fetch, tune, resize, xport

RRDtool is distributed under the Terms of the GNU General
Public License Version 2. (www.gnu.org/copyleft/gpl.html)

For more information read the RRD manpages

OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
RRDtool 1.0.46  Copyright 1997-2001 by Tobias Oetiker <tobi@oetiker.ch>

Usage: rrdtool [options] command command_options

Valid commands: create, update, graph, dump, restore,
                last, info, fetch, tune, resize, xport

RRDtool is distributed under the Terms of the GNU General
Public License Version 2. (www.gnu.org/copyleft/gpl.html)

For more information read the RRD manpages

OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.01 r:2.46
OK u:0.00 s:0.02 r:2.46
OK u:0.00 s:0.02 r:2.47
OK u:0.00 s:0.02 r:2.47
OK u:0.00 s:0.02 r:2.47
OK u:0.00 s:0.02 r:2.47
OK u:0.00 s:0.02 r:2.47
OK u:0.00 s:0.02 r:2.47
OK u:0.00 s:0.02 r:2.47
OK u:0.00 s:0.02 r:2.47
OK u:0.00 s:0.02 r:2.47
OK u:0.00 s:0.02 r:2.47
_out N:83368469:2239213440

Code: Select all

[50] SNMP v2: routername, dsname: traffic_out, oid: .1.3.6.1.2.1.2.2.1.16.12, value: 688093440
RRDCMD: update '/usr/OK u:0.00 s:0.04 r:6.29
OK u:0.00 s:0.04 r:6.29
OK u:0.00 s:0.04 r:6.29
OK u:0.00 s:0.04 r:6.29
OK u:0.00 s:0.04 r:6.29
OK u:0.00 s:0.04 r:6.29
OK u:0.00 s:0.04 r:6.29
OK u:0.00 s:0.05 r:6.29
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.00 s:0.05 r:6.30
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.31
OK u:0.01 s:0.05 r:6.33
OK u:0.01 s:0.05 r:6.33
OK u:0.01 s:0.05 r:6.33
OK u:0.01 s:0.05 r:6.33
OK u:0.01 s:0.05 r:6.33
local/cacti-0.8.5/rra/routernameremoved_in_171.rrd' --template traffic_in:traffic_out N:1734588548:688093440
[51] SNMP v2: routername, dsname: 5min_cpu, oid: .1.3.6.1.4.1.9.2.1.58.0, value: 0

Morgan · Post by **Morgan** » Sat Mar 20, 2004 6:44 am

anyone?

Post by **raX** » Mon Mar 22, 2004 12:03 am

I talked with Morgan today via AIM and suggested that he use the latest cactid version:

http://www.raxnet.net/downloads/cacti/b ... 321.tar.gz

Another Windows user of cactid suggested that I check for the return value of popen to prevent potential deadlocks. I have yet to run into this myself, but I figure it might help in this situation. I believe the problem described here is different than the problem fixed the by the original cactid patch for 0.8.5. That problem was quite related to SNMP and should not have affected scripts at all.

-Ian

Morgan · Post by **Morgan** » Mon Mar 22, 2004 5:43 am

so far so good Ian. i compiled cactid from the latest cvs at around 11 pm CET last night.
12 hours later, i still don't have any stuck processes.

I'll keep a close eye on it for the next couple of days.

M

Morgan · Post by **Morgan** » Tue Mar 23, 2004 11:17 am

Ian,

not a single stuck process yet.

This looks to be a keeper. My solaris box is know happy again

Morgan

Post by **raX** » Wed Mar 24, 2004 12:25 am

Excellent!! If I can find a spare three minutes in the next couple of days, I will definitely release 0.8.5a.

-Ian

dieter · Post by **dieter** » Wed Mar 24, 2004 1:39 am

Will this update solve my problems with cactid? The stuck sh.exe processes or is it a fault of cywin?

I saw in a book of VB.NET code for making multiple threads and I don't see any thread_mutex_lock. Maybe this is a linux call? If I find some time the next weeks I will try to convert it to a windows version but I'm afraid its going to be too difficult for me

Morgan · Post by **Morgan** » Wed Mar 24, 2004 10:46 am

not sure about this as it's windows related, and i'm running this on solaris.

this is a fix for stuck processes when cactid uses external scripts to get data as opposed to snmp queries.

M

Post by **raX** » Tue Mar 30, 2004 1:32 am

dieter wrote:Will this update solve my problems with cactid? The stuck sh.exe processes or is it a fault of cywin?

I believe so. I have yet to test this myself, but the user who originally submitted the patch to me was using cactid on Windows.

dieter wrote:I saw in a book of VB.NET code for making multiple threads and I don't see any thread_mutex_lock. Maybe this is a linux call? If I find some time the next weeks I will try to convert it to a windows version but I'm afraid its going to be too difficult for me

I use the thread_mutex_* functions internally as wrappers to the actual pthread functions. See locks.c for more information on this. If you have any Windows C programming experience, that would be great. I would love to have a natively compiled cactid at some point although it might require a decent bit of code changes.

-Ian

Morgan · Post by **Morgan** » Tue Mar 30, 2004 4:55 am

raX wrote:Excellent!! If I can find a spare three minutes in the next couple of days, I will definitely release 0.8.5a.

-Ian

i just had to kill a few cactid processes stuck again.
a lot less than before, but had about 1 a day for the past 4-5 days stuck in there.

no snmp requests stuck though....

mooseman · Post by **mooseman** » Tue Mar 30, 2004 10:38 pm

I compiled the same version that you posted, and I had some stuck processes, but they were at the rrdtool stage. I can't remember the exact command they were running.

Morgan · Post by **Morgan** » Wed Mar 31, 2004 3:53 am

rax, i confirm what mooseman said above. i have another stuck process this morning:

root 14080 14079 0 09:45:00 ? 0:01 ./cactid
root 14081 14080 0 09:45:00 ? 0:00 sh -c /usr/local/bin/rrdtool -
root 14079 191 0 09:45:00 ? 0:00 sh -c cd /usr/local/cactid/bin; ./cactid > /dev/null 2>&1
root 14082 14081 0 09:45:00 ? 0:00 /usr/local/bin/rrdtool -

btw, cactid could use the option to pass it an argument for the cactid.conf file path
let me know what kinda info you want me to gather if any

mooseman · Post by **mooseman** » Wed Mar 31, 2004 10:33 am

Yep, that's what I'm seeing. I'll grab the latest beta, and try that out today.

Post by **raX** » Sat Apr 03, 2004 5:31 pm

I have a patch that might help fix the stuck RRDTool process problem. Just download the patch to your cactid directory and apply it like so:

Code: Select all

wget http://www.raxnet.net/downloads/cacti/patches/0.8.5a/cactid_rrdtool_pipe.patch
patch -p2 < cactid_rrdtool_pipe.patch

Make sure to recompile cactid after applying this patch and let me know how it goes.

-Ian

mooseman · Post by **mooseman** » Sat Apr 03, 2004 6:09 pm

Great I'll give it a shot next week.

By the way did you get my e-mail last week? I also cc'd my work e-mail but didn't receive it. So I was wondering if you got it.

Cacti

cactid problems

cactid problems

Who is online