Spine core dump solaris

Post support questions that directly relate to Linux/Unix operating systems.

Moderators: Developers, Moderators

Post Reply
User avatar
davebassjunkie
Posts: 14
Joined: Mon Nov 19, 2007 7:56 pm

Spine core dump solaris

Post by davebassjunkie »

Hi, getting a core dump when running spine on Solaris 10, Netsnmp 5.0.9, mysql 4.1.22, Cacti 0.8.7a, Spine 0.8.7a.

Output shows the following:

Code: Select all

SPINE: Using spine config file [spine.conf]
SPINE: Version 0.8.7b starting
Host[10] ERROR: Problems parsing Multi SNMP OID! (oid: .1.3.6.1.4.1.2021.11.54.0 )

Host[10] DS[118] WARNING: Result from SNMP not valid. Partial Result: ...

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

SNMP: Error generating SNMPv3 Ku from authentication pass phrase.

Segmentation Fault (core dumped)
Backtrace shows the following:

Code: Select all

#0  0xcd2f3805 in t_splay () from /usr/lib/libc.so.1
No symbol table info available.
#1  0xcd2f36d2 in t_delete () from /usr/lib/libc.so.1
No symbol table info available.
#2  0xcd2f2f81 in _malloc_unlocked () from /usr/lib/libc.so.1
No symbol table info available.
#3  0xcd2f2e5a in malloc () from /usr/lib/libc.so.1
No symbol table info available.
#4  0xcd2e1863 in calloc () from /usr/lib/libc.so.1
No symbol table info available.
#5  0xcd6def3b in snmp_pdu_create () from /usr/sfw/lib/libnetsnmp.so.5
No symbol table info available.
#6  0xcd7032ce in snmpv3_build_probe_pdu () from /usr/sfw/lib/libnetsnmp.so.5
No symbol table info available.
#7  0xcd701946 in snmpv3_engineID_probe () from /usr/sfw/lib/libnetsnmp.so.5
No symbol table info available.
#8  0xcd702124 in _sess_open () from /usr/sfw/lib/libnetsnmp.so.5
No symbol table info available.
#9  0xcd702650 in snmp_sess_open () from /usr/sfw/lib/libnetsnmp.so.5
No symbol table info available.
#10 0x08055bab in snmp_host_init (host_id=34, hostname=0x81175fc "eddie", snmp_version=135362712, 
    snmp_community=0x81176f6 "public", snmp_username=0x8117760 "csu", snmp_password= "<removed>", 
    snmp_auth_protocol=0x81177c4 "MD5", snmp_priv_passphrase=0x81177c9 "", snmp_priv_protocol=0x8117891 "[None]", 
    snmp_context=0x8117898 "", snmp_port=134598260, snmp_timeout=500) at snmp.c:266
        sessp = (void *) 0xcd783ab4
        session = {version = 3, retries = 3, timeout = 500000, flags = 0, subsession = 0x0, next = 0x0, 
  peername = 0xcd0c7fa0 "eddie:161", remote_port = 161, local_port = 0, authenticator = 0, callback = 0, 
  callback_magic = 0x0, s_errno = 0, s_snmp_errno = 0, sessid = 0, community = 0x0, community_len = 0, 
  rcvMsgMaxSize = 1472, sndMsgMaxSize = 0, isAuthoritative = 0 '\0', contextEngineID = 0x0, contextEngineIDLen = 0, 
  engineBoots = 0, engineTime = 0, contextName = 0x8117898 "", contextNameLen = 0, securityEngineID = 0x0, 
  securityEngineIDLen = 0, securityName = 0x8117760 "csu", securityNameLen = 3, securityAuthProto = 0x8117a98, 
  securityAuthProtoLen = 10, securityAuthKey = '\0' <repeats 31 times>, securityAuthKeyLen = 32, 
  securityPrivProto = 0x8133dd8, securityPrivProtoLen = 10, securityPrivKey = '\0' <repeats 31 times>, 
  securityPrivKeyLen = 32, securityModel = -1, securityLevel = 2, securityInfo = 0x0, myvoid = 0x0}
        hostnameport = "eddie:161\000\016\bhà\021\b\001\000\000\000\b\000\000\000°\205\fÍh\016fÍià\021\b\000\000\000\000\000\000\000\000\002\000\000\000\001\000\000\000°\205\fÍ\001\000\000\000Ф7Í\b_\r\b\030\200\fÍxìdÍ°\205\fÍ\020\200\fÍ\b\000\000\000°\205\fÍh\016fÍh\016fÍh\016fÍ(\200\fÍ°\205\fÍ\000\000\000\000´\004dÍ,\200\fÍ\203\207cÍ°\205\fÍ\036\000\000\000h\016fÍ`\200\fÍk\224cÍ°\205\fÍ ·\fÍ\000\000\000\000h\016fÍø^\r\bÐ\201\021\b\031\203\021\bÐ\201\021\b\030\203\021\b°\205\fÍhà\021\b\210\200\fÍ\212µcÍ"...
#11 0x08058b7c in poll_host (host_id=34) at poller.c:351
        query1 = "SELECT action, hostname, snmp_community, snmp_version, snmp_username, snmp_password, rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, rrd_num, snmp_port, snmp_timeout, snmp_auth_protocol, snmp_pri"...
        query2 = "SELECT id, hostname, snmp_community, snmp_version, snmp_username, snmp_password, snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_port, snmp_timeout, max_oids, availabi"...
        query3 = 0x8122da0 "°\030\022\bRT I¸\201\021\bpoll"
        query4 = "SELECT data_query_id, action, op, assert_value, arg1 FROM poller_reindex WHERE host_id=34", '\0' <repeats 934 times>
        query5 = "SELECT action, hostname, snmp_community, snmp_version, snmp_username, snmp_password, rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, rrd_num, snmp_port, snmp_timeout, snmp_auth_protocol, snmp_pri"...
        query6 = "UPDATE poller_item SET rrd_next_step=rrd_next_step-300 WHERE host_id=34", '\0' <repeats 952 times>
        query7 = "UPDATE poller_item SET rrd_next_step=rrd_step-300 WHERE rrd_next_step < 0 and host_id=34", '\0' <repeats 935 times>
        query8 = "INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES", '\0' <repeats 951 times>
        query9 = "SELECT snmp_port, count(snmp_port) FROM poller_item WHERE host_id=34 GROUP BY snmp_port", '\0' <repeats 936 times>
        query10 = "SELECT snmp_port, count(snmp_port) FROM poller_item WHERE host_id=34 AND rrd_next_step < 0 GROUP BY snmp_port", '\0' <repeats 914 times>
        errstr = '\0' <repeats 1023 times>
        sysUptime = '\0' <repeats 39 times>
        result_string = '\0' <repeats 1023 times>
        num_rows = 1
        assert_fail = 0
        spike_kill = 0
        rows_processed = 0
        i = 0
        j = 0
        num_oids = 0
        out_buffer = 135365080
        php_process = 135365080
        poll_result = 0x0
        host_time = 0x8122680 "2008-01-10 11:35:50"
        update_sql = "UPDATE host SET status='3', status_event_count='1', status_fail_date='2008-01-10 11:35', status_rec_date='2008-01-08 16:26:37', status_last_error='Host did not respond to SNMP', min_time='9.999990', m"...
        temp_result = '\0' <repeats 1023 times>
        last_snmp_version = 0
        last_snmp_port = 0
        last_snmp_community = "public", '\0' <repeats 43 times>
        last_snmp_username = "csu", '\0' <repeats 46 times>
        last_snmp_password = "<removed>", '\0' <repeats 41 times>
        last_snmp_auth_protocol = "MD5\000"
        last_snmp_priv_passphrase = '\0' <repeats 199 times>
        last_snmp_priv_protocol = "[None"
        last_snmp_context = '\0' <repeats 64 times>
        previous_assert_failure = 0
        last_data_query_id = 0
        new_buffer = 1
        reindex = (reindex_t *) 0x8120078
        host = (host_t *) 0x81175f8
        ping = (ping_t *) 0x8117e50
        poller_items = (target_t *) 0x8143d88
        snmp_oids = (snmp_oids_t *) 0x811dfe8
        mysql = {net = {vio = 0x80e0e90, buff = 0x811e068 "þ", buff_end = 0x8120068 "", write_pos = 0x811e068 "þ", 
    read_pos = 0x811e068 "þ", fd = 24, max_packet = 8192, max_packet_size = 1073741824, pkt_nr = 0, 
    compress_pkt_nr = 35, write_timeout = 35, read_timeout = 31536000, retry_count = 31536000, fcntl = 1, 
    compress = 0 '\0', remain_in_buf = 0, length = 0, buf_length = 0, where_b = 0, return_status = 0x0, 
    reading_or_writing = 0 '\0', save_char = 0 '\0', no_send_ok = 0 '\0', 
    last_error = '\0' <repeats 213 times>, "(B\r\bÀ'\021\bÐí\022\b\000\000\000\000@B\r\b\020B\r\b\000\000\000\000¸*\023\b\220<fÍ\000\000\000\000 -\022\b\000\000\000\000\000\000\000\000 \000\000\000ä\037\000\000\005", '\0' <repeats 11 times>, "\001", '\0' <repeats 23 times>, "í°\000\000\000\000\000\000ê\f\000\000\215 \000\000,¢\000\000\n\000\000\000\036\000\000\000\002\000\000\000\b\000\000\000\005\000\000\000\200", '\0' <repeats 84 times>, "\001wm;~+Y\\E\000\001°\205\fÍ°\205\fÍ\000\000\000\000°\205\fÍ", '\0' <repeats 50 times>, sqlstate = "\000\000\000\000\000", last_errno = 0, error = 0 '\0', 
    query_cache_query = 0x0, report_error = 0 '\0', return_errno = 0 '\0'}, connector_fd = 0x0, host = 0x0, user = 0x0, 
  passwd = 0x0, unix_socket = 0x0, server_version = 0x0, host_info = 0x0, info = 0x0, db = 0x0, charset = 0x0, 
  fields = 0x0, field_alloc = {free = 0x0, used = 0x0, pre_alloc = 0x0, min_malloc = 0, block_size = 0, block_num = 0, 
    first_block_usage = 0, error_handler = 0}, affected_rows = 0, insert_id = 0, extra_info = 0, thread_id = 0, 
  packet_length = 0, port = 0, client_flag = 0, server_capabilities = 0, protocol_version = 0, field_count = 0, 
  server_status = 0, server_language = 0, warning_count = 0, options = {connect_timeout = 0, read_timeout = 0, 
    write_timeout = 0, port = 0, protocol = 0, client_flag = 0, host = 0x0, user = 0x0, password = 0x0, unix_socket = 0x0, db = 0x0, init_commands = 0x0, my_cnf_file = 0x0, my_cnf_group = 0x0, charset_dir = 0x0, 
    charset_name = 0x0, ssl_key = 0x0, ssl_cert = 0x0, ssl_ca = 0x0, ssl_capath = 0x0, ssl_cipher = 0x0, 
    shared_memory_base_name = 0x0, max_allowed_packet = 0, use_ssl = 0 '\0', compress = 0 '\0', named_pipe = 0 '\0', 
    rpl_probe = 0 '\0', rpl_parse = 0 '\0', no_master_reads = 0 '\0', separate_thread = 0 '\0', 
    methods_to_use = MYSQL_OPT_CONNECT_TIMEOUT, client_ip = 0x0, secure_auth = 0 '\0', local_infile_init = 0, 
    local_infile_read = 0, local_infile_end = 0, local_infile_error = 0, local_infile_userdata = 0x0}, 
  status = MYSQL_STATUS_READY, free_me = 0 '\0', reconnect = 0 '\0', scramble = '\0' <repeats 20 times>, 
  rpl_pivot = 0 '\0', master = 0x0, next_slave = 0x0, last_used_slave = 0x0, last_used_con = 0x0, stmts = 0x0, 
  methods = 0x0, thd = 0x0, unbuffered_fetch_owner = 0x0, current_stmt = 0x0}
        result = (MYSQL_RES *) 0x8117a98
        row = (MYSQL_ROW) 0x81181d8
#12 0x08059bc0 in child (arg=0x8b18fe05) at poller.c:51
        host_id = 34
#13 0xcd34fc42 in _thr_setup () from /usr/lib/libc.so.1
No symbol table info available.
#14 0xcd34ff30 in L3_doit () from /usr/lib/libc.so.1
No symbol table info available.
#15 0xcd7c2400 in ?? ()
No symbol table info available.
#16 0x00000000 in ?? ()
No symbol table info available

Any help is much appreciated, thanks!
User avatar
davebassjunkie
Posts: 14
Joined: Mon Nov 19, 2007 7:56 pm

Post by davebassjunkie »

Anyone had a chance to look at this?
User avatar
davebassjunkie
Posts: 14
Joined: Mon Nov 19, 2007 7:56 pm

Post by davebassjunkie »

Still having these issues, thought I'd try again, rebuilt and still same problems.

Code: Select all

root@monitor /local/spine # su - cacti -c "/local/spine/bin/spine --conf=/local/spine/etc/spine.conf"
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
SPINE: Using spine config file [/local/spine/etc/spine.conf]
SPINE: Version 0.8.7b starting
09/05/2008 02:30:30 PM - SPINE: Poller[0] Host[10] ERROR: Problems parsing Multi SNMP OID! (oid: .1.3.6.1.4.1.2021.11.54.0 )
09/05/2008 02:30:30 PM - SPINE: Poller[0] Host[10] DS[118] WARNING: Result from SNMP not valid. Partial Result: ...
09/05/2008 02:30:30 PM - SPINE: Poller[0] FATAL: Spine Encountered a Segmentation Fault (Spine thread)

Code: Select all

root@monitor /local/spine # su - cacti -c "/local/spine/bin/spine --conf=/local/spine/etc/spine.conf -V 5 -f 4 -l 4"
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
SPINE: Using spine config file [/local/spine/etc/spine.conf]
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'path_webroot''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'path_cactilog''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The path_php_server variable is /local/cacti-0.8.7b/script_server.php
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The path_cactilog variable is /local/cacti-0.8.7a/log/cacti.log
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_destination''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The log_destination variable is 1 (FILE)
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'path_php_binary''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The path_php variable is /local/php/bin/php
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'availability_method''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The availability_method variable is 2
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_recovery_count''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The ping_recovery_count variable is 3
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_failure_count''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The ping_failure_count variable is 2
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_method''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The ping_method variable is 2
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_retries''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The ping_retries variable is 1
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'ping_timeout''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The ping_timeout variable is 400
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_perror''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The log_perror variable is 1
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_pwarn''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The log_pwarn variable is 0
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'log_pstats''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The log_pstats variable is 0
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'max_threads''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The threads variable is 8
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'poller_interval''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The polling interval is 300 seconds
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'concurrent_processes''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The number of concurrent processes is 3
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'script_timeout''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The script timeout is 25
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'php_servers''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The number of php script servers to run is 8
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT action FROM poller_item WHERE action=2 AND host_id BETWEEN 4 AND 4 LIMIT 1'
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: StartHost='4', EndHost='4', TotalPHPScripts='0'
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The PHP Script Server is Not Required
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT value FROM settings WHERE name = 'max_get_size''
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: The Maximum SNMP OID Get Size is 15
09/05/2008 02:30:03 PM - SPINE: Poller[0] Version 0.8.7b starting
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: MySQL is Thread Safe!
09/05/2008 02:30:03 PM - SPINE: Poller[0] SPINE: Initializing Net-SNMP API
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SNMP Header Version is 5.3.0.1
09/05/2008 02:30:03 PM - SPINE: Poller[0] DEBUG: SNMP Library Version is 5.3.0.1
09/05/2008 02:30:04 PM - SPINE: Poller[0] SPINE: Initializing PHP Script Server(s)
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT id FROM host WHERE disabled='' AND id BETWEEN 4 AND 4 ORDER BY id'
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: Initial Value of Active Threads is 0
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: Valid Thread to be Created
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 1
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: In Poller, About to Start Polling of Host
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT snmp_port, count(snmp_port) FROM poller_item WHERE host_id=0 AND rrd_next_step < 0 GROUP BY snmp_port'
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: SQL:'SELECT action, hostname, snmp_community, snmp_version, snmp_username, snmp_password, rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, rrd_num, snmp_port, snmp_timeout, snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context  FROM poller_item WHERE host_id=0 and rrd_next_step <=0 ORDER by snmp_port'
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_next_step-300 WHERE host_id=0'
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: SQL:'UPDATE poller_item SET rrd_next_step=rrd_step-300 WHERE rrd_next_step < 0 and host_id=0'
09/05/2008 02:30:04 PM - SPINE: Poller[0] Host[0] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: The Value of Active Threads is 0
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: SQL:'replace into settings (name,value) values ('date',NOW())'
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: SQL:'insert into poller_time (poller_id, start_time, end_time) values (0, NOW(), NOW())'
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: Thread Cleanup Complete
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: PHP Script Server Pipes Closed
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: Allocated Variable Memory Freed
09/05/2008 02:30:04 PM - SPINE: Poller[0] SPINE: Net-SNMP API Shutdown Completed
09/05/2008 02:30:04 PM - SPINE: Poller[0] DEBUG: MYSQL Free & Close Completed
09/05/2008 02:30:04 PM - SPINE: Poller[0] Time: 1.2108 s, Threads: 8, Hosts: 1
Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests