Debian (Wheezy 64bit) - Spine 0.8.8b Lost connection

[trevorgron]

Hi everyone,

We have recently migrated our Cacti installation onto a newly built Debian (Wheezy 64 bit) Server. The Cacti version is 0.8.8b and
we are using Spine version 0.8.8b.

2/27/2014 12:46:05 PM - SYSTEM STATS: Time:4.4935 Method:spine Processes:10 Threads:10 Hosts:22 HostsPerProcess:3 DataSources:1174 RRDsProcessed:863

02/27/2014 12:47:01 PM - SPINE: Poller[0] ERROR: SQL Failed! Error:'2013', Message:'Lost connection to MySQL server during query', SQL Fragment:'UPDATE host SET status='3', status_event_count='0', status_fail_date='0000-00-00 00:00:00', status_rec_date='0000-00-00 00:00:00', status_last_error='Host did not respond to SNMP, ICMP: Ping timed out', min_time='0.040050', max_time='54.656900', cur_time='0.515940', avg_time='2.200598', total_polls='278587', failed_polls='5135', availability='98.1568' WHERE id='5''

Since the migration we have noticed repeated "SPINE: Poller[0] ERROR: SQL Failed! Error:'2013', Message:'Lost connection to MySQL server during query', SQL Fragment:" messages in the Cacti Log File for various hosts. I have tried manually running the MySQL update shown in the Cacti Log File, and it runs with out any errors/warnings.

I have read that other users are having similar problems, and that this might be related to using MySQL-Server version 5.5.35. One forum user "MrRat" posted that he had reverted back to MySQL-Server version 5.1.66, recompiled spine and problems were resolved.

My previous Cacti Server 0.8.8b (Spine version 0.8.8b) was run on Debian (Squeeze 32 bit) with MySQL-Server version 5.1.66 and had no errors
in the Cacti Log File.

Is this a known problem with using MySQL-Server version 5.5.35, and is a patch or new version of Spine going to be released that will resolve these problems?

Any help would be greatly appreciated.
Trevor

[trevorgron]

I have tried various combinations of settings in "Cacti Settings (Poller)". Reducing "Maximum Threads per Process" down to 1 appears
to have resolved the 'Lost connection to MySQL server' problem, though I see this as a workaround and not a fix.

Please could Cacti developers respond as to whether this is a known problem, and if a patch/fix will be brought out for this problem.

Regards
Trevor

[trevorgron]

I see that I am getting quite a few hits on this topic, but that no one has any comments

Am I the only person facing this problem when increasing "Maximum Threads per Process" above 1?

If you currently have, or have previously had this problem, I would appreciate if you can share
your experience.

[cigamit]

I don't believe most people have this issue. It almost sounds like a max connections issue with MySQL, but probably isn't.

Code: Select all

show variables like "max_connections";

[trevorgron]

Hi Cigamit,

Please find below the output of "max connections" :-

mysql> show variables like "max_connections";
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 151 |
+-----------------+-------+
1 row in set (0.00 sec)

Regards
Trevor

[smcline06]

Theres actually an issue when compiling SPINE with MySQL 5.5 and it causes those messages (cant recall where I found that issue, lots of hours goggling). What I had to do was compile spine with MySQL 5.1 installed then move the spine bin to the cacti server and all was well

I will see if I can locate the page again (it mentioned a specific file issue within 5.5).

Update:

The thread with the fix is http://forums.cacti.net/viewtopic.php?f ... 8&start=15

I was just able to spend some more time working on this. The problem seems to be related to spine being built against MySQL-5.5 client libraries. I reverted mysql to 5.1.66 and recompiled spine against those libraries and the problem is solved. specifically libmysqlclient_r.so.16 instead of libmysqlclient_r.so.18 where this problem was occurring. I think that I can probably reinstall MySQL 5.5.x as long as I were to keep the .so.16 for my current spine binary and not rebuild spine against the so.18. I will try to test this next week.

[trevorgron]

Hi smcline06,

Thanks for your reply to the problem I posted.

I also found the thread "spine dies with Lost connection to MySQL server during query".

I will try your tip of recompiling SPINE against MySQL 5.1.

Many thanks
Trevor

[trevorgron]

I have finally managed to resolve this problem with many thanks to smcline06 and an old post by MrRat.

Eventually I compiled MySQL 5.1 in a temporary directory, and copied libmysqlclient_r.so.16.0.0 to /usr/lib/x86_64-linux-gnu
I then removed the symbolic links for MySQL 5.5, created similar symbolic links for
libmysqlclient_r.so.16.0.0 and then recompiled SPINE.

SPINE compiled successfully and I copied it to /usr/sbin/, running SPINE threw up the following error :-

spine: error while loading shared libraries: libmysqlclient_r.so.16: cannot open shared object file: No such file or directory

Creating the symbolic link for libmysqlclient_r.so.16 using the following command resolved this problem :-
ln -s libmysqlclient_r.so.16.0.0 libmysqlclient_r.so.16

I then removed the majority of MySQL 5.1 symbolic links and recreated all the symbolic links for MySQL 5.5 that I had previously removed.