Help Sql table error

[jay]

I have sent the DB dump as requested.


Jay

[TheWitness]

Without having a backup first, truncating the graph_template_graph would have been a HUGE mistake. Do you have a backup from prior to the crash? Are you running backups? Please resond.

TheWitness

[jay]

Hi

Sorry for the delay, time zones and all that jazz.

When i migrated cacti onto a new server i forgot to set up the backup batch job Luckily our server team run backups on the server and i will ask them to do a backup for me sometime today.

Jay

[jay]

Phhheeeeww. Our server team backed up the mysql/data/cacti folder and i can now see the graphs. I only lost about 15 minutes of data while i removed the corrupt files from the folder and then restored the data from last week. I have now made a backup and have set up daily backups using mysql administrator.

I'm still unsure why the database got corrupted like it did. At the time of corruption i remember deleting a graph template which was only being used by one device. I can't see that causing a problem.

Maybe the mysqldump will show you a possible bug.

Many thanks for all your assistance. Keep up the great work. Cacti ROCKS!!

Jay

[jay]

Hi

This has now crashed again. I have just got into the office and it looks like i have got exactly the same problem with Cacti. No graphs in graph management and im seeing the same mysql table error

060509 11:14:26 [ERROR] C:\MySQL\bin\mysqld-nt: Table '.\cacti\graph_templates_graph' is marked as crashed and should be repaired

I also noticed about 90 cactid/catiplus windows open on the server.

I have tried to repair the tables but still no graphs at all in graph management.

There looks to be a deeper problem and restoring the mysql database again doesn't look like it will help at all.

Can you offer me some assitance again please??

Regards

Jay

[BSOD2600]

What else is the 'server team' doing each weekend which is corrupting MySQL/cacti ? Weekly backups or something?

[TheWitness]

Likely a hard reboot. The hosts table can tell you when the host was last down. Not from the UI, but from either Monitor plugin or via accessing the table directly using something like MySQL Query Browser.

TheWitness

[TheWitness]

"sometimes" the server teams perform weekly reboots to clear memory without assessing impact. Cacti servers (pollers and mysql) should be rebooted very infrequently. And never during major I/O operations.

Could also be bad physical disk too....

TheWitness

[jay]

I have restored the mysql DB again.

I spoke to the server team and they have been running backups on this since the server was set up. Nothing has changed with the backup process. They also dont reboot the server without our teams permission beforehand, unless its an emergency. The server does get rebooted when they are updates for windows to install that require a reboot. This occurs on a monthly basis. The best wayi can think of rebooting is after Cactid has run a polling cycle.

It has not been hard rebooted since the server was installed as far as i can tell. I checked the host table and the dates the hosts have been up for are varied.

I do notice that Cactid has multiple windows when i log in. I noticed this today after i came in after a long weekend off. There was no downtime at the weekend. I'm sure i have seen a script on the forums which would kill the cactid process if it didn't finish properly but i cant seem to find this. Would this help at all?? Maybe this is causing the table corruption error?? It also creates large gaps in the graphs due to the polling cycle not completing.

As i was writing this post i checked the cacti server and i noticed 2 cactid windows which have just hung and not finished their poll. This means that the problem with that is not due to backups over night.

I get this error

No log handling enabled - turning on stderr logging
read_config_store open failure on /var/net-snmp/cactid.conf

To add to this Cacti is built on a VM.

Any advice is appreciated.

Regards

Jay

[TheWitness]

Try this version of Cactid. Also, make sure you utilize another account to run the poller process, then you won't get those windows hanging around, unless cactid crashes.

Larry

[jay]

I have tried the version of Cactid but when i try and run it i get the following error.

cygcrypto-0.9.8.dll was not found

I only have the cygwin.dll file in the cactid folder along with the cactid.conf file. I do also have 2 stackdumps in there as well..not sure why.

I dont use cygwin on the server. All i did was to move the old cactid and put the new cactid into the folder. I dont compile Cactid at all. I takea it i need cygwin in order for this new version of Cactid to work???

Jay

[TheWitness]

Post the stackdumps. Here is the missing file. Use both the crypto*.dll and cygwin1.dll files from the zip file.

Larry

[jay]

Hi

I have tried that but i know get another .dll file not found error.

cygz.dll not found.

Even if you send me this file i have a feeling i will only get another error.

I have also noticed this error when running cactid and php -m - Failed to start up concurrent users module

My setup doesn't look very happy but is hanging on in there

Here are my poller stats

05/12/2006 11:18:34 AM - SYSTEM STATS: Time:95.9653 Method:cactid Processes:1 Th
reads:10 Hosts:189 HostsPerProcess:189 DataSources:2647 RRDsProcessed:1282



Looks ok to me.


Not really sure what to do next other than start thinking about a whole new install of Cacti and then moving the rra over to the new server.

Is there a recommended set up for Cacti e.g mysql ver, php ver etc ???

Jay

[jay]

Here is the info from the stackdumps..

Cactid.exe.stackdump


Exception: STATUS_ACCESS_VIOLATION at eip=00464F17
eax=00000000 ebx=00710C68 ecx=00000000 edx=00000000 esi=00710CC0 edi=00710C68
ebp=1972EF78 esp=1972C20C program=C:\Cactid\cactid.exe, pid 7452, thread unknown (0x23D0)
cs=001B ds=0023 es=0023 fs=003B gs=0000 ss=0023
Stack trace:
Frame Function Args
1972EF78 00464F17 (0000007F, 00000000, 00000000, 00710C68)
1972EF98 00404CFE (005E92C8, FFFFFFFF, 1972EFD8, 61003B7D)
1972EFD8 6109D17E (00710C68, 1972F010, 6109D110, 00000000)
1972F008 61003E94 (00000000, 00000000, 00000000, 00000000)
1972FFA8 61003EDA (00000000, 00000000, 00000000, 00000000)
End of stack trace
8656931 [unknown (0xC08)] cactid 7452 cygthread::detach: called detach but inuse 0, thread 0x1264?




--------


rrdtool.exe.stackdump

Exception: STATUS_STACK_OVERFLOW at eip=610DFBB3
eax=00004034 ebx=000004E4 ecx=002218B4 edx=00226900 esi=00000018 edi=00000018
ebp=002268B8 esp=002268AC program=c:\rrdtool\rrdtool.exe, pid 25684, thread main
cs=001B ds=0023 es=0023 fs=003B gs=0000 ss=0023
Stack trace:
Frame Function Args
002268B8 610DFBB3 (10041540, 00000018, 00226938, 610020CC)
0022A918 6102135B (611559C0, 10041540, 10041559, 6104FA08)
0022A958 61021BEE (611559C0, 10041540, 00000019, 00000019)
0022A9C8 6108F815 (00000001, 0022A9E0, 00000001, 00000000)
0022A9E8 6108FABD (00000001, 10041540, 00000019, 147AE147)
0022AA08 610DF696 (0022F144, 00000001, 10041540, 00000019)
0022AA38 610DC59F (610E3038, 10041540, 00000019, 0000000B)
0022AA68 610CC031 (610E3038, 0042930E, 00000001, 00000000)
0022AAA8 610DC2EB (610E3038, 0022C420, 0022AAC8, 00000001)
0022AAC8 610C807C (0022F144, 0022B06E, 0042930F, 00000001)
0022C6E8 610CB7E9 (0022F144, 610E3038, 004292F4, 0022C74C)
0022C708 610CB9C8 (610E3038, 004292F4, 0022C734, 10041370)
0022C728 610D4CA5 (004292F4, 00000000, 00000000, DF3B645A)
0022EEE8 610844FF (00000006, 10040200, 10040090, 610A5E22)
0022EFD8 61004DD2 (0022EFF0, FFFFFFFF, 00000000, 00000070)
0022FF88 6100594F (00000000, 00000000, 00000000, 00000000)
End of stack trace (more stack frames may be present)


Jay

[TheWitness]

Best to follow BSOD2600's installation instrauctions allong with Cygwin and the few modules. Once that is done, you can delete the dll's out of the Cactid directory. If you go with the minimal install of Cygwin, it only takes a few minutes.

TheWitness