Cacti Not Freeing up Semaphores???

Post support questions that directly relate to Linux/Unix operating systems.

Moderators: Developers, Moderators

Post Reply
MikeSullivan
Posts: 5
Joined: Mon Aug 19, 2002 12:52 pm

Cacti Not Freeing up Semaphores???

Post by MikeSullivan »

I installed cacti about two weeks ago. Recently I was removing some virtual hosts from my httpd.conf file and then restarted apache. I was surprised when I received the following error:

Ouch! ap_mm_create(1048576, "/var/run/httpd.mm.27323") failed
Error: MM: mm:core: failed to acquire semaphore (No space left on device): OS: Invalid argument :(

Anyway, after running the error through Google, I discovered the "ipcs" command and how to determine what was using up the semaphores:

tux:~$ ipcs

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status

------ Semaphore Arrays --------
key semid owner perms nsems status
0x00000000 1028 getreg 600 1
0x00000000 1029 getreg 600 1
0x00000000 1030 getreg 600 1
0x00000000 1031 getreg 600 1

//-- snipped the other 500 lines --//
(there was over 500 lines for the getreg user)

The user "getreg" is the user that the cacti cron job runs under. I then used a little script to free up the semaphores:

#!/bin/sh
ipcs | egrep getreg | awk '{print $2}' | xargs -n1 ipcrm sem;
ipcs | egrep getreg | awk '{print $1}' | xargs -n1 ipcrm sem

After that, apache starts up again just fine. Very quickly though, all of the semaphores are again used up.

Does anyone have any ideas on how to remedy this?? I have searched the net high and low for possible configurations problems, but there just isn't much info available.

Thanks!
Mike Sullivan

Out the NIC across the switch, off the router
down the wire, nothing but net...
raX
Lead Developer
Posts: 2243
Joined: Sat Oct 13, 2001 7:00 pm
Location: Carlisle, PA
Contact:

Post by raX »

I've never run into this problem myself, even with cacti installations that have been running for over a year. I am suspecting that the cause could be with a script you are calling to gather data (I could be wrong though).

If I were going to debug this problem, that is the first place I would check.

-Ian
MikeSullivan
Posts: 5
Joined: Mon Aug 19, 2002 12:52 pm

Post by MikeSullivan »

The only scripts I am currently using are the ones that came with cacti. I am only monitorring three routers, two switches (36 total ports), and the box on which cacti is installed. They were all setup using the SNMP interface scripts under the "Data Gathering" category of the console. Is it possible I am monitoring too much?

I didn't mean to imply necessarily that cacti is the culprit, but I am baffled as to what could be wrong and was hoping someone else had run into this particular problem. I have managed a workaround with a cron job to free up the semaphores in the meantime, but would really like to find the root cause of the problem.

Any more ideas??
Mike Sullivan

Out the NIC across the switch, off the router
down the wire, nothing but net...
phriday613
Posts: 32
Joined: Thu Aug 15, 2002 10:29 am
Location: ct
Contact:

Post by phriday613 »

this is an old thread, but ill post.. who knows, it may help someone else! :)


i have a problem with semaphores as well.. my webserver/fileserver i have x server on. In using KDE, i noticed i have 468 processes running, at once!!! it was killing everything!! apache had 50mb along with php, mysql and perl, i had no memory left! 128mb is in, but it should be fine!

first, i found a temporary fix with this little command:

echo 250 32000 32 250 > /proc/sys/kernel/sem

also, i switched over to gnome.. I know i know, i shouldnt be running x on a webserver or fileserver, but the box was a learning tool before it became a server of any kind.. since, i just left it on :)

hope this helps anyone..
-michael
raX
Lead Developer
Posts: 2243
Joined: Sat Oct 13, 2001 7:00 pm
Location: Carlisle, PA
Contact:

Post by raX »

I must admit I have seen this kind of thing happen before as well. It really only happens when there is some sort of network outtage causing timeouts all over the place. Since cmd.php is not multi-threaded it just waits for each process to finish. Then after five minutes another cmd.php starts, many times before the previous one has finished. Hence your sitution...

The obvious short term fix for this problem is to have cmd.php use a lock file and not spawn if another copy is not currently running. The long term solution is to make cmd.(c|php|pl) ;-) multi-threaded.

-Ian
deadzen

the semaphore thing

Post by deadzen »

that's a freebsd issue
gilson
Posts: 27
Joined: Wed Jan 23, 2002 7:00 pm
Contact:

Re: the semaphore thing

Post by gilson »

deadzen wrote:that's a freebsd issue
I have the same problem on RH73 and RH80.

If you need to restart Apache and there is no semaphores available, you can try this script in Red Hat (restart_apache.sh):

Code: Select all

#!/bin/sh
service httpd stop
for a in `ipcs -s|grep apache|awk '{print $2}'`; do ipcrm sem $a; done
for a in `ipcs -m|grep apache|awk '{print $2}'`; do ipcrm shm $a; done
service httpd start
-Gilson
gregm
Posts: 38
Joined: Mon Oct 21, 2002 4:19 pm

Post by gregm »

This problem is actaully a known bug between apache and MM.

Some users have it, some don't.. i had it, and downgraded, and it seems to be working..

Rather than explain it here, I'll link you to this thread in another forum:

http://forum.plesk.com/showthread.php?s=&threadid=6226
Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests