memory load is VERY high

Post support questions that directly relate to Linux/Unix operating systems.

Moderators: Developers, Moderators

Post Reply
MPI
Posts: 29
Joined: Thu Jan 11, 2007 2:36 pm

Post by MPI »

TheWitness wrote:Other than the fact that my mind is in the machine, I don't have a clue. It's somewhat an act of clairvoyance or just plain luck...

TheWiitness
can you answer my last question about mysql?

p.s. i noticed that ssl has gone down too....the certificate is still valid but ssl is sending timeouts.
Top
User avatar
TheWitness
Developer
Posts: 17064
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

Not really, it makes not sense to me that MySQL was causing excessive connections to your localhost's Apache server. You will have to research on your own.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
MPI
Posts: 29
Joined: Thu Jan 11, 2007 2:36 pm

Post by MPI »

TheWitness wrote:Not really, it makes not sense to me that MySQL was causing excessive connections to your localhost's Apache server. You will have to research on your own.

TheWitness
oh i see whats going on, i think you misunderstood...mysql is NOT causing excessive connections....its a very busy website so 1,500 was the amount of clients that connected at the time.....

so the problem is either:
1. too many apache connections are staying alive and not dying....or conntrack_timeout is too high...but i've never touched the conntrack configs on this server.
2. the process are taking TOO much memory which is not at all usual.

i've searched google, posted the same issue on 3 or 4 other forums including cacti's...but nothing....so any advice you give me would be very helpful.
Top
User avatar
TheWitness
Developer
Posts: 17064
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

So, as I suspected all along. Nothing to do with Cacti. I am glad you have discovered your problem.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
MPI
Posts: 29
Joined: Thu Jan 11, 2007 2:36 pm

Post by MPI »

TheWitness wrote:So, as I suspected all along. Nothing to do with Cacti. I am glad you have discovered your problem.

TheWitness
actually i think that it has EVERYTHING to do with cacti....

my theory is that someone got in via the cacti exploit and changed all these values...now i can't even find alot of the conntrack files!
Top
MPI
Posts: 29
Joined: Thu Jan 11, 2007 2:36 pm

wow

Post by MPI »

i just caught this....WOW

Code: Select all

top - 01:51:03 up 3 days,  5:41,  1 user,  load average: 55.29, 37.00, 16.49
Tasks: 2543 total,  19 running, 2524 sleeping,   0 stopped,   0 zombie
Cpu(s):  5.4% us, 94.3% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.2% hi,  0.0% si
Mem:   8312692k total,  8298652k used,    14040k free,     8828k buffers
Swap:  8421368k total,  1499324k used,  6922044k free,   745904k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                
20487 nobody    16   0 22156 8784 2900 S   28  0.1   0:00.57 httpd                                                                  
 5276 nobody    16   0 28256  12m 3212 R   24  0.2   0:04.81 httpd                                                                  
21787 nobody    16   0 28992  10m 2824 R   22  0.1   0:01.34 httpd                                                                  
 4955 nobody    15   0 28280  11m 2848 S   22  0.1   0:04.70 httpd                                                                  
18904 nobody    17   0 22844 8828 2836 R   21  0.1   0:00.90 httpd                                                                  
 4779 nobody    16   0 22276 8744 2864 S   19  0.1   0:02.64 httpd                                                                  
   74 root      16   0     0    0    0 R   18  0.0  60:36.88 kswapd0                                                                
26137 nobody    16   0 28088  13m 3212 S   18  0.2   0:04.31 httpd                                                                  
 4837 nobody    16   0 28088  11m 3300 R   17  0.1   0:11.02 httpd                                                                  
20599 nobody    16   0 27968  13m 2824 S   17  0.2   0:02.36 httpd                                                                  
18159 nobody    15   0 28120  14m 3336 S   15  0.2   0:02.97 httpd                                                                  
22031 nobody    18   0 22068 8800 2904 R   13  0.1   0:00.33 httpd                                                                  
22544 root      20   0  4780 2376  760 R   13  0.0   0:01.25 top                                                                    
 4560 nobody    16   0 22236 8840 2892 S   10  0.1   0:01.50 httpd                                                                  
20635 nobody    16   0 28088  11m 2844 R   10  0.1   0:01.92 httpd                                                                  
20934 nobody    15   0 27968  12m 2776 S    7  0.1   0:00.26 httpd                                                                  
20933 nobody    17   0 22328 8788 2816 R    6  0.1   0:01.26 httpd                                                                  
 4996 nobody    15   0 28120  13m 3332 S    6  0.2   0:03.19 httpd                                                                  
 2657 mysql     15   0  134m  22m 2156 S    4  0.3  91:41.41 mysqld                                                                 
 6313 nobody    15   0 28088  12m 3216 S    4  0.2   0:03.51 httpd                                                                  
20445 nobody    15   0 27968  11m 2816 S    3  0.1   0:01.90 httpd                                                                  
10561 nobody    16   0 22964 9572 3304 S    1  0.1   0:02.38 httpd                                                                  
21004 nobody    15   0 22844 8936 2820 S    1  0.1   0:00.04 httpd                                                                  
 2839 root      15   0     0    0    0 S    1  0.0   0:19.49 kjournald                                                              
 4734 root      16   0  232m 2492 1472 S    1  0.0   3:38.10 dsm_sa_datamgr3                                                        
 4867 nobody    16   0 28560  13m 2872 S    1  0.2   0:02.31 httpd                                                                  
 5245 nobody    15   0 22220 9176 3244 S    1  0.1   0:02.95 httpd                                                                  
 5920 nobody    15   0 22220 9232 3236 S    1  0.1   0:02.52 httpd                                                                  
 7384 nobody    15   0 22448 9296 3212 R    1  0.1   0:02.44 httpd                                                                  
 7850 nobody    15   0 28088  13m 2844 S    1  0.2   0:02.47 httpd                                                                  
 7899 nobody    15   0 28088  11m 2848 S    1  0.1   0:04.50 httpd                                                                  
12654 nobody    16   0 22996 9528 3236 S    1  0.1   0:02.60 httpd                                                                  
14875 nobody    16   0 22276 9248 3240 S    1  0.1   0:02.79 httpd                                                                  
19701 nobody    15   0 28088  12m 2848 S    1  0.2   0:05.18 httpd                                                                  
20702 nobody    16   0 22964 9432 3308 S    1  0.1   0:04.01 httpd                                                                  
20742 nobody    15   0 22188 8776 2844 S    1  0.1   0:02.86 httpd                                                                  
30513 nobody    16   0 22308 9260 3236 S    1  0.1   0:01.94 httpd                                                                  
12111 nobody    15   0 22188 8864 2840 S    1  0.1   0:01.08 httpd                                                                  
24346 nobody    16   0 23028 9448 3216 S    1  0.1   0:02.29 httpd


i couldn't barely ssh into the machine...looks like i caught right before it was about to crash...i just immediately stopped apache.

any ideas?
Top
User avatar
TheWitness
Developer
Posts: 17064
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

Have you ever heard of a DOS attack over HTTP? Could also be a SYN attack,etc.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
MPI
Posts: 29
Joined: Thu Jan 11, 2007 2:36 pm

Post by MPI »

TheWitness wrote:Have you ever heard of a DOS attack over HTTP? Could also be a SYN attack,etc.

TheWitness
well, i have SYN cookies enabled...and far as a ddos attack goes, i know what its purpose it but thats about all i know....i don't know much about how to detect or stop it.

but wait, a a ddos attack wouldn't stop after i restart apache, would it?
Top
User avatar
TheWitness
Developer
Posts: 17064
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

You should install Wireshark and perform a sniffer capture and attach it here or via a PM.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
MPI
Posts: 29
Joined: Thu Jan 11, 2007 2:36 pm

hmm

Post by MPI »

TheWitness wrote:You should install Wireshark and perform a sniffer capture and attach it here or via a PM.

TheWitness
tried, couldn't make it "make".

here are the last few lined before it exit

Code: Select all

rith -W  -g -O2 -I/usr/local/include -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include    -MT version_info.o -MD -MP -MF ".deps/version_info.Tpo" -c -o version_info.o version_info.c; \
then mv -f ".deps/version_info.Tpo" ".deps/version_info.Po"; else rm -f ".deps/version_info.Tpo"; exit 1; fi
In file included from version_info.c:50:
/usr/include/ucd-snmp/version.h:9:2: #error "Please update your headers or configure using --enable-ucd-snmp-compatibility"
version_info.c: In function `get_epan_compiled_version_info':
version_info.c:206: error: `VersionInfo' undeclared (first use in this function)
version_info.c:206: error: (Each undeclared identifier is reported only once
version_info.c:206: error: for each function it appears in.)
make[2]: *** [version_info.o] Error 1
make[2]: Leaving directory `/home/wireshark-0.99.4'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/wireshark-0.99.4'
make: *** [all] Error 2
Top
MPI
Posts: 29
Joined: Thu Jan 11, 2007 2:36 pm

Post by MPI »

looked all over google and a few boards..nothing.
Top
Post Reply

Return to “Help: Linux/Unix Specific”

Who is online

Users browsing this forum: No registered users and 3 guests