Submit Your CMD.PHP vs. SPINE Metrics Here

Important information about Cacti developments that all users should be interested in.

Moderators: Developers, Moderators

Post Reply
User avatar
zorrosam
Cacti User
Posts: 244
Joined: Thu May 03, 2007 3:17 pm
Location: Italy

Re: advice on my poor performance?!

Post by zorrosam »

zorrosam wrote:Hi guys,

any advice for my situation?

06/22/2009 09:49:30 AM - SYSTEM STATS: Time:269.0361 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7065

DELL Poweredge 1950
2xQUADCORE 2.00 GHZ XEON
8GB RAM
2xHDD 250GB RAID1
MYSQL ACTIVE ON ANOTHER SERVER TUNED

I don't know if will be better activate boost on my side ... the system is on production and fundamental for all my bussiness. There's the way for do that without downtime?

bye

sam
the version of cacti is 0.87b!
Top
User avatar
zorrosam
Cacti User
Posts: 244
Joined: Thu May 03, 2007 3:17 pm
Location: Italy

Re: advice on my poor performance?!

Post by zorrosam »

zorrosam wrote:
zorrosam wrote:Hi guys,

any advice for my situation?

06/22/2009 09:49:30 AM - SYSTEM STATS: Time:269.0361 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7065

DELL Poweredge 1950
2xQUADCORE 2.00 GHZ XEON
8GB RAM
2xHDD 250GB RAID1
MYSQL ACTIVE ON ANOTHER SERVER TUNED

I don't know if will be better activate boost on my side ... the system is on production and fundamental for all my bussiness. There's the way for do that without downtime?

bye

sam
the version of cacti is 0.87b!

06/22/2009 10:04:31 AM - SYSTEM STATS: Time:269.2874 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7055
06/22/2009 10:04:28 AM - RECACHE STATS: RecacheTime:299.7883 HostsRecached:66
06/22/2009 09:59:31 AM - RECACHE STATS: RecacheTime:304.3112 HostsRecached:71
06/22/2009 09:59:28 AM - SYSTEM STATS: Time:266.6671 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:54:27 AM - SYSTEM STATS: Time:266.1210 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:49:30 AM - SYSTEM STATS: Time:269.0361 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7065
06/22/2009 09:49:22 AM - RECACHE STATS: RecacheTime:297.9598 HostsRecached:70
06/22/2009 09:44:24 AM - SYSTEM STATS: Time:262.2032 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:39:25 AM - SYSTEM STATS: Time:263.7265 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:39:20 AM - RECACHE STATS: RecacheTime:297.6952 HostsRecached:71
06/22/2009 09:34:22 AM - SYSTEM STATS: Time:261.4651 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:29:22 AM - SYSTEM STATS: Time:261.6119 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:29:22 AM - RECACHE STATS: RecacheTime:297.0967 HostsRecached:71
06/22/2009 09:24:24 AM - SYSTEM STATS: Time:262.7349 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:19:26 AM - SYSTEM STATS: Time:264.5988 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:19:24 AM - RECACHE STATS: RecacheTime:297.9909 HostsRecached:71
06/22/2009 09:14:26 AM - SYSTEM STATS: Time:264.5102 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:09:20 AM - SYSTEM STATS: Time:259.1723 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:09:15 AM - RECACHE STATS: RecacheTime:292.1965 HostsRecached:74
06/22/2009 09:04:22 AM - SYSTEM STATS: Time:260.7924 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7055
06/22/2009 08:59:23 AM - SYSTEM STATS: Time:261.6565 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7055
06/22/2009 08:59:13 AM - RECACHE STATS: RecacheTime:291.8565 HostsRecached:73
06/22/2009 08:54:22 AM - RECACHE STATS: RecacheTime:300.2356 HostsRecached:71
06/22/2009 08:54:21 AM - SYSTEM STATS: Time:259.7655 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 08:49:22 AM - SYSTEM STATS: Time:260.2591 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 08:44:21 AM - SYSTEM STATS: Time:260.2246 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 08:44:20 AM - RECACHE STATS: RecacheTime:299.8413 HostsRecached:74

The IOS is RedHat 5.4 Arch 64 BIT
Top
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Re: advice on my poor performance?!

Post by TheWitness »

zorrosam wrote:
zorrosam wrote:
zorrosam wrote:Hi guys,

any advice for my situation?

06/22/2009 09:49:30 AM - SYSTEM STATS: Time:269.0361 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7065

DELL Poweredge 1950
2xQUADCORE 2.00 GHZ XEON
8GB RAM
2xHDD 250GB RAID1
MYSQL ACTIVE ON ANOTHER SERVER TUNED

I don't know if will be better activate boost on my side ... the system is on production and fundamental for all my bussiness. There's the way for do that without downtime?

bye

sam
the version of cacti is 0.87b!

06/22/2009 10:04:31 AM - SYSTEM STATS: Time:269.2874 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7055
06/22/2009 10:04:28 AM - RECACHE STATS: RecacheTime:299.7883 HostsRecached:66
06/22/2009 09:59:31 AM - RECACHE STATS: RecacheTime:304.3112 HostsRecached:71
06/22/2009 09:59:28 AM - SYSTEM STATS: Time:266.6671 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:54:27 AM - SYSTEM STATS: Time:266.1210 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:49:30 AM - SYSTEM STATS: Time:269.0361 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7065
06/22/2009 09:49:22 AM - RECACHE STATS: RecacheTime:297.9598 HostsRecached:70
06/22/2009 09:44:24 AM - SYSTEM STATS: Time:262.2032 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:39:25 AM - SYSTEM STATS: Time:263.7265 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:39:20 AM - RECACHE STATS: RecacheTime:297.6952 HostsRecached:71
06/22/2009 09:34:22 AM - SYSTEM STATS: Time:261.4651 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:29:22 AM - SYSTEM STATS: Time:261.6119 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:29:22 AM - RECACHE STATS: RecacheTime:297.0967 HostsRecached:71
06/22/2009 09:24:24 AM - SYSTEM STATS: Time:262.7349 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:19:26 AM - SYSTEM STATS: Time:264.5988 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:19:24 AM - RECACHE STATS: RecacheTime:297.9909 HostsRecached:71
06/22/2009 09:14:26 AM - SYSTEM STATS: Time:264.5102 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:09:20 AM - SYSTEM STATS: Time:259.1723 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 09:09:15 AM - RECACHE STATS: RecacheTime:292.1965 HostsRecached:74
06/22/2009 09:04:22 AM - SYSTEM STATS: Time:260.7924 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7055
06/22/2009 08:59:23 AM - SYSTEM STATS: Time:261.6565 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7055
06/22/2009 08:59:13 AM - RECACHE STATS: RecacheTime:291.8565 HostsRecached:73
06/22/2009 08:54:22 AM - RECACHE STATS: RecacheTime:300.2356 HostsRecached:71
06/22/2009 08:54:21 AM - SYSTEM STATS: Time:259.7655 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 08:49:22 AM - SYSTEM STATS: Time:260.2591 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 08:44:21 AM - SYSTEM STATS: Time:260.2246 Method:spine Processes:6 Threads:15 Hosts:179 HostsPerProcess:30 DataSources:14340 RRDsProcessed:7083
06/22/2009 08:44:20 AM - RECACHE STATS: RecacheTime:299.8413 HostsRecached:74

The IOS is RedHat 5.4 Arch 64 BIT
This should be a separate thread. The big thing to understand is this, how long does it take spine to complete. If you issue a watch command for spine processes, and time them. You should be able to determine if the problem is spine (some script or snmp call haning on too long), or RRDtool updates.

Another thing to note is that if the following command (+ 1 GByte) is more than your system ram, you issue is likely the RRDupdate process, of which Boost can help with.

du -s /var/www/html/cacti/rra

Boost can be deployed with no downtime, but you need to know what you are doing. Having a test system to "play" with is essential.

Regards,

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
jbarbieri
Cacti User
Posts: 101
Joined: Tue May 24, 2005 8:46 am
Location: Rochester, New Hampshire
Contact:
Contact jbarbieri

Post by jbarbieri »

Fully migrated to new server:

Code: Select all

07/20/2009 03:30:11 PM - SYSTEM STATS: Time:9.6051 Method:spine Processes:15 Threads:25 Hosts:53 HostsPerProcess:4 DataSources:10547 RRDsProcessed:4406
07/20/2009 03:35:10 PM - SYSTEM STATS: Time:9.0917 Method:spine Processes:15 Threads:25 Hosts:53 HostsPerProcess:4 DataSources:10547 RRDsProcessed:4406
Not writing to an NFS helps alot!

Quad Xeon 3.4Ghz with 2GB RAM.
Cacti 0.8.7e and Spine 0.8.7d


Attached is a picture of my poller stats....nice dropoff! Those "blank" spots are when the process took longer then 5 minutes.
Attachments
Poller Stats
Poller Stats
poller.png (24.59 KiB) Viewed 22051 times
Top
frdrk
Cacti User
Posts: 55
Joined: Fri Jul 06, 2007 2:27 pm

Post by frdrk »

jbarbieri wrote: Attached is a picture of my poller stats....nice dropoff! Those "blank" spots are when the process took longer then 5 minutes.
that's a nice fixed font, whats it called?
Top
jbarbieri
Cacti User
Posts: 101
Joined: Tue May 24, 2005 8:46 am
Location: Rochester, New Hampshire
Contact:
Contact jbarbieri

Post by jbarbieri »

To be honest, I don't know. I am running RRDTool 1.3.7, but my paths for fonts are all blank.
Top
kmhendress
Posts: 33
Joined: Wed Aug 27, 2008 2:49 pm

Post by kmhendress »

I'm very happy with the stats!
Cacti Version - 0.8.7e
Plugin Architecture - 2.4
Poller Type - Cactid v
Server Info - Linux 2.6.9-67.0.1.ELsmp
Web Server - Apache/2.0.52 (Red Hat)
PHP - 5.2.9
PHP Extensions - libxml, xml, tokenizer, session, pcre, SPL, PDO, SQLite, sockets, standard, Reflection, posix, pdo_sqlite, mysql, ldap, json, iconv, hash, gd, filter, date, ctype, zlib, apache2handler
MySQL - 4.1.20
RRDTool - 1.2.23
SNMP - 5.1.2
Plugins
  • Thresholds (thold - v0.4.1)
    Global Plugin Settings (settings - v0.5)
    Host Info (hostinfo - v0.2)
    Update Checker (update - v0.4)
    Device Monitoring (monitor - v0.8.2)
    PHP Network Weathermap (weathermap - v0.96a)

07/22/2009 03:05:12 PM - SYSTEM STATS: Time:298.4995 Method:cmd.php Processes:30 Threads:N/A Hosts:18 HostsPerProcess:1 DataSources:5731 RRDsProcessed:2238

07/28/2009 09:20:57 AM - SYSTEM STATS: Time:54.9774 Method:spine Processes:20 Threads:5 Hosts:18 HostsPerProcess:1 DataSources:6184 RRDsProcessed:3439
Top
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:
Contact gandalf

Post by gandalf »

kmhendress wrote: 07/22/2009 03:05:12 PM - SYSTEM STATS: Time:298.4995 Method:cmd.php Processes:30 Threads:N/A Hosts:18 HostsPerProcess:1 DataSources:5731 RRDsProcessed:2238

07/28/2009 09:20:57 AM - SYSTEM STATS: Time:54.9774 Method:spine Processes:20 Threads:5 Hosts:18 HostsPerProcess:1 DataSources:6184 RRDsProcessed:3439
I'd do using 5 processes and 20 threads unless you have a Quad-Quadcore or the like
Reinhard
Top
kmhendress
Posts: 33
Joined: Wed Aug 27, 2008 2:49 pm

Post by kmhendress »

Thanks, I have made the reccomended change. It is a VM box, with two CPU's tied to it.

Here are the new stats, even much better!

07/30/2009 03:20:38 PM - SYSTEM STATS: Time:35.9677 Method:spine Processes:5 Threads:20 Hosts:18 HostsPerProcess:4 DataSources:6186 RRDsProcessed:1474
Top
kmhendress
Posts: 33
Joined: Wed Aug 27, 2008 2:49 pm

Post by kmhendress »

kmhendress wrote:Thanks, I have made the reccomended change. It is a VM box, with two CPU's tied to it.

Here are the new stats, even much better!

07/30/2009 03:20:38 PM - SYSTEM STATS: Time:35.9677 Method:spine Processes:5 Threads:20 Hosts:18 HostsPerProcess:4 DataSources:6186 RRDsProcessed:1474
Went back to about ~60 seconds
07/30/2009 03:40:58 PM - SYSTEM STATS: Time:56.8005 Method:spine Processes:5 Threads:20 Hosts:18 HostsPerProcess:4 DataSources:6186 RRDsProcessed:3478
Top
User avatar
Howie
Cacti Guru User
Posts: 5508
Joined: Thu Sep 16, 2004 5:53 am
Location: United Kingdom
Contact:
Contact Howie

Post by Howie »

Not spine vs cmd, but spine vs cactid :-)

I just upgraded a customer's Cacti install from 0.8.7a+cactid to 0.8.7e+spine and the polling time nearly halved!

07/31/2009 03:13:26 PM - SYSTEM STATS: Time:7.4273 Method:spine Processes:1 Threads:10 Hosts:10 HostsPerProcess:10 DataSources:958 RRDsProcessed:390

Nice work!
Attachments
graph_image.php.png
graph_image.php.png (28.15 KiB) Viewed 21878 times
Weathermap 0.98a is out! & QuickTree 1.0. Superlinks is over there now (and built-in to Cacti 1.x).
Some Other Cacti tweaks, including strip-graphs, icons and snmp/netflow stuff.
(Let me know if you have UK DevOps or Network Ops opportunities, too!)
Top
kmhendress
Posts: 33
Joined: Wed Aug 27, 2008 2:49 pm

Post by kmhendress »

Howie wrote:Not spine vs cmd, but spine vs cactid :-)

I just upgraded a customer's Cacti install from 0.8.7a+cactid to 0.8.7e+spine and the polling time nearly halved!

07/31/2009 03:13:26 PM - SYSTEM STATS: Time:7.4273 Method:spine Processes:1 Threads:10 Hosts:10 HostsPerProcess:10 DataSources:958 RRDsProcessed:390

Nice work!
I like your graph, what template did you use to get that working? I'd love to have it :)
Top
User avatar
Howie
Cacti Guru User
Posts: 5508
Joined: Thu Sep 16, 2004 5:53 am
Location: United Kingdom
Contact:
Contact Howie

Post by Howie »

kmhendress wrote:
I like your graph, what template did you use to get that working? I'd love to have it :)
It's Gandalf's Poller Statistics template. Very handy for figuring out whether you are about to hit a wall, and what the impact is of any tuning.
Weathermap 0.98a is out! & QuickTree 1.0. Superlinks is over there now (and built-in to Cacti 1.x).
Some Other Cacti tweaks, including strip-graphs, icons and snmp/netflow stuff.
(Let me know if you have UK DevOps or Network Ops opportunities, too!)
Top
kmhendress
Posts: 33
Joined: Wed Aug 27, 2008 2:49 pm

Post by kmhendress »

Awesome - Thanks!
Top
ikorzha758
Cacti User
Posts: 164
Joined: Thu Jan 21, 2010 8:41 pm

Post by ikorzha758 »

Here are my results so far:
01/28/2010 01:17:29 PM - SYSTEM STATS: Time:28.0138 Method:spine Processes:6 Threads:4 Hosts:60 HostsPerProcess:10 DataSources:16713 RRDsProcessed:8516
01/28/2010 01:16:29 PM - SYSTEM STATS: Time:28.0694 Method:spine Processes:6 Threads:4 Hosts:60 HostsPerProcess:10 DataSources:16713 RRDsProcessed:8516
01/28/2010 01:15:32 PM - SYSTEM STATS: Time:29.8361 Method:spine Processes:6 Threads:4 Hosts:60 HostsPerProcess:10 DataSources:16713 RRDsProcessed:8516

1 Minute polling.
Cacti 0.8.7e and Spine 0.8.7e
2 Xeons 2.0Ghz each 6 Gigs of RAM Linux 5.2 Enterprise.
Top
Post Reply

Return to “Announcements”

Who is online

Users browsing this forum: No registered users and 0 guests