Submit Your CMD.PHP vs. SPINE Metrics Here

[otisbrown]

4/18/2008 04:45:09 PM - SYSTEM STATS: Time:7.8898 Method:spine Processes:1 Threads:10 Hosts:47 HostsPerProcess:47 DataSources:7977 RRDsProcessed:2805
04/18/2008 04:40:21 PM - SYSTEM STATS: Time:19.1519 Method:spine Processes:1 Threads:2 Hosts:47 HostsPerProcess:47 DataSources:7977 RRDsProcessed:2805

[jigarbalani]

Hello,

05/19/2008 09:30:15 PM - SYSTEM STATS: Time:293.5121 Method:cmd.php Processes:1 Threads:N/A Hosts:80 HostsPerProcess:80
DataSources:6221 RRDsProcessed:3030

I assume the number of graphs are fine, but the polling time is lot more then acceptable. I tried removing some of the graphs from graph management and it gave me the following results :

05/20/2008 12:36:43 AM - SYSTEM STATS: Time:84.0856 Method:cmd.php Processes:1 Threads:N/A Hosts:80 HostsPerProcess:80 DataSources:5259 RRDsProcessed:667
05/20/2008 12:36:06 AM - SYSTEM STATS: Time:349.7721 Method:cmd.php Processes:1 Threads:N/A Hosts:80 HostsPerProcess:80 DataSources:5259 RRDsProcessed:4170

The number of RRDs processed is also varying with no changes done from my side.

My dilemma is that, I have just added one part of the network and that is also not being fully monitored. I want to add more interfaces to cover a larger part of the network. Is there any solution?

I have also tried with cactid, but still the polling interval does not reduce. Any idea how many maximum devices/ports/graphs are supported by cacti?

Awaiting response, since I am stuck up. Thanks for the help.

Regards,
Jigar H. Balani

[gandalf]

Please upgrade to latest cacti (currently 087b). Use latest spine. Use latest rrdtool on Linux supporting fadvise. Expect a reduction of polling time of at least a magnitude. Ah, please add your servers memory footprint to your next reponse
Reinhard

[jigarbalani]

Thanks for the reponse, gandalf.

servers memory footprint means the amount of memory used by cacti?
right the cacti graph memory graph shows the following stats:
Free : 43.6M
Swap : 2.72G

All the latest versions of spine and rrdtool has been installed.

The current stats are:
05/26/2008 03:13:23 AM - SYSTEM STATS: Time:201.5580 Method:cmd.php Processes:1 Threads:N/A Hosts:80 HostsPerProcess:80 DataSources:3009 RRDsProcessed:1715
05/26/2008 03:08:15 AM - SYSTEM STATS: Time:194.4140 Method:cmd.php Processes:1 Threads:N/A Hosts:80 HostsPerProcess:80 DataSources:3009 RRDsProcessed:1732
05/26/2008 03:03:11 AM - SYSTEM STATS: Time:189.9954 Method:cmd.php Processes:1 Threads:N/A Hosts:80 HostsPerProcess:80 DataSources:3009 RRDsProcessed:1687
05/26/2008 02:58:09 AM - SYSTEM STATS: Time:188.1963 Method:cmd.php Processes:1 Threads:N/A Hosts:80 HostsPerProcess:80 DataSources:3009 RRDsProcessed:1717

But still a lot many ports remain to be added (above 10k ports).
Will it be able to scale this much. What modifications do I need to made for it?

Other problems:
Now it doesnt show the thold tab at all. Not sure what went wrong.
In the graph tree page, there is no '+' sign to minimize and maximize the tree things.

Once again, Thanks for the help and if possible please reply and guide.

-Jigar

[Exo7]

The number of RRDs processed is also varying with no changes done from my side

This happens when your polling time exceed 297 seconds. When this is the case, polling is aborted, all RRDs are not updated and you will see gaps on graphs which depends on non-updated RRDs.

] 05/26/2008 03:13:23 AM - SYSTEM STATS: Time:201.5580 Method:cmd.php Processes:1 Threads:N/A Hosts:80 HostsPerProcess:80 DataSources:3009 RRDsProcessed:1715
05/26/2008 03:08:15 AM - SYSTEM STATS: Time:194.4140 Method:cmd.php Processes:1 Threads:N/A Hosts:80 HostsPerProcess:80 DataSources:3009 RRDsProcessed:1732

You should increase number of Processes, and/or switch to cactid. How many CPU/Core your server has ?
With cactid, try 2 processes and 2 threads.
With cmd.php, try 4 processes.
Polling time should never exceed 297 seconds, and is already high if near 200 seconds (rrd updates have to happen after, can take a long time too).

[gandalf]

Thanks for the reponse, gandalf.

servers memory footprint means the amount of memory used by cacti?
right the cacti graph memory graph shows the following stats:
Free : 43.6M
Swap : 2.72G

No. I requested the physically installed amount of main memory. Swap space is of no use. Still, you're polling times are unusual high for that few data sources. Is anything else running on that server?
And yes, please answer to the question above
Reinhard

[jigarbalani]

Actually I am not quite sure what is the size of memory on that box, since that is a linux box and I am not quite good with linux. There is no GUI on it, but I will find out and get back to you.

I think there is something running on that server apart from cacti, but lets see if I get a seperate box only for cacti.

Thanks for the prompt response and I will get back with replies on ur question.

- Jigar

[gandalf]

Please invoke the "top" command. It should show memory installed/used and stuff at the top of the display. Stop "top" by pressing Ctrl-C
Reinhard

[jigarbalani]

Thanks for the response.
hope this helps

Tasks: 84 total, 3 running, 80 sleeping, 0 stopped, 1 zombie
Cpu(s): 32.1%us, 12.0%sy, 55.9%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 894976k total, 856024k used, 38952k free, 107348k buffers
Swap: 2715640k total, 62168k used, 2653472k free, 586836k cached


-Jigar

[gandalf]

Ok, seems to be about 1 Gb. That's not enough for a decent sized cacti installation, I fear. We're currently running 4GB on a dual Xeon under RHEL 4 with > 12k RRDs and a average polling time around 50 sec.
Reinhard

[jigarbalani]

Thanks for the response on this.

Although we would be having a new server for cacti, the config that you mentioned is too high!

What is the max load that cacti can take, that means how many maximum things we can monitor through cacti.

[gandalf]

There are no exact numbers. It depends on many, many items
- available memory
- cpu power
- disk speed
- network latency
- number of scripts used
You may find some numbers in this thread. Perhaps you'll find an installation that comes close to your hardware spec.
Reinhard

[jigarbalani]

ok, got it

will reinstall all the stuff and get back to you via this post.

Thank you so much for your inputs.

-Jigar

[dmojica]

hi, i have a follow problem, when i run in my linux:
(this a a little fragment)
php /var/www/html/poller.php:


07/01/2008 05:32:35 PM - SPINE: Poller[0] Host[946] DS[1062] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.9.25'
07/01/2008 05:32:35 PM - SPINE: Poller[0] Host[946] DS[1062] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.9.25'
07/01/2008 05:32:35 PM - SPINE: Poller[0] Host[946] DS[1061] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.9.25'
07/01/2008 05:32:35 PM - SPINE: Poller[0] Host[946] DS[1061] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.9.25'
Waiting on 2/20 pollers.
07/01/2008 05:32:35 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/cal046_mod_traffic_in_1061.rrd --template traffic_in:traffic_out 1214951544:U:U
OK u:0.03 s:0.07 r:42.31
07/01/2008 05:32:35 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/cal046_mod_traffic_in_1062.rrd --template traffic_in:traffic_out 1214951544:U:U
07/01/2008 05:32:35 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/gua007_mod_traffic_in_1023.rrd --template traffic_in:traffic_out 1214951543:0:9133999
07/01/2008 05:32:35 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/gua007_mod_traffic_in_1024.rrd --template traffic_in:traffic_out 1214951543:133985:230822
OK u:0.03 s:0.07 r:42.31
OK u:0.03 s:0.07 r:42.31
OK u:0.03 s:0.07 r:42.31




07/01/2008 06:08:59 PM - SPINE: Poller[0] Host[770] Hostname[10.1.11.247] ERROR: HOST EVENT: Host is DOWN Message: Host did not respond to SNMP, UDP: Ping timed out
07/01/2008 06:08:59 PM - WEATHERMAP: Poller[0] MEM Initial: memory_get_usage() says 6.6MBytes used. Limit is 256M
07/01/2008 06:08:59 PM - WEATHERMAP: Poller[0] Iterating all maps.
07/01/2008 06:08:59 PM - WEATHERMAP: Poller[0] Iterated all 0 maps.
07/01/2008 06:08:59 PM - WEATHERMAP: Poller[0] MEM Final: memory_get_usage() says 6.6MBytes used. Limit is 256M
07/01/2008 06:08:59 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
[root@localhost ~]# 07/01/2008 06:08:59 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:00 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:00 PM - SPINE: Poller[0] Host[602] DS[574] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.7.160'
07/01/2008 06:09:00 PM - SPINE: Poller[0] Host[602] DS[574] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.7.160'
07/01/2008 06:09:00 PM - SPINE: Poller[0] Host[602] DS[575] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.7.160'
07/01/2008 06:09:00 PM - SPINE: Poller[0] Host[602] DS[575] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.7.160'
07/01/2008 06:09:00 PM - SPINE: Poller[0] Host[689] DS[710] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.1.134'
07/01/2008 06:09:00 PM - SPINE: Poller[0] Host[689] DS[710] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.1.134'
07/01/2008 06:09:00 PM - SPINE: Poller[0] Host[689] DS[711] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.1.134'
07/01/2008 06:09:00 PM - SPINE: Poller[0] Host[689] DS[711] WARNING: SNMP timeout detected [1000 ms], ignoring host '10.1.1.134'
07/01/2008 06:09:01 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:01 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:01 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:01 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:01 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:01 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:01 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:01 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:02 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:05 PM - SPINE: Poller[0] ERROR: Spine Timed Out While Processing Hosts Internal
07/01/2008 06:09:22 PM - SPINE: Poller[0] WARNING: SS[3] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:09:32 PM - SPINE: Poller[0] WARNING: SS[3] The PHP Script Server did not respond in time and will therefore be restarted
çWARNING: SS[3] The PHP Script Server did not respond in time and will therefore be restarted07/01/2008 06:09:42 PM - SPINE: Poller[0] WARNING: SS[3] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:09:52 PM - SPINE: Poller[0] WARNING: SS[3] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:09:59 PM - SPINE: Poller[0] ERROR: SS[3] Script Server did not start properly return message was: 'U'
07/01/2008 06:09:59 PM - SPINE: Poller[0] ERROR: SS[3] Script Server did not start properly return message was: 'U'
07/01/2008 06:09:59 PM - SPINE: Poller[0] ERROR: SS[3] Script Server did not start properly return message was: 'U'
07/01/2008 06:09:59 PM - SPINE: Poller[0] ERROR: SS[999] Script Server did not start properly return message was: 'U'
07/01/2008 06:10:15 PM - SPINE: Poller[0] WARNING: SS[5] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:10:25 PM - SPINE: Poller[0] WARNING: SS[5] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:10:35 PM - SPINE: Poller[0] WARNING: SS[5] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:10:45 PM - SPINE: Poller[0] WARNING: SS[5] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:10:55 PM - SPINE: Poller[0] WARNING: SS[5] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:11:03 PM - SPINE: Poller[0] ERROR: SS[5] Script Server did not start properly return message was: 'U'
07/01/2008 06:11:03 PM - SPINE: Poller[0] ERROR: SS[5] Script Server did not start properly return message was: 'U'
07/01/2008 06:11:03 PM - SPINE: Poller[0] ERROR: SS[5] Script Server did not start properly return message was: 'U'
07/01/2008 06:11:03 PM - SPINE: Poller[0] ERROR: SS[5] Script Server did not start properly return message was: 'U'
07/01/2008 06:11:03 PM - SPINE: Poller[0] ERROR: SS[999] Script Server did not start properly return message was: 'U'
07/01/2008 06:11:53 PM - SPINE: Poller[0] WARNING: SS[6] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:12:04 PM - SPINE: Poller[0] WARNING: SS[6] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:12:13 PM - SPINE: Poller[0] ERROR: SS[6] Script Server did not start properly return message was: 'U'
07/01/2008 06:12:13 PM - SPINE: Poller[0] ERROR: SS[6] Script Server did not start properly return message was: 'U'
07/01/2008 06:12:13 PM - SPINE: Poller[0] ERROR: SS[6] Script Server did not start properly return message was: 'U'
07/01/2008 06:12:13 PM - SPINE: Poller[0] ERROR: SS[6] Script Server did not start properly return message was: 'U'
07/01/2008 06:12:13 PM - SPINE: Poller[0] ERROR: SS[6] Script Server did not start properly return message was: 'U'
07/01/2008 06:12:13 PM - SPINE: Poller[0] ERROR: SS[999] Script Server did not start properly return message was: 'U'
07/01/2008 06:12:23 PM - SPINE: Poller[0] WARNING: SS[7] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:12:33 PM - SPINE: Poller[0] WARNING: SS[7] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:12:43 PM - SPINE: Poller[0] WARNING: SS[7] The PHP Script Server did not respond in time and will therefore be restarted
07/01/2008 06:12:54 PM - SPINE: Poller[0] WARNING: SS[7] The PHP Script Server did not respond in time and will therefore be restarted


The problem is that I do not show spectrum in the latest graphics I've created
someone can help me please?


My configuration is:
Cacti Version 0.8.7b
Plugin Architecture 2.0
Poller Type Cactid v
Server Info Linux 2.6.9-67.0.20.plus.c4smp
Web Server Apache/2.0.59 (CentOS)
PHP Version 5.1.6
PHP Extensions libxml, xml, wddx, tokenizer, sysvshm, sysvsem, sysvmsg, standard, SimpleXML, sockets, SPL, shmop, session, Reflection, pspell, posix, mime_magic, iconv, hash, gmp, gettext, ftp, exif, date, curl, ctype, calendar, bz2, zlib, pcre, openssl, apache2handler, gd, ldap, mysql, mysqli, PDO, pdo_mysql, pdo_sqlite, snmp, eAccelerator
MySQL Version 5.0.58
RRDTool Version 1.2.23
SNMP Version 5.1.2
Plugins Global Plugin Settings (settings - v0.3)
Thresholds (thold - v0.3.9)
Large Site Performane Booster for Cacti (boost - v1.5)
Device Monitoring (monitor - v0.8.2)
Network Discovery (discovery - v0.8.4)
Network Tools (tools - v0.2)
Syslog Monitoring (syslog - v0.5.2)
Device Tracking (mactrack - v1.1)
RRD Cleaner (rrdclean - v1.1)
Update Checker (update - v0.4)
FlowView (flowview - v0.4)
Host Info (hostinfo - v0.2)
Error Images (errorimage - v0.1)
PHP Network Weathermap (weathermap - v0.941)
Create Aggregate Graphs (aggregate - v0.63)
Documents (docs - v0.1)
report it! (report it! - v0.5.0)

(sending the image of the configuration of cacti, and the graph that I am not translating this traffic is on the point ... to send you to this point ping of 800 bytes: S)

XEON 3.0Ghz
4Gb Ram
harddisk : SCSI 80 GB raid 1
[/b]

[Gamb]

07/02/2008 03:35:20 PM - SYSTEM STATS: Time:18.9624 Method:spine Processes:1 Threads:10 Hosts:70 HostsPerProcess:70 DataSources:259 RRDsProcessed:214