The I/O of my cacti server is so high

[donglee]

The load of my cacti server is so high, I don't know how to put photo in the post, so I just write values in there
this is I/0 value:
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 2269.00 0.00 132.00 0.00 6184.00 46.85 137.34 908.96 7.58 100.10
dm-0 0.00 0.00 0.00 2440.00 0.00 19520.00 8.00 2074.98 254.64 0.41 99.80
dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

I have 500 linux server and 25 switches in total, for a accurate value, I run rrdtool every one minute, this is my cacti log:

12/18/2013 01:37:13 PM - SYSTEM STATS: Time:10.9693 Method:spine Processes:12 Threads:50 Hosts:502 HostsPerProcess:42 DataSources:25237 RRDsProcessed:0

Now, the I/0 is always so high, I even cannot log in my server. And although we have 500 servers, but I only get one virutal machine to run cacti. Other parameters, like cpu and mem, are just ok. Can someone help me improve the performance of my cacti server? Thank you very much.

[npsiva]

I have this same issue, as I have around that same amount of devices in my network being monitored.

What I did was to poll every 5 minutes. When the poll starts, it starts snmpwalk(I'm using Windows) for the 500 devices, and during the poll that many processes being started, cause the processor to go to 100%, and makes me desperate when trying to use the server for other purposes.

So, I'm polling every 5 minutes (using spine)
Balance Process Load ticked
MAx Threads per process - 5
# PHP Script Servers - 10
Max SNMP OID's per SNMP Get Request 100

If anyone knows of a way to improve this, it would be very appreciated.

Thanks,
Nuno