Boost Memory

[lmouly]

Hi

I am currently setting up a new cacti server, with boost and I am a bit curious about boost memory usage.

here are my last "Runtime Statistics"
Runtime Statistics
Last Start Time: 2010-09-16 11:24:06
Last Run Duration: 23 seconds (0% of update frequency)
RRD Updates: 599712
Peak Poller Memory: 573.96 MBytes
Detailed Runtime Timers: RRDUpdates:599712 TotalTime:23.7857 range_local_data_id:0.42 rcaston_add:1.86 get_records:6.46 results_cycle:11.97 rrd_path:0.26 rrd_template:0.61 rrd_lastupdate:0.08 rrd_field_names:4.09 rrdupdate:0.63 delete:2.39
Max Poller Memory Allowed: 1024 MBytes


Is there something wrong with it ?
I am asking this because of the graph examples here : http://docs.cacti.net/usertemplate:host:cacti:poller
700k RRDupdates take only 40M of memory.

Please share your stats.

[lmouly]

I'm using a memory engine for the boost tables.
Boost Storage Statistics
Database Engine: MEMORY
Current Boost Table Size: 346 MBytes
Avg Bytes/Record: 96 Bytes
Max Record Length: 52 Bytes
Max Allowed Boost Table Size: 452 MBytes
Estimated Maximum Records: 4934475 Records

Does it have an impact on the memory usage ?

I just had access to an other Cacti server :

Boost Storage Statistics
Database Engine: InnoDB
Current Boost Table Size: 53 MBytes
Avg Bytes/Record: 109 Bytes
Max Allowed Boost Table Size: Unlimited
Estimated Maximum Records: Unlimited Records
Runtime Statistics
Last Start Time: 2010-09-21 15:02:29
Last Run Duration: 35 minutes 48 seconds (60% of update frequency)
RRD Updates: 484646
Peak Poller Memory: 12.08 MBytes
Detailed Runtime Timers: RRDUpdates:484646 TotalTime:2148.3762 range_local_data_id:13.15 rcaston_add:2.68 get_records:7.8 results_cycle:2098.05 rrd_path:27.22 rrd_template:78.58 rrd_lastupdate:413.15 rrd_field_names:5.49 rrdupdate:1563.47 delete:10.07
Max Poller Memory Allowed: 2048 MBytes


It does take much longer to write the data to the RRD because of the innoDB engine, but look at the memory used, only 12 MB.

[TheWitness]

Hmm, very interesting. Well if your server has lot's of memory, I would not worry about it and simply use memory.

TheWitness

[lmouly]

I have 8GB of RAM, but for the moment I only have 3587 datasources on it.
When I will migrate all the devices from my old cacti I will have much more.

When I look at some statistics from here : http://forums.cacti.net/about6206-0-asc-300.html
I am curious how people deal with 10 million update. For my system it should take around 10GB of RAM !

[lmouly]

I just found out why it is using so much memory.
As I don't have a lot of datasources, it does take a while to reach the maximum record.
So for each datasources, I have a lot of data to write to the rrd. As I left the default setting for the "Maximum Data Sources Per Pass" (2000) the first pass takes half of my datasources into memory to write them into the rrds.

I reduced the "Maximum Data Sources Per Pass", I will see tommorow how much memory has been used by boost during the night.

[lmouly]

That was it indeed.
I reduced to 200 datasources per pass and I am now using 250MB max