Process\render weathmap configs in parallel?

[deevee]

Hi,

Is it possible to have Weathemap process the weathermap config files in parallel rather than sequentially, as part of the poller process in Cacti? I have achieved this via the cli by manually processing each config seperately at the same time, but I ran into issues with the cactihost target (can't access Cacti database when running weathermap from the cli - WMWARN07).

The reason for this is that it currently takes 8 seconds to process one weathermap I have created. I am planning on creating a few more, and can see myself hitting the limits pretty quickly on how many I can create.

Any advice/help would be much appreciated.

Thanks,

deevee

[Howie]

You can use weathermap-cacti-rebuild.php to process your maps outside of the poller, but with access to the cacti environment. It does them sequentially, however.

You might also like to try weathermap from svn, which is about twice the speed on average. If you do decide to do that, please get revision 514 (currently the latest) and not later, as I am going to be making changes that may break things temporarily (it's the development version, after all).

If you have particular slow maps and you are able to share them with me, I'd be interested to see if there are other optimisations, too.

To answer your actual question though - no, the map processing is all done in the poller process, in sequence. It probably wouldn't be too hard to change that, but it would (at least) make for strange-looking logs, and would still need to run after the actual data collection has happened, otherwise you'll be showing old data.

[deevee]

Thanks for the quick response

I tried weathermap-cacti-rebuild.php, and as you mentioned it runs sequentially.

Do you have any plans on making weathermap multithreaded like the spine poller, so when limits are met, extra capacity can be added via CPU cores and/or memory. The weathermaps could be evenly distributed amongst the threads, and the logs for each weathermap could have a thread id as to make them less confusing... this is just an idea of course and may not actually make any technical sense from a developers perspective

When I performed some tests from the CLI though, I ran the command for two identical weathermaps at the same time and it took roughly 8 seconds, which is the time it takes to process one through cacti.

In regards to the slow map, I have roughly 20-30 devices showing their status with cactihost, and 40-50 links showing utilisation. Nothing too fancy. Is 8 seconds too long for a weathermap this size?

Thanks,
deevee

[Howie]

When I performed some tests from the CLI though, I ran the command for two identical weathermaps at the same time and it took roughly 8 seconds, which is the time it takes to process one through cacti.

Really? That's interesting... I would have expected it to just run slower. I don't think I've ever tested. I'll take a look at this some more at some point. I guess that it spends enough time waiting for i/o that the other map can run in that time.

In regards to the slow map, I have roughly 20-30 devices showing their status with cactihost, and 40-50 links showing utilisation. Nothing too fancy. Is 8 seconds too long for a weathermap this size?

I'll take a look at my own installation when I get to work. That doesn't sound too crazy. Are you using poller_output or dsstats or just straight rrd files for the utilisation? Do you have a lot of curved links? Is this with one-minute polling?

Anyway, if you don't mind PMing it to me, it'd be useful to run the php profiler with your map, and see where all the time is spent - this is what I did after 0.97a came out with my own maps, and the result is roughly doubling the speed for 0.98dev. I don't share user's data.

[ikorzha758]

Howie I would like to run PHP profiler on my huge map and provide you with results too if you can specify command syntax.

[Howie]

I'd far rather have a copy of the map config if possible - setting up xdebug wasn't trivial, from my memory of it.

I'll find a reference for how to do it though, tomorrow (not at home right now).

[Howie]

I tried weathermap-cacti-rebuild.php, and as you mentioned it runs sequentially.

I'm just adding some options to weathermap-cacti-rebuild.php so that it will take a map-id on the command-line. With that, it should be possible to experiment better with parallel map processing and still have the "cacti knowledge" available.

[deevee]

Anyway, if you don't mind PMing it to me, it'd be useful to run the php profiler with your map, and see where all the time is spent - this is what I did after 0.97a came out with my own maps, and the result is roughly doubling the speed for 0.98dev. I don't share user's data.

I have PM'd the weathermap to you. I am running 1 minute polling, straight rrd files and have a few curved links.

I'm just adding some options to weathermap-cacti-rebuild.php so that it will take a map-id on the command-line. With that, it should be possible to experiment better with parallel map processing and still have the "cacti knowledge" available.

Sounds promising. How have your tests gone so far?

[deevee]

Hi Howie,

Have you made any progress so far?

thanks,

deevee

[Howie]

No, I haven't had any time yet. I don't have any time during this week, but may do at the weekend.