performance issues using cactid with external scripts

smilerpt · Post by **smilerpt** » Tue Apr 24, 2007 8:21 pm

Hi,

I have a unique host with more than 400 interfaces, and I am using a external script to gather class statistics. Right now I am only querying about 40 interfaces, and I have around ~360 data sources.

Because in the future I will need to query more interfaces and maybe in equipments that have even more interfaces, I switched from cmd.php to cactid and unfortunately I realized that the times that I am getting are very much similar, cactid improved in less than 5% the timings I am getting with cmd.php.

I did some research on the way it was querying the router and I realized that no matter what I configure as number of processes and threads, the external scripts (perl) will run sequentially.

Imagine each query through the perl script takes 0.12 seconds, so for 360 data sources, sequentially I would have 0.12 x 360 = ~43 seconds.

Well what I thought (until know wrongly...) was that using cactid I would have concurrent queries and therefore I would reduce the timings I am getting / will get in the future with more datasources in the same host.

Theorically with 2 threads I should have 0.12 x 360 / 2 right ?

No matter what I configure, 2 process x 5 threads, 1 process x 20 threads, 4 process x 4 threads, ..., cactid will invoke the external scripts sequentially and never simultaneously.

Any ideas or comments on this ? Shouldnt cactid fork the processes and divide the \work\ - datasources ?

Thanks

smilerpt

smilerpt · Post by **smilerpt** » Wed Apr 25, 2007 2:24 pm

I did some more analisys on this and found that while poller was running, I had loads of perl defuncts processes.

I thought that maybe because of this defuncts, I didnt have concurrency in the external scripts, so according to what was said in the discussion http://forums.cacti.net/viewtopic.php?t ... ht=WNOHANG I changed my nft_popen.c file and recompiled cactid

[root@MBOX cacti-cactid-0.8.6i]# diff nft_popen_backup.c nft_popen.c
297d296
< kill(cur->pid, SIGTERM);
301c300
< do { pid = waitpid(cur->pid, &pstat, WNOHANG); }
---
> do { pid = waitpid(cur->pid, &pstat, 0); }

Well, in fact zombies (defunct processes) are gone during the polling though performance havent increased, external scripts are still beeing invoked one by one.

can anyone comment on this ?

rs

smilerpt · Post by **smilerpt** » Fri Apr 27, 2007 2:03 pm

No comments at all ?!

Post by **TheWitness** » Fri Apr 27, 2007 9:33 pm

Well, you have to understand that I have a day job too. So, sometime's I don't answer these questions right away.

In fact, in SVN (BRANCH_0_8_6) I introduced a change to allow scripts to run rampant, and I have not fixed your little nft_popen issue. But the problem that you may have is that on a per host basis, Cactid will process scripts sequentially.

Therefore, to speed those types of things up, just add more hosts and the script activity will increase significantly.

TheWitness

smilerpt · Post by **smilerpt** » Sat Apr 28, 2007 5:43 am

Hello TheWitness,

Thanks for your reply !

But adding more hosts will not solve my problem...in fact it will increase script activity but what I am afraid of is to not beeing able to poll in 300 seconds cause of this issue. If my script takes 0.2 seconds per query, and if per host basis cactid processes sequentially, then I can

smilerpt · Post by **smilerpt** » Sat Apr 28, 2007 5:44 am

Hello TheWitness,

Thanks for your reply !

But adding more hosts will not solve my problem...in fact it will increase script activity but what I am afraid of is to not beeing able to poll in 300 seconds cause of this issue. If my script takes 0.2 seconds per query, and if per host basis cactid processes sequentially, then I can not have more than 1500 ds per host basis, otherwise I will be over the 300 seconds. 1500 ds is easily reachable with QoS graphs, because each MultiInterface can have in average 4 classes of service, so it will be 8 ds = 4 x class drops + 4 x class traffic + 1.

Thanks once more for your time.

rs

Post by **TheWitness** » Sat Apr 28, 2007 6:27 am

Have you considered the script server for PHP. It will execute your scripts faster (I think).

TheWitness

smilerpt · Post by **smilerpt** » Sat Apr 28, 2007 7:42 am

No I hadn

smilerpt · Post by **smilerpt** » Sat Apr 28, 2007 7:42 am

No I had not considered that ! Thank you very much for your suggestion ! Will get back when I have any feedback on this.

rs

PS : For some reason, If I use special characters my messages get truncated.

performance issues using cactid with external scripts

performance issues using cactid with external scripts

Who is online