RRDTool crazy process and RDD...

[Iceman]

Hi all,
I've been using Cacti since september without any considerable problem...
I have only one problem which i can't find any solution.
I have about 35 hosts, and i suppose i get about 100/150 requests (snmp/personal scripts)
For some reason i can't understand, sometimes, some graph go "crazy" and cannot graph anymore. It produce a rrdtool process that may eat any system resource, eating 50%-100% of cpu.
If i udpate webpage, another rrdtool process hangs, without show graph.
In this situations, i must "kill -9 $PROCESS".
The only way to clean this situation is to rebuild rrd database by delete its db file.
I use cacti 0.8.6c (upgraded today, with same problems) on Sun Blade-1000, Solaris 5.8.

[rony]

Using cactid or cmd.php for poller?

[Iceman]

cmd.php.

Other information, prstat give:
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
5500 nobody 1640K 1568K cpu1 10 0 0:02.36 50% rrdtool/1

if i "ps -ef|grep 5500", i get:
nobody 5500 5499 49 10:59:03 ? 2:05 /usr/local/rrdtool-1.0.49/bin/rrdtool graph - --imgformat=PNG --start=111338274

If i get father process: "ps -ef|grep 5499":
nobody 5499 4677 0 10:59:03 ? 0:00 sh -c /usr/local/rrdtool-1.0.49/bin/rrdtool graph - --imgformat=PNG --start=1

In Solaris, i can test what process do with "truss". If I truss process 5500 (the process at 50% of cpu) nothing is displayed. If i "truss" father process (5499), i get:
truss -p 5499

waitid(P_PID, 5500, 0xFFBEF948, WEXITED|WTRAPPED|WNOWAIT) (sleeping...)

[rony]

Should have asked this before, php version?

[Iceman]

# php -v
PHP 4.3.7 (cli) (built: Sep 20 2004 16:53:04)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies

RRDtool 1.0.49

Not solved....

[TheWitness]

Iceman,

You have a corrupt graph definition. What you need to do is edit the graphs one at a time under graph management until your browser locks up. The you will know what template to modify to fix the problem. This is a nasty unfortunate situation that we are looking to remedy in 0.9.

It sometimes happens when your graph syntax is buggered up (invalid). Updating rrdtool doesn't always fix the problem. (aka, it's an rrdtool bug).

TheWitness

[rony]

Btw, doesn't lock up my browsers.... I think TheWitness is seeing things...

[TheWitness]

Not! Had this problem myself when creating graphs.

[Iceman]

I know that problem is rrdtool that can't manage correctly his *.rrd

Another thing I can't understand is why "rrdtool graph" start with root permission.
Do I update my rrdtool with an earlier version?

[TheWitness]

Tobi fixed the Stack without Area race condition in either RRDTool 1.2.4-1.2.6. So, those lockups should go away.

That was the issue with me, building Stack Graphs would lock up the web browser if I did them in the wrong order. Again, if you use 1.2.x, don't remember what "x" is, you should not have that problem.

TheWitness

[Iceman]

OK, i've installed rrdtool 1.2.8 (together 1.0.49)
Now i read that i have to upgrade cacti to 0.8.6d to work with rrdtool 1.2.8
I read that no new rrdtool's implementation should work in cacti (i don't care)
If I upgrade, do i loose my old RRAs? Will my RRA update correctly by "php poller.php" in crontab with new rrdtool?
What is the best "step-by-step" procedure to upgrade without loose 5minutes polls (in the 10-15 minutes required to upgrade)? (rename old directory, unpack new, copy rra to new folder, ecc...)

[TheWitness]

Your old RRD files will remain compatible between 1.0.x and 1.2.x, but new RRD files will not.

TheWitness