program to import RRD's

Post general support questions here that do not specifically fall into the Linux or Windows categories.

Moderators: Developers, Moderators

Post Reply
abush
Posts: 7
Joined: Sun Jan 27, 2002 7:00 pm
Location: Columbus, OH

program to import RRD's

Post by abush »

I have attached a program (import-rrd.php) that can be used to import existing RRD's that you may have been created using other tools or even by hand. The program is very young and done quickly so it is very messy but works for me.

To use:

1) copy attached source to a file named import-rrd.php in your cacti install directory.

2) place a RRD that you would like to import into the rra/ subdirectory of the cacti install directory.

3) go to http://cacti-server/cacti-dir/import-rrd.php page from a web browser.

4) enter the name of the RRD file you would like to import (including the .rrd extension).

5) If you would like to preview the SQL inserts and _not_ have them actually done then check "preview only".

6) The import will create Data Sources records and Graphs for each DS in the RRD, each graph will have one item assigned to it and several values involving display set to defaults. (this really needs to be improved on)

7) The import will not add the graphs to the Graph Hierarchy.

PS:
Extending cacti would be much easier if the DB and RRD manipulation was seperated from the UI via some type of abstraction, i.e. classes. It is difficult to determine what inserts/updates _must_ happen to keep the db in a stable state.
---------------------------------------------

Code: Select all

<?php

/* 
	program to import an existing rrd into cacti; this is very messy... 

	Aaron S. Bush (01/31/02)

	TODO: 
	: ability to import files not on the server via file upload.
	: ability to customize graphs...etc
	: a common API to work with the cacti DB would help prevent broken data.
*/

$section = "Add/Edit Data Sources"; 
include ('auth/include/auth.php');
header("Cache-control: no-cache");
include ('include/database.php');
include ('include/config.php');
include_once ('include/form.php');

include_once ("include/cmd_functions.php");
include_once ('include/top_header.php');


/* start element handler for import of RRD XML data */

Function start_element($parser, $name, $attribs) {
	GLOBAL $element_status;

	switch($name) {
		case 'step':
			$element_status['in_step'] = TRUE;
			break;
		case 'ds':
			$element_status['in_ds'] = TRUE;
			break;
		case 'name':
			if ($element_status['in_ds']) {
				$element_status['in_ds_name'] = TRUE;
			}
			break;
		case 'type':
			if ($element_status['in_ds']) {
				$element_status['in_ds_type'] = TRUE;
			}
			break;
		case 'minimal_heartbeat':
			if ($element_status['in_ds']) {
				$element_status['in_ds_heart'] = TRUE;
			}
			break;
		case 'min':
			if ($element_status['in_ds']) {
				$element_status['in_ds_min'] = TRUE;
			}
			break;
		case 'max':
			if ($element_status['in_ds']) {
				$element_status['in_ds_max'] = TRUE;
			}
			break;
		default:
			break;
	}
} /* start_element */


/* end element handler for import of RRD XML data */

Function end_element($parser, $name) {
	GLOBAL $element_status, $ds_import_offset;

	switch($name) {
		case 'step':
			$element_status['in_step'] = FALSE;
			break;
		case 'ds':
			$element_status['in_ds'] = FALSE;
			$ds_import_offset++;
			break;
		case 'name':
			if ($element_status['in_ds']) {
				$element_status['in_ds_name'] = FALSE;
			}
			break;
		case 'type':
			if ($element_status['in_ds']) {
				$element_status['in_ds_type'] = FALSE;
			}
			break;
		case 'minimal_heartbeat':
			if ($element_status['in_ds']) {
				$element_status['in_ds_heart'] = FALSE;
			}
			break;
		case 'min':
			if ($element_status['in_ds']) {
				$element_status['in_ds_min'] = FALSE;
			}
			break;
		case 'max':
			if ($element_status['in_ds']) {
				$element_status['in_ds_max'] = FALSE;
			}
			break;
		default:
			break;
	}
} /* end_element */


/* data element handler for import of RRD XML data */

Function data_handler($parser, $data) {
	GLOBAL $element_status, $step, $ds_import_query, $ds_import_offset, $ds_types;

	if (strlen($data = trim($data)) <= 0 ) {
		return;
	}

	if ($element_status['in_step']) {
		$step = $data;
	} elseif ($element_status['in_ds']) {
		if ($element_status['in_ds_name']) {
			$ds_import_query[$ds_import_offset] .= "DSName = '$data', ";
		} elseif ($element_status['in_ds_type']) {
			$ds_import_query[$ds_import_offset] .= "DataSourceTypeID = ".$ds_types["$data"].", ";
		} elseif ($element_status['in_ds_heart']) {
			$ds_import_query[$ds_import_offset] .= "Heartbeat = $data, ";
		} elseif ($element_status['in_ds_min']) {
			$sci_split = explode('e', $data, 2);
			if (count($sci_split) == 2) {
				$data = $sci_split[0] * (pow(10,$sci_split[1]));
			} else {
				$data = 0;
			}
			$ds_import_query[$ds_import_offset] .= "MinValue = ".$data.", ";
		} elseif ($element_status['in_ds_max']) {
			$sci_split = explode('e', $data, 2);
			if (count($sci_split) == 2) {
				$data = $sci_split[0] * (pow(10,$sci_split[1]));
			} else {
				$data = 0;
			}
			$ds_import_query[$ds_import_offset] .= "MaxValue = ".$data;
		} else {
			return;
		}
	} else {
		return;
	}
} /* data_handler */


/* == MAIN == */

switch($paction) {
	case 'import':

		$ds_import_query = array();
		$ds_import_offset = 0;
		
		/* collect existing def_ds records */

		if (! ($result = mysql_query("SELECT * FROM def_ds")) ) {
			DrawMatrixTableBegin("97%");
			DrawMatrixRowBegin();
				DrawMatrixHeaderTop("Error: Selecting def_ds",$color_dark_bar,"FF0000","4");
DrawMatrixRowEnd();
			DrawMatrixTableEnd();
			break;
		} else {
			$process_files[] = $path_rra."/".$s_fname;
		} 

		/* prep XML handlers */

		$xml_parser = xml_parser_create();
	
		xml_parser_set_option($xml_parser, XML_OPTION_CASE_FOLDING, 0);
		xml_set_element_handler($xml_parser, "start_element", "end_element");
		xml_set_character_data_handler($xml_parser, "data_handler");

		while (list(,$file) = each($process_files)) {
			
			/* initialize xml element status flags */

			$element_status['in_step'] = FALSE;
			$element_status['in_ds'] = FALSE;
			$element_status['in_ds_name'] = FALSE;
			$element_status['in_ds_type'] = FALSE;
			$element_status['in_ds_heart'] = FALSE;
			$element_status['in_ds_min'] = FALSE;
			$element_status['in_ds_max'] = FALSE;

			list($base_name) = explode('.', basename($file), 2);
			if (!($fp = popen($config['path_rrdtool']['value']." dump $file", "r"))) {
				DrawMatrixTableBegin("97%");
				DrawMatrixRowBegin();
				DrawMatrixHeaderTop("Error: Opening RRD (".$path_rra."/".$s_fname.").",$color_dark_bar,"FF0000","4");
				DrawMatrixRowEnd();
				DrawMatrixTableEnd();
				break;
			}
		
			/* extract rrd info from XML dump */


			while ($data = fread($fp, 4096)) {
				if (!xml_parse($xml_parser, $data, feof($fp))) {
					DrawMatrixTableBegin("97%");
					DrawMatrixRowBegin();
					DrawMatrixHeaderTop("Error: ".
						sprintf("XML error: %s at line %d",
						xml_error_string(xml_get_error_code($xml_parser)),
						xml_get_current_line_number($xml_parser))
						,$color_dark_bar,"FF0000","4"
					);
					DrawMatrixRowEnd();
					DrawMatrixTableEnd();
					pclose($fp);
					exit;
				}
			}
			pclose($fp);

			/* prep import queries */

			while (list($x, $y) = each($ds_import_query)) {
				$ds_import_query[$x] = "INSERT INTO rrd_ds SET Step = $step, Name = '$base_name', Active = '', $y";
				$graph_import_query[$x] = "INSERT INTO rrd_graph SET Title = '$base_name ($x)', ImageFormatID = 1, Height = 120, Width = 500, AutoScale = 'on', AutoScaleOpts = 2";
			}


			/* execute import queries */


			reset($ds_import_query);
			while (list($x, $y) = each($ds_import_query)) {
				print "Setup Data Source: $y
n";
				if ($preview != 'on') {
					if ( ! mysql_query($y) ) {
						print "ERROR on DS import";
						exit;
					} else {
						if ( ! ($result = mysql_query("SELECT LAST_INSERT_ID()")) ) {
							print "ERROR on last id";
							exit;
						} else {
							$rrd_ds_id = mysql_result($result,'',0);
						}
					}
				} else {
					$rrd_ds_id = 'x';
				}

				print "Setup Graph: $graph_import_query[$x]
n";
				if ($preview != 'on') {
					if ( ! mysql_query($graph_import_query[$x]) ) {
						print "ERROR on Graph import";
						exit;
					} else {
						if ( ! ($result = mysql_query("SELECT LAST_INSERT_ID()")) ) {
							print "ERROR on last id";
							exit;
						} else {
							$graph_id = mysql_result($result,'',0);
							$graph_item_import = "INSERT INTO rrd_graph_item SET DSID = $rrd_ds_id, ColorID = 1, GraphTypeID = 7, GraphID = $graph_id, ConsolidationFunction = 1";
						}
					}
				} else {
					$graph_id = 'y';
				}
				
				print "Setup Graph Items: $graph_item_import
n";
				if ($preview != 'on') {
					if ( ! mysql_query($graph_item_import) ) {
						print "ERROR on Graph item import";
						exit;
					} 
				}
			}
		}
		xml_parser_free($xml_parser);

		break;

	default:
		DrawFormHeader("rrdtool Data Source Import",true,false);
			
		DrawFormItem("Server File Name","The name for the RRD file to import; this is for files that are already on the server in the $path_rra/ directory.");
		DrawFormItemTextBox("s_fname","$s_fname","","19");

		DrawFormItemCheckBox("preview","$preview","Preview only","off");
		
		DrawFormItemHiddenTextBox("paction","$paction","import");

		DrawFormSaveButton();
		DrawFormFooter();
		break;
	
}
include_once ("include/bottom_footer.php");
?>
abush
Posts: 7
Joined: Sun Jan 27, 2002 7:00 pm
Location: Columbus, OH

Post by abush »

Well the copy/paste looks pretty bad...
If there is a person/place that i can send this too for a more stable home let me know.

Thanks,
-ab
ekool
Posts: 7
Joined: Tue Jan 29, 2002 7:00 pm
Contact:

Post by ekool »

abush, i can host it, email it to ekool@fiendish.net
raX
Lead Developer
Posts: 2243
Joined: Sat Oct 13, 2001 7:00 pm
Location: Carlisle, PA
Contact:

Post by raX »

I will also gladly put it in /downloads/patches if you send me a copy. (iberry@raxnet.net)

-Ian
ekool
Posts: 7
Joined: Tue Jan 29, 2002 7:00 pm
Contact:

Post by ekool »

Rax, any idea when your mp3 interface is going to be available?
raX
Lead Developer
Posts: 2243
Joined: Sat Oct 13, 2001 7:00 pm
Location: Carlisle, PA
Contact:

Post by raX »

Oh yes, lava. Not sure when I am going to release it, needs some work before it is usable though. I'll keep updates on the raXnet page.

Also you can find Aaron's rrd import script here:

http://www.raxnet.net/downloads/patches ... rt.php.txt

-Ian
Guest

Post by Guest »

Okay....I got cmd_functions.php, updated rrd_functions.php, and rrd_import.php, and tried to import an rrd made by mrtg, and I get nadda...no errors, nadda....any clues?
cocktail@shujian.net

Post by cocktail@shujian.net »

i want to use lava

how get mp3's played times?
bilbo
Posts: 7
Joined: Sat Oct 23, 2004 10:03 am

Post by bilbo »

raX wrote:Also you can find Aaron's rrd import script here:

http://www.raxnet.net/downloads/patches ... rt.php.txt

-Ian
Seems to have changed to
http://www.cacti.net/downloads/patches/ ... rt.php.txt
o019676
Posts: 7
Joined: Sat Apr 07, 2007 11:52 pm

Add Data Source From The Command Line

Post by o019676 »

untar the script

gunzip -dc <filename>|tar xvf -

might need to chmod 755 in the script, or to your liking

If you create a data template then you can use this script to create data sorces by providing the rrd, host id, and the path to the rrd file.

The data template must be created first, thats where it gets it information about the rrd.

Remember to always back up your cacti database first.

This is beta code and is written in bash shell script.

Change the global file to the setting you need, there are some variables there you might not need, especially if you dont use nagios. Just remove that line from the global file, i use it in conjuction with some other scripts to import rrd made from nagiosgraph into cacti.

use the --help to get the use of the script.

Please feel free to send some suggestions and fixes for a final scipt.
Attachments
add_datasource.tar.gz
(2.01 KiB) Downloaded 524 times
o019676
Posts: 7
Joined: Sat Apr 07, 2007 11:52 pm

opps

Post by o019676 »

re read the post

you must supply the data template ID, host id, and path to rrd file.
p.bites
Posts: 1
Joined: Mon Sep 17, 2007 9:03 am

Import File error !!!

Post by p.bites »

Hi. I have the cacti version 0.8.6J install on Fedora Core 5 and the rrd import file when run i have a blank page... not appear.
Thanks for all help.

Paulo
Brasília - Brasil :)
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Post by gandalf »

This script is not officially supported. I fear, that you will have to do it on your own
Reinhard
Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests