Creating new graphs on the same settings not working

mariusz · Post by **mariusz** » Wed May 20, 2009 5:52 am

All goes like this (all pictures below) ...

There are 4 devices (Draytek IP Access DSLAM A24S), all the same (the same model, snmp settings, timeouts, etc on devices and in Cacti):

1. dslam01
2. dslam02
3. dslam09
4. dslam10

Picture 1 - Device List

Let's see more details on specific device (Picture 2 - dslam01 settings). Three remaining devices are the same in every way (host template none, snmp timeout, associated data queries, etc.) dslam01 IS FINE. So ... let's see what am i graphing on dslam01 (24 graphs, 24 sources, see Picture 3). I can see in/out traffic on all 24 ports, and this works (graphs, Picture 4. Below graph and data source debug.

Code: Select all

 Graph Template Selection [edit: dslam01 - iface 100 (user) 512/128]: Interface Traffic (bits/sec - Total Bandwith)
Host: dslam01
Title: |host_description| - iface 100 (user) 512/128
Inbound Data Source: dslam01 - Traffic (traffic_in)
Outband Data Source: dslam01 - Traffic (traffic_out)
RRDTool Command:

C:/rrdtool/rrdtool.exe graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="dslam01 - iface 100 (rondomusic) 512/128" \
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="bits per second" \
--slope-mode \
--font TITLE:12: \
--font AXIS:8: \
--font LEGEND:10: \
--font UNIT:8: \
DEF:a="C\:/Apache2/htdocs/cacti/rra/dslam01_traffic_in_193.rrd":traffic_in:AVERAGE \
DEF:b="C\:/Apache2/htdocs/cacti/rra/dslam01_traffic_in_193.rrd":traffic_out:AVERAGE \
CDEF:cdefa=a,8,* \
CDEF:cdeff=b,8,* \
AREA:cdefa#00CF00FF:"Inbound"  \
GPRINT:cdefa:LAST:" Current\:%8.2lf %s"  \
GPRINT:cdefa:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefa:MAX:"Maximum\:%8.2lf %s\n"  \
COMMENT:"Total In\:  67.42 MB\n"  \
LINE1:cdeff#002A97FF:"Outbound"  \
GPRINT:cdeff:LAST:"Current\:%8.2lf %s"  \
GPRINT:cdeff:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdeff:MAX:"Maximum\:%8.2lf %s\n"  \
COMMENT:"Total Out\: 335.87 MB\n" 

RRDTool Says:

OK

Code: Select all

Data Template Selection [edit: dslam01 - Traffic]
Selected Data Template: Interface Traffic
Host: dslam01
Supplemental Data Template Data
Name: |host_description| - Traffic
Data Source Path: <path_rra>/dslam01_traffic_in_172.rrd
Data Source Item Fields [traffic_in]
Maximum Value: 100000000
Data Source Item Fields [traffic_out]
Maximum Value: 100000000
Custom Data
Output Type ID: In/Out Bits with Total Bandwith
Index Value: 79
Index Type: ifIndex
Data Source Debug

C:/rrdtool/rrdtool.exe create \
C:/Apache2/htdocs/cacti/rra/dslam01_traffic_in_172.rrd \
--step 300  \
DS:traffic_in:COUNTER:600:0:100000000 \
DS:traffic_out:COUNTER:600:0:100000000 \
RRA:AVERAGE:0.5:1:600 \
RRA:AVERAGE:0.5:6:700 \
RRA:AVERAGE:0.5:24:775 \
RRA:AVERAGE:0.5:288:797 \
RRA:MAX:0.5:1:600 \
RRA:MAX:0.5:6:700 \
RRA:MAX:0.5:24:775 \
RRA:MAX:0.5:288:797 \

Now i want to make the same thing on dslam02,09, and 10.

1. I click on 'Devices' (left side)
2. I click on dslam02
3. I click 'create graphs for this host' (now i see list of ifaces on dslam02).
4. I choose the one with index 77
5. I select a graph type (In/Out Bits with Total Bandwith)
6. I click 'Create'
7. I see '+ Created graph: dslam02 - Traffic'
8. I click 'Devices' again
9. I see that dslam02 has now one graph and one data source (earlier was zero)
10. I wait 2 poller runs (10 min)

In the meantime i check 'System Utilities -> Viev Cacti Log File', and i see:

Code: Select all

05/20/2009 12:25:08 PM - SYSTEM STATS: Time:7.6682 Method:cmd.php Processes:2 Threads:N/A Hosts:8 HostsPerProcess:4 DataSources:254 RRDsProcessed:96

05/20/2009 12:30:08 PM - SYSTEM STATS: Time:7.6738 Method:cmd.php Processes:2 Threads:N/A Hosts:8 HostsPerProcess:4 DataSources:254 RRDsProcessed:96

05/20/2009 12:35:08 PM - SYSTEM STATS: Time:7.6999 Method:cmd.php Processes:2 Threads:N/A Hosts:8 HostsPerProcess:4 DataSources:254 RRDsProcessed:96

What the hell ?!?! Now new DataSources ano NO NEW RRDsProcessed:96 ?!?! Let's see debugs from dslam02.

Code: Select all

Graph Template Selection [edit: dslam02 - Traffic] 
Selected Graph Template: Interface Traffic (bits/sec - Total Bandwith)
Host: dslam02
Title: |host_description| - Traffic
Inbound Data Source: dslam02 - Traffic (traffic_in)
Outband Data Source: dslam02 - Traffic (traffic_out)
RRDTool Command:

C:/rrdtool/rrdtool.exe graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="dslam02 - Traffic" \
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="bits per second" \
--slope-mode \
--font TITLE:12: \
--font AXIS:8: \
--font LEGEND:10: \
--font UNIT:8: \
DEF:a="C\:/Apache2/htdocs/cacti/rra/dslam02_traffic_in_211.rrd":traffic_in:AVERAGE \
DEF:b="C\:/Apache2/htdocs/cacti/rra/dslam02_traffic_in_211.rrd":traffic_out:AVERAGE \
CDEF:cdefa=a,8,* \
CDEF:cdeff=b,8,* \
AREA:cdefa#00CF00FF:"Inbound"  \
GPRINT:cdefa:LAST:" Current\:%8.2lf %s"  \
GPRINT:cdefa:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefa:MAX:"Maximum\:%8.2lf %s\n"  \
COMMENT:"Total In\:  0 bytes\n"  \
LINE1:cdeff#002A97FF:"Outbound"  \
GPRINT:cdeff:LAST:"Current\:%8.2lf %s"  \
GPRINT:cdeff:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdeff:MAX:"Maximum\:%8.2lf %s\n"  \
COMMENT:"Total Out\: 0 bytes\n" 

RRDTool Says:

ERROR: opening 'C:/Apache2/htdocs/cacti/rra/dslam02_traffic_in_211.rrd': No such file or directory

ERROR: opening 'C:/Apache2/htdocs/cacti/rra/dslam02_traffic_in_211.rrd': No such file or directory ???

Code: Select all

Data Template Selection [edit: dslam02 - Traffic]
Selected Data Template: Interface Traffic
Host: dslam02
Supplemental Data Template Data
Name: |host_description| - Traffic
Data Source Path: <path_rra>/dslam02_traffic_in_211.rrd
Data Source Item Fields [traffic_in]
Maximum Value: 100000000
Data Source Item Fields [traffic_out]
Maximum Value: 100000000
Custom Data
Output Type ID: Data query data sources must be created through New Graphs.
Index Value: empty
Index Type: Data query data sources must be created through New Graphs.
Data Source Debug

C:/rrdtool/rrdtool.exe create \
C:/Apache2/htdocs/cacti/rra/dslam02_traffic_in_211.rrd \
--step 300  \
DS:traffic_in:COUNTER:600:0:100000000 \
DS:traffic_out:COUNTER:600:0:100000000 \
RRA:AVERAGE:0.5:1:600 \
RRA:AVERAGE:0.5:6:700 \
RRA:AVERAGE:0.5:24:775 \
RRA:AVERAGE:0.5:288:797 \
RRA:MAX:0.5:1:600 \
RRA:MAX:0.5:6:700 \
RRA:MAX:0.5:24:775 \
RRA:MAX:0.5:288:797 \

Output Type ID: Data query data sources must be created through New Graphs.
Index Value: empty
Index Type: Data query data sources must be created through New Graphs.

What am i doing wrong ? Any advice ?

Also ...

BSOD2600 wrote:You've installed the cacti 0.8.7d patches, right?

Yes.

BSOD2600 wrote:If you manually run that 'c:/rrdtool/rrdtool.exe create' command from the cacti.log, does it work?

Yes, it created my rrd file in correct place etc. , but graphs still not work (logs -> no new rrd's in 5 min cycles)

BSOD2600 · Post by **BSOD2600** » Wed May 20, 2009 1:17 pm

What patching method did you use?

Try clearing the poller cache. Then look in the poller cache for that dslam02 device -- is there anything?

You only have those two plugins enabled?

mariusz · Post by **mariusz** » Thu May 21, 2009 12:51 am

BSOD2600 wrote:What patching method did you use?

I choose pre-patched from: http://www.cacti.net/downloads/patches/ ... e-patched/ and i restarted server/computer after that.

BSOD2600 wrote:Try clearing the poller cache. Then look in the poller cache for that dslam02 device -- is there anything?

No, dslam02 shows zero information, nothing. For example dslam01 shows nice:

SNMP Version: 1, Community: public, OID: .1.3.6.1.2.1.2.2.1.10.77
RRD: C:\Apache2\htdocs\cacti\rra\dslam01_traffic_in_170.rrd

BSOD2600 wrote:You only have those two plugins enabled?

I was playing with other (flovwiew, weather, ReportIt etc.) but i don't need them right now and i want to have my Cacti "clean" as possible. So i use only host info for making reports of my configuration, and settings for global plugins setups.

BSOD2600 · Post by **BSOD2600** » Thu May 21, 2009 1:46 am

1) The patched files have the correct NTFS security on them?

2) Does the cacti technical support page show any errors?

3) Back to the cacti devices screen you pasted. If dslam02-04 doesn't have any data sources associated with them and your downtime detection method is only snmp, the up/down status should be UNKNOWN or down, not up. Cacti will only test if a device is down (when using snmp) if it has a data source to poll. That makes me wonder what is going on, since that's not quite right...

mariusz · Post by **mariusz** » Thu May 21, 2009 2:24 am

BSOD2600 wrote:1) The patched files have the correct NTFS security on them?

I can't check this with Sysinternals Process Monitor (my system hang up all the time when i start this program). I must use other way to check security/permission.

BSOD2600 wrote:2) Does the cacti technical support page show any errors?

No, only sql table showing that someone is using some table (i think me watching technical

). I can put technical info on forum if You want.

BSOD2600 wrote:3) Back to the cacti devices screen you pasted. If dslam02-04 doesn't have any data sources associated with them and your downtime detection method is only snmp, the up/down status should be UNKNOWN or down, not up. Cacti will only test if a device is down (when using snmp) if it has a data source to poll. That makes me wonder what is going on, since that's not quite right...

Yeah. This is strange from the start. Yesterday i deleted dslam02. Status of device is now Unknown all the time. I graph earlier some octets strings from rest of dslams with good results (all goes fine, i made custom graph, data, host template with needed OIDs etc.). But i don't need this octets ;p I need to associate SNMP - Interface Statistics data query and graph some traffic (like on dslam01)

I will delete dslam09 and 10 and make new devices.

And look on next picture below. When i was writing this post i add some new statistics to dslam01 (the good one, first one). In/Out bytes or something like that, just too check this NTFS permissions or security or other issues. And .... what the hell ?! ALL IS OK. I don't understand what am i doing wrong with dslam02, 09, and 10

I added even new device DSLMAX from Lucent and this works. My local comp graphs works. Some Linux boxes works. Only this 3 dslam's don't.

Second picture - new look of Devices

BSOD2600 · Post by **BSOD2600** » Thu May 21, 2009 2:40 am

Very odd indeed.

To check ntfs permissions, I was thinking more along the lines of using the Security tab on the folder/files in Windows

.

Change the cacti logging level to debug (or dev if you're brave). Try adding something to the 'broken' dslams. See in the log file the stuff added to the database? Leave it on debug for a polling cycle. Whats cacti do with the broken dslams and the data sources?

mariusz · Post by **mariusz** » Thu May 21, 2009 3:13 am

BSOD2600 wrote:Very odd indeed.

To check ntfs permissions, I was thinking more along the lines of using the Security tab on the folder/files in Windows .

Done. Those 4 files (pre-patched) 'comes from other computer', so i repair this

No help with bad devices.

BSOD2600 wrote:Change the cacti logging level to debug (or dev if you're brave). Try adding something to the 'broken' dslams. See in the log file the stuff added to the database? Leave it on debug for a polling cycle. Whats cacti do with the broken dslams and the data sources?

I changed to dev, but .... log is huge now. Can you be more specific with "Whats cacti do with the broken dslams and the data sources?". What must i check (search string for ctrl+f, message type or something ;p ) ?

mariusz · Post by **mariusz** » Thu May 21, 2009 4:04 am

... and another strange thing (i've forgotten to tell this earlier). When i add new device (other name, but this is dslam09 for example, the same snmp settings, host temaplate none etc. only IP is other then dslam01, i used IP of dslam09) of course graphing not work. When i:

- go to 'Create graphs for this host'
- choose interface (index)
- choose 'Select a graph type'
- i select 'In Out Bits with ... '
- click 'Create'
- + Created graph: 09 - Traffic
- i choose next interface (index)

Previous choosen index should be NOT ACTIVE in the same graph type (In out bits with ...) but IT IS ACTIVE. In all other devices, Lucent, Linux boxes, XP machines, dslam01 previous choosen index is NOT ACTIVE (grayed out, i don't know how to say it

).

BSOD2600 · Post by **BSOD2600** » Thu May 21, 2009 5:39 am

For broken things to search for... try 'error' or HOST[XX] (whatever the host id is -- look in the url). Maybe try debug mode for starters.

mariusz wrote:Previous choosen index should be NOT ACTIVE in the same graph type (In out bits with ...) but IT IS ACTIVE. In all other devices, Lucent, Linux boxes, XP machines, dslam01 previous choosen index is NOT ACTIVE (grayed out, i don't know how to say it ).

umm after you've created a graph on a specific index and graph type, it SHOULD be greyed out / unelectable.

How did you come about this broken Cacti installation anyways?

mariusz · Post by **mariusz** » Thu May 21, 2009 6:39 am

BSOD2600 wrote:For broken things to search for... try 'error' or HOST[XX] (whatever the host id is -- look in the url). Maybe try debug mode for starters.

There are no errors with new added device. Just normal poller data, sql queries etc. New device is hmmm... invisible, ignored. 'error' output is clean.

BSOD2600 wrote:umm after you've created a graph on a specific index and graph type, it SHOULD be greyed out / unelectable.

Yes, it is unselectable for anything else, even some snmp enabled printer graphs. But when i choose one of this 3 broken dslams or delete one of them add new ... is selectable, even when i choose 2nd time to graph THE SAME index in THE SAME graph type.

BSOD2600 wrote:How did you come about this broken Cacti installation anyways?

What do You mean ?

BSOD2600 · Post by **BSOD2600** » Thu May 21, 2009 2:29 pm

mariusz wrote:There are no errors with new added device. Just normal poller data, sql queries etc. New device is hmmm... invisible, ignored. 'error' output is clean.

There is nothing in the cacti.log regarding that device (HOST[##]) or data source (DS[##]) ?

mariusz wrote:
BSOD2600 wrote:How did you come about this broken Cacti installation anyways?
What do You mean ?

Just that there is obviously something broken about this cacti installation. Wondering if you've had it for years or recently inherited someone else project

Just for giggles, go ahead and run \cacti\cli\repair_database.php

mariusz · Post by **mariusz** » Mon May 25, 2009 3:19 am

BSOD2600 wrote:There is nothing in the cacti.log regarding that device (HOST[##]) or data source (DS[##]) ?

Nothing, zero.

BSOD2600 wrote:Just that there is obviously something broken about this cacti installation. Wondering if you've had it for years or recently inherited someone else project

It's my own project. Fresh, alive and kicking

All is made on new box, new MS windows, new install (april this year, using Yours installer)

BSOD2600 wrote:Just for giggles, go ahead and run \cacti\cli\repair_database.php

Done, no help. Not working still.

I will try to add some new info. In weekend i made all from the start.

1. I deleted all my 4 IP DSLams, (01, 02, 09, 10) and other devices, graphs, data queries etc.

2. In meantime i made some custom graphs for other devices (just for make sure that all is fine with cacti install.) I graphed my local machine (Cacti server btw

) traffic, cpu usage, users etc, then i graphed some routers (Unix boxes) traffic, users, filesystem etc. and i graph other IP DSLAMS (other hardware, other room, other ips) and some other stuff. All goes well, graphs look fine.

3. Then i prepare all for graphing my four needed DSLAMS.

- they are four
- they are in the same server room
- they have IPs like 1.1.1.1, 1.1.1.2, 1.1.1.3 and 1.1.1.4
- all are from the same producent
- all are the same model
- all have the same snmp settings on them
- all are working

4. I add new device: dslam01 (no host template, data queries etc.) just clean device.

5. Then i add 3 remainings devices, in the same way.

6. I clicked on all four devices (for make sure that snmp is responding). Working in four.

7. Time to graph.

8. I choose dslam01 for start. I add Associated Data Queries: SNMP - Interface Statistics with Success (870 Items, 145 Rows, i will add xml query to this post)

9. I go to Create graphs for .... and so on.

10. I made 24 graphs starting from index 77 (traffic in/out with total bandwith).

11. All 24 graphs and data sources are added to dslam01 in devices list.

12. Wait 2 poller runs - all working.

13. Next i choose dslam02.

14. I add Associated Data Queries: SNMP - Interface Statistics with Success (290 items, 145 rows)

Something is wrong, why only 290 items ? dslam01 have more (870). But ok. Lets go to graphs.

15. Create new graph ... (dslam02) and this look strange (see pictures below). But i add anyway this 24 graphs.

16. New 24 graphs and 24 sources was added to dslam02 in devices list.

17. I try something. I go again to Create graphs for .... and click again type traffic in/out with ... and all 24 indexes ARE ACTIVE (like i wrote in earlier posts). Logs (like i wrote eralier) says nothing. Status in devices list is all time Unknown. Graphs not working (like i wrote earlier).

The same situation is with dslam 09 and 10. Three of them don't work.

I made new pictures.

mariusz · Post by **mariusz** » Mon May 25, 2009 6:04 am

I find next clue. Maybe it will be some help.

GetIf - from Cacti Installer for windows read ALL 4 devices (dslams 01,02,09,10) with no problems or missing data in 'Interfaces' tab.

Also i find now strange logs in getif.log:

Code: Select all

MODULE-IDENTITY(is a reserved word): At line 55 in C:\php\extras\mibs/SNMPv2-SMI.txt
Bad operator(MACRO): At line 55 in C:\php\extras\mibs/SNMPv2-SMI.txt
Did not find 'mib-2' in module SNMPv2-SMI
Unlinked OID in UDP-MIB: udpMIBConformance ::= { udpMIB 2 }
Unlinked OID in UDP-MIB: udpMIBGroups ::= { udpMIBConformance 2 }
Unlinked OID in UDP-MIB: udpMIBCompliances ::= { udpMIBConformance 1 }
Unlinked OID in UDP-MIB: udpMIBCompliance ::= { udpMIBCompliances 1 }
Unlinked OID in UDP-MIB: udp ::= { mib-2 7 }
Unlinked OID in UDP-MIB: udpMIB ::= { mib-2 50 }
Unlinked OID in UDP-MIB: udpGroup ::= { udpMIBGroups 1 }
Unlinked OID in UDP-MIB: udpTable ::= { udp 5 }
Unlinked OID in UDP-MIB: udpOutDatagrams ::= { udp 4 }
Unlinked OID in UDP-MIB: udpInErrors ::= { udp 3 }
Unlinked OID in UDP-MIB: udpNoPorts ::= { udp 2 }
Unlinked OID in UDP-MIB: udpInDatagrams ::= { udp 1 }
Unlinked OID in UDP-MIB: udpEntry ::= { udpTable 1 }
Unlinked OID in UDP-MIB: udpLocalPort ::= { udpEntry 2 }
Unlinked OID in UDP-MIB: udpLocalAddress ::= { udpEntry 1 }
TEXTUAL-CONVENTION(is a reserved word): At line 8 in C:\php\extras\mibs/SNMPv2-TC.txt
Bad operator(MACRO): At line 8 in C:\php\extras\mibs/SNMPv2-TC.txt
Did not find 'enterprises' in module SNMPv2-SMI
Did not find 'DisplayString' in module SNMPv2-TC
Did not find 'TruthValue' in module SNMPv2-TC

BSOD2600 · Post by **BSOD2600** » Mon May 25, 2009 6:48 pm

Each time, it seems it's just a few specific dslams that have issues, not them all.

You said the dslams are identical in terms of hardware and software config? Something is causing 02 to return far less rows than 01. Click the verbose query button next to the snmp interface query associated data query. What are the obvious differences? Another way to check the differences is a snmpwalk of the interfaces on each.

Also, when I said to look through the cacti.log for host[##], those ## represent the real host ID number. You got that, right?

As for the getif log errors, it's simply listing out what it thinks are non-conforming MIB files and their respective errors -- not anything with the dslam devices.

Have you enabled error logging in php.ini? Once done, restart the web service and check for any errors after using cacti a bit/poller has ran.

Creating new graphs on the same settings not working

Creating new graphs on the same settings not working

Who is online