DNS lookup problem in device management page

[rujipars]

Previously I use '/etc/hosts' for hostname to IP address mapping. Recently I setup an external DNS server and try to use the name from DNS server lookup instead of '/etc/hosts'.

I test the operation with DNS name successfully like this.

Ping from Linux CLI
[root@PRI_NMS cli]# ping tdnbkk
PING tdnbkk.sta.net (10.1.253.34) 56(84) bytes of data.
64 bytes from 10.1.253.34: icmp_seq=1 ttl=252 time=60.6 ms
64 bytes from 10.1.253.34: icmp_seq=2 ttl=252 time=71.2 ms

Manual snmpwalk from Linux CLI (SNMPv1 and v2 is OK)
[root@PRI_NMS cli]# snmpwalk -v 1 -c thaidonostc tdnbkk
SNMPv2-MIB::sysDescr.0 = STRING: Cisco Internetwork Operating System Software
IOS (tm) C805 Software (C805-Y6-MW), Version 12.2(15)T13, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Wed 16-Jun-04 1
SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.9.1.245
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (251738366) 29 days, 3:16:23.66
SNMPv2-MIB::sysContact.0 = STRING:
SNMPv2-MIB::sysName.0 = STRING: ThaiDoNo_BKK
SNMPv2-MIB::sysLocation.0 = STRING:
SNMPv2-MIB::sysServices.0 = INTEGER: 78
SNMPv2-MIB::sysORLastChange.0 = Timeticks: (0) 0:00:00.00
IF-MIB::ifNumber.0 = INTEGER: 5
IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.2 = INTEGER: 2
IF-MIB::ifIndex.3 = INTEGER: 3
IF-MIB::ifIndex.5 = INTEGER: 5

[root@PRI_NMS cli]# snmpwalk -v 2c -c thaidonostc tdnbkk
SNMPv2-MIB::sysDescr.0 = STRING: Cisco Internetwork Operating System Software
IOS (tm) C805 Software (C805-Y6-MW), Version 12.2(15)T13, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2004 by cisco Systems, Inc.
Compiled Wed 16-Jun-04 1
SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.9.1.245
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (251739020) 29 days, 3:16:30.20
SNMPv2-MIB::sysContact.0 = STRING:
SNMPv2-MIB::sysName.0 = STRING: ThaiDoNo_BKK
SNMPv2-MIB::sysLocation.0 = STRING:
SNMPv2-MIB::sysServices.0 = INTEGER: 78
SNMPv2-MIB::sysORLastChange.0 = Timeticks: (0) 0:00:00.00
IF-MIB::ifNumber.0 = INTEGER: 5

I've created a host to test with the DNS name from Cacti. Here I found some problem like this.

Information on the Device page
ThaiDono-BKK (tdnbkk)
SNMP Information
SNMP error
Ping Results
ICMP Ping timed out

Cacti log file
this message occur when open device page
02/01/2011 10:59:49 AM - CMDPHP: Poller[0] WARNING: ICMP Ping Error: gethostbyname failed for tdnbkk

this message occur in polling cycle
02/01/2011 11:10:13 AM - POLLER: Poller[0] WARNING: Poller[0] Host[482] SNMP Time was not numeric

Data Query Debug Information when I use Verbose Query for SNMP Interface Statistics
+ Running data query [1].
+ Found type = '3' [snmp query].
+ Found data query XML file at '/var/www/cacti/resource/snmp_queries/interface.xml'
+ XML file parsed ok.
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.2.1.2.2.1.1'
+ No SNMP data returned
+ Found data query XML file at '/var/www/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at '/var/www/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at '/var/www/cacti/resource/snmp_queries/interface.xml'

The SNMP queries does not work so the interface table does not show up so I can't create interface traffic graph.
I tried creating simple SNMP OID graphing like Cisco CPU and it works. I see CPU graph working correctly all the time.

I think poller is working properly but the device management, including SNMP queries that list items for administration.

Can anyone guide me further?

[rujipars]

Code: Select all

Technical Support
General Information
Date 	Tue, 01 Feb 2011 11:35:41 +0700
Cacti Version 	0.8.7e
Cacti OS 	unix
SNMP Version 	NET-SNMP version: 5.3.2.2
RRDTool Version 	RRDTool 1.2.x
Hosts 	396
Graphs 	1858
Data Sources 	Script/Command: 13
SNMP: 357
SNMP Query: 1524
Script Query: 4
Script Query - Script Server: 29
Total: 1927
Poller Information
Interval 	300
Type 	cmd.php
Items 	Action[0]: 3447
Action[1]: 21
Action[2]: 52
Total: 3520
Concurrent Processes 	15
Max Threads 	4
PHP Servers 	4
Script Timeout 	25
Max OID 	10
Last Run Statistics 	Time:52.6720 Method:cmd.php Processes:15 Threads:N/A Hosts:341 HostsPerProcess:23 DataSources:3520 RRDsProcessed:1730
PHP Information
PHP Version 	5.1.6
PHP OS 	Linux
PHP uname 	Linux PRI_NMS 2.6.18-164.11.1.el5 #1 SMP Wed Jan 20 07:39:04 EST 2010 i686
PHP SNMP 	Installed
max_execution_time 	30
memory_limit 	48M

[noname]

02/01/2011 10:59:49 AM - CMDPHP: Poller[0] WARNING: ICMP Ping Error: gethostbyname failed for tdnbkk

Please check your /etc/resolv.conf whether this line exists:

domain sta.net
OR
search sta.net

[rujipars]

Dear noname,

To verify as your suggestion.

[root@PRI_NMS ~]# cat /etc/resolv.conf
search sta.net
nameserver 203.33.240.51

[root@PRI_NMS ~]# ping tdnbkk
PING tdnbkk.sta.net (10.1.253.34) 56(84) bytes of data.
64 bytes from 10.1.253.34: icmp_seq=1 ttl=252 time=69.5 ms
64 bytes from 10.1.253.34: icmp_seq=2 ttl=252 time=81.1 ms

--- tdnbkk.sta.net ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 69.510/75.328/81.147/5.824 ms

[root@PRI_NMS ~]# ping tdnbkk.sta.net
PING tdnbkk.sta.net (10.1.253.34) 56(84) bytes of data.
64 bytes from 10.1.253.34: icmp_seq=1 ttl=252 time=64.5 ms

--- tdnbkk.sta.net ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 64.501/64.501/64.501/0.000 ms
[root@PRI_NMS ~]#

Thanks for your response.

[noname]

Hmm...
I can make similar situation as yours, but the result is different to yours.

When I delete 'search example.net' from resolv.conf,
I can see 'SNMP Error' at Device page and 'UDP Ping Error: gethostbyname failed' in cacti.log. (I use UDP ping by reason of privileges)
hostname-only-ping in CLI is failed.

Code: Select all

% ping foobar
ping: unknown host foobar

% ping foobar.example.net
foobar.example.net is alive

Furthermore, when I execute below code at this time, I couldn't get ip addresses.

Code: Select all

<?php
        echo gethostbyname("foobar") . "\n";
?>

% php test.php
foobar

And then, when I add 'search example.net' to /etc/resolv.conf,
all the above worked successfully.

% php test.php
192.168.100.1
(I got the same result when I executed this by cacti user privileges)


Sorry I can't figure out your situations yet..

[rujipars]

From the evidence in my first post, my conclusion is that I have DNS problem while working with console web page but the poller could work properly.

I'm quite sure that there should not be some problem about 'resolv.conf' because all CLI tests work fine, including poller (I can plot CPU graph).

You may be able to reproduce the same output only for the console web, but your result with poller and CLI (if you tried it) are different than mine.

Anyway, thanks for your effort to help

[rujipars]

Here is the data in cacti database

Code: Select all

mysql> select id,hostname,snmp_community,snmp_port from host where id = 498;
+-----+-----------------+----------------+-----------+
| id  | hostname        | snmp_community | snmp_port |
+-----+-----------------+----------------+-----------+
| 498 | tdn-bkk.sta.net | thaidonostc    |       161 | 
+-----+-----------------+----------------+-----------+
1 row in set (0.00 sec)

I tried PHP's snmprealwalk function with this simple code for ifTable.ifIndex.

[root@PRI_NMS ~]# cat test2.php
<?php
$a = @snmprealwalk("tdn-bkk.sta.net", "thaidonostc", ".1.3.6.1.2.1.2.2.1.1");

foreach ($a as $val) {
echo "$val\n";
}

?>

[root@PRI_NMS ~]# php test2.php
INTEGER: 1
INTEGER: 2
INTEGER: 3
INTEGER: 5
INTEGER: 6

At last, I tried verbose query for SNMP Interface Statistics with SNMPv1, SNMPv2. They show the same output


Data Query Debug Information
+ Running data query [1].
+ Found type = '3' [snmp query].
+ Found data query XML file at '/var/www/cacti/resource/snmp_queries/interface.xml'
+ XML file parsed ok.
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.2.1.2.2.1.1'
+ No SNMP data returned from tdn-bkk.sta.net <-- I modified Cacti's code to show hostname here
+ Found data query XML file at '/var/www/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at '/var/www/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at '/var/www/cacti/resource/snmp_queries/interface.xml'

[wulfhere]

Did you ever find the solution to this? I'm having the same issue with 0.8.7g on Fedora. Name lookups work fine from the CLI, and I'm using fully qualified hostnames, but I just get SNMP error and UDP Ping error in the web page. If I use IPs instead of hostnames, it works fine. Very strange.

[rujipars]

Yes, still have the problem.

I wonder whether a significant amount of cacti user use DNS name instead of IP or /etc/hosts.

[wulfhere]

It's very odd. I'm going to try posting something to the Help:General section, since there seems to be more traffic over there, and I somehow doubt this is a linux-specific issue. I'll let you know if I find out anything.

[thilak]

Fine!!!
For the good reverse DNS lookup ,
I visit this site http://www.whoisxy.com/domain-to-ip.aspx
It has the best information of IP address, IP address to domain, domain name to IP,domain name, DNS lookup., IP address lookup and ping test !!!

[marktees]

Hi there,

I had a similar issue with Cacti and by turn PHP gethostbyname.

The problem was that Cacti using gethostbyname to resolve a name was failing. Where as my PHP test script resolving the same name was working.

A packet capture revealed that the PHP instance Cacti was using was using old DNS resolvers. IE: Not what we had in /etc/resolv.conf

We had changed the resolvers on the box in the last month but httpd had not been restarted. Restarting httpd resolved the issue and Cacti's instance of PHP/gethostbyname started using the new resolvers.

So, without looking into gethostbyname too closely it appears PHP holds the configured resolvers somewhere perhaps in environment variables or elsewhere.