[solved] Can't monitor network stats for VMWare Windows VM's

[danwdoo]

Hello,
I have moved serveral VM's from MS Virtual Server 2005 to VMWare ESXi. Since doing so (everything worked fine under VS), Cacti won't graph any network data for the hosts. It sees the VMware virtual network adapter in my queries but it won't build anything from it. When I check my Cacti logs, I see that it is retrieving a value for traffic and that it is incrementing as I would expect. Is there something wrong with the format that it doesn't like?

Here are two sections from my Cacti log showing the data being retrieved:

05/18/2009 05:45:05 PM - CMDPHP: Poller[0] Host[8] DS[542] SNMP: v1: <HOST FQDN>, dsname: traffic_in, oid: .1.3.6.1.2.1.2.2.1.10.65539, output: 68405481
05/18/2009 05:45:05 PM - CMDPHP: Poller[0] Host[8] DS[542] SNMP: v1: <HOST FQDN>, dsname: traffic_out, oid: .1.3.6.1.2.1.2.2.1.16.65539, output: 631617010

05/18/2009 05:48:09 PM - CMDPHP: Poller[0] Host[8] DS[542] SNMP: v1: <HOST FQDN>, dsname: traffic_in, oid: .1.3.6.1.2.1.2.2.1.10.65539, output: 68449888
05/18/2009 05:48:09 PM - CMDPHP: Poller[0] Host[8] DS[542] SNMP: v1: <HOST FQDN>, dsname: traffic_out, oid: .1.3.6.1.2.1.2.2.1.16.65539, output: 631625871

If anyone has any ideas on how to resolve this I would really appreciate it!

Thanks!
Danny

[BSOD2600]

Follow http://forums.cacti.net/viewtopic.php?t=11747 and post again. Also, post the verbose query from the snmp interface associated data query against that device.

[danwdoo]

Here is the verbose query:

Code: Select all

+ Running data query [1].
+ Found type = '3' [snmp query].
+ Found data query XML file at 'C:/Inetpub/wwwroot/cacti/resource/snmp_queries/interface.xml'
+ XML file parsed ok.
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.2.1.2.2.1.1'
+ Index found at OID: 'interfaces.ifTable.ifEntry.ifIndex.1' value: '1'
+ Index found at OID: 'interfaces.ifTable.ifEntry.ifIndex.65539' value: '65539'
+ Located input field 'ifIndex' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.1'
+ Found item [ifIndex='1'] index: 1 [from value]
+ Found item [ifIndex='65539'] index: 65539 [from value]
+ Located input field 'ifOperStatus' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.8'
+ Found item [ifOperStatus='Up'] index: 1 [from value]
+ Found item [ifOperStatus='Up'] index: 65539 [from value]
+ Located input field 'ifDescr' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.2'
+ Found item [ifDescr='MS TCP Loopback interface'] index: 1 [from value]
+ Found item [ifDescr='VMware Accelerated AMD PCNet Adapter'] index: 65539 [from value]
+ Located input field 'ifName' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.31.1.1.1.1'
+ Located input field 'ifAlias' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.31.1.1.1.18'
+ Located input field 'ifType' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.3'
+ Found item [ifType='softwareLoopback(24)'] index: 1 [from value]
+ Found item [ifType='ethernetCsmacd(6)'] index: 65539 [from value]
+ Located input field 'ifSpeed' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.5'
+ Found item [ifSpeed=' 10000000'] index: 1 [from value]
+ Found item [ifSpeed=' 1000000000'] index: 65539 [from value]
+ Located input field 'ifHwAddr' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.2.2.1.6'
+ Found item [ifHwAddr=''] index: 1 [from value]
+ Found item [ifHwAddr='00:0C:29:60:3E:CF'] index: 65539 [from value]
+ Located input field 'ifIP' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.2.1.4.20.1.2'
+ Found item [ifIP='127.0.0.1'] index: 1 [from regexp oid parse]
+ Found item [ifIP='172.22.10.6'] index: 65539 [from regexp oid parse]
+ Found data query XML file at 'C:/Inetpub/wwwroot/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at 'C:/Inetpub/wwwroot/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at 'C:/Inetpub/wwwroot/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at 'C:/Inetpub/wwwroot/cacti/resource/snmp_queries/interface.xml'

My Install information is:

Code: Select all

Cacti Version 0.8.7d 
Plugin Architecture 2.4-1 
Poller Type CMD.php 
Server Info Windows NT 5.2 
Web Server Microsoft-IIS/6.0 
PHP Version 5.2.5 
PHP Extensions bcmath, calendar, com_dotnet, ctype, session, filter, ftp, hash, iconv, json, odbc, pcre, Reflection, date, libxml, standard, tokenizer, zlib, SimpleXML, dom, SPL, wddx, xml, xmlreader, xmlwriter, ISAPI, gd, mysql, snmp, sockets 
MySQL Version 5.0.51a-community-nt 
RRDTool Version 1.2.26 
Global Plugin Settings (settings - v0.5)
Thresholds (thold - v0.4.1)
Host Info (hostinfo - v0.2)
Update Checker (update - v0.4)
Documents (docs - v0.1)
SuperLinks (superlinks - v0.8)
RRD File Cleaner (RRD Cleaner - v0.36)
uptime
Network Tools (tools - v0.3)
PHP Network Managing (PHP Network Managing - v0.6.1)
IP subnet Calculator IPv4 IPv6 (ipsubnet - v.4d)
PHP Network Weathermap (weathermap - v0.96a)

Again, everything else works perfectly that I have tested except this one item. Disk and other graphs are working fine on the same hosts. The query being used here is the standard SNMP interface statistics.

Thanks!
Danny

[BSOD2600]

What exactly is the problem? Cacti is able to retrieve the interface info and in/out stats for the VMware Accelerated AMD PCNet Adapter.

When you followed the debugging guide, did it reveal any other issues?

[danwdoo]

Cacti won't draw the graph for the interface statistics. It will draw the graph for CPU, Disk and everything else I've tried for these hosts but the network always shows the red x graph placeholders. I see rrd files for other things I am graphing but not for the network so it seems like it is pulling the data but not storing it. There are no errors in the log so I'm not sure what is going on.

I checked the poller cache and see the following entry for the network:

Code: Select all

 SNMP Version: 1, Community: <blanked>, OID: .1.3.6.1.2.1.2.2.1.16.65539  
 RRD: C:\Inetpub\wwwroot\cacti\rra\mail_traffic_in_542.rrd  

The listed RRD is never created however. All the entires for hosts with this adapter appear the same way and also do not create the rrd files. Rebuilding the poller cache does nothing and this affects new and existing hosts (that are created or moved to VMWare).

Thanks!

[BSOD2600]

Alright so taking that specific example of vmware interface traffic, look through the cacti.log file (in debug mode after the poller has ran) for instances of mail_traffic_in_542.rrd. did cacti create and update this file?

Track the data flow of DS[542] in cacti.log.

[aerogate]

I got same problem.

RRDTool Says:

ERROR: opening 'C:/wamp/www/cacti/rra/2/14.rrd': No such file or directory

[BSOD2600]

I got same problem.

RRDTool Says:

ERROR: opening 'C:/wamp/www/cacti/rra/2/14.rrd': No such file or directory

I doubt you do. Start a new thread after reading http://forums.cacti.net/viewtopic.php?t=11747

[aerogate]

Hi, BSOD2600,

Sorry, I am a new bee of Cacti. I tried installing Cacti under CentOS and didn't get any problem. Now under Windows installation, I can get all other graphs except for network Interface-Traffic (bit/sec). Seemed RRDTool didn't create .rrd file at all. You know why?

Cheers

[danwdoo]

Ok,
I switched the log into debug mode and captured the section in question. Here are the results:

Code: Select all

06/06/2009 02:56:26 PM - CMDPHP: Poller[0] Time: 5.7886 s, Theads: N/A, Hosts: 1  
06/06/2009 02:56:21 PM - POLLER: Poller[0] CACTI2RRD: c:/rrdtool/rrdtool.exe update C:\Inetpub\wwwroot\cacti\rra\<host>_hdd_total_543.rrd --template hdd_used:hdd_total 1244271381:8491102208:19329372160  
06/06/2009 02:56:21 PM - POLLER: Poller[0] CACTI2RRD: c:/rrdtool/rrdtool.exe update C:\Inetpub\wwwroot\cacti\rra\<host>_hdd_total_544.rrd --template hdd_used:hdd_total 1244271381:341835776:536281088  
06/06/2009 02:56:21 PM - POLLER: Poller[0] CACTI2RRD: c:/rrdtool/rrdtool.exe update C:\Inetpub\wwwroot\cacti\rra\<host>_traffic_in_545.rrd --template traffic_out:traffic_in N:3582594456:531153725  
06/06/2009 02:56:21 PM - POLLER: Poller[0] CACTI2RRD: c:/rrdtool/rrdtool.exe create C:/Inetpub/wwwroot/cacti/rra/<host>_traffic_in_545.rrd --step 300 DS:traffic_in:COUNTER:600:0: 1000000000 DS:traffic_out:COUNTER:600:0: 1000000000 RRA:AVERAGE:0.5:1:600 RRA:AVERAGE:0.5:6:700 RRA:AVERAGE:0.5:24:775 RRA:AVERAGE:0.5:288:797 RRA:MIN:0.5:1:600 RRA:MIN:0.5:6:700 RRA:MIN:0.5:24:775 RRA:MIN:0.5:288:797 RRA:MAX:0.5:1:600 RRA:MAX:0.5:6:700 RRA:MAX:0.5:24:775 RRA:MAX:0.5:288:797 RRA:LAST:0.5:1:600 RRA:LAST:0.5:6:700 RRA:LAST:0.5:24:775 RRA:LAST:0.5:288:797  

I'm not sure what to make of this.

[BSOD2600]

danwdoo: Looks like cacti is trying to create the rrd file every polling cycle, which means you do not have the correct NTFS permissions on the \cacti\rrd\ folder, cmd.exe or rrdtool.exe.

Please read my installation guide.

[danwdoo]

I've verified the permissions and they are correct. Remember, the poller is creating other rrd files in the same directory with no problem whatsoever for this and any other host. The only time it has a problem is when monitoring the network traffic settings for a machine that is hosted on a VMWare server. There is something it is choking on when it comes to data from this particular object (VMWare virtual network adapter). Just to verify, I reset the permissions one last time and still see the same behavior so whatever it is, it isn't permissions. I even tagged another object to be graphed on this same server and it began graphing almost immediately without any trouble.

Thanks
Danny

[BSOD2600]

Oh I see it now, its the 'space in the DS' issue -- which has already been patched. Notice the space before the 1000000000 in the rrdtool create command?

Have you installed all the patches for 0.8.7d?
Delete the spaces in the fields for DS[545].

[danwdoo]

Ok,
I had not noticed the patches before. I'll go ahead and install them. I'm unsure of how to do the following however:

Delete the spaces in the fields for DS[545].

How do I go about doing this after I install the patches? Which file so I need to edit?

If this is RRD stuff, can I just delete the graphs and start over once I install the patches?

Thanks!

[BSOD2600]

I'm unsure of how to do the following however:

Delete the spaces in the fields for DS[545].

DS = Data Source number 545 -- which I assume is the one for <host>_traffic_in_545.rrd. In the Maximum Value fields, remove any beginning/trailing spaces.