Monitor Vmware ESX

[blak111]

Hi Guys,

Can I ask why you are trying to use SNMP to gather the interface stats? I have configured my cacti to use a script written by OP5 which utilizes the VMWare Client SDK kit to poll the vcentre servers or esx servers. The script then pulls the info it needs and graphs it in cacti.

I have this running on 60+ machines.

Any questions, be glad to help.

Lee

Can you post a quick guide on how you set it up?

Thanks

[ldjones48]

Hi Guys,

Can I ask why you are trying to use SNMP to gather the interface stats? I have configured my cacti to use a script written by OP5 which utilizes the VMWare Client SDK kit to poll the vcentre servers or esx servers. The script then pulls the info it needs and graphs it in cacti.

I have this running on 60+ machines.

Any questions, be glad to help.

Lee

Can you post a quick guide on how you set it up?

Thanks

Hi Blak111,

I need to do a write up for work so I will post what I did here afterwards.

Lee

[Hyperlord]

@ldjones48: Uuuuuooooooah!! Awesome! Can't wait

Thanks in advance!

[skneo]

nice to read the topic is not death

[ldjones48]

Hi Guys.

This is a long one, it took me around three to four days of searching the web, understanding how the cacti could interact with the vCentre server and poll the VM’s for the data. I then had to figure out how cacti could take this data and graph it.

The main thing to keep in mind is that cacti normally uses SNMP to retrieve data from remote hosts and graph them. You have to make a para-dime shift when thinking about data retrieval from the vCentre servers in that it uses SSL connections over HTTPS.

To start with you need to download the following zip file (scripts (attached to post)) which contains the following two files

esxiograph.sh
check_esx3.pl

These files are the core of how the data is retrieved from the vCentre infrastructure. Firstly the check_esx3.pl pearl script will be called from the esxiograph.sh script using the details specified in a certain format.

These two files will need to be uploaded to your cacti server and placed in the folder “<cacti_path>/scripts” mine was in “/var/www/cacti/scripts/”. Once they have been uploaded you will need to edit the esxiograph.sh to change where it looks for the check_esx3.pl.

nano <cacti_path>/scripts/esxiograph.sh

If you scroll down to the bottom you will see a section referencing the check_esx3.pl script, this needs to be changed to where we just placed the file as this was written for another program.

Before

io_vm)
type=io_vm
io_vm_all=`perl /usr/share/cacti/site/scripts/check_esx3.pl -H $2 -N $3 -u $4 -p $5 -l IO`
check_io_vm
;;
cpu_vm)
type=cpu_vm
cpu_vm_all=`perl /usr/share/cacti/site/scripts/check_esx3.pl -H $2 -N $3 -u $4 -p $5 -l CPU`
check_cpu_vm
;;
mem_vm)
type=mem_vm
mem_vm_all=`perl /usr/share/cacti/site/scripts/check_esx3.pl -H $2 -N $3 -u $4 -p $5 -l MEM`
check_mem_vm
;;
net_vm)
type=net_vm
net_vm_all=`perl /usr/share/cacti/site/scripts/check_esx3.pl -H $2 -N $3 -u $4 -p $5 -l NET`
check_net_vm
;;
io_vs)
type=io_vs
io_vs_all=`perl /usr/share/cacti/site/scripts/check_esx3.pl -H $2 -u $3 -p $4 -l IO`
check_io_vs
;;
cpu_vs)
type=cpu_vs
cpu_vs_all=`perl /usr/share/cacti/site/scripts/check_esx3.pl -H $2 -u $3 -p $4 -l CPU`
check_cpu_vs
;;
mem_vs)
type=mem_vs
mem_vs_all=`perl /usr/share/cacti/site/scripts/check_esx3.pl -H $2 -u $3 -p $4 -l MEM`
check_mem_vs
;;
net_vs)
type=net_vs
net_vs_all=`perl /usr/share/cacti/site/scripts/check_esx3.pl -H $2 -u $3 -p $4 -l NET`
check_net_vs
;;

After

io_vm)
type=io_vm
io_vm_all=`perl <cacti_path>/scripts/check_esx3.pl -H $2 -N $3 -u $4 -p $5 -l IO`
check_io_vm
;;
cpu_vm)
type=cpu_vm
cpu_vm_all=`perl <cacti_path>/scripts/check_esx3.pl -H $2 -N $3 -u $4 -p $5 -l CPU`
check_cpu_vm
;;
mem_vm)
type=mem_vm
mem_vm_all=`perl <cacti_path>/scripts/check_esx3.pl-H $2 -N $3 -u $4 -p $5 -l MEM`
check_mem_vm
;;
net_vm)
type=net_vm
net_vm_all=`perl <cacti_path>/scripts/check_esx3.pl -H $2 -N $3 -u $4 -p $5 -l NET`
check_net_vm
;;
io_vs)
type=io_vs
io_vs_all=`perl <cacti_path>/scripts/check_esx3.pl -H $2 -u $3 -p $4 -l IO`
check_io_vs
;;
cpu_vs)
type=cpu_vs
cpu_vs_all=`perl <cacti_path>/scripts/check_esx3.pl -H $2 -u $3 -p $4 -l CPU`
check_cpu_vs
;;
mem_vs)
type=mem_vs
mem_vs_all=`perl <cacti_path>/scripts/check_esx3.pl -H $2 -u $3 -p $4 -l MEM`
check_mem_vs
;;
net_vs)
type=net_vs
net_vs_all=`perl <cacti_path>/scripts/check_esx3.pl -H $2 -u $3 -p $4 -l NET`
check_net_vs
;;
Ctrl-x and save

Next we need to ensure Perl has been installed on the system. Depending on which distribution of Linux you are using you will need to utilize your own version of package management. For this guide I am using Ubuntu and all references will be apt-get.

apt-get install perl

Once Perl has been installed onto your system, you will need to download the VMWare-vSphere-SDK-for-Perl files from the following link (http://communities.vmware.com/community ... e_sdk_perl), you will need a vmware.com account to retrieve the files. Then follow the mini-guide below to install the SDK kit.

Download the latest version of the vSphere SDK for Perl package from VMware support page.

In this example we use VMware-vSphere-SDK-for-Perl-4.0.0-161974.i386.tar.gz, but the instructions should apply to newer versions as well.

Upload the file to your nagios server’s /root directory


cd /root
tar xvzf VMware-vSphere-SDK-for-Perl-4.0.0-161974.i386.tar.gz
cd vmware-vsphere-cli-distrib/
./vmware-install.pl
“Creating a new vSphere CLI installer database using the tar4 format.

Installing vSphere CLI.

You must read and accept the vSphere CLI End User License Agreement to continue.
Press enter to display it.”

<ENTER>
“Read through the License Agreement”

“Do you accept? (yes/no)”

yes
“In which directory do you want to install the executable files?
[/usr/bin]”

<ENTER>
“The following Perl modules were found on the system but may be too old to work
with vSphere CLI:

Crypt::SSLeay
Compress::Zlib

The installation of vSphere CLI 4.0.0 build-161974 for Linux
completed successfully. You can decide to remove this software from your system
at any time by invoking the following command:
“/usr/bin/vmware-uninstall-vSphere-CLI.pl”.

Enjoy,

–the VMware team”

Note: “Crypt::SSLeay” and “Compress::Zlib” are not required for check_esx3 to work.

Once the SDK kit has been installed, you will need to create a ‘readonly’ user account on your vSphere system to enable the cacti server to login and retrieve the data. To do this, login to your vCentre server or use the vSphere client to login to the control panel. Once here click on the vcentre server top level and click on permissions. In here you can add your user. If you are in a Windows domain you can add the user in active directory and then add it here. When adding the user only set the permission level to ‘readonly’. Make sure that propagate is ticked so this permission is inherited by all child nodes. On the other hand you could add the permissions on each individual server / host you wish to allow readings from.

Once you have created your user you can use command line code to test that the cacti server can communicate with the vCentre server. This is done with the following commands.

First of all we will test with the check_esx.pl script to ensure that the SDK kit can login with the details and pull the required data.

perl <cacti_path>/scripts/check_esx3.pl -H <vcentre server> -N <name of virtual machine> -u <username> -p “<password>” -l NET

Please ensure that the name of the virtual machine is exactly as the vcentre sees it. In other words it is case sensitive. I have also found that you can not have spaces in your names as this will not poll. If you find any way round this, please let me know.

Once executed you should get some data returned from the server like below.

SSL_connect:before/connect initialization

SSL_connect:SSLv2/v3 write client hello A

SSL_connect:SSLv3 read server hello A

SSL_connect:SSLv3 read server certificate A

SSL_connect:SSLv3 read server certificate request A

SSL_connect:SSLv3 read server done A

SSL_connect:SSLv3 write client certificate A

SSL_connect:SSLv3 write client key exchange A

SSL_connect:SSLv3 write change cipher spec A

SSL_connect:SSLv3 write finished A

SSL_connect:SSLv3 flush data

SSL_connect:SSLv3 read server session ticket A

SSL_connect:SSLv3 read finished A

CHECK_ESX3.PL OK – “virtual machine name” net receive=0.00 KB/s, send=47.43 KB/s | net_receive=0.00KB;; net_send=47.43KB;;

This confirms that the cacti server can login with the credentials specified and it authorized to poll data for that certain machine.

There are several things to remember here, you can either poll the vCentre server or the hyper-visors directly. I have chosen to go with the vCentre server in my environment due to the fact that DRS may start to move machines around for level distribution. This would cause issues later on with graphing if I am directly polling the hyper-visors. On the other hand if you do not have a vCentre server and only hyper-visors this would be perfect. Please also not that the passwords need to be in quotations.

Moving on you will next need to test that the ./esxiograph.sh can parse the data retrieved from the check_esx3.pl and output it in a format that is easily readable by cacti’s graphing system. Please be aware before you test the next step you need to ensure that ‘bc’ is installed on the nagios server. This caused me several hours of headache not knowing why the calculation was not working. Please first install ‘bc’ with the following command.

apt-get install bc

Once this is installed you can go ahead and test the script with the following command

./<cacti_path>/scripts/esxiograph.sh net_vm <vcentre> <vm name> <username> <”password”>

Please note you will need quotation marks around the password again. If all is ok you will get another output, this time in a more cacti friendly output and also in a rate of bits per second.

SSL_connect:before/connect initialization

SSL_connect:SSLv2/v3 write client hello A

SSL_connect:SSLv3 read server hello A

SSL_connect:SSLv3 read server certificate A

SSL_connect:SSLv3 read server done A

SSL_connect:SSLv3 write client key exchange A

SSL_connect:SSLv3 write change cipher spec A

SSL_connect:SSLv3 write finished A

SSL_connect:SSLv3 flush data

SSL_connect:SSLv3 read finished A

read:8000.00 write:16000.00

This should be all you need to do in the CLI side of the installation, next we need to import a template into cacti via the web GUI to allow us to start graphing the data collected.

First of all you need to download the following cacti_host_template_vmware_esxi (attached to this post) file. Extract this to your desktop and then login to your cacti server via the web gui. Once logged in goto the console tab at the top and then goto Import Template under the Import/Export heading. Click choose file and then navigate to the extracted file on your desktop. Then click import.

Once this has imported you will need to add a device. Under the console tab again, goto devices and then add device in the top right. Fill out the description and hostname. From the dropdown menu under host template select ‘VMWare esxi’. You will need to set the downed device detection to PING and make sure all firewalls in between your cacti server and vcentre server allow ping from the cacti IP otherwise the graphs will never be created. You can ignore all other fields and click create.

Once the device has been added goto the console tab and then under heading ‘Create’ click new graph, select your vcentre as the host and you will see 8 options to choose from. The options that start with VM_ are referring to the virtual machines VS_ are referring to the virtual host server (hyper-visor). For this guide I am only concerned with ‘VM Net Load’ select this and click create. You will now have to fill out the options as you did CLI stylie. Populate the vm name (case sensitive), username, and password. Please note the password will need to be in quotation marks such as “*nyy7^%ujl” otherwise it will not work. Click create and the graph will be created.

If after a while you have no graph and receive a error from the RRD tool, I found that the Graph Template for ‘ESXi – VM Net Load’ was using CF Type Last as opposed to Average for the two data sources inbound and outbound. I believe I exported these templates with the changes made but if not you may want to check this.

Any questions, please feel free to ask.

Regards,

Lee

[troffasky]

I've been struggling for days with this too, so thanks for your efforts on this one.

I'm getting

Error: XML: Hash version does not exist.

when I try to import your XML template. Any ideas?


Edit: Hmm, OK looks like my version of cacti [0.8.7e] is older than yours.

[ldjones48]

I've been struggling for days with this too, so thanks for your efforts on this one.

I'm getting

Error: XML: Hash version does not exist.

when I try to import your XML template. Any ideas?


Edit: Hmm, OK looks like my version of cacti [0.8.7e] is older than yours.

Hi Troffasky, I am using version "cacti-0.8.7g"

It looks like you can not import into a version lower than what the version was used to exported the template. In which case if you use the template I have attached to this reply you should be ok. Once you have imported the template you will need to edit the Graph Template "ESXi - VM Net Load". In here you will need to set the CF Type of (esxi_vmnet_read): Inbound and (esxi_vmnet_write): Outbound to Average instead of Last like in the the screenshot below. This should resolve your issues.



Any problems, please let me know.

[troffasky]

Imported that other template, made the changes you suggested to the consolidation function and...yes! This is working. You are an absolute star. I think this deserves a new thread topic of it's own.

So far I've got all the VS parameters, VM CPU, VM IO, VM NET, and VM MEM Usage working. VM MEM Active was showing -nan, so I changed the MAX from 100 to 10000, deleted and recreated the graphs and now it's working.

I am using Cacti 0.8.7e-2 packages on Ubuntu 10.04.2 LTS x86_64, with VMware-vSphere-Perl-SDK-4.1.0.

[ldjones48]

Imported that other template, made the changes you suggested to the consolidation function and...yes! This is working. You are an absolute star. I think this deserves a new thread topic of it's own.

So far I've got all the VS parameters, VM CPU, VM IO, VM NET, and VM MEM Usage working. VM MEM Active is showing -nan at the moment but I'm looking into it.

I am using Cacti 0.8.7e-2 packages on Ubuntu 10.04.2 LTS x86_64, with VMware-vSphere-Perl-SDK-4.1.0.

Hi Troffasky, glad I could help, as this took me a while to sort out I thought it deserved a post as parts of this thread helped me figure out what to do also.

A possible sticky guide?

Lee

[troffasky]

Yes, I think so. There are so many different guides for VMware + Cacti out there - it would be nice to highlight the one that works.

[ironique]

Hi ldjones48,

Have followed your guide, but when I test the check_esx3.pl to see if it returns anything, I get the following:

CHECK_ESX3.PL CRITICAL - Server version unavailable at 'https://<vCenter-IP>:443/sdk/vimService.wsdl' at /usr/share/perl/5.10/VMware/VICommon.pm line 545

Have googled around but cant seem to find a fix for it! Help!?

[ldjones48]

Hi ldjones48,

Have followed your guide, but when I test the check_esx3.pl to see if it returns anything, I get the following:

CHECK_ESX3.PL CRITICAL - Server version unavailable at 'https://<vCenter-IP>:443/sdk/vimService.wsdl' at /usr/share/perl/5.10/VMware/VICommon.pm line 545

Have googled around but cant seem to find a fix for it! Help!?

Hi Ironique,

Do you have any firewalls between the cacti server and your vCentre server? If so are they allowing your cacti source IP?

Lee

[gsaray101]

how does this handle if a new vm is created? do we have to go into cacti and manually add the new vm?

[gsaray101]

is there any template where we logon to vcenter and read through all the vms located in that vcenter?

[ldjones48]

how does this handle if a new vm is created? do we have to go into cacti and manually add the new vm?

Hi gsaray101, your are correct, each time you add a vm you need to add this in the cacti GUI. Thus if you removed one you would need to remove it. The reason being is it creates it own data collector per Virtual Machine not host.