Tech Blog

Had an issue where I’d deleted a VMDK from a LUN and was left with orphaned VM in vSphere, no problem I thought…right click and remove from inventory.  The problem is, all options were greyed out:

So what to do?

Enable SSH on the host.

Connect wit Putty and run:

List all registered VMs

vim-cmd /vmsvc/getallvms

Then to unregistered:

vim-cmd /vmsvc/unregister <id>

Today I noticed a NIC was down on a VM, had a look and noticed it was disconnected:

 So I ticked the box and clicked ok and got the follow error:

Failed to connect virtual device ethernet0.

Weird, so I thought I’d check the port ID for that virtual switch:

Looks like something else is on that port id:

Scroll down the list to find a free port ID and then go back to the VM and change the port ID and boom you should be able to connect.

Tried to increase a disk past 2TB today on a VM on a ESXi 6.0.0 (via vSphere) host but kept getting this:

Hot-extend was invoked with size (6442450944 sectors) >= 2TB. Hot-extend beyond or equal to 2TB is not supported. 
The disk extend operation failed: msg.disklib.INVAL

Well it was driving me nuts until I turned the VM off and tried again…..it worked.

As per the title:

# All hosts connected in vCenter
$scope = Get-VMHost     
#OR specify a cluster
#$scope = Get-Cluster -Name 'Vsphere.Cluster01' | Get-VMHost

foreach ($esx in $scope){
Write-Host "HostName:", $esx
$hbas = Get-VMHostHba -VMHost $esx -Type FibreChannel
foreach ($hba in $hbas){
$wwpn = "{0:x}" -f $hba.PortWorldWideName
Write-Host `t $hba.Device, "|", $hba.model, "|", "World Wide Port Name:" $wwpn
}}

So today a vCenter Server Service wouldnt start Java was going mental and we kept seeing event IDs 1105 and 1827.  So what this means is our SQL Express instance reached the limits the size of databases i.e. 10GB on SQL Express 2008 R2.

You can confirm this by looking at the size of the VIM_VCDB.MDF file in c:\program files\Microsoft SQL Server\MSSQL10_50.VIM_SQLEXP\MSSQL\Data, if its 10GB then your in trouble.

So the first thing to do is fire up Microsoft SQL Server Management studio and run a standard report against the VIM_VCDB by right clicking the DB > Reports > Standard Reports > Disk Usage by Top Tables:

From here you can see which tables are causing the problems, in my case its dbo.VPX_EVENT_ARG which is massive.

So from here you need to purge the above table:

1. From Microsoft SQL Server Management studio
2. Click databases to expand and select VIM_VCDB > Tables.
3. Right-click the dbo.VPX_PARAMETER table and select Open.Note

   : If you are using SQL Server 2008, right-click the dbo.VPX_PARAMETER table and click Edit Top 200 Rows.
   

4. Modify event.maxAge to 30, and modify the event.maxAgeEnabled value to true.
5. Modify task.maxAge to 30, and modify the task.maxAgeEnabled value to true.

1. Next run a built-in stored procedure to shink the logs:
2. Go to VIM_VCDB > Programmability > Stored Procedures.

3. Right-click dbo.cleanup_events_tasks_proc and select Execute Stored Procedure. This purges the data from the vpx_event, vpx_event_arg, and vpx_task tables based on the date specified for maxAge.

Now the records have been purged you need to shrink the DB:

1. As before connect using SQL Server Management Studio.

2. Right click on the VIM_VCDB database and New Query, when the white SQLQuery box opens type the below and click ! Execute

dbcc shrinkdatabase ( VIM_VCDB , 5 )

3. It’ll start running and when its finished you should see:

Check the size of the file again and you should be able to start the services.

So today I had a error on vSphere, specifically related to the HA agent:

On the cluster we had this error:

vCentre Server is unable to find a master vSphere HA agent in cluster

On the hosts we had this error:

vCentre Server is unavble to find the a master vSphere HA Agent is cluster.

Inscufficient resources to satisfy vSphere HA failover level o cluster.

And this is what it all looked like:

So the resolution:

Well what have I changed recently, updated vCentre and ESXi and I’ve also done some networking on the hosts (see my previous blogs).

But I’m not convinced thats what the issue is, so after some basic checks I found I couldnt ping the host names of the ESXi servers from the vCentre server.

After rebooting and clearing the DNS cache on my DC and finally rebooting the vCentre server pings were back, finially I Reconfigured the vSphere HA and bobs your uncle we’re back!!

If this doesnt work though, I suggest:

Check if you can navigate to the hosts via a web gui.

Restarting the management agents on the hosts:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003490

Check the SSL certs on vSphere:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2006210

Remove the host from the cluster and re-add.

My current Citrix XenDesktop environment has out grown its original IP scope and I wanted to move it to a new dedicated VLAN with scope to grow even more. Below is brief guide in the steps taken.
Stage 1 – Creating the DHCP Scope
•    Logged onto your DC and open DHCP, drill down on IPv4, right click and select New Scope.

•    Give the Scope a name “Citrix Guest VM’s” and description.

•    Enter the IP range you would like addresses DHCP to give out (it always a could idea to leave some spare at the end for static IPs)

•    Add any exclusions you wish
•    Lease duration I left default of 8 days
•    Proceed to “Yes, I want to configure these options now”
•    Default Gateway is the IP address you will assign your Core Switch / Router in your network when creating the VLAN in the next stage. 192.168.25.254

•    Next select/enter your DNS servers
•    WINS Server if any
•    Active Scope – Select “Yes, I want to activate this scope now”
•    Finish

Stage 2 – Creating a new VLAN
In my environment we use HP Procurve Switches, so the following config are HP cmds.
•    Log on to your Core Switch / Router and use the follow config.

conf t
vlan 3025
name "Citrix Guest VMs"
ip address 192.168.25.254 255.255.255.0
ip helper-address 192.168.250.2
ip helper-address 192.168.250.2
tagged ?????
exit
wr mem

Config Break down
1 Enter configuration mode
2. Created VLAN 3025
3. Naming the VLAN
4. Creating the default gateway ip address for the VLAN
5 & 6. DHCP Servers
7. Tagging – You need to tagged any ports you would like to traffic to run through, see below.
8. Exit VLAN config
9. Save
Tagging
You need to proceed to create the VLAN and tagged the ports through your network you intend the traffic to run through. In my situation I stated by creating the VLAN on both core switches and had to tagged ports through to my server rack switches and then uplinks to my ESXi Hosts.

Testing
Once all the tagging was done I wanted to do some testing before I moved onto VMware VCentre.
From my core switch I untagged a port on VLAN 3025 (untagged assigns an the device connected to that port the IP in that specified VLAN). Plugged in my laptop and check the address assigned. It was a success I received a address from the 192.168.25.0/24 range, I could double check this by looking at the address leased on the DHCP server.
If you was not successful at this point please go back and review as the stage will not work

Stage 3 – VMware VCentre
•    Adding the Virtual Machine Port Group  (VLAN) onto a vSwitch (virtual switch) in VMware
Open up VMware Client, navigate to the your Cluster
– Select your host (if you have mulitple you will need to do this to all of them)- Select the Configuration tab
– Select Networking from the left side menu
– Click ” Add Networking at the top”

Add Virtual Machine, click next

Select the vSwitch you are currently using, for exmaple mine is “vSwitch0″

Give the Port Group a name, i suggest using the same as the VLAN on the physical switches.
Enter the VLAN ID – 3025

View Summery and Click Finish.

You have now added a new VLAN (port Group) into VMware ESXi and can now been given to VMs, in our case our VDI solution.
To check this, edit any VM which is sitting on any of the host you have just configured, select the NIC and you should be able to drop now the menu to select “Citrix Guest VMs” network.

Now this is great, you are able to change the VM to a new network…… but I have 200 VMs!
It is not feasible to sit there and manually reconfigure and 200 machines and reboot them, plus you would be taking down your environment for a long time.
Stage 4 – VMware vSphere PowerCLI
How to automate the NIC change for multiple VMs
1. Download VMware vSphere PowerCLI

https://my.vmware.com/web/vmware/details?downloadGroup=PCLI550&productId=352

2. Install and connect to you Vcentre Server

Connect-VIServer –Server VCENTER_SERVERNAME

Once Successfully connected you can start issuing powershell commands. Use get-vicommand to see a list of all PowerCLI commands.
To complete this task we just need:
– The list of machines from the XenDesktop machine catalog.
Commands used:

Get-VM: to obtain list of VM
Get-NetworkAdapter: to find list of VMs on VLAN
Set-NetworkAdapter: to change the VLAN

Scenario:
– Machine Catalog Name: “Admin Desktop”
– Machine names: “monvdiad##”
-Current Network: “Server Network”
-New Network: “Citrix Guest VMs”
** Make sure the account you are using have elevated rights in vSphere
PowerCLI Commands to change VLAN
To obtain a list of all XenDekstop VMs in that catalog issue following command use the wildcard display all machines that start with “MONVDIAD*”

Get-vm –Name “monvdiad*”

If the above query returns the list of the correct VM’s then proceed to change the NIC with the code below.

Get-vm -Name “monvdiac*” | Get-NetworkAdapter | where {$_.NetworkName -eq "Server Network"} | Set-NetworkAdapter -NetworkName "Citrix Guest VMs" -Connected: $false | Set-NetworkAdapter -Connected: $true

Press “A” to accept the changes and watch the magic begin.
Once complete I personally would give the affected VM’s a reboot with the following cmd

 Get-VM –Name “monvdiac*” | Restart-VM

Again press “A” to accept
You can check your work by manually check a VM or issue the following cmd which will query and display any VM using “Citrix Guest VMs” Port Group as a selected network.

Get-vm –Name “monvdi*” | Get-NetworkAdapter | where {$_.NetworkName –eq “Citrix Guest VMs”}

Jobs a good un!

So I had a REALLY annoying issue where my keyboard wouldnt work when I was using the console on vSphere (on both vSphere client and the vSphere web gui) on my Windows 8.1 PC (fine on others).

After some internet searches I found that the below Windows Updates are to blame and uninstalling them fixes the issues:

KB2961072
KB2971850
KB2912280
KB2973201
KB2973351
KB2993651
KB2984972
KB3000061
KB3002885
KB2962409
KB2967917
KB2975719
KB2994897
KB2984006

Good luck.

So a client asked me to upgrade their ESXi server from 5.1 to 5.5.  Fairly straight forward if you rememeber this:

1. Update vCenter to 5.5,
2. Update vSphere Update Manager to 5.5,
3. Lastly update ESXi to 5.5.

So lets get going:

Update vCentre:

Firstly download the vCentre ISO: VMware-VIMSetup-all-5.5.0-2183112-20140901-update02.iso (or what ever is the latest version).

Logon to your vCentre server and run the ISO.

If all the components of the existing vCenter Server infrastructure are installed on the same machine, the easiest and fastest upgrade way it to select Simple Install on VMware vCenter 5.5 Installer screen. Remember that upgrading from 5.0 or 5.1 is much easier as SSO will be already installed and the upgrade of SSO will be almost seamless.

It will upgrade all necessary components of vCenter Server, starting with SSO. As you can see, I am upgrading from a version with SSO already installed (i.e. 5.1) so it says it will upgrade it.

After a few checks the upgrade process for SSO will start making sure that SSO users, groups and Lookup Services are migrated. The previous version will be uninstalled and the new one configured:


After Web Client, the next element is Inventory Service. You’ll be asked if you want to keep or overwrite your database:

And now vCenter Server itself will be upgraded:

After asking if the vCenter Server database should be upgraded

the installer will ask you a few other questions like if host agents should be updated automatically once vCenter Server upgrade is done or which account vCenter Server service should run on as well as you will be able to customized port used by vCenter Server. Finally the upgrade starts:

and the whole process completes successfully:

Once the vSphere Client is upgraded, you can check version of your vCenter Server:

Done. Almost everything has been done automatically and we were asked only for necessary information. If you want to have much more control over the upgrade process or if your vCenter Server components are installed on different machines, check the next post in the series when I show how to upgrade them.

Udate vSphere Update Manager:

Run the ISO again and choose Vsphere Update manager:

Tick the download if you have a internet connection:

Type your password for your installation:

Update:

Next through:

The installation will take the service off line so make sure no other Admins are working. Once finished you need to update your client.

vSphere Update Manager client update:

Open the vSphere Client, and log into the vCenter

From Plug-ins menu, select Manage Plug-ins

This should show the plug-in manager, that will show the VMware vSphere Update Manager, and blue link to download and install the plug-in.

At the end of the install, the plug-in will attempt to connect to the VUM server, accept the default certificate that is presented.

Note: This should add an Update Manager icon to the “Solutions and Application” portion of the vSphere Client:

Finally update the host!!

Update the host:

Download a ESXi image from VMware or if you have a HP as I do (HP ProLiant DL385 G7), then you can use the below which has all the HP drivers etc:
http://www8.hp.com/us/en/products/servers/solutions.html?compURI=1499005#tab=TAB4

First put your chosen host into Maintenance mode.

Connect to your vCenter server using vSphere client. Click on Solutions and Applications -> Update Manager. Click on ESXi Images Tab and Select “Import ESXi Image”

Click on Browse and browse towards the ESXi 5.5 ISO image from your system and click on Next

Import ESXi Image wizard will detect the ESXi 5.5 information and click on Next to complete the Import.

Create a New Baseline for ESXi host upgrade.Enter the name ,description and Select Host Upgrade under Host baselines and click on Next

Select the ESXi Image to be included as part of this Upgrade baseline and click on Next.

Review the information for this baseline and click on Finish.

Select the cluster or Host to attach the newly created Host upgrade Baseline. Select the Update Manager tab and click on Attach. Select the checkbox for the baseline to attach to the Host. I am attaching this host upgrade baseline to my ESXi 5.5 host 192.168.1.1. Click on Attach

Click on Scan to check the compliance for the Host against the attached baseline.

If the attached baseline is applicable to the selected Host, Click on Remediate and click on Next.

Accept the End user License Agreement for this ESXi 5.5 upgrade and click on Next.

Verify the Upgrade settings.Select the checkbox if you want to remove the installed third-party software that is incompatible with the upgrade, and continue with the remediation and click on Next.

Enter the Task name and description. If you want to perform this upgrade in later time,Select the checkbox At time and define the time. I am doing it with immediately option. Click on Next.

Specify the Host Remediation option as per your requirement and click on Next.

Specify the cluster Remediation options as per your requirement and click on Next.

Review the selected remediation settings and click on Finish to start the Upgrade process.

Once Upgrade Baseline Remediation is completed, your done!!

Had an annoying error today while trying to add a ESX 4.1 host to DataCenter:

Call “datacenter.queryconnectioninfo” for object “DC or Cluster Name” on vCenter Server “vCenter” failed.

Firstly I thought my password may be wrong for the host, it wasn’t!

Then, maybe my new host is in maintenance mode, it wasn’t!

Finally, and really it should of been obvious to me:

I’d changed the IP address twice while building the box via the console and had forgotten to enter a new default gateway!

And Bam, added the host to the Datacenter!!