Category Archives: Uncategorized

Failing/Failed Disk And Detailed Information From HPACUCLI

Today I was alerted by vCenter with a yellow, “Host Storage Status” alarm.
diskalert

Clicking the ESXi host in the alert I find from the description that I can check the Hardware Status tab of the host for more details
diskalertdetail

I see that the disk is failing and that RAID is in “Interim Recovery”.  This doesn’t give me much information about the failing disk though.  Maybe in HP ILO I can find more information?  Nope, ILO doesn’t seem to have any information about the disks in the host.  According to the ILO System Information, everything looks just fine.
nodiskilo

I called to open a ticket with HP and while on the phone the support technician mentioned a diagnostic tool called HPACUCLI.  If you installed your host a HP customized ESXi image, then this should be there for you.

SSH into your host then type cd opt\hp\hpacucli\bin.  If you type ls you should see the script called “hpacucli”.  Type ./hpacucli and then press “Enter”.  You should see something about detecting controllers and you should be at a new prompt.
hpacucli1

Type controller all diag file=/tmp/adu1.zip and then press “Enter”.
hpacucli2

Now you will need a program like WinSCP that will get the newly created zip file off the host.  Connect to the host with WinSCP and then navigate to the /tmp folder.  Click on the adu1.zip file and the click “Download”.
tmpadu

Looking into the zip file we see four files; double click on “ADUReport.htm”.  The report should open into your browser and should have a lot of information.  Towards the top of my report I see that the drive has failed and if I scroll down I can actually get some in depth information about the drive.  Things like the model and Serial.
Drivefaileddrivefailed2

SRM Testing…What do I do with my Domain Controllers? Part 2

Now that you have successfully prepared for the script…here it is. This script will power down the previous clone if there is one, delete the old clone, create a new clone, change the network, and then power on the clone.  Please make sure you test this in first in a test environment before deploying to production.

RedNotice

 

########################################################################
# Clone DC VM and set network to Test Bubble (clonedc.ps1)
#
# This script is used to clone a DC and then set the network port group # to the #Test Bubble group.
# The purpose of this script is to have a replica of our primary DC for # use #when testing an SRM failover. This will remove the previous clone # before #creating a new one.
#
# Created By: Matt Jones, 2014
#
# Variables: $newVMName – Name of the new clone made from the original VM
# $sourceVMName – Name of the target DC VM that will be cloned
# $VM — Name of VM that was cloned earlier
# $DestGroup — This is the port group that the network will be changed to.
#
# Usage: ./clonedc.ps1
#
#######################################################

add-pssnapin VMware.VimAutomation.Core
Connect-viserver (Your vCenter Server)

#Clone Script#
$newVMName = “DCClone”
$sourceVMName = “SourceDC”

Get-VM $newVMName | Stop-VM -Confirm:$false
Start-Sleep -s 5
Get-VM -Name $newVMName | Remove-VM -DeletePermanently -Confirm:$false
Start-Sleep -s 10
New-VM -Name $newVMName -VM $sourceVMName -Location Foldernamecreatedearlier -vmhost hostname
Start-Sleep -s 120

#Set Test Network#
$VM = get-vm “DCClone”
$DestGroup = “Test Bubble”
$VM | get-networkadapter | set-networkadapter -NetworkName $DestGroup -confirm:$false
Start-Sleep -s 30
$VM | Start-VM

taskkill /F /IM powershell.exe

#######################################################

Once you have created the clonedc.ps1 script we need to get it set up as a scheduled task on the physical machine. Log in using the script AD account that you created earlier, then go to Control Panel –> System and Security –> Administrative Tools –> Task Scheduler.  Click Create Task.

TaskScheduler

TaskCreate

Under the General Tab Name your new task, change this to Run whether user is logged on or not, and check the box Run with highest privileges.  Click the Triggers tab

TaskGeneral

Click New and then choose the frequency that you would like this to run.  Make sure to check the Stop task if it runs longer than box and then set it for 30 minutes.  This makes sure that the script stops and doesn’t continually run.  Make sure you check the Enabled box or the script won’t run automatically!  Click the Actions Tab.

TaskTrigger

Click New and then from the drop down menu choose Start a program.  In the Program/Script box type C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe and in the Add arguments (optional) box type -psc “C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\vim.psc1” -noe -c “. \”C:\scripts\clonedc.ps1\””.  Click the Settings Tab.

TaskAction

If you want to allow this to run on demand then check the Allow task to be run on demand box.  Check the Stop the task if it runs longer than box and choose 1 hour from the drop down.  Check the box If the running task does not end when requested, force it to stop.  Click OK.  You will be prompted to enter the credentials for the account running the task and then click OK.

TaskSettings

RunTaskAccount

 

You might have to do a refresh for your new task to appear.  I am guessing this might be a bug.  Manually run your task and monitor the VIC to make sure it is working properly.

Refresh

That should be it.  Now you can run an SRM test fail over and have a DC available.  Make sure that you have removed all of your DCs from SRM if you had them in there.

Yes

Installing APC Network Shutdown for ESXi – Part 3

In Part 3 we are going to install Powerchute Network Shutdown on the OVA that we deployed, then we are going to configure it to shut down the VMs in case of a problem.

  • See APC pdf FA159776. Open Putty.exe, insert the name or IP of the VMA you just deployed, and then click “Open“. Click “Yes” if you get a security alert. Login with vi-admin and your password that you set earlier.
  • Create a temp directory in opt using the command (You will be prompted for the vi-admin password): sudo mkdir /opt/temp
  •   Next we need to change the permissions to this temp directory: sudo chmod 777 /opt/temp
  • Now to check the permissions: ls -la /opt The permissions should now read drwxrwxrwx
  • Now using WINSCP we need to transfer the .tar.gz file that we downloaded earlier up to the ESXi host. Enter the appropriate information and then click “Login“. Click “Yes” or “Proceed” if prompted with a security warning.
  • Check the “Never show this banner again” box and then click “Continue“. You should now see a screen with two windows. The window on the left is your local computer and the screen on the right is the VMA. Navigate on the left window until you find the .tar.gz file.
  • On the right window the drop down where it says “vi-admin“. Change this to /<root>. Then navigate to “opt–>temp 
  • Drag the .tar.gz file from the left window to the right window. Click “Copy” when prompted.
  • Verify that the file has been copied successfully.
  • Now go back to Putty.exe and we are going to uncompress the file. The commands are: gunzip pcnsname.tar.gz then: tar -xvf pcnsname.tar
  • Use the ls-la command and you should see a new ESXi folder. Use the command cd ESXi to change to this folder.
  • List the contents of ESXi with the ls -la command. We need to change the permissions for the installation file: sudo chmod 777 install_en.sh
    Now do another ls -la to see that the permissions have changed to rwxrwxrwx.
  • Now we are ready to install PCNS. Use the command: sudo ./install_en.sh
    Press “Enter” and then use the “z” key to scroll to the end of the agreement. If you agree then type “yes” and then press “Enter“.
  • Accept the default installation path (or insert a different one if you prefer). Press “Enter“. Type “yes” and “Enter” that you are sure about the path.
  • Take the default for the java directory. Press “Enter“.
  • Next the installation looks for the ESXi host that will be shut down. First add the IP of the host and then it will ask for the username and password for the host to make this change.  Update:  Almost all of the deployments failed to add the ESXi host here, so I would choose “q” to skip and then at the command line do: sudo vifp addserver <hostname/ IP address of ESXi host>
  • Verify that the server has been added with the command: vifp listservers
  • To ensure Powerchute can shutdown the VMs on the host, we need to add the ESXi host to the fasspass. Use the command: vifptarget -s <server name or ipaddress>
    Now type the command: vicfg-nics -l
    You should see a list of nics on the ESXi host.
  • One the server has been added you should be able to open a browser and go to the powerchute configuration wizard: https://vmahostnameorip:6547
  • Click “Next” and you should see the Configuration Wizard: Security page. Insert the username and password and the authentication phrase. This must match the card in your APC device. By default this is apc/apc with the passphrase: “admin user phrase” then click “Next“.
  • On the UPS Electrical Configuration page choose the correct configuration for your company and then click “Next“.
  • On the UPS Details page choose the protocol, port, and IP for the APC network card.
  • On the Miscellaneous page check the box for “Automatically check for updates to PCNS” and then click “Next“.
  • Confirm the details and then click “Apply“.
  • Hopefully you see that the computer is now protected. Click “Next“.
  •  You should now see that the wizard is complete, now click “Finish”.
  • You will now see the main page for the Network Shutdown. Click “Configure Events” and then click the check box for “Shutdown System” on “UPS: On Battery“.
  • The “Shut Down Operating System” page will display and input 300 into the “Shut down the PCNS operating system only when the event lasts this long (seconds)
  • Finally, we need to set up the virtual machine shutdown options on the ESXi host. Open the vSphere Client, select the host, and then choose the “Configuration” tab. Under the “Software” pane click on “Virtual Machine Startup/Shutdown“.
  • In the top right corner click “Properties“. Click the box “Allow virtual machines to start and stop automatically with the system“. Set the shutdown delay (120 default) and then set the shutdown action to “Guest Shutdown“.
  • Leaving VMs under the Manual startup will make it so when the host turns back on, the VMs will not start up by themselves. Usually you want to make sure power is restored and stable before bringing up VMs. You can change your VMs to start automatically if you really wanted to. 

THAT’S IT!!

Installing APC Network Shutdown for ESXi – Part 1

Preparation:

  •  Create a password for the vi-admin account.
  • Download VMA OVA from VMware.
  • Download the Powerchute Network Shutdown for ESXi from www.apc.com. The most current version at the time of writing this is v3.0.1.
  • Download and install putt.exe.
  • Install WinSCP on your local machine. This will be used to put the tar.gz file that you just downloaded from APC on your Esxi host.
  • Make sure the vSphere Client is installed on your machine.

Installation:

  •  On your vCenter server click “File–>Deploy OVF Template“.  
    Choose the location of your ovf. Click “Next“.

  • Verify the details and click “Next“.
  •  Click “Accept” and then “Next“.
  •  Name your VM and then choose the inventory location for the VM. Click “Next“.
  •  Choose the host you wish to deploy to and then click “Next“.
  •  We don’t use resource pools. Select the top level cluster and then click “Next“.
  •  Select the datastore to deploy the VM to and then click “Next“.
  •  I prefer thin provisioning…especially for the VMA’s. Click the “Thin Provisioning” radio button and then click “Next“.
  •  Choose the appropriate source network and destination network and then click “Next“.
  •  Choose the “Fixed” radio button and then click “Next“.
  •  Enter the IP address for the new VMA and then click “Next“.
  •  Verify all of the settings, click the “Power on after deployment” box, and then click “Finish“.
  •  You will see the OVF start to deploy.

Everything look good? Proceed to Part 2

VMWORLD 2012 HERE I COME!!!

I am all signed up and ready to go!! 

Image

Here we go…

I started this blog to document everyday things that I run into for my position as a VMware Administrator.  Hopefully it benefits others and makes someones day a little easier.