#vmtip Archive From Twitter

For a few weeks now I have been posting VMware and storage related tips to Twitter.  I have been using the hash tag #vmtip for each of them.  I keep an archive of them in Evernote so I can remember what I have done but it isn't organized.  This is simply an attempt to better organize them into categories.  This won't be updated every day, but I will try to keep it somewhat up to date.

Last Update: January 14th, 2010

VMware Related Tips

• #VMware tip: vSphereU1 increases the max# of vm's per host to 160 for up to 8 hosts (was 100), still only 40 vm's per hosts if >8 #vmtip
• #VMware ESXi local boot only supported option today. ESXi Boot from SAN and PXE are both experimental right now (via @DuncanYB) #vmtip
• Need to get data from a #VMware Workstation or ESX(i) vmdk? VMware Disk Mount Utility. It saved me this week! http://bit.ly/6rtY8e #vmtip
  

VMware vCenter Related Tips

• Prior to loading #VMware vCenter, make sure you set the final machine name, static ip address, and domain membership! #vmtip
  

Virtual Machine Alignment Tips

• #VMware tip: Windows 2008 vm's do not need alignment if created fresh. If it was upgraded from W2k03, it will be misaligned. #vmtip
  

VMware Lab Manager Related Tips

• #VMware Lab Manager Tip: If using VMFS, there is a maximum of 8 hosts per datastore due to disk chains. There is no limit for NFS. #vmtip
• #VMWare Lab Manager Tip: Lab Manager disk chains can not span volumes due to the linked clone technology #vmtip
  

NetApp Related Tips

• #VMware on #NetApp tip: VSC will tweak ESXi installs for NetApp. Great since no NetApp Host Utilities Kit for ESXi #vmtip
• #NetApp on #VMWare vmdk alignment tip: Windows Dynamic Disks, Linux LVM's and Citrix Servers can not be aligned with mbralgin #vmtip

NetApp SMVI Related Tips

• When doing #VMware SRM on #NetApp and using SMVI, you CAN'T take a VMware snapshot as part of the backup! #vmtip
• This appears to be undocumented: #NetApp SMVI backup of Windows virtual machine on IDE disk are not eligible for single file restore #vmtip

NetApp SnapDrive Related Tips

• #VMware on #NetApp: When installing Snap Drive, check the install account is an admin on BOTH the server & filer before install! #vmtip
• #NetApp SnapDrive 6.2 for Windows requires .NET 3.5 SP1 and 3 MS hotfixes (with reboot) BEFORE installation of SnapDrive. #vmtip

Designing VMware on NetApp for Dedupe, SMVI, and SnapMirror

I have done a fair amount of design work around VMware solutions on NetApp storage. As many already know, I love the NetApp product line and the added value around virtualization. The down side is getting all of the products to inter operate with each other from a design stand point. The most common solution I see lately is new vSphere installations where customers are requesting NetApp deduplication for better space utilization, SnapManager for Virtual Infrastructure (SMVI) to protect the virtual machines, and SnapMirror for replication to another NetApp storage system at a remote site. Typically my installs have been NFS, but all of this could easily be done on block based storage as well.

I'm not giving away any trade secrets here, all of this can be found in the NetApp TR's for the products if you dig deep enough.

Based on what I stated above, here is some advice for designing the solution:

• Do not create a volume larger than the dedupe limits for the system (1TB for 2020, 2TB for 2050, etc.) NetApp TR-3505 has this information.  If you set autogrow for the volume, make sure it does not grow past this limit!
  
• I recommend against setting autogrow unless you understand two things: 1. don't autogrow past the volume dedupe listed above and 2. autogrow may break SnapMirror.  If you autogrow the volume at the primary site, the destination site must also be grown.   See NetApp TR-3446 for more information.  My preference is to disable autogrow on SnapMirror volumes and grow both source and destination volumes manually as needed.
• As stated in many of the TR's, you will want to put the virtual machine vSwap files on a volume that isn't replicated with SnapMirror or protected with SMVI.  This is set per vSphere host.
  
• Give careful consideration to the placement of the virtual machine page file.  You have many choices with this one but be VERY careful if you plan to implement VMware SRM.  Read NetApp TR-3671 for more details.   For SRM you will need to SnapMirror the page file or jump through some painful hoops to create the page file at the remote location without replicating it.
• If you don't want to replicate the page file, create a small independent persistent virtual disk and put it on a separate volume that doesn't get SnapShots or SnapMirror. See NetApp TR-3737 for more information.
  
• Consider the order for deduplication, SMVI, and SnapMirror.  You want to deduplicate first because dedupe runs on the Active File System, NOT SnapShot blocks.  To maximize dedupe savings, run dedupe first, then  SMVI after dedupe has completed, and lastly use SMVI to kick off the SnapMirror replication
• If you are doing cross site replication (both of your NetApp systems are both source and destinations for SnapMirror), be careful of "deswizzling".  Deswizzling is a SnapMirror post process that runs on the destination volume after a SnapMirror.  It moves the blocks around on the disk to optimize the layout.  This process can be intensive of some systems.  I have seen a 2020 with SATA disks brought to a halt because of this.  The only reference I have seen to deswizzling was in NetApp TR-3561 but it appears to have been pulled from their site.
• If you are using block based storage, you will need to thin provision the volume and the LUN. I recommend using Operations Manager to monitor the free space and send alerts as needed
• The combination of SMVI and SnapMirror only supports one datastore per volume and only one SnapMirror destination is supported
  
• SMVI will update SnapMirror relationships but it will not create them
• SMVI will not backup or restore RDM LUNs or iSCSI software connected LUNs in Microsoft virtual machines
• SMVI can't take VMware snapshots of virtual machines that have iSCSI software LUNs with the Microsoft iSCSI initiator or NPIV RDM LUNs
  
• Use Common Sense: Don't overwork the systems right out of the gate, start small, monitor, and grow

NetApp mbrscan and mbralign for Virtual Machine Alignment In-Depth

Alignment of VMware virtual machines has been an issue for quite some time. This issue exists no matter who is the storage vendor is but I will use NetApp because it is what I know. Here are some links to get you up to speed in case you don't fully understand the situation:

Link to VMWare document on alignment
Link to NetApp document on alignment

What has NetApp done about the situation? I hear that NetApp will be releasing a tool that plugs into vCenter specifically for vSphere 4.0 and vSpherei (ESXi) 4.0 shortly. In the meantime, if you are still on ESX 3.5 (or vSphere with a Service Console) there is another answer that you can use today. This will not work on ESXi since you need a service console for the tools.

Eric Forgette at NetApp created a set of tools about a year ago that has matured and found its way into the NetApp VMware Host Utilities Kit version 5.1. If you do not have this loaded on your ESX/vSphere servers and you are connecting to NetApp, go load it NOW! (A reboot is required for the settings to take affect) Most of the following is research I have conducted in conjunction with conversations directly with Eric over at this thread on NetApp Communities.

Included in this tool are two executables, mbrscan (scans a vmdk for alignment) and mbralign (performs the alignment). The default installation location is /opt/netapp/santools.

While the readme does a good job of going over the basics, there are a number of caveats to run the tools correctly. I will go into each executable in depth but before I go down into the weeds, you need to know when NOT to run it!

You can not or do not want to use the alignment tool for the following:

• Windows 2008 Server is aligned if the machine was created as a Windows 2008 server. If the machine is upgraded from Windows Server 20003, it will not be aligned.
• Citrix Servers are not supported because they remap the c:\
• Windows Dynamic Disks are not supported and will be corrupted if an alignment is performed (but mbrscan will detect them - see below)
  
• Linux LVM volumes are not supported (mbrscan may NOT detect all LVM partitions)
• Windows Server 2003 non-boot disks that have been added (d:, e:, etc) will need to be remapped in Computer Management. The drive letter will be lost on alignment.
• GRUB booted Linux and Solaris will need to have GRUB reinstalled after alignment

With all that out of the way, it is a basic two step process: 1. run mbrscan, 2. run mbralign on machines as needed.

Step #1 - mbrscan

• In order for the mbrscan to give reliable results, the machine must either be powered off or have a VMware snapshot!
• I have a simple script that I put together that just takes a VMware snapshot on all machines on an ESX/vSphere host.
• I then execute mbrscan using the scan all virtual machines parameter: mbrscan --all
• After I have the scan results I need I execute another script to remove the VMware snapshots I just created for all machines on the host
• NOTE: Windows Dynamic Disks will report a partition type of: unknown - 0x42. Do Not Align These Partitions!

Step #2 - mbralign

• In order to perform an alignment, ALL VMware snapshots MUST be removed!
• In order to perform the alignment, you NEED an amount of free space equal to the size of the vmdk. mbralign will make a backup of the file as the first step. This file have an extension: -mbralign-backup.
• In order to perform the alignment, the virtual machine MUST be powered off!
• If the virtual machine has multiple vmdk's, only one can be aligned at a time! 
  
• Execute mbralign against the vmdk - I usually get about 1-2GB per minute speed
  
• Boot up the virtual machine. If it works, delete the -mbralign-backup file
• On Windows systems it will ask you to reboot one more time because it detects the hard disks as new hard disks
  
• If it doesn't work, run mbralign again and it will detect the -mbralign-backup and ask you if you would like to restore the file. Very Nice!