Designing VMware on NetApp for Dedupe, SMVI, and SnapMirror

I have done a fair amount of design work around VMware solutions on NetApp storage. As many already know, I love the NetApp product line and the added value around virtualization. The down side is getting all of the products to inter operate with each other from a design stand point. The most common solution I see lately is new vSphere installations where customers are requesting NetApp deduplication for better space utilization, SnapManager for Virtual Infrastructure (SMVI) to protect the virtual machines, and SnapMirror for replication to another NetApp storage system at a remote site. Typically my installs have been NFS, but all of this could easily be done on block based storage as well.

I'm not giving away any trade secrets here, all of this can be found in the NetApp TR's for the products if you dig deep enough.

Based on what I stated above, here is some advice for designing the solution:

• Do not create a volume larger than the dedupe limits for the system (1TB for 2020, 2TB for 2050, etc.) NetApp TR-3505 has this information.  If you set autogrow for the volume, make sure it does not grow past this limit!
  
• I recommend against setting autogrow unless you understand two things: 1. don't autogrow past the volume dedupe listed above and 2. autogrow may break SnapMirror.  If you autogrow the volume at the primary site, the destination site must also be grown.   See NetApp TR-3446 for more information.  My preference is to disable autogrow on SnapMirror volumes and grow both source and destination volumes manually as needed.
• As stated in many of the TR's, you will want to put the virtual machine vSwap files on a volume that isn't replicated with SnapMirror or protected with SMVI.  This is set per vSphere host.
  
• Give careful consideration to the placement of the virtual machine page file.  You have many choices with this one but be VERY careful if you plan to implement VMware SRM.  Read NetApp TR-3671 for more details.   For SRM you will need to SnapMirror the page file or jump through some painful hoops to create the page file at the remote location without replicating it.
• If you don't want to replicate the page file, create a small independent persistent virtual disk and put it on a separate volume that doesn't get SnapShots or SnapMirror. See NetApp TR-3737 for more information.
  
• Consider the order for deduplication, SMVI, and SnapMirror.  You want to deduplicate first because dedupe runs on the Active File System, NOT SnapShot blocks.  To maximize dedupe savings, run dedupe first, then  SMVI after dedupe has completed, and lastly use SMVI to kick off the SnapMirror replication
• If you are doing cross site replication (both of your NetApp systems are both source and destinations for SnapMirror), be careful of "deswizzling".  Deswizzling is a SnapMirror post process that runs on the destination volume after a SnapMirror.  It moves the blocks around on the disk to optimize the layout.  This process can be intensive of some systems.  I have seen a 2020 with SATA disks brought to a halt because of this.  The only reference I have seen to deswizzling was in NetApp TR-3561 but it appears to have been pulled from their site.
• If you are using block based storage, you will need to thin provision the volume and the LUN. I recommend using Operations Manager to monitor the free space and send alerts as needed
• The combination of SMVI and SnapMirror only supports one datastore per volume and only one SnapMirror destination is supported
  
• SMVI will update SnapMirror relationships but it will not create them
• SMVI will not backup or restore RDM LUNs or iSCSI software connected LUNs in Microsoft virtual machines
• SMVI can't take VMware snapshots of virtual machines that have iSCSI software LUNs with the Microsoft iSCSI initiator or NPIV RDM LUNs
  
• Use Common Sense: Don't overwork the systems right out of the gate, start small, monitor, and grow