VMware PEX: Lab Manager Design and Implementation

VMware PEX: Lab Manager Design and Implementation (TECHMGT0922).  This is written as I sit in the session so it could be messy.

Architecture

• Lab Manager Server (Windows based)
• vCenter Server
• One or more ESX 3.5 or 4.0 servers, ESXi 4 servers
• TCP port 902/903 for virtual machine console access
• Brower client to Lab Manager (LM) server is tcp443 
• TCP 5212 from LM Server to ESX servers and TCP 443 from lm to vCetner
• Read the manual for all the requirments (installer checks for them and spits out errors)
• Install a service account on lm server - page 14-15 of user guide as details on permissions needed
• Pre-crreate resource pools if you want to use them so that it will pick them up at install time
• Pre-create all virtual switches, distributed switches, etc.
• Don't join it to the domain so nothing from AD would get pushed to it as a member server

Storage Design

• LM uses Linked Clones to save disk space
• Make sure the I/O can support your environment (just because you have the space doesn't mean you have enough I/O!)
• LUN locking and limit of 8 nodes if using vmfs (NFS doesn't have this limit)
• Understand the concept of disk chains and how they work (this isn't well documented)

Network Design

• When Setting up LM server, create the default Physical network  
• Design Considerations for Networking: Physical Network vs. Virtual, Fenced vs. Non-Fenced - Will IP's be from an IP Pool, DHCP, or Static?
• Physical network is nothing more than a connection out to a physical network
• Virtual network - a network that is may or may not be connected to a physical network (could be on a different ip, vlan, etc) - A virtual network can be connected to a physical network if needed upon deployment
• Fencing - The ability to create a fence around a configuration (group of vm's) so they are isolated from the rest of the network.  If the fenced configuration needs to get out, it will do so through a NAT router (small Linux vm).  In this case it would have an internal ip inside the fence and an external ip address outside the fence using the NAT router.  This is great for machines and applications that all have the same ip.  This way there will not be ip conflicts on the network.
• Host spanning - fencing isolation was limited to one host in version 3, the ability to cross servers with vMotion is called host spanning.  Host Spanning needs the Distributed Switch (and Enterprise Plus license to get Distributed Switch)
• IP Pools - Takes a lot of ip addresses - if using fencing remember the the NAT router needs ip's as well

Fencing Considerations

• Fancing can't use DHCP (DHCP can't cross the fencing to provide the address)
• IP Static Pool must be used
• At least one virtual machine needs to conntect to physical network (otherwise virtual network with no outside connection)
• Fence policy is traffic In and Out, All Blocked, or Out Only.  There is no In Only policy. 

• Be careful with outside communications if using fencing (many machines with same name but different ip's all hitting an outside source!)
•  Domain COntroller - member servers can be in a configuration with the same name as others as long as the machine pasword with AD doesn't expire (30 days by default).  Otherwise, put a clone of the DC in the configuration and run it private to the configuration group
• Domain Controller Clone - be careful - a cloned dc will come up with a .169 address because it detects one with the same ip address already out there.  Best way to do this is clone the DC and completely isolate it from the production network.
• SQL server - if outside the fence, what happens when multiple configurations hit it??  Maybe different instances of the same database on the same server - adds a little bit of a manual intervention to the process
• Can create a workstation that is inlcuded in the configuration for the user to use as a workstation "in the fence"

Good Article: VMware KB1000023 - How to Backup the VLM Database

VMware PEX: Site Recovery Manger "Up and Running"

This VMware Partner Exchange Session (PEX) was Site Recovery Manger "Up and Running" (TECHBC0321).  I'm writing as I go so this might be a little messy.

This session will focus on the problems typically encountered during SRM implementations.

Required Components
2x vCenter servers
2x SRM Servers
Replication Product from the Storage Vendor
SRA (Storage Array Replication) from the Storage Vendor

Install Workflow
1. vCenter at each site
2. SRM Server (seperate server)
3. SRM needs a DB instance
4. SRA (Often the most complex and causes the most problems) - Install on SRM server


SRA's Function

1. Setup - Query for replicated luns, match luns to vm inventory
2. Failover - Automates promotion of LUNS at remote site, and 
3. Testing - LUN Snapshot creation

What can go wrong during SRA install?

• Not all SRA's are create equal.  Each one is different and have different levels of effort put into the development Some require additional framework (Java JRE for example)  Always read all release notes and the install guide prior to the install attempt
• Always download a fresh SRA FROM THE VMWARE SITE NOT THE VENDOR SITE, many vendors change versions on a frequent basis
• Whatever you do on one site, do it on the other site
• When configuring SRA at the protected site, it may fail if not all components are installed at the recovery site (not configured, just installed)
• What if no datastores appear but the SRA seems to be installed OK?  This is because the datastore doesn't have any vm's on it
• Always verify you have all the needed license features on BOTH storage systems to fully support replication in BOTH directions

Design Considerations

• Disparate networks (re-ip of servers) - Most Common
• Stretch vlans (no re-ip of servers) - Less Common
• DNS services
• Active Directory services - Could be dedicated for testing and failover or same production AD 
• Considered Applications with Hard Coded IP's
• Remember Default Gateway and Subnet Mask
• When performing a recovery, the less changes the better (DOC-1491 in VMware Communities
• SRM Supports RDM's but it isn't recommended
• If using multiple virtual hard disks, make sure both of them are replicated (or exist) at both locations
• SRM does not support replicating virtual machines with snapshots
• Need port 80 https tunnel between sites for site pairing (it is encrypted but travels on port 80 instead of 443 to make security easier
• 150 protection groups / 1000 protected vm's
• A protection group can hold consist of datastores if a virtual machine spans datastores

VMware PEX: Reliable vCenter Database - Operations, Management and Troubleshooting

I  was finally able to attend a VMware Partner Exchange (PEX) session that I was able to discuss.  This session was Reliable vCenter Database - Operations, Management and Troubleshooting (TECHBC0330).  This is being written as I'm in the session so it might be a little messy.

• vCenter at startup takes data from the Windows registry, the vpxd.cfg parameter file, and the vCenter database (VCDB).
• The executable name of the service is vpxd
• vpxd -p and -P are important because they are used to reset the password
• Almost everything you do in vCenter requires interaction with the database.  For example:
• to start a vm - reads location in the db and send commands command
• If vCenter fails - VMotion and DRS will fail but hosts and vm's will continue to run
• vCenter won't start with corrupt or inaccesable database but it will run with an empty database
• HA will be able to execute commands but won't have any "eyes" to see how to execute them

What workload that is running on the vCenter Server will determine how long your environment can be down.  For example a farm containing mainly servers that aren't moving around and won't be restarted could go hours/days without a vCenter Server.  On the other hand in environments with View, SRM, etc. the lack of a vCenter server will be noticed very quickly because machines won't be able to powered on.

Sizing and Location

• You have the option of a physical or virtual machine and you can co-locate the VCDB or put the VCDB on a separate server.
• Recommendations
• vCenter and VCDB - virtual and co-location only to 40hosts or 400 vm's
• if physical - must reatrt db's together, one could take down the other
• The speaker recommended seperate virtual vCenter and db servers with an anti-affinity rule to disable both vm's from being on the same hosts. (I'm not sure how I feel about that in the case of power failures)
• If the database server is virtual, you can take a db backup by cloning the vm

Protecting the vCenter and VCDB's

• Can be protected many ways (too much information to fast to list) but methods included physical rebuild, VMware HA, vCenter Heartbeat, MSCS, and FT

Registry Components

• The DSN Information is held in the registry: HKLM\Software\VMwareInc.\VMware Virtual Center\DB 
• Four objects of value under that: 
• 1 = DSN Name 
• 2 = Login ID 
• 3 = password (encrypted) 
• 4 = driver being used

Database Structure Components

• The VCDB is mainly performance information (over 80% of the database typically)
• The other information is the configuration of accounts and security information for vCenter
• All db tables have the prefix VPX_ - It is NOT recommended to use the tables directly!