Pages

Tuesday, April 9, 2019

vMotion and Virtual Switches

vMotion vSwitche N-VDS VDS VSS

----------------
BUG DESCRIPTION:
----------------
Live migration of a VM from one NVDS to another NVDS was failing and Powering off the VM.


-----------
ROOT CAUSE:
-----------
VC
 thinks LS with same network name across datacenter to be different 
networks and send a deviceBackingChange in the migrateSpec even if the 
vMotion is over same LS.

-------
OUTPUT:
-------
1). Live migration of a VM from NVDS to same NVDS within same Datacenter and across Datacenters is now PASSING.
2). Live migration of a VM from one NVDS to another NVDS within same Datacenter and across Datacenters is now BLOCKING.
3). Live migration of a VM from VSS to NVDS and vice-versa within same Datacenter and across Datacenters is now PASSING.


-----------------------------
BUILDS USED FOR VERIFICATION:
-----------------------------
VC: 12713247 (vSphere67u2)
ESXi: 12698103 (vSphere67u2)


------------------
SCENARIOS COVERED:
------------------

(1). Live migration of VM across VSS, DVS and NVDS(OPN) both in single and across VC/Datacenters.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       Network-Path                          Same-DC       Across-DC       XVC (Across VC)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  VSS1 -> VSS1 (different PG)                 PASS(E1)        NA                NA
  VSS1 -> VSS2 (within same host)             PASS(E1)        NA                NA
  VSS1 -> VSS2 (with different configs)       PASS           PASS              PASS
  VSS1 -> VSS2                                PASS           PASS              PASS
  DVS1 -> DVS1 (same port group)              PASS            NA                NA
  DVS1 -> DVS1 (different port group)         PASS            NA                NA
  DVS1 -> DVS2 (with different configs)       PASS           PASS              PASS
  DVS1 -> DVS2                                PASS           PASS              PASS
  OPN1 -> OPN1                                PASS           PASS              PASS
  OPN1 -> OPN2                                PASS(B1)       PASS(B1)          PASS(B1)
  DVS -> VSS                                  PASS(B1)       PASS(B1)          PASS(B1)
  VSS -> DVS                                  PASS           PASS              PASS
  VSS -> OPN                                  PASS           PASS              PASS
  OPN -> VSS                                  PASS           PASS              PASS
  DVS -> OPN                                  PASS           PASS              PASS
  OPN -> DVS                                  PASS           PASS              PASS
  VSS, DVS, OPN -> OPN, DVS, VSS              PASS           PASS              PASS
  VSS, DVS, OPN -> OPN, OPN, DVS              PASS           PASS              PASS
  VSS, DVS, OPN -> VSS, OPN, DVS              PASS           PASS              PASS
  VSS, DVS, OPN -> OPN, OPN, OPN              PASS           PASS              PASS
  VSS, DVS, OPN -> VSS, OPN, OPN              PASS           PASS              PASS
  VSS, DVS, OPN -> OPN, DVS, OPN              PASS           PASS              PASS
  VSS, DVS, OPN -> DVS, DVS, OPN              PASS           PASS              PASS
  VSS, DVS, OPN -> DVS, OPN, OPN              PASS           PASS              PASS
  VSS, DVS, OPN -> VSS, OPN, VSS              PASS           PASS              PASS
  VSS, DVS, OPN -> OPN, DVS, DVS              PASS           PASS              PASS
  VSS, DVS, OPN -> DVS, OPN, VSS              PASS           PASS              PASS
  VSS, DVS, OPN -> OPN, OPN, VSS              PASS           PASS              PASS
  VSS, DVS, OPN -> DVS, OPN, DVS              PASS           PASS              PASS
  With Disconnected Network Adapter           PASS           PASS              PASS
  Different NVDS with Same Name               PASS           PASS              PASS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

E1 -> Error stack:
Migrating
 VM standalone-cbd6dd057-esx.2-vm.0 to different network without 
changing its host is not supported. Please use Reconfigure API to change
 VM's network.

B1 -> "Currently connected network interface" 
'Network adapter 1' cannot use network 'LogicalNetwork2 
(nsx.LogicalSwitch:00021250-382c-995d-2ae4-56c5c6fbe603)', because "the 
type of the destination network is not supported for vMotion based on 
the source network type".
See KB article 56991 for more details.


(2). Compatibility checks for scenarios where destination switch is without PNIC.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       Network-Path                        Same-DC         XVC (Across VC)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  VSS1 -> VSS2 (without pnics)            BLOCK(CE)         BLOCK(CE)
  VSS1 (without pnics) -> VSS2              PASS               PASS
  VSS -> DVS (without pnics)              BLOCK(CE)         BLOCK(CE)
  VSS (without pnics) -> DVS                PASS               PASS
  DVS1 -> DVS2 (without pnics)            BLOCK(CE)         BLOCK(CE)
  DVS1 (without pnics) -> DVS2              PASS               PASS
  OPN -> VSS (without pnics)              BLOCK(CE)         BLOCK(CE)
  VSS (without pnics) -> OPN                PASS               PASS
  OPN -> DVS (without pnics)                (NE)               (NE)
  DVS (without pnics) -> OPN                PASS               PASS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Compatibility
 Error (CE) -> Currently connected network interface 'device' uses 
network 'network', which is a 'virtual intranet'.

Not Expected (NE) -> Allowing migration without CE. Raised Bug – 2289453


(3). Destination network not accessible cases.
Delete the destination network before migration process starts.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Network-Path              Result
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  VSS -> DVS                PASS(E1)
  OPN -> VSS                PASS(E1)
  OPN -> DVS                PASS(E1)
  VSS -> OPN                PASS(E2)
  DVS -> OPN                PASS(E2)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Task Error (E1) - "currently connected network interface 'device' uses network 'network', which is not accessible."

Task Error (E2) - "A general system error occurred: Invalid fault"


(4). Suspended VM migration cases.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Network-Path              Same-DC      XVC (Across VC)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  VSS -> DVS                PASS             PASS
  VSS -> OPN                PASS             PASS
  DVS -> OPN                PASS             PASS
  OPN -> DVS                PASS             PASS
  OPN -> VSS                PASS             PASS
  DVS -> VSS                (CW)             (CW)
  OPN1 -> OPN2              PASS             PASS
  VSS1 -> VSS2              PASS             PASS
  DVS1 -> DVS2              PASS             PASS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Compatibility
 Warning (CW) -> Network interface" 'Network adapter 1' cannot use 
network 'VM Network', because "the type of the destination network is 
not supported for vMotion based on the source network type.


Datastore capacity expansion issue

TAM recommended to create new additional 4TB datastore, format it to VMFS 6 and use storage migration from old datastore (ibm103-xx) to new datastore (ibm104-xx).
Old datastore (ibm103-xx) will be empty and next week we will do further troubleshooting of old datastore (ibm103-xx) and will try datastore expansion in Host Client (HTML5).
TROUBLE SHOOTING COMMANDS
voma -m vmfs -f check -d /vmfs/devices/disks/naa.00000000000000000000000000000703:3
Running the esxcli storage core device list command shows the size of the extended LUN.
Running the vmkfstools -Ph /vmfs/volumes<datastore name> command shows the original size of datastore
Command partedUtil - Using the partedUtil command line utility on ESXi and ESX (1036609) - https://kb.vmware.com/kb/1036609
The whole runbook procedure is describe in VMware KB: Growing a local datastore from the command line in vSphere ESXi (2002461) - https://kb.vmware.com/kb/2002461