Pages

Monday, April 25, 2016

NSX multiple VTEPs load balancing

From: Kevin Barrass <kbarrass@vmware.com>
Date: Mittwoch, 24. Februar 2016 10:31
To: Yves Fauser <yfauser@vmware.com>, David Pasek <dpasek@vmware.com>, Emanuele Mazza <emazza@vmware.com>, Dimitri Desmidt <ddesmidt@vmware.com>
Subject: Re: NSX Question

Hi Yves, David.

Short answer is yes VM’s on the same logical switch on a host with two VTEP’s will most likely be balanced across those two VTEPs. Longer answer below :)

When you configure load balance SRCID or SRC MAC. We map dvPorts VM’s are attached to by either dvPort ID or MAC address of VM to one of the dvUplinks on the dvSwitch.
We also statically map each VTEP vmkernel interface to a dvUplink on the dvSwitch. This results in an approximate even split of VM’s across both uplinks.

Each VM will then be encapsulated in VXLAN by the IOChain of the dvUplink the VM is mapped to, the SRC IP of that encapsulation will be the VTEP vmkernel IP address that is mapped to the same dvUplink.
The Local Control Plane (netcpa) will report up to the central control plane this VM dvPort (MAC address) to VTEP mapping.

If a dvUplink on the dvSwitch should fail, all VM’s that were mapped to that dvUplink and the associated VTEP vmkernel interface will be re-mapped to one of the remaining dvUplinks. Also the Local Control Plane agent will report this re-mapping up to the central control plane.

You can view this mapping on the ESXi dataplane from using either esxtop or esxcli as well as on the central control plane as below,:

ESXTOP

 9:12:12am up 13 days 17:46, 492 worlds, 3 VMs, 4 vCPUs; CPU load average: 0.03, 0.03, 0.03

   PORT-ID              USED-BY  TEAM-PNIC DNAME              PKTTX/s  MbTX/s    PKTRX/s  MbRX/s %DRPTX %DRPRX 
  33554433           Management        n/a vSwitch0              0.00    0.00       0.00    0.00   0.00   0.00 
  33554434               vmnic0          - vSwitch0             76.29    0.19      53.41    0.06   0.00   0.00 
  33554435     Shadow of vmnic0        n/a vSwitch0              0.00    0.00       0.00    0.00   0.00   0.00 
  33554436                 vmk0     vmnic0 vSwitch0             76.29    0.19       0.00    0.00   0.00   0.00 
  50331649           Management        n/a DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331650               vmnic3          - DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331651     Shadow of vmnic3        n/a DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331652               vmnic2          - DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331653     Shadow of vmnic2        n/a DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331654                 vmk1     vmnic3 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331655          vdr-vdrPort     vmnic3 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331660 52305:Palo Alto Netw     vmnic2 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331661 37661:Dom-Ubuntu01.e     vmnic3 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  50331662 41267:Dom-Windows02.     vmnic3 DvsPortset-0          0.00    0.00       0.00    0.00   0.00   0.00 
  67108865           Management        n/a DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00 
  67108866               vmnic1          - DvsPortset-1          0.00    0.00     129.70    0.25   0.00   0.00 
  67108867     Shadow of vmnic1        n/a DvsPortset-1          0.00    0.00       0.00    0.00   0.00   0.00 


ESXCLI

To view VM to dvUplink mapping

~ # esxcli network vm list
World ID  Name                         Num Ports  Networks       
--------  ---------------------------  ---------  ---------------
   37661  Dom-Ubuntu01                         1  dvportgroup-540
   41267  Dom-Windows02                        1  dvportgroup-585
   52305  Palo_Alto_Networks_NGFW_(1)          1  dvportgroup-520

~ # esxcli network vm port list -w 37661
   Port ID: 50331661
   vSwitch: DSwitch-Res01
   Portgroup: dvportgroup-540
   DVPort ID: 222
   MAC Address: 00:50:56:87:6f:a9
   IP Address: 0.0.0.0
   Team Uplink: vmnic3
   Uplink Port ID: 50331650
   Active Filters: dvfilter-generic-vmware-swsec, vmware-sfw, serviceinstance-1

To View VM to vmkernel VTEP mapping

~ # esxcli network  vswitch dvs vmware vxlan network port list --vds-name=DSwitch-Res01 --vxlan-id=5007
Switch Port ID  VDS Port ID  VMKNIC ID
--------------  -----------  ---------
      50331655  vdrPort              0
      50331661  222                  0
~ # 

~ # esxcli network  vswitch dvs vmware vxlan  vmknic list --vds-name=DSwitch-Res01
Vmknic Name  Switch Port ID  VDS Port ID  Endpoint ID  VLAN ID  IP          Netmask        IP Acquire Timeout  Multicast Group Count  Segment ID
-----------  --------------  -----------  -----------  -------  ----------  -------------  ------------------  ---------------------  ----------
vmk1               50331654  20                     0       10  172.16.1.4  255.255.255.0                   0                      0  172.16.1.0
~ # 


Central Control Plane

htb-1n-eng-dhcp10 # show control-cluster logical-switches mac-table 5007
VNI      MAC               VTEP-IP         Connection-ID
5007     00:50:56:87:6f:a9 172.16.1.4      51           
5007     00:50:56:87:92:b1 172.16.1.5      52           
5007     00:50:56:87:6e:32 172.16.1.3      45   


Please don’t hesitate to contact me if you have anymore questions.

Kind Regards
Kev
Kevin Barrass – VCDX#191
Senior NSX Solutions Architect
Network and Security Business Unit 
+44 (0)7825 034393


From: Yves Fauser <yfauser@vmware.com>
Date: Wednesday, 24 February 2016 08:59
To: David Pasek <dpasek@vmware.com>, Emanuele Mazza <emazza@vmware.com>, Dimitri Desmidt <ddesmidt@vmware.com>, Kevin Barrass <kbarrass@vmware.com>
Subject: Re: NSX Question

{Adding Emanuele, Dimitri and Kev}

Hi David,

I must admit that I don’t know the logic of how VM traffic gets placed onto the different VTEPs in a setup where multiple VTEPs are deployed per ESXi Host.
So I can’t tell you if we pin whole logical switches or individual VMs to the different VTEPs.

I’m sure one of our colleagues I added knows this and will educate me and yourself on it ;-)

Cheers,
Yves

Yves Fauser
Senior Solutions Architect
Mobile: +49 172 254 7415


From: David Pasek <dpasek@vmware.com>
Date: Dienstag, 23. Februar 2016 12:17
To: Yves Fauser <yfauser@vmware.com>
Subject: NSX Question

Hi Yves.

I have a simple question but so far I have get several contradictory answers from different VMware’s NSX experts.

Let’s assume I have NSX with multiple VTEPs (2) per ESXi host. 

Is there a chance that each VM connected to the same logical switch will be load balanced across these two VTEPs?

Of course, it depends on hash algorithm result, but let’s assume hash result is unique for each VM.

Thanks in advance.

David Pasek, Senior Technical Account Manager

No comments:

Post a Comment