VMware ESX ethernet link binding with Cisco switches

Something that I get asked a lot is how to bind multiple ethernet links together between an ESX host and a Cisco switch using Cisco Etherchannel (802.3ad).  In reality it’s actually quite simple but does have a small gotcha.  Let’s look at both ends individually:

This is an example config taken from a Cisco switch:

port-channel load-balance src-dst-ip

spanning-tree mode rapid-pvst

interface Port-channel1
description *** ESX Trunk Group 1 ***
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk native vlan 10
spanning-tree portfast trunk
spanning-tree bpdufilter enable
spanning-tree bpduguard enable

interface GigabitEthernet1/0/1
description *** Port 1 in ESX Trunk ***
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk native vlan 10
channel-group 1 mode on
spanning-tree-portfast trunk
spanning-tree bpdufilter enable
spanning-tree bpduguard enable

interface GigabitEthernet1/0/2
description *** Port 2 in ESX Trunk ***
switchport trunk encapsulation dot1q
switchport mode trunk
switchport trunk native vlan 10
channel-group 1 mode on
spanning-tree-portfast trunk
spanning-tree bpdufilter enable
spanning-tree bpduguard enable
As you can see it’s a fairly normal Etherchannel configuration.  The gotcha that I mentioned is the ‘port-channel load-balance src-dst-ip’ line which sets the load blancing mode to src-dst-ip.  The default is src-mac which doesn’t work well with ESX.  In the next section I’ll show you how to configure this from the VMware end.  The other differences are the spanning-tree lines.  As VMware can’t bridge traffic between two physical NICs we need to disable spanning-tree on the ports connected to our ESX servers.  This isn’t totally possible but adding the lines shown will achieve the same result.

On the VMware ESX side we need to assign the trunked NICs to a vSwitch and then open up the properties of the vSwitch.  The example below shows a team of two NICs.

The important thing to check is that your Load Balancing is set to ‘Route based on ip hash’ which will interact correctly with the Cisco switch.

The next thing to do is setup our Vlans correctly and I’ll cover this in another post.