Most of our physicals network adapters are 10Gb adapters, all of these adapters are configured in SEA failover, and these SEA are configured over an Etherchannel adapter in 802.3ad (LACP) mode. Having “sleeping” adapters on the backup VIO Server is on the major drawback of this configuration. 10Gb adapters are expansive and I think It’s a shame not to use it. In most of our case Etherchannels adapters are built over two or four 10Gb physical adapters. Obviously we are NOT using 20 to 40Gb, Etherchannels adapters are used for fault tolerance. Using Shared Ethernet adapter with load sharing mode can help us to save adapters (and switch Ethernet ports), save money, save CPU on VIO Server (the two SEA are working on each VIO Server), and save bandwidth on our networks environment. Here is an example of my current configuration :

• Shared Ethernet Adapter is build over Etherchannel :

# lsdev -type adapter -virtual | grep "Shared Ethernet Adapter"
ent7             Available   Shared Ethernet Adapter
# lsdev -type adapter -virtual | grep "EtherChannel"
ent6             Available   EtherChannel / IEEE 802.3ad Link Aggregation
# lsdev -dev ent7 -attr real_adapter,ha_mode
value

ent6
auto

• Etherchannel is configured in LACP (802.3ad) mode over four 10Gb physical Ethernet ports :

# lsdev -type adapter
[..]
ent0             Available   10 Gb Ethernet-SR PCI Express Dual Port Adapter (771000801410b003)
ent1             Available   10 Gb Ethernet-SR PCI Express Dual Port Adapter (771000801410b003)
ent2             Available   10 Gb Ethernet-SR PCI Express Dual Port Adapter (771000801410b003)
ent3             Available   10 Gb Ethernet-SR PCI Express Dual Port Adapter (771000801410b003)
ent4             Available   Virtual I/O Ethernet Adapter (l-lan)
ent5             Available   Virtual I/O Ethernet Adapter (l-lan)
ent6             Available   EtherChannel / IEEE 802.3ad Link Aggregation
ent7             Available   Shared Ethernet Adapter
ent8             Available   VLAN
[..]
# lsdev -dev ent6 -attr adapter_names,mode
value

ent0,ent1,ent2,ent3
8023ad

ha_mode=auto vs ha_mode=sharing

Before trying to setup an SEA with load sharing my advice is to take a moment to understand differences between a classic SEA failover and an SEA in load sharing mode. Please click on the image below to watch it in bigger size. Here are the most important points to understand :

With the ha_mode set to auto :

SEA is configured over one trunk adapter (802.1q compatible), all accessible vlans are added to this trunk adapter.

All logical partitions are using the the same path. (yellow partitions access network through the yellow VIO Server)

Physical adapter on the blue VIO Server is not used until a failover.

All vlans are identically configured on each trunk adapter, all vlans are accessible trough the same VIO Server. (yellow one, or blue one after a failover)

With ha_mode set to sharing :

SEA is configured over two (or more) trunk adapter (802.1q compatible) (if you are using two trunk adapters on the first SEA, you have to use two trunk adapters on the second, and so on with three or four). All accessible vlans are shared among these trunk adapter. Adapters with same PVID on each VIO Server have to be configured with the exact same VID list (in our case 1122 on one trunk adapter, 1124 on the second).

Each logical partitions on the same vlans are using the same path, partitions on different vlans can use the path on the second VIO Server if the vlan id is present on a different trunk adapter (in our case vlans 1122 and 1124 are using two different path)

Each physical adapter on each VIO Server are used, in our case one for the 1222 vlan, the other for the 1124 vlan (yellow partition with VID 1224 can access network through the yellow VIO Server, blue partition with VID 1122 is going to use the blue VIO Server).

All vlans are identically configured on each trunk adapter WITH THE SAME PVID. Trunk adapters with PVID 999 are configured with ‘additional vlans’ ids set to 1122, Trunk adapters with PVID 998 are configured with ‘additional vlans’ ids set to 1124.

Creating a new load sharing SEA

HMC side

On each VIO Server add three new virtual adapters, one for control channel, the two others for trunk adapters (on theses ones share the vlans by looking the load on each one, HMC lsstat command can help you).

• Control channel adapter on VIO Server 1, choose a PVID not use by one of your vlan :



• Control channel adapter on VIO Server2, choose a PVID not use by one of your vlan :



• First trunk Ethernet adapter on VIO Server1, choose a PVID not use by one of your vlan, check 802.1q compatible adapter box, add vlans in additional VLAN ID box (in our case we have split vlans 1122 and 1124 between the trunk adapters), check Use this adapter for Ethernet Bridging and set the priority to 1 (the priority have to be the same on the second trunk adapter) :



• First trunk Ethernet adapter on VIO Server 2, choose a PVID not use by one of your vlan, check 802.1q compatible adapter box, add vlans in additional VLAN ID box (in our case we have split vlans 1122 and 1124 between the trunk adapters), check Use this adapter for Ethernet Bridging and set the priority to 2 (the priority have to be the same on the second trunk adapter) :



• Second trunk Ethernet adapter on VIO Server 1, choose a PVID not use by one of your vlan, and different from the first PVID used by the first trunk adapter, check 802.1q compatible adapter box, add vlans in additional VLAN ID box (in our case we have split vlans 1122 and 1124 between the trunk adapters), check Use this adapter for Ethernet Bridging and set the priority to 1 (the priority have to be the same on the first trunk adapter) :



• Second trunk Ethernet adapter on VIO Server 2, choose a PVID not use by one of your vlan, and different from the first PVID used by the first trunk adapter, check 802.1q compatible adapter box, add vlans in additional VLAN ID box (in our case we have split vlans 1122 and 1124 between the trunk adapters), check Use this adapter for Ethernet Bridging and set the priority to 2 (the priority have to be the same on the first trunk adapter) :

VIO Server side

All virtual Ethernet adapters are present on all VIO Servers :

• ent1 is the real physical adapter.
• ent4 is the first trunk adapter.
• ent5 is the second trunk adapter.
• ent6 is the control channel adapter.

Begining with the first VIO Server, then the second VIO Server (really important), create the new Shared Ethernet Adapter :

• First VIO Server :

# mkvdev -sea ent1 -vadapter ent4,ent5 -default ent4 -defaultid 999 -attr ha_mode=sharing ctl_chan=ent6
ent7 Available

• Second VIO Server :

# mkvdev -sea ent1 -vadapter ent4,ent5 -default ent4 -defaultid 999 -attr ha_mode=sharing ctl_chan=ent6
ent7 Available

Checking Shared Ethernet Adapter state

You can easily check the configuration by using the lsdev command, all informations about the Shared Ethernet Adapter can be gathered using the entstat, here are a few examples :

• Configuration checking using lsdev command :

lsdev -dev ent7 -attr ctl_chan,pvid_adapter,virt_adapters,real_adapter,pvid,ha_mode
value

ent6
ent4
ent4,ent5
ent1
999
sharing

• Checking state, on first VIO Server, then on the second one, mode are not primary and backup anymore, but primary_sh and backup_sh :

# entstat -all ent7 | grep "State" | grep -v LAN
[..]
    State: PRIMARY_SH
[..]

# entstat -all ent7 | grep "State" | grep -v LAN
[..]
    State: BACKUP_SH
[..]

• Checking active and inactive trunk adapter on each VIO Server. In a normal case, on each VIO Server, one adapter has to be active. (Example below : on VIO Server 1, trunk adapter with vlan id 1222 is active, and trunk adapter with vlan id 1224 is inactive. On the second one trunk adapter with vlan id 1222 is inactive, and trunk adapter with vlan id 1224 is active) :

# entstat -all ent7 |grep -E "Priority|Virtual Adapter|Shared Ethernet Adapter"
Device Type: Shared Ethernet Adapter
Statistics for adapters in the Shared Ethernet Adapter ent7
    Priority: 2
Virtual Adapter: ent4
  Priority: 2  Active: True
Virtual Adapter: ent5
  Priority: 2  Active: False

# entstat -all ent7 |grep -E "Priority|Virtual Adapter|Shared Ethernet Adapter"
Device Type: Shared Ethernet Adapter
Statistics for adapters in the Shared Ethernet Adapter ent7
    Priority: 1
Virtual Adapter: ent4
  Priority: 1  Active: False
Virtual Adapter: ent=5
  Priority: 1  Active: True

• Checking vlans configuration each VIO Server. In our case firsts adapters on each VIO Server are setup with 1122 vlan, and seconds adapters are setup with 1124 vlan (output for this command are the same on each VIO Server) :

# entstat -all ent7 |grep -E "Port VLAN ID|VLAN Tag IDs|VID shared|Virtual Adapter|Shared Ethernet Adapter"
Device Type: Shared Ethernet Adapter
Statistics for adapters in the Shared Ethernet Adapter ent7
    VID shared: 999 1122
Virtual Adapter: ent4
Port VLAN ID:   999
VLAN Tag IDs:  1122
Virtual Adapter: ent5
Port VLAN ID:   998
VLAN Tag IDs:  1124
[..]

Moving from a SEA failover to a load sharing SEA

An existing Shared Ethernet Adapter can be modify from auto to sharing without an outage, however my advice is to be careful with this kind of operation. Follow the steps below in this exact order when you have to move from ha_mode=auto to ha_mode=sharing :

• From the Hardware Management Console with a dlpar operation, add on each VIO Server a new trunk adapter, with an unused PVID, and the list of additional vlan ids (priority has to be set at the same value of the existing trunk adapter used by the Shared Ethernet Adapter).
• Run cfgdev command on each VIO Server, one by one. You can skip this step if you are using VIO Server 2.2.2.1 or later.
• On each VIO Server, add the new trunk adapter to the existing Shared Ethernet Adapter (example below : ent5 is the Shared Ethernet Adapter, ent4 the existing trunk adapter, and ent6 the new added trunk adapter) :

# chdev -dev ent5 -attr virt_adapters=ent4,ent6
# lsdev -dev ent5 -attr virt_adapters,ha_mode
value

ent4,ent6
auto

• On each VIO Server, STARTING BY THE PRIMARY SHARED ETHERNET ADAPTER, change the ha_mode to sharing :

# chdev -dev ent5 -attr ha_mode=sharing
lsdev -dev ent5 -attr virt_adapters,ha_mode
value

ent4,ent6
sharing

• If you want to migrate a vlan from one trunk adapter to another, an outage is required, remove the vlan form the first trunk adapter on each VIO Server, then add the same vlan on the second trunk adapter on each VIO Server.

Impress your network team

Network teams guy are my friends, they do not like to see unused and plunged switch Ethernet port. After a successful configuration of an SEA in sharing mode, call them, and test the failover (always test the SEA) :

• In a normal situation, vlans are shared between the to Ethernet switch port :



• After a failover, all vlans are using the same Ethernet switch port :

Hope this can help.