EtherChannel is a trademark registered by Cisco Systems and is generally called multi-port trunking or link aggregation. If your Ethernet switch device has this function, you can exploit the support provided in AIX. In this case, you must configure your Ethernet switch to create a channel by aggregating a series of Ethernet ports.
On AIX with "smitty etherchannel", in the same menu, depending how the fields are filled out, 2 distinctly separate functioning devices can be created. This can be seen:
ent2 Available EtherChannel / IEEE 802.3ad Link Aggregation
1. EtherChannel (Network Interface Backup, NIB)
2. Link Aggregation (802.3ad)
Network Interface Backup (NIB) can be used to achieve network redundancy. Two adapters are used to create an EtherChannel that consists of one primary adapter and one backup adapter. The interface is defined on the EtherChannel. If the primary adapter becomes unavailable, the Network Interface Backup switches to the backup adapter.
2. Link Aggregation
Link Aggregation is a network port aggregation technology that allows several Ethernet adapters to be aggregated together to form a single pseudo Ethernet adapter. (There are for example 2 primary adapters).
The main benefit of a Link Aggregation is that it has the network bandwidth of all of its adapters. If an adapter fails, the packets are automatically sent on the next available adapter without disruption to existing user connections. A link or adapter failure will lead to a performance degradation, but not a disruption.
For me once in order to work correctly the Link Aggregation (with LACP) I had to set these:
smitty etherchannel --> Change/Show Characteristics...:
Hash Mode src_dst_port
(I used smitty, but chdev works as well)
Etherchannel/Link Aggregation parameters:
standard: this is good for default Etherchannel (primary and backup)
8023ad: this is the LACP (Link Aggregation Control Protocol) configuration (2 primary adapters).
(ad = Autodetect: if our server approaches switch with one adapter, switch sees one adapter, if our server approaches switch with a Link Aggregation, switch auto detects that. For 10Gb, we should be LACP/8023ad.)
default: is by IP address, good fan out for one server to many clients. But will transmit to a given IP peer on only one adapter
src_dst_port: uses source and destination port numbers in hash. Multiple connections between two peers likely hash over different adapters.
(src_dst_port is the best opportunity for multiadapter bandwidth between two peers. IBM recommendation is hash_mode=src_dst_port, it should be verified at switch as well.)
In src_dst_port mode, the TCP/UDP port values are added, then divided by two. The resultant whole number (no decimals) is plugged into the standard algorithm, then TCP/UDP traffic is sent on the adapter selected by the standard algorithm. Non-TCP or UDP traffic will fall back to the default hash mode, using the last byte of either the destination IP address or MAC address.
The src_dst_port hash mode considers both the source and the destination TCP or UDP port values. In this mode, all of the packets in one TCP or UDP connection are sent over a single adapter (!), so they are guaranteed to arrive in order, but the traffic can be still spread out because additional connections (even to the same host) may be sent over on different adapters.
So, a 2x10Gb LACP channel, for 1 connection will utilize only 1 adapter at a time, and there is no guarantee that the next connection will be dispatched to the other adapter. (Theoretically it can happen that in an LACP channel, adapter1 is 100% utilized, adapter2 0% utilized, and next connection is dispatched again to adapter1, which will cause performance issues.)
Address to ping
Not typically used. Aids detection for failover to backup adapter. Needs to be a reliable address, but perhaps not the default gateway. Do not use this on the Link Aggregation, if SEA will be built on top of it. Instead use netaddr attribute on SEA, and put VIO IP address on SEA interface.
How to check which one is configured on the server:
entstat -d <etherchannel device> | head
Etherchannel for Network Interface Backup (primary - standby):
ETHERNET STATISTICS (ent6) :
Device Type: EtherChannel
Hardware Address: 00:11:25:a5:9b:2a
Elapsed Time: 0 days 12 hours 10 minutes 35 seconds
Etherchannel for Link Aggregation (primary - primary):
ETHERNET STATISTICS (ent4) :
Device Type: IEEE 802.3ad Link Aggregation
Hardware Address: 00:11:25:c5:bf:1d
Elapsed Time: 45 days 2 hours 37 minutes 55 seconds
Link Aggregation with NIB:
By using a backup adapter you can add a single additional link to the Link Aggregation, which is connected to a different Ethernet switch. This single link will only be used as a backup.
man ethchan_config manual for configuring etherchannel with commands
smitty etherchannel smitty menu for configuring etherchannel
lsattr -El entX it will show the adapters belongin to the etherchannel (and other useful settings
entstat -d entX | grep Act shows which adapter (Primary or Backup) is the active in the etherchannel (enX)
Creating an EtherChannel:
1. before creating make sure no ip on either adapter
(easiest way: first remove enX/etX/entX before starting to configure, then cfgmgr)
2. when you have clean adapters: smitty etherchannel
- choose the adapters which you want as primary adapter
- add additional adapter if needed a backup adapter
- set mode and hash mode if needed (for Link Aggreagtion once I needed 8023ad and src_dst_port)
Add /Remove and adapter from EtherChannel:
Beginning with the 5200-03, Dynamic Adapter Membership functionality is available. This functionality allows you to add or remove adapters from an EtherChannel without having to disrupt any user connections.
Removing adapter from etherchannel without stopping etherchannel:
smitty etherchannel -> Delete Main/Backup Adapter
/usr/lib/methods/ethchan_config -d <etherchannel physical device> <physical device>
/usr/lib/methods/ethchan_config -d ent4 ent0
Adding adapter to etherchannel:
(make sure no ip is configured, if needed put to detach state)
smitty etherchannel -> Add Main/Backup Adapter
/usr/lib/methods/ethchan_config -a -b <etherchannel physical device> <physical device> <--adding as backup adapter
/usr/lib/methods/ethchan_config -a <etherchannel physical device> <physical device> <--adding as primary adapter
/usr/lib/methods/ethchan_config -a ent4 ent0
(For me once, only command line worked, when smitty has been used for removing, it always said: resource is busy...)
Testing adapter failover:
Primary Adapter: ent0 (connected on VLAN with VIOS1)
Backup Adapter: ent1 (connected on VLAN with VIOS2)
(IOCTL error message during creation of an etherchannel can be ignored)
1. check active channel:
entstat -d ent2| grep Active <--Active channel: primary channel
2. unplug cable from primary channel or testing it with command:
/usr/lib/methods/ethchan_config -f ent2 <--it will do a failover
entstat -d ent2 | grep Active <--Active channel: backup adapter
errpt | head <--ETHERCHANNEL FAILOVER
4. switch back:
/usr/lib/methods/ethchan_config -f ent2 <--it will do a switch again
entstat -d ent2 | grep Active <--Active channel: primary channel
errpt | head <--ETHERCHANNEL RECOVERY