In the previous part of this whitepaper, link aggregation techniques in microwave wireless links were discussed. The second chapter of the whitepaper is aimed to emphasize several challenges, which can be seen mainly in the wireless link Layer 2 aggregation.
Challenges
In aggregated links, the link on which to transmit a given frame must be selected. Layer 1 aggregation is the most efficient aggregation technique, which equally splits traffic between aggregated links. Hashing algorithm does not require MAC-MAC, IP-IP or port-port connection diversity.
Nevertheless, the most common aggregation technique used in the majority of Ethernet equipment is Link Aggregation Control Protocol (LACP), which is Layer 2 aggregation. By default, LACP uses a MAC-MAC address hash to distribute different connections among aggregated links. Sending one long frame via one link may take longer than sending several short frames on the other; therefore, some short frames may be received earlier than the long one and the order must be restored at the receiving end. Thereby, in LACP all frames belonging to one conversation are transmitted through the same physical link. This guarantees correct ordering at the receiving end, with no need adding sequencing information to the frames. Take a look at the figure below:
Figure 1 – packet flow for LACP aggregation
Assume the LACP aggregation implemented on the external switch for two parallel microwave links. Red blocks represent the first MAC-MAC connection frames and blue blocks represent the second MAC-MAC connection frames. The LACP hashing algorithm forwards frames of the first MAC-MAC connection to Link 1 and frames of the second connection to Link 2. This depicts a normal Layer 2 (LACP) aggregation scenario for more than one connection, however, several challenges may be faced in such a scenario.
A low number of MAC-MAC connections
A greater number of MAC to MAC address pairs allows more equal traffic distribution between aggregated links and therefore more equal use of the aggregated link’s capacity. Since each particular connection may utilize significantly different throughput, the load on parallel links will be distributed unevenly. Such a drawback will be especially evident with a small number of MAC-MAC connections.
Consider two MAC-MAC connections. Connection 1 utilizes much greater capacity than Connection 2; moreover, it utilizes more capacity than a single microwave link can provide. Figure 2 shows a total input rate of 825Mbps through the 1Gbps link. The LACP aggregation engine splits the traffic between two microwave links, 440Mbps capacity each. On one hand, the total 2+0 wireless link capacity is 880Mbps and it should be capable of handling input traffic with a data rate of 825Mbps. However, the rate of Connection 1 exceeds a single wireless link’s capacity and, since the LACP engine forwards single MAC-MAC to one of the wireless links with a capacity of 440Mbps, part of the data from Connection 1 will be discarded. In other words, Link 1 is overloaded, while Link 2 has enough free capacity to handle rest of the traffic.
Figure 2 – LACP aggregation of two MAC-MAC connections
Consider a second scenario with the same total input data rate of 825Mbps, but eight approximately equal MAC-MAC connections. Figure 3 shows, how in such a scenario the LACP engine is capable of distributing many connections between two wireless links more or less equally.
Figure 3 – LACP aggregation of eight MAC-MAC connections
Scenarios with a constant single MAC-MAC connection are also possible. Consider parallel links with LACP aggregation between two routers. Since the routing algorithm implies adding the router’s Layer 2 header on the egress packets, traffic with only one MAC-MAC connection will be transmitted over the LACP link. This will result in only one aggregated microwave link utilization, while the capacity of the other links will remain unused.
Fortunately, many models of Ethernet equipment allow load balancing not only by source and destination MAC addresses but also provide source and destination IP address hash, as well as source and destination TCP/UDP port number hash. This allows splitting a single MAC-MAC connection into many streams.
Sync-Los events
There is no direct feedback about the wireless link status to the external LACP engine, see Fig. 4. In case a physical link, connected to the LACP switch goes down, the traffic is immediately re-directed from this aggregated link. In case of SyncLos events, LACP will continue sending traffic to an inoperative link, because the Ethernet links between switch and radio will remain UP at both sides. This will result in the loss of all packets on this link. Of course, LACP has its own messaging, by means of sending LACPDUs (Link Aggregation Control Protocol Data Unit) through aggregated links. However, there are fast and slow LACPDUs – first is being exchanged every sec. and second is exchanged with a rate of every 30 sec. In the fast mode, LACP may detect an inoperative logical link and re-direct traffic within a few seconds, and in slow mode, it may take more than one minute. In some middle or low-performance switches only slow mode LACP may be available, which will result in 60 seconds or even longer downtime; in modern networks, it is totally unacceptable.
Figure 4 – LSP in case of SyncLos event
To avoid such a drawback, a Link state propagation (LSP) function is implemented in some models of SAF products. Link state propagation functionality allows the shutting down of specified LAN ports if synchronization loss events occur so that the LACP engine in customer-premises equipment (CPE) can immediately detect an inoperable logical link.
Modulation downshift events
With enabled Adaptive Coded Modulation (ACM) function, modulation downshifts may occur in the wireless link for many reasons (e.g. bad weather conditions). Switching to lower modulations lowers the link capacity and therefore enhances the bottleneck effect in the network. Like in the previous case, the LACP engine on the external switch does not get any feedback about the wireless link modulation status; thus, the external switch sends traffic into the aggregated link without considering its decreased capacity. This will result in a traffic drop in the microwave radio due to packet buffer congestion.
To minimize this shortcoming, Flow Control functionality on the Ethernet link between the switch and the microwave radio is highly recommended. In this case, the radio will share its packet buffer with a packet buffer of a CPE switch. SAF Tehnika microwave radio units are suitable for the external aggregation and support the Flow Control function. For this reason, SAF recommends using switches with sufficient packet buffer size for external (LACP) aggregation purposes.
Out-band management implementation for external aggregation
Since the LACP hash algorithm can randomly forward a single connection to one of the physical links, radio management access cannot be guaranteed through the aggregation link. There is always a chance that the frames addressed to a management interface of a particular radio will be forwarded to a wrong link, therefore making management access to that particular radio unreachable.
For this reason, the external aggregation scenario requires radio management traffic to be displaced beyond the LACP channel. Isolation of the management traffic from the user traffic in the radio is necessary; in other words – out-of-band management (see Fig. 4, where management and user traffic runs on separate cables). Some microwave radio models may already come with a separate port for management access only and a separate port for user traffic. In case the microwave radio does not come with pre-designed out-of-band management, it may be implemented by means of VLANs on a radio with at least two Ethernet interfaces. If advanced Ethernet functionality is available, a bandwidth limitation can be set to a management VLAN, thus limiting the management channel on an appropriate interface. The majority of SAF equipment includes VLAN functionality and some models come with pre-designed out-of-band management.
Conclusion
Taking into account possible challenges of the Layer 2 aggregation, external aggregation becomes a great solution for extending a microwave link’s capacity at a relatively low cost, where built-in aggregation is not available.
