Journal of Undergraduate Research
Volume 7, Issue 2 - January/February 2006

Simulative Analysis of QoS in Avionics Networks for Reliably Low Latency

John Wernicke

ABSTRACT

Increasing performance demands from avionics systems requires faster and more reliable backbone networks. These networks must be capable of providing low latency and jitter while maintaining high throughput. In previous work, we developed a framework for analyzing the ability of Gigabit Ethernet switches to provide reliably low latency and jitter using Quality of Service (QoS) controls. In this paper, we use the results of the previous work to develop and validate a simulative model of an Ethernet switch from the QoS perspective. A case study based on avionics networks is constructed and analyzed to determine if a given topology is capable of providing sufficiently low latency and jitter to specific critical streams.

INTRODUCTION

Key applications for future cockpit and cabin avionics systems promise to drive the performance and reliability requirements of their integrated networking infrastructure to ever-increasing levels. Avionics networks are used to connect mission-critical systems and therefore have strict performance requirements [1]. The switches used in these networks must operate at line rate while providing low latency and jitter and offer a robust Quality of Service (QoS) scheme. For example, reliably low jitter is essential for Voice-over IP data streams. The need for higher bandwidth combined with the importance of cost makes Commercial Off-The-Shelf (COTS) networking technologies desirable. One of the most promising network protocols and technologies to provide these capabilities are those of Ethernet. The economy of scale advantages of Ethernet have led to low equipment costs and higher line rates. While Gigabit Ethernet is currently under consideration, 10-Gigabit and 40-Gigabit Ethernet are possible future choices. As the link rates increase, new switches and network interface cards will be required, but the network will remain interoperable with older equipment due to protocol backward compatibility.

In a previous study, we analyzed a number of QoS-enabled Ethernet systems [2]. An experimental framework was formulated to analyze the capabilities of modern COTS Ethernet switches to deliver desired performance levels for critical traffic.

In this study, we use the statistical results of the experimental work to accurately model Ethernet switches using the Mission-Level Designer (MLD) simulation tool for discrete-event simulation from MLDesign Technologies Inc. [3]. This model set can be used as a toolkit to create and analyze specific network topologies from the perspective of latency and jitter performance metrics. Using the simulation toolkit, accurate predictions for the performance of a large network can be studied without the prohibitive costs and time demands of prototyping. Further, the controlled domain of discrete-event simulation permits a large number of performance metrics to be analyzed simultaneously at a variable level of fidelity.

From this toolkit, we build and analyze a case study based on a specific avionics network topology provided by Rockwell Collins. The two most promising switches from the previous study, the BayStack 5510 and the Catalyst 2970, will be compared according to their ability to provide specific high-priority streams with consistently low latency on an end-to-end basis.

The BayStack 5510-48T from Nortel Networks is an inexpensive 48-port Gigabit Ethernet switch. The switch supports several QoS features including 802.1p user priority and DiffServ. The BayStack can also be stacked to provide up to 384 Gigabit Ethernet ports with a total bandwidth of 640 Gbps.

The Catalyst 2970 from Cisco Systems offers 24 ports of inexpensive Gigabit Ethernet. This switch represents the entry-level Gigabit Ethernet switch from Cisco.

The organization of the report is as follows. Section 2 is a brief overview of QoS terminology and standards for Ethernet that will be used in this report. Section 3 describes our simulative design and model construction. Section 4 presents a validation procedure using a sample data collection procedure. Section 5 presents the construction and results of a case study. Finally, section 6 contains the conclusions drawn from the study.

BACKGROUND

Normally, Ethernet provides no performance guarantees and therefore operates on a best-effort basis. Our key interest is in the study of performance-centric network QoS [4]. In this study, the primary focus is to provide reliably low latency to critical data. Performance guarantees in a physical network are traditionally specified probabilistically [4]. The metrics of latency, jitter, and packet loss can be used to analyze the success of a QoS scheme.

In QoS, control mechanisms are specified and implemented to provide performance guarantees. This paper focuses on packet-level QoS, mechanisms such as classifiers, markers, and shapers that improve switch latency for prioritized streams at the expense of less critical data. Classifiers are elements of a system that determine which level of service should be given to a specific packet. Markers use the results of the classifier to mark the packet header permanently to pass the classification to the next switch. Finally, shapers moderate the packet to provide the proper level of service inside the switch.

A number of Ethernet QoS standards have been developed to provide sophisticated control over switching performance. First, a relatively simple system, the IEEE 802.1p standard, was created as part of the IEEE 802.1D [5]. 802.1p uses three bits from the Layer 2 tag to differentiate 8 levels of service.

To provide greater fidelity of control, integrated services (IntServ) was proposed [6]. The IntServ standard requires a packet to schedule a path of appropriate resources using the Resource ReSerVation Protocol (RSVP) before transmission so that guarantees or predictions of service can be made. IntServ was a largely unsuccessful protocol in Ethernet because of the large amount of data that must be maintained in the switch about the current state of each flow and since each intermediate switch on the packet’s path must implement IntServ.

The differentiated services (DiffServ) standard extends the three-bit 802.1p marking to a full byte to provide 64 different classes of services [7]. While the Class of Service field for an 802.1p packet is part of the MAC header, the DiffServ classification uses the Type of Service (ToS) field located in the IP header. In contrast to IntServ, DiffServ stores the QoS information inside the packet header, allowing QoS packets to pass through network components that do not implement QoS.

An even more recent standard is Multi-Protocol Label Switching (MPLS). MPLS is a Layer 3 standard that tags each packet with an MPLS tag that holds a specific routing path [8]. Like integrated services, MPLS switches rely upon a full path of MPLS-enabled machines to provide performance guarantees. To date, relatively few COTS switches implement MPLS due to slow adoption of the specification by end users. Since MPLS is mainly created as an end-to-end dynamic QoS solution, embedded systems often have little use for the level of sophistication of MPLS.

SIMULATION MODELS

This section describes the creation of a general model set that can be used to construct large case studies. First, a generic switch design will be created and verified. Then, parameters for this switch model will be derived and validated to match both COTS switches. Finally, several data sources will be created to support a variety of traffic patterns.

Switch design

The Ethernet/IP switch models were designed to accurately simulate the latency and jitter characteristics of a switch from a QoS perspective. The majority of 802.1p and DiffServ QoS action takes place in ingress and egress ports where classification, marking, and shaping take place. Figure 1 shows the high-level design for the switch. Most COTS switches implement QoS shaping controls as a Weighted-Round-Robin (WRR) queue that prefers higher priority streams as specified by the frame’s 802.1p or packet’s DiffServ value. Since our investigation focuses on the QoS perspective, contention in the physical switch fabric is modeled at a low level of fidelity.

Figure 1. High-level switch design

The first process inside the ingress port is classification. Each ingress port can be configured to trust the 802.1p and DiffServ priorities of the incoming packet or to forward the packet regardless of its claimed priority. Alternatively, a new priority can be assigned based on a number of criteria such as IP destination or source, MAC destination or source, etc.

Figure 2. Ingress port detail

Once the frame has been prioritized, the ingress port maps the frame to the appropriate priority queue. The queue is serviced according to an adjustable algorithm such as WRR or Strict-Priority and then sent to the switch fabric. Figure 2 shows the steps taken by the ingress port to process each frame. Modern switches often perform many of these and other tasks in parallel to reduce the latency.

The egress port is quite similar in function to the ingress port. Since the frame has been previously marked and classified within the switch, the frame is mapped to a priority queue and then sent out of the switch according to another configurable queue algorithm.

Figure 3 shows the top-level diagram of the model for the switch. As in the design, the switch is separated into three distinct components: ingress ports, switch fabric, and egress ports. The Ethernet/IP routing switch fabric permits both unicast and multicast communication to more accurately represent a diversity of traffic patterns.

Figure 3. Switch model

If the port is trusted, input packets are sent to the classification path and then mapped to the appropriate WRR queue. Otherwise, the packet is tagged as untrustworthy and moved to the fabric through the queue which is now operating as a FIFO queue Figure 4 shows the ingress port model from MLD. A small delay is incurred during these processes to reflect the classification and address mapping delays in a switch.

Figure 4. Ingress port model

The switch fabric then forwards the packet to the appropriate output port based on routing and multicast tables that must be created before simulation. The delay and jitter caused by the switch fabric is modeled as a positive-value Poisson distribution to match the experimental data.

The egress port implementation is shown in Figure 5. The packet is checked to determine whether it should be trusted. If the packet is not to be trusted, the packet is forwarded to the WRR queue with the lowest priority so that any trusted high-priority packets are sent out before it. The WRR queue transmits a packet out of the switch whenever the output transmission line is available.

Figure 5. Egress port model

Design parameters

In order to make the switch model capable of modeling the different performance characteristics observed in the experimental study, key attributes that affect timing were parameterized. These parameters include the number of egress and ingress queues, the queue size, and the priority queue weights. The priority queue weights determine the amount of port time devoted to each level of QoS priority. Since often the number of queues in less than the number of possible priorities, each priority is treated as a service class wherein a packet is given equal (FIFO) treatment compared to any other packet in its service class.

Since the performance parameters are crucial to the model’s validity, these parameters were derived from the metrics used in our experimental study. Using the large set of data on latency and jitter characteristics obtained from our previous study [2], the values of these parameters were chosen and adjusted to fit the observed results.

At low line rates, minimum latencies of a stream demonstrate the minimum processing and forwarding latency of which a specific switch is capable. For this report, we define forwarding latency as the time from when a switch finishes processing the packet until the switch places the packet on the output port. Forwarding latency in particular is strongly dependent on frame size both because of the physical line rate of the output port and because of less-than-line-rate fabric switching. Once the minimum values for these quantities are derived, the ability of the switch to serialize data is calculated from the average latencies and jitter observed. Table 1 contains the derived parameters for the BayStack 5510 and the Catalyst 2970.

Data sources

A number of different data sources were also created from MLDesigner libraries to allow a variety of traffic patterns to be studied including bursty, periodic, uniform-distributed, and Poisson-distributed. These data sources can be set to any QoS priority level as desired. Several of these data sources can be combined to represent a single node with multiple processes sending different amounts, types, and priorities of traffic.

MODEL VALIDATION

Once the models were verified to be functionally correct, a simple test system was created to match the topology used in the experimental study [2]. Eight nodes were connected to a single switch in a many-to-one transmission scenario. This scenario effectively models a resource type problem where a large number of nodes require the use of a single resource, in this case a single destination node. The other seven nodes transmit packets of a specified line-rate and packet size. Figure 6 shows the MLD system created to represent this scenario. The green nodes are transmission nodes while the red node is the receiving node.

Figure 6. Validation system

The model’s parameters were adjusted to both of the switches and the system was measured for latency and jitter observed at the receiving node. As in the experimental study, latency was measured from the time when the first bit entered the switch until that same first bit left the switch. The latency and jitter results for both switches BayStack 5510 and the Catalyst 2970 are shown in Table 2 for both 128 and 1518-byte frames.

The simulative latency values for 128-byte frames are less than 1% off the expected values for both switches. The jitter for the 128-byte frames is slightly high, but the difference is at most 180 ns. The 1518-byte latency was slightly high for the Catalyst 2970, while the 1518-byte jitter was slightly low for the BayStack 5510. All simulated values had less than a 10% error, while most had less than 5% error. Thus, the derived performance parameters are quite accurate.

Figures 7 and 8 show scatter graphs of the latencies observed at the receiving node. While these graphs are not always the most informative method of studying the performance of a network, they demonstrate the control that MLD provides over data collection. Using MLD, each packet can be studied individually while most other performance characterization tools like Ixia are limited by the size of the hardware-level capture buffer. Thus, transient traffic phenomena are much easier to capture and examine in detail.

Figure 7. BayStack 5510 1518-byte latencies (ns)

Figure 8. Cisco 2970 128-byte latencies (ns)

CASE STUDY: AIRCRAFT LAN

After the model was validated, a case study was constructed to examine a candidate avionics network architecture provided by Rockwell Collins. The network consists of three switches and a number of subsystems that contain several nodes. Figure 9 shows the basic layout of the network used.

Figure 9. Aircraft LAN system

Each of the subsystems communicates both within itself and with other related subsystems. Many of the sources of data such as the mission processing subsystem periodically transmit data to all other subsystems.

The data sources from the aircraft LAN are listed in Table 3. The values and traffic patterns chosen were increased significantly from the original specification to model a next-generation system. Since in some subsystems different nodes transmit at different rates, the mean rate of subsystems is given as a range in the table. Each node was developed by combining a number of data sources into a unique model.

Aircraft LAN best-effort

First, the aircraft LAN system was simulated assuming that no QoS controls were used to regulate traffic. Table 4 shows the best-effort latencies and jitters of frames sent to each of the traffic groups in the best-effort aircraft LAN system. The names of the subsystems have been abbreviated in Table 4.

The subsystems show mostly low latencies and jitter, with a few notable exceptions. The streams going to the display processing subsystem have both high latencies and jitter. The sensor subsystem in particular sends a large amount of data continuously to the display processing subsystem. The 5510 has slightly lower latencies and jitter than the 2970 in this case. Obviously, the slightly larger minimum latencies of the 2970 combined with the roughly equivalent low-line rate jitter of the two switches makes the BayStack 5510 slightly faster in delivering packets, especially as the number of hops that a packet must travel increases.

The file servers also had considerable latencies and jitter. Since most subsystems communicate with the file server, the file server is unable to guarantee low latencies and jitter to all streams. Although a file server needs to be capable of low-latency responses, generally file server requests do not need to have low latency or jitter.

Finally, the communications subsystem shows moderate latency and jitter. Using QoS mechanisms is unlikely to improve the performance of the streams very much since the stream must still travel two hops. As expected, the choice of topology constrained the ability to provide low latencies for these streams. These streams will be observed, however, to determine the impact of giving an IP multicast priority throughout the network.

Aircraft LAN QoS-enabled

We will now examine the display processing subsystem, which we identified as a system with large latency and jitter. First, we will measure the best effort latencies and jitter of the critical streams entering this subsystem. Figures 10 and 12 show the Best-effort latencies and jitter respectively.

Figure 10. Best-effort stream latencies

Figure 11. QoS-enabled stream latencies

Figures 11 and 13 show latency and jitter for the QoS-enabled streams using a medium priority for the platform monitoring multicast. The latency of the platform monitoring to display processing stream decreases slightly (~2.5us) for both switches. The jitter for this stream also decreases, but by a much larger amount. The platform monitoring stream benefits greatly from these QoS controls.

Figure 12. Best-effort stream jitter

Figure 13. QoS-enabled stream jitter

Despite the benefit, none of the streams can decrease below the per-hop minimum latency, and so the communications-bound stream’s latency does not decrease. Interestingly, the jitter of the Catalyst 2970 increases slightly (by 2us) even though the average latency does not change appreciably. One possible reason is that the higher priority results in some of the packets arriving more quickly than before while the medium priority also requires that lower priority packets are occasionally transferred before the higher priority packet according to the WRR algorithm. Thus, the average latency would stay roughly the same but the jitter would increase. In other words, the mean remains the same but the addition of outliers both above and below the mean increases the standard deviation.

The Catalyst 2970 benefits the most from the QoS controls here, eventually reducing its jitter for the display processing-bound stream beneath the BayStack 5510’s jitter. Since the other two streams show little change in latency and jitter, the topology would need to be changed in order to gain lower latencies and jitter for these streams.

CONCLUSIONS

The simulation models constructed were used to extend previous work to examine performance characteristics of an IP network. By modeling the characteristics of a switch, we were able to create a toolkit capable of prototyping the performance of a specific architecture.

After the model toolkit was developed, parameters were derived from previous work to accurately model two commercial switches. Then, the switches were validated using the experimental framework switch topology and comparing the results with observed values.

Through modeling and experimental analysis, an avionics network was prototyped using two commercial switches. In the best-effort case, the BayStack 5510 was shown to be slightly superior to the Catalyst 2970 in both jitter and latency. However, the Catalyst 2970 benefited from QoS controls more than the BayStack 5510. Thus, using QoS controls allows the Catalyst 2970 to close the slight performance margin that the BayStack 5510 won in the best-effort scenario. By using the simulative toolkit, systems can be quickly and cheaply prototyped.
Further reductions in latency could be realized by using a single mesh topology with all nodes connected to a single switch. However, fault tolerance concerns often dictate that avionics networks have a ring topology which results in high latencies. Future work could investigate and classify these issues in more detail.

---

REFERENCES

1. J. Meier, S. Kim, A. George, and S. Oral, “Gigabit COTS Ethernet Switch Evaluation for Avionics,” Proc. of the 27th Annual IEEE Conference on Local Computer Networks, Nov. 2002, pp. 739-740.
   
2. A. Jacobs, J. Wernicke, S. Oral, B. Gordon, and A. George “Experimental Characterization of QoS in Commercial Ethernet Switches for Statistically Bounded Latency in Aircraft Networks,” Proc. of 29th IEEE Conference on Local Computer Networks (LCN), Tampa, FL, Nov. 16-18, 2004, pp. 190-197.
   
3. G. Schocht, I. Troxel, K. Farhangian, P. Unger, D. Zinn, C. Mick, A. George, and H. Salzwedel, “System-Level Simulation Modeling with MLDesigner,” in 11th IEEE/ACM International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), Orlando, FL, October 2003.
   
4. V. Firoiu, J. Y. Le Boudec, D. Towsley, and Z. L. Zhang, “Theories and Models for Internet Quality of Service,” Proc. of the IEEE, Vol. 90, No. 9, Sept. 2002, pp. 1565-1591.
   
5. IEEE Standard 802.1D, “IEEE Standard for Information Technology—Telecommunications and Information Exchange Between Systems—Local and Metropolitan Area Networks—Common Specifications. Part 3: Media Access Control (MAC) Bridges,” IEEE, 1998.
   
6. R. Braden, D. Clark, and S. Shenker, “Integrated Services in the Internet Architecture: an Overview,” Internet Engineering Task Force (IETF) specification, RFC 1633, Jun. 1994.
   
7. S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, and W. Weiss, “An Architecture for Differentiated Services,” Internet Engineering Task Force (IETF) specification, RFC 2475, Dec. 1998.
   
8. E. Rosen, A. Viswanathan, and R. Callon, “Multiprotocol Label Switching Architecture,” Internet Engineering Task Force (IETF) specification, RFC 3031, Jan. 2001.

---

--top--

Back to the Journal of Undergraduate Research