The utilization of Single Root I/O Virtualization (SR-IOV) technology enhances network performance by isolating PCI Express (PCIe) resources. A recent exploration by the Red Hat OpenStack Performance and Scale team illustrated the capabilities of SR-IOV with 100GbE NVIDIA ConnectX-6 adapters in a Red Hat OpenShift on Red Hat OpenStack (ShiftonStack) configuration.
Test Methodology
|
Disclaimer:Â This article’s performance results and efficiency metrics are derived from controlled tests and should only act as guidelines. Performance might differ due to hardware, network conditions, and workload characteristics. These results do not represent a Service Level Agreement (SLA). Always conduct comprehensive testing in your environment to validate the suitability and performance of SR-IOV with Red Hat OpenShift on OpenStack for your specific use cases. |
Topology: The diagram below showcases the Dpdk-TestPMD (CNF) loopback setup for the Sriov Shiftonstack network performance test:
Diagram: ShiftonStack DPDK loopback network test setup
Network Setup
The study employed the Crucible traffic generator benchmark utilizing bench-trafficgen (Trex v2.7) tools. These tools are crucial for simulating practical network conditions to enable accurate measurement of network performance across various scenarios.
Hardware Configuration
The tests were carried out on the Intel IceLake compute node with the Dell PowerEdge R650 server, optimized with specific BIOS and kernel parameters according to NFVi recommendations.
Traffic Profiles
Bidirectional traffic flows, with multiple frame sizes and latency-focused measurements, facilitated a comprehensive analysis of network performance using 1K flow/sec.
Configuration Details
To ensure a fair comparison, tests were conducted using identical configurations.
The study concluded that configuring SR-IOV compute nodes with NFV-recommended kernel boot parameters allows for identical hardware synchronization between virtual machines (VMs), resulting in more consistent and higher-performance network operations. These findings highlight the potential of SR-IOV in improving network efficiency within Red Hat OpenShift on OpenStack environments.
For optimal network performance, the Performance Profile Creator (PPC) node tuning operator was applied to the OpenShift SR-IOV Worker VM to boost network efficiency.
In our setup, both SR-IOV Worker VMs and standard SR-IOV VMs were equipped with an identical setup of 8 virtual functions (VFs) to achieve high-speed performance through 8 peering ports for the Trafficgen client. These VFs were systematically distributed and connected across two separate Neutron SR-IOV OpenStack provider networks.
The OpenShift SR-IOV Network Operator was successfully installed on a virtual OpenShift cluster, and the necessary SriovNetworkNodePolicy configurations were set. These policies mapped to two distinct OpenStack SR-IOV Neutron provider network IDs, each associated with a specific resource name and label.
Key Findings: A Performance Powerhouse
Surging Throughput
Upon evaluating the throughput between CNF and VNF based on packet-per-second rates, the results showed consistent performance. The throughput performance of both CNF and VNF was nearly identical, demonstrating efficiency and reliability in terms of packet processing rates.
Chart: Trafficgen rx-mpps & rx-l2gbps throughput distribution (bi-dir. traffic)
Microsecond-Level Latency
Specific latency tests showcased substantial improvements in latency performance for both CNF and VNF. The results displayed consistent round-trip latency performance across various frame sizes.
Chart: Mean and max roundtrip latency distribution (bi-dir. traffic)
Unwavering Stability of Interrupts Handling in Compute Node
Inspecting interrupt handling in pinned isolated VM cores within the compute node indicated consistent performance. Although CNF scenarios showcased slightly higher interrupts due to OpenShift management network interface, they did not impact data-plane performance significantly.
Chart: Sriov compute node interrupts handling
CPU Efficiency of SR-IOV VMs on Compute Nodes
`mpstat` analysis showed consistent CPU performance across dedicated SR-IOV VM cores, supporting diverse workloads effectively while enhancing resource utilization.
Chart: Isolated CPU core performance on SR-IOV compute node for VMs
Analyzing Page-Faults in Compute Node
Monitoring page-faults per second on SR-IOV compute nodes offers crucial insights into memory utilization and system performance optimization.
Chart: Memory Page Faults in Sriov Compute node
Empowering CNFs and VNFs
The deployments of both CNF and VNF indicated significant performance enhancements with SR-IOV PCI passthrough, highlighting its role in boosting network efficiency.
Wrap Up
This research emphasizes the transformative influence of integrating SR-IOV in high-speed network environments. With improved throughput, latency, and stability, organizations can enhance their network capabilities. The study validates the effectiveness of SR-IOV and provides insights for optimizing network performance within Red Hat OpenStack 17.1.2 cloud setups.
Source: Red Hat Blog
