The silent killer in high-performance computing (HPC) workflows is latency. Many complex HPC modeling operations rely on real-time data and continuous, cross-node communication for optimal accuracy and efficiency, but even small data transfer delays can have a large cumulative effect on execution times. In an environment where every millisecond counts, innovative HPC teams are turning to technologies such as RDMA and NVMe for speed and efficiency gains.
This post breaks down the most common sources of HPC latency, and evaluates how modern HPC architectures are leveraging sophisticated methods to overcome them.
Where HPC Latency Comes From
There are many ways in which latency can creep into the high-performance computing process. Below are some of the culprits most HPC teams contend with today.
I/O Bottlenecks
Traditional HPC storage systems leverage hard disk drives (HDDs) vs. solid-state drives (SSDs) for data storage, or sometimes a hybrid of the two. There are, however, a number of challenges with HDDs:
Slower Speeds: HDDs rely on spinning disks and moving read/write heads, which can’t perform data retrieval as quickly as non-moving SSDs.
Failure Potential: The moving parts within HDDs are more susceptible to wear-and-tear from non-stop use, which can lead to unexpected failures.
Fragmentation: Files can become fragmented from continuous movement over time, degrading performance and increasing maintenance requirements.
These traits slow down the input/output capabilities of high-performance storage solutions, creating unwanted HPC latency and bottlenecks.
Metadata Handling
Parallel file systems (PFS) and other distributed file systems can succumb to metadata corruption, overload, and associated delays. With hundreds or thousands of concurrent clients accessing these HPC storage systems, their centralized metadata servers often become overwhelmed with simultaneous requests for metadata operations like file lookups or directory traversals.
Under these circumstances, the metadata server itself can become a chokepoint that creates latency and limits the overall performance of the system. Such metadata delays are the primary impediment to parallel file system performance for high-performance computing.
Congestion & Complexity
Some HPC storage systems depend upon minimal network congestion in order to deliver low-latency data performance. For example, without built-in load balancing, legacy network file systems (NFS) have often been considered less efficient for large file transfers or heavy data loads, especially when network traffic is high. This can lead to latency when used for HPC initiatives.
Other high-performance storage solutions need extensive client-side tuning in order to perform quickly and efficiently, particularly in high-traffic environments. PFS, for example, require specialized clients due to their less standardized structure, which can add to both configuration complexity and performance latency.
Compounding Delays
As alluded to in this post’s introduction, even small delays caused by HPC storage system limitations and/or network congestion can have a compounding effect on multi-node simulations. Considering the importance of continuous data exchange in high-performance computing, these snowballing delays don’t just increase latency — they also hinder system scalability and impact the potential of future HPC projects.
NVMe Speed: Why It Matters for Modern HPC
Non-Volatile Memory Express (NVMe) is a modern interface protocol designed for fast communication and data transfer to and from non-moving storage devices like SSDs. When deployed, NVMe establishes a direct connection between the SSD and the storage system CPU — essentially creating an “express” tunnel that bypasses traditional storage controllers. This allows the protocol to achieve much faster data read/write speeds than both HDDs and traditional SSDs operating without NVMe.
NVMe also allows the full potential of flash-based storage devices to shine. SSDs are inherently parallel devices, capable of handling many I/O requests simultaneously. However, they’re often operated using single-queue protocols such as SATA, which limit the data load to one command at a time. NVMe supports tens of thousands of parallel command queues, each with a high depth of up to 64,000 commands, allowing for significantly more simultaneous I/O operations and maximizing the utilization of SSD storage.
This native parallelism puts NVMe in the right position to support real-time AI, analytics, and simulation workloads. The many benefits of NVMe include:
Faster Speed: By enabling many more commands to be processed at the same time, NVMe dramatically reduces read/write latency.
Improved Efficiency: By using fewer CPU cycles and consuming less power, NVMe optimizes HPC storage system efficiency and costs.
Greater Scalability: By unlocking high levels of parallelism, NVMe creates a high-performance storage solution that can readily scale to meet the demands of enterprise HPC environments.
Accelerating Access with RDMA and NFS-over-RDMA
NVMe goes a long way towards cutting down HPC latency for faster job completion times. But what if these improved processing speeds could be accelerated even further?
RDMA Overview
Remote Direct Memory Access (RDMA) is a networking technology that enables direct memory transfers between servers over a network. A challenge that’s often encountered in high-performance computing is kernel latency — the delay between the initiation of a system event and the operating system’s kernel executing the code to handle that event. RDMA eliminates kernel latency by cutting out CPU involvement and instead using network interface cards (NICs) to directly access and transfer data between server application memory.
This bypass approach frees up CPU resources and allows RDMA to achieve high-speed data transfers from server to server (or to another network device), making it ideal for high-performance computing and other demanding applications. Furthermore, by combining NVMe, which accelerates data transfer between an SSD and a CPU, and RDMA, which accelerates data transfer between a server and another system device, latency is being attacked and reduced from multiple angles.
NFS-over-RDMA
NFS-over-RDMA is a newer high-performance storage technology that combines the standardized network file system protocol with RDMA. By allowing data to be copied directly between the memory of the client and the storage server via RDMA, network file systems shed their performance limitations and become a far more attractive high-performance storage solution capable of achieving ultra-low latency and easily rivaling parallel file system performance.
Benefits
NFS-over-RDMA data storage, backed by the NVMe communication protocol, is unlocking newfound potential for shared storage and optimized read/write I/O balance in large compute environments. Here’s how:
Parallel Performance: NFS-over-RDMA has removed the performance barriers of legacy NFS and greatly accelerated data transfer speeds, allowing enterprises to achieve parallel performance with standard NFS for any high-throughput HPC workload.
Full Scalability: An NFS-over-RDMA system running on VAST’s Disaggregated, Shared-Everything (DASE) architecture avoids the PFS metadata inefficiencies that often slow and inhibit scale.
Simple Management: NFS-over-RDMA systems are much simpler and cost-effective to operate than PFS. They offer standardized client integrations and support with minimal operational lift, and require less day-to-day maintenance and oversight.
How VAST Eliminates Latency Bottlenecks with DASE
The VAST AI Operating System is the foundation upon which leading HPC teams are building and scaling their AI, machine learning, and large compute workloads. Every day, more and more enterprises are switching to NFS-over-RDMA storage powered by VAST Data, and they’re seeing meaningful results.
Disaggregated, Shared-Everything Architecture
VAST’s innovative, proprietary DASE architecture was designed to remove the performance chokepoints that so often hold HPC initiatives back. Here’s how it does exactly that:
Flash-only, metadata-aware design for consistent sub-millisecond latency.
Real-time performance with terabytes per second of data throughput and millions of input/output operations per second (IOPS).
Stateless servers deployed on low-latency Ethernet or InfiniBand NVMe fabrics.
CPU clusters that can support exabyte-scale and 10,000s of processors.
Results That Matter: Faster Jobs, Less Energy, More Research
VAST Data customers, including many of the world’s leading data-driven organizations, are experiencing the benefits of reduced latency with the DASE architecture, such as:
Reduced HPC execution times, enabling more simulations and faster innovation.
Lower power consumption than hybrid disk-based systems, generating ongoing cost savings.
Greater operational flexibility, running AI, simulation, and visualization initiatives from the same platform.
Improved speed of scale, supporting task and market expansion without performance loss.
With the right high-performance storage solution and partner, latency optimization can become a true strategic advantage. NVMe speed, RDMA efficiency, and disaggregated storage flexibility are the next-generation HPC techniques making this a reality for today’s data-driven enterprises. Talk to a VAST Solution Architect today to discuss how to eliminate latency in your own HPC environment.
