Last week at SC25 in St. Louis, we sat down with Dan Stanzione to talk about the part of the forthcoming Horizon supercomputer that matters just as much as its exaflop-scale potential. What could that be in a field historically dominated by floating point performance and compute capability? Why, IO, of course.
The conversation kept coming back to data movement because that is where Horizon’s architecture breaks from the past because it has to, Stanzione says.
The workloads TACC supports no longer resemble the orderly patterns traditional parallel filesystems were built to serve. They produce billions of small files, unpredictable 4k and 64k reads, constant metadata churn, and users who span every discipline and every style of computation. Horizon needed to meet all of that head on.
We talk about all of this and much more, including power, performance, and codes in the full-length podcast interview here but needless to say, a good deal of our time was spent on the past, present and future of data.
Stanzione has watched the center of these systems strain for years. The old idea that large scientific codes would write clean, regular data structures in large sequential bursts never held up under real workload pressure. And even before the rise of AI, auxiliary scripts and preprocessing layers surrounded every simulation and created swarms of tiny operations.
As Stanzione tells us, with AI-heavy workflows in the mix, patterns became even more irregular. Frontera’s scratch space once reached nearly two billion files, and most of them were small. The filesystem became the single shared surface on which every user could accidentally create a system-wide stall. Lose a compute node and the impact is local.
Overload a metadata server and the entire machine pauses.
That recognition shaped Horizon’s architecture more than any other factor. TACC removed disks from the active data path and built an all solid state tier that jumps directly to tape for archival work. The middle layer, the one that relied on mechanical seeks and rotational latency, no longer had a place in a machine where performance must remain stable even as thousands of users behave in unpredictable ways.
The job was not to chase theoretical peak bandwidth. It was to eliminate the points where the system’s performance could fragment under load.
This led to the decision to adopt VAST as the storage layer after several years of controlled experiments. The surprise was not the headline speeds but the refusal to fail.
TACC had long experience with storage systems they could break on demand. They knew exactly which combinations of operations would overwhelm a metadata server or send latency spikes rippling across the cluster.
As Stanzione details, VAST absorbed those tests without collapsing. It degraded cleanly when pushed to the hardware’s limits and held stable through node failures, switch drops, mixed software versions, and live upgrades. Stanzione called resilience the real differentiator because that is where every other system had faltered.
For Horizon’s users, this changes the core experience of working on a large-scale system.
Every compute node sees the same namespace. Data does not need to be staged or copied into local scratch. AI users who come from cloud workflows, where data must be migrated from object stores to ephemeral working space, find a simpler model.
The data is persistent, shared, and visible from anywhere. Scientific users who expect a POSIX file interface continue to work exactly as they always have. The complexity stays inside the architecture instead of leaking into the workflows it supports.
And while Horizon’s compute story is formidable, with racks that draw more than two hundred kilowatts and clusters of Grace Blackwell GPUs that deliver tens of exaflops in mixed precision, none of that matters if IO cannot sustain the pace.
This is why Horizon dedicates nearly a quarter of its system budget to storage. The machine delivers eight terabytes per second of write performance and sixteen terabytes per second of reads, but the priority is not peak throughput but rather consistency.
Governance pressures reinforce this design he adds. TACC supports controlled data of many kinds, from patient information to sensitive corporate research. Encryption at rest is now the default. The storage layer must separate protected partitions while presenting a coherent space to users who expect the same reliability they would find in a commercial cloud environment.
Horizon therefore marks a pivot point in how large-scale systems are built and the story is not about GPUs alone, it’s about the architecture required to keep the system coherent as AI reshapes scientific workflows, as precision models shift, and as users place new demands on shared resources.
The evolution of compute will continue, Stanzione says. GPU architectures will change. Precision formats will drift toward lower bit depths with software emulation filling the gap. And even quantum hardware may find a role in certain classes of problems he adds.
But the part of Horizon that feels most like the future is the one that carries none of the spectacle, it’s that operating system for AI and connects data with compute that stands between thousands of users and the quiet stalls that used to erode system performance.
It is the architecture built to withstand the way people actually compute.
