The AI landscape is at an inflection point. While a couple dozen companies have led the charge in developing and training leading AI models, we are now witnessing a shift toward enterprises adopting AI at scale. This transition—from model training to inference and real-world application—requires a new approach to infrastructure: one that balances performance, scalability, and efficiency.
Supermicro has been at the forefront of this transformation, helping businesses move beyond the era of experimentation into full-scale AI deployment. By delivering rack-scale, plug-and-play AI solutions, Supermicro is making AI infrastructure more accessible to enterprises of all sizes. Their ability to integrate and deliver all facets of an AI cluster makes them a one-stop shop for AI innovation.
Fresh off the announcement that Supermicro is ramping full production of rack-scale systems of Supermicro’s NVIDIA HGX B200 8-GPU systems (NVIDIA Blackwell platform) and more than a year after VAST Data and Supermicro formally announced a partnership to simplify and accelerate AI pipelines, we sat down with Ben Lee, Supermicro Director of Solution Architecture, to discuss how AI is reshaping their business, how they are helping enterprises adopt AI at scale, and what the future holds for AI infrastructure.
The Enterprise AI Revolution
John Mao: Supermicro has played a key role in AI infrastructure for years. How has the rise of AI changed your business, and what are the biggest trends you’re seeing?
Ben Lee: AI has fundamentally reshaped how businesses approach compute infrastructure. While hyperscalers initially drove AI advancements—where we’ve had a long journey of tremendous success—we’re now seeing enterprises across many industries rapidly adopting AI for large-scale implementation and operational workloads.
As a result, we’ve evolved beyond just supporting large-scale training clusters. Now, we’re delivering fully integrated, scalable AI solutions that help enterprises accelerate AI adoption.
A few key trends stand out:
Enterprises want end-to-end AI infrastructure—turnkey solutions that simplify deployment, whether on-prem, in hybrid environments, or cloud-adjacent.
AI model evolution is driving demand for more optimized hardware—especially for chain-of-thought (CoT) reasoning and real-time inference, which require high-performance compute, memory, and storage.
Power and cooling efficiency are now mission-critical—as AI clusters scale, energy consumption becomes a major consideration, and Supermicro is leading the way with innovative solutions.
John: I’m really excited about the next 12-18 months in AI. The pace of innovation is driving down inference costs, making AI accessible to every enterprise. That said, most IT organizations still face a significant gap in designing and deploying accelerated compute and AI infrastructure. Many might not realize that Supermicro delivers fully integrated, rack-scale AI solutions—a game changer for enterprises looking to scale AI. Can you walk us through what makes your approach unique?
Ben: Supermicro’s approach is built on three pillars:
Engineering excellence – We design and manufacture everything in-house, from AI-optimized servers to all-flash storage and liquid-cooled racks.
Vertical integration – By controlling the entire stack, we deliver fully integrated and validated AI clusters optimized for performance, efficiency, and scalability.
Rapid innovation – We’re time to market with next-gen AI infrastructure, integrating the latest NVIDIA accelerated computing technologies, high-speed networking, and advanced cooling systems.
These capabilities allow us to provide plug-and-play AI clusters that integrate compute, storage, networking, and power management—eliminating deployment complexity and accelerating time-to-insight.
A great example? We recently delivered a 96-node EBox rack-scale solution for a high-performance AI cluster running the VAST software.
Supermicro's rack integration team meticulously fine-tuned and validated the system before final validation by the VAST team. This collaboration showcases our ability to combine advanced compute and storage into a seamless, enterprise-grade AI solution that delivers scalability, reliability, and performance.
John: It’s a beautiful photo! Most enterprises come from a history of picking discrete infrastructure components one-by-one, but the reality is that even small AI clusters are effectively “supercomputers” where everything – compute, storage, networking – needs to work optimally together to get the best performance and ROI possible. Hopefully, Supermicro will see a lot more rack-scale integration business with the influx of AI projects this year! Now, a few weeks ago there was some big news announced that you’re now in full production of NVIDIA HGX B200 systems. What does this mean for customers looking to deploy next-generation AI at scale?
Ben: NVIDIA HGX B200 represents a major leap forward in AI compute performance and efficiency for Supermicro’s customers. With Supermicro ramping up full production of NVIDIA Blackwell-based rack-scale solutions for our customers, enterprises can now deploy the latest AI infrastructure at scale with exceptional density, optimized power efficiency, and high-performance computing.
And we have a range of newly launched products for the NVIDIA HGX B200 series - for example, our 10U air-cooled NVIDIA HGX platforms SYS-A22GA-NBRT (Intel) or AS -A126GS-TNBR(AMD) - which offers unparalleled performance per watt, optimized scalability, and better workload efficiency across AI training, inference, and generative AI applications.
Another product, the 4U MGX SYS-422GL-NR, supports the latest NVIDIA H200 NVL platform to deliver exceptional AI inference performance with its 4-way NVLink bridge, providing high-bandwidth GPU communication without external switches. It’s ideal for enterprise PCIe servers, and includes NVIDIA AI Enterprise software for streamlined development and deployment of AI workloads.
Our Blackwell-based systems feature NVIDIA’s next-generation NVLink and NVSwitch architectures, providing massive memory bandwidth and seamless multi-GPU communication—key capabilities for large-scale AI model training, enterprise AI, with great potential for different sectors for AI workload.
We hope our customers—whether CSPs or enterprises— can rapidly deploy and scale NVIDIA HGX B200 systems, driving their AI-driven initiatives at full speed.
John: That’s quite the lineup! I’ve seen all the benchmarks from NVIDIA over the last couple months about Blackwell drastically improving inferencing performance. The fact that Supermicro has multiple different shapes and sizes makes you really well-positioned to capture both model builders that need large-scale compute clusters for training, but at the same time a more flexible and tailored solution for enterprises looking to do inferencing both in the near-term and also as they scale into production.
Ben: We recognize that enterprises have different AI deployment models depending on their industry, workload requirements, and IT strategy. While hyperscalers often rely on massive, centralized AI clusters, enterprises need flexible, right-sized solutions that align with their unique operational needs.
To meet this demand, we offer the broadest product portfolio, spanning accelerated computing servers, as well as advanced storage platforms. We architect AI solutions based on specific workloads and use cases, then offer rack-scale AI clusters as turn-key solutions, enterprise AI deployments, ensuring high-performance, scalable, and efficient AI infrastructure tailored to business needs.
John: Supermicro and VAST have been working together for almost a year now, and before that, you and I had been discussing AI infrastructure for quite some time! Not many people know this, but the “EBox” concept was actually inspired by a very large mutual customer looking to attach a high-performance storage system to a massive GPU compute cluster. Customers always have the best ideas! From your perspective, what does the VAST Data Platform bring to Supermicro’s AI solutions?
Ben: Data is the lifeblood of AI, and enterprises need seamless access to vast, unstructured datasets to drive meaningful insights. That’s why Supermicro is partnering with VAST to eliminate traditional storage bottlenecks and enable real-time data processing at scale.
By combining Supermicro’s high-density, accelerated computing infrastructure with VAST’s advanced data platform, we provide:
A turnkey AI solution that maximizes compute utilization
Accelerated AI workflows with real-time data access
Simplified deployment for enterprises scaling AI workloads
Together, Supermicro and VAST are delivering the next generation of AI infrastructure, ensuring enterprises have the power, speed, and scalability they need to drive AI innovation forward.
The AI Journey Starts Here. Let’s Talk.
Supermicro and VAST are pioneering the future of AI infrastructure, ensuring enterprises, foundation model builders and AI clouds have the data foundation they need to scale AI seamlessly. If you’re looking to accelerate AI adoption with the best-in-class compute and data solutions, drop us a line or join the conversation on Cosmos.
