These days it might seem easy to conclude that AI success is mostly about choosing the right model and buying enough accelerators but as the most recent Shared Everything podcast episode with Cloudera Chief Business Officer Abhas Ricky and VAST Data co-founder Jeff Denworth makes clear, this misses the real engineering challenge.
That perspective is informed by Cloudera’s own evolution. More than a decade ago, the company helped organizations solve the problem of storing and processing massive datasets through Hadoop. As enterprises matured, simply storing data stopped being enough. They needed governance, security, metadata, streaming analytics, and the ability to operate consistently across private infrastructure and multiple public clouds.
Today, Ricky says, the challenge has shifted again. Instead of asking how to manage enterprise data, organizations are asking how to turn trusted enterprise data into AI systems that can make useful decisions. According to Ricky, that requires bringing structured data, documents, streaming information, metadata, vector search, inference, governance, and security into a single architecture instead of treating them as separate technologies.
GPU starvation is at the root of a lot of the bottlenecks. The term is usually used to describe expensive accelerators sitting idle while waiting for data, but Ricky argues that the phrase is misleading.
As he says, GPUs are rarely waiting because storage is too slow. They are waiting because the entire system has failed to assemble the information the model needs. The data may live in disconnected silos. Metadata may be incomplete. Permissions may need to be evaluated. Vector indexes may not be ready. Network congestion can introduce delays. By the time all of those pieces are resolved, the GPU has already spent valuable time idle. The visible bottleneck appears inside the accelerator, but the actual bottleneck exists throughout the infrastructure stack.
Denworth argues the industry should think less about data movement and more about context. Large language models are no longer reading a handful of documents before producing an answer. Enterprise AI systems increasingly assemble information from databases, documents, retrieval systems, metadata catalogs, knowledge graphs, streaming data, and previous interactions. From the model’s perspective, all of those sources contribute to context. That changes the architectural problem from simply delivering bytes fast enough to assembling trusted context quickly enough.
The discussion reframes GPU starvation as something closer to context starvation. Compute is no longer the scarce resource if the surrounding infrastructure cannot continuously deliver the information the model needs to reason.
That emphasis on context also changes how enterprises think about their own data. Early enterprise AI projects largely focused on applying language models to unstructured text through retrieval-augmented generation.
Real enterprise intelligence comes from combining tables, documents, logs, metadata, streaming systems, and historical records into a unified view that AI can reason over. Organizations are increasingly interested in models that understand every form of enterprise information rather than treating databases and documents as separate worlds.
The conversation also introduces another architectural shift that is likely to become more important over the next several years: memory.
In traditional infrastructure, storage primarily existed to preserve information while memory existed to accelerate computation. AI blurs those boundaries. Retrieval systems, KV caches, agent memory, conversation history, and organizational knowledge all become different forms of memory that influence inference quality and speed.
Another recurring theme both touch on is that moving from an AI demonstration to a production deployment exposes an entirely different set of engineering problems. A prototype can answer questions convincingly, but production systems must explain where answers came from, preserve lineage, enforce governance policies, version prompts, monitor drift, and satisfy regulatory requirements. Those capabilities are rarely considered exciting compared to the latest foundation model, yet they determine whether enterprises can trust AI enough to deploy it in critical business processes. The final challenge is not generating intelligence. It is embedding that intelligence into the workflows that actually run a business.
The takeaway from this episode is that models alone cannot solve enterprise AI. Neither can storage, networking, governance, or data engineering independently. Every layer must work together, from accelerated compute and high-performance data infrastructure through governance, metadata, engineering platforms, and finally the models themselves. That may be the biggest lesson from this conversation.
As AI matures, competitive advantage is likely to come less from selecting a particular model and more from building an architecture that can continuously deliver trusted enterprise context to whichever model proves best tomorrow.
