My last post covered why AI platforms need remote storage at all, and why VAST is a great choice for these workloads:
Scale
The choice to share or isolate
Cheap snapshots
Tenants instead of dedicated clusters
Replication for workloads that can't go down
Every one of those trends shows up on a normal AI platform. Agent sandboxes - safe, isolated code testing and execution environments - however, take all five and compress them into a workload that lives for minutes, which makes sandboxes the sharpest version of this story. This post dives deeper into the world of sandboxes and VAST’s approach to serving them.
A Workload Built to Disappear
When agent sandboxes spin up and tear down by the thousand, each wanting its own private, disposable, high-performance disk, you're not dealing with a handful of volumes anymore. You're dealing with a workload pattern that assumes storage can be provisioned and destroyed at a rate local disk was never built to coordinate across a fleet. A sandbox fleet doesn't create five volumes and leave them alone the way a database does. It can create and destroy thousands an hour, and every one of those events touches host and network state somewhere in the system.
It helps to be specific about why this breaks the assumptions most storage systems were built on. A typical provisioning path assumes a volume is going to exist for a while: allocate it, register it, hand it to a workload that will use it for months. The bookkeeping behind that - host registration, network path setup, metadata tracking - is priced for infrequent events. A sandbox fleet inverts that pricing model entirely. The volume exists to be destroyed within minutes of being created, and if the system treats that as a rare edge case instead of the normal case, the bookkeeping itself becomes the bottleneck long before raw throughput does.
Figure 1: Provision, run, destroy, repeated thousands of times an hour across a fleet. Nothing about this pattern resembles a database's relationship with its disk.
That churn is exactly why VAST's blockHostsAutoPrune option exists: It automatically cleans up unused VAST Host entries, the NQNs that identify NVMe/TCP initiators, once no PVC on a node references them anymore. Left unmanaged, host sprawl is the kind of thing that turns a clean architecture into a support ticket a few months in. Not because anything failed, but because nobody was watching a number that only mattered at this scale.
It's also why provisioning latency matters more here than almost anywhere else on the platform. A slow volume-creation path is invisible on a database that gets provisioned once and runs for a year. On a sandbox fleet, that same latency multiplies by every session, every minute, and shows up directly as an agent sitting idle instead of working.
VAST's Block CSI Driver was built as general-purpose NVMe/TCP block storage - not a sandbox-specific side project - which is exactly why it holds up here: The same driver that handles a long-running database handles a workload that lives ninety seconds, without needing a different code path for either.
Isolated by Default, and That's the Right Call
Most agent harnesses - the software actually orchestrating these agents - default to assuming storage is not shared: one sandbox, one volume, fully isolated. That's the right default for most of these workloads; the whole point of a sandbox is a clean, disposable, private environment. Giving two agent sessions write access to the same filesystem invites race conditions nobody wants to debug at whatever hour they surface.
Host directories may be exposed to a sandbox as read-only at most, and support for shared storage across Kubernetes environments remains limited. The pattern that matters here generalizes well beyond any one harness: isolation is the default, and the harness itself typically isn't the thing that solves shared access once you're running on Kubernetes. That gap doesn't sit there unfilled forever, it just means the storage layer underneath has to be the one to close it.
Figure 2: VAST's answer to the pattern: isolated block volumes for per-session sandboxes, shared NFS for the datasets teams collaborate over, through two separate CSI provisioners on one underlying system.
That's where VAST's structural answer matters more than it would on a less extreme workload. Block and file are two separate CSI provisioners under one platform, not one system pretending to be two. Most sandbox volumes get provisioned as private, isolated block devices - RWO, one volume, one writer - because that's what the workload wants. When a pipeline needs an agent's output visible to the next step immediately, or several agents collaborating on the same repository, that's just a different StorageClass away, pointed at the same underlying VAST cluster instead of an entirely different system.
The Narrower Middle Ground: Read-Only at Scale
Between fully isolated and fully shared sits a pattern sandbox fleets hit constantly: an evaluation run where dozens or hundreds of agents all need to read the same golden dataset, without duplicating it per session and without the write conflicts that come with true multi-writer access. VAST's block driver exposes an allowROManyBlockFsMode option for exactly this case, ReadOnlyMany semantics that let several pods on different nodes mount the same block volume read-only at once.
Figure 3: A narrower capability than NFS's full multi-writer model, but it covers a real case the harness's own tooling doesn't reach on Kubernetes.
That distinction matters because it's easy to assume the only two options are total isolation or full NFS-style sharing. There's a middle case - many readers, one writer - that fits a lot of evaluation and benchmarking workloads exactly. And, with VAST, it's available on the same block volumes a sandbox fleet is already using, without reaching for a second storage system just to fan a dataset out to a hundred readers.
Why This is the Extreme Case, Not a Special Case
Nothing about agent sandboxes requires a different storage architecture than the rest of an AI platform needs. That's the actual point:
The same block driver that handles millions of devices
The same choice between isolated block and shared NFS
The same cheap snapshots that let a sandbox template get cloned into a running volume in seconds instead of cold-initializing
The same tenant model that lets a platform team hand a team its own quota without a dedicated cluster
All of it applies here unchanged.
What's different is the intensity: the churn rate is higher, the isolation default is stricter, and the tolerance for a slow provisioning path is close to zero, because an agent waiting on its sandbox is an agent doing nothing. If a storage platform holds up under agent sandbox churn, isolation, and provisioning-speed requirements, it holds up under everything else on the platform too, because sandboxes are what happens when every one of those requirements gets turned up at once.
To Be Continued
There's more to come here. Specifically, a follow-up post will go deep on VAST's integration with modern agent execution environments, including how you trace an individual agent sandbox's activity weeks after the sandbox itself is gone - which turns out to be a harder and more interesting problem than the provisioning story alone.
