The assumptions that shaped enterprise AI infrastructure are breaking down.
Data lives in one system. Analytics runs in another. Vector databases sit off to the side. Event streams flow through separate infrastructure. Kubernetes clusters sprawl across teams.
Each layer solves a real problem. Together, they create architectures that are fragmented, expensive, and difficult to operate at scale.
As AI workloads become more demanding and more operational, real-time intelligence requires more than faster components. It requires a platform where data, vectors, events, and execution operate as one system.
That is the foundation VAST has been building. Today, VAST AI OS 5.5 is generally available.
This release advances four core dimensions of the platform: hyperscale vector retrieval, native analytical execution, managed pipeline compute, and efficient data operations without data movement. Here is what is shipping.
The VAST Hyperscale Vector Index
The prevailing assumption has been that vector search must run in a separate system built around in-memory structures and sharded indexes. That model works at modest scale. It begins to fracture at tens or hundreds of billions of vectors, when memory-resident indexes become prohibitively expensive, sharding adds operational complexity, and hybrid search requires stitching together results across systems.
The VAST Hyperscale Vector Index takes a different approach. At its core is a proprietary hierarchical clustering index in the VAST vector store, which is natively embedded in the VAST DataBase. Instead of maintaining a flat, memory-resident graph, VAST organizes vectors into a multi-level hierarchy of distance-based clusters. A query evaluates a bounded number of clusters at each level, descending only into the most promising candidates. Query latency plateaus rather than rising linearly with scale. Because only targeted subsets of quantized vectors are loaded into memory at query time, search remains memory-bounded rather than dependent on full index residency.
The result: 10x faster vector retrieval in benchmark comparisons against leading open-source vector databases at 1 billion 128-dimensional vectors.
Because vectors live alongside structured and semi-structured data inside VAST DataBase, similarity search can combine natively with SQL filters, joins, and metadata predicates in a single execution path, under unified governance.
This makes vector search practical at the scale and consistency required for real-world AI systems.
Native Analytical Execution at the Data Layer
High-performance analytics should not require moving data to an external engine. VAST 5.5 advances this principle on two fronts.
VAST Native Query Engine now supports more than 50 additional aggregation and statistical functions.
These include variance, standard deviation, regression analysis, correlation, quantile estimation, and conditional aggregation via the FILTER clause.
Analytical logic that previously required exporting data to external systems now executes directly inside VAST, where the data already lives. Complex statistical analysis, machine learning feature preparation, and exploratory data analysis run without additional infrastructure or data movement. Now, with this native VAST Engine, you can also conduct hybrid vector + SQL filter analytics in the same query path.
Vector Search with SQL Filtering
Partitioning for Large Tables improves analytical performance for large-scale workloads through SQL-native declarative partitioning. Users express high-level intent - partition by time, by category, by any logical dimension - and VAST automatically applies partition pruning, join optimization, and efficient deletes at query time. In benchmarking against Iceberg-based tables, VAST delivered more than 60% faster performance on highly selective queries, 200x faster updates, and approximately 20% aggregate runtime improvement in a real-world manufacturing environment.
This is not traditional database partitioning. There are no shards to manage, no data to rebalance, no node ownership to reason about. Performance gains emerge from engine-managed execution, not operational complexity.
DataEngine Advances & Managed AI Pipeline Compute Inside the Cluster
Modern AI pipelines require continuous execution alongside data: ingestion, chunking, embedding, enrichment, indexing. In most environments, this means running a separate Kubernetes cluster alongside the data platform. That separation introduces latency, increases cost, and creates operational friction.
With VAST 5.5, VAST Native Compute brings container orchestration directly into the VAST AI OS for DataEngine functions. Administrators can provision managed Kubernetes clusters on CNodes within the VAST cluster, allocating dedicated compute capacity for containerized pipeline workloads. Provisioning, scaling, upgrades, and lifecycle operations are handled entirely by the platform.
The practical effect: DataEngine pipelines, like vectorization jobs, document and video chunking, embedding generation, event-driven functions, can be fully orchestrated inside the cluster, adjacent to the data they operate on. Python functions for DataEngine orchestrating and animating these pipelines run easily, without requiring teams to operate, secure, or maintain separate Kubernetes infrastructure.
File-Triggered Pipelines for NFS extend VAST DataEngine's event-driven automation beyond S3 to native NFS workflows. File system events, including creation, deletion, metadata changes, can now trigger pipelines directly via NFSv4, without protocol translation or object storage re-architecture. For organizations that rely on NFS for application data, media processing, scientific data, or legacy systems, this eliminates a meaningful gap in real-time automation.
Finally, this release also introduces advanced pipeline controls for DataEngine functions. Event batching allows functions to process multiple events together, reducing overhead for high-throughput pipelines. Conditional function routing enables developers to dynamically direct events based on application logic, rather than relying on static pipeline definitions. User-defined metrics provide visibility into pipeline behavior, allowing teams to monitor and optimize execution in real time.
This release focuses on managed DataEngine serverless Python functions to orchestrate workloads. Model serving continues to run on external Kubernetes today. Future releases will extend native compute to additional workload types.
Zero-Copy Data Operations at Scale
Two additional capabilities in 5.5 strengthen pipeline automation.
Instant Data Cloning for Block converts block-level copy operations from data movement tasks into lightweight metadata operations. Instead of reading from a source range and rewriting to a target, the platform allows the target to reference the same underlying flash data. Large copy operations — VM clones, Storage vMotion, SMB server-side copy — complete almost instantly, without consuming storage bandwidth or write buffer resources.
Available Now
These capabilities extend a unified platform where data, vectors, events, and execution operate together. Data remains governed. Execution stays close to the data. Performance scales as workloads grow.
VAST AI OS 5.5 is available today.
To go deeper on the architecture and performance behind this release, read our earlier blog here and join our VAST AI OS 5.5 webinar next week.
