With the GA release of VAST AI OS 5.5, the VAST Native Query Engine takes a meaningful step forward - extending beyond its original role as a vector execution engine into a broader, database-native analytical layer. The premise is straightforward: analytical execution belongs at the data layer, not above it. Data should not have to move to be analyzed.
From Vector Execution to Analytical Processing
The VAST Native Query Engine was originally introduced to power vector retrieval within the VAST AI Operating System in our hyperscale vector store. It combines similarity search, filtering, and access control into a single execution path, operating directly on data stored within VAST.
With this release, that same execution framework now supports a broad set of native aggregation and statistical functions, extending the engine beyond vector execution into analytical workloads.
More than 50 aggregation functions have been added, spanning core operations such as SUM, MIN, MAX, AVG, and COUNT, as well as statistical measures including variance, standard deviation, skewness, kurtosis, covariance, and correlation.
The engine also introduces regression functions (such as regr_slope and regr_intercept), quantile estimation, and ordered-set statistical operations such as percentile_cont and percentile_disc. Boolean, conditional, and bitwise aggregations are supported alongside compensated summation functions (fsum, favg) to improve numerical accuracy at scale.
Support for the SQL FILTER clause enables conditional aggregation directly within query expressions, allowing more expressive analytical queries without additional processing stages.
This release focuses on enabling single-pass aggregation and statistical analysis directly on in-place data, rather than providing full warehouse-style SQL coverage.
Rethinking Where Analytics Runs
In most modern data architectures, analytical processing is separated from storage. Data is ingested, landed, transformed, and then queried through external engines such as Spark or Trino. This separation has been treated as necessary for scalability.
In practice, it introduces a different set of constraints: data movement, pipeline orchestration, duplicated storage, and latency between ingestion and analysis.
The VAST Native Query Engine follows a different model. It executes analytical logic directly within the platform, operating on data where it already resides. Because it is built on the VAST Disaggregated Shared-Everything (DASE) architecture, it can access data globally without requiring reshuffling, repartitioning, or duplication.
This changes the role of the query engine. Instead of acting as a separate system that consumes data, it becomes part of the data platform itself, executing computation in place.
A Unified Execution Path for AI and Analytics
One of the more significant implications of this release is the unification of execution across different workload types.
The same engine now supports:
Vector similarity search
Structured data access
Aggregation and statistical analysis
These operations can be composed within a single query flow for hybrid search. For example, similarity search results can be filtered and aggregated without exporting data to another system. Analysts can correlate embeddings, metadata, and metrics within a single execution path, rather than stitching together results across multiple systems.
query = f"""SELECT array_distance("vector"::FLOAT[128], ARRAY[{ref_vector}]::FLOAT[128]) as dist,
"vector" FROM "bucket-name/schema_name".table_name WHERE id > 10000000 ORDER BY dist LIMIT 10"""
Vector search with SQL filtering
Focused Scope, Predictable Performance
This release intentionally focuses on expanding analytical depth while maintaining predictable execution performance. Support for GROUP BY, DISTINCT, and ORDER BY within aggregates is not yet included in this version. In addition, several aggregation families, such as approximate functions, histogram-based aggregations, weighted aggregations, and list-producing functions, are not included in this release. These are coming soon.
Rather than attempting to replicate the full surface area of a traditional data warehouse, the engine is optimized for efficient filtering, projection, and single-pass aggregation.
This design reflects a deliberate tradeoff: prioritizing predictable, high-performance execution on in-place data over full SQL completeness.
Practical Applications
In practice, these capabilities enable several classes of workloads:
Real-time analytics on continuously ingested data without waiting for batch pipelines
Metadata and catalog queries where filtering and summarization are required across large datasets
AI-assisted investigations combining vector search with structured metrics
Operational dashboards built directly on live data without pre-aggregation layers
In each case, analytical logic executes directly on the data without requiring external processing engines or intermediate pipelines.
Toward a Unified Data and Execution Platform
This release establishes the analytical foundation. Single-pass aggregation, statistical analysis, regression, quantile estimation, and conditional filtering all executing natively inside VAST, on live data, without external engines or intermediate pipelines. GROUP BY, DISTINCT, JOIN, and ORDER BY within aggregates are coming. Approximate functions, histogram-based aggregations, and broader SQL coverage will follow.
Each addition builds on the same execution framework that already powers vector retrieval and streaming ingestion inside the platform. The goal is not to replicate a data warehouse. It is to make one unnecessary.
To learn more about all of the capabilities in our 5.5 GA release, read our launch blog.
