AI agents break at scale without real-time data, says VeloDB’s APAC head

Why pilots succeed, production fails

Regional Director for APAC at VeloDB, Nitin Waghulde, said most agentic-AI projects succeed in pilot phases—where scope, data, and questions are tightly controlled—but break down when scaling to production. The gap exposes systemic issues: spiralling costs, unpredictable query loads, and stale or fragmented data.

How AI agents stress data infrastructure

According to Waghulde, AI agents require hundreds of data lookups to answer a single complex question, compared to one lookup in traditional reporting systems. Agents operate continuously, asking multiple unscripted questions simultaneously while iterating through cycles of querying, reasoning, and retrying. This unpredictable load strains legacy data infrastructure, which was designed for human-scale batch reporting, not real-time, high-concurrency AI workloads.

What production-grade AI agents demand

Waghulde outlined four non-negotiable requirements for production-grade AI agents:

Access to every data type—structured records, documents, and unstructured meaning.

Scale to scan petabytes per query, not terabytes.

Freshness so every change (insert, update, delete) is visible immediately, without batch delays.

Audit trails for every interaction, which themselves become decision-support datasets.

A single-engine architecture for hybrid queries

VeloDB, founded by the creators of Apache Doris, offers a single-engine solution that fuses SQL, vector search, and full-text search in one query. Apache Doris, an open-source real-time analytics database under the Apache Software Foundation, claims over 14,000 GitHub stars, 700 contributors, and adoption by 5,000 enterprises globally.

Production results: latency, scale, cost

Waghulde cited two production examples. An unnamed major cryptocurrency exchange supports 33,000 concurrent AI agent conversations with 3-millisecond tail latency. Chinese AI company MiniMax ingests data at 10 Gbps and queries 10 petabytes in under two seconds, cutting costs by 70 % after replacing Grafana Loki with Apache Doris.

What this means

Lazyfounder analysis — our interpretation, not reported fact.

Waghulde’s keynote makes two points founders should weigh against their own agentic-AI pilots.

First, Gartner’s projection that over 40 % of projects may be cancelled by 2027 is not just about model performance; it centres on data-layer costs and latency. Teams burning cloud credits on hundreds of sub-queries for every user question are discovering that the unit economics break before they reach meaningful scale.

Second, the shift toward single-engine architectures (SQL, vector, and text search in one place) replaces the messy pipeline of specialised databases that most startups cobbled together for RAG last year. Founders who adopted those early RAG stacks may now face a rip-and-replace decision: retrofit real-time freshness on top of fragmented stores, or migrate to a unified analytics layer that can serve petabyte-scale hybrid queries without adding latency.

Key takeaways

• Founders scaling AI agents beyond pilots face escalating cloud costs and latency from hundreds of sub-queries per user question, not just model limitations.
• Legacy data stacks built for human-scale reporting lack the concurrency, freshness, and hybrid-query support AI agents require.
• Companies like MiniMax reduced query costs by 70 % and slashed latency from seconds to sub-3 ms after replatforming to a single real-time analytics engine.
• Gartner’s projection of 40 % attrition in agentic-AI projects by 2027 highlights data reliability and cost as key risks, not model accuracy alone.

Related on Lazyfounder

• More ai news