BatchDAG: LLM-Planned Execution Graphs for Scalable Ad-Hoc Analysis Over Enterprise Data

Summary

BatchDAG proposes a directed acyclic graph (DAG) of typed operations—such as SQL, semantic search, in-memory transformations, and parallel branching—planned by an LLM. This graph is executed by a deterministic engine using topological wave parallelism. The system focuses on cross-asset analytical queries over enterprise-scale datasets, addressing the limitations of sequential LLM tool calls.

Why it matters

Enterprise analytics teams constantly face the gap between a single LLM’s document-level inferences and the need for comprehensive, multi-asset queries over tens of thousands of records. BatchDAG offers asset-aware chunked processing (reducing LLM calls by up to 47x) and sub-60-second latency across 50,000+ meetings, with a cost of $0.02–$0.24 per query. These features transform the system from a research curiosity into a deployable orchestration layer in practice. Additionally, it shifts the LLM’s role from 'responder' to 'planner,' aligning with the direction of production AI systems.

Highlights

• Asset-aware chunked processing groups rows by logical asset before branching. This method reduces LLM calls by up to 47x without compromising asset-based citation capabilities.
• Structured JSON intermediate outputs between DAG nodes reduce hallucination by 27% compared to plain text summaries. The planner achieves a 98.8% valid DAG rate in a 300-call test.
• In a controlled test of 12 queries, BatchDAG (3.74/5) performs on par with an expert pipeline (3.25/5) and outperforms the ReAct agent (3.09/5, p<0.01). BatchDAG’s transcript evidence traceability is 77%, compared to 46–60% for baselines.

Critical take

The main comparison relies on only n=12 queries. The p-value for the claimed 27% hallucination reduction is 0.107, which is well above the traditional 0.05 threshold. Consequently, the authors report a statistically insignificant effect as a finding. Furthermore, the 'expert-designed pipeline' baseline was created by the paper itself. The paper’s own framing ('not primarily an accuracy improvement') acknowledges that the main contribution is engineering orchestration around an off-the-shelf LLM. This limits the scientific novelty to DAG planning and chunked processing heuristics.