DeepLens Diagnosis Agent: Agentic Workflow Design Lets a Small Reasoning Model Compete with Frontier LLMs

Summary

A five-stage agentic pipeline (structured inference, RAG retrieval, constrained generation, evidence triangulation, and auditable decision-making) built around a 7B-parameter medical model achieves a 60.14% top-1 accuracy rate in the 915-case DiagnosisArena benchmark. This system maintains per-case costs approximately 35-45% lower while outperforming Claude Sonnet 4.5 and Gemini 3.1 Pro by roughly 9-10 points.

Why it matters

The core claim of the study is that disciplined workflow design can bridge most of the diagnostic reasoning gap between a small model and frontier LLMs. This finding has direct cost and deployment implications for healthcare institutions with limited resources. The ability of a 7B-parameter model to compete with multi-billion-parameter APIs using structured prompts at a significantly lower price facilitates the adoption of diagnostic AI in hospitals that cannot afford frontier model subscriptions. Furthermore, the emphasis on auditable intermediate outputs addresses regulatory and clinical trust requirements that raw accuracy metrics alone cannot satisfy.

Highlights

• Workflow design alone provides a +36 point improvement compared to using the same 7B model in single-shot mode (23.99% → 60.14%).
• Per-case cost of $0.0072 and 24-second latency on a single A100; these figures remain below the costs of Claude Sonnet 4.5 and Gemini 3.1 Pro.
• The pipeline generates structured intermediate outputs at each stage, enabling error localization and audit trails.

Critical take

A 60.14% top-1 accuracy rate still means that approximately one in every four diagnoses is incorrect. Additionally, the 915-case benchmark remains modest for such a high-risk clinical claim. Moreover, the study is a technical report by John Snow Labs and contains no information regarding prospective clinical validation, human-in-the-loop testing, or real patient data. Therefore, the claim of 'outperforming frontier LLMs' relies on a single, narrow benchmark rather than proven clinical safety.