arxiv:2608.08389

Not Worth Another Token: Marginal Value Estimation for Efficient Deep Research Agents

Published on Aug 9

· Submitted by

Franck Dernoncourt on Aug 12

Upvote

Authors:

Franck Dernoncourt ,

Abstract

Pruning strategies applied at different pipeline stages reduce token usage and latency in long-horizon research agents, with early pruning yielding the greatest efficiency gains.

Generated by thinkingmachines/Inkling-Small

Long-horizon research agents solve open-ended tasks through iterative retrieval, aggregation, and synthesis, but context grows rapidly while the marginal value of additional evidence often declines. This leads to unnecessary token cost, higher latency, and noisier inputs for final report generation. We study marginal value estimation for context management in deep research agents and present the first systematic stage-aware comparison of pruning strategies across the pipeline. We evaluate lightweight heuristic criteria and a learned value model at pre-retrieval, post-retrieval, and pre-synthesis stages. Our results show that pruning effectiveness depends more on where pruning is applied than on the specific scoring rule: early pruning yields the largest end-to-end savings, while later pruning mainly refines the final synthesis context. Lightweight heuristics reduce token usage by up to 73% with little quality degradation, learned pruning remains competitive on selected trade-offs, and no single method dominates across quality, efficiency, and faithfulness. These findings provide practical guidance for designing efficient long-horizon agentic systems.

View arXiv page View PDF Add to collection

Community

Franck-Dernoncourt

Paper author Paper submitter 1 day ago

This comment has been hidden

O96a

1 day ago

The learned value model is the part I'd fight with in prod. My research agents hit a different document distribution every task — one day it's API docs, next it's legal PDFs, next it's a forum thread from 2011. A value model trained on one corpus is going to prune exactly the evidence it never saw, and that's the evidence I usually need. I'd want to see marginal value estimates transfer across domains before I trust the pruning. Maybe the fix is per-task calibration or a fallback that keeps anything the model is uncertain about. Until then, token savings on a benchmark corpus don't tell me much about my Tuesday.