ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent
What this work does
Separately prompted search/relevance/grounding steps generate trajectories that are scored and reused for self-training and smaller-model distillation.
Connection to discovery
Evidence against a blanket rule that only superficially correct intermediate steps should be retained; selection must match end-to-end objectives.
Evidence
Inspected multi-step search evaluation and the ablation in which manual filtering of locally bad steps did not improve final accuracy.
Scope and limitations
The result is task- and data-size-dependent, using small QA evaluation sets and model-based judgments; it does not prove that erroneous scientific claims are suitable positive demonstrations.
THE RESEARCH CONVERSATION
Discussion & reactions
Share a question, an insight, or a replication experience. Sign in with GitHub to join the conversation.