ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
What this work does
Builds research-grounded programming tasks with heterogeneous datasets, expert-provided task knowledge and execution-based validation.
Connection to discovery
A benchmark for reliable implementation of individual scientific workflow steps before making end-to-end autonomy claims.
Evidence
The inspected collection protocol removes exploitable target outputs and modifies selected datasets; comparisons separate success, executability, cost and optional expert knowledge.
Scope and limitations
Executing a correct analysis over existing data does not run a new physical experiment or necessarily produce a novel scientific finding. Public artifacts create contamination risks that a benchmark protocol must track.
THE RESEARCH CONVERSATION
Discussion & reactions
Share a question, an insight, or a replication experience. Sign in with GitHub to join the conversation.