MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
What this work does
Packages 75 Kaggle competitions with recreated grading, agent execution environments and leaderboard-derived human reference scores.
Connection to discovery
A practical long-horizon experimentation benchmark with executable outcomes and compute-sensitive comparisons.
Evidence
Inspected competition screening, submission-validity versus score access, solution-use rules and attempts/compute/time ablations.
Scope and limitations
Engineering performance on known competitions is not necessarily novel research. Public solutions and pretrained-data contamination require explicit controls; no new physical measurements are collected.
THE RESEARCH CONVERSATION
Discussion & reactions
Share a question, an insight, or a replication experience. Sign in with GitHub to join the conversation.