Good Papers: read the good ones

Showing Agent benchmarks & environments Show all papers

84%Must read

?Must readVote to see the score

CheckerBench: Can Long-Horizon Agents Synthesize Static-Analysis Checkers?

CheckerBench evaluates long-horizon agents on synthesizing static-analysis checkers across 300 CVE-derived tasks, finding best Pass@1 reaches 45.33%.

Hang He, Li Wang, Hao Chen, Yuchen Shao and 8 more

Published Oct 6, 2026 · ▲ 52 on Hugging Face · Code

100% Readers1 of 1 upvoted

12/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 12 of 20 reviewers recommend it

lenient 5/5

medium 5/10

strict 2/5

74%Highly rated

?Highly ratedVote to see the score

arXivAgent benchmarks & environments

MiniCorp: The Last Mile of the AI Agent Firm

MiniCorp is a simulated office environment that generates longitudinal, counterfactual enterprise data to study autonomous AI-run companies and train adaptive agents.

Jingying Zeng, Zhenwei Dai, Jinning Li, Changho Shin and 5 more

Published Oct 5, 2026 · ▲ 8 on Hugging Face

– ReadersNo votes yet

9/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 9 of 20 reviewers recommend it

lenient 5/5

medium 4/10

strict 0/5

86%Must read

?Must readVote to see the score

arXivAgent benchmarks & environments

AutoSciBench: Autonomous Benchmark Generation for Evaluating Scientific Agents

AutoSciBench autonomously generates and iteratively adapts scientific agent benchmarks via recipes and concepts, reducing solver accuracy by over 22 points versus human benchmarks while improving quality ratings.

Dongki Kim, Namkyeong Lee, Surag Nair, Carl Edwards and 6 more

Published Oct 4, 2026 · ▲ 17 on Hugging Face

100% Readers1 of 1 upvoted

13/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 13 of 20 reviewers recommend it

lenient 5/5

medium 7/10

strict 1/5

91%Must read

arXivAgent benchmarks & environments

UndoBench: Separating Task Competence from Recovery Capability in Tool-Using AI Agents

UndoBench separates tool-using agent competence from fault recovery via paired enterprise workflow trials, finding 83.54% nominal success but only 46.72% recovery success with phase-dependent vulnerabilities.

Dolly Sah, Tanmay Sah, Harshul Jain, Tanya Sah

Published Oct 4, 2026 · ▲ 11 on Hugging Face · Code ★ 1

– ReadersNo votes yet

17/20 AI panelreviewers recommend it

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 17 of 20 reviewers recommend it

lenient 5/5

medium 9/10

strict 3/5

89%Must read

?Must readVote to see the score

arXivAgent benchmarks & environments

LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures

LMBuild evaluates LLM agents on generating buildable, functional 3D structures and finds physical operability and functional affordance remain challenging despite improved soundness.

Jiateng Liu, Rushi Wang, Cheng Qian, Xuejun Zhang and 8 more

Published Oct 3, 2026 · ▲ 24 on Hugging Face · Code ★ 1

– ReadersNo votes yet

16/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 16 of 20 reviewers recommend it

lenient 5/5

medium 9/10

strict 2/5

76%Highly rated

?Highly ratedVote to see the score

arXivAgent benchmarks & environments

HyperBrowseComp: A Multilingual and Multimodal Stress Test for Web-Browsing Agents

HyperBrowseComp introduces a multilingual, multimodal web-browsing benchmark of 423 hard questions requiring obscure evidence discovery, and current agents perform poorly against human baselines.

Alham Fikri Aji, Faiz Rizki Ramadhan, Zayd M. K. Zuhri, Seung Hun Eddie Han and 13 more

Published Oct 2, 2026 · ▲ 53 on Hugging Face

100% Readers1 of 1 upvoted

6/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 6 of 20 reviewers recommend it

lenient 4/5

medium 1/10

strict 1/5

88%Must read

?Must readVote to see the score

arXivAgent benchmarks & environments

Self-Supervised Scaling of Terminal Environments for Scientific Domains

Software-in-the-loop reconstruction scales terminal-agent training by deriving verified tasks from existing scientific workflows without manual references, improving Terminal-Bench 2 performance to 53.56%.

Zhongzhi Li, Yucheng Shi, Zongxia Li, Junyao Yang and 7 more

Published Oct 2, 2026 · 0 citations · ▲ 10 on Hugging Face

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

83%Must read

?Must readVote to see the score

arXivAgent benchmarks & environments

World Editing: Intervening on Executable Worlds at Increasing Depth

World editing intervenes on executable environments at increasing depth via IGMWorld and IGMBench, where top agents achieve 78.2% task success with reliability declining by depth.

Max Ku, Nok-Kan Law, Yu-Chien Tang, Shih-Ying Yeh and 14 more

Published Oct 1, 2026 · ▲ 14 on Hugging Face · Code

– ReadersNo votes yet

13/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 13 of 20 reviewers recommend it

lenient 5/5

medium 5/10

strict 3/5

86%Must read

?Must readVote to see the score

arXivAgent benchmarks & environments

SimuVerity: Benchmarking Agents for Engineering-Grade Simulink Model Generation

SimuVerity benchmarks text-to-executable Simulink generation across engineering domains, finding best agents score only 42.86 and structural similarity poorly predicts engineering performance.

Ruiqi Zhang, Jiahao Wang, Mingxuan Li, Haichen Luo and 8 more

Published Oct 1, 2026 · ▲ 47 on Hugging Face · Code ★ 19

– ReadersNo votes yet

14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 14 of 20 reviewers recommend it

lenient 5/5

medium 7/10

strict 2/5

88%Must read

?Must readVote to see the score

NeurIPS 2026WorkshopAgent benchmarks & environments

Auditing Web Agent Evaluation on WebArena-Lite: Human Review of Outcomes and Trajectories

Human audit of 165 WebArena-Lite tasks recovers 5.45, 8.49% evaluator-missed successes, reveals trajectory errors like looping, and shows guide text and MASM improve results.

Chengguang Gan, Zimeng He, Yoshihiro Tsujii, Ken-ichiro Kobayashi and 2 more

Published Oct 1, 2026 · 0 citations

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

86%Must read

?Must readVote to see the score

arXivAgent benchmarks & environments

OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software

OSWorld-Science benchmarks VLM agents on 146 expert scientific software tasks, showing state-of-the-art models still struggle with scientific workflows and harness design.

Dingyuan Dai, Heli Qi, Lei Liu, Yinxi Li and 27 more

Published Sep 30, 2026 · 0 citations · ▲ 63 on Hugging Face · Code ★ 5

– ReadersNo votes yet

14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

76%Highly rated

?Highly ratedVote to see the score

arXivAgent benchmarks & environments

WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents

WorldAuditBench benchmarks interactive 3D world auditing with multimodal agents, finding success rates of 6.6% to 42.3% versus 83.4% human performance.

Ziyan Jiang, Jingbo Yang, Jiabao Ji, Yujian Liu and 4 more

Published Sep 30, 2026 · 0 citations · ▲ 102 on Hugging Face · Code ★ 4

– ReadersNo votes yet

10/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

91%Must read

arXivAgent benchmarks & environments

OSWorld-Pro: Process-based Evaluation for Computer Use Agents

OSWorld-Pro introduces process-based evaluation with 2,800 subgoals across 300 tasks, revealing top models achieve only 75.7% subgoal success versus 83.4% end-state performance and identifying distinct failure modes like irrelevant actions and click errors.

Zhilin Wang, Shaokun Zhang, Yifan Zhang, Hao E. Zhang and 8 more

Published Sep 21, 2026 · 0 citations · ▲ 15 on Hugging Face

100% Readers1 of 1 upvoted

16/20 AI panelreviewers recommend it

Only vote on papers you've read. Sign in with GitHub to vote.

80%Must read

?Must readVote to see the score

arXivTsinghua Zhipu AI U Electronic Science and Technol Agent benchmarks & environments

MTAC-IFBench: Benchmarking Instruction-Following in Multi-Turn Agentic Coding

MTAC-IFBench benchmarks multi-turn instruction-following in agentic coding via progressive constraints, revealing rapid performance degradation in current code agents as sessions lengthen.

Bosi Wen, Cunxiang Wang, Jiayi Gui, Haoke Zhang and 5 more

Published Sep 14, 2026 · 0 citations

– ReadersNo votes yet. 1 from authors or colleagues not counted

12/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

83%Must read

?Must readVote to see the score

arXivAgent benchmarks & environments

HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?

HarnessDev evaluates LLMs creating and evolving agent harnesses, finding generated harnesses lag human references on coding and search but match them on writing and ML tasks, with unstable, model-dependent evolution gains.

Yuhao Wu, Jingyuan Zhang, Jiajun Shi, Xinping Lei and 15 more

Published Sep 1, 2026 · 0 citations · ▲ 566 on Hugging Face

– ReadersNo votes yet

13/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

88%Must read

?Must readVote to see the score

arXivWashington University in St. Lou Google U North Carolina at Chapel Hill Agent benchmarks & environments

EnvHarness: Awakening Static Worlds for Agent Learning

EnvHarness wraps static environments with programmable components to reshape agent behavior without altering underlying logic, improving benchmarks by up to 9.0 points while enabling continuous policy-environment co-evolution.

Chengsong Huang, Zifeng Wang, Rujun Han, Jun Yan and 13 more

Published Aug 20, 2026 · 0 citations · ▲ 175 on Hugging Face · Code ★ 619

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

83%Must read

?Must readVote to see the score

arXivAgent benchmarks & environments

Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields

Workflow-GYM benchmarks long-horizon professional GUI workflows, showing top agents achieve only ~30% success due to stage omission, error propagation, and objective drift.

Liya Zhu, Jingzhe Ding, Jian Zhang, Jianbo Xue and 36 more

Published Jun 9, 2026 · 0 citations · ▲ 221 on Hugging Face

– ReadersNo votes yet

13/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

78%Highly rated

?Highly ratedVote to see the score

arXivAgent benchmarks & environments

Agents' Last Exam

ALE introduces a benchmark evaluating AI agents on long-horizon, economically valuable real-world tasks across 13 industry clusters, finding current full pass rates below 1%.

Yiyou Sun, Xinyang Han, Weichen Zhang, Yuanbo Pang and 36 more

Published Jun 3, 2026 · 0 citations · ▲ 392 on Hugging Face · Code ★ 1,083

– ReadersNo votes yet

11/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

80%Must read

?Must readVote to see the score

arXivAgent benchmarks & environments

ClawBench: Can AI Agents Complete Everyday Online Tasks?

ClawBench introduces 153 real-world online tasks across 144 platforms to evaluate AI agents, finding frontier models complete only about a third of them.

Yuxuan Zhang, Yubo Wang, Yipeng Zhu, Penghui Du and 24 more

Published Apr 9, 2026 · 0 citations · ▲ 377 on Hugging Face · Code ★ 958

– ReadersNo votes yet

12/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

75%Highly rated

NeurIPS 2026National University of Singapore National University of Singapore Nanyang Technological Singapore University of Technolo SMU Agent benchmarks & environments

MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome

MiroEval benchmarks multimodal deep research agents via process and outcome evaluation across 100 real-world tasks, finding process quality predicts outcomes and multimodal tasks reduce scores by 3, 10 points.

Fangda Ye, Yuxin Hu, Pengxiang Zhu, Yibo Li and 18 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 69 on Hugging Face · Code ★ 52

0% Readers0 of 1 upvoted

17/20 AI panelreviewers recommend it

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 17 of 20 reviewers recommend it

lenient 5/5

medium 9/10

strict 3/5

90%Must read

NeurIPS 2026Technion – Israel Institute of T Technion - Israel Institute of T Hebrew University of Jerusalem, International Business Machines Technion, Israel Institute of Te Agent benchmarks & environments

A Matter of TASTE: Improving Coverage and Difficulty of Agent Benchmarks

TASTE reverses benchmark construction by evolving tool sequences to automatically generate harder, broader-coverage agent tasks that expose severe performance drops and saturation in existing benchmarks.

Tomer Keren, Nitay Calderon, Asaf Yehudai, Yotam Perlitz and 2 more

Paris Poster Session 6, Fri, Dec 11, 2:30 PM–4:30 PM, Paris Poster Hall · Published 2026 · ▲ 74 on Hugging Face · Code ★ 4

100% Readers1 of 1 upvoted

15/20 AI panelreviewers recommend it

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 15 of 20 reviewers recommend it

lenient 4/5

medium 9/10

strict 2/5

67%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Harvard Massachusetts Institute of Techn Arcada Labs Design Arena, Harvard Agent benchmarks & environments

AudioAgentBench: Evaluating Multi-Turn Voice Agents on Real-World Tasks

Gardenia Liu, Yi-Hao Peng, Kamryn Ohly, Oliver Johansson and 3 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

2/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 2 of 20 reviewers recommend it

lenient 2/5

medium 0/10

strict 0/5

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026Institute of Software, Chinese A U the Chinese Academy of Science Beijing University of Aeronautic Institute of Software, CAS Alibaba Group Agent benchmarks & environments

AutoDataBench: How Far Are LLM Agents from Autonomously Engineering Post-Training Data Pipelines?

Qiaoyu Tang, Hao Xiang, Le Yu, Yaojie Lu and 9 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026CUHK & Microsoft Research As Research, Microsoft Microsoft U Illinois at Urbana-Champaign The Chinese University of Hong K Agent benchmarks & environments

RoutingBench: Can Agentic Routing Analysis Scale to Production Datacenter Networks?

Wenlong Ding, Zhixiong Niu, Jianan Yang, Fajun Zhang and 5 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

69%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026OralThe University of Tokyo The University of Tokyo & RI Waseda University, Japan Waseda RIKEN Agent benchmarks & environments

Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations

Junjue Wang, Weihao Xuan, Heli Qi, Pengyu Dai and 6 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

3/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 3 of 20 reviewers recommend it

lenient 2/5

medium 1/10

strict 0/5

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026Tongji Agent benchmarks & environments

Z-AXIS: From Deterministic Ground to Agentic Depth for Enterprise Evaluation

Zifan Song, Mianzhi Chang, Ziyang Liao, haiyan xu and 2 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it

lenient 0/5

medium 0/10

strict 0/5

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026Massachusetts Institute of Techn Max Planck Institute of Biochemi MIT / Seoul national university Agent benchmarks & environments

M4Bench: Evaluating Procedural Specification for Clinical EHR Derivation Agents

Hannes Ill, Rafi Al Attrach, Rajna Fani, Ahram Han and 4 more

Paris Poster Session 3, Thu, Dec 10, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it

lenient 1/5

medium 0/10

strict 0/5

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026Princeton FAIR, Meta Agent benchmarks & environments

CEO-Bench: Can Agents Play the Long Game?

Haozhe Chen, Karthik Narasimhan, Zhuang Liu

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · Code ★ 82

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it

lenient 0/5

medium 0/10

strict 0/5

67%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Peking Michigan State Southeast University / Tongyi La U Hong Kong Zero Gravity Labs Agent benchmarks & environments

SaaS-Bench: Can Computer-Use Agents Leverage Real-World SaaS to Solve Professional Workflows?

Kean Shi, Zihang Li, Tianyi Ma, Zengji Tu and 11 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

2/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026U California, Santa Cruz U California, Berkeley UC Berkeley Agent benchmarks & environments

SudoBench: A Contextual Authorization Benchmark for LLM Agents

Vincent Siu, Tianneng Shi, Shangding Gu, Zhun Wang and 2 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026U California, San Diego The University of Hong Kong UCSD Agent benchmarks & environments

ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making

Liu Dai, Haina Wang, Weikang Wan, Hao Su

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it

lenient 1/5

medium 0/10

strict 0/5

67%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Hong Kong University of Science AlignBase The Hong Kong University of Scie Alibaba Group Tongji Agent benchmarks & environments

Are Agents Ready to Teach? A Multi-Stage Benchmark for Real-World Teaching Workflows

Zixin CHEN, Peng Liu, Rui SHENG, Haobo Li and 5 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

2/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026U Michigan - Ann Arbor U Pennsylvania, University of Pe Northern Arizona Peking U Hong Kong Agent benchmarks & environments

OS-Omni: A Cross-Platform Benchmark for Generalist Computer-Using Agents

Hui Shen, Yunta Hsieh, Jianing Ma, Ziyuan Liu and 36 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026Shanghai Jiaotong Xiamen Dalian University of Technology Nankai Harbin Engineering Agent benchmarks & environments

PCBInnoBench: Benchmarking LLM Agents on Real-World PCB Design

Fuyuan Xia, Jiaxin Hu, Kewei Yu, Zihan Zhang and 12 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026Princeton Arizona State Microsoft Agent benchmarks & environments

Synthetic Tasks for Training AutoResearch

Ziyang Cai, Seyyedamirhossein Saeidi, Harkirat Singh Behl

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026U Wisconsin, Madison Microsoft Agent benchmarks & environments

Synthetic Web: Benchmarking Language Agents under Adversarial Search Ranking

Shrey Shah, Levent Ozgur

Atlanta Poster Session 1, Wed, Dec 9, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it

lenient 0/5

medium 0/10

strict 0/5

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026Renmin University of China Columbia Beijing Jiaotong U Science and Technology of Chin Beijing Academy of Artificial In Agent benchmarks & environments

SciResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery

Lei Xiong, Kun Luo, Ziyi Xia, Wenbo Zhang and 5 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it

lenient 1/5

medium 0/10

strict 0/5

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026Pivotal Research, University of U Notre Dame Hebrew Agent benchmarks & environments

ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping

Joseph Marvin Imperial, Daniel Slate, Noam Kolt

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

67%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Technical University of Munich Technische Universität München Imperial College London U Oxford Institute of Computer Science, L Agent benchmarks & environments

MedFlowBench: Auditing Medical Agents in Full-Study Workflows

Weixiang Shen, Chengzhi Shen, Che Liu, Junde Wu and 10 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

2/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 2 of 20 reviewers recommend it

lenient 2/5

medium 0/10

strict 0/5

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026Westlake Uniforce AI Zhejiang U Glasgow Agent benchmarks & environments

Evaluating Physical Reasoning in LLM Agents Requires Construction Benchmarks

Wenhao Deng, Tian Xia, Yuru Jiang, Joemon M Jose and 1 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it

lenient 1/5

medium 0/10

strict 0/5

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026U Wisconsin - Madison U Tokyo / SB Intuitions Columbia DeepMind Princeton Agent benchmarks & environments

DEBATE: A Large-Scale Benchmark for Evaluating Opinion Dynamics in Role-Playing LLM Agents

Yun-Shiuan Chuang, Ruixuan Tu, Chengtao Dai, You Li and 7 more

Atlanta Poster Session 4, Thu, Dec 10, 4:30 PM–7:30 PM, Hall C1 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026Amazon AGI AWS AI Technical University of Munich AWS Amazon AWS Agent benchmarks & environments

CREF: Forecasting Benchmarks for the Age of Agents

Andreas Auer, Abdul Fatir Ansari, Oleksandr Shchur, Xiyuan Zhang and 5 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

67%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Shanghai Jiaotong Shanghai Jiao Tong Shanghai Artificial Intelligence Agent benchmarks & environments

ABHBench: Evaluating Moral Decision-Making of Foundation Models from an Agentic Perspective

Chaoran Li, Yukun Li, Zeyuan Zhao, Shao Zhang and 2 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

2/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 2 of 20 reviewers recommend it

lenient 1/5

medium 1/10

strict 0/5

67%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Macao Polytechnic Southwest Medical Radboud University Medical Cente Agent benchmarks & environments

Task Success Is Not Enough: Side-effect-Aware Evaluation of Tool-Using Language Model Agents

Jiaju Huang, Shaobin Chen, Xinglong Liang, Xinyu Ma and 4 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

2/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026The University of North Carolina U Illinois at Urbana-Champaign USTC UNC-Chapel Hill Shanghai Jiao Tong Agent benchmarks & environments

See, Read, Compare: Candidate-Aware Verification for Agent Test-Time Scaling

Xinyu Ye, Yongliang Wu, Xingyu Zhu, Peng Xia and 6 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026Georgia Institute of Technology Agent benchmarks & environments

PhysAgentGym: Free Physical-Law Verifiers for Training Small Code-Reasoning Agents

Lama Moukheiber

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026Huawei Technologies Ltd.Harbin Institute of Technology Sichuan Northwest Polytechnical Universi Huawei Agent benchmarks & environments

ClawBenchPro: Benchmarking How Well Agent Harnesses Work

Yufei Liu, Yirong Zeng, Yuxiang He, Yutai Hou and 9 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

67%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026LG Corporation LG AI Research Agent benchmarks & environments

From Benchmark to Adoption: Coding Agent Evaluation Needs User-Level Harness

Dasol Hong, Ji Yong Cho, Bumsoo Kang, Moontae Lee

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

2/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026U Oxford Agent benchmarks & environments

TrialAgentBench: Evaluating AI Agents for Clinical-Trial Analysis and Long-Horizon Drug-Development Decisions

Bradley M Segal, William J Bolton, Philip Torr, David Clifton and 1 more

Paris Poster Session 3, Thu, Dec 10, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it

lenient 1/5

medium 0/10

strict 0/5

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026U California, Davis Microsoft Amazon UC Davis Agent benchmarks & environments

ContractBench: Can LLM Agents Preserve Observation Contracts?

Jicheng Wang, Yifeng He, Zili Wang, Hanwen Xing and 2 more

Atlanta Poster Session 6, Fri, Dec 11, 4:30 PM–7:30 PM, Hall C1 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 0 of 20 reviewers recommend it

lenient 0/5

medium 0/10

strict 0/5

45%Niche pick

?Niche pickVote to see the score

NeurIPS 2026U Southern California Agent benchmarks & environments

DDBench: A Benchmark for Agentic Debugging on Distributed Systems

Yibo Yan, Huijuan Wang, Junzhou He, Yizhuo Liang and 3 more

Atlanta Poster Session 5, Fri, Dec 11, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

0/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026U Illinois Chicago U Illinois at Chicago Intel Labs Intel Pacific Northwest National Labor Agent benchmarks & environments

Argus: A Cross-Regime Benchmark for the Transferability of Uncertainty Quantification in Computer-Use Agents

DIVAKE KUMAR, Sina Tayebati, Devashri Naik, Amanda Rios and 4 more

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

69%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026National University of Singapore Agent benchmarks & environments

Is Agentic AI Ready for Real-World Hardware Engineering? A Deep Dive with Phoenix-bench

Qingyun Zou, Feng Yu, Hongshi Tan, Jiahao Cui and 2 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

3/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026Esker / INSA Lyon / LIRIS Seoul National University, Seoul McGill Ohio State University, Columbus McGill University, McGill Agent benchmarks & environments

WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents

Imene Kerboua, Fatemeh Pesaran Zadeh, Xing Han Lu, Weijian Qi and 18 more

Paris Poster Session 4, Thu, Dec 10, 5:30 PM–7:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it

lenient 1/5

medium 0/10

strict 0/5

67%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026U Illinois Chicago U Illinois at Chicago Illinois Institute of Technology U Washington State University of New York at Agent benchmarks & environments

JARVIS-Bench: Benchmarking Personal Intelligence Agents on Long-Horizon Real-User Daily Traces

Weizhi Zhang, Wei-Chieh Huang, Yueqing Liang, Liwei Jiang and 36 more

Atlanta Poster Session 2, Wed, Dec 9, 4:30 PM–7:30 PM, Hall C1 · Published 2026

– ReadersNo votes yet

2/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026SpotlightMacau University of Science and Shanghai Artificial Intelligence Institute of Process Engineering Shanghai Jiao Tong Faculty of Electrical Engineerin Agent benchmarks & environments

BEAKER: An Expert-Curated Benchmark for Embodied Brains in Self-Driving Chemical Laboratories

Fei Lin, Tengchao Zhang, Ziyang Gong, Xiaotong Yu and 14 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

57%Worth a look

?Worth a lookVote to see the score

NeurIPS 2026McMaster Vector Institute Department of Computer Science, Carnegie Mellon MPI for Intelligent Systems, Tüb Agent benchmarks & environments

AM-Bench: A Unified Taxonomy and Evaluation Suite for Agentic Misalignment

Eric Zhang, Terry J Zhang, Chijioke Ugwuanyi, Jerick Shi and 2 more

Paris Poster Session 6, Fri, Dec 11, 2:30 PM–4:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

1/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 1 of 20 reviewers recommend it

lenient 1/5

medium 0/10

strict 0/5

80%Must read

?Must readVote to see the score

NeurIPS 2026shopify Shopify North Carolina State NC State Agent benchmarks & environments

ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agents

ShopGym converts live e-commerce sites into reproducible simulated shops and synthesizes diverse benchmark tasks, showing synthetic shops preserve live structural properties and agent performance correlations.

Yuanzheng Zhu, Mingyu Zhao, Chinmay Savadikar, Han Li and 4 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

12/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

91%Must read

?Must readVote to see the score

NeurIPS 2026DeepWisdom South China University of Techno SUN YAT-SEN UNIVERSITY Fudan The Hong Kong University of Scie Agent benchmarks & environments

DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agents

DiagEval uses trajectory-conditioned diagnostic probes to disambiguate evaluator errors from software defects in GUI-agent evaluations, recovering over 45% of misattributed failures and improving accuracy substantially.

Sirui Hong, Liuzhijie, Tengfei Li, Wei Tao and 2 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

17/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

80%Must read

?Must readVote to see the score

NeurIPS 2026School of Engineering and Applie Harvard MIT Fudan Massachusetts Institute of Techn Agent benchmarks & environments

AI GAMESTORE: Scalable, Open-Ended Evaluation of Machine General Intelligence with Human Games

AI GameStore proposes evaluating general intelligence via scalable synthesis of human games, finding frontier vision-language models score under 10% of human averages on most generated games.

Lance Ying, Ryan Truong, Prafull Sharma, Kaiya Zhao and 8 more

Atlanta Poster Session 6, Fri, Dec 11, 4:30 PM–7:30 PM, Hall C1 · Published 2026 · ▲ 9 on Hugging Face

– ReadersNo votes yet

12/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

76%Highly rated

?Highly ratedVote to see the score

NeurIPS 2025Princeton UT Austin Seoul National MAUM.AI, Soongsil Sirindhorn International Institu Agent benchmarks & environments

The PokeAgent Challenge: Competitive and Long-Context Learning at Scale

PokeAgent is a large-scale Pokémon benchmark with battling and speedrunning tracks that expose major gaps between LLMs, RL agents, and human experts.

Seth Karten, Jake Grigsby, Tersoo Upaa, Junik Bae and 27 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 12 on Hugging Face

– ReadersNo votes yet

10/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 10 of 20 reviewers recommend it

lenient 4/5

medium 6/10

strict 0/5

78%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Johns Hopkins Department of Computer Science, Lambda JHU Agent benchmarks & environments

AgentOdyssey: Open-Ended Long-Horizon Text Game Generation for Test-Time Continual Learning Agents

AgentOdyssey generates open-ended long-horizon text games to evaluate test-time continual learning, finding top agents far below human performance despite scaling with model strength.

Zheyuan Zhang, Zehao Wen, Bowei Zhang, Andrew Wang and 3 more

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026 · ▲ 8 on Hugging Face · Code ★ 55

– ReadersNo votes yet

11/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

88%Must read

?Must readVote to see the score

NeurIPS 2026XPeng Motors Southern University of Science a Fudan The Hong Kong Polytechnic Alibaba Group Agent benchmarks & environments

Done, But Not Sure: Disentangling World Completion from Self-Termination in Embodied Agents

VIGIL decouples world-state completion from terminal commitment in embodied agents, revealing that comparable execution yields up to 19.7 pp differences in correct episode termination.

Ying Chen, Lihuang Fang, Rui Jiang, Mingxu Wang and 3 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

80%Must read

?Must readVote to see the score

NeurIPS 2026ELLIS Institute Tübingen, MPI-IS Max Planck Institute for Intelli ELLIS, University of Stuttgart Eberhard-Karls-Universität Tübin Institute for AI, Univ Agent benchmarks & environments

FutureSim: Replaying World Events to Evaluate Adaptive Agents

FutureSim replays real-world events chronologically to benchmark adaptive AI agents forecasting future news, finding top accuracy at only 25%.

Shashwat Goel, Nikhil Chandak, Arvindh Arun, Ameya Prabhu and 4 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026 · ▲ 6 on Hugging Face · Code ★ 56

– ReadersNo votes yet

12/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

86%Must read

?Must readVote to see the score

NeurIPS 2026Beijing Institute of Technology Agent benchmarks & environments

MedEvoEval: Evaluating Continual Evolution of Doctor Agents through Simulated Clinical Episodes

MedEvoEval evaluates doctor agents through simulated longitudinal clinical episodes to measure experience-driven improvement, resource use, and capability retention over time.

Hui Zhang

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 14 of 20 reviewers recommend it

lenient 5/5

medium 7/10

strict 2/5

86%Must read

?Must readVote to see the score

NeurIPS 2026U Sydney Australian National Royal Melbourne Institute of Tec MINT lab, Australian National Johns Hopkins Agent benchmarks & environments

NORA: Evaluating Grounded Reasonableness in Visual First-person Normative Action Reasoning

NoRA evaluates visual first-person normative reasoning by requiring models to generate actions with fact-reason-action support graphs, revealing current VLMs struggle to bind correct justifications to actions.

Sichao Li, Sai Ma, Zhuang Li, Daniel Kilov and 2 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 14 of 20 reviewers recommend it

lenient 5/5

medium 7/10

strict 2/5

78%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Institute of Information Enginee Beijing University of Aeronautic CASIA Alibaba Group Agent benchmarks & environments

VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora

VeriTrip benchmarks travel planning agents via evidence-grounded reasoning over noisy multimodal web corpora, revealing a retrieval-reasoning trade-off that erodes instruction retention.

Yuting Xu, Jiayi Tian, Jian Liang, Xin Xiong and 3 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

11/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 11 of 20 reviewers recommend it

lenient 5/5

medium 5/10

strict 1/5

76%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Yonsei Korea Advanced Institute of Scie Sung Kyun Kwan Texas A&M University - Colle Samsung Agent benchmarks & environments

Towards Direct Evaluation of Harness Optimizers via Priority Ranking

Priority ranking directly evaluates harness optimizers by ranking harness components via potential performance impact, correlating with multi-step optimization ability without costly rollouts.

Kai Ong, Minseok Kang, Dongwook Choi, Junhee Cho and 10 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 2 on Hugging Face

– ReadersNo votes yet

10/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

86%Must read

?Must readVote to see the score

NeurIPS 2026U Pennsylvania Scale AI Dyna Robotics U Nevada, Las Vegas U Arizona Agent benchmarks & environments

SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution

SWE Atlas benchmarks coding agents on codebase Q&A, test writing, and refactoring, finding frontier models lead but all struggle with edge cases and engineering quality.

Mohit Raghavendra, Soham Dan, Miguel Romero Calvo, Yannis He and 11 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

89%Must read

?Must readVote to see the score

NeurIPS 2026Australian Institute for Machine Southeast Hunan U Adelaide Singapore University of Technolo Agent benchmarks & environments

LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation

LangMap introduces human-verified hierarchical open-vocabulary navigation benchmarks across scene, room, region, and instance levels with 18K tasks, and PlaNaVid achieves top RGB-only success via planning and memory.

Bo Miao, Weijia Liu, Jun Luo, Lachlan Shinnick and 7 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 1 on Hugging Face · Code ★ 53

– ReadersNo votes yet

16/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

76%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026City University of Hong Kong The Chinese University of Hong K The Chinese University of Hong K Shanghai University of Finance a Agent benchmarks & environments

LiveOption: Evaluating LLM Agents in Structured Option Trading with Nonlinear Payoffs

LiveOption evaluates LLM option-trading agents via structured sequential decision-making with nonlinear payoffs, showing current agents rarely achieve competitive returns.

Haochen Luo, YIFAN LI, An B Minh, Xiaolong Luo and 3 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

10/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 10 of 20 reviewers recommend it

lenient 5/5

medium 4/10

strict 1/5

89%Must read

?Must readVote to see the score

NeurIPS 2026Renmin University of China Beijing Institute of Technology Aptura AI Columbia Massachusetts Institute of Techn Agent benchmarks & environments

SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

SpreadsheetBench 2 evaluates agents on end-to-end spreadsheet workflows, finding best models achieve only 34.89% accuracy with debugging at 12%.

Jian Zhu, Yuzheng Zhang, Zeyao Ma, Bohan Zhang and 10 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · Code ★ 39

– ReadersNo votes yet

16/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 16 of 20 reviewers recommend it

lenient 5/5

medium 9/10

strict 2/5

83%Must read

?Must readVote to see the score

NeurIPS 2026GRADIENT DATA Soochow Independent Researcher Amazon Microsoft Agent benchmarks & environments

TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routing

TwinRouterBench introduces static and live dynamic tracks to benchmark LLM routing at agent step-level using deterministic scoring and live execution on SWE-bench.

Pei Yang, Wanyi Chen, Tongyun Yang, Pengbin Feng and 11 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026 · ▲ 5 on Hugging Face

– ReadersNo votes yet

13/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

88%Must read

?Must readVote to see the score

NeurIPS 2026U Hong Kong The University of Hong Kong The Chinese University of Hong K The Chinese University of Hong K Agent benchmarks & environments

UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks

UniClawBench introduces a capability-driven benchmark evaluating proactive agents via 400 real-world tasks with live Docker evaluation and multi-turn feedback.

Zhekai Chen, CHENGQI DUAN, Kaiyue Sun, Bohao Li and 3 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026 · ▲ 34 on Hugging Face · Code ★ 39

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

92%Must read

?Must readVote to see the score

NeurIPS 2026OralProgramma Labs New York Facebook FAIR, Meta AI Meta FAIR Agent benchmarks & environments

Computer Use at the Edge of the Statistical Precipice

A 1MB replay script outperforms frontier agents on static benchmarks because of flawed environment design and evaluation; the paper proposes PRISM principles, DigiWorld, and hierarchical bootstrap aggregation to fix both.

Pierluca D Oro, Sneha Silwal, William R Wong, Yuxuan Sun and 5 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

19/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

89%Must read

?Must readVote to see the score

NeurIPS 2026Yale U California, Berkeley Bloomberg Rutgers Agent benchmarks & environments

PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarios

PieArena benchmarks LLM negotiation via multi-agent MBA scenarios, ranking agents with order-invariant payoffs and finding GPT-5 matches trained human baselines while profiling cross-model behavioral heterogeneity.

Chris Zhu, Sasha Cui, Will S Dufallo, Runzhi Jin and 3 more

Paris Poster Session 6, Fri, Dec 11, 2:30 PM–4:30 PM, Paris Poster Hall · Published 2026

– ReadersNo votes yet

16/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

92%Must read

?Must readVote to see the score

ICLR 2026WorkshopInternational Business Machines Hebrew University of Jerusalem, IBM Research IBM MIT, IBM Agent benchmarks & environments

General Agent Evaluation

A systematic comparison of general agent architectures finds backbone choice dominates performance while architecture shifts results up to 12pp, and open models suffer generality sinks.

Elron Bandel, Asaf Yehudai, Lilach Edelstein, Yehoshua Sagron and 11 more

Paris Poster Session 5, Fri, Dec 11, 11:30 AM–1:30 PM, Paris Poster Hall · Published 2026 · ▲ 14 on Hugging Face · Code ★ 76

– ReadersNo votes yet

19/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

88%Must read

?Must readVote to see the score

NeurIPS 2026Université de Montréal Google DeepMind / U. de Montreal Mila - Québec AI Institute &Agent benchmarks & environments

Agentick: A Unified Benchmark for General Sequential Decision-Making Agents

Agentick unifies RL and foundation model agent evaluation across 37 tasks, finding no dominant approach and substantial room for improvement.

Roger Creus Castanyer, Pablo Samuel Castro, Glen Berseth

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

88%Must read

?Must readVote to see the score

NeurIPS 2026Handshake AI Handshake Independent Collinear AI UC Berkeley Agent benchmarks & environments

BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows

BankerToolBench benchmarks AI agents on multi-hour investment banking workflows using expert rubrics, finding frontier models fail nearly half of criteria with zero client-ready outputs.

Elaine Lau, Markus Dücker, Ronak Chaudhary, Hui Wen Goh and 24 more

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

89%Must read

?Must readVote to see the score

NeurIPS 2026BenchFlow Ohio State University, Columbus Amazon Dartmouth College U California, Davis Agent benchmarks & environments

SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks

SkillsBench benchmarks agent skills across 87 tasks, finding curated skills boost pass rates by 16.6 points, with focused small bundles often outperforming larger ones.

Xiangyi Li, Yimin Liu, Wenbo Chen, Shenghan Zheng and 36 more

Atlanta Poster Session 3, Thu, Dec 10, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

16/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

91%Must read

?Must readVote to see the score

NeurIPS 2026Cornell Cornell Tech Agent benchmarks & environments

Agent Meltdowns: The Road to Hell Is Paved with Helpful Agents

Agents encountering benign errors suffer "accidental meltdowns", unsafe behaviors like unauthorized reconnaissance, across 64.7% of error rollouts, often unreported.

Rishi Jha, Harold Triedman, Vitaly Shmatikov, Arkaprabha Bhattacharya

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

18/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 18 of 20 reviewers recommend it

lenient 5/5

medium 9/10

strict 4/5

91%Must read

?Must readVote to see the score

NeurIPS 2026Peking U Hong Kong Fudan Department of Computer Science, Agent benchmarks & environments

Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents

Claw-Eval introduces a trajectory-aware benchmark with 300 tasks, finding opaque grading misses 44% of safety violations and agent rankings vary across multi-dimensional capabilities.

Bowen Ye, Rang Li, Qibin Yang, Yuanxin Liu and 9 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026 · ▲ 117 on Hugging Face · Code ★ 778

– ReadersNo votes yet

18/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 18 of 20 reviewers recommend it

lenient 5/5

medium 9/10

strict 4/5

86%Must read

?Must readVote to see the score

NeurIPS 2026Technische Universität Darmstadt Technical University of Darmstad Cornell Technical University Darmstadt TU Darmstadt Agent benchmarks & environments

Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Induction

ZendoWorld evaluates AI agents on active visual rule induction and finds high prediction accuracy does not imply rule recovery, with VLM agents proposing near-uninformative experiments.

Sophia Koehler, Antonia Wüst, Inga Ibs, Top Piriyakulkij and 4 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 14 of 20 reviewers recommend it

lenient 5/5

medium 7/10

strict 2/5

83%Must read

?Must readVote to see the score

NeurIPS 2026Northeastern Avid ML U Michigan - Ann Arbor AT&T AT&T Labs Research Agent benchmarks & environments

Consistency as a Testable Property: Statistical Methods to Evaluate AI Agent Reliability

The paper proposes statistical consistency metrics for AI agents that reveal strategy breakdowns hidden by standard pass rates, isolating architectural reliability flaws.

Harsh Raj, Niranjan Orkat, Suvrorup Mukherjee, Aritra Guha and 2 more

Sydney Poster Session 4, Wed, Dec 9, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

13/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 13 of 20 reviewers recommend it

lenient 5/5

medium 7/10

strict 1/5

83%Must read

?Must readVote to see the score

NeurIPS 2026Tsinghua Carnegie Mellon CMU, Carnegie Mellon School of Computer Science, Carn Meta Agent benchmarks & environments

Evaluating Test-Time Scaling of General LLM Agents

Realistic benchmark reveals LLM agents suffer scaling plateaus and verification gaps that prevent meaningful test-time compute gains.

Xiaochuan Li, Tianshi Ming, Pranav Setlur, Abhijay S Paladugu and 5 more

Sydney Poster Session 6, Thu, Dec 10, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026 · ▲ 10 on Hugging Face · Code ★ 25

– ReadersNo votes yet

13/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 13 of 20 reviewers recommend it

lenient 5/5

medium 7/10

strict 1/5

86%Must read

?Must readVote to see the score

NeurIPS 2026Zhejiang Beijing University of Aeronautic The Chinese University of Hong K Shandong nanjing university Agent benchmarks & environments

EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies

EcoGym benchmarks long-horizon LLM economic planning across open-source environments, revealing no single model dominates and exposing strategic and execution suboptimalities.

Xueyu Hu, Jinxiang Xia, Shengze Xu, Kangqi Song and 12 more

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 11 on Hugging Face · Code ★ 99

– ReadersNo votes yet

14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 14 of 20 reviewers recommend it

lenient 5/5

medium 8/10

strict 1/5

89%Must read

?Must readVote to see the score

NeurIPS 2026Korea Sookmyung Women's Agent benchmarks & environments

AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering

AgentHop diagnoses agent failures via multi-hop scientific QA under constraints, finding model-family tool-use fingerprints and hidden within-family differences.

Chanhee Park, Jeongho Yoon, Sungbin Han, Hyeonseok Moon and 1 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

16/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 16 of 20 reviewers recommend it

lenient 5/5

medium 9/10

strict 2/5

78%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Eigent AI Hassana Labs Sambanova Systems King Abdullah University of Scie Camel AI Agent benchmarks & environments

SETA: Scaling Environments for Terminal Agents

SETA scales verifiable terminal RL environments via synthesis and evolution, yielding 4,500+ tasks and boosting 8B terminal-agent pass rates to 12%.

Qijia Shen, Zhiqi Huang, Vamsidhar Kamanuru, Aznaur Aliev and 14 more

Sydney Poster Session 1, Tue, Dec 8, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

11/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 11 of 20 reviewers recommend it

lenient 5/5

medium 5/10

strict 1/5

88%Must read

?Must readVote to see the score

NeurIPS 2026CMU Carnegie Mellon Agent benchmarks & environments

Gym-Anything: Turn Any Software into an Agent Environment

Gym-Anything converts any software into interactive agent environments via multi-agent setup and auditing, yielding CUA-World with 10K long-horizon tasks and improved agent performance.

Pranjal Aggarwal, Graham Neubig, Sean Welleck

Sydney Poster Session 3, Wed, Dec 9, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026 · ▲ 1 on Hugging Face

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 15 of 20 reviewers recommend it

lenient 5/5

medium 8/10

strict 2/5

88%Must read

?Must readVote to see the score

NeurIPS 2026National University of Singapore Tsinghua National university of Singapore Suzhou Automobile Research Insti Meta Agent benchmarks & environments

FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics

FML-bench isolates agent strategy from infrastructure across 18 ML tasks, finding greedy hill-climbing nearly matches tree search, while adaptive exploration switching outperforms fixed strategies.

Qiran Zou, Hou Hei Lam, Wenhao Zhao, Tingting Chen and 10 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 15 of 20 reviewers recommend it

lenient 5/5

medium 9/10

strict 1/5

89%Must read

?Must readVote to see the score

NeurIPS 2026Indian Institute of Technology, Zillow Group, Inc.Agent benchmarks & environments

Proper Scoring Rules for Agentic Uncertainty Quantification

Trajectory Proper Score is a strictly proper family of trajectory-level scoring rules that elicits full prefix-conditioned success probabilities, unlike resolution-blind calibration or collapsed scalar metrics.

Suresh Raghu, Satwik Pandey, Shashwat Pandey

Sydney Poster Session 5, Thu, Dec 10, 10:00 AM–1:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

16/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 16 of 20 reviewers recommend it

lenient 3/5

medium 10/10

strict 3/5

86%Must read

?Must readVote to see the score

NeurIPS 2026Shandong U Illinois Urbana-Champaign nanjing university Shandong University of Triaditio Sichuan Agent benchmarks & environments

OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectories

OpenClawBench benchmarks process-side agent anomalies via 31,264 annotated trajectories, revealing 2,904 process failures among 31,135 oracle-passing executions.

Yibing Liu, Yangze Liu, Xiao-Long Yin, Bin Wang and 3 more

Sydney Poster Session 2, Tue, Dec 8, 5:00 PM–8:00 PM, Hall 1-4 · Published 2026

– ReadersNo votes yet

14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 14 of 20 reviewers recommend it

lenient 5/5

medium 7/10

strict 2/5

86%Must read

?Must readVote to see the score

NeurIPS 2026UNC-Chapel Hill Tongji Agent benchmarks & environments

DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution

DriveHierarchy hierarchically benchmarks VLM driving across four ranks from perception to closed-loop execution, linking open-loop understanding to embodied performance for diagnosing 15 models.

Chengkai Xu, Jiaqi Liu, Yicheng Guo, Peng Hang and 1 more

Paris Poster Session 3, Thu, Dec 10, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026 · ▲ 1 on Hugging Face

– ReadersNo votes yet

14/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 14 of 20 reviewers recommend it

lenient 5/5

medium 9/10

strict 0/5

83%Must read

?Must readVote to see the score

NeurIPS 2026Renmin University of China Kuaishou- 快手科技 Kuaishou Agent benchmarks & environments

GISA: A Benchmark for General Information-Seeking Assistant

GISA introduces 373 human-crafted information-seeking queries with structured answers, live updates, and search trajectories to benchmark autonomous search agents, revealing state-of-the-art models achieve under 20% accuracy.

Yutao Zhu, Xingshuo Zhang, Maosen Zhang, Jiajie Jin and 8 more

Paris Poster Session 1, Wed, Dec 9, 12:30 PM–2:30 PM, Paris Poster Hall · Published 2026 · ▲ 26 on Hugging Face · Code ★ 37

– ReadersNo votes yet

13/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 13 of 20 reviewers recommend it

lenient 5/5

medium 6/10

strict 2/5

88%Must read

?Must readVote to see the score

NeurIPS 2026Stanford Division of Hospital Medicine St Agent benchmarks & environments

PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments

PhysicianBench evaluates LLM agents on 100 real-world EHR tasks across 21 specialties, finding top models achieve only 46% success.

Ruoqi Liu, Imran Mohiuddin, Austin J Schoeffler, Kavita Renduchintala and 9 more

Atlanta Poster Session 5, Fri, Dec 11, 10:00 AM–1:00 PM, Hall C1 · Published 2026 · ▲ 8 on Hugging Face · Code ★ 59

– ReadersNo votes yet

15/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 15 of 20 reviewers recommend it

lenient 5/5

medium 7/10

strict 3/5

78%Highly rated

?Highly ratedVote to see the score

NeurIPS 2026Skyfall.AI U Toronto Tufts The University of Texas at Dalla SkyfallAI Agent benchmarks & environments

Mini Amusement Parks (MAPs): A Testbed for Modelling Business Decisions

MAPs introduces a mini amusement-park simulator benchmarking integrated business decision-making, finding experts outperform state-of-the-art agents by over 11x due to weaknesses in long-horizon planning, sample-efficient learning, and spatial reasoning.

Stéphane Aroca-Ouellette, Ian Berlot-Attwell, Panagiotis Lymperopoulos, Abhiramon Rajasekharan and 4 more

Atlanta Poster Session 1, Wed, Dec 9, 10:00 AM–1:00 PM, Hall C1 · Published 2026

– ReadersNo votes yet

11/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 11 of 20 reviewers recommend it

lenient 5/5

medium 6/10

strict 0/5

91%Must read

?Must readVote to see the score

NeurIPS 2026OralU California, Berkeley Tsinghua Tsinghua University, Tsinghua Purdue U Washington Agent benchmarks & environments

MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI

MLS-Bench evaluates AI agents on inventing scalable ML methods across 140 tasks, finding current systems fail to reliably surpass human-designed approaches due to insufficient scientific validation insight.

Bohan Lyu, Yucheng Yang, Siqiao Huang, Jiaru Zhang and 24 more

Atlanta Poster Session 2, Wed, Dec 9, 4:30 PM–7:30 PM, Hall C1 · Published 2026 · ▲ 8 on Hugging Face · Code ★ 120

– ReadersNo votes yet

17/20 AI panelreviewers recommend it

Readers and the AI panel: vote on this paper to see what they said.

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel: 17 of 20 reviewers recommend it

lenient 5/5

medium 8/10

strict 4/5

91%Must read

?Must readVote to see the score

NeurIPS 2026University College London nanjing university Nanjing ByteDance Facebook Agent benchmarks & environments

TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks

TerminalWorld automatically builds terminal benchmarks from wild recordings, yielding 1,530 tasks where top agents achieve only 62.5% success with weak correlation to expert benchmarks.

Zhaoyang Chu, Jiarui Hu, Xingyu Jiang, Pengyu Zou and 7 more

Paris Poster Session 6, Fri, Dec 11, 2:30 PM–4:30 PM, Paris Poster Hall · Published 2026 · ▲ 8 on Hugging Face · Code ★ 48