Research
We publish what we find, including what did not work.
Sixty-two papers since 2023, covering four programmes. Code, datasets and evaluation harnesses are released with the work wherever a licence allows.
Long-horizon reasoning
Most agent failures are not reasoning failures. They are bookkeeping failures: state that was forgotten, a plan that was never revised, an error that was silently absorbed. We study the mechanics of staying coherent over tens of thousands of tokens of accumulated context.
Our current approach separates a slow planner from a fast executor and lets the planner rewrite its own scratchpad when a sub-goal fails. On unattended data-engineering tasks this raises end-to-end completion from 43.2% to 71.4% at equal inference cost.
Selected work
- Sierra: an agent that recovers from its own failures
- Scratchpad rewriting under distribution shift
- How far can a plan survive? Context rollover in practice
Tool use & embodied control
A model that can call a tool is not the same as a model that knows when to stop calling it. We train and evaluate agents against real interfaces — a browser, a terminal, a set of production APIs — and measure the cost of the wrong call, not just the accuracy of the right one.
- Calibrated tool selection with an explicit abstention action
- Terminal-2: a benchmark for reversible and irreversible actions
Evaluation & interpretability
Benchmarks decay. A suite that discriminated well in 2024 may carry almost no information about a 2026 model family. We build estimators for that decay, and we apply dictionary learning to whole agent trajectories rather than to single activations.
- Tessera: measuring what an evaluation set no longer tests
- Harbour: sparse features of agent trajectories
- Predicting specification violations from internal state
Safe deployment
Deployment is where research meets consequences. Our hosted agents run in a sandbox by default, every release passes a documented review, and we publish incident reports when a review misses something.
- Capability thresholds as a release gate: two years in production
- Monitoring for drift in long-running agents
All publications
| Title | Venue | Year |
|---|---|---|
| Sierra: an agent that recovers from its own failures | Preprint | 2026 |
| Tessera: measuring what an evaluation set no longer tests | NeurIPS | 2026 |
| Harbour: sparse features of agent trajectories | ICML | 2026 |
| Capability thresholds as a release gate | Preprint | 2026 |
| Monitoring for drift in long-running agents | MLSys | 2026 |
| Calibrated tool selection with an explicit abstention action | Preprint | 2026 |
| Scratchpad rewriting under distribution shift | ICML | 2026 |
| Terminal-2: reversible and irreversible actions | NeurIPS D&B | 2025 |
| Predicting specification violations from internal state | Preprint | 2025 |
| Context rollover without plan collapse | ICLR | 2025 |
| An empirical account of tool overuse | EMNLP | 2025 |
| Reward hacking in tool-augmented training loops | NeurIPS | 2024 |
Showing 12 of 62. Full list available on request or via our arXiv listing.
Code & datasets
We release evaluation harnesses and training data under Apache 2.0 wherever third-party terms allow. Model weights are covered separately — see the models page.
agentbench-2— evaluation harness, 41 tasks, Apache 2.0terminal-2— reversible/irreversible action suite, CC-BY 4.0trajectory-sae— dictionary learning over trajectories, Apache 2.0recovery-20m— annotated failure trajectories, research licence