Released Apr 2024

Record sheet · No. 015
SWE-agent
A research project that argued the model was never the bottleneck; the interface was.
GitHub avatar of SWE-agent, not the project’s own logo — taken from github.com on 2026-10-01.

What it is
A research agent that takes a GitHub issue and tries to fix it, built around commands and feedback formats designed for a model to read: it reported 12.29% of issues resolved on SWE-bench.
Its actual contribution is the Agent-Computer Interface, a concept and not a product: the argument that models fail at software engineering partly because the tools they are given were designed for humans. The team redesigned the commands and feedback formats around the model, and reported resolving 12.29% of issues on SWE-bench, state of the art at the time.
Who built it
SWE-agent comes out of a Princeton research group rather than a product team: the NeurIPS 2024 paper lists seven authors — John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik R. Narasimhan and Ofir Press — and the repository’s first commit is by Jimenez. Six of the seven also wrote SWE-bench, the benchmark the 12.29% figure is measured on, which the same group published in October 2023 (arXiv:2310.06770). The repository has since taken 101 contributors, and the group’s follow-up EnIGMA extends the same interface idea to capture-the-flag security tasks with researchers from Tel Aviv University and New York University.
Build log
3 stages- 01
Reframing the problem as an interface problem
The research question was not “which model is strongest” but “what should the model be looking at”. The team designed purpose-built commands and output formats so a language model could browse a repository, view, edit and execute code without wading through terminal output meant for a person. They named this the Agent-Computer Interface.
- 02
A number, published with a date on it
On the SWE-bench test set the agent resolved 12.29% of issues, state of the art in April 2024. That figure is two years old at the time of writing and should always be quoted with the date attached: a benchmark score without a date reads as a current claim.
- 03
The researchers now recommend their own successor
The documentation for SWE-agent carries a banner at the top recommending mini-swe-agent instead, describing it as the same performance with far more simplicity and flexibility. The original is still maintained, and the people who wrote it point new users elsewhere.
What they would tell you
- When progress stalls, the interface between the model and the work is a legitimate place to look — not just the model.
- A benchmark result is a claim with an expiry date, and should be published with one.
- A research project that recommends its own replacement is doing its job, not failing.
Adjacent records
All records →No. 128
sepia
A portable de-AI writing skill: four operations over one canonical rules file, narrative architecture repaired before word choice on fiction, a thin rule file matched to the venue on professional prose, and every rule labelled as measured, consulted or the project’s own inference.
No. 127
Lemmalog
A Rust Datalog engine that treats agent memory as a deductive database, not a bigger vector store: facts asserted at the extraction boundary, stratified rules deriving closures and temporal views, provenance back to the source episode on every derived fact, and views maintained one epoch at a time — served to Claude Code and Kimi CLI as twelve MCP tools.
No. 143
AntOmniEvo
A Python framework that runs an evolution loop over a directory of your files: a coding agent rewrites them from the failure trajectories of the previous candidate, parent and child are scored on the same training batch, and only an improvement that clears a threshold is re-scored on the validation set and kept.


