Can Exo improve itself during a task?
Exo made zero self-edits and passed zero tasks.
9 Sept 2026 · 6 Terminal-Bench 4.0 tasks · GPT-5.6 Luna, xhigh reasoning
Task outcomes
Loading task outcomes…
Three failed, one hit an error, and two were blocked when that error stopped the run. No source edits or rebuilds.
Background
Exo can edit its own source, rebuild, and resume the same conversation. That could help when an agent is stuck with the wrong tools. We tested whether that freedom leads to self-improvement during coding tasks.
Methodology
GPT-5.6 Luna at xhigh reasoning got six Terminal-Bench 4.0 development tasks, one attempt each. Each task started with a fresh, writable Exo and a prompt to read SELF.md and improve itself when useful. Changes did not carry over between tasks. We tracked source edits, tool changes, rebuilds, and task results separately, with a one-hour limit per task.
Results
Exo made no source edits and no rebuilds. All three scored tasks failed. The fourth hit an error that stopped the run, so the last two never started. A setup check had already shown that editing, rebuilding, and resuming worked; Exo just never used it on a task.
Discussion
This is a failed attempt to trigger useful self-editing, not proof that self-editing agents can’t work. The tasks were seen development cases, not a holdout, and the error cut the run short. Errors and blocked tasks are not scored failures. Without a fixed-runtime control, we can’t measure any benefit from source access alone.
Conclusion
Writable source is not self-improvement. First get the agent to make and use a relevant change on a real task, then compare it with the same agent on a fixed runtime. This setup never cleared the first step.
Data and run details
Loading run status…
Results
Errors and blocked tasks are not scored failures. Scroll sideways for usage and rebuilds.
| Task | Status | Reward | Cost¹ | Time | Calls | Tokens (in / cached / out) | Source changes | Rebuilds (ok / failed / queued) | Notes |
|---|
Method
Each task gets a fresh, writable Exo that can edit its source, rebuild, and resume the same conversation. Changes do not carry between tasks.
The six tasks are seen development cases, not a holdout. Source access alone is not improvement, so tool edits, source edits, and rebuilds are counted separately.
Runtime and task limits
Exo uses its practical profile: shell, inspect_tools, manage_tool, and rebuild_and_restart_exo. The guardian builds its writable clone, and the driver resumes it after a rebuild.
Terminal-Bench images, tasks, resources, and verifiers stay fixed. Harbor attaches the task sandbox, and the driver asks Exo to read SELF.md and improve when useful.
The driver enforces a one-hour task limit, an estimated $2 cap per task, and two provider retries. An infrastructure or provider failure stops the remaining tasks; a scored task failure does not.
This run uses xhigh reasoning. Earlier max-reasoning trials are not pooled with it.
Setup checks and cost
Setup fixed the restart path, guardian cleanup, and scheduler secret store. Keys stayed outside the sandbox.
The first setup check rebuilt but could not resume through macOS Keychain ($0.0039). The secret-store fix came before any benchmark task.
¹ Costs use provider-reported usage where available, otherwise Luna pricing ($0.20 input, $0.02 cached input, $1.20 output per million tokens). In-flight calls count only once usage arrives, so a cost cap can overshoot by one call. Setup spend is separate.