Does encrypted compaction help in coding tasks?
Encrypted compaction cut token use by 29%.
25 Sept 2026 · 20 Terminal-Bench 2.1 tasks · GPT-6 Luna, high reasoning · version 1, replaced by version 2
Pass rate, tokens, and cost
Background
When a coding agent runs out of context, should it keep a text summary or the provider’s encrypted state? This first pass compared both on GPT-6 Luna. It found a useful resource difference, but the quality question needed a better experiment.
Methodology
Pi 0.80.10 ran 20 selected Terminal-Bench 2.1 tasks once per method in fresh containers, with high reasoning and compaction at 50000 tokens. We picked tasks from their instructions and resource metadata, not model scores. Resource means use tasks where both methods scored, and cost includes task and summary calls at OpenAI’s published rates.
Results
Encrypted compaction used 1.03 million tokens per task versus 1.45 million for textual, 29% fewer, and cost 15% less ($0.0382 versus $0.0449). Encrypted passed 80% of tasks to textual’s 70%, but only four tasks had different outcomes: three encrypted-only passes and one textual-only.
Discussion
Eight tasks never compacted, and two of the four differing outcomes came from those. The pass-rate gap had a 95% interval of −10 to +30 points. We also found setup differences: the encrypted method dropped Pi’s developer message, and the textual method got an extra finish-and-verify instruction. That makes this a poor test of quality.
Conclusion
Treat the token saving as a reason to test encrypted compaction, not the 80% as proof it codes better. Version 2 adds repeats, fixes the prompt differences, and separates tasks that actually compact. Read version 2 for the current comparison.
Data and run details
Partial data
Results
Only compactions followed by another model request count as qualified.
| Task | Textual | Encrypted | Outcome |
|---|
Method
Pi 0.80.10 runs 20 tasks with GPT-6 Luna at high reasoning. At 50000 tokens, the textual method keeps a Pi summary and the encrypted method keeps OpenAI’s encrypted state. Each task runs once per method in a fresh container.
Tasks are the original seven plus thirteen longer implementation, debugging, scientific-computing, and build tasks. We picked them from task instructions and resource metadata, not GPT-6 results.
Scoring and cost
Headline scores and resource means use tasks where both methods scored. Each method’s card also shows all its completed results. Compaction counts separate observed events from those followed by another agent request.
Costs include task and summary calls at OpenAI’s published standard rates. Cached input counts once in token totals. Spend includes preflight calls; no GPU rental was needed. Errors are counted separately from verifier failures.
Tasks keep their original timeouts and resource limits, with four attempts at a time and one run per method per task. This selected public subset is not an official leaderboard result. Tasks: Terminal-Bench 2.1 (license).