r/codex • u/MattAlreadyHere • 13h ago
Showcase Tool-output compression for Codex CLI on Terminal-Bench 2.1: −22.7% on the API bill, no significant change in solve rate (open source, MIT)
Hello everyone! I work at Hystax and we built the compressor (open source, MIT). The benchmark ran on Codex CLI, so the numbers seemed worth sharing here.
Why compress at all. Codex re-sends the whole conversation every turn, including every apt-get log and progress bar it has already read. In our traces, tool output is about half of all prompt bytes. About 93% of input tokens are cache reads at a tenth of the price, so a compressor has to produce byte-identical output every turn. Otherwise it re-prices the conversation and costs more than it saves. Ours is rule-based and deterministic: no ML, no LLM calls.
Setup. Terminal-Bench 2.1, 89 tasks, Codex CLI on GPT-5.5, 3 concurrent trials via Harbor. We compared actual provider bills with the cache discount included. Figures are per full run over the 87 tasks every arm completed.
| Per full run | No compression | With compression |
|---|---|---|
| Cost | $107.89 | $83.40 (−22.7%) |
| Input tokens | 96.1M | 72.2M (−25%) |
| Tasks solved | 82.2% | 81.1% (not significant) |
| Median trial time | 4.8 min | 4.5 min |
Two trials had to compact their context without compression; one did with it.
Where it helps. Building from source −28%, ML training −26%. Algorithm/puzzle tasks −8%, reverse engineering −5%. If your tasks are short with small outputs, expect little.
Codex-specific:
- Codex wraps tool results in its own framing (Chunk ID, Wall time, exit code). We split that off and re-attach it verbatim. One Codex Python result went from 0% to 87% removed.
- Codex has a 5-second WebSocket timeout. The engine runs at about 0.2 ms per KB, so it never gets close.
Nothing is silently dropped. Every elision becomes a marker like [... 412 log lines omitted [ref:k_…]; 3 ERROR, 1 WARN], and the model can fetch the original back for an hour.
Repo: https://github.com/hystax/optscale-kompressor
Methodology and per-task results: https://optscale.ai/blog/context-compression-coding-agents
If you think the measurement is flawed somewhere, I'd rather hear it.
•
u/dexterthebot 13h ago
You might want to consider listing your project on the weekly Show-Us-What-You-Built post. Look out for it on Tuesday/Wednesday. Highest commented project wins a week promotion on r/Codex and gets on the Hall of Fame sidebar.