Output tokens get the attention because they're expensive per token. Prompt tokens are the bigger line: they're on every call, they never change, and nobody is reading them. Here is what removing the ones that don't matter is worth.
Every call to a model costs prompt_tokens × input_price + output_tokens × output_price. The prompt half is the template you wrote plus the per-request input. The template part is identical on every call. In a classifier, a router, an extractor or a tool-call formatter, it is most of the input — and in an agent loop it is sent once per step, several steps per task.
That makes the template the one lever that is free to pull: no model change, no architecture change, no runtime component. Change the string, and every subsequent call is cheaper and faster.
Monthly savings from compressing one template. Adjust the fields; the arithmetic is tokens saved × calls per month × price per million.
Defaults: 10M calls, 120 → 40 tokens (a 67% cut, mid-range for what the compression runs achieved on a 97-token prompt), $3 per million input tokens. Prices vary by model and cache state; use your own invoice.
Compression is a one-time search, and the search is cheap because it runs on your dataset with a small model doing the rewriting:
Against the calculator above, the search pays for itself in the first hour of production traffic for any system making more than a few thousand calls a day.
Agentic systems multiply the number of prompts you run and the number of times you run each one.
Nothing you didn't agree to. You set an accuracy floor; the optimizer minimizes tokens subject to it and reports the whole tokens-vs-accuracy front on a held-out set the search never saw. In the compression runs the shortest prompts sat 0.02–0.08 F1 below the 97-token original; prompts within 0.03 of it were 14–23 tokens. You choose the operating point — and the number you read is the held-out one, not the training one.
Two things compression will not do: it will not rescue a prompt that was wrong to begin with (fix accuracy first, then compress — same loop, different objective), and it will not compress the per-request input; that is what LLMLingua-style tools do, and the two compose.