Agent instructions: how to compress a prompt with bpto

When to use: the user wants a shorter prompt (fewer tokens, lower cost or latency) without losing accuracy, and can supply or generate labeled examples.

What you need from the user before starting:

The current prompt text, with placeholders for the per-request inputs (e.g. {text}).
Labeled examples: inputs and the correct output. 50–100 is a good start; 30 is too few for reliable selection.
Which model the prompt runs on in production, and an API key for it.
An accuracy floor — how much accuracy they will trade for tokens (default: within 0.05 of the original).

Step 1 — Install

git clone https://github.com/sign-of-fourier/bpto
cd bpto
pip install -e .                      # Python ≥ 3.12; deps: anthropic, pydantic, httpx, numpy
python -m pytest -q                   # optional: offline tests, no API key needed

Step 2 — Write the dataset as JSONL

One JSON object per line. inputs is a dict whose keys match the placeholders in the prompt; answer is the reference output.

{"inputs": {"text": "After the merger, Priya Natarajan moved the Denver office to a four-day week; Marcus Hale objected."}, "answer": ["Priya Natarajan", "Marcus Hale"]}
{"inputs": {"text": "Rainfall in Portland exceeded the September average, the National Weather Service reported."}, "answer": []}
{"inputs": {"text": "The Lakers signed Charles Okafor to a two-year deal, sources told Reuters."}, "answer": ["Charles Okafor"]}

Keep a held-out split (e.g. 2:1) that the optimizer never sees; report the held-out score to the user, not the training score.

Step 3 — Run a smoke test offline (no API key)

python -m tasks.compression.run --mock --rounds 2

Confirms the install. Outputs go to runs/compression/.

Step 4 — Run for real

# Anthropic
ANTHROPIC_API_KEY=... python -m tasks.compression.run --data train.jsonl --rounds 4 --model claude-opus-5

# Any OpenAI-compatible endpoint (OpenAI, vLLM, Ollama, OpenRouter)
OPENAI_API_KEY=... python -m tasks.compression.run --data train.jsonl --rounds 4 \
    --provider openai --base-url https://api.openai.com/v1 --model gpt-5

# Constrained mode: shrink the token budget each round instead of a weighted sum
python -m tasks.compression.run --data train.jsonl --rounds 6 --constrained --start-tokens 80 --shrink 10

# Continue a checkpointed run
python -m tasks.compression.run --data train.jsonl --rounds 8 --resume

Useful flags: --n-random / --n-guided (children per expansion, default 3 / 2), --expand-k (leaves expanded per round, default 3), --cheap-n (examples in the first successive-halving rung, default 12), --token-weight (per-token penalty for the weighted objective, default 0.002), --concurrency (default 8), --out (default runs/compression).

Step 5 — Read the outputs

file	what it is
runs/compression/report.txt	Pareto table (template_tokens vs f1 for every non-dominated prompt) and the best prompt under the objective. Show this table to the user.
runs/compression/pareto.png	The same front as a plot.
runs/compression/tree.json	Checkpoint of the whole search tree; needed for `--resume`.
runs/compression/cache.jsonl	Completion cache keyed on (prompt, config, schema). Re-scoring under a new objective is free.

Report to the user: the original prompt's tokens and held-out score, the row on the front that meets their accuracy floor, and its prompt text. Offer 2–3 points on the front (shortest feasible, safest, in between) rather than one.

Step 6 — Adapting to a task that is not name extraction

The tasks/compression runner is wired for a list-of-names schema and set-F1. For another task, write a small task module: a Pydantic schema for the output, a scorer returning a dict of metrics per example, and use combine(scorer, template_tokens(), token_count()) with a LinearObjective or ConstrainedObjective. The bpto README's question-answering example (LLM judge + token count) is the template. The tree, operators and search loop are unchanged.

from bpto import (AnthropicClient, ConstrainedObjective, Dataset, LinearObjective, Task, Tree,
                  combine, evaluate, guided, llm_judge, random, select, template_tokens, token_count)

task = Task(
    root=open("prompt.txt").read(),
    description="one line describing what the prompt does",
    dataset=Dataset.from_jsonl("train.jsonl"),
    schema=YourPydanticModel,
    scorer=combine(llm_judge("Is the answer equivalent to the reference?", client=AnthropicClient("claude-opus-5")),
                   template_tokens(), token_count()),
    objective=ConstrainedObjective(LinearObjective(judge=1.0), metric="template_tokens", bound=lambda ctx: 40, penalty=0.01),
    client=AnthropicClient("claude-opus-5", max_concurrency=16),
)
tree = Tree(task)
await tree.apply(random(n=4), select=select.leaves)
await tree.apply(guided("make it as short as possible without losing precision", n=3), select=select.leaves)
await tree.apply(evaluate(), select=select.unevaluated)
print(tree.best().prompt)
print(tree.pareto({"judge": True, "template_tokens": False}))

Things that go wrong

Every child scores the same as the root. The rewriting model is paraphrasing instead of editing. Use a stronger model for proposals, or a more forceful directive ("remove every sentence that is not necessary").
Training score is high, held-out is low. Overfitting a small dataset. Add examples or raise the floor.
Output doesn't parse. The schema is enforced client-side; some small models echo the schema. Check cache.jsonl for raw completions.
Budget. A 4-round run on 100 examples is a few hundred to a few thousand model calls. Set --concurrency to respect rate limits; use --resume after a crash.

Compress a prompt from Claude Code.

Install the skill (one command)