# promptcompression.ai — full site

Generated 2026-09-15. Index: https://promptcompression.ai/llms.txt



---

# promptcompression.ai — prompt learning: ML-driven prompt compression and optimization
URL: https://promptcompression.ai/

# Prompts are code.
Optimize them like code.

 Most teams tune prompts by hand and hope they hold. We call the alternative prompt learning: treat a prompt like a model parameter — data in, objective set, optimizer runs, held-out set decides. promptcompression.ai applies it to manage prompts for stability, quality, efficiency and performance.

 [See how it works](https://promptcompression.ai/how-it-works)
 [View on GitHub](https://github.com/sign-of-fourier/bpto)

## What is prompt compression?

 Prompt compression is finding the shortest prompt that still does the job — measured, not guessed. You give it your prompt, a labeled dataset and an accuracy floor; an optimizer searches rewrites and hands back the tokens-vs-accuracy trade-off. It is one objective of a general prompt optimizer: the same loop also improves accuracy, format compliance and latency.

## What is prompt learning?

 Machine learning applied to prompts. A trained model has a dataset, a loss, an optimizer and a validation set. Prompt learning gives a prompt the same four things: labeled examples, an objective (accuracy, tokens, format, latency, or a combination), a search algorithm that proposes and selects rewrites, and a held-out set that decides. Prompt optimization is the search step; prompt compression is one objective. [Why the search is hard →](https://promptcompression.ai/prompt-optimization)

## Four things prompt learning delivers

 Each one is measurable. Each one is optimizable. None of them survive a model upgrade on their own.

### Stability
Prompts drift every time a model changes. A scored dataset turns "it seems fine" into a regression test you can re-run in minutes.

### Quality
Accuracy, format compliance, judge scores — pick the metric that matters and let the optimizer climb it instead of guessing.

### Efficiency
Shorter prompts, fewer tokens, lower latency. Compression is an objective, not an afterthought.

### Performance
Trade accuracy against cost on a Pareto front and choose the operating point your product actually needs.

## The numbers

 Agentic systems multiply the number of prompts you run, and the number of times you run each one.

 97 → 6

 tokens: a name-extraction prompt compressed by bpto; held-out F1 0.984 → 0.965. Across 12 seeds: 6–29 tokens, mean 12–16.

 Compression v2 experiment, Sept 2026 — [see findings](https://promptcompression.ai/findings)

 40%

 of enterprise applications will include task-specific AI agents by the end of 2026 — up from under 5% in 2025.

 [Gartner](https://www.gartner.com/en/newsroom/press-releases/2025-08-26-gartner-predicts-40-percent-of-enterprise-apps-will-feature-task-specific-ai-agents-by-2026-up-from-less-than-5-percent-in-2025)

 62%

 of organizations are experimenting with or scaling agentic AI; 23% have a system in production.

 [McKinsey, State of AI](https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai)

 35%

 average increase in cloud spend attributed to unmonitored token usage at enterprises using generative AI.

 [Deloitte 2025 AI expenditure report, via PremAI](https://www.premai.io/blog/llm-token-cost/)

 $2–4k

 per month saved by trimming ~80 prompt tokens from a system that makes 10M calls a month at $2.50–$5 per million input tokens.

 Worked example: 80 tokens × 10M calls = 800M tokens/month. Your numbers will differ.

 3–6

 prompts in a typical agent loop (system, planner, tool formatter, summarizer, answer). Each one is a separate optimization target.

 See [optimizing a whole system](https://promptcompression.ai/how-it-works#system)

## Two ways to run it

 The same optimizer, as a library today and as a point-and-click studio next.

 Available now

### bpto — the open-source engine

 A Python library. Define a task (root prompt, dataset, scorer, objective), compose search operators, run with a budget and checkpoints. Everything on this site was measured with it.

 [github.com/sign-of-fourier/bpto](https://github.com/sign-of-fourier/bpto)

 Coming

### promptcompression.ai Studio

 Every step of bpto without writing code. Upload your labeled data, paste your starter prompt, pick an objective, watch the tree grow, and read the result off the Pareto front. No notebooks, no config files.

 [Get notified](https://promptcompression.ai/contact)

## Why prompts belong in your AI strategy

 Agents multiply the prompt surface. A single chat assistant has one prompt. A ReAct-style agent has a system prompt, a planning prompt, a tool-call formatter, an observation summarizer and a final-answer prompt — and it runs each of them several times per task. Small inefficiencies compound per call, per step, per user.

 Every model upgrade silently re-tunes every prompt. Prompts written for one model are not optimal for the next. Without a scored dataset and an optimizer, "upgrading the model" means re-doing the prompt work by hand, or not at all.

 A measurable prompt strategy is the difference between a pilot and a product. Teams that treat prompts as versioned, evaluated, optimized artifacts ship faster and spend less. Teams that treat them as text in a config file discover regressions from customers.

 The optimizer is abstract: it takes a starter prompt, a dataset and an objective, and searches. Compression, accuracy, format compliance, latency — same loop, different objective.

 [How it works](https://promptcompression.ai/how-it-works)
 [Why prompt optimization is hard](https://promptcompression.ai/prompt-optimization)

## Frequently asked questions

### How is this different from LLMLingua-style compression?

 LLMLingua compresses the input of each request at inference time by dropping low-information tokens, and needs a small model in the request path. promptcompression.ai compresses the prompt template once, at optimization time, by searching over rewrites and scoring them on your data. The result is a plain shorter prompt with nothing extra at inference. The two compose: a compressed template with compressed inputs.

### How much data do I need?

 Enough to detect the differences you care about. 100 training examples plus a held-out set was enough to separate search strategies in our compression runs; 30 was not. If you have production traces, you already have the data.

### Which models does it work with?

 Any: Anthropic models directly, and anything with an OpenAI-compatible endpoint (OpenAI, vLLM, Ollama, OpenRouter, Bedrock via a proxy). The optimizer only needs a client that returns completions.

### Does compression hurt accuracy?

 It's a trade-off you control. You set the accuracy floor; the optimizer minimizes tokens subject to it and reports the whole Pareto front, so you choose the operating point. Always read the held-out number, not the training number.

### Can I use it from Claude Code or another coding agent?

 Yes — there is a [step-by-step page written for agents](https://promptcompression.ai/claude-code) and an installable skill file.

### Is it open source?

 The engine, bpto, is open source on [GitHub](https://github.com/sign-of-fourier/bpto). The point-and-click Studio is in development.


---

# How prompt compression works — promptcompression.ai
URL: https://promptcompression.ai/how-it-works

# Data in. Better prompt out.

 This is the prompt-learning loop: dataset → objective → optimizer → validation. It needs three things from you — labeled examples, a starter prompt and an objective. We'll walk through prompt compression, but the same loop optimizes anything you can score.

 1

### Start with labeled data

 A dataset is a list of inputs with known answers. Here is a name-extraction set: passages of text and the people mentioned in them.

 | inputs.text | answer |

 | "After the merger, Priya Natarajan moved the Denver office to a four-day week; Marcus Hale objected in the March board meeting." | ["Priya Natarajan", "Marcus Hale"] |

 | "Rainfall in Portland exceeded the September average, the National Weather Service reported." | [] |

 | "Dr. Elena Vasquez will present the findings; Tuesday's session is chaired by Tom Bright of Unilever." | ["Elena Vasquez", "Tom Bright"] |

 | "The Lakers signed Charles Okafor to a two-year deal, sources told Reuters." | ["Charles Okafor"] |

 Run a prompt over the dataset and score each output against the answer. Now every prompt has a number — or a vector of numbers. That's what the optimizer sees. It never needs to understand the task; it only needs (prompt, score) pairs.

 | prompt | f1 | template_tokens |

 | Extract the full names of all people mentioned in the text below. Return only real people (not places, companies, months …). Output JSON: {"names": [...]} | 0.97 | 97 |

 | List every person's full name in the text as JSON {"names": [...]}. Skip organizations and places. | 0.96 | 31 |

 | People named, JSON {"names":[]} | 0.93 | 9 |

 | Names: | 0.61 | 2 |

 2

### Give it a starter prompt and an objective

 The starter prompt is the root of the search tree — usually whatever you have in production today. The objective says what "better" means. Scores are vectors; the objective is a small function that turns a vector into a scalar, so it's cheap to swap.

### Prompt compression
Maximize accuracy minus a per-token penalty, or hold accuracy above a floor and minimize tokens. f1 − 0.001·template_tokens, or a token budget that tightens each round.

### Accuracy
Exact match, set-F1, or an LLM judge asked "is this equivalent to the reference?" Pure quality, no cost term.

### Format & reliability
Fraction of outputs that parse as valid JSON, follow a schema, or stay within a length limit. Great for tool-call formatters.

### Latency & cost
Output tokens, prompt tokens, wall time. Combine with a quality metric to trade them off explicitly.

```
from bpto import AnthropicClient, Dataset, LinearObjective, Task, combine, template_tokens, token_count
from tasks.compression import Entities, set_f1

task = Task(
 root=open("prompt.txt").read(), # your current prompt, with {text} placeholder
 description="extracts the names of all people mentioned in a passage",
 dataset=Dataset.from_jsonl("train.jsonl"), # {"inputs": {"text": ...}, "answer": [...]}
 schema=Entities,
 scorer=combine(set_f1(), template_tokens(), token_count()),
 objective=LinearObjective(f1=1.0, template_tokens=-0.002),
 client=AnthropicClient("claude-opus-5"),
)
```

 3

### Search

 The optimizer grows a tree. An LLM proposes rewrites of a node (random variants, or directed: "make it shorter", "be more precise", or a reflection on the examples the parent got wrong). Each child is evaluated on the dataset. A selector decides which node to expand next — greedy, a Pareto pool, or a Bayesian surrogate over prompt embeddings that predicts where the good children are.

 A prompt tree. Nodes are prompts scored on the dataset; edges are LLM rewrites. The selector picks which node to expand next — that choice is where the sample efficiency comes from. [Why that's hard →](https://promptcompression.ai/prompt-optimization)

 4

### Read the result off the Pareto front

 For a two-objective problem like compression, the answer isn't one prompt — it's a front. bpto keeps every non-dominated (tokens, accuracy) pair so you can choose the operating point your product needs.

 Before · 97 tokens · held-out F1 0.97

```
You are an information extraction system. Read the passage below carefully and extract the full names of every person who is mentioned. Include each person only once, preserve the order in which they first appear, and do not include organisations, places, or pronouns. Titles such as Dr. or Professor are not part of the name. If no people are mentioned, return an empty list.

Passage:
{text}
```

 After · 6 tokens · held-out F1 0.965

```
Extract human names from: {text}
```

97

6

 Measured, not assumed. In the compression v2 run (12 seeds × 2,000 rollouts, 100 training / 200 held-out examples) both search strategies compressed the 97-token root to 6–29 tokens; the best seed reached 6 tokens at held-out F1 0.965 vs 0.984 for the original. Full tables and front plots on the [findings page](https://promptcompression.ai/findings). (Table prompts above are illustrative; the before/after pair is real.)

## It's not just compression

 Nothing above was specific to shortening prompts. The optimizer sees a root, a dataset, a scorer and an objective. Change the objective and the same tree search does something else:

 - Accuracy — objective is the judge score; directives like "be more precise" instead of "be shorter".

 - Migration — you upgraded models and the old prompt regressed; the old prompt is the root, the objective is accuracy on the new model.

 - Reliability — objective is the fraction of outputs that parse; the dataset is real production traces.

 - Constrained — maximize accuracy subject to a token budget, or minimize tokens subject to an accuracy floor. bpto supports both weighted-sum and constrained objectives, and objectives that tighten with tree depth.

 Completions are cached on (prompt, config, schema), so re-scoring the whole tree under a new objective costs nothing.

## Prompt learning for a whole system

 Real products aren't one prompt. A ReAct-style agent is a loop of several prompts, each with its own job, each run several times per task. Every box below is a prompt you can optimize — with its own objective — using the same loop.

 A ReAct loop. Five prompts, five objectives, one end-to-end metric. The summarizer is the compression target — it runs once per tool call, so its tokens compound fastest.

### How you optimize it

 - Collect traces. Run the agent on labeled tasks and log every module's inputs and outputs. Each module now has its own dataset — for free.

 - Give each module an objective. The formatter wants valid JSON; the summarizer wants fewer tokens with no lost facts; the final answer wants judge score. Plus one end-to-end metric (did the task succeed?).

 - Optimize one module at a time, holding the others fixed — the prompt-learning analogue of training a multi-component model with coordinate ascent. Round-robin through them. Re-collect traces when an upstream module changes, because its outputs are the downstream module's inputs.

 - Check the end-to-end metric. A module that got "better" on its local objective but hurt task success is rejected. That's what the labeled tasks are for.

 Because the optimizer only ever sees (prompt, score), a module's "prompt" can be anything text-shaped: a system prompt, a few-shot block, a tool description, or the instructions inside a chain-of-thought template.

 Today all of this runs as the open-source [bpto](https://github.com/sign-of-fourier/bpto) library. The promptcompression.ai Studio will put the same steps — upload data, set a starter prompt and objective, search, pick from the front — behind a point-and-click interface.

 [Get the code](https://github.com/sign-of-fourier/bpto)
 [Get notified about Studio](https://promptcompression.ai/contact)


---

# Prompt optimization: why it's a hard search problem (DSPy, GEPA, Bayesian optimization) — promptcompression.ai
URL: https://promptcompression.ai/prompt-optimization

# Prompt optimization is a strange search problem.

 The objective is flat almost everywhere, jumps when it moves, and lives on a thin slice of the space that nobody can write down. Here's what that means and how the current methods deal with it.

## Prompt learning, and where optimization sits in it

 Prompt learning is machine learning applied to prompts. The analogy is exact enough to be useful:

 | training a model | prompt learning |

 | weights | the prompt text |

 | training set | labeled (input, answer) examples |

 | loss function | the objective: accuracy, tokens, format, latency, or a weighted / constrained combination |

 | optimizer (SGD, Adam) | prompt optimization: an LLM proposes rewrites, a selector decides which to evaluate and expand |

 | validation set | a held-out set the optimizer never sees |

 | overfitting | a prompt that wins on 30 examples and loses on the next 30 |

 Where the analogy breaks is the optimizer row, and that break is the subject of this page. Weights are continuous and the loss is differentiable; prompts are discrete text and the objective has no gradient. Everything else transfers — which is why the discipline that comes with it (held-out sets, budgets, reporting fronts instead of single numbers) transfers too.

 Not to be confused with prompt tuning (Lester et al. 2021) or prefix tuning, which learn continuous "soft prompt" vectors by gradient descent. Prompt learning in the sense used here keeps the model frozen and the prompt human-readable; the output is a string you can paste into a config file.

## The problem

 Prompt optimization means: given a task and a scoring function, find the string that scores best. The space is every possible string. The score is what an LLM does when you feed it that string, averaged over a dataset. That's a well-posed optimization problem — and almost none of the usual machinery applies.

### 1. The derivative is zero almost everywhere

 Change one word and the score usually doesn't move at all. The model reads "Extract the names" and "Extract all the names" the same way. Gradient descent has nothing to follow: the surface is piecewise flat, and the pieces are wide. The only way to learn anything is to make a real edit and pay for a full evaluation.

### 2. When it does move, it jumps

 Drop the word "only" and precision collapses. Add "as JSON" and format compliance goes from 60% to 99%. The landscape is a plateau riddled with cliffs and thousands of tiny basins. Local search gets stuck in the first basin it finds; every basin looks like an optimum from inside.

 A one-dimensional caricature of the prompt landscape. Flat almost everywhere, discontinuous where it moves, many small local optima, and a usable region whose edges you can't compute in advance.

### 3. Only a sliver of the space is usable — and you can't enumerate it

 Almost every string is a useless prompt. The good ones live on a thin manifold of fluent, task-relevant text — and there is no formula for its boundary. You can't grid-search it, you can't sample it uniformly, and you can't project onto it. The only thing that reliably stays on the manifold is another language model generating text. So every practical method has the same shape: an LLM proposes candidates, a scorer evaluates them, and a selector decides where to look next. The differences are all in the proposer and the selector.

### 4. Evaluations are expensive and noisy

 Every point on the landscape costs one LLM call per dataset example. Scores carry sampling noise. Small datasets overfit: a prompt that wins on 30 examples may lose on the next 30. So the budget is measured in rollouts — unique (prompt, example) calls — and the question every method has to answer is: what do I evaluate next, given what I've already paid for?

## The approaches

 Three families, each getting one thing right.

### DSPy and MIPROv2 — prompts as programs, search as hyperparameter tuning

 [DSPy](https://github.com/stanfordnlp/dspy) separates the program (a pipeline of typed modules) from the prompt text each module ends up using. Its optimizers then fill in the text. MIPROv2 proposes a pool of candidate instructions and bootstrapped few-shot demonstrations for each module, then runs Bayesian optimization over the discrete choice of which instruction and which demos to combine — a surrogate model predicts which combinations will score well, so the expensive full evaluations go where they're most informative.

 - What it gets right: treating multi-module systems as first-class, and casting prompt search as a well-understood hyperparameter problem with a surrogate.

 - Where it's limited: the search space is a fixed pool of pre-generated candidates. It can pick the best of what it proposed; it can't discover what it didn't propose.

### GEPA — reflective evolution with a Pareto pool

 [GEPA](https://arxiv.org/abs/2507.19457) (Agrawal et al., 2025) treats optimization as evolution, with two ideas that matter. First, the mutation operator is reflective: an LLM reads the parent prompt, the examples it failed on and textual feedback about why, then rewrites. A scalar reward says "0.7"; text feedback says "you extracted the company name as a person" — far more information per rollout. Second, the parent pool is a per-example Pareto front: a candidate stays alive if it's the best on any single training example, and parents are sampled in proportion to how many examples they win. That preserves diversity — a prompt that's mediocre on average but nails a hard case is kept around to breed from.

 Reference numbers from the paper on Qwen3-8B (test accuracy, %): HotpotQA 42.3 → 62.3, HoVer 35.3 → 52.3, PUPA 80.8 → 91.9, IFBench 36.9 → 38.6, within ~7k rollouts — beating MIPROv2 and GRPO-style RL at a fraction of the rollouts.

 - What it gets right: the proposer. Reflection on failures is the richest signal available, and the Pareto pool keeps the search from collapsing into one basin.

 - Where it's limited: parent selection is stochastic sampling from the pool; nothing predicts which parent is worth expanding before paying for its children.

### Bayesian optimization — learn where the good prompts are

 Bayesian optimization is the standard answer to "expensive, noisy, black-box objective". Embed each evaluated prompt as a vector, fit a Gaussian process from embedding → score, and use an acquisition function (expected improvement, UCB, Thompson sampling) to choose the next candidate: the one with the best combination of predicted score and uncertainty. The GP handles noise natively — it knows a single high score on a small sample might be luck.

 Applied to a prompt tree, BO answers two questions: which node should I expand next? (fit the surrogate to a node's descendants' value, so it learns which regions breed good children) and which of these freshly-proposed children are worth a full evaluation? (pre-screen by predicted score, evaluate only the top ones).

 - What it gets right: sample efficiency and robustness to evaluation noise; it's the only one of the three with a principled model of uncertainty.

 - Where it's limited: it needs an embedding in which "nearby" means "scores similarly" — not guaranteed for text — and a scalar value; a threshold-tied scalar leaves the rest of a Pareto front to chance.

## What we learned running them side by side

 bpto implements GEPA-style selection and BO selection as peers on the same tree, with the same mutation operators, cache and budget, so they can be compared at equal rollouts with one ingredient swapped at a time. The [experiment log](https://github.com/sign-of-fourier/bpto/tree/main/experiments) is in the repo; the standing conclusions so far:

 - The Pareto pool is what works in GEPA; its randomness isn't. On synthetic landscapes (8 arms × 40 seeds × 6 conditions), greedy argmax within the pool beat GEPA's weighted sampler everywhere except a deliberately deceptive landscape, where they tied.

 - BO's edge is noise and weak mutators. Under evaluation noise BO beat GEPA's sampler (0.97 vs 0.86); surrogate pre-screening of children helped most when the proposer was weak (0.91 vs 0.83). On clean additive landscapes nothing beat greedy.

 - Live, the proposer is the binding constraint. If the rewriting model only paraphrases, no selection strategy can help — there are no substantive children to select among. Get the mutator making real edits before comparing selectors.

 - Report the front, not a number. On a real compression task (12 seeds, 2,000 rollouts each), BO's pooled (tokens, F1) front lay left of GEPA's at every accuracy level; at the targeted accuracy BO was ~3.5 tokens shorter. At stricter accuracy bars GEPA's pool hedged better, because BO's scalar value ignored accurate-but-not-shorter children. A single-row comparison would have hidden that.

 - Flat benchmarks can't tell methods apart. IFBench moves by 1–2 points under every published optimizer; with a few hundred rows that's undetectable. Use it as a plumbing test, not a decider.

 - Gains on weak models don't transfer proportionally. A restatement that fixes a small model's "cheap" failures does nothing for a model that has already absorbed them.

 The framing that made the comparison possible: proposals are operators on a tree, selection is a pluggable strategy, and the tree never knows which one is running. GEPA's reflective proposer and BO's acquisition-based selector are not rivals — the current best configuration uses both.

## Further reading

 - [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://arxiv.org/abs/2507.19457) — Agrawal et al., 2025 · [code](https://github.com/gepa-ai/gepa)

 - [Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs](https://arxiv.org/abs/2406.11695) (MIPROv2) — Opsahl-Ong et al., 2024

 - [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://arxiv.org/abs/2310.03714) — Khattab et al., 2023

 - [bpto — Bayesian Prompt Tree Optimization](https://github.com/sign-of-fourier/bpto) — the implementation and experiment logs behind this page


---

# Findings: prompt compression benchmarks, GEPA vs Bayesian optimization — promptcompression.ai
URL: https://promptcompression.ai/findings

# What the measurements say.

 Every number on this site comes from a logged experiment in the bpto repo. This page collects the ones that matter: how far a prompt compresses, and which search strategy gets there with fewer rollouts.

## Prompt compression: 97 tokens → 6–29, GEPA vs Bayesian optimization

 12 seeds × 2,000 rollouts per arm · 100 training / 200 held-out examples per seed · task: extract people's names from a passage · model under optimization: Amazon Nova Micro; rewrites by Nova Lite · total cost $0.41 · September 2026.

 Both arms use the same mutation operator (an LLM reads a minibatch of the parent's failures and rewrites), the same evaluation gate, the same budget. They differ only in which parent gets expanded and how many children are proposed and screened. The objective was constrained: minimize template tokens subject to training F1 ≥ a floor that tightened from root − 0.15 to root − 0.05 over the run.

 [figure: Scatter plot of template tokens (log scale) versus train F1 for every prompt evaluated by the GEPA and Bayesian optimization arms, with each arm's Pareto front; the BO front lies to the left of GEPA's at every accuracy level. Right panel: per-seed fronts.]
 Left: every prompt each arm fully evaluated, pooled over 12 seeds, with each arm's non-dominated front. Right: per-seed fronts. Star = the 97-token root.

### Best prompt per seed (held-out F1 in parentheses; root held-out F1 0.97–0.99)

 | seed | GEPA | tok | BO | tok |

 | 0 | List all unique full names of people in: {text}, ignoring titles and organizations. | 19 (0.982) | Extract full names from: {text} | 6 (0.904) |

 | 1 | List unique full names from this text, ignoring titles: {text} | 14 (0.907) | Extract full names from text, excluding titles: {text} | 12 (0.910) |

 | 2 | Extract human names from: {text} | 6 (0.965) | Extract only personal names from this passage: {text} | 12 (0.951) |

 | 3 | Extract all unique full names from: {text}, ignoring titles and organizations. | 18 (0.982) | Extract personal names from {text}, avoid titles and organizations. | 15 (0.902) |

 | 4 | Extract names from: {text}. List only people, no titles or orgs. | 14 (0.880) | Extract full human names from: {text} | 8 (0.891) |

 | 5 | List all names of people in this passage, excluding titles: {text} | 15 (0.994) | Extract all human names from: {text} | 8 (0.952) |

 | 6 | Identify all people's full names mentioned in: {text}. List them once, in order, without titles. | 22 (0.992) | Extract only person names from: {text} | 8 (0.987) |

 | 7 | Extract full names from this text, omitting titles and entities: {text} | 16 (0.990) | Identify human names in: {text}. | 6 (0.942) |

 | 8 | Extract all full names of individuals from: {text} | 11 (0.916) | List all full person names from the following text.\nPassage:\n{text} | 15 (0.953) |

 | 9 | Extract full names from: {text}. Ignore titles and locations. | 14 (0.916) | List names in this text, excluding titles and organizations: {text} | 15 (0.972) |

 | 10 | Identify all full names of individuals in this passage, avoiding titles and organizations: {text} | 23 (0.996) | Identify and list all names of people mentioned in the text below, ignoring titles and non-person entities. Names:\n{text} | 29 (0.946) |

 | 11 | Identify all full names of people in this passage and list them in order: {text} | 18 (0.897) | Extract full names from: {text}. Exclude titles. | 10 (0.987) |

 Summary: GEPA 15.8 ± 1.3 tokens, held-out F1 0.951; BO 12.0 ± 1.8 tokens, held-out F1 0.941. Paired difference 3.8 ± 2.0 tokens, BO shorter on 8 of 12 seeds. At the accuracy the floor targeted (root − 0.05 to − 0.10) BO is 3–5 tokens shorter with the ±2 SE band clear of zero; at stricter bars (within 0.03 of root) GEPA's Pareto pool hedges better. BO makes ~3× the reflector calls (3 children per round vs 1); a "GEPA + 3 children, random keep-1" control is planned.

 [figure: Mean shortest feasible prompt length at each accuracy gap from the root, GEPA versus BO, with standard-error bands, and the paired difference showing BO shorter by 3 to 5 tokens for gaps above 0.03.]
 Same data averaged: at each accuracy bar (train F1 ≥ root − gap), the shortest prompt each seed found, mean ± 1 SE, and the paired GEPA − BO difference.

 Source: [experiments/2026-09-12-compression-v2-gepa-vs-bo](https://github.com/sign-of-fourier/bpto/tree/main/experiments/2026-09-12-compression-v2-gepa-vs-bo) (NOTES.md, results.jsonl, analysis.md).

## Earlier and supporting runs

 | date | experiment | result |

 | 2026-09-10 | Synthetic BO benchmark, 20 seeds, hash embedder + GPR + EI | BO finds the planted optimum in 1.45 rounds vs 7.35 for random selection. Synthetic landscape only. |

 | 2026-09-11 | Synthetic ladder: 8 selection arms × 40 seeds × 6 landscape conditions, $0 | The Pareto pool matters, stochastic sampling does not: greedy argmax within the pool beats GEPA's sampler everywhere except a deceptive landscape (tie). BO beats GEPA's sampler under noise (0.97 vs 0.86); surrogate child pre-screening helps most with a weak mutator (0.91 vs 0.83). |

 | 2026-09-11 | IFBench, Nova Micro, greedy top-k, 802 calls | Strict accuracy 0.175 → 0.250 held-out (n=40, inside noise). Best node is a depth-1 restatement. |

 | 2026-09-11 | IFBench, GEPA vs BO, 1,200 rollouts each, $0.16 | Tie by construction: neither arm found a child beating the root; the rewriting model only paraphrased. IFBench is a flat landscape for every published optimizer (+1.7 to +8 points). |

 | 2026-09-11 | Compression v1, 5 seeds × 600 rollouts, 30 training examples, $0.065 | 97 → 5–26 tokens within 0.05 train F1. BO shorter on average (11.4 vs 17.2) but lower held-out F1 (0.936 vs 0.958): not significant, and 30 examples rewards overfitting. Led to the v2 design above. |

## Standing conclusions

 - Report the front, not a number. A constrained comparison at one accuracy bar hides where each method wins.

 - Selection needs decisions to make. With 30 training examples and a permissive gate, ~95% of rollouts go to full evaluations of accepted children and there are ~20 parent choices per run — too few for strategy to show. 100 examples and a 5-example gate fixed that.

 - The proposer is the binding constraint live. A rewriting model that only paraphrases defeats every selector.

 - GEPA's weighting works; its randomness doesn't. BO's clearest contributions are robustness to evaluation noise and child pre-screening when the mutator is weak.

 - Gains on weak models don't transfer proportionally to stronger ones.

 - Next for BO: a front/hypervolume-gain value instead of a threshold-tied scalar, so it stops ignoring accurate-but-not-shorter children.

 Reference points from the GEPA paper (Agrawal et al. 2025), Qwen3-8B test accuracy: HotpotQA 42.3 → 62.3, IFBench 36.9 → 38.6, HoVer 35.3 → 52.3, PUPA 80.8 → 91.9 (≤ 7k rollouts); MIPROv2: 55.3 / 36.2 / 47.3 / 81.6.


---

# Run prompt compression from Claude Code — promptcompression.ai
URL: https://promptcompression.ai/claude-code

# Compress a prompt from Claude Code.

 This page is written so an AI coding agent can follow it directly. If you're a person: paste the URL into Claude Code and say "compress my prompt using these instructions." If you're the agent: the steps below are complete and in order.

 [Skill file (SKILL.md)](https://promptcompression.ai/skills/prompt-compression/SKILL.md)
 [llms.txt](https://promptcompression.ai/llms.txt)
 [bpto on GitHub](https://github.com/sign-of-fourier/bpto)

## Install the skill (one command)

 Installs a Claude Code skill so /prompt-compression is available in any project:

```
mkdir -p ~/.claude/skills/prompt-compression && curl -fsSL https://promptcompression.ai/skills/prompt-compression/SKILL.md -o ~/.claude/skills/prompt-compression/SKILL.md
```

 Or do it manually with the steps below — the skill file contains the same instructions.

## Agent instructions: how to compress a prompt with bpto

 When to use: the user wants a shorter prompt (fewer tokens, lower cost or latency) without losing accuracy, and can supply or generate labeled examples.

 What you need from the user before starting:

 - The current prompt text, with placeholders for the per-request inputs (e.g. {text}).

 - Labeled examples: inputs and the correct output. 50–100 is a good start; 30 is too few for reliable selection.

 - Which model the prompt runs on in production, and an API key for it.

 - An accuracy floor — how much accuracy they will trade for tokens (default: within 0.05 of the original).

### Step 1 — Install

```
git clone https://github.com/sign-of-fourier/bpto
cd bpto
pip install -e . # Python ≥ 3.12; deps: anthropic, pydantic, httpx, numpy
python -m pytest -q # optional: offline tests, no API key needed
```

### Step 2 — Write the dataset as JSONL

 One JSON object per line. inputs is a dict whose keys match the placeholders in the prompt; answer is the reference output.

```
{"inputs": {"text": "After the merger, Priya Natarajan moved the Denver office to a four-day week; Marcus Hale objected."}, "answer": ["Priya Natarajan", "Marcus Hale"]}
{"inputs": {"text": "Rainfall in Portland exceeded the September average, the National Weather Service reported."}, "answer": []}
{"inputs": {"text": "The Lakers signed Charles Okafor to a two-year deal, sources told Reuters."}, "answer": ["Charles Okafor"]}
```

 Keep a held-out split (e.g. 2:1) that the optimizer never sees; report the held-out score to the user, not the training score.

### Step 3 — Run a smoke test offline (no API key)

```
python -m tasks.compression.run --mock --rounds 2
```

 Confirms the install. Outputs go to runs/compression/.

### Step 4 — Run for real

```
# Anthropic
ANTHROPIC_API_KEY=... python -m tasks.compression.run --data train.jsonl --rounds 4 --model claude-opus-5

# Any OpenAI-compatible endpoint (OpenAI, vLLM, Ollama, OpenRouter)
OPENAI_API_KEY=... python -m tasks.compression.run --data train.jsonl --rounds 4 \
 --provider openai --base-url https://api.openai.com/v1 --model gpt-5

# Constrained mode: shrink the token budget each round instead of a weighted sum
python -m tasks.compression.run --data train.jsonl --rounds 6 --constrained --start-tokens 80 --shrink 10

# Continue a checkpointed run
python -m tasks.compression.run --data train.jsonl --rounds 8 --resume
```

 Useful flags: --n-random / --n-guided (children per expansion, default 3 / 2), --expand-k (leaves expanded per round, default 3), --cheap-n (examples in the first successive-halving rung, default 12), --token-weight (per-token penalty for the weighted objective, default 0.002), --concurrency (default 8), --out (default runs/compression).

### Step 5 — Read the outputs

 | file | what it is |

 | runs/compression/report.txt | Pareto table (template_tokens vs f1 for every non-dominated prompt) and the best prompt under the objective. Show this table to the user. |

 | runs/compression/pareto.png | The same front as a plot. |

 | runs/compression/tree.json | Checkpoint of the whole search tree; needed for --resume. |

 | runs/compression/cache.jsonl | Completion cache keyed on (prompt, config, schema). Re-scoring under a new objective is free. |

 Report to the user: the original prompt's tokens and held-out score, the row on the front that meets their accuracy floor, and its prompt text. Offer 2–3 points on the front (shortest feasible, safest, in between) rather than one.

### Step 6 — Adapting to a task that is not name extraction

 The tasks/compression runner is wired for a list-of-names schema and set-F1. For another task, write a small task module: a Pydantic schema for the output, a scorer returning a dict of metrics per example, and use combine(scorer, template_tokens(), token_count()) with a LinearObjective or ConstrainedObjective. The bpto README's question-answering example (LLM judge + token count) is the template. The tree, operators and search loop are unchanged.

```
from bpto import (AnthropicClient, ConstrainedObjective, Dataset, LinearObjective, Task, Tree,
 combine, evaluate, guided, llm_judge, random, select, template_tokens, token_count)

task = Task(
 root=open("prompt.txt").read(),
 description="one line describing what the prompt does",
 dataset=Dataset.from_jsonl("train.jsonl"),
 schema=YourPydanticModel,
 scorer=combine(llm_judge("Is the answer equivalent to the reference?", client=AnthropicClient("claude-opus-5")),
 template_tokens(), token_count()),
 objective=ConstrainedObjective(LinearObjective(judge=1.0), metric="template_tokens", bound=lambda ctx: 40, penalty=0.01),
 client=AnthropicClient("claude-opus-5", max_concurrency=16),
)
tree = Tree(task)
await tree.apply(random(n=4), select=select.leaves)
await tree.apply(guided("make it as short as possible without losing precision", n=3), select=select.leaves)
await tree.apply(evaluate(), select=select.unevaluated)
print(tree.best().prompt)
print(tree.pareto({"judge": True, "template_tokens": False}))
```

### Things that go wrong

 - Every child scores the same as the root. The rewriting model is paraphrasing instead of editing. Use a stronger model for proposals, or a more forceful directive ("remove every sentence that is not necessary").

 - Training score is high, held-out is low. Overfitting a small dataset. Add examples or raise the floor.

 - Output doesn't parse. The schema is enforced client-side; some small models echo the schema. Check cache.jsonl for raw completions.

 - Budget. A 4-round run on 100 examples is a few hundred to a few thousand model calls. Set --concurrency to respect rate limits; use --resume after a crash.

## Machine-readable resources

 - [/llms.txt](https://promptcompression.ai/llms.txt) — site index for LLMs

 - [/llms-full.txt](https://promptcompression.ai/llms-full.txt) — every page of this site as one markdown document

 - [/skills/prompt-compression/SKILL.md](https://promptcompression.ai/skills/prompt-compression/SKILL.md) — the Claude Code skill

 - [bpto README](https://github.com/sign-of-fourier/bpto/blob/main/README.md) — full library API


---

# Contact — promptcompression.ai
URL: https://promptcompression.ai/contact

# Get in touch.

 Questions about the optimizer, a task you'd like to try it on, a system you want to compress — or a note saying you want early access to the Studio.

### Email

 The fastest way to reach us.

 [markshipman4273@gmail.com](https://promptcompression.aimailto:markshipman4273@gmail.com)

### GitHub

 Bugs, feature requests and experiment questions belong in [the issue tracker](https://github.com/sign-of-fourier/bpto/issues).

### Or send a message

 Send