Prompt optimization is a strange search problem.

The objective is flat almost everywhere, jumps when it moves, and lives on a thin slice of the space that nobody can write down. Here's what that means and how the current methods deal with it.

Prompt learning, and where optimization sits in it

Prompt learning is machine learning applied to prompts. The analogy is exact enough to be useful:

training a modelprompt learning
weightsthe prompt text
training setlabeled (input, answer) examples
loss functionthe objective: accuracy, tokens, format, latency, or a weighted / constrained combination
optimizer (SGD, Adam)prompt optimization: an LLM proposes rewrites, a selector decides which to evaluate and expand
validation seta held-out set the optimizer never sees
overfittinga prompt that wins on 30 examples and loses on the next 30

Where the analogy breaks is the optimizer row, and that break is the subject of this page. Weights are continuous and the loss is differentiable; prompts are discrete text and the objective has no gradient. Everything else transfers — which is why the discipline that comes with it (held-out sets, budgets, reporting fronts instead of single numbers) transfers too.

Not to be confused with prompt tuning (Lester et al. 2021) or prefix tuning, which learn continuous "soft prompt" vectors by gradient descent. Prompt learning in the sense used here keeps the model frozen and the prompt human-readable; the output is a string you can paste into a config file.

The problem

Prompt optimization means: given a task and a scoring function, find the string that scores best. The space is every possible string. The score is what an LLM does when you feed it that string, averaged over a dataset. That's a well-posed optimization problem — and almost none of the usual machinery applies.

1. The derivative is zero almost everywhere

Change one word and the score usually doesn't move at all. The model reads "Extract the names" and "Extract all the names" the same way. Gradient descent has nothing to follow: the surface is piecewise flat, and the pieces are wide. The only way to learn anything is to make a real edit and pay for a full evaluation.

2. When it does move, it jumps

Drop the word "only" and precision collapses. Add "as JSON" and format compliance goes from 60% to 99%. The landscape is a plateau riddled with cliffs and thousands of tiny basins. Local search gets stuck in the first basin it finds; every basin looks like an optimum from inside.

prompt space (one dimension of a very large space) score unusable (score ≈ 0) unusable ← global optimum flat steps = zero gradient · dots = local optima the usable manifold — boundary unknown
A one-dimensional caricature of the prompt landscape. Flat almost everywhere, discontinuous where it moves, many small local optima, and a usable region whose edges you can't compute in advance.

3. Only a sliver of the space is usable — and you can't enumerate it

Almost every string is a useless prompt. The good ones live on a thin manifold of fluent, task-relevant text — and there is no formula for its boundary. You can't grid-search it, you can't sample it uniformly, and you can't project onto it. The only thing that reliably stays on the manifold is another language model generating text. So every practical method has the same shape: an LLM proposes candidates, a scorer evaluates them, and a selector decides where to look next. The differences are all in the proposer and the selector.

4. Evaluations are expensive and noisy

Every point on the landscape costs one LLM call per dataset example. Scores carry sampling noise. Small datasets overfit: a prompt that wins on 30 examples may lose on the next 30. So the budget is measured in rollouts — unique (prompt, example) calls — and the question every method has to answer is: what do I evaluate next, given what I've already paid for?

The approaches

Three families, each getting one thing right.

DSPy and MIPROv2 — prompts as programs, search as hyperparameter tuning

DSPy separates the program (a pipeline of typed modules) from the prompt text each module ends up using. Its optimizers then fill in the text. MIPROv2 proposes a pool of candidate instructions and bootstrapped few-shot demonstrations for each module, then runs Bayesian optimization over the discrete choice of which instruction and which demos to combine — a surrogate model predicts which combinations will score well, so the expensive full evaluations go where they're most informative.

GEPA — reflective evolution with a Pareto pool

GEPA (Agrawal et al., 2025) treats optimization as evolution, with two ideas that matter. First, the mutation operator is reflective: an LLM reads the parent prompt, the examples it failed on and textual feedback about why, then rewrites. A scalar reward says "0.7"; text feedback says "you extracted the company name as a person" — far more information per rollout. Second, the parent pool is a per-example Pareto front: a candidate stays alive if it's the best on any single training example, and parents are sampled in proportion to how many examples they win. That preserves diversity — a prompt that's mediocre on average but nails a hard case is kept around to breed from.

Reference numbers from the paper on Qwen3-8B (test accuracy, %): HotpotQA 42.3 → 62.3, HoVer 35.3 → 52.3, PUPA 80.8 → 91.9, IFBench 36.9 → 38.6, within ~7k rollouts — beating MIPROv2 and GRPO-style RL at a fraction of the rollouts.

Bayesian optimization — learn where the good prompts are

Bayesian optimization is the standard answer to "expensive, noisy, black-box objective". Embed each evaluated prompt as a vector, fit a Gaussian process from embedding → score, and use an acquisition function (expected improvement, UCB, Thompson sampling) to choose the next candidate: the one with the best combination of predicted score and uncertainty. The GP handles noise natively — it knows a single high score on a small sample might be luck.

Applied to a prompt tree, BO answers two questions: which node should I expand next? (fit the surrogate to a node's descendants' value, so it learns which regions breed good children) and which of these freshly-proposed children are worth a full evaluation? (pre-screen by predicted score, evaluate only the top ones).

What we learned running them side by side

bpto implements GEPA-style selection and BO selection as peers on the same tree, with the same mutation operators, cache and budget, so they can be compared at equal rollouts with one ingredient swapped at a time. The experiment log is in the repo; the standing conclusions so far:

The framing that made the comparison possible: proposals are operators on a tree, selection is a pluggable strategy, and the tree never knows which one is running. GEPA's reflective proposer and BO's acquisition-based selector are not rivals — the current best configuration uses both.

Further reading