Mise Engine · Conversions · Testing Playbook
mise convert CLI
The LLM talks to the engine exactly the way the future production agent will — through a tool call, never through code access.
Generates realistic asks in rotation: line cook ("#10 can", "a quart of pickle"), pastry chef (baker's %, grams), old-school ("a glug", "two handfuls"), invoice-speak ("4/1gal case", "CS"). Works against a coverage checklist so probing is systematic, not wandering.
Every call is logged and every response is invariant-checked automatically. The agent doesn't have to notice a failure — the proxy catches it.
The top-200 ConvMissing failures (what real kitchens actually got stuck on), the fixture site's 21K conversion facts, and recipe lines from the beta kitchen's book. The LLM dresses these in chef language — invention is the garnish, not the base.
"Accurate" means three different things depending on the case — a number can be checked, a reference can be matched, or the right answer is a behavior (ask, or confess an assumption). Each layer below has its own judge; none of them is a model.
True regardless of input — no answer key needed:
convert(2x) = 2 × convert(x)For any answer built from stored facts, the harness recomputes the expected value with separate, deliberately dumb code — walk the same facts, multiply the factors, no engine involved — and diffs. Agreement by two different routes is strong evidence; disagreement localizes the bug to a specific hop.
A frozen table of known answers: exact constants (1 lb = 453.59237 g), the master-list facts, curated chef-verified conversions. Ordinary spec tests. (Reminder of why even "known" truths get checked once: our own docs shipped an inverted fl-oz constant.)
For ambiguous or unanswerable inputs, the correct result is not a number:
The check is "did the ledger confess correctly" — not "was the number right."
Weekly: review a small, dollar-weighted sample of transcripts end-to-end. Every disagreement becomes a permanent fixture. The only layer that can catch "the engine and the checker are wrong the same way."
"Two 750ml bottles of Sauv Blanc, batches take 180ml — how many full batches?"
Engine: 8 batches, 60ml left. Layer 1: conservation holds (8×180+60 = 1500 ✓). Layer 2: independent recompute agrees ✓. Ledger carries the bottle-size source.
Same question, but "two bottles of wine."
Engine answers "8 batches" with no assumption entry. The arithmetic is flawless — and it's a Layer-4 failure: a silent assumption. The number was right; the behavior was wrong. That distinction is the reason this harness exists.
Every session ends with a report:
Two trends prove the system is healthy: findings per session fall on old territory (it's all fixtures now) and coverage climbs. When a full persona rotation of fresh chef-chaos produces zero new math bugs and zero silent assumptions, the engine isn't "probably fine" — it has survived a measured, adversarial, physics-checked interrogation, and we can say so to a chef in September and an investor after that.