Level$’s voice line answers spoken questions about a household’s spending — “how much have we spent on dining this month,” “are we over on groceries” — over a real phone call, against real account data. Anyone can wire an LLM to a set of tools and call it a voice agent in a weekend. What follows is the part that took longer: proving the thing doesn’t lie.
In progress — measurement phase, not a finished resultThe brief for this project was never “build a voice bot.” It was: build one, then instrument it well enough to produce an honest writeup of where it breaks. The deliverable is the measurement, not the agent. Success looks like 30–50 logged calls, a computed metrics table, and a failure taxonomy sorted by cause — not a demo reel of the calls that went well.
Four things had to exist before that measurement meant anything:
Grounding attestation checks that a number the agent says out loud came from a real tool call, not a hallucination. That catches the obvious failure. It does not catch a tool that runs a subtly wrong calculation and returns a confident, well-formed, entirely traceable wrong answer.
A voice tool issuing its own group by categoryin SQL would disagree with the number on the user’s screen for most transactions — split transactions, ignore rules, and pending-charge suppression all live outside a naive SQL aggregate — and grounding attestation would bless it anyway, because the number really did come from a tool call.The fix was refusing to let a second implementation of that math exist at all. Category totals, split-transaction distribution, and ignore-rule matching live in one shared module, called by the app screen and by every voice tool that touches spending. A permanent parity test asserts the voice path and the app path agree — per category, per period, down to the cent — over the household’s real transaction history. If someone edits one and not the other, the test fails before a call ever happens.
Two layers, because a voice agent fails in two different ways — it reasons about the wrong thing, or it executes correctly on the wrong plan. Reporting only one hides the other.
Share of spoken numerals that trace to a real tool return, with rounding and percent tolerance rules applied — not exact-string matching.
The failure every other metric here misses: a correct answer reached by an incorrect plan looks identical to success on grounding rate, task completion, and containment alone.
Counted as a signal the boundary works, not as a defect — a violation caught and bounded is a better outcome than a boundary that was never tested.
Treated as a first-class outcome rather than an edge case — a judgment question answered against a defined threshold, qualified rather than asserted as fact.
The shared spend module, the numeral-attestation script, and the parity assertion are built and tested against the household’s real transaction history. The metrics table and the failure taxonomy are not written yet, because they don’t exist until enough real calls do — and this page won’t claim results that aren’t measured yet. When they are, this case study gets updated with the actual numbers, not a projection of them.