Evaluations and guardrails, so the system holds up once real users touch it.
AI evaluation and guardrails engineering, out of New York. Guardrails ship with the system. They are the part that decides whether the thing survives contact with a real Tuesday, and they are cheapest to build while the code is still being written.
On the routing platform that means three things you can check. Geocoding fails closed, so a location the app cannot trust becomes a flagged task for a human. A per-solve travel-matrix budget makes a runaway maps bill structurally impossible. And 1,278 tests cover the business rules, which is what lets a dispatcher trust a sheet they did not plan themselves.
The same discipline applies well outside models. The food-truck site carries 141 Vitest tests guarding its URL structure, including one that fails the run if a second page claims the New York rental query the owner page holds. Its published prices live in one module that generates the price sheet, both llms files and the JSON-LD offers on seven pages, with another test scanning every page for a superseded price.
Workflow
How a system gets ready for real usersHow we work
Four that are running now
Fail closed:
The routing platform refuses to invent anything it cannot read with confidence. An unreadable row and an untrustworthy geocode both become the same thing, which is a flagged task for a person rather than an address a van would have driven to.
Budgets in the structure:
A per-solve travel-matrix budget makes a runaway maps bill structurally impossible. The limit is part of the solver, so nobody has to remember it at 6am.
Tests that block the merge:
1,278 tests cover the routing rules. On the food-truck site, 141 Vitest tests guard the URL structure, including one that fails the run if a second page claims the search query the owner page holds.
One source for a published number:
Published prices on that site live in one module that generates the price sheet, both llms files and the JSON-LD offers on seven pages, and a test fails the run if a superseded price reappears in any page's copy.
We hold the pager
We stay on after launch. We fix what breaks and hand over once your team wants it. The same people are still reachable six months after launch.
Why that belongs on this page
An evaluation written the week before launch describes the week before launch. The ones that matter are still running six months later, when the data has drifted and somebody has pasted something nobody anticipated into a field. Being reachable is part of the guardrail.
Who this is for
You want a system that still behaves the way you agreed it would once real users are inside it. The people who scope your project are the people who write it, and they are still reachable six months after launch.
