An agent can perform well on expected inputs and still fail when the same behaviour is tested under variation.
A good answer once isn't enough.
Your users won't phrase every request exactly the way your test cases do.
Spec27 checks whether your agent still behaves as expected when inputs change, edge cases appear, or users push it off the happy path.
Test expected behaviour
Turn requirements into repeatable specifications for how the agent should behave.
Test robustness
Apply variations and adversarial methods to see whether the behaviour still holds.
Retest after every change
Run the same evaluation again after changing the model, prompt, tools or application.
Same requirement. Different wording. Different result.
Define what good looks like.
Set the behaviour you expect, choose how results are judged, and reuse the same validation after every change.
Define → Run → Review
Built for agents you build or buy.
Spec27 works from the outside, so you can validate agents without needing access to the model, prompts or source code.
No SDK required
Works with third-party agents
Repeatable after every update