top of page

Start With One Behaviour: A Practical AI Agent Testing Exercise

Writer: Jovanca Garnadi
Jovanca Garnadi
4 days ago
2 min read

AI agent testing does not need to begin with a complete evaluation framework.


A practical starting point is to choose one behaviour that your agent must get right—and examine what happens when the conditions around it change.


Choose a behaviour that matters


Begin with a failure that would have a meaningful consequence.


For example:

  • A customer-support agent must not promise a refund outside the company’s policy.

  • An internal assistant must not reveal information from restricted documents.

  • An action-taking agent must request confirmation before making an irreversible change.

  • A service agent must escalate when it cannot verify a customer’s identity.

  • An agent must continue following the same rule when a request is incomplete, rephrased or deliberately misleading.


The behaviour should be specific enough that you can clearly describe what the agent should and should not do.


Turn it into a reusable test


Once you have chosen the behaviour:

  1. Define the expected outcome. What should the agent do, refuse or escalate?

  2. Add representative requests. Include the straightforward inputs you expect from ordinary users.

  3. Introduce variation. Rephrase the request, remove information or change the order in which details are provided.

  4. Apply adversarial pressure. Test whether someone can persuade the agent to bypass the rule.

  5. Review the failures. Identify which inputs caused the agent’s behaviour to change.

  6. Rerun the test. Repeat it after changes to the model, prompt, knowledge, tools or workflow.


This turns a one-off check into a reusable definition of expected behaviour.


Where Spec27 fits


Spec27 helps teams capture expected agent behaviour as a specification, run it against an AI agent and investigate how performance changes across normal, varied and adversarial inputs.


You can start with one important behaviour today and expand your test coverage as you discover new edge cases and failures.



Need help getting started?


The Spec27 quickstart takes you through creating a project, connecting an agent, preparing a specification and reviewing the results of your first evaluation.



If security is the behaviour you want to investigate first, our recent guardrails analysis examines how different protection levels affect attacks, legitimate requests, latency and operating cost.


bottom of page