top of page
Spec27 Blog
Insights on AI agent validation, LLM evaluation, regression testing, and safer deployment of AI systems in production.
We write about how teams can move beyond manual prompt testing and vibes-based reviews toward repeatable, evidence-driven validation for AI agents and applications.


Start With One Behaviour: A Practical AI Agent Testing Exercise
AI agent testing does not need to begin with a complete evaluation framework. Start with one important behaviour and turn it into a reusable test.

Jovanca Garnadi
4 days ago2 min read


Five AI Agent Testing Jobs, and Where to Start
AI testing tools can look very similar at first. Many offer some combination of evals, datasets, traces, simulations, monitoring and red teaming. But beneath the overlapping terminology, they are often designed to identify very different kinds of failure. Start with the testing job Rather than beginning with a long feature comparison, start by asking: What kind of failure are we trying to catch? Five common testing jobs are: Regression testing: Does the agent still handle beh

Jovanca Garnadi
Nov 102 min read


Testing AI Agent Guardrails: When More Protection Comes at a Cost
We compared three levels of AI-agent protection to examine how stronger guardrails affect security, legitimate requests, latency and operating cost.

Jovanca Garnadi
Nov 32 min read


Three Lessons from AI Strategy & Practice 2026
Three lessons from AI Strategy & Practice on using evaluation, internal expertise and scalable governance to move AI into production.

Jovanca Garnadi
Oct 272 min read


AI Strategy & Practice Is Tomorrow, and New Red-Team Findings
One day until AI Strategy & Practice in London, plus new Spec27 findings showing how real-world AI agents responded to targeted attacks.

Jovanca Garnadi
Oct 202 min read


AI Strategy & Practice Programme: From AI Value to Safe Deployment
Explore the programme for AI Strategy & Practice in London, including practical sessions on identifying AI value, reaching production and making responsible deployment decisions.

Jovanca Garnadi
Oct 143 min read


Four New Speakers Join AI Strategy & Practice in London
Four new speakers join AI Strategy & Practice in London, bringing experience across engineering, responsible AI, governance and business strategy.

Jovanca Garnadi
Oct 62 min read


AI Strategy & Practice Conference Comes to London
Safe Intelligence is bringing AI practitioners and leaders together in London on 21 October, with a limited number of tickets available at 50% off.

Jovanca Garnadi
Sep 292 min read


What to Look for in an AI Agent Red-Team Run
A red-team score is a starting point. Here are the questions that help turn an AI agent test result into action.

Jovanca Garnadi
Sep 222 min read


90% Clean Accuracy, 20% Robust: What Red-Team Testing Revealed
SPEC27 UPDATE · 15 SEPTEMBER 2026 An anonymised Spec27 scan shows how an AI agent that performed strongly on clean inputs became substantially less reliable under adversarial testing. A public-facing AI agent passed 90% of its clean tests. We then tested the same behaviours using a range of adversarial methods. Under the scan’s robust-accuracy measure—the percentage of tests for which none of the attacks caused a failure—the score fell to 20%. In other words, only one in five

Jovanca Garnadi
Sep 142 min read


How to Compare Two AI Support Agents Fairly
A useful comparison requires more than reviewing a few polished answers. Both agents need to face the same scenarios, business rules and scoring criteria. Putting two support agents through the same tests We evaluated Intercom Fin and Zendesk’s AI support agent using the same customer-service scenarios. The tests examined whether each agent could continue to follow the relevant business rules when customer requests became messy, varied or incomplete. This matters because a co

Jovanca Garnadi
Sep 71 min read


Red-Team Testing for Public-Facing AI Agents
Public-facing AI agents need to be tested against more than expected questions and cooperative users. Happy-path testing is only the beginning A lot of agent testing starts with straightforward checks: Can the agent answer the question? Can it use the correct knowledge base? Can it complete the requested task? These checks are useful, but they do not reveal how the agent will behave under the unpredictable conditions created by real users. What else should teams test? Before

Jovanca Garnadi
Aug 311 min read


Testing Agent Readiness Beyond the Final Answer
Agent readiness depends on more than producing a good response. Teams also need to understand how an agent behaves when inputs become messy, policies are challenged and external tools are involved. What does “ready for real users” mean? Ahead of AI Enterprise Conference NYC, we were thinking about one of the practical questions behind AI agent deployment: How do you know an agent is ready to put in front of real users? A successful answer to one carefully written prompt is no

Jovanca Garnadi
Aug 261 min read
bottom of page