AI features feel untestable because the output changes from run to run. That feeling hides a useful fact: most of an AI feature is not the model. It is the code that builds the prompt, validates the answer, stores it, calls other systems, and decides what the public can see. All of that is ordinary code, and it can be tested the ordinary way, without calling a model.

On this site the AI work covers an article-drafting agent, three MCP servers, and a WHMCS integration. Here is how they are tested, in layers.

Layer 1: fake the agent

The Laravel AI SDK lets you fake an agent with canned structured responses. The test then exercises everything around the model:

ArticleWriter::fake([
    [
        'title' => 'Shipping n8n Workflows That Survive Production',
        'slug' => 'shipping-n8n-workflows',
        'excerpt' => 'A practitioner take on workflow reliability.',
        'body' => "## Why this matters\n\nProduction workflows need real error handling.",
        'topic_tag' => 'n8n',
        'meta' => ['confidence' => 'high', 'sources_cited' => ['https://example.com/n8n-post']],
    ],
]);

$draft = app(GenerateArticleDraftAction::class)->handle($item);

$this->assertSame(DraftStatus::PendingReview, $draft->status);
$this->assertSame(SourceItemStatus::Used, $item->fresh()->status);

The fake also records what the agent was asked, so a test can confirm that the grounding actually reached the prompt:

ArticleWriter::assertPrompted(function (AgentPrompt $prompt): bool {
    return str_contains((string) $prompt->agent->instructions(), 'n8n ships a new workflow feature');
});

That second assertion matters. A refactor that silently drops the source material from the instructions would still produce plausible drafts. Only a test like this notices.

Test what you do with bad output

The most valuable tests feed the code the answers you hope the model never gives:

  • An empty body must throw before anything is saved, and the test checks that no draft row exists afterwards.
  • A topic tag outside the allowed list must fall back to the source's tag.
  • A slug that is already taken must get a suffix rather than overwrite another draft.

Models will eventually return every one of these. The question is whether your code was tested against them first.

Layer 2: fake the HTTP

The WHMCS integration goes through Laravel's HTTP client, so tests use Http::fake() and then assert on the exact request that would have been sent:

Http::fake([
    '*' => Http::response(['result' => 'success', 'status' => 'available'], 200),
]);

DomainServer::tool(CheckDomainAvailabilityTool::class, ['domain' => 'example.com']);

Http::assertSent(fn ($request) => $request['action'] === 'DomainWhois'
    && $request['domain'] === 'example.com'
    && $request['responsetype'] === 'json');

There are separate tests for an unavailable domain, an uppercase domain that must be lowercased, WHMCS returning an error inside a successful HTTP response, and WHMCS being unreachable. The last two are where integrations break in real life.

It is also worth calling Http::preventStrayRequests() in tests, so any request without a fake fails loudly instead of reaching a real API. The client classes behind these tests follow the patterns in API Integration Patterns I Use in Every Laravel Project.

Fakes are worth shipping with your own packages, too. Our open-source Laravel SMS gateway package, the client for the SMS service VividSol runs for its applications, includes test fakes so an app can assert what it would have sent without sending a real message.

Layer 3: test MCP tools through their server

laravel/mcp lets a test call a tool the way a client would and assert on the result:

$response = ArticleServer::tool(SubmitArticleDraftTool::class, $payload);

$response->assertOk()->assertHasNoErrors()->assertSee('pending_review');

The rejection cases get the same treatment: an invalid topic tag must produce an error and leave the drafts table empty. For a tool that writes data, "nothing was stored" is the assertion that protects you.

Layer 4: test the boundary that matters most

For this site, the most important rule is that nothing an AI writes is public until a person publishes it. So there is a test that creates a pending draft, checks that it is missing from the Insights page and that its URL returns a 404, publishes it through the same action the admin uses, and checks that it now appears in both places.

If you test only one thing about an AI feature, test its boundary with the outside world.

Layer 5: evaluate the model separately

None of the above says whether the drafts are any good. That is a different question, and it needs real model calls, so it does not belong in the main suite.

Pest 5 made this explicit with an evals plugin: it scores model output through the familiar expect() API, combining exact checks with AI-based scorers. Because evals call real models, they are skipped by default and run only with the --evals flag. This project uses PHPUnit, but the same split works there with a group that the normal run excludes.

For a writing agent, my evaluation criteria would be concrete:

  • No version numbers or dates that are not in the source.
  • Every cited URL appears in the source material.
  • Confidence is "low" when the source is a two-line summary.
  • No banned filler phrases.

Run evals before changing the model, the prompt, or the guidelines, and compare against the previous results. That is how a prompt change stops being a matter of taste.

What runs in CI

Only the deterministic layers: fakes, no network, and fast. A CI run that depends on a model's mood will fail randomly, and a suite that fails randomly gets ignored.

Deterministic checks are not limited to unit tests. EduManage, our school management software, runs automated accessibility checks against WCAG 2.1 AA on every build. The principle is the same: if a rule matters, make a machine check it every time.

My take

Do not let "it's AI" become a reason to skip tests. Fake the model and test your code properly: prompt construction, validation, storage, integrations, and above all the line between draft and public. Then evaluate the model on its own schedule, with criteria written down. The first keeps the system correct. The second tells you whether the output is good enough. Mixing them gives you neither.