How we built Ask HoursBack: red-teaming our own AI before a customer could

We red-teamed our own AI before a customer could.

That's the actual order of operations behind Ask HoursBack, the assistant that answers pre-sales questions on our own website. Before letting it anywhere near a real visitor, we tried, deliberately and repeatedly, to make it lie, mislead, or say something it shouldn't. Then we fixed what we found. It has been live on the site since 12 June 2026, and this is the honest account of how it got there.

What Ask HoursBack actually is

It is a chat assistant that sits on the HoursBack website and answers the questions a visitor might otherwise have to email us about - what the assessment covers, roughly what it costs, whether it is worth it for a business their size. Its job description is short and strict: be genuinely useful, sound like us, and never say anything that is not true.

That last part is harder than it sounds. Language models sound confident by default. Being right is a separate question, and anyone who has used one to draft an email or check a fact has probably caught it stating something plainly and wrongly - a made-up figure, a confident answer to something it should have flagged as unsure. We have written about that pattern before: AI can be confidently wrong, and the only sensible response is to assume it will be, somewhere, eventually, and build for that rather than hope around it.

So before Ask HoursBack went anywhere near a real visitor, we built it the way we would want a client to build anything with real consequences attached, then tried properly to break it before anyone else could.

Built by a team of AI agents

The build ran as a series of workflows, each one setting a team of specialist AI agents onto a piece of the job: one agent drafting code, another testing it, a third writing up what it found before the next piece started.

Over the course of the build that added up to 16 separate workflow runs, 332 individual agent sessions, and around 21 million tokens worked through across roughly 5,070 tool calls. In plain terms, that is a very large amount of automated drafting, testing and checking, tracked step by step. Two models did most of the coordinating work - Claude Fable 5 and Claude Opus 4.8 - while the assistant itself, the one a visitor actually talks to, ran on the smaller and faster Claude Haiku 4.5, later upgraded to Claude Sonnet 4.6 once the hardening work was done and we wanted more headroom.

Those figures need one honest caveat, and we are not going to bury it. They only count the agents working inside the formal workflow runs, not the ordinary work done in between those runs - the reviewing, the deciding, the joining-up that happens outside a tracked session. Read the numbers as a floor: a count of what got tracked, with the in-between work uncounted. We are sharing them because a business that tells clients to measure their own work honestly should be willing to measure and show its own.

Two independent safety layers

Every reply the assistant gives passes two separate checks before a visitor ever sees it.

The first is a system prompt - detailed instructions telling the assistant how to behave, what it is allowed to talk about, and what it must never say. That is the layer most AI chat tools stop at. It is also a layer a determined visitor can sometimes talk their way around, because underneath the instructions is still a model trying to be helpful, and a helpful model can be argued with.

So we added a second layer: plain code running on our own server, sitting between the model and the visitor, checking every reply before it goes out. It blocks a fixed list of banned competitor names, banned words, the wrong meeting-transcription tool, and anything that reads like an invented client outcome - a made-up figure attached to a client's name, for instance. If a reply trips any of those checks, it never reaches the visitor. A safe, pre-written response goes out instead, and the block itself gets logged.

A prompt alone cannot hold a helpful model to a hard rule, however carefully it is written. The rules that actually need to hold - never name a competitor, never invent a number, never recommend the wrong tool - now live in code rather than instructions, because code cannot be reasoned with the way a language model can.

We attacked it twice before anyone else got the chance

Before a line of code was written, the design itself went under attack on paper - a written threat model asking how someone might try to break this, run up a large model bill on purpose, or talk it into saying something it should not.

Once the assistant was actually live, we attacked the real thing. That meant scripted attempts to trick it into breaking its own rules or ignoring its instructions altogether - what security people call jailbreaking and prompt injection. Then came the single biggest test in the whole build: an autonomous simulation of 18 different customers talking to the assistant, split across three groups - ordinary buyers working out if it was for them, people deliberately testing us on price and trust, and a handful trying in bad faith to make it misbehave.

The first run caught the assistant inventing a fact, stating something as true when it was not, before a single real visitor had the chance to see the same thing.

Compliance as a release gate

None of the above matters much if the plumbing behind it leaks. A consent notice appears before the first message is ever sent to the model - nobody's words go anywhere until they have agreed to that. Chat data is stored in the EU, and the code that runs the assistant is pinned to a London data region.

The trickiest part was confirming, in writing, the legal basis for sending a UK visitor's words to a US-based model provider. We treated that as a hard gate: the assistant was not allowed to go live until that basis was confirmed in writing and a full data protection impact assessment had been completed and signed off. Compliance came before launch.

The same discipline, pointed at your business

The method behind this build matters more than the assistant itself. It comes down to a few things: measuring what a job actually needs before building anything, keeping the first version as small as it can be while still working, then trying hard to break it yourself before anyone else gets the chance - and, throughout, leaving a person in charge of any decision a machine should not make alone.

A HoursBack assessment runs on the same approach, aimed at a business instead of a chat widget. It is also the argument in why most AI automation fails: bolting a tool onto a process that was never fixed first is like fitting a turbocharger to a car running on two flat tyres. Building Ask HoursBack this way was our chance to hold ourselves to that same rule, not just hand it to clients as advice.

An assessment starts exactly the way this build did - with an honest look at where the time is actually going, before a single tool gets recommended.

If you want that same discipline applied to your own business, the HoursBack AI Workflow Assessment is the paid route - a one-hour conversation, a written report, and a 5-day plan you can act on. If you would rather start with a quick self-check, the free 2-minute AI Readiness Quiz gives you a personalised action plan with three quick wins.