The demo gap

Somebody on your team built a working version of the thing over a weekend. It demos beautifully. The board and a customer have seen it, and someone has already asked when it ships.

Building it was the right call. It was cheap, and it de-risked the idea faster than any specification document would have. The trouble starts a bit later, at the moment everyone in the room quietly agrees that the hard part is behind you.

A prototype is a hypothesis and a system is a promise. They’re different jobs with different ways of failing, and from the outside the distance between them looks like the last 10% of the work. In practice it’s the lion’s share, and it happens to be the part AI is currently worst at.

A demo has to work once, for one person, on one path, with data somebody picked. A system has to work every time, for everyone, on paths nobody thought of, with data that arrives broken or hostile or missing, and it has to keep doing that for years while a team who didn’t write it keeps changing it.

Think of it as the difference between a show flat and the building. The show flat is real; you can walk through it, the taps work, the light is nice. Nobody would buy the building on the strength of it.

Concretely: the demo has a happy path, and the system needs the ugly ones too, like the upload that times out at 94%, the user who hits back mid-transaction, the third-party API that returns a 200 with an empty body. In the demo there’s one user, and that user is you, so nothing needs tenancy, roles, or rate limits. In the system, the second customer must not see the first customer’s work, and neither of them should be able to take the service down with a loop.

Then there’s load. It worked when you ran it, and it now has to work when four hundred people run it at 09:00 on a Monday, which is a different claim about queues, pools, and timeouts than “it worked on my machine” ever made.

And there’s what you can prove afterwards. A demo has tests that pass, where a system needs tests that would fail if the code were wrong, and that’s a much rarer thing than a green pipeline. Where the demo has a screenshot and whatever the model happened to choose, the system needs an audit trail and decisions somebody made on purpose and can still explain eighteen months later in a post-incident review.

We build prototypes ourselves, and they’re a useful artefact. The mistake is arithmetic: the prototype gets treated as 80% of the system when it’s closer to the first 5%, and it’s the 5% that carries none of the risk.

Two findings are worth having in your head, and neither of them says AI is bad.

The first is about what generated code does with security. Veracode, who sell code scanning, updated its AI code security work in spring 2026. They ran more than 150 models through 80 coding tasks in four languages, each task written so there was a secure way to finish it and an insecure one. Almost all the code came back syntactically fine. 55% of it passed the security check, which leaves 45% carrying one of the flaws they were testing for, and cross-site scripting passed only 15% of the time, and log injection 13%.

It’s vendor research, so keep that in mind. The trend is the part we’d still take seriously: that pass rate has sat around 55% for two years while the models improved at nearly everything else. (Veracode)

That makes sense once you look at what’s being asked. Syntax has an answer the model can learn. A secure implementation is a separate decision about what the system must never allow, and if nobody states that decision, nobody makes it.

The second is Google’s DORA research, and it’s the framing we keep coming back to: AI amplifies whatever organisation it enters. Adoption raises throughput and instability together, so teams with real testing and clear workflows compound their gains, while fragmented teams get faster at producing their own technical debt. (DORA)

So before you ask whether to use AI, ask what AI would be amplifying in your organisation right now.

Here’s what this looked like on a real engagement. A creative technology partner came to us with a vibe-coded prototype of an animation production platform. It was good. It demonstrated the idea, it convinced the people it needed to convince, and it had been built fast. There was no brief – there was a demo, and a request to own every technical decision from that point on.

A partial list of what stood between that demo and a system, none of it visible at demo time:

There was no authentication and no permission management at all. A prototype has one user and trusts them completely, so everything about who may see, edit, or export a given asset had to be designed from scratch. There was no tenancy isolation either, which is fine when all the data belongs to one person and is the entire product when the system holds several client organisations. Authorisation has to come from the session, never from what the request body claims about itself. And there were no rate limits, so expensive rendering jobs could be fired off as fast as anyone cared to click.

A schema was defined in two repositories, and the two definitions had drifted apart. Both sides passed their own tests and disagreed in production about what a field meant. Contracts have to be defined once and shared, with parity fixtures and automated drift detection, or client and server diverge quietly and you find out from a customer.

The data model was wrong in one central place. It made sense on day one and stopped making sense the moment real production workflows touched it. This was the one-way door on that project: a decision taken before anyone knew what the product would become, and expensive to walk back by the time we found it. We retired it with a named migration path and a planned, deliberate break, rather than a compatibility shim that would have lived forever and taxed every feature built after it.

Configuration was compiled in. Fallback content was baked into the binary so the demo would never show an empty screen, which is fine for a demo and misleading in production. We replaced it with real configuration, so a broken config shows up as an error instead of a working screen.

And we deleted tests. The suite was green, and in places it was empty: those tests passed whether or not the code was correct, because they asserted on mocks, or on the shape of a response rather than what was in it. A green suite that asserts nothing is worse than no suite, because it sells you confidence you haven’t earned.

That’s the whole list, and none of it was visible at demo time. Every item on it would have surfaced later, at several times the cost.

None of the work above is hard because the code is hard. It’s hard because you need to know what the system has to be true about: which decisions are one-way doors, which invariant has to hold across two repositories, which test is lying, what “done” means for this business rather than in general.

Give a model a vague ticket, and you get plausible code that closes the ticket. Give it a ticket that names the acceptance criteria, the architectural decision it must respect, the file it must not break, the assertion that now has to be real, and the boundary it must not cross, and you get production code. The model is the same in both cases. What’s different is twenty years of knowing what belongs in the ticket.

That’s also why I’m not convinced this fixes itself with the next release. A better model given a vague specification produces more convincing wrong code, faster, so specification and verification get more valuable as the models improve.

We use AI heavily ourselves. Our agents have their own accounts in the tracker; they get assigned tickets, and they can’t pick up work until a human has marked the specification complete enough to trust. (The gate is for our benefit more than theirs.) We’d still rather have the amplifier than not, as long as we know what it’s amplifying.

If you have a prototype and a date you’ve promised someone, this is worth doing before anyone writes a line of the “real” version.

• Read the prototype as a document of intent rather than as a codebase. What is it claiming? What is it faking? What did it quietly decide on your behalf? Most prototypes contain at least one architectural decision nobody made deliberately, and it’s cheaper to find it now.
• Find the one-way doors you’ve already walked through: data models, identity, tenancy, anything customer-visible. These get more expensive every week they stand.
• Audit your tests before you trust the number. Take three passing tests, deliberately break the code they cover, and see whether they fail. If they don’t, your coverage figure is decoration.
• Measure throughput and stability together. DORA’s point is that watching one without the other is how organisations get very fast at breaking things.

This is a practice you keep doing, and it’s the one we’ve been at for twenty years. We were doing it before the code was generated, and generated code only makes it more necessary.

If you’re standing at this gap right now, ping us – this is what our Production Readiness Review is for. It’s one to two weeks sized to the codebase, at a fixed cost agreed up front and with read-only access, and you get a written assessment: what your prototype is claiming versus what it’s doing, the one-way doors already behind you, the security and tenancy exposure, an audit of which tests actually test something, and a prioritised hardening plan with estimates. You walk away with a document and a decision. What you do with it is up to you.