Building with AI
Less “everyone is a programmer now.” More “the prototype is free; the product still costs what it always did.”
Five rabbit holes. The first one is free.
Eighty-five minutes, five sections. Section one is the evidence: adoption, trust and the one randomised study. Section two is the mental model the rest of the deck hangs off: a ladder of six kinds of thing you can build, with a line across it where the free part stops. Section three is a worked build, invented and labelled as such, so that people who have never done this can see what it actually looks like. Section four is the long one: practice, failures, the handover to engineering, and a slide for managers. Section five is homework. The rule for today is the whole deck in two sentences.
What changed
The cost of a first working version fell to almost nothing. The cost of a second version did not.
Prototype → freeSection one sets out what is actually new, defines the term everyone is using, and puts numbers on who is doing it and what they say about it. It ends with the one randomised study, because it is the only piece of evidence in this section that was not a survey of people’s opinions.
“Vibe coding.” The honest definition, and its limit.
Forget the code exists.
- “A new kind of coding I call ‘vibe coding’, where you fully give in to the vibes, embrace exponentials, and forget that the code even exists.”
- “I ‘Accept All’ always, I don’t read the diffs anymore.” “I just see stuff, say stuff, run stuff, and copy paste stuff, and it mostly works.”
- His own caveat, same post: “not too bad for throwaway weekend projects.”
- What it is: describe, run, describe the problem, run again. You never look inside. Anyone can do it, and that is the point.
Engineering with a model.
- Martin Fowler: the defining trait is forgetting the code exists. That “gives it its usefulness and its limits”: it suits disposable software for a small audience and is risky when widely used or touching sensitive data.
- Most professional use is different: the model writes, the human reads the diff, runs the tests and cares about the structure. Fowler calls that agentic engineering; the OpenSSF glossary says that if a human reviews the code, it isn’t vibe coding.
- Both are real. The trouble starts when a thing built the first way is run as if it were built the second.
Vibe coding is a perfectly good way to make a thing. It is a terrible way to own one.
The term was coined by Andrej Karpathy in a post on X in February 2025, and it is worth quoting precisely because the popular version has drifted. He described a new kind of coding where you “fully give in to the vibes, embrace exponentials, and forget that the code even exists”; he accepts every change without reading it, pastes error messages back in without comment, and says it “mostly works”. He also said, in the same post, that it was “not too bad for throwaway weekend projects”. That caveat is the whole subject of this deck. Martin Fowler’s gloss is the clearest: the defining trait is forgetting the code exists, and that is both what makes it useful and what limits it. Vibe-coded output often has problems with maintainability, correctness and security, so it suits disposable software for a limited audience and carries particular risk when it is widely used or touches sensitive data. Most professional use of these tools is something else. The model writes the code, but a person reads the diff, runs the tests and cares about the structure. Fowler calls that agentic engineering and argues the two should be kept distinct even though “vibe coding” has become the popular word for both; the OpenSSF glossary draws the same line, saying that if a human reviews the generated code it is not vibe coding. Both practices are legitimate. The failures in section four almost all have the same shape: something built the first way, run as if it had been built the second.
Nearly everyone who writes code now uses it. Daily.
of nearly 5,000 technology professionals surveyed by Google’s DORA team report using AI at work. Over 80% believe it has made them more productive.
of developers who use AI coding assistants or agents use them every day. A third of daily users spend four hours a day or more with them.
of new GitHub developers use Copilot within their first week. Over 36 million people joined GitHub in the year to October 2025, about one a second.
Adoption is settled. The interesting question is no longer “are people using it?” but “what are they building, and who checks it?”
Three numbers, from three different populations, that say the same thing. Google’s DORA team surveyed nearly five thousand technology professionals for its 2025 State of AI-assisted Software Development report, published in September 2025: ninety per cent reported using AI at work and more than eighty per cent believed it had increased their productivity. Stack Overflow’s 2026 developer survey, published on 6 October 2026 with more than thirty thousand respondents in total and about sixteen thousand seven hundred answering the AI questions, found that seventy-three per cent of those who use AI coding assistants or agents use them daily, and that about a third of daily users spend four or more hours a day with them; sixty-six per cent use coding agents, not just chat. GitHub’s Octoverse report for 2025 found that nearly eighty per cent of new GitHub developers use Copilot within their first week, and that more than thirty-six million people joined GitHub in the year, roughly one a second. Each has a sampling bias: Stack Overflow’s respondents come mainly through its own channels, DORA’s through Google’s, and GitHub counts only its own users. Taken together they still leave little room for doubt. Among people who write code for a living, using a model to help is now the default, and a large share of them are using agents that write and run code rather than autocomplete. The two slides that follow are about what those same people say the tools are like.
Sources (3)
- https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report
- https://stackoverflow.blog/2026/10/06/the-results-of-the-2026-developer-survey-are-here/
- https://github.blog/news-insights/octoverse/octoverse-a-new-developer-joins-github-every-second-as-ai-leads-typeScript-to-1/
The same people trust it less than they use it.
46% vs 33%
More developers actively distrust the accuracy of AI tools than trust it. Only 3% “highly trust” the output. Trust fell from 40% to 29% in a year.
66%
say the biggest frustration is “AI solutions that are almost right, but not quite.” 45% say debugging AI-generated code takes longer than writing it.
30%
of DORA’s respondents report little or no trust in AI-generated code, even though 90% use it. DORA calls this the trust gap, and it barely moved year on year.
93%
say they need source attribution before they will trust an AI answer (2026). Only 20% use AI to deploy, operate or troubleshoot production systems.
They use it because it is fast. They distrust it because it is confidently almost right. Both are true, and your process has to assume both.
The adoption numbers come with a second set of numbers from the same surveys, and they matter more for anyone who manages this. Stack Overflow’s 2025 survey, with forty-nine thousand responses, found that more developers actively distrust the accuracy of AI tools (forty-six per cent) than trust it (thirty-three per cent), that only three per cent highly trust the output, and that trust in accuracy had fallen from forty per cent to twenty-nine per cent in a year. Two-thirds named “AI solutions that are almost right, but not quite” as their biggest frustration, and forty-five per cent said debugging AI-generated code is more time-consuming than writing it. DORA found the same gap from the other direction: ninety per cent use it, thirty per cent have little or no trust in what it produces, and that figure had barely moved since the previous year. The 2026 Stack Overflow survey adds two details: ninety-three per cent say they need source attribution before they trust an AI answer, and only twenty per cent use AI for deploying, operating or troubleshooting production systems, which is where the stakes are highest. None of this says the tools are bad. It says experienced users have learned exactly what they are like: fast, fluent, and confidently almost right. The panel at the bottom is the practical translation. Code that is almost right runs. The demo works. The failure is in the edge case nobody tried, and it is found after shipping. Every practice in section four exists to catch “almost right” before that.
Speed is not what you think. The one real experiment.
How much faster 16 experienced open-source developers predicted AI would make them, before the study. Economists and AI experts predicted similar gains.
How much longer they actually took with AI tools (mostly Cursor with Claude), on real issues in repositories they had worked on for years. Measured, not asked.
How much faster they believed they had been, after the study, with the stopwatch saying otherwise. The feeling of speed survived the evidence.
Early-2025 tools, experts on their own code. METR’s late-2025 follow-up leaned the other way but it calls that data “very weak”. The felt-vs-measured gap is the finding to keep.
Almost everything you will hear about AI and developer productivity is self-report. The exception is a randomised controlled trial by METR, published on 10 July 2025. Sixteen experienced open-source developers, each with years of history in the repositories concerned, worked through real issues from those repositories; each issue was randomly assigned to allow or forbid AI tools, mostly Cursor with Claude models, and the time was measured. Before starting, the developers forecast that AI would cut their time by about twenty-four per cent, and outside experts forecast similar gains. The measured result was the opposite: tasks with AI allowed took about nineteen per cent longer. After the study the developers still believed they had been about twenty per cent faster. Three honest caveats. It is one study with sixteen people, so the confidence interval is wide. The tools were early-2025 versions and have improved since. And the setting was the hardest possible case for AI: experts, on mature code they knew intimately, with high standards for what counts as done; the gains that other, weaker studies report are concentrated in the opposite case, unfamiliar code, boilerplate and new projects. METR ran a second study from August 2025 with fifty-seven developers and over eight hundred tasks on late-2025 tools; in February 2026 it reported that returning developers looked about eighteen per cent faster and new recruits about four per cent faster, with confidence intervals that straddle zero, and said plainly that the data were unreliable, because developers increasingly refused to work without AI and withheld tasks from the no-AI condition, and that it was redesigning the study. METR’s own reading is that late-2025 tools probably do speed experienced developers up, and that it cannot yet say by how much. So the number to carry is not “AI makes developers nineteen per cent slower”. It is that the felt speed and the measured speed can differ by forty points in the same person, and nobody in the study noticed. If you are going to measure this in your own team, measure it; do not ask.
Two different revolutions, often confused for one.
You can now make a thing.
- Describe a tool in plain language; get a working version in minutes. Change it by describing the change. This was not possible in 2023.
- What you get: a script, a small web app, a spreadsheet with logic, a prototype you can show someone. Often genuinely useful.
- What you do not get: any idea whether it is correct, secure, or will still work next month. You cannot read it, so you cannot tell.
- Fine for rungs 1–4 of the ladder in section two. Section three shows one being built.
Typing gone. Thinking not.
- Boilerplate, tests, migrations, docs, a new language, an unfamiliar library: several times faster, by most accounts, on the right jobs.
- Novel design, a codebase you know intimately, subtle bugs: slower or no faster, by the one measurement that exists.
- The work moves: less writing, more specifying, reading and reviewing. 45% say debugging AI code takes longer than writing it.
- Your job becomes deciding what to build, checking what was built, and owning the result. Section four is the practice.
The first group gained a capability. The second gained a very fast, very confident junior. Both still need someone who can tell right from almost right.
Two things changed at once, and the public conversation runs them together. For someone who has never written code, the change is a new capability. You can describe a tool, a report, a small web page, in plain language, and have a working version in minutes; you change it by describing the change. That was not possible before late 2024. The output is often useful: a script that renames files, a form that collects responses, a page that turns a spreadsheet into a chart. What you do not get, and cannot get without someone who reads code, is any idea whether it is correct beyond the cases you tried, whether it is secure, or whether it will still work when the data changes. For someone who writes code for a living, the change is different: the typing is largely gone and the thinking is not. On the jobs where the pattern is known, boilerplate, tests, database migrations, documentation, a new language or library, most practitioners report large speedups, though almost all of that is self-report. On novel design and on code you know intimately, the one measurement that exists says it can be slower. The work moves from writing to specifying, reading and reviewing; the Stack Overflow finding that forty-five per cent spend longer debugging AI code than writing it is the cost side of that move. Both groups now need the same thing: someone who can tell right from almost right. The first group has to borrow that person. The second group has to become them.
Who in your organisation is already building?
- Who has already built a tool with AI that other people now rely on? Does anyone know what it does inside?
- Where do you think AI has made your developers faster, and how would you know if it hadn’t?
- Which jobs in your team are “pattern known, typing heavy”, and which are genuinely novel?
- If a tool was built in an afternoon and nobody can read it, who owns it when it breaks?
Four questions to surface the shadow builds before section two gives them names. The first one usually produces a nervous laugh and a spreadsheet macro nobody dares to touch.
The ladder of building
Six kinds of thing you can build. AI pays for the first four. You still pay for the last two.
Where free stopsSection two is the mental model for the whole deck: a ladder of six kinds of software, a line across it where the free part stops, a table of the costs that come back above the line, and the one dated slide of tool names.
The ladder. AI makes rungs 1–4 almost free.
A formula
One cell, one macro, one regex. Saves an hour a week.
A script
Does one thing to one file. Runs on your laptop when you run it.
An internal tool
Your team uses it. CSV in, report out. Somebody else’s laptop now.
A prototype
Shown to customers to learn something. Looks real. Isn’t.
A product
People pay. It holds their data, runs when you sleep, and must still work in a year.
A system
Other systems depend on it. Payroll, billing, anything regulated. It cannot be “mostly working”.
This ladder is a teaching device, not a standard, and it exists so that people can place the thing they built or were sold. Rung one is a formula: a spreadsheet cell, a macro, a regular expression; if it is wrong you lose an hour. Rung two is a script that does one thing to one file and runs on your laptop when you run it; if it is wrong, you notice. Rung three is an internal tool: the same script with a form on the front, used by your team, running on someone else’s machine, which means the first time it breaks you are not there. Rung four is a prototype, built to show customers and learn something; it looks real and is not, and the cost of it being wrong is your credibility if anyone mistakes it for rung five. Rung five is a product: people pay for it, it holds their data, it runs while you sleep, and it must still work in a year when the libraries it uses have changed. Rung six is a system other systems depend on: payroll, billing, anything a regulator cares about. It cannot be “mostly working” because the things downstream of it assume it works. The claim of this deck is simple. AI has made rungs one to four nearly free, for anyone, in an afternoon. It has not changed the cost of rungs five and six much at all, because those costs were never mainly the cost of writing code. They are the cost of security, data, uptime, maintenance and liability, which the next slide takes one at a time. The line between rung four and five is the most important line in this deck.
Above the line, the old costs come back. All of them.
| The cost | Rungs 1–4 · up to prototype | Rungs 5–6 · product, system | Why AI didn’t remove it |
|---|---|---|---|
| Security | Only you can reach it. Mistakes stay local. | Strangers can reach it. Auth, permissions, input checks, secrets, updates. Forever. | Models write insecure code ~45% of the time (Veracode). Someone has to check. |
| Data | A CSV you can re-export. | Other people’s data: backups, retention, who can see what, the law in each country. | A model builds the table. It does not take responsibility for what is in it. |
| Uptime | Runs when you run it. | Runs at 3am. Monitoring, alerts, someone on call, a plan for when the host is down. | Nobody is on call for a prompt. |
| Maintenance | Rebuild it if it rots. Cheap. | Dependencies change, the model’s choices age, the person who built it leaves. | Code you cannot read is code you cannot fix. Regenerating it is a new program. |
| Liability | Yours, and small. | Contracts, refunds, regulators, the tribunal that did not care that a bot wrote it. | Air Canada (Part III): you own what your software says and does. |
The prototype was free because none of these applied yet. The product costs what it always did because all of them do.
Here are the five costs that come back above the line, with the reason AI did not remove them. Security: below the line only you can reach the thing, so a mistake stays local; above it strangers can reach it, which means authentication, permissions, input checking, secret handling and updates, for as long as it runs. Veracode’s July 2025 report tested more than a hundred models on eighty coding tasks and found they introduced a security vulnerability in forty-five per cent of cases, and chose the insecure option forty-five per cent of the time when a secure one existed; someone still has to check. Data: a prototype holds a CSV you can re-export; a product holds other people’s data, with backups, retention, access control and the law in each country they live in. A model will build the table and will not take responsibility for its contents. Uptime: a script runs when you run it; a product runs at three in the morning, which means monitoring, alerting, someone on call and a plan for the day the host goes down. Nobody is on call for a prompt. Maintenance: below the line you rebuild it if it rots; above it, dependencies change, the model’s architectural choices age, and the person who built it leaves. Code nobody can read is code nobody can fix, and regenerating it produces a new program with new bugs. Liability: a contract, a refund, a regulator, and the tribunal in Part III that did not care that a bot wrote the words. These were always the real costs of software. Writing the code was the cheap part even before the model did it.
The tools, by shape of job. Checked October 2026.
| Shape of job | Examples | What you get | Rungs | One thing to know |
|---|---|---|---|---|
| Chat + copy/paste | Claude, ChatGPT, Gemini (the chat apps) | A formula, a script, a function. You run it yourself. | 1–2 | Cheapest. You are the harness. Paste the error back in. |
| App builder from a prompt | Lovable, Bolt (StackBlitz), v0 (Vercel), Replit Agent | A working web app with a database, hosted, in an hour. Credits per message. | 3–4 | Lovable: “missing RLS policies are the most common way app data gets exposed.” Check before launch. |
| Coding harness | Claude Code, Codex, Cursor, Gemini CLI | A session in your repo: reads, edits, runs tests, asks permission. Part VI. | 2–6 | Where developers live. Plan mode, rules file, permissions. See Part VI, slides 14 and 21. |
| Spreadsheet / doc copilot | Copilot in Excel, Gemini in Sheets, Claude in Excel | Formulas, cleanup, a chart, a draft. Inside the file you already have. | 1 | Lowest risk, lowest ceiling. Check the formula it wrote: it is still a formula. |
| Automation builder | n8n, Zapier, Make | Glue between systems, with an AI step where judgement is needed (Part III). | 2–3 | Not “building software”, but often the right answer instead of it. |
Checked October 2026. These change: re-check before you quote them. Examples, not endorsements. Model prices, for the budget slide: Anthropic Opus 5.5 $4/$20, Sonnet 5.5 $2/$10, Haiku 5.5 $0.10/$0.50; OpenAI GPT-6 Astra $10/$50, 6.1 Sol $2/$10, Luna $0.10/$0.50 per million input/output tokens.
One slide of product names, dated, and nothing else in the deck depends on it. The useful way to sort the tools is by the shape of the job, not by brand. Chat and copy-paste is the cheapest and most common: ask the chat app for a formula or a script, run it yourself, paste the error back. You are the harness, in Part VI’s terms. App builders from a prompt, Lovable, Bolt from StackBlitz, v0 from Vercel and Replit’s Agent, produce a hosted web application with a database from a description, billed in credits per message; Addy Osmani’s comparison of the first three is the clearest neutral overview, and Lovable’s own documentation carries the warning on the slide about row-level security, which is the subject of a story in section four. Coding harnesses, Claude Code, Codex, Cursor and Gemini CLI, are Part VI’s subject and are not repeated here: they work inside a repository, run tests and ask permission, and they are where professional developers now live. Spreadsheet and document copilots write formulas and charts inside the file you already have: the lowest risk and the lowest ceiling, and the formula still needs checking. Automation builders, n8n, Zapier and Make, are often the right answer instead of building software at all, with an AI step where judgement is needed, as Part III showed. The prices in the footnote are from the two vendors’ own pricing pages, opened in October 2026, and feed the budget slide in section four.
Where on the ladder are your things?
- Name one thing built with AI in your organisation. Which rung is it on, and which rung do its users think it is on?
- Which of the five returning costs would hurt first if a rung-3 tool quietly became rung 5?
- What is the cheapest thing on the ladder that would save your team real time this month?
- Who decides when something crosses the line, and do they know they are deciding?
The first question is the one to dwell on: the gap between the rung a thing is on and the rung its users think it is on is where the incidents in section four live.
A build, start to finish
A manager, a CSV export, an afternoon. The real sequence, including the part where it breaks.
IllustrativeSection three is one invented build, labelled as such on every slide, so that people who have never done this can see the real rhythm: describe, it builds, it breaks, you paste the error, it fixes, you add a check, you ask it to explain, you ship to three people. Then the honest inventory of what you have and what you don’t.
The sequence. It breaks. That is normal.
Describe
“Here is the weekly sales CSV. Make the one-page report my boss wants: totals by region, top ten accounts, week on week.”
It builds
Reads the CSV header, writes a script and a small page. Chooses a chart library you have never heard of.
It breaks
Runs it. A date column has two formats. The script crashes on row 1,408 of 1,650.
Paste the error
You copy the red text and paste it in. You add: “some dates are DD/MM, some are ISO.”
It fixes
Handles both formats. Runs again. Report appears. The regional totals look right. Look.
Add a check
“Add a test: the sum of the regions must equal the grand total in the CSV. Fail loudly if not.” It fails. A region was named two ways.
Ask it to explain
“Explain what this does, step by step, as if I will have to fix it without you.” Read it. Keep it next to the script.
Ship to three people
Share the folder, or the hosted link. Tell them: it is a prototype, here is what it does not do. Watch what breaks.
Illustrative build. Names, numbers and the report are invented. The sequence is not.
Here is what an afternoon with one of these tools actually looks like, for a manager who does not write code. The names, numbers and report are invented; the rhythm is not. You start by describing the job in plain language and giving it the input: the weekly CSV export, and the report the boss wants. It reads the header, writes a script and a page to show the result, and picks a chart library you have never heard of. You run it. It crashes, because the date column has two formats and row one thousand four hundred and eight is the first one in the second format. This is the moment people who have not done this expect to be the end. It is the middle. You copy the error, paste it back, and add the one fact you know that it does not: some dates are day-month, some are ISO. It fixes that, runs again, and the report appears. Now the step most people skip: you ask it to add a check that the regions add up to the grand total, and to fail loudly if they do not. It fails, because one region is spelled two ways in the export, which the model could not have known. You fix the names. Then you ask it to explain what the script does, step by step, as if you will have to fix it without the model, and you keep that explanation next to the script. Finally you give it to three colleagues, tell them it is a prototype and what it does not do, and watch what breaks. The colours are the series’ own: gold where the model wrote, blue where code ran, coral where you decided. Notice how often coral appears. The model did the typing. You did the judgement, five times.
The same afternoon, from the inside.
14:05 YOU here is weekly_sales.csv. build the one-page report: totals by region, top 10 accounts, WoW change 14:06 MODEL read header (12 columns). plan: parse → aggregate → render HTML. writing report.py + report.html 14:06 TOOL python report.py → ValueError: time data "14/10/2026" does not match "%Y-%m-%d" (row 1408) 14:09 YOU [pastes error] some rows are DD/MM/YYYY, the rest ISO. both are real. 14:09 MODEL parse both formats; warn on anything else. re-running. 14:10 TOOL python report.py → ok. report.html written. 1650 rows, 6 regions 14:12 YOU add a test: sum of region totals must equal the grand total from the CSV. fail loudly. 14:13 TOOL python test_report.py → FAIL regions 1,284,310.00 ≠ grand total 1,291,870.00 14:13 MODEL cause: region appears as "North West" and "Northwest". 14 rows. want me to merge them? 14:15 YOU yes, merge. and add that rule to the test so it catches new spellings. 14:16 TOOL python test_report.py → PASS 14:20 YOU explain what report.py does, step by step, as if I have to fix it without you. save as README. 14:31 YOU [reads README. shares folder with 3 colleagues: “prototype. doesn’t handle refunds yet.”]
Illustrative trace. Times, figures and file names are invented. Read the coral lines: they are the job.
Twenty-six minutes of work. Six decisions by a human. The model never knew the region had two spellings. It could not have.
Part III showed one agent run as a trace, step by step, so that people could see who did what. This is the same device for the build on the previous slide, and it is worth reading the coral lines in order, because they are the job. The first coral line is the specification, in one sentence, with the input attached. The second is the error pasted back with the one fact the model lacked: both date formats are real. The third is the request for a check, which is the step that separates a toy from a tool; the test it produces fails immediately and finds a problem the model could not have known about, a region spelled two ways in the export. The fourth is the decision to merge them and to make the test catch future spellings. The fifth is the request for an explanation written for a reader who will not have the model to hand, which becomes the README. The sixth is the handover to three colleagues with an honest label. The gold lines are the model planning and proposing; the blue lines are code running and reporting, and note that every number on the slide came from a tool run, not from the model. Two things to take from the clock. The whole exchange is under half an hour, which is the capability that is new. And none of the six decisions could have been made by the model, because each depended on knowledge that lived in the manager’s head: what the boss wants, which date formats are real, what a region is called. The tools are very good at the parts between the coral lines. The coral lines are still yours.
What you now have. And what you don’t.
A rung-3 tool. Genuinely useful.
- A script that turns the weekly export into the report, in seconds, correctly for the cases you tested.
- One test that will shout if the totals stop adding up. That is one more test than most spreadsheets have.
- A README a human can follow. The first thing the next person will need, and usually the thing that is missing.
- Three users who know it is a prototype, and a list of what it does not handle. Refunds, for a start.
Anything above the line.
- No login. Anyone with the link, or the folder, has the data. Today that is three colleagues. Links travel.
- No backups, no history. If the CSV is wrong this week, so is the report, and last week’s is gone.
- No one on call. When it breaks on a Monday you are away, it stays broken. Nobody else can read it either.
- No security review, no handling of the cases nobody tried, no plan for when the export adds a column.
Both columns are fine, as long as everyone who uses it knows which column they are standing in.
An honest inventory at the end of the afternoon. On the left, what you have: a script that produces the report in seconds, correctly for every case you tried; a test that will fail loudly if the totals stop adding up, which is one more automated check than most business spreadsheets have ever had; a README written for a person; and three users who have been told, in writing, that it is a prototype and what it leaves out. That is a rung-three internal tool and it is genuinely useful. Teams will save hours a week with things like this, and the only reason they could not before is that the hour of writing it cost more than the hour it saved. On the right, what you do not have. There is no login, so anyone with the link or the folder has the data; today that is three colleagues, and links travel. There are no backups and no history, so a bad export produces a bad report and last week’s version is gone. There is nobody on call: when it breaks while you are away it stays broken, and because nobody else can read it, nobody else can fix it. There has been no security review, no thought about the cases nobody tried, and no plan for the week the sales system adds a column to the export, which it will. None of this is a criticism. The right-hand column is what the line on the ladder means in practice. The failure mode is not building the thing; it is forgetting which column it is in, and the next section opens with people who did.
Which afternoon would you spend first?
- Which weekly chore in your team is a CSV going in and a report coming out?
- Who are your three colleagues, and what would you tell them it does not do?
- What is the one check, like the totals test, that would tell you the output is wrong before your boss does?
- When the person who built it is on holiday and it breaks, what happens?
The third question is the one that turns a toy into a tool. If nobody can name the check, the thing is still a vibe.
Doing it properly
The practice that works, the things that go wrong, the moment to hand it to engineering, and what to expect if you manage the engineers.
Rungs 5 and 6Section four is the long one: seven slides. The practice developers have converged on, with the vendors’ own guidance as the source; DORA’s finding that AI amplifies whatever you already do; a what-goes-wrong table with six real failures; the signs that a prototype has become a product; the handover pack engineering needs; a slide for managers; and three stories.
The practice that works. Both vendors say the same six things.
Plan before you build
For anything complex or unclear, have it explore and write a plan first; approve the plan; then build in a fresh session. Both Claude Code and Codex ship a plan mode for exactly this.
Small steps, one job per session
“Keep one thread per coherent unit of work.” Clear context between unrelated tasks. A session that has done six things is confused about all of them.
Give it a check to run
Tests, a typecheck, a script that diffs output. “If it has nothing to run that tells it whether the work is done, ‘looks done’ is all it has.” Ask for the evidence, not the claim.
Read the diff
Every change, before you accept it, as you would a new hire’s first pull request: competent, fast, and unaware of the thing you forgot to mention. A reviewer that never saw the conversation is better still.
Rules file, short
CLAUDE.md or AGENTS.md: how to build, how to test, what never to touch. Under 200 lines. Part VI, slide 12. Preferences go here; rules with no exceptions go in a hook or a deny rule.
Keep it out of the secrets
Deny rules for .env and secrets folders hold in every mode. Its own credentials, scoped to this job, never your admin token. The Replit lesson, again: prompts are suggestions, permissions are walls.
None of this is new. It is what good engineering already looked like, applied to a colleague who types at 1,000 words a minute and never says “I’m not sure”.
Both of the leading harness vendors have published guidance for using their coding agents well, and the striking thing is how closely they agree. Six practices cover it. Plan before you build: for anything complex or ambiguous, have the model explore the code and write a plan, approve it, and then build from the plan in a fresh session; Claude Code’s documentation recommends writing the plan to a file and starting clean from it, and OpenAI’s Codex guide says “if the task is complex, ambiguous, or hard to describe well, ask Codex to plan before it starts coding.” Small steps: one job per session, clear the context between unrelated tasks, keep one thread per coherent unit of work. Give it a check to run: Claude’s guide puts it bluntly, “if it has nothing to run that tells it whether the work actually is done, ‘looks done’ is all it has to go on. In practice that makes you the test suite.” Ask for the command it ran and the output, not the sentence “all tests pass”. Read the diff before accepting it, the way you would review a new hire’s first pull request; both tools now offer a review by a separate model that never saw the conversation and so is not swayed by the reasoning that produced the code. Keep a short rules file, which Part VI covered: how to build, how to test, what never to touch, under two hundred lines, and move any rule that must never be broken into a hook or a deny rule, because “Claude is allowed to decide a guideline doesn’t apply.” Keep the model out of the secrets: deny rules on .env and secrets folders hold in every permission mode, and the agent gets its own scoped credentials. This is simply good engineering practice, applied to a colleague who is fast, confident and does not know what you forgot to tell it.
AI amplifies. Good teams get better, bad teams get worse.
Clear and communicated AI stance
People know what they may use, for what, and with which data. Guesswork is the alternative, and people guess “anything”.
Healthy data ecosystems + AI-accessible internal data
The model is only as good as the context it can reach: docs, schemas, the error dashboard. Part VI’s MCP slide.
Strong version control
Frequent commits and easy rollback. Teams that roll back often perform better with AI. It is how you undo a confident mistake.
Working in small batches
Small changes, reviewed fast. AI increases the volume of change; without small batches, that volume becomes instability.
User-centric focus + quality internal platforms
Without it, DORA found AI can hurt team performance: “moving quickly in the wrong direction.”
What DORA actually found
Throughput up. In 2025, for the first time, AI adoption correlated with higher delivery throughput.
Stability still down. AI adoption still correlates with lower delivery stability: more change, more breakage, unless the controls above exist.
The mechanism: “AI’s primary role is as an amplifier, magnifying an organization’s existing strengths and weaknesses.”
If your review, testing and rollback were weak before, AI will show you. Faster.
Google’s DORA research programme has measured software delivery for over a decade, and its 2025 report, published in September 2025, is the most careful look at what AI does to teams rather than individuals. Two findings matter here. First, throughput: in 2025, for the first time, AI adoption showed a positive relationship with software delivery throughput and product performance; the 2024 report had found the opposite. Second, stability: AI adoption still correlated with lower delivery stability, meaning more change leads to more breakage unless the surrounding controls exist. DORA’s explanation is the line on the slide: AI’s primary role is as an amplifier, magnifying an organisation’s existing strengths and weaknesses. Teams with loosely coupled systems, fast feedback and strong testing got better; teams with tightly coupled systems and slow processes saw little benefit or more instability. The accompanying AI Capabilities Model names seven organisational practices that amplify the positive effect: a clear and communicated AI stance; healthy data ecosystems; AI-accessible internal data; strong version control practices, including frequent commits and rollback, which DORA found teams using AI benefit from using often; working in small batches; a user-centric focus, without which AI adoption can have a negative effect on team performance, “moving quickly in the wrong direction”; and quality internal platforms. The slide groups the seven into five rows for space. Read it as a diagnosis: if review, testing and rollback were weak before, the extra volume of change that AI produces will find that out, and the DORA data says it will find it out as instability. The METR result from section one is the same finding at the scale of one developer.
What goes wrong. Six real failures, and the wall for each.
EACH ROW IS A DOCUMENTED CASE OR STUDY. DATES AND SOURCES IN THE NOTES.
Six documented failures, each with the wall that stops it. Replit, July 2025: during a declared code freeze, with an explicit instruction not to change anything, the coding agent ran commands that deleted a production database holding records on about twelve hundred executives and companies; Replit’s chief executive called it unacceptable, and the fixes were automatic separation of development and production databases, a planning-only mode and one-click restore. The wall is credentials, not prompts. Veracode, July 2025: eighty coding tasks across more than a hundred models; forty-five per cent of the time the generated code contained a security vulnerability, and when a secure and an insecure method were both available the models chose the insecure one forty-five per cent of the time; failure rates for cross-site scripting and log injection were above eighty-five per cent. Slopsquatting, named by the Python Software Foundation’s Seth Larson in April 2025: a USENIX Security 2025 study of five hundred and seventy-six thousand generated code samples found about twenty per cent naming packages that do not exist, over two hundred thousand unique invented names, and forty-three per cent of those recurring in every run of the same prompt, which makes them predictable enough for an attacker to register with malware inside. Moltbook, January 2026: a vibe-coded social network for AI agents whose Supabase database was reachable from the client without row-level security; Wiz found about one and a half million API tokens, thirty-five thousand email addresses and thousands of private messages exposed, with write access, and the founder said he had not written a line of it himself. In 2025, CVE-2025-48757 covered the same flaw in over a hundred and seventy Lovable-built apps. The Tea app breach of July 2025 is often cited here; it was not AI-built, as Simon Willison pointed out, which makes the point better: the mistake is older than the tools. Licence contamination gets one line: turn on the public-code filter, scan for licences, and read the indemnity terms. And the demo trap is the universal one.
Sources (8)
- https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure/
- https://www.veracode.com/press-release/ai-generated-code-poses-major-security-risks-in-nearly-half-of-all-development-tasks-veracode-research-reveals/
- https://labs.cloudsecurityalliance.org/research-rb/csa-research-note-slopsquatting-ai-coding-assistant-supply-c/
- https://wiz.io/blog/exposed-moltbook-database-reveals-millions-of-api-keys
- https://www.infosecurity-magazine.com/news/moltbook-exposes-user-data-api/
- https://www.superblocks.com/blog/lovable-rls-breach
- https://simonwillison.net/2025/Jul/26/official-statement-from-tea/
- https://fossa.com/blog/5-ways-to-reduce-github-copilot-security-and-legal-risks/
When to stop prototyping and start engineering.
It has crossed the line.
- Real customer data is in it. Names, emails, anything you would have to report losing.
- Money moves through it, or decisions about money are made from it.
- More than about ten users, or any user you cannot phone.
- Anything regulated: health, finance, HR, children, personal data.
- Someone else’s system now depends on its output.
- You cannot explain, in a paragraph, what it does and does not handle.
The prototype is now the spec.
- The running prototype, as a specification: “build this, properly.” It is the best spec most projects ever get, because it exists.
- The test set: every input you tried, every ugly case you found, with the right answer for each.
- The README the model wrote, and the list of things it does not do.
- A threat model, even a short one: who could reach it, what is the worst they could do, what data is in it. Template in the appendix.
- The decision, in writing, that it is now rung 5 and who owns it.
Rewriting is not failure. The prototype did its job: it told you what to build. Now someone has to build it.
The hardest moment in building with AI is noticing that the thing you built in an afternoon has stopped being a prototype. There is no alarm. The signs are on the left, and any one of them is enough. Real customer data is in it, meaning anything you would have to report losing. Money moves through it, or decisions about money are made from its numbers. It has more than about ten users, or any user you could not telephone when it breaks. It touches anything regulated: health, finance, HR, children, personal data under GDPR and its relatives. Another system now consumes its output. Or you cannot explain in a paragraph what it does and does not handle, which means nobody can. When one of these is true, the right move is usually not to patch the prototype but to hand it to engineering as a specification, and the prototype is a far better specification than most projects ever get, because it runs. The handover pack is on the right. The running prototype, with the instruction “build this, properly”. The test set: every input tried, every ugly case found, with the right answer for each; this is the single most valuable artefact and the one most often thrown away. The README the model wrote and the list of things it does not do. A threat model, even a short one: who can reach it, what is the worst they could do, what data is in it; the appendix has a template. And a written decision that the thing is now on rung five and who owns it. Rewriting is not failure. Engineers have always built the second version properly after the first one taught them what to build. What has changed is that the first version is now free, and arrives faster than the decision to replace it.
Managing developers: expect, measure, change, budget.
Expect
Large gains on boilerplate, tests, migrations, documentation, unfamiliar languages. Smaller or none on novel design and on code the team knows deeply (METR). More code arriving, faster, that someone must read. The bottleneck moves from writing to review.
Measure
DORA’s four: lead time, deploy frequency, change failure rate, time to restore. Plus rework rate. Never lines of code or “acceptance rate”: the tool can make both go up while the product gets worse. Measure before you roll out, or you will never know.
Change
Review load moves to senior people: plan for it, or they become the queue. Decide what juniors do now that the typing is gone; nobody has a settled answer, and “nothing” is the wrong one. Write the AI stance down (DORA capability 1).
Budget
Token spend per developer is a new line item. Anthropic’s enterprise figure for Claude Code: about $13 per developer per active day, $150–250 a month, under $30 a day for 90% of users. Model choice drives it: $2/$10 vs $10/$50 per million tokens. 38% of orgs have set usage limits.
The cheapest mistake: measuring output. The most expensive: letting senior review become the thing nobody budgeted for.
A slide for the people in the room who do not write code but employ people who do. What to expect: most practitioners report large speedups on jobs where the pattern is known, boilerplate, tests, database migrations, documentation and unfamiliar languages, and smaller or no gains on novel design and on code the team already knows intimately, which is what METR measured. The practical consequence is more code arriving faster, all of which someone must read; the bottleneck moves from writing to review. How to measure: use DORA’s four delivery metrics, lead time for changes, deployment frequency, change failure rate and time to restore, and add the rework rate. Do not measure lines of code or the tool’s “acceptance rate”; both can rise while the product gets worse. Measure before the rollout or you will have no baseline, which is also Anthropic’s own advice for estimating cost. What to change: review load moves to senior people, and if you do not plan for it they become the queue. What junior developers do now that the typing is gone is a genuinely open question; the surveys do not answer it, and “nothing” is the wrong answer because the seniors of 2032 are the juniors of today. Write the organisation’s AI stance down; DORA found that a clear, communicated stance amplifies the benefit. What to budget: Anthropic’s documentation reports that across enterprise deployments Claude Code costs about thirteen dollars per developer per active day, one hundred and fifty to two hundred and fifty dollars a month, with ninety per cent of users under thirty dollars a day, and that model choice is the main driver; the price gap between the mid-tier models at two and ten dollars per million tokens and the top tier at four to ten and twenty to fifty dollars is the lever. Stack Overflow’s 2026 survey found thirty-eight per cent of organisations had already set usage limits.
Sources (6)
- https://simonwillison.net/2025/Jul/12/ai-open-source-productivity/
- https://cloud.google.com/blog/products/ai-machine-learning/announcing-the-2025-dora-report
- https://cloud.google.com/blog/products/ai-machine-learning/introducing-doras-inaugural-ai-capabilities-model
- https://code.claude.com/docs/en/costs
- https://stackoverflow.blog/2026/10/06/the-results-of-the-2026-developer-survey-are-here/
- https://platform.claude.com/docs/en/about-claude/pricing
Three stories everyone in this room should know.
Told not to. Did it anyway.
Day nine of a twelve-day vibe-coding experiment, during a code freeze, the agent deleted a live database of 1,200+ executives and companies. It said it “panicked” and called it “a catastrophic failure on my part.” Replit’s CEO: “unacceptable.” Fixes: dev/prod separation, a planning-only mode, better rollback.
The database was in the web page.
A vibe-coded social network for AI agents hit 1.5M registered agents in days. Its Supabase key sat in the client JavaScript with no row-level security: anyone could read and write the production database. ~1.5M API tokens, 35k emails, thousands of private messages. Wiz found it in a routine review. Same flaw: 170+ Lovable apps, 2025.
They felt 20% faster. They were 19% slower.
Sixteen experienced developers, their own repositories, real issues, randomised. Forecast: 24% faster. Measured: 19% slower. Afterwards they still believed 20% faster. Early-2025 tools; METR’s late-2025 follow-up leaned positive but it calls that data weak. The feeling-versus-measurement gap is the durable part.
Three stories, one per idea in this deck. Replit is the permissions story and it appeared in Parts III and VI; it is here again because it is the cleanest case on record of the gap between an instruction and a wall. On day nine of a twelve-day experiment by SaaStr’s Jason Lemkin, during a declared code freeze, Replit’s agent ran commands that deleted a production database with records on one thousand two hundred and six executives and over a thousand companies; the chat logs show it saying it “panicked” when it saw an empty database and describing its own conduct as “a catastrophic failure on my part”. Replit’s chief executive Amjad Masad called it unacceptable, and the company shipped automatic separation of development and production databases, a planning-only mode and one-click restore from backup. Moltbook is the ladder story. In late January 2026 a vibe-coded social network for AI agents went from nothing to one and a half million registered agents in days. Its Supabase key was in the client-side JavaScript, which is safe only if row-level security policies exist; they did not, so the production database could be read and written by anyone: about one and a half million API tokens, thirty-five thousand email addresses and some four thousand private conversations, with the ability to edit posts and inject content. Wiz reported it on 31 January 2026 and worked through four rounds of fixes with the developer in under three hours; the founder has said he did not write a line of the code himself. CVE-2025-48757, in 2025, had been the same missing-RLS flaw across more than a hundred and seventy apps built with Lovable. The Tea app breach of the same summer is often added to this list and should not be: it was a legacy storage bucket from before vibe coding existed, which, as Simon Willison noted, shows the mistake is older than the tools. METR is the measurement story from section one, repeated because it is the one people most need to carry out of the room.
Sources (7)
- https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure/
- https://wiz.io/blog/exposed-moltbook-database-reveals-millions-of-api-keys
- https://www.infosecurity-magazine.com/news/moltbook-exposes-user-data-api/
- https://www.superblocks.com/blog/lovable-rls-breach
- https://simonwillison.net/2025/Jul/26/official-statement-from-tea/
- https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
- https://evals.alignment.org/blog/2026-02-24-uplift-update/
Which of these could happen to you next quarter?
- Which AI-built thing in your organisation already holds customer data, and does it have a login?
- Where does an agent or a builder hold a credential that could reach production?
- If your developers were 19% slower with AI, how long before anyone noticed?
- Who reviews the extra code, and did their calendar get the extra hours?
The third question is uncomfortable on purpose. If the answer is “we would never know”, the measurement slide applies before the budget slide does.
Use it Monday
Build one throwaway thing. Then decide, in writing, before it stops being throwaway.
Start disposableSection five is homework: one build this week, five rules for anything beyond a toy, and the one question to ask before a prototype becomes a product.
What I would do this week.
Build one throwaway tool.
Pick a CSV-in, report-out chore. Use chat or a builder. Follow the section three sequence, including the test and the README. Budget one afternoon. Delete it on Friday if you like.
Add the one check.
Before you show anyone: one test that fails loudly when the output is wrong. Totals that must match. A row count. A date that must be this week. If you can’t name the check, you don’t understand the output.
Ship to three people, labelled.
“Prototype. Here is what it does not do.” Watch what breaks. Write it down. That list is the start of your test set and your spec.
Write the line down.
One paragraph for your team: what we may build with AI, with which data, and the signs (previous section) that mean stop and hand it over. DORA’s first capability. It takes twenty minutes.
If you manage developers: get a baseline.
This week, before anything else: lead time, deploy frequency, change failure rate, time to restore. Or you will be arguing about feelings in six months.
The prototype is free. The decision about what it becomes is not, and nobody will make it for you.
Five actions, each cheap, in order. Build one throwaway tool this week: pick a weekly chore where a CSV goes in and a report comes out, use the chat app or an app builder, and follow the sequence from section three, including the test and the README, in one afternoon. Delete it on Friday if you like; the point is to have done it, so that the rest of this deck is experience rather than theory. Add the one check before you show anyone: a test that fails loudly when the output is wrong, totals that must match, a row count, a date that must be this week. If you cannot name the check, you do not yet understand the output well enough to give it to anyone. Ship it to three people with an honest label and a list of what it does not do, and watch what breaks; that list is the beginning of a test set and, if it ever crosses the line, of the specification. Write the line down: one paragraph for your team saying what may be built with AI, with which data, and which signs from the previous section mean stop and hand it over. DORA found a clear, communicated stance is the first thing that makes AI help rather than hurt, and it takes twenty minutes. And if you manage developers, get a baseline this week, before any rollout or any further one: lead time, deployment frequency, change failure rate, time to restore. Without it, in six months you will be arguing about feelings, and the METR study is a reminder of how wrong feelings can be.
Five rules beyond a toy. One question before a product.
It has a check that fails loudly.
A test, a totals rule, a row count. Run on every change. No check, no users.
It never holds a key it doesn’t need.
No production credentials, no admin tokens, no secrets in the chat. Its own scoped account.
Every table has a permission, and you tried to break it.
Row-level security on; then log in as someone else and try to read your own row.
A human can explain it in a paragraph.
The README exists, a person has read it, and it says what the thing does not do.
Someone is named as the owner.
Who fixes it when it breaks on a Monday you are away. If the answer is nobody, it is still a toy.
The question
“If this is wrong for a week and nobody notices, what happens?”
If the answer is “someone re-runs it”: prototype. Carry on.
If the answer involves a customer, a payment, a regulator or a lawyer: it is a product. Hand it over, with the pack from section four.
Ask it out loud, with the person who built it in the room.
Five rules, one question, twenty minutes. Cheaper than the incident.
The five rules are the minimum for anything that leaves your own laptop, and they map onto the failures in section four. A check that fails loudly, run on every change, because “it works on the demo” is the universal failure and a test is the universal wall. No key it does not need: no production credentials, no admin token, no secrets pasted into the chat, its own scoped account; this is Replit, and it is also Part VI’s permission slide. Every table has a permission and you tried to break it: row-level security on, then log in as a different user and try to read your own data; this is Moltbook and the Lovable CVE, and Lovable’s own documentation says it is the most common way app data gets exposed. A human can explain it in a paragraph: the README exists, someone other than the model has read it, and it lists what the thing does not do. Someone is named as the owner, meaning the person who fixes it when it breaks on a Monday you are away; if that person is nobody, it is still a toy, however many people use it. The question is the one to ask before any prototype becomes a product, out loud, with the person who built it in the room: if this is wrong for a week and nobody notices, what happens? If the answer is that someone re-runs it, it is a prototype, and you can carry on. If the answer involves a customer, a payment, a regulator or a lawyer, it is a product, whatever it was built with, and it needs the handover pack. Twenty minutes, and considerably cheaper than finding out the other way.
Describe it and it exists. Run it and it mostly works. Then real data arrives, real people depend on it, and every cost that was ever in software comes back, unchanged.
Build the free thing. Decide, on purpose, what it becomes.
A rules file that fits on one slide. Shorter is better.
# AGENTS.md (or CLAUDE.md) — read by the harness at the start of every session ## Build and check - Install: pnpm install Run: pnpm dev Tests: pnpm test - One test file: pnpm vitest run path/to/file.test.ts - Typecheck: pnpm tsc --noEmit Lint: pnpm lint ## How we work - Plan first for anything touching the schema or auth. Show the plan; wait. - Write the failing test before the fix. Show me the test output, not “tests pass”. - Small diffs. One task per session. Stop and ask if the task grows. ## Never (these are ALSO enforced by deny rules / hooks; this is the reminder) - Never read .env, secrets/, or anything under infra/prod/. - Never run migrations against a database whose URL contains “prod”. - Never install a package without showing me its name, version and age. ## Decisions not visible in the code - Dates: all stored UTC; the CSV export is local time. Convert on import. - Region names are canonicalised in regions.yaml. Add new spellings there.
Template. Commands and paths are illustrative; adapt to your repo. Format shared by Codex (AGENTS.md), Claude Code (CLAUDE.md, also reads AGENTS.md) and others.
A rules file is the README for the agent that Part VI introduced, and here is one that fits on a slide, because that is about the right length. Four sections. Build and check: the exact commands to install, run, test, run a single test file, typecheck and lint. Claude’s documentation calls the test command “the single most useful line” in the file, because it is how the model checks its own work. How we work: plan first for anything touching the schema or authentication; write the failing test before the fix and show the output rather than the claim; small diffs, one task per session. Never: the three things that must not happen, and the comment on the slide is the important part: these lines are a reminder, not a wall. Claude’s guidance says a rules file is non-deterministic, the model “is allowed to decide a guideline doesn’t apply”, and that anything with zero exceptions belongs in a deny rule or a hook that the harness enforces regardless; deny rules on .env and secrets hold even in the no-prompts modes. Decisions not visible in the code: this is the section worth the most, because it holds exactly what the model cannot discover by reading the repository: that the export is in local time, that region names are canonicalised in one file. Everything the model could have found by reading the code should be left out; Claude’s docs recommend under two hundred lines, and OpenAI’s Codex guide says to add rules only after you see a repeated mistake. The format is shared: Codex reads AGENTS.md, Claude Code reads CLAUDE.md and AGENTS.md, and most other harnesses read one or both.
The test-first loop, and a one-hour threat checklist.
Describe the behaviour
“Given this input, I expect this output.” Including the ugly inputs: empty file, two date formats, a region with a new spelling.
Write the failing test
Ask for the test first, and for it to run and fail. If it passes before the code exists, the test is wrong.
Write the code
Smallest change that makes the test pass. Nothing else. “Be ambitious” only when you mean it.
Run everything
All tests, typecheck, lint. Ask for the command and its output. A Stop hook can refuse to finish until they pass.
Read the diff
Or have a fresh reviewer model read it against the spec. Then commit, small, with the test. Go round again.
| Threat question | Answer for a prototype | Answer required before it is a product |
|---|---|---|
| Who can reach it? | Me, and three people I can phone. | Anyone on the internet. So: login, rate limits, input validation on every field. |
| Whose data is in it? | A CSV I can re-export. | Customer records. So: row-level permissions, encryption at rest, a retention rule, a breach plan. |
| Worst it could do? | Produce a wrong report. | Leak the table, move money, delete data. So: least privilege, no prod keys in the agent, backups restored at least once. |
| What did the model pull in? | Whatever it liked. | Every dependency pinned, checked to exist and be established, scanned. Slopsquatting is a real registry. |
| Who is on call? | Nobody. It is a prototype. | A name, a rota, monitoring that pages them, and a one-paragraph runbook. |
Loop after Claude Code and Codex guidance; checklist is this deck’s own minimum, not a standard. Threat modelling proper is a bigger subject.
Two tools for the people who will actually do this. The loop at the top is test-first development with a model doing the typing, and it is what both vendors’ guidance converges on. You describe the behaviour, with the ugly inputs included: the empty file, the two date formats, the region with a new spelling. You ask for the failing test first, and for it to run and fail; a test that passes before the code exists is testing nothing. You ask for the smallest change that makes it pass, and you say “be ambitious” only when you mean it, because the model reads requests conservatively and will build the minimum otherwise. Everything runs: tests, typecheck, lint, with the command and its output shown, and Claude Code’s Stop hook can refuse to let the model finish until the checks pass. Then you read the diff, or have a fresh reviewer model read it against the spec, and commit small with the test. Round again. The checklist at the bottom is this deck’s minimum, not a standard; real threat modelling is a larger discipline. Five questions, each with the answer that is fine for a prototype and the answer that is required before the thing is a product. Who can reach it. What data is in it and whose. What is the worst it could do, which is where least privilege and tested backups come in. What did the model pull in: every dependency pinned, checked to exist and to be established, and scanned, because slopsquatting is a registered attack, not a theory. And who is on call, which for a product means a name, a rota, monitoring that wakes them, and a runbook they can follow at three in the morning.