← Home

What Building With AI Actually Costs

The failure modes that show up when an AI writes the code, and why most of them would never survive a normal engineering process.

Most writing about AI-assisted development argues about whether it works. That question is settled in my experience: it works, and the projects on this site would not exist at this scale without it. The more useful question is what it costs, and the answer is not bad code. The cost is a specific category of failure that traditional engineering practice had quietly solved, and that AI-assisted development reintroduces. This essay is that category, drawn from real incidents rather than general concern.

Where the machinery does not reach

A professional codebase is surrounded by machinery that catches errors before a human sees them: compilers, linters, review, and framework conventions. AI-assisted development produces a lot of artifacts that fall outside all of it: configuration files, deployment scripts, glue between systems, SQL run through a command-line tool, and markdown that controls behavior. These are the places where nothing checks your work, and where an AI, working fast and plausibly, will hand you something that looks right and does nothing at all.

The clearest example from my own work is a text-formatting tool whose configuration was correct in every field and sat in a folder where configuration is ignored. It did nothing for months, with no error, because from the system’s point of view nothing was wrong. I then made the same class of mistake a second time while renaming it. The skills library tells that story in full; here it stands for the whole category.

Silence is the common thread

Once I started looking, the pattern was everywhere. A deployment verification step reported a fix as missing across three successful deployments, because its pattern-matching produces false negatives against binary data; a full debugging cycle chased a problem that did not exist. A database script with ordinary typographic punctuation deployed cleanly and quietly wrote corrupted characters, because the command-line tool reads files in an older encoding. A script that never named its database created its tables in the server’s default one; it succeeded, in the wrong place. A comment placed between two branches of a template conditional silently broke the chain and rendered the condition on screen as text. And a retry loop with no memory of prior failures retried a broken operation every few seconds for five weeks, until the log table reached roughly 41 gigabytes. Nothing crashed. Nothing alerted.

None of these produce a stack trace. All of them produce confident, plausible, wrong behavior, and a fast development loop is very good at generating more of them before you notice the first one.

Confident wrong answers

The subtler category cost me two documented conclusions that turned out to be false: a capability failure that was really an unlucky draw, and a lucky first attempt I promoted into a documented strength. Both are corrected in place on the evaluation page. The rule that came out of them: no single run drives a routing, deletion, or purchasing decision, because one clean result from a system that speaks fluently feels like evidence and is not. How the first error surfaced says something too: someone challenged my conclusion with a source that was itself partly mistaken, and chasing it down anyway is what uncovered the real flaw.

Documentation becomes executable

In a conventional project, stale documentation is a nuisance. With AI-assisted development it is an instruction to do the wrong thing, delivered with full authority, to something that acts on it immediately. I ran into this on the warehouse project: a committed plan described an all-or-nothing migration of 149 database pointers, a later decision replaced it with an incremental approach, and both versions existed at once. The fix was not updating the document but writing an explicit note that a specific section is now wrong, what supersedes it, and the trap the stale version would create. The habit that followed: when a change alters behavior, the documentation update belongs in the same commit.

What works

None of this argues against building this way; it argues for compensating discipline. Verify the artifact, not the intent: read the deployed binary, query the real database, never the assistant’s report of success. Assume silence means untested. Write down the mistakes, not just the fixes, in files the next session reads. Discuss before code, every session, because the constraint is not trust but keeping a mental model of a system growing faster than a person naturally absorbs. And multi-sample anything probabilistic, because one good answer is an anecdote.

The build that was dropped

Not every attempt earned a page on this site. We tried to build our own solution around a vendor reporting product, a modern interface in front of the reports it serves. Even with AI assistance, the time the build needed was more than we could give it, so we made the call and dropped the work. AI lowers the cost of building, but it does not lower it past the cost of buying, and it does not add hours to the people doing the building. Some builds still lose to the product that already exists, and this one did.

What it does not fix, which is most of the timeline

This is the correction I would most want made if someone concluded from this site that AI-assisted development compresses delivery. Getting the first application I built this way to a state where an operational team fully relies on it is roughly a year’s work, and the code is not the part taking a year: review cycles, security approval, testing with the people who do the work, training, and waiting for a safe moment in an operational calendar barely notice how the code was written. What the tooling changed was whether the application could exist at all, which is a different and narrower claim. And the gap between writing fast and being exercised slowly is where every silent failure above gets easier to introduce and slower to discover.

The honest summary is that AI-assisted development moves the difficulty instead of removing it. Less time writing code, considerably more time verifying that what got written does what it appears to do. That trade has been worth it for me, several times over, but it is a trade, and pretending otherwise is how the 41-gigabyte log file happens.

What still surprises me

The part that still surprises me is the low end. I remember spending days getting a Raspberry Pi set up to do anything at all, and the same class of weekend project, a dashboard of one’s own life, a small tool for one person, is now a prompt and an evening. After a year of measuring this work I expected the surprise to wear off. It has not, and the fun things are the ones that keep proving the point.

Have a comment on this page? Send it to me →

Home