Imagine a developer who normally writes 500 lines of code in a week.
They understand most of it.
They remember why a function exists, which alternative they rejected, where they made an uncomfortable compromise, and which section caused them to whisper something inappropriate at their monitor.
Now give the same developer an AI coding agent.
By Friday, they have produced 10,000 lines.
Let us be generous. Assume the generated code is just as good, line for line, as the code they would have written themselves.
There is still an awkward question:
Who understands the other 9,500 lines?
We made one station in the factory much faster
Software development is not one activity.
It is a pipeline:
Problem → design → code → review → testing → deployment → monitoring → maintenance
AI can help at every stage, but its most spectacular ability is generation. It can produce functions, tests, infrastructure, documentation and entire applications before a human has finished explaining why the ticket was marked “small.”
That sounds like a productivity breakthrough.
It may also be a queueing problem.
Imagine a factory that installs a machine capable of producing twenty times as many parts. Production celebrates. The dashboard turns green.
Unfortunately, the inspection station still has two people and one slightly temperamental X-ray machine.
The factory has not increased verified output twentyfold. It has increased the rate at which unverified material reaches inspection.
Software teams may be doing the same thing.
The assurance gap
I think we need a name for the difference between how quickly a team can produce software changes and how quickly it can establish that those changes are safe.
Let us call it the assurance gap.
Conceptual model—not measured data
Generation capacity may rise quickly while the capacity to do these things rises much more slowly:
- understand architectural consequences;
- identify incorrect assumptions;
- review security-sensitive behaviour;
- test unusual user journeys;
- detect interactions between services;
- monitor production behaviour;
- and diagnose failures under pressure.
This does not require AI-generated code to be unusually bad.
It only requires change to arrive faster than the organization can absorb it.
Suppose AI increases the rate of change by 30 percent rather than twentyfold. If review, testing and operational capacity remain fixed, that extra volume still has to go somewhere.
Queues grow.
Reviews become shallower.
Changes become larger.
The pressure to approve rises.
The assurance gap widens quietly.
The evidence is mixed—and that is the point
Research does not currently support a simple story in which AI either makes every developer dramatically faster or makes software uniformly worse.
GitHub studied 202 experienced developers completing a bounded coding task. Code produced with Copilot performed better on the study’s tests and received modestly higher ratings for readability, reliability and maintainability. The study is useful, although GitHub evaluated its own product and the task was deliberately constrained. (GitHub, 2024)
METR studied experienced open-source developers working in mature repositories they already knew. Using early-2025 AI tools, those developers took an estimated 19 percent longer to complete assigned tasks. METR now labels that result out of date: its 2026 follow-up suggests newer tools probably improve productivity, but participation and task-selection biases made the size of the effect difficult to estimate reliably. (METR, 2025; 2026 update)
Google’s 2024 DORA report found a similar tension at the organizational level: AI adoption was associated with benefits to individual work, while higher adoption was also associated with worse delivery stability and throughput. These are observational results, not proof that AI caused the outcomes. (DORA, 2024)
These findings do not cancel one another.
AI may be excellent when the task is bounded, context is available and correctness can be checked quickly. It may help less—or even slow work—when success depends on tacit knowledge spread across a large existing system.
The important question is not:
Does AI make coding faster?
It is:
Which part of the delivery system becomes the bottleneck when it does?
Code review is not a spell
Most software organizations have a reassuring ritual.
A developer opens a pull request.
Another developer scrolls through it.
Comments are exchanged.
Eventually, somebody presses Approve.
The workflow has now recorded that a review occurred.
That is not the same as another human understanding the change.
Consider receiving a 6,000-line AI-generated pull request late on Thursday.
You can inspect the naming. You can notice duplication. You can verify that the tests are green. You can ask another AI whether the first AI made a mistake.
But did you reconstruct the model behind the change?
Did you trace what happens when authentication succeeds but a dependency times out?
Did you notice that two individually sensible components create an unsafe interaction?
Did you question whether the feature needed 6,000 new lines at all?
A study of Chromium OS security reviews found that vulnerabilities requiring broader contextual understanding were more likely to escape review. Changes spanning more directories were also harder to assess successfully. (Thompson et al., 2021)
The code does not need to be obviously bad.
It merely needs to be larger than the reviewer’s available attention.
The hidden debt is comprehension
Before AI, writing code was not only a way of producing an implementation.
It was also one of the ways a developer learned the system.
While building a feature, they gradually discovered:
- how the data moves;
- where state is hidden;
- which dependency is unreliable;
- which assumption is doing suspiciously heavy lifting;
- and why the “two-hour change” is entering its fourth day.
The friction was not entirely waste.
Some of it was cognition.
When an AI produces the implementation quickly, the developer can inspect the result without reconstructing the reasoning that might have emerged while creating it.
The code may work. The tests may pass. The pull request may be approved.
Yet the organization’s software has acquired more behaviour than its people have acquired understanding.
I think this creates comprehension debt.
Technical debt means a system will be expensive to change later.
Comprehension debt means the system already contains behaviour its owners cannot adequately explain.
It is possible to owe both at once, which feels unnecessarily ambitious.
The bill arrives during an incident
Comprehension debt can remain invisible for months.
Then something breaks.
An engineer opens the affected component and finds generated abstractions, generated tests, generated configuration and a commit message explaining that the change “improves robustness.”
The original developer remembers asking the AI to solve the problem.
The reviewer remembers that the tests passed.
The model that produced the code has since been updated.
Nobody remembers the assumptions.
The organization owns the software legally, depends on it financially and operates it publicly.
But it understands it statistically:
This usually works.
That is not where you want to begin during an incident involving money, health information, identity or physical infrastructure.
Can AI review the AI?
The obvious response is to apply AI downstream.
Use AI to review generated code. Generate more tests. Automate threat modelling. Monitor deployments. Summarize incidents caused by the other AI.
This will help enormously.
But independent safety layers are valuable partly because they fail differently.
The developer may misunderstand the requirement.
The reviewer may challenge the architecture.
QA may behave like a confused user.
Security may behave like a hostile user.
Production may produce a condition nobody imagined.
If generation, review and testing rely on systems with similar training, context and assumptions, their mistakes may become correlated. The reviewer may approve the generator’s error because it shares the generator’s blind spot.
This is a risk to test, not a settled fact about AI review.
Adding more AI stages does not automatically create defence in depth.
Sometimes it may create the same confidence several times.
What responsible acceleration might look like
The answer is not to reject AI tools. That would be like responding to industrial machinery by insisting every chair must once again be carved by hand.
The answer is to redesign assurance around the new imbalance.
Keep changes small
AI makes enormous changes easy to generate. That is precisely why pull requests may need stricter limits on size and scope.
The ability to generate a large change is not evidence that someone can review one.
Preserve the reasoning
Future maintainers need more than a description of what changed.
They need the assumptions, rejected alternatives, expected failure modes and reason the change exists. AI can help create that record, but a human owner must verify it.
Use genuinely different checks
Generated tests are useful, but they should not be the only evidence that generated code is correct.
Consequential changes may need specialist review, adversarial testing, different tools or a human walkthrough that begins from the requirement rather than the implementation.
Design for failure
Limited permissions, staged deployments, feature flags, observability and rapid rollback reduce the damage caused by a defect that escapes every earlier layer.
The goal is not to prove that failure is impossible.
It is to prevent one local misunderstanding from becoming everyone’s emergency.
The harder problem is incentives
The technical responses are imaginable.
The incentive response is harder.
Imagine two developers.
One completes ten visible tasks using AI.
The other completes three, removes unnecessary code and spends time testing failure cases.
The first developer’s output is immediately visible.
The second developer’s avoided incidents are not.
There is no Jira ticket for the outage that never happened.
There is no dashboard showing the vulnerability that was never introduced.
There is only a lower feature count.
If organizations reward tickets closed, pull requests merged or features shipped, AI can increase the measured output while the unmeasured assurance gap grows.
That may be the deepest danger—not that AI writes uniquely terrible code, but that producing unverified change becomes economically irresistible.
What I still cannot figure out
For most of software history, implementation was expensive. Organizations had more ideas than engineering capacity.
AI weakens that constraint.
But removing one scarce resource reveals another.
The scarce resource may now be trustworthy review, deep system knowledge, independent verification and human attention.
So perhaps the important engineering question is no longer:
How much software can we produce?
It is:
How much change can this organization responsibly understand?
And what happens when the honest answer is smaller than the number a competitor is willing to ship?
Until the next strange question,
Osagie