Source code quality in the AI era

I think that even if, in the near future, AI does all coding in our place, the quality of source code will still matter. There are short-term reasons and long-term reasons.

Short-term reasons

Since, for now, AI is in a situation of imitating humans, it will also handle well the kinds of code humans handle well. Therefore, code quality will still matter. (According to the distinction in Welcome to the Era of Experience, this is still the “era of human data”)
The better the quality of the code in an existing project, the more likely coding agents are to produce better code as well. In some agents’ system prompts, there are explicit instructions to imitate the style of the existing code, and even when such instructions are not explicit, it is highly likely that the quality of the existing code included in the context will have large and small effects on various aspects of the generated code’s quality. For example, according to Improving frontend design through Skills, there was an observation that when design instructions included instructions about typography, parts other than typography (e.g. layout) improved as well.
When code quality declines (long code, circular dependencies, high coupling, etc.), the number of tokens that must be put into an LLM’s context increases. The context size supported by LLMs is steadily increasing, but when the context gets longer, a phenomenon occurs in which the LLM starts doing weird things (context rot). (This is a relatively minor issue, but using more tokens also costs more and takes more time.)
As AI begins producing code quickly, the bottleneck is shifting from coding to review. To relieve this bottleneck, AI needs to take on larger units of work for longer and produce output in a form that is easy for humans to review. Even if it uses more tokens and more time (AI’s time), it is better to steer it toward producing high-quality code (and therefore code that is easy to review). The same applies even if the review is done by AI.

Long-term reasons

In the long term, one might think that since AI will not need to imitate humans in the first place, “high-quality code” by human standards will become meaningless. It may even become unnecessary to create text-based “source code” at all and then convert it into machine code.

But even if things reach that point, I still think high-quality code will matter. There are two reasons.

First, characteristics like recursive decomposition and composition, and isolation between modules, are good properties in themselves regardless of human understandability. This is speculation, but I think the reason the human mind evolved to be comfortable with recursive decomposition and composition in the first place is that such abilities are necessary to interact with the world in meaningful ways. That is, I do not think recursive decomposition and assembly are properties limited to the human mind. That said, there will also be hierarchies between modules, and forms of interfaces between modules, that are not compatible with human understandability. (See: Understandable modularity)
Second, humans will probably still have to perform software audits in the future anyway (with AI’s help), from the standpoint of legal/ethical responsibility, and for that, code that humans can understand will still be necessary. However, the detailed criteria for what “code quality” means in this context may change. For example, “how easy is it to read” may become (much more than now) more important than “how easy is it to modify.” Meanwhile, the kind of “audit” meant here seems much closer to “were the goals set correctly” than to “was the code implemented without bugs” (see: Verification and validation), so it does not seem necessary for humans to read every nook and cranny of the code. Whether the code is free of bugs is something that can be resolved through program verification within a practically useful range. Things like the halting problem or Gödel’s incompleteness theorems are not as realistic a barrier as one might think.

How should we manage quality?

See: Enforcing the quality of AI-generated code

References

Software engineering in the AI era
The friction is your judgment by Armin Ronacher, Cristina Poncela Cubeiro
- “Your codebase is infrastructure for the agent. Design it that way.”