New Practices for AI-Aided Programming

Unorganized thoughts on new programming practices suited to an AI-aided programming environment.

Deciding when to ask AI to do what

According to a meta-study on what situations human-AI collaboration produces better results in (When combinations of humans and AI are useful - A systematic review and meta-analysis)…

attaching AI to work humans do better than AI led to better results, and
attaching humans to work AI does better than humans led to worse results.

The authors speculate that this is because only when humans are generally better than AI are they also better able to judge whether to accept AI’s suggestions. In other words, when humans are worse than AI, they either fail to be suspicious when they should doubt the AI (overreliance), or stubbornly refuse when they should accept the AI’s suggestion (underreliance).

The ability to distinguish when to assign what kind of work to AI, and in what situations and to what extent to trust the results AI suggests, seems important. It also seems worth thinking about what kinds of everyday practices could help develop this ability. See also: Building intuition for using LLMs

In the future (whether 6 months from now or 10 years from now, I don’t know), AI will probably become good at everything, but not right now, so for the time being this requires thought and preparation.

Deciding how big a chunk to hand over

Similar to consciously controlling the stride of the TDD cycle, it seems important to consciously decide how large a unit of work to give to AI.

To do that, I think you need to be comfortable with both very narrow strides (coding in the traditional way while accepting suggestions in the form of ghost text) and very wide strides (vibe coding, agentic coding). That way, you can choose the stride you want at the moment you want it.

You need to develop a feel for how much work a given model can accomplish depending on how you instruct it, but I still need to think about concrete practices during work that would improve that feel. Maybe before long AI will route things well on its own. Even now (September 2024), tools like Cursor already have automatic model selection options, but I’m skeptical about how well they work.

Creating high-quality feedback loops for AI

It was always important to have a fast, accurate, and rich feedback environment through things like unit tests and static type checking. The same is true when coding with AI agents. We need to help AI quickly check and fix its own mistakes. In fact, once you try using agent mode, you can tell that whether unit tests and static type checking are in place is tremendously important.

Existing good practices (isolated tests, fast tests, high coverage, and so on) have become even more important.

Detecting the feedback AI gives

Something I noticed on April 20, 2025. When I’m pressed for time and end up writing shitty code, suddenly the AI agent or ghost text features stop working properly. If AI suddenly gets stupid, maybe I should look back and ask whether the code I’m writing is shitty code.

Responding to hallucinations

Considering that the cost of software defects is proportional to the time until the defect is discovered (DCI), how quickly you detect hallucinations is a very important issue.

It’s also a problem when unnecessary code gets generated and then you keep carrying it around. Code is an asset, but on the other hand it is also debt, so if the functionality is the same, shorter code is better. But AI that lacks context tends to generate unnecessary code, and since that is not itself a bug, there is a tendency to just let it accumulate. However, unnecessary code imposes a cost every time humans and AI read the code later, causes confusion, and can increase the burden of refactoring.

Once again, good code, unit tests, static type checking, and so on are important.

Distinguishing Verification and Validation

Verification means checking whether the intended task was done correctly, while Validation means checking whether the goal itself was appropriate in the first place. For example, after deciding, “To find the average income of Koreans, I should use the arithmetic mean,” checking “Did I calculate the arithmetic mean correctly?” is “Verification.” Meanwhile, checking “Is it actually right to use the arithmetic mean when finding average income?” is “Validation.”

If you do only Verification without Validation, you end up doing the wrong thing correctly. Advice about being careful of AI hallucinations and so on is mostly focused on “Verification.” But if you gave AI the wrong instruction in the first place, there is no point in checking whether it carried out that instruction well.

This too was important regardless of AI, but as the Verification stage gradually becomes automated, it seems the importance of Validation has grown even greater. (See also: Verification and validation)

The changing balance between writing code and reading it

I spend a decreasing share of my time writing code myself and an increasing share reviewing code suggested by AI. It seems like at least half of the code is written by AI. The ability to read code well will probably become much more important than the ability to write code well. I think training in code reading will be necessary.

The problem seems to be that code suggested by AI (at least for now) always carries the possibility of errors, and the kinds of errors it makes are often somewhat different from the errors human programmers make. I think some kind of practice is needed that selectively takes advantage of things like unit tests, powerful type systems, and program proofs.

The changing role of comments and names

Until tools like GitHub Copilot or Cursor appeared, comments had meaning only for people, and for computers they had no meaning at all or only very limited meaning (JavaDoc or Python docstrings, for example). The same was true of all kinds of names (variable names, function names, class names, and so on).

For example, the two pieces of code below are identical from the computer’s point of view ( $α$ -equivalence).

const blah = a => Math.PI * a * a
const area = r => Math.PI * r * r

But after the emergence of LLM-based coding support tools, the IDE started to understand comments and names. As a result, in an AI-aided programming environment, when the programmer

types const blah = a =>, the editor cannot give useful suggestions, but
types const area = r =>, it suggests Math.PI * r * r with high probability.

The same goes for comments. If you write comments well, it suggests code that better fits the context.

Giving things good names and writing necessary comments in the right places was already something we were supposed to do anyway, so if you look only at the result there is not much difference. But there seems to be quite a large difference in the process that produces that result. For example:

When doing TDD, after adding a new failing test, it is important to see the green bar as quickly as possible (usually within 30 seconds) (see also: TDD cycle). So at this stage I often give variables rough names like blah, foo, a, b and then, once the test passes, gradually refactor. But if you name things carelessly, you can’t properly benefit from AI’s coding support.
When doing TDD, you first put in the minimum implementation that can make the test pass (Fake it) and then gradually guide the implementation forward, but AI often suggests a proper implementation in one shot. After that, even if you add the next test, the test does not break (because a proper implementation already exists), so the feedback loop is cut off. As a result, there is a risk that test coverage will become lower.

Meanwhile, there are also changes that do not conflict with existing practices. Some changes make existing good practices even more beneficial. For example:

Before coding, I have started more often to write in comments, in the form of pseudocode or a TODO list, what I plan to do in this coding session. This is similar to starting a pair programming session with a short design session, scribbling with your pair on a whiteboard or a piece of paper, and it is a practice worth encouraging. The fact that I have come to do this more consciously even when coding alone is a good thing.
Thanks to AI understanding pseudo code written in comments, the technique of stepwise refinement has become much more useful, and accordingly I use it more often.
The same goes for literate programming. In environments like Google Colab it still does not work well enough yet, but when using VSCode with Jupyter Notebook, AI support works fairly well.

As for some changes, I still don’t really know whether they are good or bad. For example:

Comments inside a function are generally a smell, but it is good to write careful comments for function or module declarations (especially when they are a “published” interface). However, if you want to refactor frequently enough, it is more efficient to write declaration comments right before committing. But now that AI is good at helping write comments, and also good at helping revise comments to match refactored code, writing comments in advance is no longer a major waste.

Software design

In Refactoring by Martin Fowler, there is this line:

After all, the compiler doesn’t care whether the code is ugly or clean. But when I change the system, there is a human involved, and humans do care. —p4

This means that whether the structure of the code is clean or not does not matter to the computer (compiler), but it matters to humans. But now clean code has become important to computers as well. The more clearly code reveals intent and the better its design is, the more likely it is that AI will work well with it.

Because of this change, it has also become necessary to think more about when it is best to do design.

Changes in the order of writing code

There is a need to think, on a smaller scale and at narrower intervals, about which part of the code to write first in order to communicate my intentions better. If you are using Cursor rather than GitHub Copilot, then even where you leave the cursor after finishing an edit becomes important. (I’m not sure about this yet, but) it also seems to take into account what you’ve copied to the clipboard.

Commit more often

Since the LLM writes the commit messages for me anyway, it seems better to commit more often. So that I can always go back even if an AI agent wrecks the code.

Excessively frequent commits are not a problem, because you can just git squash them appropriately.

Productivity improvement vs. capability improvement

Even though short-term productivity improvement and long-term capability improvement often conflict with each other (Exploration-exploitation dilemma), a considerable amount of discussion of generative AI focuses on the former or loosely muddles the two together.

The same goes for domestic discussions. For example, Musinsa’s Did GitHub Copilot Really Increase Our Productivity? focuses mainly on productivity. SK Planet’s Using GitHub Copilot also focuses mainly on productivity. Still, in “5. Things to keep in mind,” it does briefly mention capability.

If the productivity perspective asks, “Can I do the work faster?” then the capability perspective asks, “Are my capabilities improving?” I’m more interested in the latter. (As an approach that maximizes the former, Vibe coding would be an obvious example.)