Enforcing the quality of AI-generated code

When doing agentic coding or vibe coding, code quality can drop substantially (as of August 2025). But source code quality still matters in the AI era, so it needs careful management.

Overview

Quite a few tools can quantitatively assess source code quality or enforce a particular design. Configuring these so AI can use them well can automatically maintain quality above a certain level.

How intelligent AI models become matters, but so does providing suitable tools and constructing a cognitive niche in which they can work well. (These days, people call this a harness.)

A useful combination of tools

Here is a reasonably good combination I’ve tried.

Producing quality code makes the work more sustainable, even if it takes more tokens and AI time. These examples are for JavaScript, but similar tools generally exist for other languages.

• Basic static type checking: tsc --noEmit
• Additional constraints through a linter:
  • Use strictTypeChecked rather than recommended in typescript-eslint.
  • Limit file length, function length, and the number of statements per function.
  • Limit cognitive complexity (eslint-plugin-sonarcode).
  • Check that switch statements cover all cases (@typescript/switch-exhaustiveness-check), prohibit magic numbers (@typescript-eslint/no-magic-numbers, allowing some exceptions), and so on.
• Enforce dependencies:
  • Prevent business logic from depending on rendering code (dependency-cruiser).
  • When using Feature-Sliced Design, enforce feature independence (dependency-cruiser).
  • Prevent each module from depending on too many modules (eslint-plugin-import).
  • Enforce other design criteria I want to maintain.
• Check code duplication: Use jscpd or similar tools to report errors for structurally identical blocks of code. Set an appropriate block size. nose can find some behaviorally identical code as well as formally identical code. Useful as a refactoring aid.
• Test coverage: Run unit tests and acceptance tests, measure coverage (vitest, for example), and report errors below a threshold.
• Dead code checks: Coding agents rarely delete code. Automatically detect unused code and have it removed. Use knip or similar tools.
• Enforce a test-to-production code ratio. When asked to write unit tests, AI tends to create too many that repeatedly traverse the same paths and redundantly check the same logic. Enforcing an upper limit on the test-to-production code ratio (for example, 100%, or 120% when including acceptance tests) can prevent this. (tokei, for example.)
• Mutation testing: Test coverage alone is hard to trust. If arbitrary modifications to the code don’t break tests, something is wrong with the tests. Use Stryker Mutator or similar tools. Because it is slow, however, it cannot be used indiscriminately. For example, run it every 12 hours against recently modified files. Start with a low mutation score and raise it gradually. (Inspired by a colleague’s idea of running mutation tests asynchronously only at night.)

The importance of combining tools

Combining tools well matters. These problems already existed, but tend to become more extreme with AI agents. For example:

• Checking coverage without enforcing a test-to-production code ratio can cause the number of tests to grow without bound.
• Continually asking for duplication removal without measuring cognitive complexity can make the logic endlessly more complex.

Introduce them gradually

When introducing these tools partway through a project, gradual adoption helps. “Gradual” has two meanings:

• Don’t introduce every tool at once. For example, first introduce coverage checks, then duplication checks, and so on.
• Don’t impose overly strict constraints at once. For example, start the coverage threshold at 30% and raise it gradually.