A process, tool, and specification format for stateless AI–human interaction. The result of Agentic coding experiment 3.

Explanation

s4 is a conventional CLI program that statically analyzes a YAML specification in a particular format (s4.yaml) and the source code, then reports what to do next.

Running s4 status checks:

Whether the specification is internally consistent (whether every business goal has at least one concrete feature, whether feature dependencies contain no cycles, whether every feature has at least one acceptance criterion, and so on).
Whether the specification and code agree (whether acceptance tests corresponding to the specification’s acceptance criteria exist in the code).
Whether all checks registered in the specification (acceptance tests, unit tests, coverage, type checks, and so on) pass.

If it finds problems, it identifies which to solve first and what needs to be done to solve it. s4’s output supplies enough context for solving the problem to enable stateless AI–human interaction.

Further explanation

This can be viewed as a form of recursive neuro-symbolic inversion of control: the AI agent controls whether to call a tool; the tool performs work the agent is not good at, then gives the agent specific instructions for work the tool is not good at, triggering the agent’s next activity.

Improvements

Formalization using Alloy or similar tools

Perhaps semi-automatically? For example, write a specification in natural language and have AI produce an Alloy model. For ambiguous parts, show several examples and ask the user which fits their intent, refining the specification through this process.
Specification written in natural language
Model-driven testing

Prompt templates similar to MCP. For example:

Check whether ATs in the specification and ATs implemented in code agree semantically.
Check whether two different features in the specification are semantically identical.
Check whether two different acceptance tests in the specification are semantically identical.

Forward engineering vs. reverse engineering

Use reverse mode when applying this method to a legacy project; forward mode for a new project.
In forward mode, the specification is the Single source of truth. In reverse mode, start with code as the SSoT and gradually transfer authority(?) to the specification.

Connect with Git

Apply expensive operations (taking a long time, consuming many tokens, and so on) only to changed code.

Dependency graph

Selecting a feature should display its dependencies, features depending on it, connected acceptance tests, and so on in a form easy for an LLM to read.
Allow prohibited dependencies to be specified.

Something to help refactor specifications. What is refactoring?

Split one FEAT into two.
Merge two FEATs into one.
…