Skip to content

Agentic engineering experiment 4

A continuation from Agent-based coding experiment 1, Experiment 2, and Experiment 3. Organized on March 13, 2026.

I extended s4 and made something called specdown. The idea is to write specifications in Markdown while inserting executable acceptance tests and Alloy models here and there along the way.

What was unsatisfying about the previous experiments

  • In semi-structured software specifications, I had used YAML because I wanted specifications to be both easy for humans to read and write and also easy for logic to analyze, but YAML was not as comfortable for humans to read as Markdown.
  • In trying to put all executable specifications into a single file to reduce the mismatch problem between specifications and code, there was a problem where the specification would become endlessly long.
  • Introducing the concept of software traceability was good, but I didn’t like that it was hardcoded into the flow of business goals → features → acceptance tests. I would like to be able to freely express other kinds of graphs as well (e.g. epic → story → acceptance test, etc.).
  • It was disappointing that I did not properly address the problem of specifications written in natural language.
  • The idea of giving an LLM a CLI and having the CLI guide the LLM (recursive neuro-symbolic inversion of control) was also decent. It seems that now the perception that LLMs and CLIs are a perfect match has become widespread.

Improvements

I made something called specdown. Like s4, which I made last year, it is a way of expressing executable specifications as semi-structured documents.

  • Instead of making the whole thing YAML, I changed it to a Markdown format with YAML frontmatter. Even this much is structured enough.
  • Taking inspiration from Ward Cunningham’s FIT from about 20 years ago (and, further back, from Donald Knuth’s literate programming), I tried implementing a way for documents and executable code to blend well together.
  • I made it possible to use Alloy Analyzer. Now natural language, modeling languages, and acceptance tests are woven together a bit better within a single document. I made some progress on the problem of specifications written in natural language a little more.
  • I implemented software traceability naturally through Markdown links, and made it possible to define the properties of graphs simply in a configuration file.

Eating my own dog food

By
ak
Created
Updated

Machine-translated from the Korean original.

2026 © ak | markdown | 한국어