Speculative tree search
While thinking about human–AI collaboration patterns for improving productivity, I came up with a method called speculative tree search.
Overview
Using 10x or 100x more LLM tokens is easy, but achieving 10x or 100x productivity as a result is difficult. Up to now (October 2025), I had mainly been thinking about ways to improve capability rather than productivity, but once I actually tried to seriously build something (after November 2025), my attention naturally shifted toward productivity improvement.
Attempts I had tried so far
It is not that I had never thought about improving productivity before, so I had intermittently tried this and that; I took this opportunity to organize those attempts.
Running multiple projects in parallel
I tried running multiple projects in parallel with the same specification (simple Minecraft clone). What motivated me to do this:
- Even when given the same specification, the quality of the outputs produced by the agent often differs quite a bit. If producing an output from a specification is treated as one trial, this means that increasing the number of trials raises the chance of getting a good output.
- In agentic coding, AI writes most of the code anyway, and I mostly just review it. So while the AI is coding, I am waiting, and while I am reviewing, the AI is waiting; in terms of actual physical elapsed time, half of it is idle time.
- Excessive parallel work increases the cost of context switching and can actually lower productivity, but I thought an appropriate level of parallel work could improve productivity by reducing idle resources.
- (There was also the motive that, since I subscribed to the Claude Code Max plan, I should use lots of tokens lol.)
I used two projects this way, switching between them at roughly 5–10 minute intervals, concentrating for about 40 hours over around 2.5 days (during that time, I repeated the process of leaving only the specification, deleting everything else, and starting over several times). It required extreme concentration, and my mental energy was depleted very quickly. This does not even get close to 10x; it is not even 2x. On top of that, there was substantial interference from moving back and forth between two similar projects (e.g. “Did I implement coyote time here? Or was it in the other project?”).
Trying to assign long tasks at once
(As of October 2025), it is still not easy to assign long tasks to coding agents.
Research such as How Does Time Horizon Vary Across Domains? says they can carry out software-related tasks for more than an hour, but the median task duration I feel in everyday use seems to be somewhere between 3 and 5 minutes. Even when I deliberately assign something that ought to take a long time, it usually finishes within 30 minutes. (Tasks that inevitably take a long time, such as building, evaluating, and improving ML models, are exceptions.)
Even when assigning a very long task to an LLM (several hours or more) occasionally succeeds, it typically suffers from two problems.
- The longer the AI works, the larger the output generally becomes. As the output gets larger, the human review time gets longer.
- The longer the AI works, the worse the quality of the output generally becomes. When the quality is bad, the human review time gets longer.
This method also does not seem capable of achieving 10x productivity, and in some cases it even lowers productivity instead.
Scheduling it to do small units of work step by step
I tried this approach based on the observation that, assuming the amount of work is the same, having several cohesive small commits is much easier to review than having one large commit. (The real prompt is longer, but I shortened it here.)
#!/bin/bash
while true
do
claude -p "다음 작업을 하나 수행하고 문서를 갱신한 다음에 커밋해."
claude -p "직전 커밋을 검토하고 관련 테스트를 추가한 다음에 커밋해."
claude -p "직전 두 개 커밋을 검토하고 버그를 수정한 뒤 커밋해. 버그가 없으면 아무것도 하지 마."
claude -p "구현과 명세 사이에 불일치를 찾아서 고치고 커밋해."
claude -p "전체 코드를 검토하고 가장 지저분한 코드를 찾아서 최소한으로 리패토링한 뒤 커밋해."
done
To have it do small tasks independently like this, as with the method explained in Stateless AI-human interaction, you have to explicitly externalize as much context as possible (mostly into markdown files).
I made a simple infinite-loop script like the above and ran it for two days, and it produced 470 commits. In other words, it produced one commit every 6 minutes on average.
The result was fairly decent, but there were various things to improve. A few examples:
- If you just tell it to “refactor” without any direction, it often bounces back and forth, doing Extract function and then Inline function and so on.
- I cannot even bring myself to review 470 commits in just two days.
This is better than the other methods I tried above, but even this method does not seem capable of achieving 10x productivity.
Lessons
While thinking about what lessons could be drawn from the attempts above, I explicitly organized the premises for 10x productivity. Once written down, they look obvious, but they still helped me develop the next step of thinking.
Premises:
- Humans are much more expensive than AI. (That is, using a lot of AI to reduce a small amount of human effort is a good trade.)
- Time is much more expensive than money. (That is, spending a lot of money to shorten time a little is a good trade.)
- Human context switching is expensive.
- There are tasks that require human judgment.
- The cost of human judgment (time and effort) is proportional to the size of the output produced by AI and inversely proportional to its quality.
- Mistakes in human judgment are proportional to human cost (context-switching cost, judgment cost).
- Mistakes in human judgment are expensive.
If we accept that the premises above are true, what should a human–AI collaboration pattern for maximizing productivity look like?
- We should separate and automate as much as possible the parts that do not require human intervention.
- In processes that do require human intervention, longer AI work should not make the output larger. Instead, it should improve the quality of the output.
- Even when a human is handling multiple things at once, context switching should be minimized.
Of these, 1 is the obvious thing everyone was already saying, so I thought more about 2 and 3.
Speculative tree search
There is a performance optimization technique called speculative execution. When it is uncertain whether execution will be needed, instead of waiting until that becomes certain, you go ahead and execute in advance. If the conclusion is that execution was needed? The waiting time disappears because it has already been executed. If the conclusion is that execution was unnecessary? You can just throw away the result.
Speculative execution is an effective strategy in situations where the following condition holds.
cost spent executing in advance < cost spent waiting
Usually, execution is done using idle resources, so unless you count electricity costs and the like, the “cost spent executing in advance” is close to 0.
Let us consider whether the concept of speculative execution can be applied to the context of human–AI collaboration, taking into account the premises derived above.
- No matter how well you write the specification, situations arise that require human judgment.
- If the AI enumerates in advance the candidate judgments a human might make and executes them ahead of time, then 1) human waiting time can be reduced, and 2) physical elapsed time can also be reduced.
- Humans are more expensive than AI, and physical time is more expensive than money, so for AI, “cost spent executing in advance < cost spent waiting” holds.
Meanwhile, if multiple projects are being carried out at the same time, a human incurs context-switching costs when moving around the projects and making judgments in an interleaved manner. Therefore, even when multiple projects are ongoing simultaneously, this cost can be reduced by visiting each project in a way that makes a series of judgments consecutively there before moving on to the next project.
To do that, recursive speculative execution is needed, and this can ultimately be seen as speculative tree search.
Pseudocode:
- Create a good specification.
- Have the AI work based on the specification.
- The AI proceeds in small, cohesive units of work.
- If the work is finished? Stop.
- If human judgment is needed to continue the work? The AI first lists several options that seem plausible on its own, then proceeds with each of them in advance. (Recursively)
The problem with this approach is that the search space grows rapidly as the tree gets deeper, but there are many strategies for making tree search efficient, so this does not seem like a major problem. For example, one could think of a slightly modified version of MCTS. Among the four stages of MCTS (selection, expansion, rollout, backprop), you might omit the rollout stage and instead simply have the LLM estimate the score.
Of course, the most effective way to reduce the search space in the first place is to write the specification as concretely as possible, but since that is a relatively independent issue, let us set it aside here. (Related posts: s4, specifications written in natural language)
Fields where it can / cannot be applied
The method above can only be applied when speculative execution does not create side effects. (If there are side effects? Spectre, hello?)
For example, if speculative execution sends emails as a result, you cannot UNDO that later, so this method cannot be applied. Creating Git branches and making commits on a local machine depending on branches is safe (in most situations), so it can be applied there.
It seems applicable to document writing, code writing, artwork generation, and so on.
Result
TODO: Actually try it and add more content.