Exploration over exploitation
Deliberately increasing exploration and reducing exploitation in the exploration-exploitation dilemma.
Algorithm
It is usually better to explore more in situations such as these:
- Many attempts remain.
- Uncertainty is high (because we have not explored enough yet, or because the situation is changing quickly).
Otherwise, it is better to exploit more.
If the agent performing these actions is human, however, I think it is worth deliberately increasing exploration in light of the following points.
A tendency toward conservatism
People gain experience as they age. If they studied hard when young, they will know even more. The more you know, the easier it is to fall into thinking “there is nothing new in the world.” Moreover, personality psychology suggests that humans naturally become more inclined to seek stability as they age (this is a good MAB algorithm: exploration should decrease as the “horizon” approaches).
Together, these tendencies make people increasingly “conservative.” In other words, the natural inclination to explore less grows over time.
Path dependence
Although not always, increasing exploitation also means using relatively less System 2. Habit formation improves efficiency but creates path dependence.
We need to keep using our brains to prevent or delay the cognitive decline that begins around our forties. As exploration decreases and exploitation increases, we become increasingly dependent on that direction, as a habit, and it becomes difficult to reverse.
Increasing life expectancy and healthspan
Human life expectancy continues to rise, and healthspan may also increase considerably through better diets and so on (some researchers argue that human lifespan could reach 140 years; for example, Lifespan: Why We Age - and Why We Don’t Have To). In light of this, we need to deliberately explore more.
An increasingly fast-changing world
We should also consider that increasing exploration is an efficient learning algorithm as the world changes faster. It varies by field, but we live in a time of rapid change. We need to deliberately explore more.