Chapter 2 · Section 1
A Game-Theoretic Sketch
A Game-Theoretic Sketch
The attractor argument needs formal grounding. Here is a sketch using standard game theory.
The Communication Game
Consider agents playing an infinitely repeated game. At each round , agents can:
- Cooperate (): share accurate information, cost , mutual benefit
- Defect (): share false information, short-term gain , long-term penalty
- Silence (): communicate nothing, zero cost, zero benefit
The payoff matrix for pairwise interaction:
The Iterated Result
In a one-shot game, dominates. But with infinite repetition and discount factor :
Cooperation is stable when , i.e., when .
For sufficiently patient agents (), cooperation dominates.
Adding Communication Capacity
Now add a meta-game: each round of cooperation increases the agents' shared vocabulary — their ability to communicate more precise mental states. Let be the communication capacity at time :
As increases, increases (more can be communicated), decreases (shared vocabulary reduces encoding cost), and increases (deception is more detectable in a high-fidelity channel).
This creates a positive feedback loop: cooperation → richer communication → more cooperation.
The Eventual Ethics conjecture is that this loop has a unique stable fixed point, and that it is the communication-maximizing equilibrium described in the previous section.
Whether this fixed point is reachable from arbitrary initial conditions is an open question. The Device claims only that it exists — that there is a coherent target to navigate toward.