I came across this simple theory of overoptimism recently (though it was published years ago). Suppose an agent has at least two actions from which to choose. An action gives either a payoff one or zero. For each, the agent has a subjective probability that the action gives a payoff of one. The probabilities of success are drawn independently from the same distribution G. Agent A then chooses one his actions, the one with the highest mean, according to his subjective beliefs. How do his beliefs about this action compare to those of an arbitrary observer?
Here’s where it gets interesting. The observer’s beliefs are different from agent A’s. They are drawn from the same distribution G but there is no reason that the observer’s beliefs are the same as agent A’s. In fact, the action agent A took will only be the best one from the observer’s perspective by accident. Actually, the observer’s beliefs will be the average of the distribution G which is lower than the belief of agent A since agent A deliberately took the action which he thought was the best. This implies that the agent A who took the action is “overoptimistic” relative to an arbitrary observer.
There are two further points. If there is just one action, this phenomenon does not arise. If agents have the same beliefs (a common prior), it also does not arise. So it relies on diverse beliefs and multiple actions. The paper is called “Rational Overoptimism and Other Biases” and is by Eric Van den Steen.