Tim van Gelder describes four grades of collective wisdom with examples:
- Simple voting
- Deliberative voting
- Averages, e.g. Prediction Markets
- Deliberative averages
Prediction markets and opinion pools only reach level three because the participants still largely disagree with each other. But experiments in peer instruction and facilitated dialogue show that deliberation can often improve individual estimates, and hence improve the average.
For this reason DAGGRE is investigating both prediction-market-like and Delphi-like approaches. Initially, participants will see only the edit-based interface — similar to prediction markets. However, we will soon enable the “ACERA” panel which allows direct estimation and discussion. Furthermore, some participants will have the opportunity to form ACERA groups to engage in facilitated dialogue.
We are interested to see whether our data support van Gelder’s ranking.
The value of deliberation depends on group composition. In a diverse group, it can greatly improve estimates, but in a homogeneous group, it can just create overconfidence. This result has been discussed by Mercier & Sperber (and blogged here, here, here, and recently team member Robin Hanson here; see also my earlier callout in this blog). It’s also been investigated by team member Anamaria Berea using agent-based models. Anamaria would like to quantify the heterogeneity / asymmetry tradeoff for various kinds of aggregation in networks that exchange information.