Of course this is a topic I have long discussed, calling for some philanthropist to fund what may be the most effective single endeavor of all time -- better methodologies for finding out who is right a lot, and for denying credibility to those who are consistently wrong -- in the form of a Predictions Registry.Has there been any progress, at all? The concept that has momentum, at present (a small amount), is "Predictions Markets".
Long Bets, An Arena for Accountable Predictions, is "a public arena for enjoyably competitive predictions, of interest to society, with philanthropic money at stake," overseen by The Long Now Foundation. Another example is Intrade, which allows you to make predictions on real world events by buying and selling shares in each event.
I have long argued with Robin Hanson, one of the founders of the prediction markets concept, over whether this is a more useful approach than my own Prediction Registries. (Recall how DARPA got in trouble for sposoring a study of "betting" about future terrorism? One case where the press and the left were loony, because this concept was and remains valid and potentially useful.)
How does the "markets" concept work? Wagering is posited to force people to be both explicit and serious, allocating their predictions carefully in a "market" style competitive arena. I grasp this advantage... and disagree. This kind of thing attracts people with little to lose and with brash axes that they want to grind. So far, the advantages of market style accountability seem not to have appeared.
In any event, I believe these prediction markets have a crucial fault... people want to make predictions and forecasts, but they do NOT want to lose wagers. Hence, while many loudmouths will make public pronouncements, they will not explicitly stake anything on the outcome. Others who have strong reason to believe that a course of action will go badly are intimidated from expressing these views openly.
Hence, these arenas will not attract
(1) politicians,
(2) civil servants who believe their political bosses are mistaken,
(3) lesser-knowns who have no "name-recognition" or resources, but who happen to be right an anomalously large fraction of the time.
My alternative model of registries (or rather, a vast and diverse community of registries) would have a crucial advantage. Participation need not be a voluntary decision on the part of some willing and eager wagerer. Rather, a public figure can be "outed" into having a prediction registered, whether they wanted it made explicit or not. (There should be an appeals process by which someone can say "that's out-of-context" and I did not predict that, specifically. Fine, let them replace the outed forecast with what they DO believe will happen. There can also be room for scenarios weighted by probabilities.)
Moreover, through pseudonymity, even civil service personel (say experts in the MIddle East) could forecast what they believe will happen. While retaining confidentiality and trust within their agency, they would nevertheless be able to establish a clear record. Later they can point to a string of successful predictions and gain deserved credibility.
Again, it seems an experiment worth trying. It would be incredibly cheap. (Like proving the poisonous effects of indignation addiction.)
It could change the world.
And why am I bothering?
