Prediction Machines
"Human reason can neither predict nor deliberately shape its own future. Its advances consist in finding out where it has been wrong." - F.A. Hayek
When I was in Panama, Bocas del Toro, a tropical Caribbean set of islands, the motorbike rentals were more expensive than I anticipated. So were the ATVs, then even the manual regular bikes. It was more economical to hire 2-3 round-trip taxis a day. At first, I attributed the prices to the "tourist tax", but then I learned the local rental prices were not considerably better. Then I suspected a price cartel between the two or three main players on the islands, but even "black markets" like Facebook and second-hand were also not considerably better than the market-setting prices. I learned one of the rental companies went from 8 to over 100 bikes in about 8 years. I wondered if there was a play for a motorbike rental business.
Spending more time in Bocas, I realized the prices seemed extortionate because of the deterioration of the vehicles. Bocas has a notorious climate known to completely degrade equipment - electronics, building materials, paper - everything on the island had a heavy deterioration tax. Even long-term and frequent visitors, who eventually earned a bit of a discount, were expected to bring their bikes and ATVs in for weekly checkups. Maintenance was a top priority and a top expense. On top of the mechanic's fees, Bocas is an island. Parts and equipment were slow to arrive and expensive. Even hardware stores seemed to have low inventory and plenty of demand. Beyond the balance sheet, could I expect fair treatment from local servicemen? Could I trust employees? Would my bikes be targeted for vandalism for undercutting local businesses?
I don't think my bike rental business would be successful. The current prices signaled an underlying issue that all Panamanians and investments on the island deal with. When I first arrived, the transport seemed expensive and the property seemed cheap, a local would (begrudgingly) admit the transport prices were reasonable and the property pricing was opportunistic.
The story of why I did a lot of walking through Bocas is a story of price signals and simple predictions. The Austrian economist Friedrich August von Hayek wrote the foundational work on price signals in his 1945 work "The Use of Knowledge in Society". Hayek demonstrated that prices are the best aggregators and transmitters of localized information such as supply, demand, substitutes, and complements of goods and services. Buyers are incentivized to seek the lowest price possible, sellers are incentivized to sell for the highest price possible. Where buyers and sellers meet in the middle is the price equilibrium. Where there's a fair deal and a handshake (a fair deal: when both sides are equally unhappy with it).
Hayek and other free-market economists argue against centralized planning and coordination of prices. Hayek believed that knowledge (what is a fair price?) is irreducibly local and the "dispersed bits of incomplete and frequently contradictory knowledge" held by individuals could never be properly aggregated by a central authority. Most champions of free-market economics would not say that central planners are incompetent, but that the nature of the data like 'how much will this neighborhood pay for bread' or 'how much can this machine shop produce by Tuesday' is knowledge that "by its nature, cannot enter into statistics". The only incompetent central planner is one who thinks they can accurately represent localized knowledge.
Hayek's price signal mechanism has a second property that has been realized today by prediction markets such as Polymarket and Kalshi. Price signals can serve as future event prediction machines. Prediction markets allow participants to trade futures on specific events such as number of federal rate cuts, March Madness winners, who will have the number 1 song on Spotify. If you believe a presidential candidate has a 70% chance of winning and the futures are priced at $0.55, then the future price - the aggregate of all market participants - is undervalued by your prediction and you should buy.
Prediction markets are the wisdom of the crowds in a formal market structure, quantitative odds of a specific event happening with a price signal. Ask 10 people how many jellybeans are in a jar, you'll have 10 wildly different estimates. Ask 1,000 people and the average guess is usually pretty accurate. Ask 1,000 people to bet on it, and those estimates tighten. Economists Michael Maloney and J. Harold Mulherin documented the stock market's response to the Challenger Spacecraft explosion as one of the fastest prediction machines. Minutes after the explosion, the stocks of major contractors involved in the build all dropped about 3-6%, the O-ring manufacturer's dropped 12%. The stock market successfully identified the culprit before the official Challenger investigation (which included Richard Feynman) took months to confirm.
Companies like Hewlett-Packard ran an internal prediction market amongst their employees to forecast how many printers they should expect to sell - the prediction market results consistently outperformed top-down management predictions. Imagine asking a software team to bet on when a feature would actually ship: management sets the over/under at three months, six of the eight team engineers bet the over. That's a signal that there is localized knowledge the timeline should account for.
Prediction markets are not perfect. Their participants are limited to people who 1) know that online prediction markets exist, 2) are willing to open an account on one, 3) understand how futures contracts work (to an extent), and 4) are willing to deposit money to make a bet. There is also a signal extraction problem: does the price of oil rising mean there is a war in the Middle East or an incoming subsidy for electric vehicles? The reason for price movement still relies on some amount of localized context. Prediction markets don't solve the distributed knowledge problem. Participants are self-selected, signals need interpretation, but within those boundaries is the most honest price on a future event - quantifiable and able to be evaluated further by those who own the markets.
The first ambitious attempt at a centralized prediction machine was by The Simulmatics Corporation, who rose to prominence as political consultants when the JFK campaign hired them as the nation's first "social scientists". Simulmatics were kept quietly employed through the LBJ years. They promised a machine that could effectively tell you what to say, to whom, and to what effect. How would swing state voters respond to a Catholic candidate? How would bus desegregation move the southern vote? For the JFK election - tightly defined criteria, target markets - Simulmatics delivered. They built a prediction machine that segmented American voters into 480 demographics and ran punch-card simulations to predict how each segment would respond to a candidate's positions on certain issues. The localized knowledge Hayek argued could never be fully aggregated seemed like it was about to crack open. Simulmatics had a machine that could process dispersed local data at scale, and use it to poke and prod at the hearts and minds of the American voters. Then, Simulmatics were sent to Vietnam.
In Vietnam, the weaknesses of rudimentary data collection were revealed. Simulmatics sent researchers in American suits to Vietnamese villages to learn about their attitudes on the war and the Viet Cong. Nobody in a hamlet shares honest feelings with men who move like central intelligence officers. Simulmatics modeled Vietnamese sentiment as more favorable than it was. Some of their native Vietnamese researchers caught wind that the Viet Cong would not honor the planned Tet ceasefire. The Simulmatics higher-ups did not process the signal and missed the Tet Offensive entirely - a devastating surprise attack by the Viet Cong that erased the little remaining American public support for the war.
Simulmatics eventually collapsed, but their members' work on digital networks and data collection heavily influenced ARPANET and early data privacy policy. Simulmatics set out to win elections but their lasting impact was the social science frameworks and collection methods that were iterated upon, eventually contributing to Cambridge Analytica's "Big Data" and Facebook's targeted advertising of the world today. Hayek as an economist was obligated to make economic forecasts. His most lasting work was not any singular prediction, but his price signal mechanism that explains how prices are set and the difficulty of aggregating localized knowledge.
The value of a prediction is not whether it is right, it's what building it forces you to understand.
For more on Simulmatics, I recommend If Then by Jill Lepore. I wrote a companion note on Simulmatics here.
For more on prediction markets, I recommend this podcast and work by Robin Hanson and Bryan Caplan.