Why Energy Experts Get Things Wrong So Often

After the unpredicted fall of the Soviet Union in the late 1980s due primarily to an oil price shock, Philip Tetlock, a U.S. psychologist, started to question the wisdom of experts.

It seemed rather incredible to Tetlock that political commentators, including Soviet scholars, had uniformly failed to forecast the dissolution of the world's second mightiest empire. Tetlock wondered why and how they could be so blind and dumb?

So the psychologist assembled 234 experts, including government forecasters, university professors, "talking head" journalists, and economists.

All together the group made approximately 28,000 expert predictions about everything from economic growth to nuclear proliferation. Tetlock then compared the accuracy of the experts to dilettantes, dart-throwing chimps, and computer algorithms.

Sadly, the experts did only slightly better than the "proverbial dart-throwing chimpanzee," and a lot worse than mathematical equations predicting business as usual.

The findings astounded academia, but were ignored by the mass media. In fact, the modern news machine devotes a majority of its space to so-called economic and political experts who remain about as authoritative as a gaggle of unaccountable bonobos in an Irish pub.

However, Tetlock found two distinguishable groups of thinkers among the experts: hedgehogs and foxes.

The hedgehogs, which tend to dominate the airways, behaved in an authoritative manner and overconfidently relied on one source of information. In other words, they knew one thing well.

In contrast, the foxes, a humble lot rarely sought out by reporters, cautiously drew from a diversity of sources. Not surprisingly, the foxes produced more accurate judgments than the hedgehogs.

These findings taught Tetlock, the author of Expert Political Judgment, a few lessons about pundits. They "were hard pressed to do better than chance, were overconfident, and were reluctant to change their minds in response to new evidence. That combination doesn't exactly make for a flattering portrait of the punditocracy," he recently told Edge.org.

Tetlock also concluded that experts weren't really in the business of making accurate predictions, but served entrenched multi-billion dollar forecasting industries.

These paid prognosticators are "in the business of flattering the prejudices of their base audience and they're in the business of entertaining their base audience." Accuracy is about the last thing many an expert worries about these days.

Now, Tetlock's insights apply to energy analysts in barrels. Oil price predictors and forecasters get things so wrong that energy and business reporters have unwittingly become pipelines for misinformation as black as bitumen.

University of Manitoba scholar Vaclav Smil, a genuine energy fox, devoted an entire chapter in one of his many energy books (Energy At the Crossroads) to the phenomenon and called it "Against Forecasting."

(An historical aside: prior to the advent of cheap fossil fuels, energy forecasting really didn't exist: there were slave and animal audits as well as ship inventories, but no clique of energy forecasters.)

Too complex to predict

Given the grim historical record on modern energy forecasting, Smil refuses to engage in the game. In fact, Smil offers only one energy prediction: "We will spend more time and money on playing the future game" and most predictions will continue to be dead wrong.

Energy systems are so large and complex that unpredictable political events, economic disasters, disruptive technologies and public policy changes can quickly alter the course of events. More