Home > AGW, Catastrophism, Climate Change, Data, Global Warming, Omniclimate, Policy, Science, Skepticism > Financial Crisis to Climate Negotiators: DO NOT OVER-RELY ON MODELS!!!

Financial Crisis to Climate Negotiators: DO NOT OVER-RELY ON MODELS!!!

The ongoing financial crisis can and will teach all of us many lessons, also in terms of climate and AGW. And no, I do not mean the rather naive made-up litany visible at ClimateProgress.

I refer to something much more profound, especially since there is now evidence that even the guys at RealClimate do not fully understand what they are dealing with, when they deal with climate models (even after loving models to death).

Consensus is in fact emerging about three advices that went missing during the build-up of the financial troubles:

  • DO NOT OVER-RELY ON MODELS
  • DO NOT JUST PUSH TOWARD EVER MORE COMPLEX MODELS
  • BE SENSIBLE WHEN MANAGING RISK

====

I have prepared a couple of quick lists from three recent articles on the business pages of the International Herald Tribune (full attributed quotes at the bottom of the blog; and yes, do keep in mind those are from analysts that are experts in their field indeed).

First, what went wrong. It’s evident that plenty of it directly related to the climate debate:

  • Models gave a false sense of precision. They can now be seen as educated guess calculated to many decimal places. At the time, they appeared precise, and yet proceeded to ultimately demonstrate themselves as totally off base
  • Until the crisis, the field (of financial risk modeling) enjoyed a halo of academic credibility
  • In general, there was too much focus on quantitative issues and data and models. People did not know what to do with things that cannot generally modeled as a quantifiable risk
  • Risk managers were also too busy with models and bringing up data that could not be absorbed by senior management
  • Better modeling, more wisely applied, would have helped, but so would have common sense in senior management

Obviously, the danger lies in the fact that to confuse the model with the world is to embrace a future disaster, as humans (or the climate) do not just obey mathematical rules that can be modeled.

What should be done? I wish the negotiators in Poznan had the following list in mind:

  • Understand that risk is a function of behavior more than of models
  • Consider that risk management is about making big-picture choices, not just trying to prevent losses
  • Acknowledge that risk may mean different things, like hazard, threat, gamble, chance, possibility, or opportunity
  • Accept that models are useful as points of information. They shouldn’t drive risk tolerance and shouldn’t be used to tell anybody how to manage firms (or nations)

Models getting translated to the real world of company or national policy suffer indeed from a “chinese whispers syndrome”, with the original caveat-full expert statements awfully simplified and distorted for the benefit of the business directors (or national politicians).

At the end of the day, the problem is not the models. The models are tools, perhaps the devil’s but still just tools. The problem is putting all eggs in the models basket, in financial just as in AGW terms.

====

(original quotes)

(1) From In fallout from crisis, rethinking risk and human judgment, by Lynnley Browning; IHT, Wednesday, November 19, 2008

[…] to cope with uncertainty and “slippery slopes” […] “With this crisis, everybody is re-evaluating the concept of risk management,” said Richard Phillips, a professor of risk management and insurance at Georgia State University […]

The scrutiny goes beyond a dissection of the complex mathematical models created by financial engineering [and focuses] “on the overreliance on models,” said Carol Fox of the Risk and Insurance Management Society […]

Because nearly all risk-management models failed to predict or protect against the crisis, Fox said, insurers will increasingly view risk “more as a function of behavior than of models.”

Going forward, she said, insurers will use models “as a point of information, but it won’t drive risk tolerance” […].

“People have been managing the wrong risk […] ” said Peter Bernstein, a historian and the author of “Against the Odds: The Remarkable Story of Risk.” ”Risk management is about making choices, not preventing losses. […]

the financial crisis has made clear is that risk, and how one deals with it, can mean wildly different things to different companies, from gamble, hazard or chance to threat, possibility or opportunity. It can be a bucket of nasty things to be avoided, or a daring play. […]

It didn’t help matters that until the crisis, the field enjoyed a halo of academic credibility. “All these rocket scientists with Ph.D.s provided reassurance to decision makers and buyers,” said Paul Bracken, a professor of political science at Yale University.

[According to] Robert Merton, the Harvard Business School professor who received the Nobel in economic science in 1997 […] “A lot of it is straightforward things, like judgments made to accept ratings. We’ve got to get these financial engineers and quant types out of the banks and get sensible types in.” […]

“Our definition of risk became confused with obeying the law,” said Bill Sharon, chief executive of Sorms, a risk-management consulting firm. […]

Now, insurers are increasingly looking at risk management as a process applying […] to big-picture questions […].

After all, said Martin Grace, associate director of the Center for Risk Management and Insurance Research at Georgia State University, “you can have math models, but that doesn’t tell you how to manage the firm.”

(2) From When crisis hit, a global framework for limiting risk proved ineffective by Conrad de Aenlle, IHT, Wednesday, November 19, 2008

[…] Even if [The Basel II international accord on banking supervision] had been put into practice immediately, it might not have averted the crisis. Critics contend that the various models, formulas and equations used to determine asset quality provide a false sense of precision, leaving bankers and regulators with no clear idea of where they stand. The numbers that are derived amount to an educated guess calculated to umpteen decimal places.

“There has been too much focus on quantitative issues and data and models and a lack of understanding of what the main risks are in the business model,” said Peter Neu, a principal in Frankfurt for the Boston Consulting Group. “Risk managers are too busy with models and bringing up data that can’t be absorbed by senior management.”

A shortcoming of some models is that their risk projections come with a caveat that they are assumed to be accurate during normal market conditions. […]

(3) From Wall Street’s extreme sport: Financial engineering by Steve Lohr, IHT, November 5, 2008

“Complexity, transparency, liquidity and leverage have all played a huge role in this crisis,” said Leslie Rahl, president of Capital Market Risk Advisors, a risk-management consulting firm. “And these are things that are not generally modeled as a quantifiable risk.”

The miss by Wall Street analysts shows how models can be precise out to several decimal places, and yet be totally off base

The quantitative models typically have their origins in academia and often the physical sciences. In academia, the focus is on problems that can be solved, proved and published — not messy, intractable challenges. In science, the models derive from particle flows in a liquid or a gas, which conform to the neat, crisp laws of physics.

“To confuse the model with the world is to embrace a future disaster driven by the belief that humans obey mathematical rules.”

Better modeling, more wisely applied, would have helped, Lindsey said, but so would have common sense in senior management

  1. Alan McIntire
    2008/12/06 at 02:20

    I enjoyed your post. Speaking of underestimating risk, my father gave me a book recently, “Fortune’s Formula”, by William Poundstone. The book covers risk and risk management- including Edward Thorp, who used the “Kelley” money management system to win at blackjack, and later to make money in a Warrant hedge fund.

    The book also covered not so successful funds, including “Long Term Capital Management” , established by Robert Merton and Myron Scholes, famed for the
    “Black-Scholes” equation for determining stock varibility. “Long Term Capital Management” went bust in a very short term, partly because you cannot rely on short term measurements to determine longer term variance, partly because the variance in the companies LTCM invested in were not independent runs, but had a common risk factor. In that regard, it was sort of like the housing market. Models were built showing a small chance of default. . It was assumed that the risks would be reduced by investing in packages of large numbers of subpar loans. Unfortunately for the investors, those loans weren’t varying independently, they all went bust together.- A. McIntire

  2. 2008/12/05 at 23:24

    Excellent post! And on this subject, I found the following editorial today in the latest edition of New Scientist (06/12/08) and scribbled it down. I don’t buy NS but usually glance at it while in the supermarket: I find it makes me annoyed and thus gives me a useful boost of energy when walking around Tesco. Apologies for including the whole thing in this comment, but I thought it quite extraordinary (any inaccuracies and misspellings are probably mine.)

    “It’s the carbon, stupid. Despite uncertainties about climate change, we know enough to act.

    “It’s time for heretical thinking on climate change. After two decades in which science has told us more and more about global warming, climate modellers may have to recognise that we have learned most of what we can from their number-crunching.

    “Some of the detailed forecasts about exactly what the climate will be like in Albuquerque or Basingstoke in 2050 or 2080 are little more than statistical noise, as physicist Lenny Smith underlines this week (see page 42.) Even the global picture may depend more than we would like on feedbacks and tipping points produced by a system that is inherently chaotic. We need to beware of the known unknowns and – whisper it – the unknown unknowns.

    “Some politicians still demand certainty from climate scientists and are sitting on their hands until they see it. But certainty may not be more available here than in that other troublesome discipline, economics. This is not a counsel for inaction, but for grown-up government: for doing what we know is needed in the face of uncertainty, and for taking actions like those called for this week by the British government’s Committee on Climate Change, from decarbonising electricity generation to culling carbon-spewing vehicles and aircraft.

    “Here’s another heresy. (Realising?) the endless negotiations to frame a successor to the Kyoto Protocol – currently in mid-grind in Poznan, Poland – are becoming an impediment to action. The protocol’s various market devices, like cap-and-trade and the clean development mechanism, could now be holding up the technologies we know will do the job. Invented by the Clinton/Gore administration, should they now be jettisoned by Barack Obama? Michael Le Page believes so (see page 20) and argues that taxing carbon would be a better plan. It would be a bold move. But just as past economic certainties are failing, maybe it is time to think the unthinkable here too.”

    So, let’s see if I’ve understood the gist of the New Scientist editorial:

    1) The climate is inherently chaotic, full of uncertainties, and unknowns. Go to 2.
    2) Climate models are thus prone to producing forecasts that are little more than statistical noise. Go to 3.
    3) This means we have reached the limitations of climate models and there’s not much point in trying to learn more from them. Go to 4.
    4) However, we know exactly what we must do. Go to 5.
    5) What must we do? Take immediate, drastic and hideously expensive measures to reduce CO2 in the atmosphere. Go to 6.
    6) Why must we do this? Because otherwise, we face an unstoppable and unprecedented global catastrophe as the world overheats. Go to 7.
    7) How do we know this will happen? Because climate models are telling us in their forecasts. Go to 8.
    8 ) Are these models and forecasts reliable? No, they are not, any more than economic models and forecasts have been. Go to 9.
    9) Why are the climate models and forecasts not reliable? Because… Go to 1.

    If this circular argument was a perpetual motion machine, just think of all the Green energy it would generate. :o)

  1. 2009/09/10 at 21:12
  2. 2009/07/01 at 11:14
  3. 2008/12/06 at 07:52

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: