An essay I wrote a few years ago. Perhaps we can start the new year with fewer turkeys..?
The Cold, Dry and Bitter Truth about Seasonal Forecasts
“Some suffer from an acute expert problem, producing cosmetic but fake knowledge, particularly in narrative disciplines…” Nassim Taleb, The Black Swan.
Recent headlines scream ‘Environment Canada Blows Winter Forecast’. The next day ‘Environment Canada Forecasts Warm Spring Ahead…’ The following day it’s ‘Groundhog Prophecies Mixed on spring’s Arrival’. The media seem stuck in a loop, rushing headlong every three months to create the seasonal weather story, often overselling it as Weather You Should Fear Today! In the rapid-fire nothing-is-too-inconsequential-to-be-made-consequential twitter-verse, the next weather horror is never far away. Once established, the hair-brained scare is impossible to unseat. In 2010 it was ‘The Worst Winter in 50 Years’. In 2011 it was ‘the Coldest Winter in 20 Years’. The oversell nicely sets up the inevitable end-of-season follow-up story about the ‘blowing’ of it. It’s really much ado about nothing but enough to make a meteorologist moan.
In my 28-year career, I’ve been asked for the seasonal ‘outlook’ more than any other forecast. The question assumes an answer is possible, that such a forecast exists, and that a meteorologist can provide it. Three cold, dry and bitter truths are: a useful answer is impossible, no such forecast exists and, you’re asking the wrong expert anyway.
1) A useful answer is impossible because a highly variable element cannot be described by its average. For example: the mean temperature of a fall day that starts frosty at 0° and peaks at 20° in glorious sunshine is identical to that of a rainy day where the temperature never budges from 10°.
Tmn frosty sunshine day = 0°+ 20° = 10° Tmn rainy day = 10°+ 10° = 10°
This averaging is a ‘first order of smoothing’ that clearly limits our capacity to describe the day’s weather from the temperature alone. Now, consider that the seasonal temperature forecast is the average of 90 days of mean temperatures! Or, working in reverse: ask yourself what might be said about today’s weather in any city when given the average of today’s mean temperatures from 90 cities? Answer: absolutely nothing and likewise for the average of 90 days worth of forecast temperatures for a single city. The ‘average of averages’ is a second order of smoothing that yields a result devoid of useful information.
2) Believe it or not, there is no such thing as a ‘seasonal weather forecast’. Weather (meteorology) is simply not predictable beyond a week at best. Predictions of ninety-day average temperature and total precipitation (climatology) are made by the climatology divisions of various national and international agencies. Short of a grocery-stand almanac – whose forecasts come from ‘a secret formula that was devised in 1792 and that remains locked in a black box in a New Hampshire office’, no credible agency even attempts to produce a weather forecast to day-90. Furthermore, candid climatologists concur; verification demonstrates very little ‘skill’ in 90-day temperature outlooks and practically zero skill in the precipitation outlooks.
The sole exception to death-by-smoothing is the case of an extreme season where the mean temperature or the total precipitation may hint at the weather experienced. This however, only applies looking backward at what has happened, rather than looking forward to what may happen because of a third order of smoothing essential to the forecast strategy: Seasonal outlooks are merely predictions of the broad ranges into which the 90-day mean temperature or the total precipitation is expected to fall: below normal, near normal or above normal. With only three possible outcomes, these predictions cannot identify or ‘capture’ extreme seasons. The ranges are broad for good reason: to quote Nils Bohr, “prediction is very difficult, especially about the future”. A consequence of this scheme is that even a perfect forecast won’t discern anything about the weather. Predictions of 90-day averages and totals may be valuable to research climatologists or to utilities considering seasonal energy consumption, but not for decision-making by anyone else.
3) You’re asking the wrong expert anyway because meteorologists have no training, experience or skill at forecasting beyond five days. The 2011 worst-winter-in-20-years story was hatched by the so-called Canadian-weather expert at a private company in the United States. To find out how this forecast was created, I emailed the expert a few times. After four weeks without a response, I left a voice-mail posing as a reporter. Within an hour I had a call-back.
Reporter: Are you a climatologist? Expert: No
Reporter: Do you have any training in climatology? Expert: No
Reporter: Are there any climatologists working for your company? Expert: No
Reporter: Are these forecasts based upon climate models? Expert: No
Reporter: Do you know the accuracy of your previous forecasts? Expert: No
Reporter: How did you arrive at the ‘third coldest winter’ prediction?’
Expert: ‘Well, I just sorta looked at about the last twenty winters in Vancouver and it wouldn’t take much more than a degree or two colder conditions to put this into the top three coldest’.
The 2011 winter narrative – spread like wild-fire across all major media – was based on worthless speculationby someone with no climatological credibility. Meteorologists don’t hesitate to defer questions about flooding, avalanches and meteors to hydrologists, avalanche forecasters, and astronomers. Why do some pretend to be experts in a field where they’ve never had any training or experience? To quote Nassim Taleb again:
“At the core of the expert problem is that people are suckers for charlatans, particularly when the charlatan is invested with some institutional authority… they serve as experts while offering the scientific reliability of astrologers. Anyone relying on them is a turkey.’’
Don’t be a turkey.