’Tis the season of crystal balls as outlooks and predictions once again proliferate. Yet, at the end of a year like this, you’d be forgiven if you questioned the value of these exercises or dismiss them entirely. Not only did the same efforts last year tells us little about what 2020 had in store, but even when in the midst of the Covid storm few got the recovery dynamics right.

Yet, while we ourselves are reluctant about many types of macro forecasts, we recognize two simple realities: First, forecasts will continue to be demanded and supplied — so how does one make the best use of them? Second, the value of forecasting, a bit like planning, can be in the journey more than the destination. So, what is there to know about the journey of macro forecasts? With that in mind, we’ve put together seven learnings we’ve accumulated, as authors and as users, about how to navigate the forecasting season.

1. Be realistic — and lenient. The failure of macro forecasts is so ubiquitous that it’s easy to be cynical. To set realistic expectations — and to elicit lenience — let’s start by acknowledging the difficulty and limits of the task. Simply put, macro forecasts’ weak performance is not (primarily…) because of economists’ hubris and incompetence, but because they operate in an open system with changing dynamics and, relatively speaking, little data. To illustrate, consider a closed system like Chess — the range of possible games is huge, but the rule set is fixed and stable and the data around game play enormous. But macro forecasts are not dealing with a closed system or a stable set of dynamics, and have very limited data — maybe 70 years of decent data from widely different economic environments, policy backdrops, etc. So even when it seems like economic forecasts are modeled within an appropriate empirical range, that range is filled with a small amount of data and its usually easy to question its applicability. No reason to throw it all away, but every reason to be realistic about what’s doable.

2. Know the tools and when to use them. “Forecast” is a broad term for a toolkit that consist of a range of instruments. On one end of the spectrum is the point forecast, a single numerical quantification of the future which by definition will almost certainly be “wrong” and often riles those not used to its context (see point 3 below). On the opposite end are longlists of risks, or outright opinionating (often about doomsday outcomes). In between are scenario exercises describing multiple futures and their contingencies.

Each of these (and their permutations) represent a different trade-off between clarity and completeness. And, depending on the user, their purpose, and their context, each can have different values and different pitfalls. Consider market-implied point forecasts are key for portfolio managers placing their bets but are not useful for stimulating thoughtful debate about the future. Meanwhile, even the best-intentioned Cassandras are less helpful for portfolio construction but can be useful for scenario building or for “red teaming” internal processes through provocation. Beyond knowing the instrument and its best use, it also pays to consider who is speaking and with what agenda. Often forecasts emanating from the finance world are accused of being too bullish, allegedly because analysis and business objective are hard to separate. Others may have thematic biases. All this makes good knowledge of the toolkit, the instruments, their authors, and when to use them indispensable.

3. If using point forecasts, know how to. Precise numerical predictions of the future — the GDP growth rate, the 10Y yield, the unemployment rate, etc… — are a particular nuisance to many users as they misleadingly convey the precision and accuracy of a natural science (and can do more to promote the “physics envy” reputation of the economics discipline than anything else). And while point forecasts are almost guaranteed to be wrong, they are also broadly misunderstood. Few of those constructing them think of point forecasts as immutable truths about the future. Rather, they represent a snapshot of the base case of the future, using the forecaster’s best available knowledge and tools, in a given moment in time. When any of those evolve, the forecast has to change and conveying that change is part of the service. In their eyes, an ever-changing forecast is less a matter of defeat, and more like a subscription service, delivered with the convenience of a precise single number. They are designed for clarity and precision at any given moment in time at the expense of hiding the distribution of outcomes and risks. Their users are empowered if they know their design and purpose — even more if they understand the mechanics and assumptions that they are built upon. In many ways, knowing what is assumed and inferred is more powerful than knowing the point estimate.

4. Know the limits — Most forecasts are based on rules and relationships, such as when the unemployment rate is very low, expect wage and price pressures. Often simple, sometimes complex, these rules are rooted in conceptual and empirical insight. But when we are in new territory, thus without an established empirical basis, forecasts cease to provide meaningful guidance. 2020 clearly is such a case of missing empirical foundation — widespread lockdowns had never been tried to counteract a public health crisis. The data is off the chart, the empirical basis and dynamics at these extremes are simply unknown — the normal approaches and models should be viewed with extraordinary skepticism as modeling outside of the empirical range is prone to extraordinary error .

Knowing these limits matters. As was the case in Q2, the extreme intensity of the data fell outside of the experience of models, and that drove too much pessimism when forecasts extrapolated from the known empirical range (e.g. linear extrapolations of recovery times from unemployment levels, which had never spiked so high, were grossly misleading). Sometimes it’s better to explicitly abandon the dataflow and forecasts and focus on drivers, dynamics and relationships of variables. In 2020, a structural reading of the crisis stood a better chance of anticipating the vigorous recovery that followed the extreme intensity of the initial shock.

5. Don’t be pushed around — by forecasts, new data, or commentary. In a Bayesian sense, having a strong foundational view (i.e. priors) that is grounded in structural analysis of how the economy works is critical. Practically this is the key to avoiding being whipsawed by incremental data, or someone’s interpretation of it, and provides a basis for reasonably incorporating new information. Without it, the risk is being dragged around by the shifting winds of commentary is high. Yet this is not an argument for stubbornness. There may appear to be only a fine line between dogmatic obstinacy and the power of conviction, but the gulf is very wide. A strong foundational view is a powerful reference point, but one must also be open to asking what it would take to change it.

The value of strong foundations was also on display in 2020. When the pandemic struck, all manner of forecasts and predictions were on offer, strongly biased towards the dark and systemic kind (recall there were broad predictions of a new Great Depression, debt crises, or the US dollar cracking). But a strong framework, even in the midst of the Covid onslaught, discouraged rushing to conclusions and encouraged asking how high the bar is for such outcomes.

6. Beware the flashing indicator — risk indicators are tempting in their precision and their simplicity and dashboards (collections of such indictors that are, essentially, automated rolling forecasts) that promise to flash when warranted, are popular. However, there are no silver bullets, nothing works all the time, and understanding the weakness of indicators is a key to their effective usage. Here we are reminded of financial markets’ favorite recession indicator — the yield curve (or more precisely the inversion of the yield curve when short rates stand above long rates and presumably foretells recessions). Its historical record may end up looking enhanced by the Covid crisis, having inverted some time before the crisis, yet it would be a stretch to argue it held any foresight that a pandemic was going to end the cycle. Yes, it might be argued that the yield curve’s inversion successfully reflected slowing growth and thus vulnerability to a shock such as a pandemic — but in truth the pandemic was a big enough shock to end any expansion in history, at any point in the cycle.

7. Value the process over the outcome — Given the dismal performance of forecasts we’re not surprised at the widespread indifference or contempt. Yet, a great deal of the value is in the process of developing a forecast, rather than in the final product, particularly in the business context. Users of forecasts can gain insight into the process by understanding how the forecasts are made and the assumptions that they lean on. The quip often attributed to Eisenhower of Planning is essential, plans are useless is even better in its original form, peace-time plans are of no particular value, but peace-time planning is indispensable. It is the process of planning (particularly before a crisis) that can help build organizations’ resilience that allows them to succeed in multiple scenarios — not omniscience. When a crisis comes — which will almost certainly be a surprise — is there familiarity with the features, with mechanisms and frameworks of thought to respond — or will the response by pulled along by the current and sentiment of the crisis? Forecast cynicism should give way to appreciation of the process and the value that it holds for organizations. This preparatory value is enhanced when mental preparedness is accompanied by behavioral preparedness, through wargaming and other simulation

This year, in under three months, all economic forecasts lay by the side of the road discarded as the year fell apart. Those who placed great faith in forecasting were not only dismayed but also disoriented. Those who were clear-eyed about the intricacies of forecasts were also dismayed but less disoriented. We hope our seven insights about forecasting will help you stay oriented next time forecasts inevitably fail. Till then we wish you all the best for 2021 — it can’t be worse than 2020, right?