The process of innovation engenders more emotional highs and lows than any other in business, from the frustration of repeated failure to the ecstasy of achieving a long-sought-after breakthrough. These emotions might seem entirely peripheral to the business of innovation, or even positively dysfunctional, distracting us from making the next best move. But could these emotions in fact be a critical feature of an effective innovation process, rather than a bug, and can they be harnessed and exploited?

The Efficiency Fallacy

Imagine a ball on a landscape of possibilities, where the lower the ball sits, the greater the value unlocked. A ball sitting motionless at the bottom of a crater on the landscape is the epitome of efficiency. It is also the epitome of stagnation, since a deeper basin might lie just beyond the rim of the current one, and the laws of physics would do nothing to move the ball to the new, more attractive position.

Now imagine that the ball has agency (an ability to move around the landscape) and emotion (different internal states which inform this motion). What sequence of emotions might successfully navigate the ball to a new, more valuable position? Perhaps first would come a feeling of ennui, a recognition of the present stagnation. This might give rise to frustration with the lack of progress and a desire to do something about it. Hope, a belief in beneficial possibilities beyond what is currently known, might then support curiosity to explore these new possibilities. Determination might kick in as the steepness of the ascent of the current crater wall steepens, to reinforce continuing with the journey of exploration. Surprise and delight would likely kick in as the new deeper crater is glimpsed over the rim of the current one, propelling the ball forward toward a new reality. Finally, restlessness might play a role in avoiding settling too firmly into the new position, realizing that the journey of innovation never ends.

Emotion and Innovation in Business

Fanciful though it might seem, innovators like GE veteran Beth Comstock and Disney’s Shelby Jiggets-Tivony shared their own experience of the power of emotion in the innovation process, including apparently “negative” ones like frustration and ennui, at a dinner we held at TED with House of Beautiful Business to explore the topic.

The importance of emotions in innovation is also supported in our research for The Imagination Machine. Our interviews with business leaders suggest that the emotions of fear and complacency are highly corrosive of imagination and innovation. Conversely, imagination is supported by optimism (a belief in beneficial possibilities), curiosity (joy in exploring the unknown), and surprise (openness to anomalies, accidents, and analogies that suggest new possibilities and ways of looking at things).

Playfulness and laughter also emerged in our research as important to exploration. After all, biologically speaking, play can be defined as spontaneous, pleasurable, accelerated, de-risked exploration. One respected senior executive told us that amid a very tough transformation, laughter in senior meetings was adopted as a key performance metric, under the logic that optimism in finding a new path under exceptionally difficult circumstances was perhaps the most important variable.

The notion that sentiment affects results also gains some support from our work on the resilience of companies during the COVID pandemic and its aftermath. We could demonstrate that in this and other crises, competitive performance and competitive rankings diverge; counterintuitively, crises therefore represent greater opportunities for competitive gains than stable times. The competitive winners from such crises frame and pursue them as opportunities early in their evolution, rather than merely seeking protection from downside risks.

One can say that crises trigger a race to optimism which separates innovators from mitigators. We encountered some vivid examples of this in our research. Throughout its history, Berkshire Hathaway has made almost all its supernormal gains during times of crisis, which it routinely regards as opportunities. Aphorisms like being “greedy when others are fearful, and fearful when others are greedy” suggest an anti-fragility which is borne out by results.

The importance of emotions also has some support in science. We know that in Bayesian statistics, pessimistic beliefs can be self-fulfilling. Possibilities that we discount are not sought out, and therefore not found. We also know from neuroscience that imagination—adopting new mental models of the world—is triggered by the surprise of noticing a mismatch between our current mental model and an anomalous observation that doesn’t fit with it.

Harnessing Emotions for Innovation

Leaders set the emotional tone in an organization by how they communicate and act and therefore impact the climate for imagination and innovation. Our research suggests several practical ways in which they can act to harness the emotions that shape and propel imagination and innovation:

• Model and reward curiosity and outward orientation
• Don’t discourage dissatisfaction with the status quo: it’s a key motive force for change
• See possibility in change and adversity and encourage strategic optimism. Make sure that your company wins the “race to optimism”
• Avoid a culture of fear and compliance which will undermine exploration. Doing the right things needs to be supplemented by trying new things
• Celebrate the courage and curiosity of pioneers and intrapreneurs
• Encourage the instinct to play that we are all born with. Better that counterfactuals are explored hypothetically than with much higher stakes during the heat of a crisis
• Don’t expect emotional constancy but rather allow the natural cycle of emotions to propel the succession of the innovation process

While scientific management has perhaps encouraged us to be wary and skeptical of emotions in the workplace, they can be harnessed to good effect. In an age where AI will increasingly displace routine cognitive operations, we must learn to understand, embrace, and harness our humanity, and harnessing the power of emotion is one important aspect of this agenda.