There is much chatter about large language models of late, divided more or less equally between optimism and skepticism. On the one hand are claims that the technology will revolutionize all aspects of our work and lives. On the other is great skepticism about its limitations, driven by concerns about truthfulness and transparency.

But asking what models like ChatGPT can do is the wrong question. As a tool, we must instead look at the system of “human + technology” and ask how we can best use it, examining our own objectives and competence in using it, as well as its fitness for this purpose. As strategists, we explored these questions in the specific context of business strategy.

ChatGPT is a powerful predictive language tool. It can create surprisingly plausible responses to prompts, having been trained on vast swathes of textual data. Its responses are constructed without regard to truthfulness and, in its current form, without transparency into which sources the responses are based upon. Unsurprisingly therefore, it has been demonstrated to sometimes base responses on unreliable sources and to quote as fact things which are not true.

How could such a tool possibly be useful in generating business strategies, upon which the economic fate of companies depends? If painstaking human analysis of facts about markets and competitors, carefully crafted hypotheses about competitive advantage, and internally consistent and plausible action paths are replaced by unsourced and questionable factoids, how could this possibly represent progress?

We conclude, perhaps surprisingly, that if you use GPT in a very specific manner, it can in fact be helpful in multiple ways.

Ask a friend

Think of the GPT as a knowledgeable, confident, and very persuasive friend who is prone to sometimes making things up or confidently asserting as facts things they are as likely to have read on the side of cereal box as in a scientific journal.

While it would be foolish to rely unquestioningly on the strategies generated by such a friend, they could in fact be very useful in several ways, providing we fully apply our powers of judgement. We could tap into this friend’s extensive worldly knowledge to characterize the competitive environment. We could generate ideas to base a strategy upon. We could simulate how different strategies might play out in different scenarios and become aware of risks and contingencies. We could tap into the friend’s powers of verbal dexterity to better articulate our strategy story.

Extracting benefit and avoiding being misled would all depend upon exercising our powers of discretion, by asking the right questions, by judging the consistency of suggestions with our own knowledge and experience, by validating things that are asserted as facts, and by testing the resulting strategies. In other words, we stand to benefit by applying the very powers of discrimination that any competent strategist must regularly employ.

All strategists know to be skeptical of “facts.” What is true in general may not be true in a specific case. What was true in the past may not be true now. What is true may not be inevitably and reliably so. Our understanding of a complex, dynamic situation is always incomplete. And even if something is true, there are always alternative ways of framing problems and solutions. Furthermore, strategy is in one important sense based as much on fiction as on fact. By aiming to shape circumstances to bring about a new goal, strategies create new facts while being grounded in existing ones. As such, strategies have always been in the business of discriminating useful fictions from misleading ones.

One consequence is that ChatGPT and similar tools are more likely to be useful to an experienced strategist than to a naïve beginner. Such tools are therefore no substitute for the cultivation of strategic minds.

We explored the ways in which the powers of strategists might be extended through a series of experiments on different aspects of the strategizing process. To organize our experiments, we set up a tree of hypotheses to test.

Ideation

The best strategies are often creative and counterintuitive. Chengwei Liu, a professor at ESMT, notes that contrarian thinking is required. That sounds easier than it is. Typically, prevailing industry or company logic drowns out “crazy” ideas. Opening up strategy to outsiders can mitigate against this constraint. We can view a large language model as such an outsider.

We conducted two experiments to explore how this might work. In the first one, the objective was to create a new concept for a bakery. The initial response from the tool was rather conventional, suggesting the sale of savory products only. Further prompting produced more radical ideas, including 3-D printing of pastries and the use of AI to create unique flavors. We wanted to find out more about the latter, and eventually obtained the suggestion to start by collecting data and feedback from easy to access sources such as family and friends, social media posts, and product testing at local farmer’s markets.

In a second experiment, we asked ChatGPT to come up with new ideas for a video streaming service. From a list of suggestions, we picked educational streaming services and deliberately steered the conversation towards the possibility of partnerships with universities, and then to the tactic of getting individual professors at a partner university to participate.

The two experiments demonstrated that responses tend to be broad or generic at first. If a strategist does not know how to ask the right questions and push the conversation into a more fruitful direction, this can result in a dead end. Once the ideas start to be more unusual, discriminatory power is required. For example, a partnership with a university for an educational streaming service might sound like a winner, but knowing how universities work, we were able to redirect towards getting individual professors involved. The final idea was to focus on early career professors who need more support creating content, and ChatGPT suggested routes of reaching them such as conferences and social media.

Despite some obvious limitations, ChatGPT has three big advantages when it comes to ideation. First, it is fast and easy to use. You can avoid all the effort of getting different people in the room. Second, the tendency toward conventional thinking can easily be overcome. Third, it’s easy to generate many ideas. The most interesting ideas were however mixed with the implausible, impractical, and untested ones. Only by sifting through these in a discriminating manner, can you get a better understanding of what might actually work.

Experimentation

Once a company has sketched out a set of ideas, the conversation turns toward selection. Scientists would conduct a series of experiments to do so. Companies can follow the same path. For example, start-ups often do this by introducing ideas in a quick set of successions and pivoting whenever necessary.

Established companies are more reluctant to experiment this way. Large language models themselves are a good example! Google developed such a technology but was initially worried that applying it to search was too risky.

In practice, for many companies, experimentation starts with thought experiments. The strategic implications of new ideas (or adjustments to existing approaches) are explored against the backdrop of different scenarios.

At the heart of the scenario planning process sit the descriptions of situations in which your strategy needs to prevail. A strategist’s judgement is required to pick the relevant parameters, but ChatGPT can take over from there, turning them into a vivid and usable narrative.

We tried this out, asking for two stories capturing the future of the travel industry. It took several iterations, but eventually we obtained two detailed and convincing stories covering both economic development in different parts of the world and consumer preferences linked to climate change as well as demography (see Sidebar for one of those stories).

Future of the travel industry – Story created by ChatGPT

In the future, the travel industry will continue to grow, with the market for senior-friendly vacations and cruises in developed countries expected to be significant. According to the United Nations, by 2050, the number of people aged 60 or over is projected to reach almost 2 billion, or 22% of the global population. This demographic shift will lead to an increase in demand for senior-friendly vacations and cruises, as older travelers are more likely to have the time and disposable income to travel.

On the other hand, the market for eco-friendly and sustainable options in emerging economies is also expected to grow as more and more younger people in these countries enter the middle class. According to the World Bank, by 2030, the number of middle-class consumers in developing countries is projected to reach 1.75 billion. This will lead to an increase in demand for sustainable and responsible travel options, as younger people in these countries are more likely to be interested in responsible tourism.

It’s worth noting that the prediction of the size of the market is difficult to make as it depends on many factors like economic and political stability, health crises, and travel restrictions. However, it is clear that the travel industry will have to adapt to these demographic shifts and changing preferences to stay competitive.

ChatGPT is much less attuned to actually testing your ideas. We asked whether investments into Caribbean Cruises would make sense, considering the scenario described in the sidebar. Several avenues of exploration were suggested by ChatGPT that could be interpreted as skeptical, but the details were insufficient. A further complication is that scenario planning requires a strategy to be constant and robust under a series of different scenarios, a specification we found hard to realize with ChatGPT.

In short, you won’t be able to outsource experimentation to ChatGPT, but its ability as a storyteller makes it easier and faster to consider alternative futures.

Evaluation

Before a company is ready to launch a new strategy, it requires a clear picture of the market and it’s likely response. Will there be sufficient demand? Are there existing competitors it needs to be prepared for?

In our fourth experiment, we used ChatGPT to evaluate a business idea with these questions in mind. The idea was simple. We want to introduce a strategy making app. The potential market would likely be small companies, as many of them do not have sufficient budgets to hire consulting firms and have limited strategy expertise in-house. We first asked ChatGPT how big this potential market is. The answer was too generic, noting that 99.9% of all US businesses are small businesses. A more fruitful avenue was asking for potential competitors, which eventually lead to a plausible list.

The main advantage here—as in all the other steps—is speed. It only takes a few minutes to go through the entire interaction. That’s sufficient as a first test and crucially allows you to narrow the search field for further analysis.

Storyfication

Humans understand the world through stories. Strategists often underestimate the importance of how ideas are communicated, wrongly assuming that what they say matters most. With only 28% of managers being able to correctly name three strategic priorities, this is an obvious miscalculation.

One of main attractions of ChatGPT is its ability to write well, complementing the blind spot of many strategists. Thousands of students have used it to help with their essays, in some instances writing much better than they could themselves.

In experiment five, we instructed ChatGPT to write up the strategy for a UK window manufacturer. After we provided the content, an initial list was produced. A further iteration separated the challenge from the solutions. In some settings (for example, government institutions) this might be sufficient. But the real value was generated in a final step, when we asked ChatGPT to write this up in vivid and engaging manner.

With storyfication being both time-consuming and beyond the capacity of many managers, this might be the most useful application of ChatGPT in the strategy making process. If you provide the facts, the tool can add style, which is crucially important to communication and implementation.

What if you are not an experienced strategist?

Thus far, we relied on the discretionary power of a strategists to leverage ChatGPT. But what if you are a novice? ChatGPT can still be helpful as trainer! Think of the tool as a fellow student: not every answer you’ll get from it is correct, but by in large you will receive answers aligned with textbook answers.

What the tool cannot offer you is experienced judgement on whether a line of inquiry will likely work. We tested this in our final experiment by having a conversation about a fictitious streaming service’s new strategy to introduce an ad-supported service. ChatGPT offered suggestions aligning well with conventional wisdom—which is great if you are just starting out in the world of strategy.

At the same time, the interaction also demonstrated the tool’s limitations. Strategic advisors are (at least in part) advocatus diaboli. For example, a reasonable question would be whether an ad-supported service is not a sign of weakness? A further development of the argument along these lines, might lead for example to the company increasing its footprint in Nollywood, which could conceivably be a more attractive strategy.

Because the value of ChatGPT which we observed is dependent upon the judgment of the strategist using it, the technology will likely polarize more that it democratizes strategy. We demonstrated that it could help the best strategists to be even better. But the technology won’t replace their powers of discrimination. If anything, it will shine a brighter spotlight on them.