Adam Smith’s “invisible hand” famously guides market economies, in which unplanned transactions by different agents lead to collectively beneficial outcomes. This logic governs most interactions between enterprises and consumers and between enterprises today. But work within enterprises is mostly organized according to the very different concept of bureaucracy, the principles of which were codified and popularized by a later philosopher, Max Weber. In this model, activities are planned, mandated, and guided by the very visible hand of organizational hierarchy.

Despite massive changes in the business environment that have put a premium on agility and innovation — and despite “bureaucracy” having become a synonym for inertia and stagnation — the concept has shown remarkable staying power. How many of us could name more than a handful of enterprises that are not fundamentally organized as bureaucracies? We explore this seeming paradox by examining the nature and functions of bureaucracy, why it has remained the dominant paradigm, and whether and how emerging challenges will require leaders to reinvent their organizational models.

How bureaucracy works

According to the transaction cost theory of the firm, companies exist because they can eliminate certain transaction costs that would be incurred in free markets, such as the effort required to discover prices or renegotiate contracts. As a result, companies can often better coordinate tasks that require a high degree of alignment between multiple parties or involve nuance or context. For example, car manufacturers typically outsource the production of car parts to suppliers through markets, because the exact shape and functionality of each part can be specified in advance. In contrast, car assembly typically happens within a firm, because the assembly process has many highly interdependent steps, and can therefore be error-prone and require nuanced judgement.

Bureaucracy, according to Weber, emerged as an organizational form to create stability and predictability, making enterprises more efficient. In particular, he identified six essential characteristics of bureaucracy that set it apart from other forms of organization:

1. Division of labor with clearly defined roles
2. Hierarchical management structure with clear lines of authority
3. Documentation that specifies required decisions and actions
4. Specialized training and meritocratic selection for each role
5. Full-time managers appointed to operate the organization
6. Static, depersonalized rules that exhaustively guide management

Most large companies today fundamentally fit Weber’s definition of a bureaucracy. They task employees with job descriptions to facilitate the division of labor and arrange them in multilayered hierarchies through which directives are transmitted, executed and monitored. Hiring and promotion decisions are based on objective criteria specific to each role. And extensive and relatively static rule sets authorize some actions and forbid others, with limited leeway for judgement and initiative based on circumstances.

This organizational form thrived in the stable, predictable environments in which most businesses historically developed. Hierarchical management enabled detailed planning at all levels of the organization; clearly defined roles allowed tasks to be decomposed and executed efficiently; and comprehensive, unchanging rules allowed new hires or resources to be easily integrated into the organization in pursuit of economies of scale.

Bureaucracy’s staying power

For all its popularity, the effectiveness of bureaucracy has increasingly been called into question because of structural changes in the business environment. Uncertainty is rising, disruption is increasing, and what it takes to succeed is changing faster than ever. As a result, companies must increasingly compete on adaptiveness, learning, and innovation. But bureaucracies are inherently ill-suited for these new imperatives: static organizational rules inhibit adaptation; top-down hierarchies are predicated on forecasting and planning rather than experimentation and learning; and highly codified tasks often do not leave sufficient room for discretion or imagination. Indeed, bureaucracy’s demise has been predicted for decades: “The conditions of our modern industrial world will bring about the death of bureaucracy,” wrote leadership scholar Warren Bennis in 1966.

Why, then, has bureaucracy remained the dominant organizational paradigm? The bureaucratic paradigm is simple and well-codified, allowing it to be easily understood and applied by managers in any industry. The things it does well — such as planning what needs to be done, decomposing it into specific actions, and coordinating those among employees — are still valuable in many situations. Bureaucracy is a also convenient mental model that is often employed even if the actual network of the organization departs somewhat from the strict model.

Some companies have boldly experimented with alternative forms of organizing, moving away from the planned, top-down nature of bureaucracy. (See Exhibit 1). These philosophies share some commonalities, including a focus on avoiding top-down hierarchy and allowing teams to self-manage, with the aims of increasing adaptiveness through decentralized action and increasing innovation through employees being more engaged and motivated. Zappos has gone further than most to break free from the restrictions of bureaucracy, by fully empowering each team to decide on which value propositions they offer and make their own investment decisions, effectively bringing market forces inside the organization. (See sidebar, “Zappos’ journey to bring the market into the organization.”) However, such alternative organizational models are still not proven, scaled, or codified to the degree necessary to make them an accepted alternative for mainstream companies today.

Sidebar: Zappos’ journey to bring the market into the organization

In 1999, Tony Hsieh joined Zappos as an advisor and investor, and eventually became CEO. After 10 years of rapid growth, Zappos was bought by Amazon in 2009, but as part of the deal Hsieh negotiated that Zappos would retain significant independent operational control. As a result, Zappos has had the freedom to experiment with its management philosophy, which has increasingly broken with the bureaucratic model.

Hsieh believes that bureaucratic organization models ultimately limit adaptability and innovativeness and are not conducive to human flourishing and happiness. This is in contrast to market-based models like cities or economies. As Hsieh says, “Companies don’t innovate, markets do.” That is why Zappos is increasingly introducing market-based elements into its organizational model.

Zappos’ first large departure from the standard bureaucratic model came in 2015 when it formally converted to Holacracy, a model focused on breaking down hierarchy and enabling self-organization. The company has since evolved its organizational model into a form it calls “market-based dynamics,” — which extends the logic of self-organization to decentralize financial and investment decisions as well — because in Holacracy budgeting decisions were still centralized, which ultimately limited the capacity of teams to self-manage. Zappos is still experimenting with its new model, but it has demonstrated that there are alternative viable models to bureaucracy.

Structurally, Holacracy breaks a firm into independent teams called circles. They function like start-ups with their own independent purpose, but each purpose is aligned with the overall organizational purpose. Whilst the goal of each circle is prescribed, circles are free to organize however they wish to achieve that goal. Holacracy does have some element of hierarchy, as circles can be nested inside other circles. However, in contrast to typical bureaucracies, circles can have overlapping responsibilities. Additionally, lines of accountability between super-circle and sub-circle are bidirectional and consent-based, so super-circles don’t control sub-circles.

Employees are highly autonomous in this model — they can choose which circle or circles to join, and while they agree to deliver specified outcomes for each role they take on, they are completely free to decide how to achieve these outcomes. Unlike bureaucracies, circles don’t have full-time managers; instead, decision-making happens through structured group conversations. Consequently, career development works in a very different way. Career paths are characterized by either developing expertise or broadening skill sets, in contrast to the typical hierarchical career ladder. As Lead Organizational Designer John Bunch says, “Career progression moves from a ladder to a jungle gym. Don’t look up, but rather look around for opportunities and do what fits you best.”

Market-based dynamics extends self-management by not only giving circles the freedom how to achieve their goals, but also allowing circles to decide what their goals are, as long as they are profitable and consistent with Zappos’ purpose and values. Circles decide on their goals by specifying a ‘menu of services’ they offer (e.g. customer-facing services like sales, or internal services like development), and they are free to sell these services to both internal and external stakeholders. Each circle is free to reinvest any profit as they wish, within the constraints of the purpose of Zappos. As Hsieh says, “If an employee wants to start up a circle to set up a cupcake bakery, that would be great. The only demand is that this bakery abides by our purpose to deliver the very best customer service, customer experience, and company culture.”

Additionally, if a circle needs to use specific services to deliver its offering, it can source these services internally or externally, except for a small set of centralized services — like payroll, core IT systems, or legal — that each circle has to use and for which it pays a contribution margin. Recruitment and salary levels are currently centralized, but the company plans to decentralize this too over time, which will give circles the complete freedom to recruit new employees and set salary levels within the bounds of their own profitability.

New tools are needed to support this decentralized model, as typical tools assume standard hierarchical structures and decision rights. For instance, Zappos has its own internal platform that allows for decentralized budgeting, and an HR tool that allows for people having multiple roles in overlapping circles.

Success is defined as a balance between efficiency, stability, and resilience. Typical business metrics focus heavily on efficiency and stability, but neglect resilience. In order to address this gap, one of Zappos’ primary metrics is the percentage of circles’ income that comes from new sources, because increased income from new sources means that circles are more entrepreneurial, which makes the larger company more resilient.

Eventually Zappos aspires to blur the boundaries between the firm and the market even further by allowing non-employees to set up Zappos circles as well. These ‘external circles’ would leverage the basic set of services that Zappos offers, for a fixed percentage contribution margin. Just as Software as a Service and Computing as a Service (cloud) lowered the barrier to developing new technology, Zappos hopes that this model will lower the barriers to start a business by offering ‘entrepreneurship-as-a-service’, while allowing Zappos to scale faster. This would move their model even further away from the typical bureaucratic command-and-control structure, as it would incorporate people that are not directly managed by the company.

Zappos acknowledges it does not yet have firm data to prove that its model is superior to bureaucratic alternatives. However, it does see encouraging signs that this model could in the long run deal much more effectively in dynamic business environments. As Hsieh says, “Typically, reorganizing is a long process that is fraught with failure. In contrast, our organization chart changes multiple times a day — that’s our new normal. This allows us to respond to changes incredibly quickly.” Zappos is still experimenting at a rapid clip, and they eventually hope to develop a full-fledged, market-based organizational philosophy.

Zappos’ experimentation teaches us valuable lessons about the power — and the limits — of current alternative organizational forms. It is a clear demonstration that the tenets of bureaucracy are by no means the only way to operate a successful business, and even quite radical departures from the bureaucratic model can deliver commercial success. It also shows that the process to develop an alternative is by no means rapid or straightforward, as Zappos has experimented with alternative models for half a decade and still hasn’t achieved its full ambition.

Toward a new organizational paradigm

So is bureaucracy’s dominance here to stay, in spite of its frequently heralded demise? We believe not, as further changes in the business environment will likely push it past its breaking point. Uncertainty and dynamism are still rising, which will put a higher priority on learning, resilience, adaptiveness, and innovation. In addition, AI’s rapidly increasing power presents new challenges, because organizational models must encompass not only human but also algorithmic decision making. Technology also expands the range of possible organizational solutions, because algorithmic decision-making allows activities to be coordinated on much shorter timescales and across a much wider set of participants (within and beyond of the legal entity).

Leaders therefore need to reconceive the organization — from a static hierarchy and rule set to a continuously evolving model; from being human-centric to encompassing both humans and algorithms; and from being contained within company boundaries to encompassing connections and activities with external partners. They also need to reframe how their enterprises compete — from serving a relatively standardized and static set of offerings more efficiently to competing on the rate of learning in order to discover and act on new opportunities. For that we will need a new generation of organizational models, which we collectively call the “hybrid learning organization.”

While we don’t have a blueprint for the hybrid learning organization, we can already identify several emerging design principles. These include:

• Integrated data systems, data communications and decision engines so that routine decisions can be made at algorithmic speed;
• New human ways of working that foster imagination and higher-level cognition rather than rote decisions and actions, which will increasingly be replaced by AI and automation.
• Data-driven feedback loops to facilitate learning on multiple levels — from product offerings, to culture and organizational models, to business models, to the learning approach itself;
• The ability to operate and learn at multiple clock speeds — from split second algorithmic timescales through to the decadal timescales of social and ecological change;
• The ability to be ambidextrous, balancing exploitation of current models and exploration to find new ones, developing a mosaic firm that uses bureaucracy when appropriate, but also employing more dynamic organizational forms when needed;
• Seamless coordination with both internal and internal and external stakeholders, through digital platforms and multi-company ecosystems;
• A dynamic organizational model that is continuously adapted and refined based on the context

Perhaps not surprisingly, the first glimpses such an organization can be found in some technologically advanced companies. Some of the new ways of working can be seen in Zappos, as described above. Alibaba has developed the “self-tuning enterprise” concept, in which algorithmic learning principles are applied throughout the organization. Its decision engines autonomously experiment, modulate and improve over time, relying on information from its vast data ecosystems, and decentralized teams have the freedom to trigger new initiatives when they see market potential. And whereas most organizations have a static vision and organizational structure, Alibaba continuously evolves its vision and organization design. Amazon (Zappos’ parent company) demonstrates similar elements, like data systems that are interconnected throughout the organization, allowing all parts of the company to react to new information without guidance from a centralized authority. Employees often take a “hands off the wheel” approach, validating algorithmic decision processes and setting guardrails rather than intervening directly, allowing them to focus on more creative efforts such as imagining new businesses and business models.

The challenges facing businesses today are evolving rapidly, yet most firms are still organized along the entrenched 19th-century paradigm of bureaucracy. Winning organizations of the future will recognize the new imperatives they face, embrace bureaucracy where useful but boldly experiment with new organizational models that harness both technology and human ingenuity where needed. The exact shape of these new models is still undetermined, but enterprising leaders are currently shaping them. In some of the companies described in this article, we can see hints of a revolution which will likely affect all companies and sectors eventually.

The authors would like to thank Tony Hsieh and colleagues at Zappos for allowing us to observe closely and discuss the Market Based Dynamics model. The views in this article are the authors’ own.