The efficient-market hypothesis is a theory that suggests that the stock market reflects all publicly available information, and that this causes stock prices to be a very accurate reflection of reality. In short, the rationality of the market means that stock prices should be, in general, highly optimized.
I think professional organizations are often viewed the same way. That surely, over time, the best methods, the best tools and the best ideas will prevail. And of course this notion often holds true. It is for example easy to see that the myriad of ideas and methods known as agile project management have become increasingly popular in a variety of organizations in different fields.
Originally popularized in (and by) the software industry, agile is now entering all kinds of organizations. Frameworks such as a Scaled Agile Framework (SAFe) and Large-scale Scrum have been adopted by companies ranging from Lego to Huawei. The music streaming giant Spotify has even famously created their own approach to an agile framework called Spotify Tribe.
It is easy to see that agile has also found success in many other fields, beyond the software industry. This means the agile philosophy and the related toolkits have some claim to success, as 19 years have passed since the publication of the Agile Manifesto (a popular point for timing agile adoption). But what is the exact nature of this success?
We could suggest an efficient organization hypothesis: that eventually any organization will absorb the best methods, tools and ideas from the marketplace. After all it seems reasonable to assume that for the people in a professional organization, there are enough internal and external incentives to try to and optimize their work and leadership.
But is this really how we adopt new things? Did you or your organization choose (or not choose) agile because you have gathered information and evidence in support of a purely rational decision? I think regardless of whether you're a trainee or a senior executive, it would be hard to point to such conclusive evidence. Besides, in a competitive environment, sitting still and waiting for scientific proof (which might never arrive) is not a very wise strategy.
Of course then we are left with another common tool of decision making: the observation that agile is very popular nowadays. In itself, popularity is a very reasonable heuristic, but it leaves us with two caveats: 1) it does not explain the nascent success of agile 2) plenty of things that were once popular in organizations are now considered permanently outdated.
So what drives the adoption of new methods and ideas in organizations, if they can't really be proven to be effective and popularity only accounts for the decisions of later adopters?
There is a potential answer in some of the other decisions the people in those organizations make every day. Something that plays into every decision from buying lunch, to buying a pair of shoes, to buying a house. It is our emotions and values.
Psychological research has long indicated that people buy with their emotions. Advertising and marketing make great use of this every day. The point is that what we buy (and buy into) comes from within us. Buying lunch has arguably very little to do with the rational considerations (energy intake, nutrients, distance), and way more to do with emotional considerations (taste, decor, social dynamics). And if we buy lunch with our emotions, could it be that we also buy our tooling, methodology and even change initiatives with emotions?
If we want to think along these lines, then the result is that our premise of efficiency gets flipped around. It is not the organization choosing the "best" change initiatives, but the people in the organization choosing the most emotionally fitting ones. Despite the still inconclusive evidence for its efficiency, consider for the original Agile Manifesto principles:
Individuals and Interactions over processes and tools
Working Software over comprehensive documentation
Customer Collaboration over contract negotiation
Responding to Change over following a plan
These to me are quite obviously value statements. Aspirations for change. The original agile principles very much address the individual needs first and the organization's needs only by extension. These are the kinds of things that most of us want to do anyway. And rarely are these conclusions drawn from some rational analysis, but rather from what would make us feel better emotionally.
The difference between having frameworks and tooling, versus having aspirations, was probably not lost on the group that arrived at the above principles. Their motivation was after all to sell the rest of us on this (at the time) less-known set of ideas.
The thing about frameworks is that they are tremendously useful. Structuring and codifying complicated patterns and interactions into something more palatable is something we do need. Following rules and guidelines allows individuals and organizations to function even on their worse days.
It is important, however, not to put your framework in front of the proverbial horse. The horse in this case being the emotions and aspirations that the framework can help us fulfil. Even the best framework can't skip over the thinking and feeling required to both create and spread change within the individuals that ultimately any organization is made of.
Personally, my takeaway from this is as follows:
While we can endlessly stare into change initiatives and new frameworks and say that we're only rationally gauging for efficiency and clever mechanics (and this is what the organization context would often have us do), it would be smart to carefully consider if we really believe the efficient organization hypothesis, and if we can find the supporting evidence.
Alternatively, we could say that whatever we as people want to be – whatever change speaks to us – is the change we will not merely adopt, but actively invite in.
