“Should” we follow a plan or test hypotheses? I don’t see these as opposites, but I do see an underlying system that tends to result in a continuum. And there’s a missing third item. Let’s explore.
Along the way, we’ll discover a primary source of technical debt, of organizational mis-alignment, and what we can do to solve that root cause.
What are Plans and Hypotheses anyway?
In a business context, I use these definitions.
- Hypothesis: an explicit, falsifiable statement about the present state and interactions of a system that predicts its future state.
- Plan: an explicit story that describes our future actions and the desired resulting state of some system.
The key part of each definition is the word explicit. Because both simply describe a thing that already exists in the underlying system. By describing that thing, they allow us to manipulate that thing.
Both Hypotheses and Plans are explicit Understandings of an underlying system.
Thus, we have a Context: a system + our Understanding of that system (defined in Hypotheses and Plans).
In the absence of Hypotheses or Plans, the system and our understanding still have underlying factors.
- Without Hypotheses, we have Assumptions: an implicit state or interaction within the system/context that affects its future state.
- Without Plans, we have Independent Beliefs: a set of implicit stories held by each actor in the system that that actor uses to make sense of the current context and take actions, with the desire of attaining some independently-desired state.
Notice that Assumptions/Hypotheses describe the system, and Beliefs/Plans describe the people / teams. Thus, when a hypothesis is missing, it “breaks down” into a single assumption – there is still a single system interaction, it is just implicit / unknown. But when a plan is missing, it “breaks down” into many independent Beliefs – without a unified story or goal, each person makes up their own.
Should we Plan, Hypothesize, or both?
The answer depends on an underlying factor: uncertainty. I see two kinds of uncertainty, and total uncertainty is (I conjecture) the product of these.
- How Wrong/Unaware we are: the degree to which the actors know the system interactions. How much their Hypotheses or “working hypotheses”/assumptions actually match the system dynamics.
- How Dynamic the system is: the degree to which the system’s interactions change over time. How long does it take for a valid Hypothesis to become invalid?
I believe that doesn’t actually matter whether we are wrong/unaware, or whether the system is dynamic. Because we only interact with the system through our beliefs and behaviors, the effect of us being wrong about a system is the same as the effect of us having been right previously, but the system has changed such that we’re wrong now.
Therefore, I conjecture that we can measure any Context as having a single uncertainty value. And that uncertainty value determines which kind of Understanding is most useful.
OK, so … both, then?
Well, almost. Because executive function is a limited resource. People only have so much time/space/energy to think. Being explicit about anything allows us to manipulate that thing, but it also costs executive function.
And it doesn’t just cost executive function once to make the Plan or Hypothesis. Each Plan / Hypothesis requires ongoing executive function to keep explicit. We have to maintain it or we forget it. We pay that maintenance with constant communication, writing things down, defining and analyzing data, and so on.
Since every Plan or Hypothesis has weight (cost in executive function), we have to limit ourselves to just a few.
Worse yet, each large system (like a company) has many smaller sub-systems that have different (often opposing) dynamics. So people involved in each sub-system need to understand both their sub-system and the larger system. That means more Understanding elements, which means more weight.
Thus, our Understanding of the larger system has to be limited enough to leave executive function free for each person/team to also Understand their own part. And leave even more free so that they can make decisions and take actions!
Our available executive function is extremely limited, indeed.
And that’s why we tend to see teams either Follow a Plan or Test Hypotheses, but not both. This is actually a healthy action! (It just may not be optimal, more on that later…)
How people/teams/companies Understand
The group tends to start out in an open state. They don’t have any understandings of either kind. They follow a process like this to decide what Understanding to create (usually implicitly).
- The team identifies their degree of uncertainty. Because executive function is limited, they don’t measure it as a value on a continuum. Instead, they measure it as a boolean value: high or low.
- If uncertainty is high, they start Testing Hypotheses. If uncertainty is low, they start Following Plans.
- Because executive function is limited, they never think to re-assess their uncertainty. They continue their mode from step 2 even if the situation changes, until the change is too drastic to ignore.
The result is good…
Teams that start in low uncertainty Follow Plans. They are efficient, aligned, and action-oriented. They deliver strong output.
Meanwhile, teams that start in high uncertainty Test Hypotheses. They learn rapidly about their system and interactions. They understand their customers and know what things actually deliver business value.
The resulting process is good:
- Teams are efficient when they can be, and exploratory when they must.
- Teams conserve executive function so they can spend it on taking actions and attaining results.
…and bad…
Teams that Follow Plans often go the wrong direction, because implicit Assumptions change (over time or between sub-systems). This results in two problems: conflict between teams at a higher level (often showing as problematic cross-team dependencies during planning), and output that doesn’t deliver outcomes / business value.
Meanwhile, teams that Test Hypotheses often end up going nowhere, because their Independent Beliefs average out. This results in high expense, analysis paralysis, internal conflict, and a lack of measurable output. They understand business value and empathize with the customer, they just aren’t able to deliver the goods.
Over time, the result is bad:
- First, the high-expense, low-output teams get cut / not promoted. So teams that Test Hypotheses don’t survive.
- We are left with an organization of teams that Follow Plans.
- Uncertainty arises, as does variability between sub-systems.
- Now teams are following plans that conflict with each other and are not aligned to actual business value.
- Teams try to become more effective in the way that survived the first cuts: they increase efficiency and output. This decreases their ability to accommodate variations caused by other teams.
- That results in painful dependencies. All actual business value is no longer aligned with the team structure (because the system changed), so doing anything useful requires coordination between independent and contradictory plans.
- Result: outcomes drop to 0.
- We observe organizational mis-alignment.
Side note: this is also the source of most large-scale tech debt.
- In step 4 above, each team changes the code to align with their plan. But these plans contradict, so the code becomes internally self-contradictory.
- In steps 5 and 6, each team attempts to protect its ability to deliver by locking down their section of the code to match their plan. Testing artifacts, tools, and other elements now contain the same contradictions.
- These internal contradictions are the technical debt. They surface as near-duplication, as inconsistent names, as alternative designs, as mis-aligned tests, as tests that break when unrelated code changes, and as integration bugs on every change.
We need a way for teams to do both…but without spending more executive function.
I see 2 ways to improve
First, we can’t just “be more careful” or “pay more attention.” We don’t have the attention to pay. We also can’t just “not fire the teams that Test Hypotheses”. Those teams are ineffective. I’d rather change them than fire them, but either way they are not going to survive in their current form.
Here are two things we can do.
- Consider uncertainty to be a range, not a boolean.
- Know when to change modes.
We just need to find a way to do each of these without increasing executive function. And that tells me something is missing.
To consider uncertainty as a range without spending executive function measure it, we need a balance.
We need to balance Hypotheses with Plans, “never” going beyond a 75%/25% imbalance.
This is a quick rule: make sure that you have both Hypotheses and Plans, but not in equal number or emphasis. One good guide is to spend at least 25% of your executive function in the kind of Understanding that applies less well to your current situation. That way your maintenance costs will keep the other mode visible.
And Troy Magennis happened to post a solution to knowing when to change modes just before I wrote this.
Measure when your balance is off by using Nelson’s rules to detect recent shifts in trends. Automate this analysis, then forget it until an alert fires.
Taken together at the team level, teams can gain the effectiveness of both styles.
The Mostly-Plan team
When uncertainly starts low, the team has a Plan and that Plan includes two elements: Testing one Hypothesis at a time, and updating the Plan based on the results. This requires some iterative process to execute the Plan, and something more like Lean Startup than like Retrospectives in order to work with Hypotheses.
The Mostly Plan-Following team delivers nearly the same output as it would with plan-only, but they also learn more about business value and customer needs. They incorporate that learning into their plan, so they deliver outcomes, not just outputs.
This team also inflicts less uncertainty on other teams. It is more predictable. That means that other teams can take it as a dependency more easily and build on its results. It also means that the organization will take those dependencies, making it harder to change when the Hypotheses start shifting. And that makes it poor at taking dependencies on other teams – it will magnify the impact of any shift in those teams.
The Mostly-Hypothesis team
When uncertainty starts high, the team Tests Hypotheses and attempts to weave the results into one Shared Story. The Shared Story is a very lightweight Plan. It ends up being a decision-support tool that each person uses to make sense of each situation and guides their actions.
The Mostly Hypothesis-Testing team’s actions tend to align around their System Story, and they get cohesive output. Because the Story remains lightweight, it is easy to update when Hypotheses shift. It doesn’t attempt to predict the decisions or actions the team will take; it just states the shared understanding of the goal and the kinds of interactions that are likely to get there. The team delivers outcomes that match its expense, and it doesn’t get fired.
However, this team inflicts a fair amount of uncertainty on the organization. They need to keep the cost of change low, so they don’t predict the future. Other teams can’t take them as a dependency, though they are great at taking other teams as a dependency.
The Balanced team
There is also a third option: the Balanced team. This team has multiple Hypotheses and a rolling-wave Plan. The Plan more detailed in some places, and more like a Shared System Story in others. There is a general Shared Story. Additionally, there is a detailed plan in 2 places: the near future, and any area well-mapped with established Hypotheses that haven’t changed in a while. These are the two areas of greatest certainty.
The Balanced team ends up between the other two teams in its results.
It offers well-defined regions of stability and uncertainty. Other teams can depend on it in regions of stability, and it can depend on others in regions of uncertainty.
Local results
| Mostly-Plan | Balanced | Mostly-Hypothesis | |
| Output | High | High | Moderate |
| Business Value | Moderate | High | High |
| Depend on others | Poor | Great in some areas, Poor in others | Great |
| Be depended on | Great | Great in some areas, Poor in others | Poor |
Over time, the team is able to shift between modes. Its Hypothesis Testing allows it to define measures and controls to detect trend shifts. It can then assess the rate of trend-shift alerts, and know when to shift modes.
Natural business cycles will cause the team to shift between modes like a pendulum every couple of years. Things become stable and the number of alerts drops, so the team transforms more of its Story into a Plan. It becomes Balanced so that others can build on its stable points. Those stable points accumulate until nearly everything is stable, and others depend on it utterly.
Then a disruption happens and the alerts start firing. The team replaces some of its plans with more hypotheses and a clearer shared story with areas of explicit uncertainty. Partner teams shift away from this dependency or shift themselves to address the growing uncertainty. As this continues, the team starts taking dependencies on more platform elements to gain leverage and free resources to test more hypotheses. It makes fewer and fewer detailed plans, becoming more innovative.
And then the cycle repeats.
System results
The system as a whole adapts to teams shifting to manage their uncertainty. It can also shift its stance overall between the modes. Dependencies are always clear. Even when they aren’t great, it’s obvious why not, and the dependency can shift.
Technical debt will remain, but will remain manageable. Different teams will still have semi-contradictory plans. But the areas of explicit uncertainty and shared stories operate like a clutch, allowing different parts of the code to be different and still work together well enough.
There is still some organizational mis-alignment. Important domains will shift, and we’ll want to re-organize teams to match the new strengths. But those re-orgs can be far less disruptive. Again, the explicit uncertainty mechanism in each team allows that team to better in areas of unknown. So we can keep more of the teams intact in a re-org, shifting their domains more than their people.
Of course, this company will never be as flat-out efficient as a Plan-only company. And that matters when commoditizing your business. This approach isn’t for every business, but I think it does work well for the majority.