Overcoming the Resource Efficiency Paradox

Many Scrum teams’ sprint burndowns look like a cliff, with numerous backlog items in progress at the same time, and most not marked ‘done’ until the end of the sprint.

If you’re not familiar with burndowns, the grey line represents an ideal rate of story closure. The red line represents the actual rate. Theoretically, the red line should track pretty closely to the grey line. In the example above, few stories were completed (marked “done”) until the end of the sprint.

Notwithstanding basic administrative failure to keep the team’s board up to date, in my experience this is a problematic pattern we see all too frequently in software development, caused by the Resource Efficiency approach to work assignment and execution.

This is how most teams operate. It’s what we were taught, and it’s what many believe is still the most productive way to do work. Resource Efficiency is a concept that comes from Manufacturing, evolved out of Frederick Taylor’s theory of Scientific Management.

In fact, in manufacturing, where you’re producing identical widgets, it makes total sense. But in knowledge work, like Software Engineering, where every widget is a one-off, it is problematic.

The Resource Efficiency Paradox

Sprint burndowns that look like cliffs are a symptom of the problems with Resource Efficiency.

• Excessive context switching by team members throughout the sprint.
• The team is unable to focus on one (or a few) epics at a time.
• Multiple sprint goals (or none!) in each sprint.
• You can’t tolerate multiple stories in a sprint that touch the same area of the code.
• Low collaboration among team members, as each person is working independently.
• Resource constraints that prevent pulling desired stories into sprints.
• Numerous backlog items in ‘waiting states’ after they start.
• Excessive handoffs, passing a backlog item from individual to individual.
• Stand-ups that feel more like individual status reporting than daily team planning.
• Inability to achieve “potentially shippable software” at the end of each sprint.
• Inability to frequently and consistently release fully completed features (epics).
• Team members lacking “big picture” perspective, as they focus on their specialty.

In their book This Is Lean, Niklas Modig and Par Ahlstrom coined the term the Efficiency Paradox to describe these problems, writing: “when organisations focus too much on utilising resources efficiently – the traditional and most common form of efficiency – it tends to lead to an increase in the amount of work there is to do. Consequently, the more organisations try to be efficient (being busy), the more inefficient they will actually become.” (to learn more, I recommend Niklas’ TED Talk on the Efficiency Paradox.)

We can do Better.

Benefits of the Flow Efficiency Approach

In my endeavor to solve these problems, I discovered some fundamental aspects of Kanban and Lean that I believe can help. They’re based on another type of efficiency approach, known as Flow Efficiency.

Flow Efficiency evangelists believe that you maximize effectiveness when you focus on getting things out the door quickly. In other words, when teams minimize cycle time, they reduce the time it takes to get backlog items from “in progress” to “done.”

A focus on cycle time is a forcing function. To reduce cycle time, you must:

• Limit the number of backlog items in progress at any one time.
• Pair-up or swarm or mob-develop each backlog item.
• Create smaller backlog items.
• Remove potential risks/blockers before starting a backlog item.
• Focus on one aspect (epic) of the product at a time.
• Explore ways to eliminate external dependencies.

The Flow Efficiency approach embraces the focus on reducing cycle time, claiming that any decrease in busyness is more than made up for by other efficiencies. Here are 10 reasons why that might be true:

• Context Switching is reduced significantly, as the team focuses on 1-2 items at a time. Technical team leaders can get much more involved in the implementation, because they’re not juggling the need to support every story simultaneously.
• The learning curve is less steep, and everyone learns faster because they learn together.
• Because everyone learns together, the risk of over-specialization is mitigated. Why? Because more team members know more stuff, and the team becomes more capable and flexible.
• Internal bottlenecks and dependencies evaporate. You might not need internal code reviews, for example, because pairing up or mob developing is self-reviewing.
• The end-of-sprint test crunch goes away. Correspondingly, the risk of several not-really-done stories at the end of a sprint goes away.
• A constant stream of “done” work imparts a better feeling of progress by stakeholders, which can lead to their increased trust in the team.
• Sprints will be more focused, and the team will probably be able to do smaller, more frequent releases, as they can work on one big thing (epic) at a time.
• More broadly knowledgeable team members means a better understanding of the big picture, which leads to a more product-oriented team.
• Daily stand-ups can be improved, becoming the mini-planning meetings they are intended to be, instead of the status meetings they’ve become. Application of practices like swarming or mobbing keeps the team in constant contact and focused.
• Story points may no longer be needed. Story counting and/or cycle time and work-in-progress measurements can be used in place of points. This might save time in refinement sessions and might even yield a reduction in Estimation Dysfunction (a topic for a future blog post).

Challenges with a Flow-Based Approach

To be fair, a flow-based approach isn’t perfect; risks and challenges do exist, including:

• Some will protest the idea of pairing, swarming, or mobbing. You risk losing people who don’t want to work this way or who won’t do well working this way. The team chemistry needs to be very good, and teams run the risk of fatigue caused by frequent face-to-face remote discussions.
• Without the pressure of a sprint deadline, or (assuming you keep sprints) if you fail to monitor your progress against plan and/or fail to take reduced cycle-time or work-in-progress seriously, teams may tend to spend too much time on each story, leaving one or more stories unstarted or unfinished.
• Because unanticipated blockers don’t show up until stories are started, doing stories as a team one at a time in a sprint risks more stories not getting done because blockers aren’t identified soon enough.
• External dependencies could be a problem for minimizing work-in-progress, causing whole-team context switching as they put stories “on hold” and switch back and forth as dependencies get resolved.
• It requires all team members’ work schedules to significantly overlap. A team can’t work together if they don’t work the same hours.

Addressing Common Misconceptions About a Flow-Based Approach

In spite of these challenges, I remain a strong advocate for a flow-based approach – especially in the field of software product development. So in closing, it’s important to correct some of the common misconceptions about Flow Efficiency –

• For remote teams, the belief that everybody is on a call all day with the entire team. Teams can design an approach where this is not the case.
• The opportunity for independent work does not exist. Again, you can design an approach where this is not the case.
• There is no opportunity for specialization. This is definitely not the case. We want to get rid of over-specialization, but we need specialists.
• Adopting a flow mindset means abandoning resource efficiency completely. This is inaccurate; it means only that you prioritize flow efficiency over resource efficiency.

Resources

A case study of a team that switched to a flow-based approach.

And a video from Johanna Rothman that is very approachable to this topic.

TED Talk on the Efficiency Paradox (not specific to software, but highly applicable).

Evidence of the high cost of context switching:

Forbes

Psychology Today

UC Irvine study

American Psychological Association

Harvard Business Review

USC

Context-switching waste is easily visualized in a short video by Henrik Kniberg.

Regarding over-specialization, read up on the concept of Generalizing Specialist.

EFFICIENT PROCESSES AND WAYS OF WORKING.

ITX is ready to partner with your team to expand the markets you serve. Learn more.

---