“What’s the secret to backlog refinement?”
Eighteen pairs of eyes turned to look at me, waiting for my answer to this product manager’s question. I’d spent the last two days with the group working through the challenges they faced using Scrum in their company. We discovered that most of their delivery problems stemmed from the teams not understanding what was needed. They’d identified with the story I’d told about the team that hated Sprint Planning and hit the reset button on their process. They knew they weren’t doing refinement and could see the effect. They wanted to know how to make it work for them.
“The secret to refinement,” I said, “is doing it.”
They laughed – which is what I was hoping for. They realized that they were potentially going to let the perfect be the enemy of the good – that they wouldn’t start if they weren’t sure how to do it “right.” Frankly, anything they did would be better than what they were (not) doing now. After the laughter died down, I shared with them five tips to avoid common problems with backlog refinement.
The Purpose of Refinement
Before I shared my advice, I wanted to be clear about what they were trying to accomplish with their refinement activities. After all, they didn’t need more conversations and meetings just to have them. They needed to have them to make their planning process better. Every agile team that I’ve been part of since 2008 has been able to plan a two-week iteration in under 90 minutes. They could do this because they spent time continuously looking at upcoming work, understanding its value, and breaking it down into smaller, more precise pieces. This refinement process helps them understand the work well enough to have a reasonable degree of confidence that they can complete it within an agreed-upon window of time. When that is true, planning becomes a matter of deciding what work to start and figuring out how to finish it.
Some organizations seem to believe that refinement (which they call often call “requirements gathering” or “story writing”) is something that a product manager – or sometimes a business analyst or domain expert – does on their own. It’s not. It’s a team-level activity that, when done well, makes planning sessions relaxed, useful, and productive. As I mentioned before, the first step is ensuring that the team sets aside time to do it. Beyond that, there are five principles that I’ve seen teams use to get the most out of refinement:
- Focus on Outcomes First, Then Activities
- Include the Right Perspectives
- Prioritize Shared Understanding Over Sizing
- Stay at the Right Level of Detail
- Perform Just-In-Time Refinement
Tip #1: Focus on Outcomes First, Then Activities
When I ask about a team’s refinement activities, I sometimes hear, “Yes, we break down all of our work into tasks.” Task breakdown is often helpful. What worries me about this answer is that these tasks are often divorced from the intended outcome.
Joshua Sieden, in Outcomes over Outputs explains,
An outcome is a change in human behavior that drives business results. Outcomes have nothing to do with making stuff – though they sometimes are created by making the right stuff. Instead, outcomes are changes in customer, user, employee behavior that lead to good things for your company, your organization, or whomever is the focus of your work.”
Outcomes are why the team is doing the work, and they must be kept front and center. Refinement helps a team understand what customer problem you are trying to solve and why.
Task breakdown focuses a team at the level of activities. This is a useful place to end up, but if a team starts there, they can easily come up with tasks unrelated to the results you need. Instead, I recommend working backward from outcomes through outputs to activities. During refinement, start with questions like “What does the user need?” and “What do we want them to be doing differently than what they are doing now (and how could we know)?” Then move on to “What might we need to build in order to fulfill that need?” or “What would we need to create in order to get them to do to that?” These descriptions should describe the qualities of a solution rather than specific technologies or interfaces. Once the team is clear about the problem and the potential solution space, you can be reasonably sure that the tasks and activities the team comes up with align with your desired outcome.
NOTE: When you ask questions like these, you may discover that all sense of who the user is and what they need is lost before the work gets to the team. This often happens when a team is given requirements to implement rather than problems to solve. Dealing with that challenge is beyond the scope of this article, but be aware that you may run into it.
Tip #2: Include the Right Perspectives
Effective refinement needs the right people involved. Who are the right people? Whoever can provide enough of the perspectives necessary to deliver value to your customers and users. You need to know what the customer and user needs are. You need to know the business value of the work. You must know how to design, build, test, and deploy things in your domain. And you need to know any other constraints on your solution, such as legal, regulatory, or budgetary concerns. The more of these that are not known or considered during the refinement, the more likely the team will encounter unpleasant surprises while carrying out the work. The more complex your product and process are, the more perspectives to include.
Teams sometimes fall into the trap of thinking they must include a person from every department in every refinement discussion. Not every perspective needs a person. Often, one person can bring multiple perspectives. Representing perspectives from people who aren’t present is a crucial skill for effective teams. This is where having a robust Definition of Done can help. Suppose your Definition of Done includes “End-user documentation is updated to incorporate any user-visible changes” as a part of each feature. In that case, you might need to include a technical writer in the conversation. If you don’t have a tech writer on the team, then at least one team member needs to know what it will take to update the documentation. The Definition of Done clarifies what perspectives to include to ensure the work is well-understood.
Tip #3: Prioritize Shared Understanding Over Sizing
Another answer I sometimes get when I ask about refinement is, “Yes, we estimate all of our stories.” There are two problems buried in this answer. First, it tells me that the team emphasizes estimating how big a piece of work is rather than “right-sizing” it, i.e., breaking it down into the ideal size for moving through their system. The most productive teams I’ve worked with all have an upper limit on how big a piece of work can be. If they need to work on something that appears larger, they carve off a coherent chunk smaller than their limit.
The second thing that “we estimate all of our stories” tells me is that the emphasis is on the sizing, not on understanding the work. When the team comes up with a number they “agree on” (spoilers: they probably don’t actually agree), they can move on to the next thing. The goal of refinement is for the people doing the work to understand what is needed well enough to start it. Having an estimate is no guarantee that they have that understanding.
One way to prevent this problem is to ask the team, “Could someone explain what must be done to complete this work? Not just what you would work on, but everything the team would need to do to deliver this.” Once you get an answer, ask the rest of the team, “Who has a different understanding of what we would need to do?” Checking for shared understanding highlights where you don’t agree on what needs to be done and avoids the problem of thinking that you do.
Tip #4: Stay at the Right Level of Detail
It’s essential to have a high-level understanding of how to complete a piece of work. Where some teams get stuck, however, is trying to define – in great detail – everything that needs to happen before starting. When teams do this, they’re trying to use specifications in place of collaboration. This type of handoff causes a remarkable amount of waste in the name of “efficiency.” Worse, the team may discover they were wrong about what tasks were needed. Without higher-level guidance, it’s hard for them to recover.
I’ve helped teams avoid this problem with Acceptance Criteria and “good enough to get started” tasks. Acceptance Criteria are how you turn your understanding of what the user needs from this work into a way the team can know they’ve achieved it. They answer, “What would this look like when it’s done?” There should be multiple ways that the team could satisfy these criteria. When this is true, it means the team isn’t tied to a specific implementation. When the team discovers what they were planning won’t work, they can find another way to meet the criteria.
Another good practice for avoiding the trap of too much detail is creating tasks that are just detailed enough to get started. Many of the software teams I’ve worked with have included UX designers. Most front-end implementation tasks on those teams never had wireframes or high-fidelity mockups to code. They didn’t specify what windows and workflows would look like upfront. Instead, they relied on software engineers and UX designers pairing to implement the best solution to the problem. During refinement, they focused instead on describing the critical details the solution needed to include or address. Once they had described the work well enough for them to get started – knowing they had the support they needed to figure out the rest as they worked on it – they were done with refinement.
Tip #5: Perform Just-In-Time Refinement
Refinement has an expiration date – it spoils quickly. The longer the gap in time between when a team refines some work and when it starts, the less value they get out of refinement. The shared understanding created by refinement is ephemeral and quickly forgotten, no matter how much you try to write it down. People forget. They work on other things that push that knowledge out of their heads. The further in advance of the work they do refinement, the more likely the team will need re-learn what they learned.
Another reason to refrain from refining too far in advance is to have the best chance of being right. Most of the work that a team does is connected. It might be on the same project, in the same product, or using the same technology. Things the team learns today will help them with tomorrow’s work. Someone once told me, “Today is the dumbest day of the rest of your project.” Every day after today you will know more about your work than you do today. The longer you wait to refine, the more you know about the work.
A third reason for not refining everything in your backlog to the point that it’s ready to pick up is that work gets preempted. Priorities change. Issues come in. Just because a team has refined piece of work doesn’t mean they’re going to do it. All refinement is potential waste. Doing refinement close in time to when the work will start reduces the risk of wasted time and energy.
I’ve seen teams deal with these issues successfully by understanding their flow and doing refinement “just-in-time.” One Scrum team I worked with looked at the rate at which they completed work from their backlog. They used this to forecast how many sprints away a piece of work should be. Let’s say they completed ten items per sprint. In theory, the top ten items in their backlog would get scheduled for the next sprint, the eleventh through the twentieth following, and the twenty-first through the thirtieth in the third. There was some variation in these numbers from sprint to sprint, but this approach worked well enough.
Next, they looked at the probability that something scheduled for one, two, or three sprints would get worked on that number of sprints out. Their data showed that a piece of work within the top ten items of their backlog would start in the following sprint 90% of the time. Meanwhile, something in the next ten would only begin in the second sprint 50% of the time; the rest of the time, it would get pre-empted or deprioritized, or the work in front of it would take longer than expected. Work items in the third set of ten had only a 15% chance of starting within three sprints – and a 30% chance of never getting started.
Based on this data, the team decided to limit their refinement work to the top 15 items in their backlog. This buffer gave them between one and two sprints worth of work ready to pick up at any given time. They never ran out of work to pick up, and they only rarely refined anything that didn’t start within three sprints.
Pulling It Together
Almost every agile team I’ve worked with over the last ten years has struggled with one or more of these aspects of backlog refinement. It’s easy to fall into the trap of either doing not enough refinement or doing too much – or both simultaneously! One team I worked with had spent months breaking down their work fifteen or more Sprints in the future before they started, only to discover that they hadn’t thought about the impact of the work on operations.
If your team struggles to refine work well, consider using these five principles as a diagnostic lens. For each, you can ask the team, “How often do we do this? Always? Most of the time? Usually? Sometimes? Hardly Ever? Never?” Expect to you get different answers from different people. One team might say that you always stay at the right level of detail, while another thinks you only sometimes do. Explore the differences in people’s answers. See how you might apply one or more of these principles to improve your refinement process. “Perfect” refinement is not the goal. Good enough refinement is.
