Is Aiming For Potentially Shippable Good Enough? [Agile Safari]

What if we used accidentally shippable instead of potentially shippable? Would that help us aim higher?

Tweet the Agile Safari Cartoon!

I was working with some new agile teams and in the process of explaining potentially shippable, I said that it should be okay if it accidentally ships. That seemed to freak some people out. I guess using ‘accidentally’ is not a word people tend to like to see. The term seems to get people’s attention as I’ve continued to use it.

So who is right?  Is the pig right? Are we aiming for only “potentially” shippable? Or is the rabbit? Should completed work actually be ready to ship, even “accidentally?”

To answer that, we have to have at least a basic agreement on what all of the various terms related to completing work in a sprint mean. That might seem obvious . . .  but the confusion on all the terms (acceptance criteria, done, definition of done, done-done, potentially shippable) is rampant.

So lets start with a review of the many terms.

Coming to Terms

Teams using Scrum select work they can complete in a sprint (or iteration). I’ll use the term “story” (user story) although there are other types of work a team may have in a sprint. We say that work is supposed to be ‘potentially shippable’ at the end of the sprint. Potentially shippable often turns into a big question mark. To get to potentially shippable, what is required?

Step 1: Acceptance Criteria

The first part of the equation is that for a story to be complete, it needs to pass it’s unique acceptance criteria. So each story has its acceptance criteria…

• In order to [value], As a [role/actor], I want [feature].
  • Acceptance Criteria
    • Given ___, When ___, Then ____.
    • Given ___, When ___, Then ____.
    • Given. . .

Passing the acceptance criteria is step 1. Ideally, the acceptance criteria is turned into automated tests, but for our purposes here, I’ll leave it at that.

Step 2: Definition of Done

Definition of Done is more or less acceptance criteria that apply to everything in the sprint (there are some variations if the team is working on different products). It would not make sense to list all of these acceptance criteria for every story, so using a Definition of Done helps simplify each stories acceptance criteria (to be unique for each story).

Some of the common (high-level) criteria included in a Definition of Done include:

• code is checked in
• tests are created and run
• PO has seen the feature working (yes, before the review at the end of the iteration)
• automated tests are run
• no known defects are in the code
• release documents have been updated
• training materials have been updated (ideally integrated with the application)

People typically get more detailed on a Definition of Done (e.g. what kinds of tests are created and run), but this list serves as a starting point. When you say something is “done” it should abide by all of the items on the Definition of Done (and of course pass the unique acceptance criteria for the story as noted in Step 1).

It’s quite common to hear “we have a Definition of Done, but we are not really using it.” Too common in fact!  Sometimes there are certain items that everyone aspires to doing, but they are not happening or not even possible, yet they still get included in the Definition of Done. Your Definition of Done should represent reality!

If you have criteria on your Definition of Done that are currently impossible (e.g. ‘automated acceptance testing is executed’ — but you don’t have any automated tests), remove the criteria! If you include ‘code is released’ but you can’t release it given your existing constraints, remove it from your Definition of Done! Many ideas may look great in an agile book (or a blog post) and are great to aspire to, but if they are not possible, it simply diminishes the value of your Definition of Done! In fact, you might need to start with two Definitions of Done to reflect your current reality!

Sidebar On ‘Done-Done’: I’m not a fan of done-done. The idea of done-done is that if I ask someone if something is done and they say yes, I then say “is it done-done?” then that often gets me a “no.” I will contend that this does actually happen, but to simply repeat the word because people are not clear on what done means, defeats the purpose of trying to have clear communications. Additionally, I find myself asking, “if this is true, what else is true?” Do I now need to ask “is that ready-ready?” “is it shippable-shippable?”, “released-released?”, “tested-tested?”, “developed-developed?” Try this: if someone says something is done, say “Great! We are shipping it in an hour!”

Some people also say “done” is “sprint-done” and “done-done” is “release-done” (or they may even add a done-done-done!). Just say what you mean, instead of repeating the same word. It is confusing.

Potentially Shippable

Completed Story (Potentially Shippable) = Meets Story Acceptance Criteria + Meets Definition of Done

We finally arrive at potentially shippable! The idea of potentially shippable is supposed to indicate that it COULD ship if it needed to, but that you may choose not to do it, for a variety of reasons. Often, work that the team completes is not shipped (or released), since the business value of those items may not outweigh the cost of releasing only a few items.

Imagine the potential cost of rolling out a few minor new features to thousands of people across the world. This is not to say these types of costs and issues can’t be mitigated with design, integrated training, and smaller rollouts, but there are cases where this is not feasible. Of course the worst-case scenario would be releasing stories that confuse your employees and customers. When renting a car this past summer, there was some confusion behind the counter and the person helping me said to her co-worker, “I can’t modify this option anymore?” Her co-worker then remarked “looks like they made another change.” I asked what happened and she said that they were getting updates to their system all the time without any information about the changes, so they were struggling to figure out the system each day! They were quite good-natured about it, but were also very matter-a-fact, which I see as an indicator that ‘this is the way it is and it won’t change.’  URG!

Ultimately, a completed story or potentially shippable story, meets the story’s unique acceptance criteria and meets the Definition of Done.

Potentially Shippable vs. Shippable

Is potentially shippable actually shippable? Is it accidentally shippable? There seems to be some agreement that potentially shippable and shippable are different, but those conversations seem to be murky to me.  The idea that you might need a “hardening sprint” or some other mechanism to do all the items you did not do in the sprint seems like a setup. My experience is that, while there might be some exceptions where that is needed in the short term, there should be clear intention, transparency, and planning, and action steps around when you would STOP doing a “hardening sprint.”  I’d much rather have a very clear Definition of Done even if it’s not possible for the team to do right now. We need to have an honest and upfront conversation about what we want to define as done in the future and what is actually possible today!

So who is right (back to the pig and rabbit)? The pig certainly has a point in that they clearly are not in agreement on what the terms mean. You have to start there. On the other hand, the rabbit has a point as well in aiming for something more! Both of them need to start by getting on the same page with terms.

Exercise: Ask everyone to write down (silently and individually) their definition of acceptance criteria, definition of done, potentially shippable, and shippable on stickies. Then post all of the definitions on the wall. How close is the group to the same definitions? If you are way off, you need to start with agreeing on the terms. It is important to do this without judgment and with everyone involved.

Accidentally Shippable

Accidentally shippable should cause people to pause and think much harder about what is going on in each sprint, about our definition of done, and about any work we are deferring to some future “hardening sprint.” In fact, it might make you feel downright uncomfortable! To some extent it should. If we are trying to deliver working software, shouldn’t we be aiming for accidentally shippable? Are we simply assuming the code will not go out till the release in 6 weeks so we can just catch things up sometime later? Shouldn’t completed stories really be accidentally shippable if they could be? Why would we not aim for that?

We know that if we have to go back and try to figure out everything we did over the last 6-8 weeks so we can create release documents, training materials, finish up code, or creating user acceptance tests we will be creating a lot of waste. We also defer learning and ROI in many cases. I realize what I’m about to suggest might seem crazy to some, but at what point do we question our assumptions and actually improve?

Try this as a retrospective approach — Ask:

1. I want you to imagine that we configure Jenkins (or whatever is handling your builds or could) to randomly deploy 1 out of every 6 stories that are completed. What are all the reasons that will not work?

2. Identify at least 4 ways you can resolve each of those reasons, if you could do anything.

3. Choose one to resolve in the next sprint (and one organizational impediment to escalate for the next sprint).

What is really constraining your ability to use accidentally shippable?