Backlog Refinement Meeting

Note:  This post was originally published on January 15. 2014, on AgileAtlas.org, a site of Scrum Alliance.  This site is now defunct.  Re-posting the article here.

Update: Sept 2018 – The agile atlas article can now be found on the Scrum Alliance site at this link: Backlog Refinement Meeting

Authors: Aashish Vaidya & Dorothy Murray

In Agile development, we engage in adaptive planning based on two of the 12 Agile Manifesto principles of “Welcome changing requirements, even late in development”; and “Regular adaptation to changing circumstance”.

Product Backlog Refinement is a core adaptive planning activity in Scrum.  Agile Planning is an ongoing activity throughout the project, not just one-time upfront activity done at the start of the project.  It is generally assumed that most team will have a Product Backlog that is not well defined, large and not finely grained.  The Product Backlog also doesn’t remain static, as new ideas flow in, as existing product needs are refined, new discoveries are made, and as priorities and business need change throughout the project timeframe.

Inadequately refined Product Backlog can result into many team issues such as:

• Performing refinement during Sprint Planning leads to long, contentious meeting that blows past meeting time-box.
• Sprinting on stories with great deal of ambiguity leads to unfinished stories and results into volatile team velocity
• Work  is done on stories that have lesser business value, or are of lower priority
• Delivery of user stories fails to meet desired functionality or fulfill stakeholder needs
• Change of scope or priority during in-progress Sprint

Constant and incremental Backlog Refinement becomes a highly desired activity that a team must master to become more productive.  For newer and maturing Scrum teams, one effective way of perform Backlog Refinement is to conduct regular Product Backlog Refinement meetings outside of the Sprint Planning meeting.  Many Agilists recommend spending up to 5-10% of team’s time in planning future Sprints and Releases.   Over time, the need for formal meetings recedes as the team matures and is able to refine its backlog constantly and incrementally.

Generally, teams whose Sprint durations are from 2-4 weeks can benefit from conducting weekly meetings during non-Sprint Planning week(s).  For teams with 1-week Sprint duration, it may be more efficient to conduct refinement during Sprint Planning meeting itself.

The attendees to these meetings are those from the Development Team, but other subject matter experts, whose input is required or desired, should be invited as well.  Here are some additional pointers to consider in organizing Backlog Refinement Meetings:

• ScrumMaster, in consultation with the Product Owner, organizes and facilitates the Refinement Meeting during non-Sprint Planning weeks.  We recommend the Refinement Meeting should be 1-2 hours per session.
• Teams that don’t regularly refine their Backlogs should consider additional meeting time and higher meeting frequency.
• Product Owner brings an ordered list of prospective stories for an upcoming Sprint.  Team reviews the list of stories, asks the product owner clarifying questions, refines the stories, acceptance criteria, and estimates un-estimated stories.  The goal is to have an ordered list of stories that are examined with the INVEST criteria and can be set to Ready to be worked on during the upcoming Sprint.

Outside of the regularly scheduled weekly Refinement meetings, a Team may also decide to have special, extended sessions to plan for subsequent Release or for some other type of mid planning horizon that extends beyond a single Sprint.  The rationale here is that it allows the Team and the Product Owner to refine a larger portion of their Product Backlog, and set up a workable Release roadmap.

A team can spend their time engaged in these activities:

• Revising larger or coarsely defined stories into finer grain, well-defined stories for subsequent Sprints.  Consider merging other Product items together for better delivery preparation.
• The Product Owner and the team engage in pruning and removing Backlog items that are no longer needed.
• The Product Owner and the team add items that are newly identified and refine them and determine whether those are needed during upcoming Sprint or can be road-mapped into future Sprints.

The Scrum Core framework only asks that Backlog Refinement activity takes place.  But engaging in planned refinement meetings between Sprint and Release Planning sessions is one practical way for agile teams to achieve adaptive planning, and deliver on Sprint and Release objectives.

Are HiPPOs Hijacking Your Team?!

(image from geograph.org.uk)

In one of my previous post titled You Might Not Be Agile If , I outlined some agile anti-patterns.  Recently, I come across couple of anti-patterns that make good fodder for discussion.  Both these patterns are something a Scrum Master should be able to avoid, corrrect.

Anti-Pattern: Only using partial team to size the backlog

 

The first one is where a team used only developers in sizing stories.  Well, is it any wonder then, that the team discovered itself in bind? Testing tasks turned out to be too large.  Surprised? The sprint committments got  jeopardized?  And is it suprising that team members ended up putting in extra hours to try and salvage the sprint?  In this particular instance, the reason was:  some team member felt the sprint planning sessions were getting too long.  So a select few team members pre-sized the backlog and tasked out the stories ahead of the planning meeting.

 

Now, there is a reason why we strive to construct cross-functional teams.  And we prefer using a consensus based estimation process such as planning poker.  We want to avoid, what some quant-driven companies have started calling –  HiPPOs hijacking and monopolizing the decision making process.  HiPPO stands for highly paid person’s opinion or alternatively, a highly positional person’s opinion. 

 

If a team lead, a Scrum Master, a Product Owner, a single team member, or a functional group “Hippos” the team, then that stunts the team from self-organizing.  You are back to command-and-control decision making structure and you miss out on the wisdom of the team – not very agile!  The planning poker or similar agile estimation process involves all team members, and it expects that all views and perspectives are allowed to be expressed, shared and discussed.  Two good things come out of the process, one is that you get a team members’ committment, not just compliance.  The other is that you get a much better and realistic view of your work, and you have intrinsically motivated agile team that owns the estimate and will do everything in its power to deliver on it.

 

But aren’t agile teams suppose to move fast?  Time-boxes are of course very important, but so is discipline.  If you don’t involved all team members, or if you short-change the consensus building process, you get uneven results.  What’s more involving the team allows it to gain tacit knowledge through meetings and conversations, which makes them better at getting things done.   

 

At the same time, agile teams are not deliberating, debating bodies.  They exist to produce great working software.  Reason given here was that the team wanted to have a shorter planning meeting.  So let’s talk about that.

 

The general rule of thumb that has worked for many teams is to allocate 1-2 hours per sprint week for the planning meeting.  Say you have a 3-week sprint, then normally your team should allocate anywhere from 4-6 hours of planning.  To make sprint planning go faster, there are few practices you can follow:

• PO has already communicated her priority stories for the upcoming sprint during grooming sessions, which are taking place during off-planning weeks.  All prioritized stories are well understood and sized by all team members.
• Most agile planning software allow you to copy a story.  So a good time saving practice is to create a “template” story which contained often encountered tasks.  The Scrum Master as part of his sprint planning session preparation can then copy this story, overlay the actual story card and the acceptance criteria.  This approach has another benefit too.  You can embed tasks associated with your story done criteria, which then serves as a good checklist for the team.  For example, if  your team forgets peer code review task on every story, then having an explicit task  as part of the template serves as a good reminder for the team to include for every story.

By the way, the team size matters too when it comes to having an efficient sprint planning meeting.  Having a goldilock-sized team of 7 (plus or minus 2), allows consensus building to occur without a huge time sink and makes for faster sprint planning.  I recall, a team that chose to go with 15 team members, their planning meetings used to drag out to almost two full days (14-16 hours) for a two week sprint!  How productive is that?!

 

Anit-Pattern # 2: Committing on more work than what can be delivered.

But is there maybe another deeper anti-pattern as a cause for this team that chose expediency over allocating “right” amount of time for sprint planning.  Could be that, they are trying to shove a whole lot more work than they can realistically take on during a sprint?.  (No time to waste, can’t you see, I have more code to shovel over to production, every day, so I will take any and all shortcuts possible, especially if I can avoid having uncomfortable conversation with my stakeholders!)

 

If that is really the issue, than there are good practices you can follow.  I have found that when doing capacity planning, it is better to start the team’s capacity at 80%.  So, if you have 14 working days (counting 1 sprint day towards demo, retrospective and planning meetings for a 3-week sprint), each team member will have , roughly, 90 working hours (14x8x0.8).  Using 80% capacity means that you do not have to account for tracking all the overhead minutiae such as company or team meetings, emails, water cooler chats, birthday celebrations, or just team members taking time to be helpful, or honing their own skills.  (An aside: this is also another reason, why I think it is a bad idea to use agile lifecycle management software for project time tracking.  It distracts from delivering good software and makes accountants out of team members).

 

Then, you subtract any “planned” PTO time.  Next comes any production support work that your team generally has to respond to.  The percentage varies from team to team.  Using a rolling 3 sprint average to even out the spikes as your starting point and adjusting according to your circumstances, works.  It is also a good practice to task out 1 or 2 more stories, over and above your team velocity.  So in case, you end up using less than your allocated production support capacity, you have a ready story to work on.

 

And finally, subtract any earmarks from your capacity.  For instance, many teams negotiate with their Product Owner and stakeholder, part of their capacity towards fixing defects, or doing test automation, or for code refactoring. 

 

If your team doesn’t account for all these things up front, you are probably operating on an illusion that you are going faster.  In reality, you are probably spending more and more time in remediating code and responding to production emergencies.  If you want to read more on this anti-pattern, check out Mark Lawler’s recent post called Sprint Planning Like It’s 1999. 

 

You say,  if I do all these things, there is no way, we are going to converge towards our project goals?  Maybe.  But, equipped with data, you may have a difficult, but an honest conversation with your stakeholders.  Should quality and delighting customers be sacrificed to meet the deadline, or should other creative solutions be explored?  Should we invest some time in building good software, or should we pay a whole lot more later?