...
An outcome is a measurable change in user behavior.
The goal of any feature you ship to a user should be user-facing feature/deliverable is to change the user’s behavior in a measurable and positive way. What matters is whether the deliverables are impacting user behavior, which in turn impacts product indicators, which in turn impacts business indicators.
User behavior <A> changes from <x> to <y> over timeframe <t>. |
...
The average amount of time for a developer to start a task to merge the changes into version control is reduced from 10h to 7h (by 30%). (Developer value stream prior to SRE’s Accelerate metrics.) |
...
The average number of times that a developer context-switches to another task on a daily basis is reduced from 60 to 42 (by 30%). |
C. Traction metrics: feature usage and performance
The 95% duration that a developer waits on the most frequently used command in their local development is reduced by 30%from 60mns to 30mns (by 50%). |
When developers have a question on local development, they discover the answer 3x faster than beforewithin 2mns instead of 20mns (10x faster). |
Developers run (3x more) local development commands in a week. |
Developers post 3x less frequently in the devstack-questions slack channel. |
...
🚫 Anti-pattern: Push Adoption of unneeded solutions
“If I had an hour to solve a problem, I’d spend 55mns thinking about the problem and 5mns thinking about solutions.” - Einstein
By emphasizing the prioritization of problems instead of the prioritization of technical solutions, teams can focus energy on continual learning of the user’s problem space. This shift in mindset from delivering solutions to solving problems enables the team to discover users' needs, desires, and delights, which can also evolve over time.
...
Opportunities = user needs, user pain points, user desires, or user questions, at a moment in time.
Opportunity Tree Example
...
...
Process & Externalizing Work
Rituals
Org-wide
We will externalize our work as part of the following organization-standard rituals:
Squad Demos (/wiki/spaces/AT/pages/1742471458, sre) are)
With Director(s)
In addition to the above org-wide rituals, work with your engineering director to establish a cadence of regular review of the squad’s process and artifacts. For example, the director may request a weekly touchpoint to review the squad’s WIP and artifacts from steps 3-5 below.
Process & Artifacts
This section lists standard ways for our teams to explore the problem space, prioritize, externalize, and iterate.
...
Starting with the user journey gives a high-level understanding of our user’s needs and a user-centric perspective of the high-level touchpoints with the products and services that we provide.
Cadence | Participants |
|---|---|
Create once | Have an ideation and affinity-mapping (to group into high-level categories) exercise with members of the squad. If possible, invite stakeholders to the session. 🙊 Remind attendees to ideate on a “moment of time” in the user’s journey and to keep it technology-agnostic and not solution-oriented. |
Review and update every 6 months | With members of the squad and stakeholders. |
Arch example (link)
...
SRE example (link)
...
Idea-1: Products, Tools, and Services offered by the team can then be mapped to which the part of the user journey each addressof them addresses.
Idea-2: Later (step 5 below), the team’s roadmap can be externalized by mapping upcoming work to each part of this user journey.
...
Go Wide
Go Deep
Interview users and identify common issues and patterns (/wiki/spaces/AT/pages/2289173034)
Use UX guidelines to prevent biasing interviewees.
Focus on gaining a deeper understanding of the user’s needs; avoid exploring solutions.
No need to interview more than 5 diverse users (link)
Go Observe
Implement telemetry to gather ongoing quantitative data (example-1, example-2)
Note: Of all the above, observations are likely the most reliable since what people do differs from what they say and remember.
However, while observations provide more accurate information on what is happening, “going deep” with user interviews can explain why it is happening.
Find and analyze pre-existing channels of activity that can provide any useful information. For example, analyzing user-generated tickets and user-generated questions in slack (example).
Cadence | Participants |
|---|---|
Initial data gathering | A few members of the squad can timebox an exercise to gather and analyze data from existing resources. There will always be some initial data that can form a basis for hypotheses. 🙊 Remind attendees to focus on the problem space and not ideate on solutions, just yet (that’s step 3). |
Every sprint | See step 5 below on continuously learning with every sprint. |
3. IDEATE: Opportunity Tree
...
Idea: The top-level depth of the opportunity tree can be taken from the developer journey described above. This allows the team to ideate against the desired outcome across each part of the developer journey.
Cadence | Participants |
|---|---|
Any time a problem needs to be explored:
| Members of the squad, along with stakeholders (if possible). 🙊 Remind attendees to first focus on going wide with opportunities before zeroing-in on solutions. 🙊 Remind attendees to phrase opportunities from the user perspective, to ensure user-centricity: user needs, user pain points, user desires, or user questions, at a moment in time. |
Arch example (link)
...
SRE example (link)
...
🕰 4. PRIORITIZE: HPS & RICE & CD3
...
| Warning |
|---|
TODO: Create a reusable template based on Arch and SRE’s sheets. |
Cadence | Participants |
|---|---|
Any time a new epic or reasonably sized story is considered for prioritizing in the squad’s backlog. | Members of the squad can do this together as part of their planning and/or grooming process. 🙊 Remind the squad that anything can be measured. Without validation, you would be prioritizing output and not an outcome. |
👩🎓 5. CONTINUOUSLY LEARN: Kata Table
“The process is about progressively solving problems, identifying the next step, and expanding understanding. Once there is sufficient understanding to anchor knowledge and take the next step, do so. Step and repeat. This framework will deepen your user knowledge and expose a host of opportunities and solutions to impact the business objective.” [reference]
Cadence | Participants |
|---|---|
| The squad’s assumed product manager and engineering manager can determine who and how this is maintained. |
Example
See how this team went from 85% of users not activating to 38% of users not activating, via iterations of learning. This is well externalized via this “Product Kata” table.
...
Columns: Places in the User Journey OR Business outcomes (see examples below)
Rows: Recent, Now, Next, Future
Cadence | Participants |
|---|---|
Every quarter | The squad’s assumed product manager can take the lead of creating and maintaining the roadmap visualization. Assuming the squad participates in the steps above, the roadmap would not be a surprise for them since it would be a result of continuously learning (step 5), ideating (step 3), and understanding user insights (step 2). |
Examples
Mobile squad's roadmap: aligns efforts to business outcomes, which makes it easier to understand themes of investment across outcomes.
FY21 Architecture User Story Roadmap: aligns efforts to user journey and Open edX named releases.