...
...
Outcomes over Output
An outcome is a measurable change in user behavior.
...
A hierarchy of outcomes allows a team to connect their team’s outcomes to an overall business impact: business outcome, product outcomes, traction metrics.
Example
A. Business outcome: reduce time-to-market, via developer efficiency
The average amount of time for a developer to start a task to merge the changes into version control is reduced by 30%. (Developer value stream prior to SRE’s Accelerate metrics.) |
B. Product outcomesoutcome: reduce developer context-switching while working on a developer task
The average number of times that a developer context-switches to another task on a daily basis is reduced by 30%. |
...
The 95% duration that a developer waits on the most frequently used command in their local development is reduced by 30%. |
When developers have a question on local development, they discover the answer 3x faster than before. |
Developers run 3x more local development commands in a week. |
Developers post 3x less frequently in the devstack-questions slack channel. |
Product
...
outcome
Product outcomesoutcome:
connect connects to the business outcome
connect connects to the human behavioral change, and to a moment in time of the user’s journey
must be a metric that the team is able to move on their own
are is negotiated with the leader - are they is it aggressive enough?
...
Anti-Patterns
🚫 Anti-pattern: Feature Factory
...
product and engineering teams stopped talking to users
they don’t measure whether what they delivered (features) had the desired impact
they don’t measure at all
they measure the wrong thing
They are stuck in a feature factory, just “building stuff” and churning out features — and hoping that someone will use it.
...
🚫 Anti-pattern: Metrics that don’t connect to business outcomes
The following measures, by themselves, tell us very little about the business impact of the work:. They need to be connected to the higher-level business outcome.
Feature metrics: Performance of a feature (How many people are using it? Is it quicker or easier for a user to do X now than it was before?)
Product metrics: Performance of specific product objectives (Because of feature X, more people are using functionality Y)
Team metrics: How many story points/features were launched at the end of the month/quarter? (This will only lead to — at best — success theatre, and at worse, the wrong behaviors from all parties)
...
Continuous Learning via User-centricity & User-first
Using data that is regularly gathered from users, we build user insights, which inform our hypotheses.
Being user-first, we ask questions and actively listen in order to form hypotheses, without making assumptions and jumping to conclusions/solutions on behalf of our users.
Being user-centric, we keep the voice and perspective of the user in mind as we prioritize, design, develop, validate → and continuously learn and iterate.
Opportunity trees and Product Katas (see below) are tools to enable continuous learning and refining of the user-problem space. We continue to discover opportunities through regular user touchpoints.
To ensure user-centricity, opportunities are phrased from the user’s point of view. Tie the opportunity to a specific moment in time.
Opportunities = user needs, user pain points, user desires, or user questions, at a moment in time.
...
Squad Demos (/wiki/spaces/AT/pages/1742471458, sre)
Process & Artifacts
...
This section lists standard ways for our teams to explore the problem space, prioritize, externalize, and iterate.
| Note |
|---|
While the following steps are listed linearly, they are intended to be developed and updated continuously with deeper customer insights through each iteration. |
🔍 1. MAP: User Journey
Starting with the user journey gives a high-level understanding of our user’s needs and a user-centric perspective of the high-level touchpoints with the products and services that we provide.
Arch example (link)
...
SRE example (link)
...
Idea-1: Products, Tools, and Services offered by the team can then be mapped to which part of the user journey each address.
Idea-2: Later (step 5 below), the team’s roadmap can be externalized by mapping upcoming work to each part of this user journey.
📈 2. DISCOVER: User Insights
Develop an understanding of the problem space by gaining user insights:
Go Wide
Go Deep
Interview users and identify common issues and patterns (/wiki/spaces/AT/pages/2289173034)
Use UX guidelines to prevent biasing interviewees.
Focus on gaining a deeper understanding of the user’s needs; avoid exploring solutions.
No need to interview more than 5 diverse users (link)
Go Observe
Implement telemetry to gather ongoing quantitative data (example-1, example-2)
Note: Of all the above, observations are likely the most reliable since what people do differs from what they say and remember.
However, while observations provide more accurate information on what is happening, “going deep” with user interviews can explain why it is happening.
Find and analyze pre-existing channels of activity that can provide any useful information. For example, analyzing user-generated tickets and user-generated questions in slack (example).
3. IDEATE: Opportunity Tree
Here are some benefits to using an opportunity tree to explore the problem space:
...
Arch example (link)
...
SRE example (link)
...
🕰 4. PRIORITIZE: HPS & RICE & CD3
In order to prioritize amongst the various opportunities ideated, we use a structured template to further define and calculate the value and cost of each opportunity under consideration. The template has the following components:
...
Weekly Impact = (Weekly frequency of issue) * (# of hours lost each time) Cost-of-delay (hours lost weekly) = (Reach of developers) * (Weekly Impact) CD3 = Cost-of-delay / Effort |
| Warning |
|---|
TODO: Create a reusable template based on Arch and SRE’s sheets. |
👩🎓 5. CONTINUOUSLY LEARN: Kata Table
“The process is about progressively solving problems, identifying the next step, and expanding understanding. Once there is sufficient understanding to anchor knowledge and take the next step, do so. Step and repeat. This framework will deepen your user knowledge and expose a host of opportunities and solutions to impact the business objective.” [reference]
...
See how this team went from 85% of users not activating to 38% of users not activating, via iterations of learning. This is well externalized via this “Product Kata” table.
...
🗓 6. SET EXPECTATIONS: Roadmap
Provide a concise way to externalize the team’s roadmap. Rather than focusing on the date of delivery, include the following:
...