[Draft] Definition of Product Characteristics

This is still work on progress

See also [Draft] Product Quality Definition , which takes some of the ideas from this work and builds out a definition of Product Quality, done in collaboration with the Product Working Group in Dec 2022.

Component/Feature - Attributes

Note: I might use component/feature interchangeably, I will stick with the component, essentially I am referring to a building block of our product.

A sample illustrating how product characteristics relate to product features.

Intro

Design is an iterative process, and thus product is expected to be always in a continuous process of change. A change in product is not only in regard of its features but also in regard of its characteristics and this document focus is on the later.

What is the goal of this

To define how to measure product characteristics
To provide a mechanism to measure where a component/feature of the product is on our quality bar.
To detect/predict a weakness or area of improvement in a component(s) of the product.
To suggest a list of attributes which can help to measure the quality of product component(s).

Open Questions:

Data Sources: Making data driven decision is easy, but getting the data might not be that simple for some attributes, mainly usage.

What is our quality bar. Does this document needs to define it, or should it rely on external resources, like other established OEPs, and in case such reference doesn’t, what to do about that…
What what is the smallest unit or building block of the product, and how does it relate or map to the technical product. (While I try to call it component, no matter if we agree about the name/notion it would be important to explicitly define it, and list it… Can it be taken from the mindmap, and if yes then at which level
- Do we need to group them per technical context? To group them per technical core would simplify the measure of attributes (i.e. measuring an attribute of a technical core component would affect all product components that use that technical core component)
In case when different stakeholders have different priorities, or would optimize for different attribute, how can that be resolved?
- For example an enterprise scale provider of Open edX it might not be concern if a change/feature increases maintainability cost in terms of human or/and IT resources, because they can afford that however other providers would find it a blocker.
How does this relate to different use cases (ref Use Cases and MindMaps )
How does this relate to different user bases ( small scale, higher scale…etc)

In order to convert product components into artifacts we propose to create a list of attributes, that would belong to product components. Just as Open edX Proposal #57: Core Product is defining what are the components of the product are, this is defining their attributes.

What do we mean by an Attribute:

An attribute can be though of as a signal, a character of the component, or for the product as a whole. As the product is a composition of its components, its attributes or characteristics are a combination of all its components. Simply put, an attribute can be defined either at the component or level or at the product level.

Characteristics of Attributes

Just as each component product has a collection of attributes, each of its attributes can be defined as its attributes, or meta attribute, those are just

An attribute must be measurable so that we can define its criteria. We should aim for quantitative non-subjective type of data. But it might not be always the case; i.e. what does the UX look like? It might be a subjective opinion; in such cases, we can try to revert to the census agreement or/and to the subject matter expert.

Attributes might conflict with each other, for example optimizing for modularity might affect the performance or/and maintainability.
1. We might try to overcome this by setting the importance of each attribute relative to other attributes.
2. Re-architecture: Can we rebuild it in a way that is try to take the best of each attribute… Is the overall cost worth the outcome?
When developing new features/components it's important to take the previous point into consideration, i.e. we should strive to ensure that adding a new component does not affect attributes of other components in a negative way, unless there is a strong need to. I.e. adding more MFEs improves the modernity of UX but decreases the efficacy/performance of the dev tool.
Some attributes might have well-defined OEP, which is a good thing, in that case, we don’t want to redefine that OEP but rather use it for PoV

Component Attributes

Below are example of suggested list of attribute.

Performance
Modernity
a11y
i18n/l10n
Modularity
Maintainability
Adoption Rate
Usability
Scalability
Developer Friendliness
Can it be deployed and enabled in the stable releases? (cf MFEs)
Is development active on it?
How many different community members depend on it? Is there collaboration on it?
Documentation - does it exist? Is it up to date?
User awareness - do users know about it?

A suggested list of Attributes:

This list is not complete, nor is it accurate, it's still not meant to be consumable yet

Performance/Effieciy
1. Importance: 8/10
2. Goal: To make sure our product performs well, is efficient, and doesn’t consume more resources than it needs to
3. Measured by:
  1. Speed/Core web vitals
4. criteria:
  1. Takes X seconds to load at least.
  2. doesn’t load or request resources that it needs to not use.
Modernity
1. Importance: 8/10
2. Goal: To ensure it looks modern
3. Measured by: This is a subjective signal
A11y: Already has its own OEP
1. Goal: To ensure it complies with our standard of A11y
2. Measured by: Lighthouse, subject matter experts
Modularity
1. Importance: 7/10
2. Goal: So that our product can be extended, without the lowest effort possible.
3. Measured by:
  1. Need to be forked (Boolean)
Maintainability
1. Importance: 7/10
2. Goal: To ensure a particular component doesn’t take too much time and hassle to be maintained. This might be relative, that is a component that provides many functionalities would expect to have high maintenance. On the other hand, if take a relatively high time and is getting complicated that it would be subject to Rearchticure.
3. Measured by: A ratio of how long it takes to maintain it / How impactful is it
Adoption Rate
1. Importance 5/10 Should be
2. Goal: An important metric to influence future decisions; i.e. a component with a low adoption rate but with high maintainability might be subject to a lower tier. On the other hand, a component with an increasing adoption rate might be subjected to a higher tier… However, the adoption rate might be low because it's not useable, or is not discoverable.
3. Measured by:
  1. Learner/Author: Average usage among the instances
  2. Instance: (Total of instances using it / Total Instance)
Usability
1. Importance: 8/10
2. Goal: To ensure it's easy to use, not too complicated,
3. Measured by:
  1. A number of steps are needed to use it.
  2. Does its target audience has the leverage to use it? i.e. Do the course author needs support from the operator?
  3. How easy is it to discover that such a feature exists?
i18n/l10n
1. Importance: 8/10
2. Goal: To ensure a component complies with our minimum
3. Measured by:
  1. Number of languages it supports
  2. Translation completion rate.
  3. How easy is it to add an unsupported language?
  4. Is it customizable in terms of local requirements?
Scalability
1. Importance 6/10
2. Goal: How easy is it to use for different use cases or installation sizes?
3. Measured by: (Is subjective)
Developer Friendliness
1. Importance: 7/10
2. Goal: How easy is it to develop, install/set up, and maintain
Component health/ Product health
1. Goal: Is a meta attribute that can be calculated relative to all other attributes per component, then we can take that index per component to calculate overall product health.
2. Measured by: Come up with a math equation all the above attributes into consideration,
  1. Component health Ch = Sum of (attribute[i] * importance[i]) / Total attributes; where a is attribute
  2. Product health Ph = Sum (compoenet[c] * importance[c] / total components ) ; where c is componenet
    1. I assume some components might be more important than others.

Act: Based on the above signals we would define the status of a component, and we can then have a systematic way of suggesting action items. To ultimately improve the overall healthiness of the product.
Map: We should strive to map the current weak points and pain points of the product to its corresponding attribute, if a common weak/pain point doesn’t have an attribute that measures it, then try to create it.
Prevent: Try to find common attribute deficiencies among components, i.e if a particular attribute is usually low among many components, then why did we fail at this, do we need to change something in our workflow?

Refs

To get overview of product features/components)
Table/Sheet example for components/attributes
Visual chart detailing the relation between product features and characteristics