Partners as Maintainers - Proposals
- Xavier Antoviaque
- Régis Behmo
- Juan Camilo Montoya
Goals
This document aims to list the proposals that have been brought up so far, in the responses document, during the 2024-12-13 meeting and during the 2025-01-17 meeting. If you see any proposal missing, don’t hesitate!
The goal is to build a list to base our discussions on for the next meeting, and to eventually vote to decide which one to adopt. To ensure there is no penalty to having multiple proposals that are similar, we would use ranked voting, like for the TOC elections.
To allow for slight variations as well as mix&matching without multiplying the number of proposals, they are broken down in several parts, which would be voted on individually.
Part 1 - Level of commitment
1-1) Partners & Verified Providers should commit to contributing core contributor time (excluding Axim-funded work) based on one of the following criteria:
A percentage of activity (determined in a follow-up question), ensuring proportional effort across providers regardless of size.
A fixed amount (e.g. a specific number of hours or contributions), ensuring equal volume of work per provider within a tier.
1-2) If the criteria is a percentage, it should be:
5%
10%
15%
20%
1-3) If the criteria is a percentage, it should be based on:
the annual income derived from Open edX projects
the workforce hours dedicated to Open edX projects
1-4) If the criteria is a fixed amount, it should use:
A tiering system: Assign gold/silver/bronze tiers based on contributions above a baseline.
Assigned areas of ownership: Long-term focus on specific features or repositories to build expertise
1-5) If a tiering system is used, how should the tiers be structured? (Choose one of the following options for defining contribution levels within a tiering system):
Option A - Bronze: 10h/month, Silver: 50h/month, Gold: 100h/month, Platinum: 200h/month, Diamond: 300h/month
Option B - Bronze: 50h/month, Silver: 100h/month, Gold: 150h/month, Platinum: 200h/month, Diamond: 300h/month
Option C: Redefine Partner tier to 150h/month and Verified tier to 50h/month, maintaining a simplified structure without additional tiers.
Option D: Number of core contributors: Bronze: 1, Silver: 2, Gold: 4, Platinum: 8, Diamond: 16 – with at least 1 medium-sized or 2 small-sized contributions for each quarter per core contributor. A quarterly impact report is sent by each company company & would be public.
Part 2 - How Contributions Are Made
How should partners fulfill their platform contribution commitment?
Core contributor and maintainer time
Financial donations toward project maintenance
A combination of both 1. and 2.
A fixed number of pull requests, bug fixes, or feature enhancements annually
Part 3 - Maintenance
How should repository maintenance be structured?
Maintain at least one repository for every $100K in revenue derived from Open edX related sales initiatives
Maintain at least one repository per core contributor from the Partner/Verified Provider
Maintain at least one repository for every 10h/month contributed.
Note: Larger/complex repositories may count as more than 1 repository at the discretion of Axim
Part 4 - Partner Program Benefits
4-1) What benefits should the Open edX Partner program provide?
Rank the following benefits in order of priority. Drag and drop to reorder them based on your preferences.
Ability to advertise being an “Open edX Partner” to clients and publicly
Leads forwarded from the Open edX website or stakeholder discussions
Transparency in lead attribution, with aggregated reports sharing the distribution of leads (e.g. by region, vendor, or category)
Leads allocated equitably by region, ensuring contributors in various areas receive fair opportunities
Leads allocated by categories, such as pricing structure, company size, or geographic focus.
Renewed focus on the referral program to increase conversion rates and address market-specific challenges (e.g. pricing or demand disparities)
Consultation on decisions that affect partners, convening an assembly or shared mailing list for input before major decisions are made
Priority access to beta features, roadmap discussions, or technical support
Priority access to aggregated market insights, such as annual market size estimates split by hosting, development, instructional design, etc.
Opportunities for collaborative marketing campaigns and inclusion in global Open edX outreach efforts
Exclusive training resources for partner teams, ensuring they stay up-to-date with Open edX features and best practices
More visibility into Axim’s partnerships, encouraging collaboration between partner companies
Transparency regarding internal Axim decisions, with discussions held publicly or at least accessible to Open edX partners
Highlighting smaller companies' contributions on official Open edX marketing channels (e.g. blog, YouTube, etc.)
4-2) Should partners contributing above the baseline (e.g., 15% or 20%) qualify for additional benefits?
Yes
No
4-3) If additional benefits are offered, they should include:
Priority access to beta features or roadmap discussions
Increased visibility on the Open edX website or marketing materials
Recognition in the community (e.g. badges or highlights)
Access to large deal-sized customer contracts
Part 5 - Marketing
Marketing contributions by Partners & Verified Providers should include the following (ranked individually - 1-4):
A specified number of case studies per year on Open edX projects for the website
A specified number of articles or blog posts on Open edX topics
A specified number of client testimonials published on the Open edX website
Adding a specified number of client logos to Open edX marketing materials
The specified number for marketing contributions should be based on:
20% of acquired clients (e.g. 2-5 case studies/testimonials per 10 clients)
A fixed amount (e.g. 1 case study, 1 testimonial per quarter).
Note: For marketing materials that require client approval, such as logos or testimonials, an exception is granted if the clients refuse publication. However, the other criteria would remain as requirements on an anonymous basis for the client.
Part 6 - Commitment Mechanism
6-1) How should commitments and benefits be tied?
Post-moderation: Benefits are removed after unmet contributions
Pre-fulfillment: Partner levels are granted based on prior contributions
6.2) If post-moderation is chosen, how should warnings be structured?
3 warning process with associated deadlines (e.g. first warning: private notification, second warning: stop referrals privately, third warning: removal from the Partner/Verified Provider list)
Immediate removal from the Partner/Verified Provider Program, if the requirements are not established within a reasonable time
I was never a big fan of counting contributions in terms of #hours/month. If tiers were measured in #hours/month, we could jump to the last tier simply by being less efficient, and thus spending more hours on maintenance – and obviously we don’t want to do that.
Core contributors, on the other hand, are a great way to measure contributions. So I think that thresholds should be defined in terms of #CCs. I suggest we define thresholds with powers of 2: 1, 2, 4, 8, 16, 32.
Is there a way to ensure accountability for CCs today? I do not know of anything personally, so I am wondering how this pathway would be less “gaming the system” than hours?
I would say that currently, there are not a lot of accountability systems in place. However, as of this year, we’re starting to build out the framework of an accountability system. We’re starting with annual check-ins with CCs that will begin soon, and I’m hoping that over time we can add on reporting and verification systems to help us better understand who’s doing the work and where.
I’m not concerned by accountability right now. It’s an important topic, but we will have to take care of it whatever KPI we end up with. My concern is not about bad actors gaming the system, but having the right incentives in place. And measuring investment in terms of hours spent is a bad incentive – just like counting #lines of code for a software engineer.
@Régis Behmo We can add a proposal here that includes the number of core contributors as a criteria.
We need to specify a bit further though - what would the providers commit to per core contributor?
Note that one of the recurring issues core contributors have mentioned is the lack of time to contribute. So if we just incentivize companies to have more CC without ensuring that then companies give CC the time to contribute, we will just reinforce the current issue. So even if the criteria is not directly hours, it should be specific enough to guarantee that the core contributors can actually spend a meaningful amount of time doing CC work.
Honestly, I don’t care much for the amount of time spent as long as the task of repo maintenance is done correctly. I get it that maintainers have trouble allocating time to their maintenance tasks, beside of their other, daily responsibilities. But that’s a problem of enforcing ownership, not relying on some arbitrary metric.
Making sure that people do their job well is a difficult topic, and it can’t be boiled down to a single KPI. Someone needs to be the bad cop here and evaluate CCs, report on their impact, discuss with them on how to improve.
Making sure that CCs do their job well should be done just like we do in any other company, and no successful company is going to rely on a single metric like hours of presence/# lines of code written/# PRs merged. It’s a multi-factor evaluation that can’t be boiled down to some automated script in CI. This is an important discussion to have, but I suggest we keep it separate from the current discussion.
@Régis Behmo That is the whole point of the summit proposal - see https://openedx.atlassian.net/wiki/spaces/COMM/pages/4517134415/Open+edX+Partners+As+Maintainers#Overview, it is focused on making sure we provide core contributors with enough resources to do their work, by defining more specifically and precisely the resources the companies commit to provide their core contributors. It has been accepted as a goal, so the proposals we include here need to match the goal approved by the core contributors during the summit.
Just specifying a number of core contributors without giving them the means to do that work would be a step backward compared to the current contractual partner agreement (10% of size) & the commitment most of the companies provide (20h/month/contributor).
Can we collaborate on this point to find a middle ground, so we can include your proposal? Something which is not hours if you don’t like that, but is at least specific enough to incentivize companies beyond just the CC nomination?
Also, note that core contributor work includes maintenance, but not only - there is a lot more than maintenance that counts as core contributor work. Cf https://openedx.atlassian.net/wiki/spaces/COMM/pages/3593502844/Core+Contributor+-+Work+Scope#Definition
I suggest we request quarterly impact reports from partner companies. These reports will provide answers to the following questions for each core contributor:
1) Which tasks did they perform? (as per the work scope)
2) What impact did these tasks actually make on the Open edX project? (minor/medium/major)
3) What are their plans for the next quarter?
It’s important that these reports are made public, such that partner companies can be evaluated fairly by the rest of the community.
If the total impact of a CC does not reach a minimal threshold (which should be fairly low), this CC should be put on review. If the CC again does not reach the required threshold the next quarter, then their CC status should be revoked. The company is then moved to the tier that matches its number of CCs.
@Régis Behmo
I suggest we request quarterly impact reports from partner companies.
Sounds good to add a proposal about the periodicity and the format of reports. That would probably fit into part 6, which deals with the commitment/control mechanism. We had actually mentioned revising this during the meeting, so that would be a good subquestion there.
If the total impact of a CC does not reach a minimal threshold (which should be fairly low)
What would that minimal threshold be? This is what would be important to be more precise about, so that we can know what is actually minimally expected - ie what companies would need to allocate in their budgets/resources.
I think we should have a really low bar for that threshold: maybe 2 minor contributions or 1 medium contribution for each quarter. Making the report public will incentivize companies to have high impact contributors: no one want to be labelled a “minor impact company”.
@Régis Behmo Thank you for detailing it! I will add a proposal for what you have suggested - check that it matches what you have in mind?
Btw, I think with low thresholds we will end up with a significant portion of companies barely matching that minimum per core contributor. Which in turn will disincentivize the ones who do provide significant time or budget for their core contributors from keeping to do so.
But maybe I’m wrong, and in any case it is worth having the group weight all the options. After all, we currently all have an incentive to set a good standard, to ensure the contribution workload is high and evenly shared.
We’ll see!
@Régis Behmo To clarify further the choices in this section, I have broken it down into more questions - in particular, expanding the partt about the tiering system, so that if it is chosen we have clear criteria for what it would be - I have added some answers, but you might want to add your own? I am not sure what tiers levels you had in mind for it, exactly.