2021-09 Committers Survey Results

Owned by Sarina Canelake (Do Not Use) (Deactivated)
May 18, 2023
5 min read

Core Contributor Committer Survey (9/2021) Results

A survey was run over the first two weeks of September 2021, and asked the 15 members of the Committer pilot program questions in three categories: Overall Program Satisfaction, Declaration of Commitment to the Program, and Platform Tech (focusing on Boundaries, Extensions, Events, & Standards). Results are summarized in this doc; we did not look at how individuals answered any particular question, but rather at what the broader themes were.

At a glance:

  • 14/15 Committers responded

  • Program NPS of 72

  • 12/14 feel they are sometimes or frequently able to work faster as a result of Committer status

  • 20 hour per month contributions:

    • 4/14 say they do not contribute 20 hours/month

    • The rest say they are sometimes or frequently able to

    • The majority of the 20 hour time is spent writing, reviewing, or merging code; a small handful of hours is additionally spent on working groups, communications, & documentation

    • Average 3-5 OSPR reviews per month

      • A few indicated they also review edX-authored PRs & didn’t count those as OSPRs

Deeper Dive: Overall Program Satisfaction

Question: What did being a Committer enable you to do?

Lots of disparate answers here, but all very positive.

  • Iterate faster by not having to wait weeks on reviews & merges

  • Sense of satisfaction and respect at being granted ownership & being seen as an authority for the repos I have access to

  • More motivated and empowered to help out the broader community; pulled in more frequently to OSPRs (positively)

  • Seen as a partner with the edX.org organization: I have a more direct line of communication with edX & my technical decisions are listened to and respected.

Question: What could be improved about the Program?

Lots of answers here ranging from broad to actionable, critical to gentle. Themes are: Onboarding/Documentation, how edX blocks progress, Committer ownership, and inter-Committer coordination.

  • Onboarding of Committers needs to be smoother, with clearer, advertiseable criteria for joining, clear onboarding steps, training courses that don’t include edX-specific language like “OGSP”, and possibly a Committer-specific course covering topics related to “how to be a good maintainer” (beyond just knowing the codebase)

    • Similarly, how can we do a better job documenting potential edx-platform failure modes? For example, what do I do when e2e tests fail on a branch I merged?

    • How do I know when to merge?

  • Product review is the single biggest blocker. More resourcing/attention there is desperately needed; this continues to be a source of frustration.

    • Similarly, edX’s business needs rather than overarching platform needs seems to be the biggest hindrance to a healthy community

  • Can we narrow down ownership scope of edx-platform, perhaps by djangoapp?

  • How can we increase Committer write/review access to more repos (called out: ecommerce, discovery, analytics, frontend-*, edx-platform).

    • Similarly, there is only 1 Committer on many repos, which makes getting review from edX just as hard as it is without being a Committer

  • Theme: Coordination of Committers with one another

    • Can we somehow expose what all Committers are working on, perhaps a board, so we can coordinate efforts?

    • Can we surface & advertise statistics around how Comitters are contributing?

    • How can Committers communicate more effectively, especially outside of working groups?

    • How can we get Committers more involved in both each other’s work and the community’s (particularly open PRs)?

Question: Any other thoughts (open-ended)?

  • I enjoy the program, and feel that edX’s investment in an Open edX team has greatly improved the overall experience

  • Confusion around why Open edX is open source, when the vast majority of its policies and procedures privilege one organization / “is still by far a one-way upstream project”. Does this discourage potential contributors?

  • Are we still in the pilot phase of the Committer program? When will we move forward with the suggestions/phases brought up previously?

  • I really enjoy being a community liaison, that community members who don’t have a direct line into edX come to me as an expert.

Deeper Dive: Declaration of Commitment to the Program

Question: if you review OSPRs, how do you find out about them? If you don’t, why not?

  • (11x) I get tagged by the author (often members of my own team or community members - primarily, BTR members) or the Open edX team

  • (4x) I watch the PRs that are opened in the repos I have write access to

  • (2x) I look at the Jira board; often I do not find unassigned tickets tagged for CC review

  • (1x) I have scaled back my involvement in some repos post-Blended project, primarily because I am no longer tagged for review.

  • (1x) I really struggle with a lack of time to get to my 20h/month

Deeper Dive: Platform Tech

Question: Which aspects of BEES (Boundaries, Extensions, Events, Standards) are working for you?

  • (8x) I do not know what this means/what the question is asking/no answer

  • Not sure if this abstract notion trickles down to day-to-day work

  • Boundaries: Splitting the monolith into smaller components/repos. Proposals around extracting things, better separation between LMS & Studio.

  • Extensions: Open edX plugins (and looking forward to the hook events to make this extension more capable), LTI, backend Django plugins are working really well

  • Standards: Generally a strong suit of the platform - drives quality.

    • Embracing LTI as a standard to follow

    • (3x) DEPR/ADRs/OEPs

Question: Which aspects of BEES (Boundaries, Extensions, Events, Standards) need improvements?

  • (8x) don’t know/no answer/”BEES” is not referenced anywhere that I can find, either in documentation or PR descriptions/comments.

  • Really doesn’t apply to PRs I create or review.

  • Boundaries - Not always sure why certain boundaries were put in place the way they were. There’s some ambiguity around where boundaries should/shouldn’t be. Small components in their own repos don’t adequately provide READMEs to explain why they exist in the way that they do.

  • Extensions - What JS & Python APIs exist and how stable are they?; no elaboration

  • Events - Confusion/lack of knowledge about event bus; no elaboration

  • Standards - Lack of knowledge around what standards exist (ex: Caliper Analytics). Lack of consolidated documentation, particularly “how things work”, what standards are being followed, and what ADRs have been added/changed (“ADR changelog” as a recommendation)

Question: Which part(s) of the platform would you like to be more flexible and extensible? What would that enable you to do/create?

  • Overall: Feeling that the platform has not been designed with extensibility in mind, and extensibility as an afterthought/in response to community complaints does not feel great

  • Theming & overall UI - many responses

    • Theming needs to be easier, more dynamic, & broad in scope

    • It’s not clear how to customize/override MFEs as we could do with templates

  • Ability to write frontend plugins in MFEs

  • Stable platform core with a solid API layer that enables Open edX consumers to build IDA services without touching the core

  • Removing edX-specific code everywhere, specifically removing tightly-coupled code that’s only used on edX.org

  • “I would like to see more extensibility and flexibility in the way the 5 primary interactions (register/login/enrolling/grading/giving_certs) work.”

  • Scalable approaches to codejail, error logging

  • XBlocks