Open

Why

A technology radar serves as a reference and an onboarding tool for the technologies used in the Open edX Platform, giving us all deeper insight into the composition and history of the software which we collectively steward. It’s a living document that records our technology decisions as they change over time, evaluating their level of adoption and suitability to the platform’s needs.

Today we don’t have a “one-stop shop” for this sort of information, and often teams and individuals are left to reinvent the wheel when making technology choices. We hope that creating a comprehensive Technology Radar for Open edX will save engineers time and make our code more approachable to each other through consistency of approach and implementation.

Creating a technology radar involves gathering information from all corners of the engineering organization, so it only makes sense to do this in a collaborative way.

See Thoughtworks Tech Radar FAQ for more information.

Tech Radar Process

Step 1: Learning

In this step, we learned what a tech radar is and how to apply the concept to Open edX. We took an initial stab at what our quadrants and rings might be, and thought about what sort of blips belong on the radar, and what don’t. We also came up with a plan for how we could collaboratively build the radar with the community.

Step 2: Ideation (Workshop Step)

Each participant individually brainstormed blips based on their areas of expertise. The goal was to come up with a corpus of technologies that might become blips.

Open

Step 3: Analysis (Workshop Step)

Participants gathered in breakout rooms by expertise area: Architecture, Backend, Frontend, and SRE / DevOps. We also had breakout rooms for Data and “Rebels”, but they were unused during the workshop for lack of participants.

Step 4: Retrospective (Workshop Step)

We paused after our analysis to reflect on the process so far. This included feedback on the workshop itself, the emerging shape of our radar, how we should categorize blips, and thoughts on how we should proceed for the next steps.

Step 5: Quadrants (Miro Board Step)

In this step we looked at the output of Step 3 (Analysis) and categorized technologies into quadrants using our learnings and predictions from Step 1 (Learning) to guide us. We confirmed - based on the analysis in the workshop - that having quadrants of Techniques and Open edX Technologies made sense for our Tech Radar. The former is also present on Thoughtworks' Radar, and the latter makes sense given the number of Open edX-specific technologies in use in our platform.

We also decided that we had enough technologies in the frontend development ecosystem, and that they were distinct enough from our other technologies, to justify having a Frontend quadrant. Finally, all other technologies effectively went into the Technologies quadrant. The majority are backend oriented, but there are also a number of services and operational/process technologies in this space.

Step 6: Consolidation (Miro Board Step)

In this step, we refined what potential blips should be included in the radar. We came up with a set of criteria by which we could combine some blips and remove others, with the goal of having roughly 25-50 blips in each quadrant.

We also made some decisions around the shape of the radar and the next steps in the process.

Approaches

1. Combine

   1. Combine listing of tools into the single 'technique' that they support.

   2. Combine or exclude blips that only a small percentage of engineers need to know about. For example, engineers are consumers of all the coverage and linting tools. They don't need to know about each one we use and don't use. They can be listed within a description, if needed.

   3. Combine blips where one is an important subset of the other, or a sub-technology integral to the adoption of the other.

2. Remove

   1. Remove blips that are obvious and can be taken for granted. (eg Django).

   2. Remove blips that are not used in the platform at all right now. If the idea is a consideration for the future, have it first be prototyped with a provisional OEP. Exception: If it's a commonly asked question of our opinion on it and we've made a decision against it, consider including it.

3. Move

   1. Move blips that are abstracted away by cookie-cutters or Open edX APIs (since those choices are made for developers) to the 2nd tier Radar.

   2. Move blips that developers need not know about within their first 30 days to the 2nd tier Radar.

Decisions

• Tiers of Radars.

  • 1st tier includes information that we expect developers will want to know within 1st 30 days.

  • 2nd tier includes additional decisions to learn post-30 days.

  • 3rd tier includes edX.org-specific decisions.

• Rings. Align name of rings with status of OEPs. We want to update OEP-1 to remove the "Final" state and clarify the difference between Provisional and Accepted. We also want to add a description for the "Adopted" state we propose using in the Tech Radar.

• Descriptions Next. We'll write blip descriptions first, then use that additional context to determine what ring it belongs in.

• Quadrants. Our quadrants will differ from those used by Thoughtworks. One of our quadrants should be "Open edX Technologies" and include technologies specific to the Open edX ecosystem. The "Techniques" quadrant from Thoughtworks radar feels appropriate for ours, so we will use it. The remaining two quadrants currently differentiate between "Frontend" specific technologies and “Technologies” in general, reflecting the natural divisions in our code-base.

We then populated a spreadsheet with all of our blips and set the quadrants appropriately, and the rings to “Hold” pending Step 8. https://docs.google.com/spreadsheets/d/1ntg2fy7EBR0TFGktyORyv3W-K1bOmhr5Z4EU6WzdSWE/edit#gid=0

Step 7: Descriptions (Spreadsheet)

The goal of this process is to generate descriptions for all of the blips on the radar. We believe that we will more accurately categorize blips into rings equipped with descriptions of what each blip is, and what it means - qualitatively - to the Open edX platform. We intend to do this collaboratively in the Tech Radar spreadsheet. Participants can sign up to be an Author or Reviewer of a blip description.

Step 8: Rings

Once we had descriptions of our blips, we categorized them into rings by their level of adoption. This was done by a small group for expediency, and then submitted for review from a broader audience to catch any discrepancies.

Step 9: Publish!

We want to share our Technology Radar with the broader Open edX community to help onboard and educate everyone about our technology stack. As part of our broader information architecture goals, we expect to be prominently linking to the Tech Radar from our documentation. New engineers should be familiar with the technologies in the “Primary Radar” within their first 30 days.

➡️ Step 10: Iterate

Once we’ve published the radar, that’s not the end. As we all know, technology continues to change constantly, and our Tech Radar will be both the vehicle for how we adapt to changes in technology, as well as our outward expression of those decisions.

Definitions

Blips

A blip is a technology or technique that plays a role in software development. Blips are things that are ‘in motion’ - that is we find their position in the Radar is changing - usually indicating that we’re finding increasing confidence in them as they move through the rings.

Our definition of what constitutes a blip may be slightly different in spirit than Thoughtworks' definition. We want our radar to represent a historical record of technologies used as part of Open edX. Since engineers may find references to technologies in our codebase years later, it makes some sense that our blips may be more long-lived than Thoughtworks'.

Rings

Our rings are intended to correspond to OEP states. The “Adopted” state is a new addition identifying those technologies that are not only Accepted - a comparatively low bar for an OEP - but thriving and recommended. The intention is to update OEP-1: OEP Purpose and Guidelines to adjust the states to map onto the rings in our radar.

Quadrants (To be Finalized)

Techniques

These include elements of the software design process, architectural principles, as well as ways of structuring software. In general, these are more abstract and less about particular code artifacts, which are covered by the other quadrants.

Open edX Platform frameworks

These include libraries and frameworks developed as part of the Open edX Platform, which we will ultimately want to evaluate in terms of adoption.

Frontend

We have enough technologies in the frontend development ecosystem, and they are. distinct enough from our other technologies, to justify having a frontend quadrant.

Technologies (All other technologies)

This quadrant includes both backend technologies and services used in the Open edX platform. As an editorial note, it feels a bit like a catch-all given the variety of technologies added to it.

Appendix A: Workshop Plan (2021-06-09)

Important! Before the workshop, sign up for a breakout room by adding your name to a column here: Breakout Room Signup Sheet

Board where we’ll be working: https://miro.com/app/board/o9J_lAwGZaM=/

In the Tech Radar workshop, we will individually brainstorm and collaboratively vet technologies (a.k.a., 'blips', see definition below) we believe to belong on the Open edX Tech Radar. The group will use breakout rooms organized by expertise to work with engineers with a similar background to organize and evaluate their ideas. We believe this will result in a higher-quality list, since the group will have a more homogenous well of knowledge to draw on to evaluate technologies. See the Agenda below for the list of breakout rooms.

Note that in this session, we will be focusing on what technologies should be on the radar, not where they belong. It’s okay to defer conversations about exactly where on the radar a given technology belongs.

The output of this session will be a catalog of technologies used in our software, vetted by engineers who are familiar with them. In a later workshop, we can then use this list to orient technologies (blips!) onto our radar by quadrant and level of adoption.

Agenda

• 5 mins review of this document.

• 15 mins quiet time, self-ideation of technologies.

• 40 mins breakout rooms

  • Breakout rooms:

    • Frontend, Backend, Data, Arch, SRE/DevOps, Rebels

  • Goal is to define what blips, not where they should be added to the radar.

  • Blip by blip, pitch and upvote/downvote

• 20 mins for group retro and discussion of our radar and base assumptions