Open edX Development Environment Plans 2U/OCM: Proposal

Owned by Rebecca Graber (Deactivated)
Last updated: Nov 21, 2022
4 min read

Purpose of this Document

The purpose of this document is to explain and compare different plans for improving the development environment for 2u/OCM employees, and ultimately to choose which one(s) we should pursue.

Background

The Development Environment (DevEnv) working group was created in February 2022 with the initial purpose of facilitating the adoption of the Tutor development environment across OCM. Tutor is the current standard environment for Open edX deployment and is becoming widely used in the Open edX community for development as well. The latest releases were created without any devstack-compatible images, so anyone developing off the release branches (as opposed to master) cannot use devstack at all. See https://openedx.atlassian.net/wiki/spaces/COMM/pages/3324149773/Development+Environment+DevEnv+Working+Group#Why-Tutor%3F for more information on the initial choice to move to Tutor.

As the DevEnv working group worked on Tutor, it became clear that Tutor was not yet in a state where it would be a suitable development environment for most OCM employees. Meanwhile, as new employees joined and were onboarded, it likewise became clear that devstack was not fully compatible with new Mac hardware. Teams with new employees, along with members of the DevEnv working group, looked for quick ways to unblock the new hires. This resulted in two particular efforts: a cloud-hosted devstack, and an M1-compatible local devstack (with MySql 8.0).

Goals

Ultimately, we are working towards a development environment with the following properties:

  • Fast to install

  • Easy to maintain and/or regenerate

  • Usable by the broader Open edX community

  • Reasonably consistent with main production environments

  • Easy to understand

  • Easy to troubleshoot problems

  • Well-documented

On a user-level, we’d like the environment to facilitate developer experiences such as:

  • A new developer (OCM or community) sits down and within a short amount of time (on the order of minutes or hours rather than days), is able to start working on a task

  • A developer can easily recreate and trace a production bug in their local setup

  • A developer can easily create the test data they need for a task, keep it when generating a new environment, and provide it to other developers who may want to work in the same area

Possible Approaches

Optimizing Tutor for OCM development

Completed Work

Remaining Work

  • Reduce Tutor build time on each launch

  • Verify that Tutor can be used for edx-platform plugin development work

  • Filling out

  • Productizing the plugins created here

  • determining the structure of OCM-specific plugins

    • Creating those plugins which make an OCM dev’s life very easy to develop with.

  • an opt-in beta group of developers at OCM once all blocking issues are resolved

  • Documentation and guidance on how OCM dev teams can add applications under their ownership to tutor

Pros

  • Strongly recommended and supported by tCRIL

    • We get on-call help from Kyle and Regis who are very interested in us adopting Tutor.

  • Consistent with most of the Open edX community

Cons

  • (I’ve started tracking blockers under the 2u-blocker label: https://github.com/overhangio/2u-tutor-adoption/labels/2u-blocker -Kyle)

  • Tutor is not fully optimized for development and still suffers from some slowness and missing features

    • For example, the default of rebuilding the image every time you want to run the service is very slow, especially as it for some reason it needs to collect assets every time

  • High activation energy cost to switching the entire OCM organization off devstack

  • Context switching between Tutor and Devstack is costly time-wise

    • Sometimes prune all images, which takes a while

Creating a cloud-hosted devstack

Completed Work

  • Functional infrastructure for launching an EC2 instance running devstack

Remaining Work

  • Monitoring and automation to avoid unnecessary cost of cloud devstack instances that outlive their usefulness (some of this may be cribbed from existing code for sandbox management)

Pros:

  • Removes issues of 2U hosted macs having problems with sudo, etc

  • Allows for prebuilt AMI snapshots with complex data without having to wait for data provisioning at launch time

  • Potentially shareable, which makes Dev lives very easy

  • Could be used in conjunction with Tutor using same cloud-hosting tech

Cons

  • Incurs cloud hosting costs per instance deployed

  • That cost increases the need to throw away and rebuild dev environments, which can make it difficult to retain custom sets of test data until other tooling (like for OEP-37: Dev Data) matures

  • Another service which needs to be maintained (namely, the infrastructure for launching and managing these VMs)

Continuing incremental improvements to local devstack

Completed Work

  • Branch that solves the most acute pain points on Apple Silicon laptops

Remaining Work

  • Finish and merge the Apple Silicon fixes

  • Establish migration plan for developers upgrading mysql locally

  • Switch from Ansible-using images from the configuration repo to simpler images generated from Dockerfiles in each service repo -

Pros

  • People’s lives are made easier, work at 2U is sped up when Devstack works better/ isn’t broken

Cons

  • Duplicates effort from the Open edX community going into Tutor

  • Makes it harder to collaborate with Open edX developers who have already switched to Tutor (sometimes out of necessity)

Improving sandboxes for development

Completed Work

Remaining Work

Pros

Cons

  • Installing everything on a VM is an anti-pattern

 

  • - Arbi-BOM is starting work on parts of this

Concerns

  • Staffing - What are the top priorities? What’s important enough to preempt other projects in progress?

  •  

 

Proposal: