What should the code structure be of the edx-platform? 

Do we agree that the current state of tangled inter-dependencies is hard to maintain, comprehend, extend, and test?

Goals

Code Structure

• Clearly named, single-purpose, apps and folders, with clearly defined interfaces.

  • So they are easier to maintain, test, and comprehend.

  • So they can be substituted, if needed.

  • Easier to transition to a separate repo, if we choose to do so.

  • Easier to transition to a separately deployed process, if we choose to do so.

• Dependencies

  • Layered:

    • Low-level apps/modules do not depend on higher layers.

    • Higher layers can call exposed APIs on lower layers.

    • It's Ok for higher layers to call lower layers that are not immediately below them (i.e., skipping layers is Ok for now).

  • Loosely coupled:

    • Where possible, use django signals to notify peer layers.

    • Use clear API abstractions defined in api.py or via REST APIs

      • which include the exposed public interfaces

      • and the expected integration points for those modules

  • Separate CMS/LMS: No code dependencies between LMS and CMS. Their lower layer common folders should not call back up into LMS/CMS.

• Assets

  • Assets that belong to an app should be co-located within the app, this includes:

    • Mako Templates

    • Underscore Templates

    • App-specific Images

  • For any assets shared across apps, they can go into

    • UI toolkit - Shared Sass

    • Pattern Library - Shared JS

    • common folder - Shared Images

• Models

  • Need to consider them on a case-by-case basis.

  • By using foreign keys, it tightly couples the relationship - and requires the apps deploy within the same container.

• URLs

  • URLs exposed by an app should be co-located within the app.

  • The top-level urls.py should then include the urls from each app's urls (with specified namespaces).

• Interfaces

  • api.py should include all public in-process python interfaces

  • urls.py should include all public REST APIs

• Tests (exact details TBD - here's a proposal)

  • Unit tests 

    • Definition

      • Do not have dependencies on peer or higher layers.  

      • May either depend on or mock lower layers.

      • White-box tests of all edge conditions and paths.

    • Guidelines

      • Can use/test internal methods of the module.

      • Use override_settings instead of requiring settings in a central envs/tests.py.

  • Integration tests 

    • Definition  (Note: this differs from testing.rst.)

      • End-to-end python integration tests that test dependencies between apps.

      • Inter-dependencies are not mocked.

      • May test multiple paths, including non-happy paths if impacted by inter-dependencies.

    • Guidelines

      • Stored separately (maybe centrally).

      • Only use exposed public interfaces (see Interfaces above)

  • Acceptance tests

    • Definition

      • End-to-end UI-level black-box tests.

      • Mostly only happy paths are tested.  Regularly occurring exceptional paths that are integral to the user-facing feature may also be tested.

      • All scenarios described in user-documentation are typically covered.

Enforcement

• Have edx-linter automatically find exceptions to prevent regressions to the agreed rules.

Non-Goals

• Don't over-package.

Long-term Goals

• edx.org project is separate from the open-edx platform.

• Use decorators on views to conditionally enable them instead of using django settings in urls.py.

• Use course-scoped xblock settings (once implemented) instead of adding app-specific fields in course_module.py.

• Have unit tests in openedx/core run on their own, outside of the context of LMS/CMS.

Short-term Goals

Open Questions

• How do we support composability using Require.js?

  • For now, just keep as is - where we configure all js files in one location.

• Karma cannot discover decentralized tests.

• Can each app locally declare its own django configuration settings so they are part of its public interface?

  • They can then be defined/set centrally for the entire project, as needed.

Initial Cleanup Tasks

Here's a list of directories to address in common:

To see all the open PRs in GitHub for the Hackemon Hackathon project, look in the "clean_platform" GitHub milestone:

https://github.com/edx/edx-platform/milestone/1

Important Hackathon Points

• edx-platform/common should no longer be used, in favor of openedx/core.

• Don't move any code to openedx/core that depends on code in lms or cms!

  • Some dependencies have snuck in to openedx/core and common.

  • We want openedx/core to be a place to put lower-level code.

    • Modules in openedx/core can depend on each other (and common for now) - but never lms/cms.

• In some cases, it's appropriate to move Django applications completely outside of edx-platform.

  • The config_models Django application was moved out of edx-platform for the hackathon.

  • edx-platform and django-splash (and maybe others) were dependent on the app.

• Avoid gigantic urls.py files!

  • Keep the url patterns for particular Django applications with the actual Django application.

  • Include those urls into the lms/cms urls.py files.

• Django signals can be used to de-couple code and remove dependencies.

  • Instead of direct calls into higher-level code, lower-level code can emit a signal which is caught by higher-level code.

• GitHub milestones are great!

  • Consider using milestones for informal PR grouping.

Current issues in common folder

CMS

LMS