| Priority | Project Name | Dependencies | Description | Projects Which Benefit | Notes | Pain (0-5) | Frequency (0-5) | Cost (0-5) |
|---|---|---|---|---|---|---|---|---|
| 1 | Speed up tests locally | None | Currently, running a full test suite takes 'a long time', so people push to CI instead, which also takes a long time. If we shortened the full tests suite, or gave a good selection of fast smoke tests, devs could have a tighter turnaround time. Perhaps using per-test coverage data to do a reverse lookup based on edited lines?
| All projects in edx-platform | BJP (4/2015) - TestEng undertook an effort in summer 2014 to evaluate a tool to do this for us (i.e. auto-determine which tests to run using nose-knows ). However, we think it is worth revisiting. See | 1 | 5 | 2 |
| 1 | Speed up tests in CI | None | For an average PR, just running the test jobs to completion takes 40 - 50 minutes, not including time in queue. That is enough time that developers context switch, which costs time in addition
| All projects in edx-platform | TestEng is actively working on this, which lowers the usefulness of Platform putting time in.
BJP (4/2015) - A truer solution for this would be a more mature build pipeline (likely coupled with the above item). That would involve testing a subset on every PR commit, and having a way to run more tests at/near merge-time. This could also imply a different branching strategy. | 1 | 5 | 2 |
| Move grade storage to submissions app | None | Dave Ormsbee can fill in more, but essentially, we can change the storage model for grades such that they are append-only, and use that to make score computation for things like the progress page more predictable and less expensive. |
| 4 | 1 | 4 | ||
| Make Runtimes per-app, rather than per-modulestore | None | Convert from Storage-centric runtimes to Application-centric runtimes | 3 | 2 | 5 | |||
| Provide access to field storage via FieldData interface, rather than via Modulestore api methods | None | We can move towards the XBlock-standard model of field storage, which should make our runtimes have to jump through fewer hoops to work correctly, and will let us re-use some building blocks across different data stores. | 3 | 1 | 5 | |||
| 2 | Kill XModule | None | Lots of technical debt here, some of which will go away with conversion to XBlock. |
| 4 | 3 | 5 | |
| 4 | Kill XModule-specific APIs/workarounds | Kill XModule | Lots more technical debt related to making two classes look like one class. Contributes many of the most brittle code pieces. |
| 5 | 1 | 5 | |
| Switch InstructorTask to generate micro-tasks and use a distributed counter (memcache) for results on particular jobs | None | Asynchronous tasks pattern gives a strategy option. This code is really hairy right now, and jumps through a lot of hoops in order to not spam the relational database. |
| 4 | 1 | 3 | ||
| 1 | Set up infrastructure for building performance tests (working with Corey, locust.io) | None | Setting up performance tests from scratch introduces risk to any project that needs to measure serverside performance. |
| 3 | 3 | 2 | |
| 3 | Move LMS to require.js | None | Integration of Require JS into the system |
| 4 | 2 | 3 | |
| 3 | Load XModule/XBlock javascript with require.js | Move LMS to require.js | Moving XModule to require.js would allow us to get rid of the large blob of all of the XModule js that's loaded on every courseware page. Moving XBlock to requirejs would allow us to stop including js snippets on the main page directly (and hopefully cache the offloaded js). |
| 4 | 2 | 4 | |
| Clean up certificates code | None | The certificates generation code is a mess |
| 5 | 1 | 4 | ||
| 1 | Pre-process and check-in Javascript and CSS | None | Currently, django collectstatic, and the requirejs optimizer, eat up lots of time any time they're invoked. If we separated the build of all of our javascript and CSS from the execution of the platform, then we'd save time on tests and on deployment, with the only cost being a test that verifies that no one forgot the asset compilation/checkin step. |
| 1 | 5 | 1 |
Pain: How bad is this right now?
Frequency: How often do we have to experience the pain?
Cost: How much will this cost to fix?
All estimates are rough and relative only to the rest of the table, in Calen Pennington (Deactivated)'s estimation.
I talked to most of the development teams about what they felt had made them slow historically. The two most common themes were that Jenkins and the test suite are slow to run, and that devstack setup is somewhat unstable, especially when switching between environments, or for occasional devstack users. Improving the LMS javascript architecture also came up from two different teams (T&L and destEdX).