edx-platform uses Django. Django is in active development and keeps advancing with continual releases. edx-platform must incorporate these new releases:
https://www.djangoproject.com/download/
This upgrade was a big project, taking ~5 engineers 4+ months.
For details, view the subpages under this top-level page:
/wiki/spaces/TNL/pages/34013439
Here's the notes from the actual release:
/wiki/spaces/RELEASES/pages/46794141
Here's the retro notes:
As of June 2016, Django 1.9 is out and on version 1.9.7. It is not a long-term-support (LTS) release and we haven't made the decision to upgrade edx-platform's Django to it yet. However, it must happen eventually.
Official:
https://docs.djangoproject.com/en/1.9/releases/1.9/
Broken down onto a Confluence page with created JIRA tickets:
Django 1.9 Release Notes and Stories
In the current Django 1.8 that edx-platform uses, we've eliminated nearly all the deprecation messages that warn us about functionality that will not be supported in Django 1.9. However, one particularly annoying message remained:
In case that's not readable, it's this warning message:
WARNINGS:
wiki.ArticleRevision.ip_address: (fields.W900) IPAddressField has been deprecated. Support for it (except in historical migrations) will be removed in Django 1.9.
HINT: Use GenericIPAddressField instead. |
I first made a branch to get rid of this warning message - this branch:
https://github.com/edx/django-wiki/tree/jeskew/fix_model_deprecation_warning
Unfortunately, it can't be merged and used by edx-platform until "Comprehensive Theming" lands. But it's ready after that happens!
The first thing I ran into was this Django 1.9 change:
app_label. Furthermore, it isn’t possible to import them before their application is loaded. In particular, it isn’t possible to import models inside the root package of an application.This subtle point means that upon Django startup - when all INSTALLED_APPS are being imported - no models can be imported from the __init__.py of those applications. Unfortunately, edx-platform and its module dependencies do this all over the place.
NOTE: If you're writing new Django apps, do not import models inside the __init__.py (root package) of the application. |
The errors caused by this premature model importing will look similiar to this one:
Traceback (most recent call last):
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "/home/jenkins/edx-venv/local/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/home/jenkins/edx-venv/local/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/home/jenkins/edx-venv/local/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/home/jenkins/workspace/edx-platform-test-subset/common/lib/xmodule/xmodule/video_module/__init__.py", line 8, in <module>
from .transcripts_utils import *
File "/home/jenkins/workspace/edx-platform-test-subset/common/lib/xmodule/xmodule/video_module/transcripts_utils.py", line 18, in <module>
from .bumper_utils import get_bumper_settings
File "/home/jenkins/workspace/edx-platform-test-subset/common/lib/xmodule/xmodule/video_module/bumper_utils.py", line 16, in <module>
import edxval.api as edxval_api
File "/home/jenkins/edx-venv/local/lib/python2.7/site-packages/edxval/api.py", line 13, in <module>
from edxval.models import Video, EncodedVideo, CourseVideo, Profile
File "/home/jenkins/edx-venv/local/lib/python2.7/site-packages/edxval/models.py", line 48, in <module>
class Profile(models.Model):
File "/home/jenkins/edx-venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 94, in __new__
app_config = apps.get_containing_app_config(module)
File "/home/jenkins/edx-venv/local/lib/python2.7/site-packages/django/apps/registry.py", line 239, in get_containing_app_config
self.check_apps_ready()
File "/home/jenkins/edx-venv/local/lib/python2.7/site-packages/django/apps/registry.py", line 124, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
AppRegistryNotReady: Apps aren't loaded yet. |
After modifying 5+ external Python modules, changing their requirements entries, and changing around the startup.py files, I was able to run the following command to completion:
python manage.py cms --settings=devstack reindex_course --setup
However, when I started up LMS or Studio, I got errors indicating that I had unapplied migrations.
When I attempted a migration, I got errors about importing South in djcelery! What??! Turns out the version that edx-platform uses has some remaining migrations that are pre-Django-1.7? I upgraded djcelery to the most recent version and tried migration again. Got a message about a table already existing. When I compared the existing schema and the new single 0001_initial.py migration in djcelery, the schema did not match. So some tricky database figuring-out will be required to proceed. The next step:
The branch I created for this work is here:
https://github.com/edx/edx-platform/tree/jeskew/django_1_9_compat
And the PR I used to run tests is here:
https://github.com/edx/edx-platform/pull/12770
I also created branches/forks in other repos. To find all those changes, review the base.txt/github.txt files from the requirements files on the branch above.