Django 4.2 Upgrade: IDA Upgrade Guidelines

For upgrading an IDA from Django 3.2 to Django 4.2, following milestones need to be completed. Each milestone has the necessary guidelines and plan of action mentioned to make it easier for the teams to identify the needed actions.

Upgrade Milestones

Under the effort of Django 4.2 Upgrade, complete all of the following steps to complete the upgrade.

• Fix Django 4.0 and Django 4.1 Deprecation Warnings

  • When running tests against Django 3.2, there would be Django 4.* Deprecation warnings in the test logs.

  • You can use third party django-upgrade tool to fix most of the deprecation warnings and refactor your code automatically.

  • If there are still any deprecation warnings left, you can go through the edx-platform deprecation issues to identify any specific warning and the method to resolve it.

• Upgrade to Django 4.2 in tox & Github action workflow

  • This step can be done by using modernisers created by the tech-arbi-bom team.

  • You can simply run the provided modernisers after installing the repo-tools package within a virtual env and it’ll make all the needed changes for Django 4.2 support in both tox.ini and ci.yml files.

  • If there are any Python<3.8 and Django<4.2versions present in the tox.ini or ci.yml which are not updated by the moderniser script, those should be removed.

• Update Requirements

  • Update the pinned Django version in the requirements to Django==4.2

  • You’ll need to override the common constraints in the make upgrade compile target for upgrading the dependencies. For that the approach can be used to override the Django<3.2 common constraint.

  • Run make upgrade to update all dependencies for Django 4.2.

• Run tests successfully with Django 4.2

  • Creating a PR after updating the workflows would automatically start executing tests against Django 4.2.

  • Fix all the failing tests to make the CI runners green with Django 4.2.

• Update IDA status

  • Update the repo support field in the IDA Upgrade Sheet

Major Blockers in the Upgrade

Dropped Support for MySQL 5.7

• Upstream support for MySQL 5.7 ends in October 2023. Django 4.2 supports MySQL 8 and higher. Changelog

• SRE team is currently working to resolve this blocker. Devstack and Sandbox are already running with MySQL8 now.

• You can reach out to the SRE team for upgrading your service from MySQL57 to MySQL8 on stage and production along with the upgrade work.

MemcachedCache Removal

• The django.core.cache.backends.memcached.MemcachedCache backend is removed.

• See the Changelog for details.

• See the issue for details and all the efforts made to resolve this issue in edx-platform.

  • Some PRs would also be needed in edx-internal and sandbox-internal to update respective configurations.

New Migrations when upgrading to Django 4.2

• When running make migrations after upgrading to Django 4.2, following migrations will be created so this needs to be handled accordingly for each service to avoid any down time when deploying the service with Django 4.2 on stage and production

  • https://docs.djangoproject.com/en/4.2/releases/4.0/#migrations-autodetector-changes

  • https://docs.djangoproject.com/en/4.2/releases/4.1/#migrations

  • https://docs.djangoproject.com/en/4.2/releases/4.2/#index-together-option-is-deprecated-in-favor-of-indexes

Major Changes in Django 4.* releases

Many more minor changes can be found in the change-logs for Django 4.0, 4.1, and 4.2. A few major changes identified through the changes by the tech-arbi-bom team while adding Django 4.2 support in edx-platform are mentioned below.

Changes in Django 4.0 Release

• The get_response argument for django.utils.deprecation.MiddlewareMixin.__init__() is required and doesn’t accept None.

• The providing_args argument for django.dispatch.Signal is removed.

• The NullBooleanField model field is removed, except for support in historical migrations.

• The DEFAULT_HASHING_ALGORITHM transitional setting is removed.

• Support for the pre-Django 3.1 encoding format of cookies values used by django.contrib.messages.storage.cookie.CookieStorage is removed.

• The HttpRequest.is_ajax() method is removed.

Changes in Django 4.1 Release

• CsrfViewMiddleware now uses request.META['CSRF_COOKIE'] for storing the unmasked CSRF secret rather than a masked version. This is an undocumented, private API.

• The exc_info argument of the undocumented django.utils.log.log_response() function is replaced by exception.

• Support for assigning objects which don’t support creating deep copies with copy.deepcopy() to class attributes in TestCase.setUpTestData() is removed.

• The default_app_config application configuration variable is removed.

• The django.core.cache.backends.memcached.MemcachedCache backend is removed.

Changes in Django 4.2 Release

• Setting update_fields in Model.save() may now be required. Changelog

• ManifestStaticFilesStorage now replaces paths to CSS source map references with their hashed counterparts. See Django ticket for details. Basically if any js or css 3rd party package has map reference but does not contain map file it will break the collect static.