How to fix edx-platform Unit Test Warnings

Current warnings in edx-platform

Total warnings count: 5060

Categories: 13, sub-categories: 196

• DeprecationWarning, count: 2660

  • common, count: 1633, types: 18

  • python, count: 655, types: 26

  • other, count: 246, types: 19

  • openedx, count: 85, types: 6

  • lms, count: 37, types: 7

  • cms, count: 4, types: 2

• RuntimeWarning, count: 1531

  • python, count: 1531, types: 40

• UserIdDeprecationWarning, count: 391

  • python, count: 311, types: 1

  • openedx, count: 80, types: 1

• ModifyingEnforceTypeWarning, count: 149

  • common, count: 146, types: 3

  • python, count: 3, types: 1

• UnorderedObjectListWarning, count: 145

  • python, count: 134, types: 13

  • openedx, count: 10, types: 1

  • lms, count: 1, types: 1

• ResourceWarning, count: 90

  • python, count: 56, types: 14

  • other, count: 36, types: 12

  • lms, count: 11, types: 5

  • common, count: 6, types: 5

  • cms, count: 2, types: 2

  • openedx, count: 1, types: 1

• FailingEnforceTypeWarning, count: 41

  • common, count: 36, types: 9

  • lms, count: 5, types: 3

• SystemTimeWarning, count: 5

  • python, count: 5, types: 1

• UserWarning, count: 5

  • common, count: 5, types: 1

• FutureWarning, count: 2

  • python, count: 2, types: 1

• CacheKeyWarning, count: 1

  • python, count: 1, types: 1

• InsecureRequestWarning, count: 1

  • python, count: 1, types: 1

• PytestUnhandledThreadExceptionWarning, count: 1

  • python, count: 1, types: 1

How to get current warnings report?

• Currently, edx-platform contains a CI job to compile and generate warnings report after running all the unit tests successfully.

• The compiled warning report is saved in both html and pdf formats as the build artefacts in the unit tests and can be downloaded from the Summary section from the CI checks layout.

How to deal with current warnings output format?

• Before starting the process to fix all the warnings, the existing check can be modified to only display the warnings output in the generated report and not on console.

• For warnings that are probably harmless and individual developers aren't chasing down, we should suppress the warning in the tests, with instructions how to re-enable them if you are working on them.

• A flag could be introduced for the devs working on fixing the warnings to enable the warnings console output when trying to fix the warnings.

How to fix existing warnings?

• An instruction guide with guidelines to fix warning for the community would be created.

• Any existing default or third party tools and code-mods to fix the warnings automatically can be found and shared with the community to be used.

• For 2u-internal devs:

  • If a tool can be run to fix the warnings in bulk, then the Clean Up Python Code Jenkins job can be used to run it across edx-platform in batches and create PRs automatically for fixing the warnings.

Guidelines to fix warnings

• Fixing ResourceWarnings

  • ResourceWarnings occur when any resource has been allocated but isn’t completely unallocated/removed. https://2u-internal.atlassian.net/browse/BOM-3346 documents an initial effort on fixing such warnings.

    • Use tracemalloc() to find the point of allocation of any unallocated resource in the code.

    • Update/Refactor the code to properly un-allocate the resource after usage by using either try/catch or preferably using with context manager.

• Fixing DeprecationWarnings

  • DeprecationWarnings occur when the deprecated code is still being used anywhere in the code. https://github.com/openedx/edx-platform/pull/30398 is an initial attempt to fix some of the DeprecationWarnings occurring due to the opaque-keys package.

    • This PR fixes one type of warnings and resolves around ~1000 deprecation warnings.

How to engage Open Source Community on this?

• Each warning section could have a GitHub Issue created to differentiate and streamline the process of working on the warnings.

• GitHub Issues can be announced and shared with the open source community to get them onboard for fixing these warnings and earn open source contributions.

• Get the word out to the community that we are trying to squash warnings.  We've done this before, and got people to help (announcement: https://discuss.openedx.org/t/squash-a-warning-earn-a-badge/3497 , and people who earned the badge: https://discuss.openedx.org/badges/104/warning-squasher).

Future plan after fixing all warnings?

• Once the warnings are squashed, make the tests stricter to prevent warnings from coming back.

• Create an ongoing process to keep the community engaged in fixing the warnings in parallel with the upgrades in edx-platform.