Debugging edX platform tests in PyCharm

Owned by Andy Armstrong (Deactivated)
Last updated: Nov 25, 2020 by Feanil PatelVersion comment
3 min read

Table of Contents

Testing

General Testing configurations: https://github.com/edx/edx-platform/blob/master/docs/guides/testing/testing.rst

We start by creating a base configuration which corresponds for the edX common unit tests.  Most other test configurations will be copied and adapted from the base configuration.

Create a debug configuration for an edX common unit test

  • Choose "Run > Edit Configurations..."
  • Select the "LMS" configuration.
  • Click the "Copy configuration" button (next to the "-" button).
  • Change the name to "Common Tests".
  • Change the script to /edx/app/edxapp/venvs/edxapp/bin/nosetests
  • Change the "Script parameters" to run the test:
    • e.g. common/lib/xmodule/xmodule/tests/test_resource_templates.py
  • Change the environment variables by clicking the "..." button. 
    • In the modal, click the + to add a new variable.
    • The variable name is DISABLE_MIGRATIONS and the value is true.
    • Note: this drastically reduces the amount of time it takes to run tests. 

Create a test configuration for an individual LMS unit test

  • Choose "Run > Edit Configurations..."
  • Select the "Common Tests" configuration.
  • Click the "Copy configuration" button (next to the "-" button).
  • Change the name to "LMS Tests"
  • Change the "Script parameters" to lms --settings test test lms/djangoapps/courseware/tests/test_views.py
    • Note that the last parameter is the individual class that we're testing
  • Click "OK" to save the new configuration

Create a debug configuration for an individual CMS unit test

  • Choose "Run > Edit Configurations..."
  • Select the "Common Tests" configuration.
  • Click the "Copy configuration" button (next to the "-" button).
  • Change the name to "Studio Tests".
  • Change the "Script parameters" to cms --settings test test cms/djangoapps/contentstore/views/tests/test_helpers.py
    • Note that the last parameter is the individual class that we're testing
  • Click "OK" to save the new configuration

Create a debug configuration for an edX acceptance test

  • Choose "Run > Edit Configurations..."
  • Select the "Common Tests" configuration.
  • Click the "Copy configuration" button (next to the "-" button).
  • Change the name to "Studio Acceptance Tests".
  • Change the "Script parameters" to run the test:
    • e.g. cms --settings acceptance harvest --traceback --debug-mode --verbosity 2 --with-xunit --xunit-file /edx/app/edxapp/edx-platform/reports/acceptance/cms.xml cms/djangoapps/contentstore/features/problem-editor.feature
  • Click "OK" to save the new configuration

Setting up Bokchoy Test Configuration (can't catch breakpoints)

Bokchoy Test Configuration

  • Go to Run -> Edit Configurations -> Add New Configuration (usually a +sign on the left).
  • Script: /edx/app/edxapp/venvs/edxapp/bin/paver.
  • Script parameters: test_bokchoy or test_bokchoy --fasttest (if you have already compiled the assets).
  • Choose the remote interpreter (usually named as "Remote Python 2.7.3 (ssh://edxapp.127.0.0.1:2222/...)").
  • Working directory: /edx/app/edxapp/edx-platform
  • Path mappings: /Users/[username]/devstack/edx-platform=/edx/app/edxapp/edx-platform

Setting up Bokchoy Test Configuration for catching break points

See also Working with Vagrant devstack and PyCharm, where there are screenshots of example configurations.

The idea is to start up the LMS and CMS servers with a special configuration (see bleow) and then run tests individually while those services are up.

Note: You may need to set FEATURES['MILESTONES_APP'] and FEATURES['ORGANIZATIONS_APP'] to False in cms/envs/bok_choy.py and lms/envs/bok_choy.py.

1. In Preferences/ Project Structure mark ./common/djangoapps/terrain/stubs as a source.

  • Make sure that Integrate XQuartz into PyCharm is working and to remember the DISPLAY variable. (in the terminal you can do echo $DISPLAY and make sure that is non-null)
    • You must start a new SSH session and verify the DISPLAY every time.
  • Configure and start Bokchoy cms server
    • Go to Run -> Edit Configurations.
    • Clone the earlier Studio or LMS server configuration.
    • Change the name to something more memorable, e.g. "CMS Bokchoy server".
    • Script: ./manage.py.
    • Script parameters: cms --settings bok_choy runserver 0.0.0.0:8031 --traceback --noreload (if you have already compiled the assets).
  • Configure and start Bokchoy lms server
    • Clone the previous setting
    • Change the name to something more memorable, e.g. "LMS Bokchoy server".
    • Script: ./manage.py.
    • Script parameters: lms --settings bok_choy runserver 0.0.0.0:8003 --traceback --noreload (if you have already compiled the assets).
  • Configure and debug Bokchoy test run
    • Clone the previous setting
    • Change the name to something more memorable, e.g. "Bokchoy test".
    • Script: /edx/app/edxapp/venvs/edxapp/bin/nosetests.
    • Script parameters: /edx/app/edxapp/edx-platform/common/test/acceptance/tests/discussion/test_cohort_management.py:CohortConfigurationTest --with-xunit --xunit-file=/edx/app/edxapp/edx-platform/reports/bok_choy/xunit.xml --verbosity=2
    • Click to edit the "Environment variables" property
    • Set the DISPLAY variable to whatever value you obtained when setting up XQuartz (see instructions) i.e. DISPLAY=localhost:11.0

Troubleshooting

Error: The browser appears to have exited before we could connect.

If you see the following error, either:

  • You have not successfully set up XQuartz
  • You haven't used the correct DISPLAY in your Run Configuration in PyCharm, or
  • You have closed the SSH session required to run the tests.  

See Integrating XQuartz into PyCharm to review instructions on any of the above.

NOTE: You may need to log out and log in to the terminal you opened in the XQuartz integration a second time if PyCharm can't find your display.

BrokenPromise: Promise not satisfied: Browser is instantiated successfully.
-------------------- >> begin captured logging << --------------------
bok_choy.browser: INFO: Using local browser: firefox [Default is firefox]
bok_choy.browser: INFO: Using default firefox profile
bok_choy.browser: DEBUG: Failed to instantiate browser: Message: The browser appears to have exited before we could connect. If you specified a log_file in the FirefoxBinary constructor, check it for details.

Error: You must link this course to an organization in order to continue.

If you see the following error, you may need to set FEATURES['MILESTONES_APP'] and FEATURES['ORGANIZATIONS_APP'] to False in cms/envs/bok_choy.py and lms/envs/bok_choy.py.

Response content was: {
  "error": "You must link this course to an organization in order to continue. Organization you selected does not exist in the system, you will need to add it to the system"
}