XQueue/XQWatcher Development

For more general architectural details:

 

So you need to develop/troubleshoot XQueue and its external grading? This page contains helpful details.

xqueue

A devstack xqueue Docker container is available, which supports the same development as other Python/Django IDAs.

To run xqueue in devstack, use: make dev.up.xqueue

xqueue-watcher

Local Development

Here’s the steps required to get a functional xqueue-watcher running locally (without codejail):

  1. Start up the xqueue container in devstack.

    1. make dev.up.xqueue

  2. Run xqueue migrations to create DB tables.

    1. make dev.shell.xqueue Then: # source ../xqueue_env # python manage.py migrate
  3. Create a Django user to use for xqueue-watcher authentication into XQueue.

    1. make dev.shell.xqueue Then: # source ../xqueue_env # python manage.py shell Then: >>> from django.contrib.auth.models import User >>> user = User.objects.create_user('lms', 'test@example.com', 'lms') >>> user.save()
  4. Clone the xqueue-watcher repo.

    1. git clone https://github.com/edx/xqueue-watcher.git

  5. Create a Python 3.8 virtualenv.

  6. Install the xqueue-watcher requirements.

    1. pip install -r requirements/production.txt

  7. Create a directory structure that mirrors the one below:

    1. . └── root ├── 600 ├── config │   ├── conf.d │   │   └── 600.json │   └── logging.json └── xqueue-watcher ├── AUTHORS ├── LICENSE.TXT ├── Makefile ├── README.md ├── conf.d ├── coverage.xml ├── grader_support ├── load_test ├── openedx.yaml ├── requirements ├── setup.py ├── tests └── xqueue_watcher
  8. Now, run the xqueue-watcher:

    1. python -m xqueue_watcher -d ../config

Troubleshooting

If the xqueue-watcher doesn’t connect successfully, look at its debug logs to see why. The logs are output to stdout when bringing up xqueue-watcher.

To view the xqueue logs and look for errors/activity, run this devstack make command:

  • make dev.logs.xqueue

Suggested Future Development Work

Developing graders or fixes to xqueue-watcher is not easy! Here’s some suggested items to make things easier:

  • Provide a Docker devstack xqueue-watcher container

    • This container would be built with:

      • the same Python virtualenvs used in production

      • the same codejail environments (using AppArmor)

      • the ability to use arbitrary grading repos/code in the container

  • Make xqueue-watcher a Python module

    • Currently, several xqueue-watcher files are duplicated across grader repos - which is the very problem that Python modules are made to solve.

    • Have each grader install the xqueue-watcher Python module and derive its graders from the common code.

    • Would include a script/method for testing all grader code written by a course team.

      • Accepts a configuration file which specifies all tests to run.

        • Code to grade along with the expected results.

  • Define a clear best practice for writing graders.

    • Create an edX-written grader repo which demonstrates all the best practices.