/
Tutor troubleshooting notes

Tutor troubleshooting notes

Owned by Kyle McCormick
Last updated: Mar 18, 2022
7 min read

This is not an official guide! This is just a log of my (@Kyle McCormick's) experiences running Tutor locally, both for my own reference, and for the community’s reference as we work to improve Tutor and it’s official docs via the Archive: Tutor DevEnv Adoption.

Task 1: Set up Tutor for local development

Goal

  • Using a branch B, which is based on openedx/edx-platform:master, I want to:

    • Run LMS and CMS with local code

    • Run static analysis & tests

    • Run a management command

Relevant Tutor Docs

Issues Encountered

  • When working out of a repo generated by tutor dev bindmount:

    • git state was confusing as a newcomer

      • no branch information

      • patches applied on top of master - wasn’t clear where they came from at first

      • copied static assets and custom settings files made git state dirty

    • git remote protocol needed to be switched from https to ssh to allow push

    • git had to be configured to pull relevant branches

  • When using tutor dev start ... or tutor dev exec ... :

    • settings weren’t set correctly? Regis mentioned that exec doesn’t set DJANGO_SETTINGS_MODULE properly…

  • When setting COMMON_OPENEDX_VERSION to my feature branch and trying tutor images build openedx :

    • If branch has updates, needed to use --no-cache, which took a very long time.

    • Had to override openedx-dockerfile-git-patches-default to be empty, otherwise cherry-picking would conflict with the state of my branch (because said commits are already on master, it seems?)

  • Volume mounting requires full path; ~ for home didn’t work.

  • Had to start from scratch due to < insert requirements problem that carlos had >

Successful Approach

# Clone edx-platform and switch to my branch. $ git clone git@github.com:openedx/edx-platform $ (cd edx-platform && git checkout $B) # Install Tutor Nightly. $ virtualenv tutor-venv $ source tutor-venv/bin/activate $ git clone --branch nightly git@github.com:overhangio/tutor $ cd tutor $ pip install -r requirements/dev.txt $ pip install -e . # Provision Tutor. $ tutor local quickstart $ tutor local stop # Copy edx-platform virtual environment to host. $ tutor dev start lms $ tutor dev bindmount lms /openedx/venv $ tutor dev stop lms $ tutor dev dc rm # Configure Tutor to mount your local edx-platform and virtual environment # by writing a env/dev/docker-compose.override.yml file to your Tutor env. # The syntax for each mount is $HOST_PATH:$DOCKER_PATH. # Be sure to substitute in the appropriate $HOST_PATH for each mount. $ cat > $(tutor config printroot)/env/dev/docker-compose.override.yml <<- EOF version: "3.7" services: lms: volumes: - /home/kyle/openedx/edx-platform:/openedx/edx-platform - /home/kyle/.local/share/tutor-nightly/volumes/venv:/openedx/venv cms: volumes: - /home/kyle/openedx/edx-platform:/openedx/edx-platform - /home/kyle/.local/share/tutor-nightly/volumes/venv:/openedx/venv lms-worker: volumes: - /home/kyle/openedx/edx-platform:/openedx/edx-platform - /home/kyle/.local/share/tutor-nightly/volumes/venv:/openedx/venv cms-worker: volumes: - /home/kyle/openedx/edx-platform:/openedx/edx-platform - /home/kyle/.local/share/tutor-nightly/volumes/venv:/openedx/venv EOF # Install requirements, provision demo course, admin user and static assets. $ tutor dev run lms make requirements $ tutor dev run lms npm install $ tutor dev run lms openedx-assets build --env=dev $ tutor dev createuser admin admin@example.com --password admin --staff --superuser $ tutor dev importdemocourse # (NOT REQUIRED - JUST EXAMPLES) Run tests, linting, and a management command. $ tutor dev run lms pytest path/to/some/code $ tutor dev run lms pylint path/to/some/code $ tutor dev run lms ./manage.py lms run_some_management_command # Run LMS and CMS. $ tutor dev start -d lms cms

Suggested Improvements

This was just a first pass at suggested improvements. We’ve been iterating these ideas in some GitHub issues:

# Clone edx-platform and switch to my branch. $ git clone git@github.com:openedx/edx-platform $ (cd edx-platform && git checkout $B) # Install Tutor Nightly. $ virtualenv tutor-venv $ source tutor-venv/bin/activate # Either: $ git clone --branch nightly git@github.com:overhangio/tutor $ cd tutor $ make requirements # OR: # tutor-nightly could be a metapackage depending on the latest nightly (`N.dev`) tutor release $ pip install tutor-nightly # Configure mounting: # * from my edx-platform to the container's /openedx/edx-platform, and # * from the default location in tutor-nightly config to /openedx/venv. $ tutor config save --set OPENEDX_MOUNTS=/home/kyle/openedx/edx-platfrom:/openedx/edx-platform,/openedx/venv # Provision Tutor, to include a default user as well as static assets. $ tutor dev quickinit # Run tests, linting, and a management command. $ tutor dev run bash app@lms$ pytest path/to/some/code app@lms$ pylint path/to/some/code app@lms$ ./manage.py lms run_some_management_command app@lms$ exit # Run LMS, with virtual environment and application code from host. $ tutor dev runserver -d lms cms

Task 2: Set up Tutor in a local Kubernetes (k8s) cluster

Goal

Run LMS, CMS and the Learning MFE in a k8s cluster on my local machine using Tutor

Relevant Tutor Docs

Kubernetes deployment — Tutor documentation

Notes

Stream of consciousness… follow-up items are denoted with

  • Docs recommend Minikube for trying things out. Great.

    • Minikube setup was easy.

  • tutor k8s start: Failing. Unable to connect to MySQL.

  • tutor k8s quickstart:

    • Hung when it got to discovery.

    • Disabled discovery for now.

    • Without discovery plugin enabled, quickstart succeeds 👍🏻

  • How do I view LMS in the browser?

    • local.overhang.io : unable to connect.

    • docs tell me to look at caddy's external IP and configure my DNS server with it.

    • kubectl --namespace openedx get services/caddy says that EXTERNAL_IP is <pending>

    • StackOverflow tells me that I need to run minikube tunnel in order for the load balancer to work within minikube

    • Now, EXTERNAL_IP shows up.

    • Putting that IP address in the browser hangs in an encouraging way but doesn't show anything.

    • Caddy's logs show that I am successfully making requests to Caddy.

    • I'm looking at the Caddyfile. Caddyfile is defined in terms of the hostnames I have from config.yaml. That is, local.overhang.io.

    • Brain: Caddy is expecting URLs to be in the form of *.local.overhang.io/*

    • Edit /etc/hosts, pointing local.overhang.io at the value of EXTERNAL_IP

    • Go to https://local.overhang.io in the browser... hangs still

    • Brain: Right, I don't have TLS set up

    • Go to http://local.overhang.io: LMS loads!

    • Takeaway: Would be good to note minikube tunnel and /etc/hosts changes that were necessary for this to work locally.

  • Trying to log in...

    • Need to make a superuser

    • tutor k8s exec lms bash ... ./manage.py lms manage_user ... --superuser

    • Could not load django_debug_toolabr

    • DJANGO_SETTINGS_MODULE=lms.envs.production ./manage.py lms manage_user ... --superuser

      • TODO: File an issue for DJANGO_SETTINGS_MODULE being mis-set?

    • Log in works!

  • Trying to enroll...

    • Whoops, no courses.

    • tutor k8s importdemocourse.

    • That worked! Enrolled.

  • Trying to start the learning mfe...

    • tutor plugins enable mfe

    • tutor config save

    • tutor k8s init --limit=mfe -> exited successfully

    • MFE app isn't running, logs say Error from server (BadRequest): container "mfe" in pod "mfe-7757f78d77-qcpmr" is waiting to start: image can't be pulled

      • according to deployments.yml, the assigned image is