Tutor troubleshooting notes
This is not an official guide! This is just a log of my (@Kyle McCormick's) experiences running Tutor locally, both for my own reference, and for the community’s reference as we work to improve Tutor and it’s official docs via the Archive: Tutor DevEnv Adoption.
Task 1: Set up Tutor for local development
Goal
Using a branch B, which is based on openedx/edx-platform:master, I want to:
Run LMS and CMS with local code
Run static analysis & tests
Run a management command
Relevant Tutor Docs
“Open edX Development”: https://docs.tutor.overhang.io/dev.html
Particularly, this section: https://docs.tutor.overhang.io/dev.html#setting-up-a-development-environment-for-edx-platform
“Running Open edX on the master branch (“nightly”)”: Obsolete: Tutor nightly — Tutor documentation
Issues Encountered
When working out of a repo generated by
tutor dev bindmount
:git state was confusing as a newcomer
no branch information
patches applied on top of master - wasn’t clear where they came from at first
copied static assets and custom settings files made git state dirty
git remote protocol needed to be switched from
https
tossh
to allow pushgit had to be configured to pull relevant branches
When using
tutor dev start ...
ortutor dev exec ...
:settings weren’t set correctly? Regis mentioned that
exec
doesn’t setDJANGO_SETTINGS_MODULE
properly…
When setting
COMMON_OPENEDX_VERSION
to my feature branch and tryingtutor images build openedx
:If branch has updates, needed to use
--no-cache
, which took a very long time.Had to override
openedx-dockerfile-git-patches-default
to be empty, otherwise cherry-picking would conflict with the state of my branch (because said commits are already on master, it seems?)
Volume mounting requires full path;
~
for home didn’t work.Had to start from scratch due to < insert requirements problem that carlos had >
Successful Approach
# Clone edx-platform and switch to my branch.
$ git clone git@github.com:openedx/edx-platform
$ (cd edx-platform && git checkout $B)
# Install Tutor Nightly.
$ virtualenv tutor-venv
$ source tutor-venv/bin/activate
$ git clone --branch nightly git@github.com:overhangio/tutor
$ cd tutor
$ pip install -r requirements/dev.txt
$ pip install -e .
# Provision Tutor.
$ tutor local quickstart
$ tutor local stop
# Copy edx-platform virtual environment to host.
$ tutor dev start lms
$ tutor dev bindmount lms /openedx/venv
$ tutor dev stop lms
$ tutor dev dc rm
# Configure Tutor to mount your local edx-platform and virtual environment
# by writing a env/dev/docker-compose.override.yml file to your Tutor env.
# The syntax for each mount is $HOST_PATH:$DOCKER_PATH.
# Be sure to substitute in the appropriate $HOST_PATH for each mount.
$ cat > $(tutor config printroot)/env/dev/docker-compose.override.yml <<- EOF
version: "3.7"
services:
lms:
volumes:
- /home/kyle/openedx/edx-platform:/openedx/edx-platform
- /home/kyle/.local/share/tutor-nightly/volumes/venv:/openedx/venv
cms:
volumes:
- /home/kyle/openedx/edx-platform:/openedx/edx-platform
- /home/kyle/.local/share/tutor-nightly/volumes/venv:/openedx/venv
lms-worker:
volumes:
- /home/kyle/openedx/edx-platform:/openedx/edx-platform
- /home/kyle/.local/share/tutor-nightly/volumes/venv:/openedx/venv
cms-worker:
volumes:
- /home/kyle/openedx/edx-platform:/openedx/edx-platform
- /home/kyle/.local/share/tutor-nightly/volumes/venv:/openedx/venv
EOF
# Install requirements, provision demo course, admin user and static assets.
$ tutor dev run lms make requirements
$ tutor dev run lms npm install
$ tutor dev run lms openedx-assets build --env=dev
$ tutor dev createuser admin admin@example.com --password admin --staff --superuser
$ tutor dev importdemocourse
# (NOT REQUIRED - JUST EXAMPLES) Run tests, linting, and a management command.
$ tutor dev run lms pytest path/to/some/code
$ tutor dev run lms pylint path/to/some/code
$ tutor dev run lms ./manage.py lms run_some_management_command
# Run LMS and CMS.
$ tutor dev start -d lms cms
Suggested Improvements
This was just a first pass at suggested improvements. We’ve been iterating these ideas in some GitHub issues:
# Clone edx-platform and switch to my branch.
$ git clone git@github.com:openedx/edx-platform
$ (cd edx-platform && git checkout $B)
# Install Tutor Nightly.
$ virtualenv tutor-venv
$ source tutor-venv/bin/activate
# Either:
$ git clone --branch nightly git@github.com:overhangio/tutor
$ cd tutor
$ make requirements
# OR:
# tutor-nightly could be a metapackage depending on the latest nightly (`N.dev`) tutor release
$ pip install tutor-nightly
# Configure mounting:
# * from my edx-platform to the container's /openedx/edx-platform, and
# * from the default location in tutor-nightly config to /openedx/venv.
$ tutor config save --set OPENEDX_MOUNTS=/home/kyle/openedx/edx-platfrom:/openedx/edx-platform,/openedx/venv
# Provision Tutor, to include a default user as well as static assets.
$ tutor dev quickinit
# Run tests, linting, and a management command.
$ tutor dev run bash
app@lms$ pytest path/to/some/code
app@lms$ pylint path/to/some/code
app@lms$ ./manage.py lms run_some_management_command
app@lms$ exit
# Run LMS, with virtual environment and application code from host.
$ tutor dev runserver -d lms cms
Task 2: Set up Tutor in a local Kubernetes (k8s) cluster
Goal
Run LMS, CMS and the Learning MFE in a k8s cluster on my local machine using Tutor
Relevant Tutor Docs
Kubernetes deployment — Tutor documentation
Notes
Stream of consciousness… follow-up items are denoted with
Docs recommend Minikube for trying things out. Great.
Minikube setup was easy.
tutor k8s start
: Failing. Unable to connect to MySQL.Tried
tutor k8s exec 'mysql mysql --username=... --password=...'
, could not connect.Also,
tutor k8s exec command arg1 arg2
: this doesn't work. Needs to betutor k8s exec 'command arg1 arg2'
.Went back to
dev
mode, confirmed that I can connect to MySQL.Brain: These are two different databases, so I need run
quickstart
again.
tutor k8s quickstart
:Hung when it got to discovery.
Disabled discovery for now.
Without discovery plugin enabled,
quickstart
succeeds 👍🏻
How do I view LMS in the browser?
local.overhang.io : unable to connect.
docs tell me to look at caddy's external IP and configure my DNS server with it.
kubectl --namespace openedx get services/caddy
says that EXTERNAL_IP is<pending>
StackOverflow tells me that I need to run
minikube tunnel
in order for the load balancer to work within minikubeNow, EXTERNAL_IP shows up.
Putting that IP address in the browser hangs in an encouraging way but doesn't show anything.
Caddy's logs show that I am successfully making requests to Caddy.
I'm looking at the Caddyfile. Caddyfile is defined in terms of the hostnames I have from config.yaml. That is, local.overhang.io.
Brain: Caddy is expecting URLs to be in the form of
*.local.overhang.io/*
Edit /etc/hosts, pointing local.overhang.io at the value of EXTERNAL_IP
Go to https://local.overhang.io in the browser... hangs still
Brain: Right, I don't have TLS set up
Go to http://local.overhang.io: LMS loads!
Takeaway: Would be good to note
minikube tunnel
and/etc/hosts
changes that were necessary for this to work locally.
Trying to log in...
Need to make a superuser
tutor k8s exec lms bash
..../manage.py lms manage_user ... --superuser
Could not load django_debug_toolabr
DJANGO_SETTINGS_MODULE=lms.envs.production ./manage.py lms manage_user ... --superuser
TODO: File an issue for DJANGO_SETTINGS_MODULE being mis-set?
Log in works!
Trying to enroll...
Whoops, no courses.
tutor k8s importdemocourse
.That worked! Enrolled.
Trying to start the learning mfe...