Skip to end of metadata
Go to start of metadata


The Open edX devstack is a Vagrant instance designed for local development. The instance:

  • Uses the same system requirements as production. This allows developers to discover and fix system configuration issues early in development.
  • Simplifies certain production settings to make development more convenient. For example, it disables nginx and gunicorn in favor of runserver for Django development.

The Open edX devstack is also designed to run code and tests, but you can do most development in the host environment:

  • Git repositories are shared with the host system, so you can use your preferred text editor/IDE.
  • You can load pages served by the running Vagrant instance.

The Open edX devstack configuration has the following components:

  • LMS (student facing website)
  • Studio (course authoring)
  • Forums / elasticsearch / ruby (discussion forums)

For comparison, look at the Full Stack page.

Pre-Requisites and Installation

Follow the instructions at http://edx.readthedocs.io/projects/edx-installing-configuring-and-running/en/latest/installation/devstack/index.html

In section 3.3.1.2 Install Devstack, be sure to set the OPENEDX_RELEASE environment variable to "master". This downloads the latest version of devstack, instead of a named release.  

Once Devstack is Running

  • Once the VM is created, log in to the VM via the command: vagrant ssh

  • If you need more control over the vagrant configuration, you can download the Vagrantfile and run vagrant yourself:

    curl -OL https://raw.github.com/edx/configuration/$OPENEDX_RELEASE/vagrant/release/devstack/Vagrantfile
    vagrant plugin install vagrant-vbguest
    vagrant up
    • Note you may need to pass a "--provider virtualbox" to "vagrant up" when running it for the first time, for example in case of some libvirt-related error occurring if you have more Vagrant providers installed on your system.
  • OPTIONAL [pertains to Windows and others having issues with NFS]: To use VBOXFS, set in your shell the environment variable VAGRANT_USE_VBOXFS = true

  • OPTIONAL [pertains to unix/linux and mac hosts]: To use "preview" from within Studio, you will need to add a line, 192.168.33.10 preview.localhost to your development machine's/etc/hosts file. For example:

    $ cat /etc/hosts
    # Host Database
    127.0.0.1   localhost
    255.255.255.255 broadcasthost
    ...
    $ # if preview.localhost doesn't already exist:
    $ sudo bash -c "echo '192.168.33.10  preview.localhost' >> /etc/hosts"
    $ cat /etc/hosts
    # Host Database
    127.0.0.1   localhost
    255.255.255.255 broadcasthost
    ...
    192.168.33.10  preview.localhost
    
    
  • OPTIONAL - if you wish to customize the location of the source code that gets cloned during devstack provisioning, or if you prefer to work on source trees that already exist on your system, you can set the VAGRANT_MOUNT_BASE env var, which sets the base dir for the edx-platform and cs_comments_service. By default it will be wherever you run vagrant up.Example:


    $ cd $HOME/my-workspace/my-edx-workspace
    $ # This is where you either want vagrant provisioning to clone,
    $ #  or you have existing repositories (although note that vagrant provision will overwrite them):
    $ ls -F
    $ #  You might optionally need to rename edx-ora to ora
    cs_comments_service/   edx-platform/   ora/
    $ # Now, set the environment for vagrant
    $ # (you may want to put this in a file, and 'source' it instead)
    $ export VAGRANT_MOUNT_BASE=$PWD

Using the edX devstack

LMS Workflow

  • Bring up the VM and log into it: vagrant upvagrant ssh
  • Within the VM instance, switch to the edxapp user: sudo su edxapp
    • NOTE: By default, the edxapp user has no password set.
  • This will source the edxapp environment (/edx/app/edxapp/edxapp_env) so that the venv python and rbenv ruby are in your search path.
  • It will also set the current working directory to the edx-platform repository (/edx/app/edxapp/edx-platform). If you get an error: Unknown task: devstack, the working directory has not been updated properly, simply do cd /edx/app/edxapp/edx-platform and re-run the command.

If you are running the developer stack with the standard environment file, lms/envs/devstack.py, then you can use this command:

paver devstack lms

There is a faster version of this command that starts the server without updating requirements or compiling static assets. To ONLY start the server:

paver devstack --fast lms

Note: you can type paver --help to list commands, or paver [command] --help for command options.

Open a browser on your host machine and navigate to http://localhost:8000/ to load the LMS. (Vagrant will forward port 8000 to the LMS server running in the VM.)

The latest version of devstack has the demo course pre-loaded and dummy accounts, you can log in to the website as:

  • staff@example.com / edx
  • verified@example.com / edx
  • audit@example.com / edx
  • honor@example.com / edx

If you are bringing up LMS with a custom configuration (For example devstack as for developer stack and devstack.py, but you name yours as you require):

  • Update requirements, and compile Sass and CoffeeScript:

    paver update_assets --settings devstack lms

If there is a change to layout, or if you ran the server with the --fast flag from the start and are wondering why all of the css/images are missing, you will need to recompile the css and other assets as well:

paver update_assets


Notepaver -h will show other available commands

  • Start your custom configured server:

    ./manage.py lms runserver --settings=devstack 0.0.0.0:8000

Note./manage.py lms -h --settings=devstack will show other available commands on the server.

Studio Workflow

Within the VM instance, switch to the edxapp user:

sudo su edxapp

This will source the edxapp environment (/edx/app/edxapp/edxapp_env) so that the venv python and rbenv ruby are in your search path. It will also set the current working directory to the edx-platform repository (/edx/app/edxapp/edx-platform).

Note: the following steps are for running with the standard configuration file. To run Studio with a custom configuration, follow the instructions as for LMS, above, replacing lms with cms in the commands.

If you are running the developer stack with the standard environment file, cms/envs/devstack.py, then you can use this command:

paver devstack studio

There is a faster version of this command that starts the server without updating requirements or compiling assets. To ONLY start the server:

paver devstack --fast studio

Open a browser on your host machine and navigate to http://localhost:8001/ to load Studio. (Vagrant will forward port 8001 to the Studio server running in the VM.)

  • OPTIONAL: you can see what other commands are available on the server (such as creating users, collecting static files, etc) with:

    ./manage.py cms -h --settings=devstack

Forum Workflow

  • Within the VM instance, switch to the forum account

    sudo su forum
  • Update Ruby requirements (Note that if you get a message for entering a password to install the bundled RubyGems to the system, you can safely ctrl-c out of that. The gems will still be installed correctly for the forum user.)

    bundle install
  • Start the server

    ruby app.rb -p 18080
  • Access the API at http://localhost:18080 (Vagrant will forward port 18080 to the Forum server running in the VM.)


Running LMS/Studio simultaneously

If you are running both LMS and Studio without the --fast switch, you will notice that while starting the either server, the LMS server will continuously redeploy itself and show a variant of this message repeatedly:

June 09, 2016 - 14:50:05
Django version 1.8.7, using settings 'lms.envs.devstack'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
June 09, 2016 - 14:50:14
Django version 1.8.7, using settings 'lms.envs.devstack'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
.
.
.

This is due to the fact that when not runing in --fast mode, there is a watcher in place that will redeploy the server if a file change is detected, and during the startup of the Studio server, several files in the LMS instance are touched.  As a workaround for this issue, either run LMS in --fast mode, or just wait for the file touching to complete.

Updating devstack to latest

If you plan to make changes that will result in a pull request for any of the devstack applications, you will likely want to update your devstack to the latest from the upstream master.  To do so, first make sure that your devstack instance was from the latest devstack download and NOT from one of the named releases.  If you defined OPENEDX_RELEASE before downloading devstack, than this is NOT the case.  Once this is true, you'll want to switch to the edxapp user and update the code and databases.

  1. First, Bring up the VM and log into it:

    $ vagrant up
    .
    .
    .
    $ vagrant ssh
  2. Within the VM instance, switch to the edxapp user:

    sudo su edxapp
  3. This will bring you to the edx-platform directory.  Next, update the code to the latest on master:

    $ git checkout master
    $ git pull
  4. Once you have the latest code, the database will be out of sync with the code changes.  You will need to update the database by running the scripted migrations:

    $ paver update_db
  5. At this point, you should be set.  Bring up cms and/or lms (as directed above) and verify that you have working instances.

Developing on devstack

See Developing on the edX Developer Stack once you are ready to start developing on devstack.

Running LMS/Studio Tests

  • Within the VM instance, switch to the edxapp user:
sudo su edxapp
  • Run the Python unit tests:
paver test_python

NOTE: if these tests fail with an error about needing a sudo password, use visudo to update your sudoers file for the edxapp user:

visudo -f /etc/sudoers.d/01-sandbox
  
  edxapp ALL=(sandbox) SETENV:NOPASSWD:/edx/app/edxapp/venvs/edxapp-sandbox/bin/python
  edxapp ALL=(sandbox) SETENV:NOPASSWD:/usr/bin/find
  edxapp ALL=(ALL) NOPASSWD:/usr/bin/pkill


See also the CodeJail readme.

  • Run the JavaScript unit tests

    paver test_js
  • Run the LMS and Studio acceptance tests

    paver test_acceptance

See the docs on testing edx-platform for detailed information about writing and running tests.

Updating Application Versions

Use vagrant provision to update the configuration and all repositories to the master branch. NOTE: this will perform a git clean on your repositories, so make sure you have committed any changes you want to save.

If you want to only update the configuration repo, vagrant ssh, sudo /edx/bin/update configuration master.

Logs

Logs for the running applications can be found in various application specific folders in/edx/var/log.

Troubleshooting / Issues / Workarounds

If you can't find the solution to your problem here, there is more documentation on the Open Source Project Page and ReadTheDocs

Provisioning Issues

edx-platform checkout failure

Sometimes there's a failure to checkout the edx-platform repo (as shown below). This can occur if the edx-platform repository is already checked out on your machine and the git URL for the origin remote is set to an SSH URL (e.g. git@github.com:edx/edx-platform.git) instead of HTTPS e.g.https://github.com/edx/edx-platform.git).

SSH keys are not installed on the Vagrant box so SSH remotes will not work. Use an HTTPS remote instead to resolve this issue. This can be set by using git remote set-url origin <https url> as the edxapp user.

TASK: [edxapp | checkout edx-platform repo into {{edxapp_code_dir}}] **********
failed: [localhost] => {"cmd": ["/usr/bin/git", "ls-remote", "origin", "-h", "refs/heads/master"], "failed": true, "item": "", "rc": 128}
stderr: Warning: Permanently added 'github.com,192.30.252.130' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

msg: Warning: Permanently added 'github.com,192.30.252.130' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
           to retry, use: --limit @/root/vagrant-devstack.retry

localhost                  : ok=125  changed=20   unreachable=0    failed=1



Stderr from the command:

stdin: is not a tty

VirtualBox 4.3.10 NFS Mounts

  • If you get the following error:

    mount.nfs: mount to NFS server '192.168.33.1:/path/to/edx-platform' failed: timed out, giving up
    

    This can be caused by a few issues:

    • your development directory is in a FUSE mounted directory on Linux
    • conflict with a VPN

    Some Linux-based NFS servers do not support shares inside a FUSE mounted directory. The easiest fix is to move your development directory somewhere else.

    You can fix the VPN conflict with these steps:

    • Stop your VPN
    • vagrant halt
    • Open the VirtualBox GUI app, go to Preferences > Network > Host-Only Networks. Remove all of them.
    • vagrant up

    NFS will now work whether your VPN is running or not.

  • VirtualBox can have issues when Guest Additions don't match the version installed when it comes to NFS mounts, which is fixed in version 4.3.12. In particular, VirtualBox 4.3.10 Guest Additions has an NFS mounting bug and can be fixed with

    vagrant up
    vagrant ssh -c 'sudo ln -s /opt/VBoxGuestAdditions-4.3.10/lib/VBoxGuestAdditions /usr/lib/VBoxGuestAdditions'
    vagrant reload
    
  • If you try to reinstall devstack you might come upon this error:

    NFS is reporting that your exports file is invalid. Vagrant does
    this check before making any changes to the file. Please correct
    the issues below and execute "vagrant reload":
    

    that is because there are entries already in your /etc/exports file. To fix this issue, delete the block of the devstack that you're reinstalling within your /etc/exports file. An export block for devstack looks like this:

    # VAGRANT-BEGIN: 502 xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    "/Users/user/path/to/devstack/cs_comments_service" 192.168.33.10 -alldirs -mapall=502:20
    "/Users/user/path/to/devstack/edx-platform" 192.168.33.10 -alldirs -mapall=502:20
    "/Users/user/path/to/devstack/ora" 192.168.33.10 -alldirs -mapall=502:20
    "/Users/user/path/to/devstack/themes" 192.168.33.10 -alldirs -mapall=502:20
    # VAGRANT-END: 502 xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    

    You can have a few this is you installed more devstacks, make sure you delete the right one. You can find the vagrant id for a particular devstack install by running this inside the devstack folder:

    cat .vagrant/machines/default/virtualbox/id

MySQL time zone definitions missing

If you see a ValueError like the following when working in the Django admin:

ValueError at /admin/certificates/certificatehtmlviewconfiguration/
Database returned an invalid value in QuerySet.datetimes(). Are time zone definitions for your database and pytz installed?

Run the following:

$ mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql

If that doesn't work, follow the instructions at https://bugs.mysql.com/bug.php?id=68861.

General tips

  • If you want to build an edX devstack environment from a base precise64 image, you can do the following
git clone https://github.com/edx/configuration.git
cd configuration/vagrant/base/devstack
vagrant up
  • If the /edx/app/edx/edx-platform directory in the Vagrant image is empty, it means the provisioning scripts did not complete successfully. Try re-running the provisioning scripts using vagrant provision.

Signing up a new user (activation message problems):

  • When signing up as a new user, this message is shown in the browser:
We need to verify your email address

Almost there! In order to complete your sign up we need you to verify your email address. 
An activation message and next steps should be waiting for you there. 

However, when running the stack locally, there is no email message sent. Rather the message comes out on the console, giving the activation key to use:

Thank you for signing up for edX Studio! To activate your account, please copy and paste this address into 
your web browser's address bar:

            http://localhost/activate/0699e17df6024fb382fadfabed100691

If you didn't request this, you don't need to do anything; you won't receive any more email from us. Please do
not reply to this e-mail; if you require assistance, check the help section of the edX web site. 

Using the domain address ‘http://localhost/activate/… may not work when you copy this address into the web browser’s address field as instructed. if a ’Server not found’ message comes up in the browser, substitute

http://127.0.0.1:8001

for localhost. For example:

http://localhost/activate/a19de8c6330f4213ac3313c147297990

becomes

http://127.0.0.1:8001/activate/a19de8c6330f4213ac3313c147297990
  • Database migrations: If you update the platform code, then when using the application you get an error message in your browser like this:
Exception Type:   DatabaseError
Exception Value:  

(1146, "Table 'edxapp.new_foo' doesn't exist")

Exception Location:   /edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/MySQLdb/connections.py in defaulterrorhandler, line 36

there have been database migrations that you need to perform. Note this will not destroy your data. You can use the following command:

paver update_db -s devstack
  • If you see this error when attempting to run the server:
pymongo.errors.ConnectionFailure: could not connect to localhost:27017: [Errno 111] Connection refused

or this one, when attempting to run the forums:

MOPED: Could not connect to any node in replica set <Moped::Cluster nodes=[<Moped::Node resolved_address="127.0.0.1:27017">]>, refreshing list.

The Vagrant instance not shut down properly. Use vagrant halt to safely stop the Vagrant instance. To fix this, try:

vagrant up
vagrant ssh
sudo rm /edx/var/mongo/mongodb/mongod.lock
sudo mongod -repair --config /etc/mongod.conf
sudo chown -R mongodb:mongodb /edx/var/mongo/.
sudo service mongod start

If using a pre-lavash version of devstack, do this instead:

vagrant up
vagrant ssh
sudo rm /edx/var/mongo/mongodb/mongod.lock
sudo mongod -repair --config /etc/mongodb.conf
sudo chown -R mongodb:mongodb /edx/var/mongo/mongo/.
sudo service mongod start
  • If you see a database error something like this when running the server:
DatabaseError: (1146, "Table 'edxapp.dark_lang_darklangconfig' doesn't exist")

Then this means that your database is outdated and needs to be migrated.

sudo su edxapp
 ./manage.py lms migrate --settings=devstack

and then do the same thing for Studio with

sudo su edxapp
./manage.py cms migrate --settings=devstack

FactoryError on oAuthTests:

If you are seeing the following error when running your python tests, you will need to force an upgrade on one of the dependencies:

FactoryError: Cannot generate instances of abstract factory ClientFactory; Ensure ClientFactory.Meta.model is set and ClientFactory.Meta.abstract is either not set or False.

If you want to verify that you have this edx-oauth2-provider issue, you can use the following:

sudo su edxapp
~/edx-platform$ cat ~/edx-platform/requirements/edx/github.txt | grep "edx-oauth2-provider"
# Should display 0.5.7 or greater if this is your issue.  This is the dependency you want.
pip freeze | grep “edx-oauth2-provider"
# Should display 0.5.1.  This is the dependency you have.

To fix (still as edxapp user):

# force uninstall of bad dependency
pip uninstall edx-oauth2-provider
# run previously failing test that will install updated dependency and now pass
paver test_system -t lms/djangoapps/oauth2_handler/tests.py:UserInfoTest.test_request_staff_courses_with_claims

Forums / Ruby Issues:

  • Bundler::GemNotFound on ruby startup

Your gems need updating (might happen when switching versions of cs_comment_service). Runbundle install to resolve this, and then retry.

If you are getting a password prompt after running bundle install, you will need to fix some directory permissions:

# NOTE do this as user vagrant, not user forum
sudo chmod -R 0770 /edx/app/forum/.rbenv
sudo chmod -R 0770 /edx/app/forum/.gem

Then sudo back to user forum, do bundle install again, and you should no longer be prompted.

Running Tests is Broken, aka Insecure String Pickle

If running tests seems broken, try removing the .testids from your project root. "git clean -fxd .testids" should also work. Aborting a test mid-run can leave those in a bad state. One error message we've seen a few times from this is "insecure string pickle". No doubt there are many other error messages which might result from this problem, but none that are as poetic!

Ubuntu:

  • if you see this message:
It appears your machine doesn't support NFS, or there is not an
adapter to enable NFS on this machine for Vagrant. Please verify
that `nfsd` is installed on your machine, and try again. If you're
on Windows, NFS isn't supported. If the problem persists, please
contact Vagrant support.

you need to install nfs using this command:

sudo apt-get install nfs-common nfs-kernel-server
  • If you get an error such as NS_ERROR_FAILURE, try upgrading VirtualBox

  • If you are working on an Ubuntu workstation and your home directory is encrypted you will likely run into NFS problems. If you see this message:

exportfs: Warning: ... does not support NFS export.

To work around the issue set VAGRANT_MOUNT_BASE to a dir outside the encrypted volume.

  • If you see the message  IOError: [Errno 37] No locks available, that indicates that the host machine isn't running rpc.statd or statd. This has been seen to affect Ubuntu >= 15.04 (running systemd), and the resolution is to run systemctl rpc-statd start and systemctl rpc-statd enable.
    • Or run these commands if that doesn't work: sudo systemctl enable rpcbind.service, sudo systemctl start rpcbind.service, sudo systemctl restart rpcbind.service


Windows

  • NPM might fail because of symlinks. If you see this message:
TASK: [edxapp | Install edx-platform npm dependencies] ************************

failed: [localhost] => {"changed": true, "cmd": "npm install ", "delta": "0:00:0
3.358306", "end": "2014-03-24 16:51:37.509494", "item": "", "rc": 1, "start": "2
014-03-24 16:51:34.151188"}
stderr: npm http GET http://registry.npmjs.org/coffee-script
npm http 200 http://registry.npmjs.org/coffee-script
npm ERR! error installing coffee-script@1.6.1

npm ERR! Error: UNKNOWN, unknown error '../coffee-script/bin/coffee'
npm ERR! You may report this log at:
npm ERR!     <http://bugs.debian.org/npm>
npm ERR! or use
npm ERR!     reportbug --attach /edx/app/edxapp/edx-platform/npm-debug.log npm
npm ERR!
npm ERR! System Linux 3.2.0-23-generic
npm ERR! command "node" "/usr/bin/npm" "install"
npm ERR! cwd /edx/app/edxapp/edx-platform
npm ERR! node -v v0.6.12
npm ERR! npm -v 1.1.4
npm ERR! path ../coffee-script/bin/coffee
npm ERR! code UNKNOWN
npm ERR! message UNKNOWN, unknown error '../coffee-script/bin/coffee'
npm ERR! errno {}
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /edx/app/edxapp/edx-platform/npm-debug.log
npm not ok

FATAL: all hosts have already failed -- aborting

Go to Troubleshooting Vagrant and follow the instructions towards the bottom of the page about this and other symlink errors in Windows.

Mac OS X

Sometimes, NFS fails the first time you start Vagrant on Mac OS X. If you see this message:

[default] Mounting NFS shared folders...
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!
mount -o 'vers=3,udp' 192.168.33.1:'/path/to/devstack/edx-platform' /edx/app/edxapp/edx-platform
Stdout from the command:
Stderr from the command:
stdin: is not a tty
mount.nfs: requested NFS version or transport protocol is not supported

then reload and re-provision the instance using these commands:

vagrant reload
vagrant provision

Fedora 23 (NFS mount hanging indefinitely)

If the virtual machine hangs on the "Mounting NFS shared folders..." stage indefinitely without any errors when attempting to run it, it could mean the default NFS version (3) used by Vagrant may be incompatible with your host system. It was noticed for instance on a Fedora 23 installation. The workaround is to edit the Vagrantfile and add "nfs_version: 4" there in the MOUNT_DIRS part, as follows:

MOUNT_DIRS.each { |k, v|
  config.vm.synced_folder v[:repo], v[:local], create: true, nfs: true, nfs_version: 4
}



2 Comments

  1. If you have support questions, see the Getting Help page (https://open.edx.org/getting-help) for places to ask.  Questions posted here will be deleted.