Running Open EdX in Production

Owned by Fred Smith (Deactivated)
Mar 02, 2016
2 min read

 

Small Single Server Installation:

Assumptions:

  • You are using a mac
  • You know git, brew, pip, virtualenv and have it working
  • You have activated a NEW virtualenv and are working inside that NEW env
  • You have an ubuntu precise box running somewhere on the public internet
  • You have root access to that box
  • You have your private key ~/.ssh/id_rsa stored safely in your mac

Prepare the ubuntu precise box - has at least 4GB RAM

sudo apt-get update -y
sudo apt-get upgrade -y
sudo reboot
sudo apt-get install -y build-essential software-properties-common python-software-properties curl git-core libxml2-dev libxslt1-dev libfreetype6-dev python-pip python-apt python-dev
sudo pip install --upgrade pip
sudo pip install --upgrade virtualenv

On your mac

git clone https://github.com/edx/configuration.git
cd configuration
git checkout named-release/birch
pip install -r requirements.txt
cd playbooks

Edit the filed named inventory.ini and replace the ip address with the ip address of your precise box and run this command.

ansible-playbook ./vagrant-fullstack.yml \
-i inventory.ini -u root \
--private-key ~/.ssh/id_rsa \
-e edx_platform_version=named-release/birch \
-e ora2_version=named-release/birch \
-e certs_version=named-release/birch \
-e forum_version=named-release/birch \
-e xqueue_version=named-release/birch \
-e configuration_version=named-release/birch

Minimal scalable reference architecture deployment

openssl genrsa 2048 > test_private_key.pem
openssl req -new -key test_private_key.pem -out test_csr.pem
openssl x509 -req -days 365 -in test_csr.pem -signkey test_private_key.pem -out wekanban.crt
brew install awscli
aws iam upload-server-certificate --server-certificate-name wekanban --certificate-body file://wekanban.crt --private-key file://test_private_key.pem
  {
      "ServerCertificateMetadata": {
          "ServerCertificateId": "ASCAIVCA5BMRD5YUHLKY2",
          "ServerCertificateName": "wekanban",
          "Expiration": "2016-03-03T03:51:00Z",
          "Path": "/",
          "Arn": "arn:aws:iam::320670401404:server-certificate/wekanban",
          "UploadDate": "2015-03-04T03:51:37.847Z"
      }
  }

edit edx/configuration/cloudformation_templates/edx-reference-architecture.json and replace arn by `arn:aws:iam::320670401404:server-certificate/wekanban`
brew install s3cmd
s3cmd put /Users/nilesh/Curriculum/edx/configuration/cloudformation_templates/edx-reference-architecture.json s3://cloudgenius
aws cloudformation create-stack --stack-name cloudgenius \
   --template-url https://s3.amazonaws.com/cloudgenius/edx-reference-architecture.json \
   --capabilities CAPABILITY_IAM \
   --region=us-east-1
{
   "StackId": "arn:aws:cloudformation:us-east-1:320670401404:stack/cloudgenius/37246d40-c294-11e4-9dba-50fa5262a89c"
}

aws cloudformation describe-stack-events --stack-name cloudgenius --region=us-east-1
aws cloudformation describe-stack-resources --stack-name cloudgenius --region=us-east-1

chmod 400 deployment.pem
ssh-add deployment.pem
cd configuration/playbooks
virtualenv /tmp/venv/configuration
source /tmp/venv/configuration/bin/activate
pip install -r ../requirements.txt

python ../util/vpc-tools/vpc-tools.py ssh-config stack-name cloudgenius \
identity-file /Users/nilesh/Curriculum/edx/configuration/deployment.pem \
user ubuntu jump-box ec2-54-173-30-232.compute-1.amazonaws.com > ~/.ssh/config

ansible-playbook -c ssh -vvv --user=ubuntu edx-east/edxapp.yml -i ./ec2.py  -e 'region=us-east-1 key=deployment name=cloudgenius group=cloudgenius' -e@/path/to/edx-secret/server-vars.yml

if cloudformation gives you trouble:

Value (us-east-1b) for parameter availabilityZone is invalid. Subnets can currently only be created in the following availability zones: us-east-1e, us-east-1c, us-east-1a, us-east-1d.

change 1-b to 1-a

s3cmd put /Users/nilesh/Curriculum/edx/configuration/cloudformation_templates/edx-reference-architecture.json s3://cloudgenius

and visit cloudformation again https://s3.amazonaws.com/cloudgenius/edx-reference-architecture.json

Your requested instance type (m1.small) is not supported in your requested Availability Zone (us-east-1c). Please retry your request by not specifying an Availability Zone or choosing us-east-1d, us-east-1a, us-east-1e, us-east-1b.

Change us-east-1c to us-east-1e

s3cmd put /Users/nilesh/Curriculum/edx/configuration/cloudformation_templates/edx-reference-architecture.json s3://cloudgenius

and then visit cloudformation again https://s3.amazonaws.com/cloudgenius/edx-reference-architecture.json

edX.org Deployment

EdX.org is deployed via Amazon's cloudformation service to Amazon EC2. You can see the cloudformation template athttps://github.com/edx/configuration/blob/master/cloudformation_templates/edx-reference-architecture.json.

As of 2015-02-18, the deployment looks like this:

EdXApp Servers (10 c3.4xlarge instances)

Forum Servers (2 m1.large instances)

CommonCluster Servers (3 m3.large instances)

Worker Servers (2 m2.4xlarge instances)

xqwatcher Servers (2 m3.medium instances)

Insights Servers (2 m3.medium instances)

Certificates Servers (1 m3.medium instance)

MongoDB Hosted via Compose.io

MySQL Hosted via AWS RDS (Multi-AZ deployment of size db.m2.4xlarge)

  • Can also be deployed using a cluster of any mysql compatible database server.

Misc

  • Various legacy graders
  • Analytics related servers
  • Admin & Monitoring servers