CCX Loadtesting Notes

Owned by Calen Pennington (Deactivated)
Last updated: May 24, 2023 by Brian Mesick
3 min read

On Matasano LT Environment

cpennington-load.sandbox.edx.org

env COURSE_ID='course-v1:edX+DemoX+Demo_Course' sudo -E -u locust ../../../venvs/locust/bin/locust -f locustfile.py --no-web -r 40 -c 400 -n 1200 --host=https://courses-matasano.edx.org

12 boxes in production, 4 in matasano

18k rpm in production edxapp => 300 rps => 100 rps in matasano

 

AMI:  BuildID: cale-matasano-edge-edxapp-1440698168, Message: Finished baking AMI ami-1fcf7474 for matasano edge edxapp.


Couldn't get past an issue where saving a course in Studio would fail with a MongoDB integer overflow error

On Loadtest Environment

cpennington-load.sandbox.edx.org

12 boxes in production, 4 in loadtest

 

18k rpm in production edxapp => 300 rps => 100 rps in matasano
BuildID: cale-loadtest-edx-edxapp-1441721622, Message: Finished baking AMI ami-afbed1ca for loadtest edx edxapp.

env COURSE_ID='course-v1:edX+DemoX+Demo_Course' sudo -E -u locust ../../../venvs/locust/bin/locust -f locustfile.py --no-web -r 2 -c 400 -n 1200 --host=https://courses-loadtest.edx.org

Test Plan

Using the following command line, with a different course id for each course, run the load test against 1) a non-ccx course, 2) a ccx base course, and 3) a ccx instance. Each of these should be freshly created for the official run.

(Use cpennington-load.sandbox.edx.org)

env COURSE_ID='course-v1:edX+DemoX+demo-ccx-baseline' sudo -E -u locust ../../../venvs/locust/bin/locust -f locustfile.py --no-web -r 8 -c 800 -n 20000 --host=https://courses-loadtest.edx.org --precision 3 --write-stats

Upload the stats output CSVs (with labels) to this page, and paste the results into the wiki for comparison.

Comparison criteria:

  1. Median and 95% throughput for all measured pages shouldn't vary by more than the confidence interval.

 

Results

Tarball: 

IPython Notebook with Analysis: