Forums Performance

TODO:

Add links to other performance information, especially under mobile.
Add documentation for performance testing

Setting up Discussions Load Testing Sandbox

Provision LMS/Discussions Sandbox

Provision sandbox using Jenkins (requires VPN)
- Set dns_name and name_tag.
- Set sandbox life of you need for longer than a week.
- Set reconfigure checkbox IF adding New Relic (costs $$$, ask first).
- Set instance_type to m4.large.
- Set enable_newrelic IF adding New Relic (costs $$$, ask first).
- For a Ruby 2.3 sandbox
  
  We are likely upgrading to Ruby 2.3, so this section will probably go away shortly.
  - Rebase configuration branch robrap/ruby2 with updates in master
  - Set configuration_version to robrap/ruby2
  - Rebase cs_comments_service branch robrap/ruby2 with updates in master
  - Set forum_version to robrap/ruby2

Manual changes to Sandbox (SSH)

Update the sandbox for auto auth:

sudo vi /edx/app/edxapp/lms.env.json
# Set AUTOMATIC_AUTH_FOR_TESTING to true

# restart application
sudo /edx/bin/supervisorctl restart edxapp:*

Update mongo indices:

mongo
> use cs_comments_service_development
> # COPY contents of edx-load-tests/discussions_api/db/create_indexes.js and PASTE in shell
> exit

Fix Forums to work with New Relic:

sudo vi /edx/app/forum/cs_comments_service/config/newrelic.yml
# Under development:
# - Set monitor_mode to true
# - Set developer_mode to false

# Restart application
sudo /edx/bin/supervisorctl restart forum

Other resources

Performance Testing for Sandboxes

Setting up a Locust box

Review How to Run Performance Tests. Also see edx-load-tests repo README.

Provision Locust Box

Provision locust box using Jenkins (requires VPN)
- Set loadtester_name.
- Set locust_target_host if testing against a sandbox.
- Leave instance_type as m3.medium (when should we upgrade to m4?).
  
  It's ok if it has problems starting the locust server, because you want to run from command line anyway.

Manual changes to Locust Box (SSH)

In case locust is running:

sudo /edx/bin/supervisorctl stop locust

Update branch (if needed):

LMS tests require updates on branch in this PR: https://github.com/edx/edx-load-tests/pull/60

This note can be removed once this has been merged.

cd /edx/app/locust/load-tests
git remote -v
# if you don't see https://github.com/edx/edx-load-tests.git
git remote set-url origin https://github.com/edx/edx-load-tests.git

# run necessary git commands
sudo git pull

Setup to run locust:

cd /edx/app/locust
source venvs/locust/bin/activate
cd load-tests

Seed long thread (after setup above):

# Seed long thread (Example Sandbox):
sudo vi settings_files/lms.yml
  # Update BASIC_AUTH_USER, BASIC_AUTH_PASSWORD
  # Update COURSE_ID: course-v1:edX+DemoX+Demo_Course (or any existing course)
  # Update LOCUST_TASK_SET: SeedForumsTasks
locust --host="https://discussions.sandbox.edx.org" -f lms -c 10 -r 1 -n 4500 --no-web
  # Capture output of LARGE_TOPIC_ID=AAA LARGE_THREAD_ID=BBB
 
# Seed long thread (LoadTest Environment):
sudo vi settings_files/lms.yml
  # Comment out BASIC_AUTH_USER, BASIC_AUTH_PASSWORD
  # Update COURSE_ID: course-v1:edX+DemoX+demo-ccx-baseline (or any existing course)
  # Update LOCUST_TASK_SET: SeedForumsTasks
locust --host="https://courses-loadtest.edx.org" -f lms -c 10 -r 1 -n 4500 --no-web
  # Capture output of LARGE_TOPIC_ID=AAA LARGE_THREAD_ID=BBB

Run Forums Tests (after setup above):

# Run tests against sample sandbox (using large thread):
sudo vi settings_files/lms.yml
  # Update BASIC_AUTH_USER, BASIC_AUTH_PASSWORD
  # Update COURSE_ID: course-v1:edX+DemoX+Demo_Course (or any existing course)
  # Update LOCUST_TASK_SET: ForumsTasks
  # Update LARGE_TOPIC_ID and LARGE_THREAD_ID with output from SeedForumsTasks
locust --host="https://discussions.sandbox.edx.org" -f lms -c 10 -r 1 -n 3000 --no-web

Determining Weightings

Run script

There is a determine weightings script which can be run on a Splunk export to get information that can be used to determine the request call distribution for the lms locust tests.

This weighting script should be migrated to a Splunk dashboard and the script should be deleted at some point.

To run the script, first run the following Splunk query:

index="prod-edx" sourcetype=nginx URI="*/discussion/*" AND NOT URI="*/static/css/discussion/*"

Export the results as a CSV, and then run the following:

cd edx-load-tests/lms/scripts
./determine_call_distribution.py discussions-2016-06-13.csv

A sample file discussions-2016-06-13.csv is attached to this page.

Update Weightings

The newest weightings were taken directly from the output of the script for some calls. The percentage of heaviest read threads, which were large welcome threads on a very popular course, helped determine the ultimate read large thread distribution. Some other adjustments were made, like mobile api calls were temporarily ignored. Also, some weight was given to the original weightings.

The final weightings can be seen in the ForumsTasks for the lms locust tests.

	Original Weightings			New Weightings
Read thread count: 49088 (58%)	45	103	44%	36	93	35%
Create thread count: 2439 (2%)	4		4%	4		4%
Update thread count: 99 (0%)
Create response count: 3866 (4%)	3		3%	4		4%
Create comment count: 791 (0%)
Update comment count: 150 (0%)
Forum search count: 1522 (1%)	5		5%	4		4%
Read inline count: 7511 (8%)	15		15%	10		10%
User profile count: 1123 (1%)	1		1%	1		1%
User count (???): 644 (0%)
Forum tab count: 6169 (7%)	19		18%	10		10%
Topic filter count: 3101 (3%)
Mobile API count: 7012 (8%)
Other count: 762 (0%)
TOTAL count: 84277
Followed Threads	1		1%	1		1%
Large Thread	10		10%	23		22%