Reduce Queries when Saving XBlock State in FieldDataCache

Pull Request: https://github.com/edx/edx-platform/pull/7258

Description

DjangoKeyValueStore.set_many was causing multiple writes whenever it saved data for a newly created XBlock. It called FieldDataCache.find_or_create which, if the block wasn't cached, did a get_or_create call to create it. Then DjangoKeyValueStore immediately called save on that block. This results in both an insert of empty state, and then an update of that record. This change means that FieldDataCache no longer calls get_or_create and instead just returns a new, in-memory django model object, which is then saved by DjangoKeyValueStore.

Method

Load Driver: https://github.com/edx/load-tests/tree/f9542ac4159928f155f27a835d3ba92783c1c15e/locust/lms

#! /bin/bash
TITLE=$1
echo "STARTING LMS TEST"
cd /edx/app/locust/load-tests/locust/lms
locust --host http://courses-loadtest.edx.org --no-web -c 300 -r 5 -n 30000 --print-stats &> /home/cpennington/perf-results/lms.${TITLE}.$(date -Ins).txt
echo "STARTING VIDEO TEST"
cd /edx/app/locust/load-tests/locust/mobile_api
for users in 10 20 30 40 50 100 150 200 300 400 450 460 500; do
    echo "STARTING ${users}"
    locust --host http://courses-loadtest.edx.org --no-web -c ${users} -r 5 -n ${users}00 --print-stats &> /home/cpennington/perf-results/video.${users}.${TITLE}.$(date -Ins).txt
done
echo "TEST COMPLETE"

The test script was run (with a larger number of users) against revision 13ba2c4, and then was run against the candidate revision af12b1b, and then again against the baseline revision 13ba2c4.

Results

The raw results are attached to the page. An analysis comparing each of the runs against the others as attached in .ods format. Those comparisons are also reproduced below.

Summarizing, we can draw several important conclusions:

1. It's likely that the candidate revision isn't causing performance degradation.
   
   1. Candidate beat's Re-baseline by more than the difference between how much both Re-baseline and Candidate lose to Baseline
2. Our test infrastructure isn't providing enough isolation to make tests repeatable.
   1. Re-baseline is slower than baseline, despite running the same driver and server code.