Reduce Queries when Saving XBlock State in FieldDataCache
Pull Request: https://github.com/edx/edx-platform/pull/7258
Description
DjangoKeyValueStore.set_many
was causing multiple writes whenever it saved data for a newly created XBlock. It called FieldDataCache.find_or_create
which, if the block wasn't cached, did a get_or_create
call to create it. Then DjangoKeyValueStore
immediately called save
on that block. This results in both an insert of empty state, and then an update of that record. This change means that FieldDataCache
no longer calls get_or_create
and instead just returns a new, in-memory django model object, which is then saved by DjangoKeyValueStore
.
Method
Load Driver: https://github.com/edx/load-tests/tree/f9542ac4159928f155f27a835d3ba92783c1c15e/locust/lms
The test script was run (with a larger number of users) against revision 13ba2c4
, and then was run against the candidate revision af12b1b
, and then again against the baseline revision 13ba2c4
.
Results
The raw results are attached to the page. An analysis comparing each of the runs against the others as attached in .ods format. Those comparisons are also reproduced below.
Summarizing, we can draw several important conclusions:
- It's likely that the candidate revision isn't causing performance degradation.
- Candidate beat's Re-baseline by more than the difference between how much both Re-baseline and Candidate lose to Baseline
- Our test infrastructure isn't providing enough isolation to make tests repeatable.
- Re-baseline is slower than baseline, despite running the same driver and server code.
Name | # reqs | 50.00% | 66.00% | 75.00% | 80.00% | 90.00% | 95.00% | 98.00% | 99.00% | 100.00% | |
---|---|---|---|---|---|---|---|---|---|---|---|
Candidate/Re-baseline | POST handler:video:save_user_state | 5110.5 | -7.94% | -8.96% | -8.70% | -9.86% | -11.69% | -12.20% | -8.70% | 20.00% | 141.68% |
GET handler:capa:problem_get | 3437 | -16.87% | -16.67% | -15.79% | -14.14% | -10.00% | -8.33% | -7.69% | 66.67% | 175.93% | |
GET courseware:index | 3416.5 | - | 1.59% | -1.47% | -1.43% | -3.95% | -7.78% | 9.09% | - | 45.95% | |
POST handler:capa:problem_get | 2774.5 | -1.79% | -3.28% | -7.35% | -13.16% | -11.36% | -10.10% | -9.09% | -8.33% | 109.00% | |
GET handler:video:get_transcript | 2518 | - | - | - | - | -1.56% | -1.47% | -1.23% | -9.09% | 134.47% | |
GET courseware:mktg_course_about | 2235 | -1.82% | -1.69% | -1.64% | -1.61% | -1.54% | 4.29% | 32.65% | 7.69% | 88.91% | |
POST handler:goto_position | 1877.5 | -6.74% | -9.09% | - | -8.33% | -7.69% | -7.14% | - | 26.67% | 150.42% | |
GET handler:goto_position | 1805 | -10.10% | -8.33% | -8.33% | -7.69% | -7.14% | -13.33% | -12.50% | -5.88% | 98.89% | |
GET handler:capa:problem_check | 1762.5 | -13.25% | -15.22% | -13.27% | -11.00% | -11.82% | -16.67% | -21.43% | -18.75% | 54.87% | |
GET forums:single_thread | 724.5 | -4.41% | -5.48% | -5.33% | -3.90% | -2.38% | 18.18% | 12.50% | 20.00% | -4.72% | |
GET courseware:course_info | 694 | -1.05% | -1.00% | - | - | - | - | - | -57.58% | 68.25% | |
GET handler:video:save_user_state | 599 | -1.92% | - | - | - | -1.64% | -1.49% | -3.45% | -38.13% | 1.49% | |
POST handler:capa:problem_check | 468 | -3.23% | -8.00% | -9.76% | -10.47% | -12.37% | -18.18% | -23.85% | -21.43% | 404.52% | |
GET handler:capa:problem_show | 428 | -15.00% | -17.98% | -18.09% | -17.17% | -12.73% | -16.67% | - | 53.33% | 941.87% | |
GET auto_auth | 395.5 | - | - | - | 4.55% | 4.17% | 55.56% | 36.36% | 5.56% | 1.91% | |
POST enroll | 394 | -2.15% | -3.00% | - | -9.09% | - | -8.33% | - | - | 317.68% | |
GET forums:forum_form_discussion | 294.5 | - | - | -1.69% | - | 4.55% | 12.50% | 6.67% | 20.00% | 5.36% | |
GET handler:capa:problem_save | 277 | -14.81% | -15.73% | -18.75% | -19.19% | -12.73% | -16.67% | -15.38% | - | 49.35% | |
GET courseware:progress | 268 | -3.70% | -9.78% | -13.00% | -18.18% | -19.17% | -28.57% | -6.25% | -15.79% | -21.07% | |
GET forums:inline_discussion | 209.5 | -3.45% | -3.23% | - | -2.94% | -2.70% | -13.56% | -28.57% | - | -9.94% | |
GET forums:search_topic | 80 | - | -2.50% | -4.76% | -2.38% | -6.38% | -18.97% | -20.63% | -66.84% | -66.67% | |
GET courseware:about | 69 | - | - | - | - | 6.67% | 137.50% | 900.00% | 900.00% | 887.28% | |
POST handler:capa:problem_show | 27.5 | -12.68% | -8.00% | -15.85% | -16.87% | -9.78% | -10.42% | -23.08% | -23.08% | -21.97% | |
GET forums:get_user_profile | 19 | - | -4.17% | -4.17% | - | - | 83.33% | 83.33% | 83.33% | 80.78% | |
POST forums:create_comment | 17.5 | - | 3.33% | 3.33% | 3.33% | -8.57% | -2.78% | -2.78% | -2.78% | -2.24% | |
GET forums:followed_threads | 12.5 | -7.14% | -6.67% | -26.32% | -56.25% | -88.46% | -88.57% | -88.57% | -88.57% | -88.75% | |
POST handler:capa:problem_save | 6 | -24.44% | -22.22% | -29.09% | -29.09% | -9.09% | -9.09% | -9.09% | -9.09% | -3.81% | |
POST forums:create_thread | 2 | 14.29% | 14.29% | 14.29% | 14.29% | 14.29% | 14.29% | 14.29% | 14.29% | 12.15% | |
Name | # reqs | 50% | 66% | 75% | 80% | 90% | 95% | 98% | 99% | 100% | |
Candidate/Baseline | POST handler:video:save_user_state | 10986.5 | 11.54% | 12.96% | 12.50% | 12.28% | 11.48% | 12.50% | 18.31% | 39.53% | 3.54% |
GET handler:capa:problem_get | 4405.5 | - | 1.35% | 2.56% | 4.94% | 3.13% | 10.00% | 9.09% | 108.33% | 13.33% | |
GET courseware:index | 8196.5 | 6.67% | 8.47% | 6.35% | 6.15% | 7.35% | 12.16% | 30.43% | 7.69% | 4.48% | |
POST handler:capa:problem_get | 8503 | 10.00% | 3.51% | -4.55% | -4.35% | - | -1.11% | - | - | -16.83% | |
GET handler:video:get_transcript | 5316 | 20.00% | 18.75% | 20.41% | 20.00% | 18.87% | 21.82% | 31.15% | 42.86% | -6.19% | |
GET courseware:mktg_course_about | 5481 | 12.50% | 13.73% | 15.38% | 17.31% | 18.52% | 30.36% | 113.11% | 79.49% | -4.23% | |
POST handler:goto_position | 5405.5 | 9.21% | 5.26% | 10.00% | - | - | 8.33% | 7.69% | 35.71% | 1.66% | |
GET handler:goto_position | 2225.5 | 4.71% | 10.00% | - | 9.09% | 8.33% | - | - | 14.29% | 96.50% | |
GET handler:capa:problem_check | 2841.5 | 4.35% | 4.00% | 7.59% | 5.95% | 3.19% | - | - | 8.33% | -1.86% | |
GET forums:single_thread | 1838.5 | 3.17% | 4.55% | 2.90% | 4.23% | 5.13% | 38.30% | 12.50% | 14.29% | -28.79% | |
GET courseware:course_info | 1678 | 9.30% | 11.24% | 9.89% | 8.70% | 15.79% | 11.11% | 18.18% | - | -9.50% | |
GET handler:video:save_user_state | 661.5 | 15.91% | 17.02% | 18.75% | 20.83% | 20.00% | 24.53% | 42.37% | 62.30% | 12.79% | |
POST handler:capa:problem_check | 1836.5 | 5.26% | 2.99% | 5.71% | 6.94% | 3.66% | -1.10% | -1.00% | - | -10.73% | |
GET handler:capa:problem_show | 830 | 7.94% | 4.29% | 2.67% | 5.13% | 4.35% | - | 18.18% | 91.67% | 27.64% | |
GET auto_auth | 792.5 | 10.53% | 10.00% | 10.00% | 9.52% | 13.64% | 61.54% | 7.14% | 11.76% | -4.98% | |
POST enroll | 791 | 22.97% | 24.36% | 23.46% | 21.95% | 30.95% | 25.00% | 26.32% | 18.18% | -60.47% | |
GET forums:forum_form_discussion | 662 | 1.89% | 1.82% | 1.75% | 3.45% | 11.29% | 28.57% | 14.29% | 26.32% | 1.07% | |
GET handler:capa:problem_save | 548.5 | 6.15% | 5.63% | 2.63% | 1.27% | 5.49% | - | - | 27.27% | -88.92% | |
GET courseware:progress | 692 | 9.86% | 10.67% | 11.54% | 9.76% | - | -9.09% | 7.14% | 6.67% | -7.62% | |
GET forums:inline_discussion | 505 | 7.69% | 11.11% | 10.34% | 10.00% | 9.09% | 18.60% | -9.09% | - | -14.96% | |
GET forums:search_topic | 181.5 | 11.76% | 11.43% | 11.11% | 10.81% | 7.32% | 4.44% | -24.24% | -62.94% | -70.99% | |
GET courseware:about | 163.5 | 16.67% | 7.69% | 15.38% | 15.38% | 23.08% | 171.43% | 1114.29% | 962.50% | 7.56% | |
POST handler:capa:problem_show | 113.5 | 12.73% | 15.00% | 4.55% | -1.43% | 5.06% | - | 8.70% | -9.09% | -33.97% | |
GET forums:get_user_profile | 36.5 | - | - | -4.17% | - | - | 62.96% | 41.94% | 25.71% | 24.98% | |
POST forums:create_comment | 28 | -3.33% | - | -3.13% | -6.06% | -5.88% | 2.94% | -2.78% | -2.78% | -3.86% | |
GET forums:followed_threads | 30.5 | - | - | - | -6.67% | -25.00% | -23.81% | -38.46% | -38.46% | -40.68% | |
POST handler:capa:problem_save | 33.5 | 13.33% | 7.69% | 9.86% | 5.41% | 21.95% | 14.94% | 11.11% | 8.70% | 9.78% | |
POST forums:create_thread | 12 | 33.33% | 33.33% | 33.33% | 26.32% | 20.00% | 14.29% | 4.35% | 4.35% | 6.19% | |
Name | # reqs | 50% | 66%` | 75% | 80% | 90% | 95% | 98% | 99% | 100% | |
Re-baseline/Baseline | POST handler:video:save_user_state | 10792 | 21.15% | 24.07% | 23.21% | 24.56% | 26.23% | 28.13% | 29.58% | 16.28% | -57.16% |
POST handler:capa:problem_get | 8464.5 | 12.00% | 7.02% | 3.03% | 10.14% | 12.82% | 10.00% | 10.00% | 9.09% | -60.21% | |
GET courseware:index | 8426 | 6.67% | 6.78% | 7.94% | 7.69% | 11.76% | 21.62% | 19.57% | 7.69% | -28.41% | |
GET courseware:mktg_course_about | 5597 | 14.58% | 15.69% | 17.31% | 19.23% | 20.37% | 25.00% | 60.66% | 66.67% | -49.30% | |
POST handler:goto_position | 5390 | 17.11% | 15.79% | 10.00% | 9.09% | 8.33% | 16.67% | 7.69% | 7.14% | -59.40% | |
GET handler:video:get_transcript | 5257 | 20.00% | 18.75% | 20.41% | 20.00% | 20.75% | 23.64% | 32.79% | 57.14% | -59.99% | |
GET handler:capa:problem_get | 4354.5 | 20.29% | 21.62% | 21.79% | 22.22% | 14.58% | 20.00% | 18.18% | 25.00% | -58.93% | |
GET handler:capa:problem_check | 2800 | 20.29% | 22.67% | 24.05% | 19.05% | 17.02% | 20.00% | 27.27% | 33.33% | -36.63% | |
GET handler:goto_position | 2186.5 | 16.47% | 20.00% | 9.09% | 18.18% | 16.67% | 15.38% | 14.29% | 21.43% | -1.20% | |
GET forums:single_thread | 1897 | 7.94% | 10.61% | 8.70% | 8.45% | 7.69% | 17.02% | - | -4.76% | -25.26% | |
POST handler:capa:problem_check | 1833.5 | 8.77% | 11.94% | 17.14% | 19.44% | 18.29% | 20.88% | 30.00% | 27.27% | -82.31% | |
GET courseware:course_info | 1706 | 10.47% | 12.36% | 9.89% | 8.70% | 15.79% | 11.11% | 18.18% | 135.71% | -46.21% | |
GET handler:capa:problem_show | 806 | 26.98% | 27.14% | 25.33% | 26.92% | 19.57% | 20.00% | 18.18% | 25.00% | -87.75% | |
GET auto_auth | 783 | 10.53% | 10.00% | 10.00% | 4.76% | 9.09% | 3.85% | -21.43% | 5.88% | -6.76% | |
POST enroll | 782 | 25.68% | 28.21% | 23.46% | 34.15% | 30.95% | 36.36% | 26.32% | 18.18% | -90.54% | |
GET courseware:progress | 703 | 14.08% | 22.67% | 28.21% | 34.15% | 23.71% | 27.27% | 14.29% | 26.67% | 17.05% | |
GET forums:forum_form_discussion | 682.5 | 1.89% | 1.82% | 3.51% | 3.45% | 6.45% | 14.29% | 7.14% | 5.26% | -4.07% | |
GET handler:video:save_user_state | 654.5 | 18.18% | 17.02% | 18.75% | 20.83% | 22.00% | 26.42% | 47.46% | 162.30% | 11.14% | |
GET handler:capa:problem_save | 539.5 | 24.62% | 25.35% | 26.32% | 25.32% | 20.88% | 20.00% | 18.18% | 27.27% | -92.58% | |
GET forums:inline_discussion | 528.5 | 11.54% | 14.81% | 10.34% | 13.33% | 12.12% | 37.21% | 27.27% | - | -5.57% | |
GET forums:search_topic | 189.5 | 11.76% | 14.29% | 16.67% | 13.51% | 14.63% | 28.89% | -4.55% | 11.76% | -12.96% | |
GET courseware:about | 169.5 | 16.67% | 7.69% | 15.38% | 15.38% | 15.38% | 14.29% | 21.43% | 6.25% | -89.11% | |
POST handler:capa:problem_show | 113 | 29.09% | 25.00% | 24.24% | 18.57% | 16.46% | 11.63% | 41.30% | 18.18% | -15.38% | |
GET forums:get_user_profile | 38.5 | - | 4.35% | - | - | - | -11.11% | -22.58% | -31.43% | -30.87% | |
POST handler:capa:problem_save | 31.5 | 50.00% | 38.46% | 54.93% | 48.65% | 34.15% | 26.44% | 22.22% | 19.57% | 14.13% | |
GET forums:followed_threads | 31 | 7.69% | 7.14% | 35.71% | 113.33% | 550.00% | 566.67% | 438.46% | 438.46% | 427.38% | |
POST forums:create_comment | 30.5 | -3.33% | -3.23% | -6.25% | -9.09% | 2.94% | 5.88% | - | - | -1.65% | |
POST forums:create_thread | 12 | 16.67% | 16.67% | 16.67% | 10.53% | 5.00% | - | -8.70% | -8.70% | -5.31% |