Discussion API Performance

Initial observations:

from NewRelic Data from our current Forums service: 

• What is HttpDispatcher? 

  • Seems like it gets called per other request. Is it something that connects services?

• Sinatra::Application/GET api/v1/users/([^/?#]+) and Sinatra::Application/GET api/v1/threads/([^/?#]+) are called the most and account for 2/3 of the time.

  • Users is called many times in conjuction with other calls. Discussion API may not have to do this.

• Notifications is not called that many times but it's average response time is the slowest and accounts for about 5% of the total time.

  • Talk to @Kevin Falcone (Deactivated) about rabbits.

• The average of most responses are 250 ms or less. Standard Deviation varies a lot. 

  • Given these large SDs, we can hopefully be able to replicate the behaviour when changing some of the variables such as # of threads in a course, or size of the text body.

Seeding forums data vs productions data:

Process:

https://openedx.atlassian.net/browse/MA-1099 

Part 1: Confirm performance factors for Discussion API (Endpoints are below)

https://openedx.atlassian.net/browse/MA-1102 

Part 2: Define data to be used in load tests.

• Leaning towards seeding the forums data. 

  • Write a script to seed data

• Example: Measure out # of threads if threads are a factor and comparing it to what is in production and using some intuition. If a course has 1000 threads, than do some tests with 0, 100,1000,10000 threads? 

https://openedx.atlassian.net/browse/MA-1042 

Part 3: Create loadtest file and do single flow test

• Create locustfile for single endpoints

• Use loadtest env

• Get preliminary results for any glaring issues

https://openedx.atlassian.net/browse/MA-1082

Part 4: Take NewRelic forums data and create+run flow test

• Take NewRelic data and run a Flowtest

Endpoints: 

• /courses/(course_id)/

  •  GET depends on # of courses

• /course_topics/(course_id)

  • GET depends on # of topics, threads, children

•  /threads/

  • GET parameters below

    • course_id depends on # of threads/comments

    • topic_id depends on # of threads/topics (Is this faster than a normal search?)

    • following, view, test_search should be the same, except it has a filter

    • order_by might be the same as the other fitlers, or faster

    • order_direction, page_size, page shouldn't matter

  • POST parameters should not affect performance

• /threads/(thread_id)

  • GET Not implemented

  •  PATCH/DELETE Should be fast? Depends on # of threads?

•  /comments

  • GET parameters

    • thread_id depends on # of threads

    • endorsed should be the same, except with filter

    • page_size, page, shouldn't matter

  • POST parameters

    • thread_id depends on the # of threads

•  /comments/(comment_id)

  • GET not implemented

  • PATCH 

  • DELETE

Concerns:

• If an endpoint that creates Threads/Comments varies in performance depending on the # of Threads/Comments, how would we go about this test?

  • Revisit this if this actually happens. Else, don't worry about this for now.

• Will existing forums data affect performance of newly seeded data? Do we need to wipe the database?

  • Figure this out

  • TnL testing did not test for this

Notes:

• Ideally, these test would be part of the new open sourced edx load tests repo

• @dfriedmanR (Deactivated) and @Diana Huang are doing performance work on the ruby code of the forums

• @Brian Beggs is upgrading the mongo database so the ruby forums will be updated in the process. 

• If DAPI and TnL performance tests vary too much, an LMS test may also be required.

July 28th - Aug 3rd NewRelic data for the Sinatra Ruby Forums service.