Discussion API Performance

Initial observations:

from NewRelic Data from our current Forums service: 

• What is HttpDispatcher? 

  • Seems like it gets called per other request. Is it something that connects services?

• Sinatra::Application/GET api/v1/users/([^/?#]+) and Sinatra::Application/GET api/v1/threads/([^/?#]+) are called the most and account for 2/3 of the time.
  • Users is called many times in conjuction with other calls. Discussion API may not have to do this.
• Notifications is not called that many times but it's average response time is the slowest and accounts for about 5% of the total time.
  • Talk to Kevin Falcone (Deactivated) about rabbits.
• The average of most responses are 250 ms or less. Standard Deviation varies a lot. 
  • Given these large SDs, we can hopefully be able to replicate the behaviour when changing some of the variables such as # of threads in a course, or size of the text body.

Seeding forums data vs productions data:

Process:

MA-1099 - Getting issue details... STATUS  

Part 1: Confirm performance factors for Discussion API (Endpoints are below)

MA-1102 - Getting issue details... STATUS  

Part 2: Define data to be used in load tests.

• Leaning towards seeding the forums data. 
  • Write a script to seed data
• Example: Measure out # of threads if threads are a factor and comparing it to what is in production and using some intuition. If a course has 1000 threads, than do some tests with 0, 100,1000,10000 threads? 

MA-1042 - Getting issue details... STATUS  

Part 3: Create loadtest file and do single flow test

• Create locustfile for single endpoints
• Use loadtest env
• Get preliminary results for any glaring issues

MA-1082 - Getting issue details... STATUS

Part 4: Take NewRelic forums data and create+run flow test

• Take NewRelic data and run a Flowtest

Endpoints: 

• /courses/(course_id)/
  •  GET depends on # of courses
• /course_topics/(course_id)
  • GET depends on # of topics, threads, children
•  /threads/
  • GET parameters below
    • course_id depends on # of threads/comments
    • topic_id depends on # of threads/topics (Is this faster than a normal search?)
    • following, view, test_search should be the same, except it has a filter
    • order_by might be the same as the other fitlers, or faster
    • order_direction, page_size, page shouldn't matter
  • POST parameters should not affect performance
• /threads/(thread_id)
  • GET Not implemented
  •  PATCH/DELETE Should be fast? Depends on # of threads?
•  /comments
  • GET parameters
    • thread_id depends on # of threads
    • endorsed should be the same, except with filter
    • page_size, page, shouldn't matter
  • POST parameters
    • thread_id depends on the # of threads
•  /comments/(comment_id)
  • GET not implemented
  • PATCH 
  • DELETE

Concerns:

• If an endpoint that creates Threads/Comments varies in performance depending on the # of Threads/Comments, how would we go about this test?
  • Revisit this if this actually happens. Else, don't worry about this for now.
• Will existing forums data affect performance of newly seeded data? Do we need to wipe the database?
  • Figure this out
  • TnL testing did not test for this

Notes:

• Ideally, these test would be part of the new open sourced edx load tests repo
• dfriedmanR (Deactivated) and Diana Huang are doing performance work on the ruby code of the forums
• Brian Beggs is upgrading the mongo database so the ruby forums will be updated in the process. 
• If DAPI and TnL performance tests vary too much, an LMS test may also be required.

July 28th - Aug 3rd NewRelic data for the Sinatra Ruby Forums service.