Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

With the addition of new query param (i.e. requested_fields=profile_image) to discussion API GET endpoints for thread and comment; the discussion APIs got exposed to the possibility of performance degradation because this new param fetches and prepares the profile image related data of all the users that are participating in thread or comment.

We have gathered and compared two sets of load test results. The former results (i.e. with profile_image) are run on the latest changes and the later are run on older implementation (i.e. without profile_image).

There appears to be anomaly in the results below (highlighted 99%) where the new implementation is faster than the old. Where as, for other results the percentile of new implementation increases significantly with the increase in load. For example, the first pair of results where no. of clients = 48, the difference is in tens or hundrend and for the last pair of results where no. of clients = 610, the difference is in thousands.

Logically; there should only be increase in response time for GET endpoints, but we see in results that all other endpoints show differences too. This is because all PATCH, POST and DELETE endpoints first call GET endpoint to retrieve an 'id' of thread and/or comment and then make further processing on it.

New Relic:

Here is the new relic permaLink_without_profile and permaLink_with_profile where the average rpm for former is 1.55k and for later is 1.48k and 84.5% of requests are made for AccountViewSet.list (i.e. user accounts API for multiple usernames) for later.

 

With Profile Image

Without Profile Image

No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

 

  

DELETE_thread

 

  

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%
DELETE_comment

 

  
DELETE_thread   
GET_comment_list   
GET_thread   
GET_thread_list   
PATCH_comment   
PATCH_thread   
POST_comment_comment   
POST_comment_response   
POST_thread   
auto_auth   


No. of clients = 
req/s =
Methodsmedian response time95%99%
DELETE_comment   
DELETE_thread   
GET_comment_list   
GET_thread   
GET_thread_list   
PATCH_comment   
PATCH_thread   
POST_comment_comment   
POST_comment_response   
POST_thread  
auto_auth   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

2703501300

DELETE_thread

180210280

GET_comment_list

170250300

GET_thread

170230260

GET_thread_list

240590740

PATCH_comment

270350450

PATCH_thread

190260370

POST_comment_comment

330440650

POST_comment_response

280350400

POST_thread

170230250

auto_auth

220230230


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

270340380

DELETE_thread

180240360

GET_comment_list

150210330

GET_thread

160220300

GET_thread_list

170340480

PATCH_comment

260340370

PATCH_thread

150260270

POST_comment_comment

330410550

POST_comment_response

280360440

POST_thread

170230270

auto_auth

210210210


No. of clients = 192
req/s = 16.10
Methodsmedian response time95%99%

DELETE_comment

280370550

DELETE_thread

180230290

GET_comment_list

180260370

GET_thread

170240340

GET_thread_list

240610780

PATCH_comment

2603801100

PATCH_thread

170260300

POST_comment_comment

340430540

POST_comment_response

290370400

POST_thread

180230350

auto_auth

210220220


No. of clients = 192
req/s = 16
Methodsmedian response time95%99%

DELETE_comment

280360430

DELETE_thread

180290290

GET_comment_list

150210290

GET_thread

160220310

GET_thread_list

180350490

PATCH_comment

250340450

PATCH_thread

190260430

POST_comment_comment

330430550

POST_comment_response

280370470

POST_thread

170230290

auto_auth

220230230


No. of clients = 240
req/s = 18.70
Methodsmedian response time95%99%

DELETE_comment

290390470

DELETE_thread

180250350

GET_comment_list

180270400

GET_thread

180240300

GET_thread_list

240620800

PATCH_comment

270400860

PATCH_thread

2002702300

POST_comment_comment

340470760

POST_comment_response

290390760

POST_thread

180240390

auto_auth

220230230


No. of clients = 240
req/s = 19.7
Methodsmedian response time95%99%

DELETE_comment

280370460

DELETE_thread

180250300

GET_comment_list

150220280

GET_thread

160230280

GET_thread_list

180350500

PATCH_comment

260380500

PATCH_thread

190250380

POST_comment_comment

330420500

POST_comment_response

280370460

POST_thread

180240340

auto_auth

210220220


No. of clients = 288
req/s = 23.20
Methodsmedian response time95%99%

DELETE_comment

290380460

DELETE_thread

190290380

GET_comment_list

180270340

GET_thread

180240300

GET_thread_list

250630790

PATCH_comment

260350440

PATCH_thread

170240280

POST_comment_comment

340430560

POST_comment_response

290390470

POST_thread

180240320

auto_auth

210210210


No. of clients = 288
req/s = 22.2
Methodsmedian response time95%99%

DELETE_comment

2804201400

DELETE_thread

180240270

GET_comment_list

160240660

GET_thread

160240490

GET_thread_list

180360570

PATCH_comment

270360740

PATCH_thread

190260630

POST_comment_comment

340450620

POST_comment_response

2904401300

POST_thread

1802401500

auto_auth

220220220


No. of clients = 336
req/s = 26.70
Methodsmedian response time95%99%

DELETE_comment

290410570

DELETE_thread

1903201200

GET_comment_list

190270370

GET_thread

180250350

GET_thread_list

250630800

PATCH_comment

240370410

PATCH_thread

180270330

POST_comment_comment

350460560

OST_comment_response

300390470

POST_thread

180250320

auto_auth

240250250


No. of clients = 336
req/s = 28.2
Methodsmedian response time95%99%

DELETE_comment

2904402400

DELETE_thread

190260440

GET_comment_list

160240550

GET_thread

170240690

GET_thread_list

180370540

PATCH_comment

2703801700

PATCH_thread

1902801500

POST_comment_comment

3604802400

OST_comment_response

3004301300

POST_thread

180240290

auto_auth

240260260


No. of clients = 384
req/s = 31
Methodsmedian response time95%99%

DELETE_comment

300400890

DELETE_thread

190300320

GET_comment_list

190280410

GET_thread

180260360

GET_thread_list

260660830

PATCH_comment

230380430

PATCH_thread

200260400

POST_comment_comment

360500590

POST_comment_response

300410470

POST_thread

180250290

auto_auth

290360360


No. of clients = 384
req/s = 31.4
Methodsmedian response time95%99%
DELETE_comment300430550
DELETE_thread190250720
GET_comment_list160240350
GET_thread170240330
GET_thread_list190360510
PATCH_comment230380500
PATCH_thread180270320
POST_comment_comment360480590
POST_comment_response100430540
POST_thread180250370
auto_auth210270270


No. of clients = 480
req/s = 36.80
Methodsmedian response time95%99%

DELETE_comment

320470600

DELETE_thread

2002901300

GET_comment_list

210320450

GET_thread

190280410

GET_thread_list

270710900

PATCH_comment

290450530

PATCH_thread

200300380

POST_comment_comment

390550680

POST_comment_response

330470600

POST_thread

190280370

auto_auth

340350350


No. of clients = 480
req/s = 38.8
Methodsmedian response time95%99%
DELETE_comment320440860
DELETE_thread2003001300
GET_comment_list170260390
GET_thread170260370
GET_thread_list190390550
PATCH_comment280420640
PATCH_thread200290410
POST_comment_comment3805301400
POST_comment_response320450610
POST_thread190260410
auto_auth260270270


No. of clients = 510
req/s = 41.40
Methodsmedian response time95%99%

DELETE_comment

320480570

DELETE_thread

200340470

GET_comment_list

210330440

GET_thread

190290390

GET_thread_list

280710880

PATCH_comment

280400460

PATCH_thread

190320420

POST_comment_comment

390560690

POST_comment_response

330490590

POST_thread

190270400

auto_auth

230250250
No. of clients = 510
req/s = 40.4
Methodsmedian response time95%99%

DELETE_comment

320510780

DELETE_thread

190250360

GET_comment_list

170280450

GET_thread

180280630

GET_thread_list

200400640

PATCH_comment

290460740

PATCH_thread

200340850

POST_comment_comment

400580820

POST_comment_response

330510780

POST_thread

190270460

auto_auth

240250250
No. of clients = 544
req/s = 28.10
Methodsmedian response time95%99%

DELETE_comment

34015004700

DELETE_thread

21019002900

GET_comment_list

22014003200

GET_thread

20021005400

GET_thread_list

29016003300

PATCH_comment

29018003200

PATCH_thread

22012002900

POST_comment_comment

41019003800

POST_comment_response

35018003800

POST_thread

20014002900

auto_auth

260270270
No. of clients = 544
req/s = 43.50
Methodsmedian response time95%99%

DELETE_comment

320490710

DELETE_thread

200340500

GET_comment_list

170280550

GET_thread

180300870

GET_thread_list

210420700

PATCH_comment

290500670

PATCH_thread

200340440

POST_comment_comment

400600840

POST_comment_response

3405301000

POST_thread

190290590

auto_auth

210250250
No. of clients = 578
req/s = 45.90
Methodsmedian response time95%99%

DELETE_comment

37021005500

DELETE_thread

2109501600

GET_comment_list

25017004100

GET_thread

21028005700

GET_thread_list

31018004200

PATCH_comment

33011002600

PATCH_thread

23017004200

POST_comment_comment

44019005700

POST_comment_response

38020005400

POST_thread

21017005300

auto_auth

53053005300
No. of clients = 578
req/s = 47.7
Methodsmedian response time95%99%

DELETE_comment

3305402300

DELETE_thread

200320470

GET_comment_list

1803101100

GET_thread

1803301900

GET_thread_list

2104501200

PATCH_comment

3006103600

PATCH_thread

210360940

POST_comment_comment

4006001300

POST_comment_response

3405501400

POST_thread

1903101200

auto_auth

240280280
No. of clients = 610
req/s = 36
Methodsmedian response time95%99%

DELETE_comment

150056007600

DELETE_thread

130054006500

GET_comment_list

120053006600

GET_thread

190067009100

GET_thread_list

130054006900

PATCH_comment

140060006800

PATCH_thread

110045005900

POST_comment_comment

160060007600

POST_comment_response

130056007600

POST_thread

120052006400

auto_auth

130058005800
No. of clients = 610
req/s = 48.9
Methodsmedian response time95%99%

DELETE_comment

3606301900

DELETE_thread

2106501600

GET_comment_list

1803701300

GET_thread

1904002100

GET_thread_list

2205001200

PATCH_comment

3106101100

PATCH_thread

2204801400

POST_comment_comment

4308902400

POST_comment_response

3606801800

POST_thread

2004201800

auto_auth

270430430

 

UPDATE:

To narrow down the behaviour of feature endpoints on high traffic and huge data, I have conducted some more tests and here are the results.

Case 1:

I ran a few tests with a fresh new course for each run and initial data of 100 threads, 10 responses each thread and 7 comments to each response in each course. The percentile shows acceptable numbers as opposed to above results where data was increasing in a single course with each run (see above "with profile image" column). Hence we know the increasing data in any course has directly proportional effect to response percentile.

No. of clients = 336: (error rate = 595 (1.65%) )

Methodsmedian response time95%99%
DELETE_comment320460530
DELETE_thread200270300
GET_comment_list200300380
GET_thread190270360
GET_thread_list270680850
PATCH_comment220400500
PATCH_thread210290330
POST_comment_comment380520590
POST_comment_response330460540
POST_thread190260300
auto_auth240320320


No. of clients = 510: (error rate = 560 (1.61%) )

Methodsmedian response time95%99%
DELETE_comment 350 520 990
DELETE_thread220 330 1100 
GET_comment_list220 360 490 
GET_thread200310 450 
GET_thread_list290 760 980 
PATCH_comment290 490 540 
PATCH_thread210 320 390 
POST_comment_comment420 600 780 
POST_comment_response350 530 720 
POST_thread200 310 500 
auto_auth230 240 240


 No. of clients = 544: (error rate = 364 (1.57%) )

Methodsmedian response time95%99%
DELETE_comment360  600 840
DELETE_thread220 330 460 
GET_comment_list230 380 520 
GET_thread220 370 530 
GET_thread_list300 800 1000 
PATCH_comment320 580 700 
PATCH_thread230 360 510 
POST_comment_comment440 670 930 
POST_comment_response370 610 860 
POST_thread210 320 420 
auto_auth250 260 260

Case 2:

I had created a new fresh course populated in it a huge number of threads, responses and comments (in thousands) and then run GET thread and comment endpoints for both with profile image and without (as these are the only endpoints this changes is reflected in). 

Comparing the two set of results; we can see the difference in 99% for with profile image but I believe the numbers are acceptable, only the last two cases that I have highlighted shows anomaly.

  • For profile image: when no. of clients = 510 shows greater response time than no. of clients = 544. Its reason that I could assume is the users involved in thread and comments for later test are less in number possibly.
  • For without profile image: 
    • 99% for no. of clients = 544 is greater than that of with profile image which is weird.
    • there is an instant rise for clients=544 than for clients=510; to see if its a valid increase I used no. of clients somewhere between the two numbers i.e. 522, but the 99% was even higher than for 544, again weird.
With Profile ImageWithout Profile Image
No. of clients = 48
req/s = 4.10

error rate = 1(0.03%)

Methodsmedian response time95%99%
GET_comment_list210290370
GET_thread170230260
GET_thread_list250650780
auto_auth210220220
No. of clients = 48
req/s = 4

error rate = 0

Methodsmedian response time95%99%
GET_comment_list 170220250
GET_thread170 230 250 
GET_thread_list180 360 480 
auto_auth210 220 220
No. of clients = 144
req/s = 12

error rate = 0

Methodsmedian response time95%99%
GET_comment_list 210290360
GET_thread170 240 270 
GET_thread_list250 690 830 
auto_auth230 230 230
No. of clients = 144
req/s = 11.4

error rate = 0

Methodsmedian response time95%99%
GET_comment_list160230280
GET_thread170 240 260 
GET_thread_list180 360 500 
auto_auth220 290 290
No. of clients = 288
req/s = 23.9

error rate = 0

Methodsmedian response time95%99%
GET_comment_list 220310420
GET_thread180 240 280 
GET_thread_list260 710 870 
auto_auth220 230 230
No. of clients = 288
req/s = 23.3

error rate = 0

Methodsmedian response time95%99%
GET_comment_list 170230310 
GET_thread180 240 280 
GET_thread_list190 370 510 
auto_auth230 230 230
No. of clients = 510
req/s = 42.3

error rate = 1(0.00%)

Methodsmedian response time95%99%
GET_comment_list 240410 1900
GET_thread190 360 2600 
GET_thread_list290 880 2000 
auto_auth300 390 390
No. of clients = 510
req/s = 40.8

error rate = 0

Methodsmedian response time95%99%
GET_comment_list 180 270580
GET_thread190 290 860 
GET_thread_list200 420 640 
auto_auth320 1300 1300
No. of clients = 544
req/s = 44.40

error rate = 0

Methodsmedian response time95%99%
GET_comment_list250 360 480 
GET_thread200 290 390 
GET_thread_list290 810 970 
auto_auth280 310 310
No. of clients = 544
req/s = 45.4

error rate = 0

Methodsmedian response time95%99%
GET_comment_list 190 4202500
GET_thread200 480 4500 
GET_thread_list210 550 2500 
auto_auth280 290 290


 

  • No labels