Versions Compared

Old Version 10

changes.mady.by.user Wajeeha Khalid (Unlicensed)

Saved on

compared with

New Version 11

changes.mady.by.user Wajeeha Khalid (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The load tests were run on 4x c4.2xlarge instances.

 

 

 

There appears to be anomaly in the results below (highlighted 99%) where the new implementation is faster than the old. Where as, for other results the percentile of new implementation increases significantly with the increase in load. For example, the first pair of results where no. of clients = 48, the difference is in tens or hundrend and for the last pair of results where no. of clients = 610, the difference is in thousands.

Logically; there should only be increase in response time for GET endpoints, but we see in results that all other endpoints show differences too. This is because all PATCH, POST and DELETE endpoints first call GET endpoint to retrieve an 'id' of thread and/or comment and then make further processing on it.

New Relic:

Here is the new relic permaLink_new_index and permaLink_old_index

where the average rpm for former is 1.55k and for later is 1.48k and 84.5% of requests are made for AccountViewSet.list (i.e. user accounts API for multiple usernames) for later.

 

...

Old Index

...

New Index

...

No. of clients = 48
req/s = 9.4
Methodsmedian response time95%99%

DELETE_comment

250

3101200

DELETE_thread

160

190200

GET_comment_list

150190310

GET_thread

140180220

GET_thread_list

160370520

PATCH_comment

2303601200

PATCH_thread

190240370

POST_comment_comment

290360430

POST_comment_response

250310450

POST_thread

160190200

auto_auth

200250250
No. of clients = 48
req/s = 9.6 --> 18min = 8.9
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

260250310300340480

DELETE_thread

160170200210220230

GET_comment_list

160150200200290320

GET_thread

150150190190260280

GET_thread_list

170170370360520520

PATCH_comment

250220410320410410

PATCH_thread

1901802503601200330

POST_comment_comment

300290390340420490

POST_comment_response

260250320300410390

POST_thread

160160210200310370

auto_auth

190220190230190230
No. of clients = 96
req/s = 18.5
Methodsmedian response time95%99%
DELETE_comment

270

340390
DELETE_thread170230330
GET_comment_list170210330
GET_thread160200250
GET_thread_list190440610
PATCH_comment240330360
PATCH_thread200260350
POST_comment_comment310380470
POST_comment_response260330440
POST_thread170210340
auto_auth180220220
No. of clients = 96
req/s = 18.7  27min = 18.6
Methodsmedian response timemedian response time95%95%99%99%
DELETE_comment280270370340440360
DELETE_thread180170220220270230
GET_comment_list170160230210350340
GET_thread160160220200300270
GET_thread_list210190500460640600
PATCH_comment2502203603101400340
PATCH_thread200190280260380350
POST_comment_comment320300400370510480
POST_comment_response270260350330430420
POST_thread170170220210260340
auto_auth190180260230260230
No. of clients = 162
req/s = 31
Methodsmedian response time95%99%

DELETE_comment

320400490

DELETE_thread

190240280

GET_comment_list

190250370

GET_thread

190250340

GET_thread_list

220520660

PATCH_comment

270360390

PATCH_thread

230310370

POST_comment_comment

350450540

POST_comment_response

300400490

POST_thread

190240340

auto_auth

200200200
No. of clients = 162
req/s = 31.5    31.6
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

350310500410660590

DELETE_thread

210190300240390250

GET_comment_list

210190350250460380

GET_thread

210180320250430360

GET_thread_list

250220620530800670

PATCH_comment

2902604403805701400

PATCH_thread

260220430280580400

POST_comment_comment

380340560450710540

POST_comment_response

330290480400610510

POST_thread

210190310240430370

auto_auth

210210230220230220
No. of clients = 240
req/s = 44.6
Methodsmedian response time95%99%

DELETE_comment

400600700

DELETE_thread

240320530

GET_comment_list

240350480

GET_thread

240360470

GET_thread_list

270630800

PATCH_comment

330510600

PATCH_thread

290420530

POST_comment_comment

420640820

POST_comment_response

360560730

POST_thread

240310440

auto_auth

210210210
No. of clients = 240
req/s = 44.7   45.40
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

540390100065018001300

DELETE_thread

3402406803702000430

GET_comment_list

3402307503501500500

GET_thread

3402307103701500600

GET_thread_list

41027010006401700850

PATCH_comment

42030080049013001200

PATCH_thread

4002807504601300640

POST_comment_comment

580410110067025001000

POST_comment_response

5003609405601600860

POST_thread

3402307103001000500

auto_auth

490190610220610220
No. of clients = 336
req/s = 53
Methodsmedian response time95%99%

DELETE_comment

250055006500

DELETE_thread

98023002900

GET_comment_list

80021002700

GET_thread

92027003300

GET_thread_list

85024003300

PATCH_comment

150039005000

PATCH_thread

120034004100

POST_comment_comment

230054007000

POST_comment_response

170044005900

POST_thread

74020002700

auto_auth

33013001300
No. of clients = 336
req/s = 48.9  54.3
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

2300150034009200440012000

DELETE_thread

17004702600410032006700

GET_comment_list

19005102900500039006800

GET_thread

18005902800650037009900

GET_thread_list

23005805200490068006800

PATCH_comment

22008103200760042001100

PATCH_thread

19007903200540055007800

POST_comment_comment

2400130035009300430012000

POST_comment_response

2200100032007700390010000

POST_thread

18004702800460036006400

auto_auth

180032021003202100320
No. of clients = 460
req/s = 44.5
Methodsmedian response time95%99%

DELETE_comment

5600980012000

DELETE_thread

370062007900

GET_comment_list

370061008100

GET_thread

5200980013000

GET_thread_list

370068008600

PATCH_comment

480079009500

PATCH_thread

460073009900

POST_comment_comment

5500930011000

POST_comment_response

5000830010000

POST_thread

350060009200

auto_auth

200026002600
No. of clients = 460
req/s = 32.4   21min = 49.1

...

DELETE_comment

...

DELETE_thread

...

GET_comment_list

...

GET_thread

...

GET_thread_list

...

PATCH_comment

...

PATCH_thread

...

POST_comment_comment

...

POST_comment_response

...

POST_thread

...

auto_auth

...

New Relic:

Here is the new relic permaLink_new_index and permaLink_old_index

578267095002600032000870025000320008600220003100013000270004000086002200032000880022000320008800200002900092002400031000OST9300240003200083002000032000360052005500578 40.2median response time95%99% 26000 32000 40000 32000 3100028000 9000OST 32000 32000 5400                                                                                                                                                                                                                              POST                                 POST         

Old Index

New Index

No. of clients = 51048
req/s = 329.4
Methodsmedian response time95%99%

DELETE_comment

7700

250

15000310270001200

DELETE_thread

5400

160

1200019015000200

GET_comment_list

63001501400019031000310

GET_thread

86001401900018036000220

GET_thread_list

64001601400037032000520

PATCH_comment

700023014000360250001200

PATCH_thread

69001901600024035000370

POST_comment_comment

74002901500036036000430

POST_comment_response

70002501400031026000450

POST_thread

60001601100019032000200

auto_auth

290020040002504400250
No. of clients = 51048
req/s =     30min = 338.9
median response time95%         
Methodsmedian response time95%99%99%

DELETE_comment

2507600 14000 19000300480

DELETE_thread

1706700 11000 14000210230

GET_comment_list

1506700 11000 16000200320

GET_thread

1509400 16000 22000190280

GET_thread_list

 7000 11000 17000

170360520

PATCH_comment

2206900 12000 16000320410

PATCH_thread

1806900 12000 21000360330

POST_comment_comment

2907600 14000 19000340490

POST_comment_response

2507200 13000 18000300390

POST_thread

 6700 11000 15000160200370

auto_auth

2202400 3200 3300230230


No. of clients = 
96
req/s = 
18.
5
Methodsmedian response time95%99%
DELETE_comment

270

340
390
DELETE_thread
170
230
330
GET_comment_list
170
210
330
GET_thread
160
200
250
GET_thread_list
190
440
610
PATCH_comment
240
330
360
PATCH_thread
200
260
350
POST_comment_comment
310
380
470
POST_comment_response
260
330
440
POST_thread
170
210
340
auto_auth
180
220
220


No. of clients = 
96
req/s = 
18.6
Methods
median response time
95%
99%
DELETE_comment

DELETE_thread

 8500 22000 
270
9100 24000 32000
340360
DELETE_thread170220230
GET_comment_list
160
8100 22000 
210340
GET_thread
160
12000 26000 
200270
GET_thread_list
190
8300 22000 
460600
PATCH_comment
220
8300 23000 
310340
PATCH_thread
 8000 23000 
190260350
POST_comment_comment
300
370
 24000 32000
480
POST_comment_response
260
8500 23000 
330420
POST_thread
170
8300 22000 
210340
auto_auth
180
4100 5100 
230230


No. of clients = 162
req/s = 31
Methodsmedian response time95%99%

DELETE_comment

 320 400 490

DELETE_thread

 190 240 280

GET_comment_list

 190 250 370

GET_thread

 190 250 340

GET_thread_list

 220 520 660

PATCH_comment

 270 360 390

PATCH_thread

 230 310 370

POST_comment_comment

 350 450 540

POST_comment_response

 300 400 490

POST_thread

 190 240 340

auto_auth

 200 200 200


No. of clients = 162
req/s = 31.6
                      
Methodsmedian response time95%99%

DELETE_comment

310410 590

DELETE_thread

190240 250

GET_comment_list

190250 380

GET_thread

180250 360

GET_thread_list

220530 670

PATCH_comment

260380 1400

PATCH_thread

220280 400

POST_comment_comment

340450 540

POST_comment_response

290400 510

POST_thread

190240 370

auto_auth

210220 220


No. of clients = 240
req/s = 44.6
Methodsmedian response time95%99%

DELETE_comment

 400 600 700

DELETE_thread

 240 320 530

GET_comment_list

 240 350 480

GET_thread

 240 360 470

GET_thread_list

 270 630 800

PATCH_comment

 330 510 600

PATCH_thread

 290 420 530

POST_comment_comment

 420 640 820

POST_comment_response

 360 560 730

POST_thread

 240 310 440

auto_auth

 210 210 210


No. of clients = 240
req/s = 45.40
                      
Methodsmedian response time95%99%

DELETE_comment

390650 1300

DELETE_thread

240370 430

GET_comment_list

230350 500

GET_thread

230370 600

GET_thread_list

270640 850

PATCH_comment

300490 1200

PATCH_thread

280460 640

POST_comment_comment

410670 1000

POST_comment_response

360560 860

POST_thread

230300 500

auto_auth

190220 220


No. of clients = 336
req/s = 53
Methodsmedian response time95%99%

DELETE_comment

2500
5500
6500

DELETE_thread

980
2300
2900

GET_comment_list

800
2100
2700

GET_thread

920
2700
3300

GET_thread_list

850
2400
3300

PATCH_comment

1500
3900
5000

PATCH_thread

1200
3400
4100

POST_comment_comment

2300
5400
7000

POST_comment_response

1700
4400
5900

POST_thread

740
2000
2700

auto_auth

330
1300
1300


No. of clients = 336
req/s = 54.3
Methodsmedian response time95%99%

DELETE_comment

1500
9200
12000

DELETE_thread

470
4100
6700

GET_comment_list

510
5000
6800

GET_thread

590
6500
9900

GET_thread_list

580
4900
6800

PATCH_comment

810
7600
1100

PATCH_thread

790
5400
7800

POST_comment_comment

1300
9300
12000

POST_comment_response

1000
7700
10000

POST_thread

470
4600
6400

auto_auth

320
320
320


No. of clients = 460
req/s = 44.5
Methodsmedian response time95%99%

DELETE_comment

5600
9800
12000

DELETE_thread

3700
6200
7900

GET_comment_list

3700
6100
8100

GET_thread

5200
9800
13000

GET_thread_list

3700
6800
8600

PATCH_comment

4800
7900
9500

PATCH_thread

4600
7300
9900

POST_comment_comment

5500
9300
11000

POST_comment_response

5000
8300
10000

POST_thread

3500
6000
9200

auto_auth

2000
2600
2600


No. of clients = 460
req/s = 49.1
Methodsmedian response time95%99%

DELETE_comment

6100
10000
11000

DELETE_thread

3500
5700
6400

GET_comment_list

3000
5000
5700

GET_thread

4200
7100
8700

GET_thread_list

3100
5300
6200

PATCH_comment

4400
7800
8600

PATCH_thread

4100
6900
7600

POST_comment_comment

5700
9900
11000

POST_comment_response

4800
8500
10000

POST_thread

3000
4800
5500

auto_auth

1700
2700
2700


No. of clients = 510
req/s = 32.4
Methodsmedian response time95%99%

DELETE_comment

7700
15000
27000

DELETE_thread

5400
12000
15000

GET_comment_list

6300
14000
31000

GET_thread

8600
19000
36000

GET_thread_list

6400
14000
32000

PATCH_comment

7000
14000
25000

PATCH_thread

6900
16000
35000

POST_comment_comment

7400
15000
36000

POST_comment_response

7000
14000
26000

POST_thread

6000
11000
32000

auto_auth

2900
4000
4400


No. of clients = 510
req/s = 33.9
Methodsmedian response time95%99%

DELETE_comment

7600
14000
19000

DELETE_thread

6700
11000
14000

GET_comment_list

6700
11000
16000

GET_thread

9400
16000
22000

GET_thread_list

7000
11000
17000

PATCH_comment

6900
12000
16000

PATCH_thread

6900
12000
21000

POST_comment_comment

7600
14000
19000

POST_comment_response

7200
13000
18000

POST_thread

6700
11000
15000

auto_auth

2400
3200
3300


No. of clients = 578
req/s = 26.70
Methodsmedian response time95%99%

DELETE_comment

9500
26000
32000

DELETE_thread

8700
25000
32000

GET_comment_list

8600
22000
31000

GET_thread

13000
27000
40000

GET_thread_list

8600
22000
32000

PATCH_comment

8800
22000
32000

PATCH_thread

8800
20000
29000

POST_comment_comment

9200
24000
31000

OST_comment_response

9300
24000
32000

POST_thread

8300
20000
32000

auto_auth

3600
5200
5500


No. of clients = 578
req/s = 40.2
Methodsmedian response time95%99%

DELETE_comment

9100
24000
32000

DELETE_thread

8500
22000
26000

GET_comment_list

8100
22000
32000

GET_thread

12000
26000
40000

GET_thread_list

8300
22000
32000

PATCH_comment

8300
23000
31000

PATCH_thread

8000
23000
28000

POST_comment_comment

9000
24000
32000

OST_comment_response

8500
23000
32000

POST_thread

8300
22000
32000

auto_auth

4100
5100
5400