Versions Compared

Old Version 12

changes.mady.by.user Wajeeha Khalid (Unlicensed)

Saved on

compared with

New Version Current

changes.mady.by.user Wajeeha Khalid (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Background:

In reference to 

Jira Legacy
serverJIRA (openedx.atlassian.net)
serverId13fd1930-5608-3aac-a5dd-21b934d3a4b4
keyMA-2678
; we have changed forums implementation with following details (for reference; see PR#192):

...

unread_comment_count = Comment.collection.find(:comment_thread_id => t._id, :author_id => {"$ne" => user.id}, :created_at => {"$gte" => read_dates[thread_key]}).count

So, we removed index
index({_type: 1, comment_thread_id: 1, author_id: 1, updated_at: 1})

and added a new one

index({comment_thread_id: 1, author_id: 1, created_at: 1})

 

...

Results:

The load tests were run on 4x c4.2xlarge instances .

New Relic:

Here is the new relic permaLink_new_index and permaLink_old_index

...

Old Index

...

for lms and 3x m4.large instances for forums.

The results of load tests below show differences between the old and new implementation. The two set of results looks quite similar except when it reaches "No. of clients = 336"; where there is huge difference between old and new percentiles as well as sudden rise in percentile for both old and new index with respect to "No. of clients = 224". For all the next tests (i.e. No. of clients = 460, No. of clients = 510, No. of clients = 578), the difference between old and new percentile is minimised and the new index results have lower percentile for most of the endpoints.

I have captured new relic charts too; permaLink_old_index with average rpm = 1.86k and permaLink_new_index with average rpm = 1.81k


48942503101200160190200150190310140180220160370520comment230120019024037029036043025031045016019020020025025048894802103202805204103304903903702309618.52703403901702303301702103301602002501904406102403303602002603503103804702603304401702101802202209618360170220340270600340350comment48042034023024044640060070024032053024035048024036047027063080033051060029042053042064082036056073024031044021021021024045.40390650130024037043023035050023037060027064085030049012002804606404106701000360560860230300500190220220

Old Index

New Index

No. of clients = 48
req/s = 9.4
Methodsmedian response time95%99%

DELETE_comment

250

3101200

DELETE_thread

160

190200

GET_comment_list

150190310

GET_thread

140180220

GET_thread_list

160370520

PATCH_comment

2303601200

PATCH_thread

190240370

POST_comment_comment

290360430

POST_comment_response

250310450

POST_thread

160190200

auto_auth

200250250
No. of clients = 48
req/s = 8.9
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

250270300470480520

DELETE_thread

170160210300230310

GET_comment_list

150160200260320350

GET_thread

150150190250280310

GET_thread_list

170190360370520530

PATCH_comment

220240320400410460

PATCH_thread

180190360300330330

POST_comment_comment

290310340510490680

POST_comment_response

250260300450390520

POST_thread

160160200250370340

auto_auth

220210230230230230


No. of clients = 
96
req/s = 
18.
5
Methodsmedian response time95%99%
DELETE_comment

270

340
390
DELETE_thread
170
230
330
GET_comment_list
170
210
330
GET_thread
160
200
250
GET_thread_list
190
440
610
PATCH_
comment240
330360
PATCH_thread
200
260
350
POST_comment_comment
310
380
470
POST_comment_response
260
330
440
POST_thread
170
210
340
auto_auth
180
220
220


No. of clients = 
96
req/s = 
18.
6
Methodsmedian response timemedian response time95%95%99%99%
DELETE_comment
250300
270280340360360980
DELETE_thread170170220
230230290
GET_comment_list
150200
160160210220340400
GET_thread
150190
160160200210270340
GET_thread_list
170360
190190460460600640
PATCH_comment220
320
230310310340460
PATCH_thread
180360
1902002602803501000
POST_comment_comment
290340
300310370410480680
POST_comment_response
250300
260260330350420620
POST_thread
160200
170170210210340330
auto_auth180210230220230
220


No. of clients = 
162
req/s = 
31
Methodsmedian response time95%99%

DELETE_comment

320
400
490

DELETE_thread

190
240
280

GET_comment_list

190
250
370

GET_thread

190
250
340

GET_thread_list

220
520
660

PATCH_comment

270
360
390

PATCH_thread

230
310
370

POST_comment_comment

350
450
540

POST_comment_response

300
400
490

POST_thread

190
240340

auto_auth

200
200
200


No. of clients = 
162
req/s = 
31.6
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

270340
310320410420590530

DELETE_thread

190
190
240230250250

GET_comment_list

160210

190190250250380380

GET_thread

160200

180180250250360340

GET_thread_list

190460
220210530520670660

PATCH_comment

220310
2602503803601400430

PATCH_thread

190260

220230280310400370

POST_comment_

300370

comment

340340450470540620

POST_comment_response

260330
290290400400510550

POST_thread

170210

190190240230370360

auto_auth

180230

210200220200220200


No. of clients = 162240
req/s = 3144.6
190
Methodsmedian response time95%99%

DELETE_comment

320400400600490700

DELETE_thread

240320280530

GET_comment_list

190240250350370480

GET_thread

190240250360340470

GET_thread_list

220270520630660800

PATCH_comment

270330360510390600

PATCH_thread

230290310420370530

POST_comment_comment

350420450640540820

POST_comment_response

300360400560490730

POST_thread

190240240310340440

auto_auth

200210200210200210


No. of clients = 162240
req/s = 3145.640
400
Methodsmedian response time95%99%

DELETE_comment

310410590

DELETE_thread

190240250

GET_comment_list

190250380

GET_thread

180250360

GET_thread_list

220530670

PATCH_comment

2603801400

PATCH_thread

220280median response time95%95%99%99%

DELETE_comment

39041065069013001100

DELETE_thread

240230370280430340

GET_comment_list

230240350350500480

GET_thread

230240370370600520

GET_thread_list

270270640630850790

PATCH_comment

3003104904801200610

PATCH_thread

280290460470640630

POST_comment_comment

340450540

4104306706601000970

POST_comment_response

290400510

360360560600860850

POST_thread

190240370

230230300310500430

auto_auth

190210220220220220



No. of clients = 
270
req/s = 
51.
1
Methodsmedian response time95%99%

DELETE_comment

560
7900
10000

DELETE_thread

280
620
5200

GET_comment_list

300
3100
4000

GET_thread

310
3400
5000

GET_thread_list

330
3000
4200

PATCH_comment

400
1700
6500

PATCH_thread

380
5100
6300

POST_comment_comment

560
7900
10000

POST_comment_response

470
6100
8200

POST_thread

290
2900
3900
auto_auth
320
320
320

No. of clients = 
300
req/s = 
48
Methodsmedian response time95%99%

DELETE_comment

1600
4600
5900

DELETE_thread

460
1600
2500

GET_comment_list

480
1300
1900

GET_thread

560
2100
2900

GET_thread_list

530
1700
2500

PATCH_comment

800
2800
3500

PATCH_thread

710
2700
3600

POST_comment_comment

1400
4300
5500

POST_comment_response

1000
3600
4800

POST_thread

430
1100
1700
auto_auth
240
250
250
No. of clients = 336
req/s = 53
Methodsmedian response time95%99%

DELETE_comment

250055006500

DELETE_thread

98023002900

GET_comment_list

80021002700

GET_thread

92027003300

GET_thread_list

85024003300

PATCH_comment

150039005000

PATCH_thread

120034004100

POST_comment_comment

230054007000

POST_comment_response

170044005900

POST_thread

74020002700

auto_auth

33013001300


No. of clients = 336
req/s = 54.3
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

15005300920096001200011000

DELETE_thread

470230041004100460067005100

GET_comment_list

51018005000370068004500

GET_thread

59021006500480099005700

GET_thread_list

58018004900390068004800

PATCH_comment

81035007600640011009700

PATCH_thread

79029005400630078007300

POST_comment_comment

13004800930093001200011000

POST_comment_response

1000370077007800100009600

POST_thread

47017004600350064004000

auto_auth

320710320830320830


No. of clients = 460
req/s = 44.5
Methodsmedian response time95%99%

DELETE_comment

5600980012000

DELETE_thread

370062007900

GET_comment_list

370061008100

GET_thread

5200980013000

GET_thread_list

370068008600

PATCH_comment

480079009500

PATCH_thread

460073009900

POST_comment_comment

5500930011000

POST_comment_response

5000830010000

POST_thread

350060009200

auto_auth

200026002600


No. of clients = 460
req/s = 49.1
Methodsmedian response time95%99%

DELETE_comment

61001000011000

DELETE_thread

350057006400

GET_comment_list

300050005700

GET_thread

420071008700

GET_thread_list

310053006200

PATCH_comment

440078008600

PATCH_thread

410069007600

POST_comment_comment

5700990011000

POST_comment_response

4800850010000

POST_thread

300048005500

auto_auth

170027002700


No. of clients = 510
req/s = 32.4
Methodsmedian response time95%99%

DELETE_comment

77001500027000

DELETE_thread

54001200015000

GET_comment_list

63001400031000

GET_thread

86001900036000

GET_thread_list

64001400032000

PATCH_comment

70001400025000

PATCH_thread

69001600035000

POST_comment_comment

74001500036000

POST_comment_response

70001400026000

POST_thread

60001100032000

auto_auth

290040004400


No. of clients = 510
req/s = 33.9
Methodsmedian response time95%99%

DELETE_comment

76001400019000

DELETE_thread

67001100014000

GET_comment_list

67001100016000

GET_thread

94001600022000

GET_thread_list

70001100017000

PATCH_comment

69001200016000

PATCH_thread

69001200021000

POST_comment_comment

76001400019000

POST_comment_response

72001300018000

POST_thread

67001100015000

auto_auth

240032003300


No. of clients = 578
req/s = 26.70
Methodsmedian response time95%99%

DELETE_comment

95002600032000

DELETE_thread

87002500032000

GET_comment_list

86002200031000

GET_thread

130002700040000

GET_thread_list

86002200032000

PATCH_comment

88002200032000

PATCH_thread

88002000029000

POST_comment_comment

92002400031000

OST_comment_response

93002400032000

POST_thread

83002000032000

auto_auth

360052005500


No. of clients = 578
req/s = 40.2
Methodsmedian response time95%99%

DELETE_comment

91002400032000

DELETE_thread

85002200026000

GET_comment_list

81002200032000

GET_thread

120002600040000

GET_thread_list

83002200032000

PATCH_comment

83002300031000

PATCH_thread

80002300028000

POST_comment_comment

90002400032000

OST_comment_response

85002300032000

POST_thread

83002200032000

auto_auth

410051005400