Page Comparison

Background:

In reference to

Jira Legacy

server	JIRA (openedx.atlassian.net)
serverId	13fd1930-5608-3aac-a5dd-21b934d3a4b4
key	MA-2678

; we have changed forums implementation with following details (for reference; see PR#192):

...

'unread comment count' was being calculate as:

unread_comment_count

...

= Comment.collection.find(:comment_thread_id

...

=>

...

t._id,

...

:author_id

...

=>

...

{"$ne"

...

=>

...

user.id},

...

:updated_at

...

=>

...

{"$gte"

...

=>

...

read_dates[thread_key]}).count
and

...

had

...

a

...

compound

...

index

...

against

...

it
index({_type: 1,

...

comment_thread_id: 1,

...

author_id: 1,

...

updated_at: 1})

With new implementation:

unread_comment_count

...

= Comment.collection.find(:comment_thread_id

...

=>

...

t._id, :author_id

...

=>

...

{"$ne"

...

=>

...

user.id}, :created_at

...

=>

...

{"$gte"

...

=>

...

read_dates[thread_key]}).count
So,

...

we

...

removed

...

index
index({_type: 1,

...

comment_thread_id: 1,

...

author_id: 1,

...

updated_at: 1})

and

...

added

...

a

...

new

...

one
index({comment_thread_id: 1,

...

author_id: 1,

...

created_at: 1})

...

Results:

The load tests were run on 4x c4.2xlarge instances for lms and 3x m4.large instances for forums.

The results of load tests below show difference differences between the old and new implementation.

There appears to be anomaly in the results below (highlighted 99%) where the new implementation is faster than the old. Where as, for other results the percentile of new implementation increases significantly with the increase in load. For example, the first pair of results where no. of clients = 48, the difference is in tens or hundrend and for the last pair of results where no. of clients = 610, the difference is in thousands.

Logically; there should only be increase in response time for GET endpoints, but we see in results that all other endpoints show differences too. This is because all PATCH, POST and DELETE endpoints first call GET endpoint to retrieve an 'id' of thread and/or comment and then make further processing on it.

New Relic:

Here is the new relic permaLink_without_profile and permaLink_with_profile where the average rpm for former is 1.55k and for later is 1.48k and 84.5% of requests are made for AccountViewSet.list (i.e. user accounts API for multiple usernames) for later.

...

Old Index

...

New Index

...

No. of clients = 48

req/s = 9.4

Methods	median response time	95%	99%
DELETE_comment	250	310	1200
DELETE_thread	160	190	200
GET_comment_list	150	190	310
GET_thread	140	180	220
GET_thread_list	160	370	520
PATCH_comment	230	360	1200
PATCH_thread	190	240	370
POST_comment_comment	290	360	430
POST_comment_response	250	310	450
POST_thread	160	190	200
auto_auth	200	250	250

No. of clients = 48

req/s = 9.6

Methods	median response time	95%	99%
DELETE_comment	260	310	340
DELETE_thread	160	200	220
GET_comment_list	160	200	290
GET_thread	150	190	260
GET_thread_list	170	370	520
PATCH_comment	250	410	410
PATCH_thread	190	250	1200
POST_comment_comment	300	390	420
POST_comment_response	260	320	410
POST_thread	160	210	310
auto_auth	190	190	190

No. of clients = 96

req/s = 18.5

Methods	median response time	95%	99%
DELETE_comment	270	340	390
DELETE_thread	170	230	330
GET_comment_list	170	210	330
GET_thread	160	200	250
GET_thread_list	190	440	610
PATCH_comment	240	330	360
PATCH_thread	200	260	350
POST_comment_comment	310	380	470
POST_comment_response	260	330	440
POST_thread	170	210	340
auto_auth	180	220	220

No. of clients = 96

req/s = 18.7

Methods	median response time	95%	99%
DELETE_comment	280	370	440
DELETE_thread	180	220	270
GET_comment_list	170	230	350
GET_thread	160	220	300
GET_thread_list	210	500	640
PATCH_comment	250	360	1400
PATCH_thread	200	280	380
POST_comment_comment	320	400	510
POST_comment_response	270	350	430
POST_thread	170	220	260
auto_auth	190	260	260

No. of clients = 162

req/s = 31

Methods	median response time	95%	99%
DELETE_comment	320	400	490
DELETE_thread	190	240	280
GET_comment_list	190	250	370
GET_thread	190	250	340
GET_thread_list	220	520	660
PATCH_comment	270	360	390
PATCH_thread	230	310	370
POST_comment_comment	350	450	540
POST_comment_response	300	400	490
POST_thread	190	240	340
auto_auth	200	200	200

No. of clients = 162

req/s = 31.5

Methods	median response time	95%	99%
DELETE_comment	350	500	660
DELETE_thread	210	300	390
GET_comment_list	210	350	460
GET_thread	210	320	430
GET_thread_list	250	620	800
PATCH_comment	290	440	570
PATCH_thread	260	430	580
POST_comment_comment	380	560	710
POST_comment_response	330	480	610
POST_thread	210	310	430
auto_auth	210	230	230

No. of clients = 240

req/s = 44.6

Methods	median response time	95%	99%
DELETE_comment	400	600	700
DELETE_thread	240	320	530
GET_comment_list	240	350	480
GET_thread	240	360	470
GET_thread_list	270	630	800
PATCH_comment	330	510	600
PATCH_thread	290	420	530
POST_comment_comment	420	640	820
POST_comment_response	360	560	730
POST_thread	240	310	440
auto_auth	210	210	210

No. of clients = 240

req/s = 44.7

Methods	median response time	95%	99%
DELETE_comment	540	1000	1800
DELETE_thread	340	680	2000
GET_comment_list	340	750	1500
GET_thread	340	710	1500
GET_thread_list	410	1000	1700
PATCH_comment	420	800	1300
PATCH_thread	400	750	1300
POST_comment_comment	580	1100	2500
POST_comment_response	500	940	1600
POST_thread	340	710	1000
auto_auth	490	610	610

No. of clients = 336

req/s = 53

Methods	median response time	95%	99%
DELETE_comment	2500	5500	6500
DELETE_thread	980	2300	2900
GET_comment_list	800	2100	2700
GET_thread	920	2700	3300
GET_thread_list	850	2400	3300
PATCH_comment	1500	3900	5000
PATCH_thread	1200	3400	4100
POST_comment_comment	2300	5400	7000
POST_comment_response	1700	4400	5900
POST_thread	740	2000	2700
auto_auth	330	1300	1300

No. of clients = 336

req/s = 48.9

...

DELETE_comment

...

DELETE_thread

...

GET_comment_list

...

GET_thread

...

GET_thread_list

...

PATCH_comment

...

PATCH_thread

...

POST_comment_comment

...

POST_comment_response

...

POST_thread

...

auto_auth

...

The two set of results looks quite similar except when it reaches "No. of clients = 336"; where there is huge difference between old and new percentiles as well as sudden rise in percentile for both old and new index with respect to "No. of clients = 224". For all the next tests (i.e. No. of clients = 460, No. of clients = 510, No. of clients = 578), the difference between old and new percentile is minimised and the new index results have lower percentile for most of the endpoints.

I have captured new relic charts too; permaLink_old_index with average rpm = 1.86k and permaLink_new_index with average rpm = 1.81k

51032477001500027000540012000150006300140003100086001900036000640014000320007000140002500069001600035000740015000360007000140002600060001100032000290040004400 _list response POST POST

Old Index

New Index

No. of clients = 46048

req/s = 449.54

Methods	median response time	95%	99%
DELETE_comment	5600 250	980031012000	1200
DELETE_thread	3700 160	62001907900	200
GET_comment_list	3700150	61001908100	310
GET_thread	5200140	980018013000	220
GET_thread_list	3700160	68003708600	520
PATCH_comment	4800230	79003609500	1200
PATCH_thread	4600190	73002409900	370
POST_comment_comment	5500290	930036011000	430
POST_comment_response	5000250	830031010000	450
POST_thread	3500160	60001909200	200
auto_auth	2000200	26002502600	250

No. of clients = 46048

req/s = 328.49

41000420004700043000430004300042000420005400

Methods	median response time	median response time	95%	95%	99%	99%
DELETE_comment	7600	38000	43000	DELETE_thread	6700	26000		250	270	300	470	480	520
DELETE_thread	170	160	210	300	230	310
GET_comment_list	7200	38000		150	160	200	260	320	350
GET_thread	10000	42000		150	150	190	250	280	310
GET_thread_list	7600	40000		170	190	360	370	520	530
PATCH_comment	7300	28000	43000	PATCH_thread	7300	27000		220	240	320	400	410	460
PATCH_thread	180	190	360	300	330	330
POST_comment_comment	7800	41000	290	310	340	510	490	680
POST_comment_response	7800	38000		250	260	300	450	390	520
POST_thread	6900	28000	160	160	200	250	370	340
auto_auth	4300	4800		220	210	230	230	230	230

No. of clients =

req/s =

18.

Methods	median response time	95%	99%
DELETE_comment

270

340

390

DELETE_thread

170

230

330

GET_comment_list

170

210

330

GET_thread

160

200

250

GET_thread_list

190

440

610

PATCH_comment

240

330

360

PATCH_thread

200

260

350

POST_comment_comment

310

380

470

POST_comment_response

260

330

440

POST_thread

170

210

340

auto_auth

180

220

No. of clients = 96

req/s = 18.6

Methods	median response time	median response time	95%	95%	99%	99%
DELETE_comment

	270	280	340	360	360	980
DELETE_thread

	170	170	220	230	230	290
GET_comment_list

	160	160	210	220	340	400
GET_thread

	160	160	200	210	270	340
GET_thread

_list	190	190	460	460	600	640
PATCH_comment

	220	230	310	310	340	460
PATCH_thread

	190	200	260	280	350	1000
POST_comment_comment

	300	310	370	410	480	680
POST_comment_response

	260	260	330	350	420	620
POST_thread

	170	170	210	210	340	330
auto_auth

180

210

230

220

230

220

No. of clients = 162

req/s = 31

Methods	median response time	95%	99%
DELETE_comment	320	400	490
DELETE_thread	190	240	280
GET_comment_list	190	250	370
GET_thread	190	250	340
GET_thread_list	220	520	660
PATCH_comment	270	360	390
PATCH_thread	230	310	370
POST_comment_comment	350	450	540
OSTPOST_comment_response	300	400	490
POST_thread	190	240	340
auto_auth	200	200	200

No. of clients = 162

req/s = 31.6

_listOST

Methods	95%	95%	99%	99%
DELETE_comment		310	320	410	420	590	530
DELETE_thread		190	190	240	230	250	250
GET_comment	_list	190	190	250	250	380	380
GET_thread	180	180	250	250	360	340
GET_thread_list		220	210	530	520	670	660
PATCH_comment	260	250	380	360	1400	430
PATCH_thread		220	230	280	310	400	370
POST_comment_comment			340	340	450	470	540	620
POST_comment_response		290	290	400	400	510	550
POST_thread		190	190	240	230	370	360
auto_auth		210	200	220	200	220	200

No. of clients = 240

req/s = 44.6

Methods	median response time	95%	99%
DELETE_comment	400	600	700
DELETE_thread	240	320	530
GET_comment_list	240	350	480
GET_thread	240	360	470
GET_thread_list	270	630	800
PATCH_comment	330	510	600
PATCH_thread	290	420	530
POST_comment_comment	420	640	820
POST_comment_response	360	560	730
POST_thread	240	310	440
auto_auth	210	210	210

No. of clients = 240

req/s = 45.40

Methods	95%	95%	99%	99%
DELETE_comment		390	410	650	690	1300	1100
DELETE_thread		240	230	370	280	430	340
GET_comment_list		230	240	350	350	500	480
GET_thread		230	240	370	370	600	520
GET_thread_list		270	270	640	630	850	790
PATCH_comment		300	310	490	480	1200	610
PATCH_thread	280	290	460	470	640	630
POST_comment_comment		410	430	670	660	1000	970
POST_comment_response		360	360	560	600	860	850
POST_thread		230	230	300	310	500	430
auto_auth		190	210	220	220	220	220

No. of clients = 270

req/s =

 51.1

Methods	median response time	95%	99%
DELETE_comment

560

7900

10000

DELETE_thread

280

620

5200

GET_comment_list

300

3100

4000

GET_thread

310

3400

5000

GET_thread_list

330

3000

4200

PATCH_comment

400

1700

6500

PATCH_thread

380

5100

6300

POST_comment_comment

560

7900

10000

POST_comment_response

470

6100

8200

POST_thread

290

2900

3900

auto_auth

320

No. of clients = 300

req/s = 48

Methods	median response time	95%	99%
DELETE_comment

1600

4600

5900

DELETE_thread

460

1600

2500

GET_comment_list

480

1300

1900

GET_thread

560

2100

2900

GET_thread_list

530

1700

2500

PATCH_comment

800

2800

3500

PATCH_thread

710

2700

3600

POST_comment_comment

1400

4300

5500

POST_comment_response

1000

3600

4800

POST_thread

430

1100

1700

auto_auth

240

250

No. of clients = 336

req/s = 53

Methods	median response time	95%	99%
DELETE_comment

2500

5500

6500

DELETE_thread

980

2300

2900

GET_comment_list

800

2100

2700

GET_thread

920

2700

3300

GET_thread_list

850

2400

3300

PATCH_comment

1500

3900

5000

PATCH_thread

1200

3400

4100

POST_comment_comment

2300

5400

7000

POST_comment_response

1700

4400

5900

POST_thread

740

2000

2700

auto_auth

330

1300

No. of clients = 336

req/s = 54.3

Methods	median response time	median response time	95%	95%	99%	99%
DELETE_comment

	1500	5300	9200	9600	12000	11000
DELETE_thread

470	2300	4100	4600	6700	5100
GET_comment_list

GET_thread

	510	1800	5000	3700	6800	4500
GET_thread	590	2100	6500	4800	9900	5700
GET_thread_list

	580	1800	4900	3900	6800	4800
PATCH_comment

	810	3500	7600	6400	1100	9700
PATCH_thread

	790	2900	5400	6300	7800	7300
POST_comment_comment

	1300	4800	9300	9300	12000	11000
POST_comment_

response	1000	3700	7700	7800	10000	9600
POST_thread

	470	1700	4600	3500	6400	4000
auto_auth

320

710

320

830

320

830

No. of clients = 460

req/s = 44.5

Methods	median response time	95%	99%
DELETE_comment

5600

9800

12000

DELETE_thread

3700

6200

7900

GET_comment_list

3700

6100

8100

GET_thread

5200

9800

13000

GET_thread_list

3700

6800

8600

PATCH_comment

4800

7900

9500

PATCH_thread

4600

7300

9900

POST_comment_comment

5500

9300

11000

POST_comment_response

5000

8300

10000

POST_thread

3500

6000

9200

auto_auth

2000

2600

No. of clients = 460

req/s = 49.1

Methods	median response time	95%	99%
DELETE_comment

6100

10000

11000

DELETE_thread

3500

5700

6400

GET_comment_list

3000

5000

5700

GET_thread

4200

7100

8700

GET_thread_list

3100

5300

6200

PATCH_comment

4400

7800

8600

PATCH_thread

4100

6900

7600

POST_comment_comment

5700

9900

11000

POST_comment_response

4800

8500

10000

POST_thread

3000

4800

5500

auto_auth

1700

2700

No. of clients = 510

req/s = 32.4

Methods	median response time	95%	99%
DELETE_comment

7700

15000

27000

DELETE_thread

5400

12000

15000

GET_comment_list

6300

14000

31000

GET_thread

8600

19000

36000

GET_thread_list

6400

14000

32000

PATCH_comment

7000

14000

25000

PATCH_thread

6900

16000

35000

POST_comment_comment

7400

15000

36000

POST_comment_response

7000

14000

26000

POST_thread

6000

11000

32000

auto_auth

2900

4000

4400

No. of clients = 510

req/s = 33.9

Methods	median response time	95%	99%
DELETE_comment

7600

14000

19000

DELETE_thread

6700

11000

14000

GET_comment_list

6700

11000

16000

GET_thread

9400

16000

22000

GET_thread_list

7000

11000

17000

PATCH_comment

6900

12000

16000

PATCH_thread

6900

12000

21000

POST_comment_comment

7600

14000

19000

POST_comment_response

7200

13000

18000

POST_thread

6700

11000

15000

auto_auth

2400

3200

3300

No. of clients = 578

req/s = 26.70

Methods	median response time	95%	99%
DELETE_comment

9500

26000

32000

DELETE_thread

8700

25000

32000

GET_comment_list

8600

22000

31000

GET_thread

13000

27000

40000

GET_thread_list

8600

22000

32000

PATCH_comment

8800

22000

32000

PATCH_thread

8800

20000

29000

POST_comment_comment

9200

24000

31000

OST_comment_response

9300

24000

32000

POST_thread

8300

20000

32000

auto_auth

3600

5200

5500

No. of clients = 578

req/s = 40.2

Methods	median response time	95%	99%
DELETE_comment

9100

24000

32000

DELETE_thread

8500

22000

26000

GET_comment_list

8100

22000

32000

GET_thread

12000

26000

40000

GET_thread_list

8300

22000

32000

PATCH_comment

8300

23000

31000

PATCH_thread

8000

23000

28000

POST_comment_comment

9000

24000

32000

OST_comment_response

8500

23000

32000

POST_thread

8300

22000

32000

auto_auth

4100

5100

5400

Versions Compared

Old Version 5

New Version Current

Key

Background:

Results:

Old Index

New Index

Old Index

New Index

Page Comparison

Versions Compared

Old Version 5

New Version Current

Key

<span class="diff-html-added" data-a11y-before="Start of added content" data-a11y-after="End of added content" id="added-diff-0">[data-colorid=m52zw5dwvb]{color:#222222} html[data-color-mode=dark] [data-colorid=m52zw5dwvb]{color:#dddddd}</span>Background:

Results:

Old Index

New Index

Old Index

New Index

Background: