Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

Background:

In reference to  MA-2678 - Getting issue details... STATUS ; we have changed forums implementation with following details (for reference; see PR#192):

  1. Changed post/response/comment behaviour to update post's 'last_activity_at' only at time of creation of post and creation of response/comment on a post. Previously post's 'last_activity_at' was being updated for both creation and update.
  2. To calculate 'read' status of a post, used 'last_acitvity_at' instead of 'updated_at'.
  3. To calculate 'unread comment count' for a post, used 'created_at' instead of 'updated_at'.

'unread comment count' was being calculate as:

unread_comment_count = Comment.collection.find(:comment_thread_id => t._id, :author_id => {"$ne" => user.id}, :updated_at => {"$gte" => read_dates[thread_key]}).count

and had a compound index against it
index({_type: 1, comment_thread_id: 1, author_id: 1, updated_at: 1})
 

With new implementation:

unread_comment_count = Comment.collection.find(:comment_thread_id => t._id, :author_id => {"$ne" => user.id}, :created_at => {"$gte" => read_dates[thread_key]}).count

So, we removed index
index({_type: 1, comment_thread_id: 1, author_id: 1, updated_at: 1})

and added a new one

index({comment_thread_id: 1, author_id: 1, created_at: 1})

 

The results of load tests below show difference between the old and new implementation.

The load tests were run on 4x c4.2xlarge instances.

 

 

 

There appears to be anomaly in the results below (highlighted 99%) where the new implementation is faster than the old. Where as, for other results the percentile of new implementation increases significantly with the increase in load. For example, the first pair of results where no. of clients = 48, the difference is in tens or hundrend and for the last pair of results where no. of clients = 610, the difference is in thousands.

Logically; there should only be increase in response time for GET endpoints, but we see in results that all other endpoints show differences too. This is because all PATCH, POST and DELETE endpoints first call GET endpoint to retrieve an 'id' of thread and/or comment and then make further processing on it.

New Relic:

Here is the new relic permaLink_new_index and permaLink_old_index

where the average rpm for former is 1.55k and for later is 1.48k and 84.5% of requests are made for AccountViewSet.list (i.e. user accounts API for multiple usernames) for later.

 

Old Index

New Index

No. of clients = 48
req/s = 9.4
Methodsmedian response time95%99%

DELETE_comment

250

3101200

DELETE_thread

160

190200

GET_comment_list

150190310

GET_thread

140180220

GET_thread_list

160370520

PATCH_comment

2303601200

PATCH_thread

190240370

POST_comment_comment

290360430

POST_comment_response

250310450

POST_thread

160190200

auto_auth

200250250
No. of clients = 48
req/s = 9.6 --> 18min = 8.9
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

260250310300340480

DELETE_thread

160170200210220230

GET_comment_list

160150200200290320

GET_thread

150150190190260280

GET_thread_list

170170370360520520

PATCH_comment

250220410320410410

PATCH_thread

1901802503601200330

POST_comment_comment

300290390340420490

POST_comment_response

260250320300410390

POST_thread

160160210200310370

auto_auth

190220190230190230


No. of clients = 96
req/s = 18.5
Methodsmedian response time95%99%
DELETE_comment

270

340390
DELETE_thread170230330
GET_comment_list170210330
GET_thread160200250
GET_thread_list190440610
PATCH_comment240330360
PATCH_thread200260350
POST_comment_comment310380470
POST_comment_response260330440
POST_thread170210340
auto_auth180220220


No. of clients = 96
req/s = 18.7  27min = 18.6
Methodsmedian response timemedian response time95%95%99%99%
DELETE_comment280270370340440360
DELETE_thread180170220220270230
GET_comment_list170160230210350340
GET_thread160160220200300270
GET_thread_list210190500460640600
PATCH_comment2502203603101400340
PATCH_thread200190280260380350
POST_comment_comment320300400370510480
POST_comment_response270260350330430420
POST_thread170170220210260340
auto_auth190180260230260230


No. of clients = 162
req/s = 31
Methodsmedian response time95%99%

DELETE_comment

320400490

DELETE_thread

190240280

GET_comment_list

190250370

GET_thread

190250340

GET_thread_list

220520660

PATCH_comment

270360390

PATCH_thread

230310370

POST_comment_comment

350450540

POST_comment_response

300400490

POST_thread

190240340

auto_auth

200200200


No. of clients = 162
req/s = 31.5    31.6
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

350310500410660590

DELETE_thread

210190300240390250

GET_comment_list

210190350250460380

GET_thread

210180320250430360

GET_thread_list

250220620530800670

PATCH_comment

2902604403805701400

PATCH_thread

260220430280580400

POST_comment_comment

380340560450710540

POST_comment_response

330290480400610510

POST_thread

210190310240430370

auto_auth

210210230220230220


No. of clients = 240
req/s = 44.6
Methodsmedian response time95%99%

DELETE_comment

400600700

DELETE_thread

240320530

GET_comment_list

240350480

GET_thread

240360470

GET_thread_list

270630800

PATCH_comment

330510600

PATCH_thread

290420530

POST_comment_comment

420640820

POST_comment_response

360560730

POST_thread

240310440

auto_auth

210210210


No. of clients = 240
req/s = 44.7   45.40
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

540390100065018001300

DELETE_thread

3402406803702000430

GET_comment_list

3402307503501500500

GET_thread

3402307103701500600

GET_thread_list

41027010006401700850

PATCH_comment

42030080049013001200

PATCH_thread

4002807504601300640

POST_comment_comment

580410110067025001000

POST_comment_response

5003609405601600860

POST_thread

3402307103001000500

auto_auth

490190610220610220


No. of clients = 336
req/s = 53
Methodsmedian response time95%99%

DELETE_comment

250055006500

DELETE_thread

98023002900

GET_comment_list

80021002700

GET_thread

92027003300

GET_thread_list

85024003300

PATCH_comment

150039005000

PATCH_thread

120034004100

POST_comment_comment

230054007000

POST_comment_response

170044005900

POST_thread

74020002700

auto_auth

33013001300


No. of clients = 336
req/s = 48.9  54.3
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

2300150034009200440012000

DELETE_thread

17004702600410032006700

GET_comment_list

19005102900500039006800

GET_thread

18005902800650037009900

GET_thread_list

23005805200490068006800

PATCH_comment

22008103200760042001100

PATCH_thread

19007903200540055007800

POST_comment_comment

2400130035009300430012000

POST_comment_response

2200100032007700390010000

POST_thread

18004702800460036006400

auto_auth

180032021003202100320


No. of clients = 460
req/s = 44.5
Methodsmedian response time95%99%

DELETE_comment

5600980012000

DELETE_thread

370062007900

GET_comment_list

370061008100

GET_thread

5200980013000

GET_thread_list

370068008600

PATCH_comment

480079009500

PATCH_thread

460073009900

POST_comment_comment

5500930011000

POST_comment_response

5000830010000

POST_thread

350060009200

auto_auth

200026002600


No. of clients = 460
req/s = 32.4   21min = 49.1
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

7600610038000100004300011000

DELETE_thread

67003500260005700410006400

GET_comment_list

72003000380005000420005700

GET_thread

100004200420007100470008700

GET_thread_list

76003100400005300430006200

PATCH_comment

73004400280007800430008600

PATCH_thread

73004100270006900430007600

POST_comment_comment

780057004100099004300011000

POST_comment_response

780048003800085004200010000

POST_thread

69003000280004800420005500

auto_auth

430017004800270054002700


No. of clients = 510
req/s = 32.4
Methodsmedian response time95%99%

DELETE_comment

77001500027000

DELETE_thread

54001200015000

GET_comment_list

63001400031000

GET_thread

86001900036000

GET_thread_list

64001400032000

PATCH_comment

70001400025000

PATCH_thread

69001600035000

POST_comment_comment

74001500036000

POST_comment_response

70001400026000

POST_thread

60001100032000

auto_auth

290040004400


No. of clients = 510
req/s =     30min = 33.9
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

 7600 14000 19000

DELETE_thread

 6700 11000 14000

GET_comment_list

 6700 11000 16000

GET_thread

 9400 16000 22000

GET_thread_list

 7000 11000 17000

PATCH_comment

 6900 12000 16000

PATCH_thread

 6900 12000 21000

POST_comment_comment

 7600 14000 19000

POST_comment_response

 7200 13000 18000

POST_thread

 6700 11000 15000

auto_auth

 2400 3200 3300


No. of clients = 578
req/s = 26.70
Methodsmedian response time95%99%

DELETE_comment

95002600032000

DELETE_thread

87002500032000

GET_comment_list

86002200031000

GET_thread

130002700040000

GET_thread_list

86002200032000

PATCH_comment

88002200032000

PATCH_thread

88002000029000

POST_comment_comment

92002400031000

OST_comment_response

93002400032000

POST_thread

83002000032000

auto_auth

360052005500


No. of clients = 578
req/s =      40.2
Methodsmedian response timemedian response time95%95%99%99%

DELETE_comment

 9100 24000 32000

DELETE_thread

 8500 22000 26000

GET_comment_list

 8100 22000 32000

GET_thread

 12000 26000 40000

GET_thread_list

 8300 22000 32000

PATCH_comment

 8300 23000 31000

PATCH_thread

 8000 23000 28000

POST_comment_comment

 9000 24000 32000

OST_comment_response

 8500 23000 32000

POST_thread

 8300 22000 32000

auto_auth

 4100 5100 5400


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%
DELETE_comment   
DELETE_thread   
GET_comment_list   
GET_thread   
GET_thread_list   
PATCH_comment   
PATCH_thread   
POST_comment_comment   
POST_comment_response   
POST_thread   
auto_auth   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%
DELETE_comment   
DELETE_thread   
GET_comment_list   
GET_thread   
GET_thread_list   
PATCH_comment   
PATCH_thread   
POST_comment_comment   
POST_comment_response   
POST_thread   
auto_auth   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
  • No labels