Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Background:

In reference to  MA-2678 - Getting issue details... STATUS ; we have changed forums implementation with following details (for reference; see PR#192):

  1. Changed post/response/comment behaviour to update post's 'last_activity_at' only at time of creation of post and creation of response/comment on a post. Previously post's 'last_activity_at' was being updated for both creation and update.
  2. To calculate 'read' status of a post, used 'last_acitvity_at' instead of 'updated_at'.
  3. To calculate 'unread comment count' for a post, used 'created_at' instead of 'updated_at'.

'unread comment count' was being calculate as:

unread_comment_count = Comment.collection.find(:comment_thread_id => t._id, :author_id => {"$ne" => user.id}, :updated_at => {"$gte" => read_dates[thread_key]}).count

and had a compound index against it
index({_type: 1, comment_thread_id: 1, author_id: 1, updated_at: 1})
 

With new implementation:

unread_comment_count = Comment.collection.find(:comment_thread_id => t._id, :author_id => {"$ne" => user.id}, :created_at => {"$gte" => read_dates[thread_key]}).count

So, we removed index
index({_type: 1, comment_thread_id: 1, author_id: 1, updated_at: 1})

and added a new one

index({comment_thread_id: 1, author_id: 1, created_at: 1})

 

The results of load tests below show difference between the old and new implementation.

The load tests were run on 4x c4.2xlarge instances.

 

 

 

There appears to be anomaly in the results below (highlighted 99%) where the new implementation is faster than the old. Where as, for other results the percentile of new implementation increases significantly with the increase in load. For example, the first pair of results where no. of clients = 48, the difference is in tens or hundrend and for the last pair of results where no. of clients = 610, the difference is in thousands.

Logically; there should only be increase in response time for GET endpoints, but we see in results that all other endpoints show differences too. This is because all PATCH, POST and DELETE endpoints first call GET endpoint to retrieve an 'id' of thread and/or comment and then make further processing on it.

New Relic:

Here is the new relic permaLink_new_index and permaLink_old_index

where the average rpm for former is 1.55k and for later is 1.48k and 84.5% of requests are made for AccountViewSet.list (i.e. user accounts API for multiple usernames) for later.

 

Old Index

New Index

No. of clients = 48
req/s = 9.4
Methodsmedian response time95%99%

DELETE_comment

250

3101200

DELETE_thread

160

190200

GET_comment_list

150190310

GET_thread

140180220

GET_thread_list

160370520

PATCH_comment

2303601200

PATCH_thread

190240370

POST_comment_comment

290360430

POST_comment_response

250310450

POST_thread

160190200

auto_auth

200250250
No. of clients = 48
req/s = 9.6
Methodsmedian response time95%99%

DELETE_comment

260310340

DELETE_thread

160200220

GET_comment_list

160200290

GET_thread

150190260

GET_thread_list

170370520

PATCH_comment

250410410

PATCH_thread

1902501200

POST_comment_comment

300390420

POST_comment_response

260320410

POST_thread

160210310

auto_auth

190190190


No. of clients = 96
req/s = 18.5
Methodsmedian response time95%99%
DELETE_comment

270

340390
DELETE_thread170230330
GET_comment_list170210330
GET_thread160200250
GET_thread_list190440610
PATCH_comment240330360
PATCH_thread200260350
POST_comment_comment310380470
POST_comment_response260330440
POST_thread170210340
auto_auth180220220


No. of clients = 96
req/s = 18.7
Methodsmedian response time95%99%
DELETE_comment280370440
DELETE_thread180220270
GET_comment_list170230350
GET_thread160220300
GET_thread_list210500640
PATCH_comment2503601400
PATCH_thread200280380
POST_comment_comment320400510
POST_comment_response270350430
POST_thread170220260
auto_auth190260260


No. of clients = 162
req/s = 31
Methodsmedian response time95%99%

DELETE_comment

320400490

DELETE_thread

190240280

GET_comment_list

190250370

GET_thread

190250340

GET_thread_list

220520660

PATCH_comment

270360390

PATCH_thread

230310370

POST_comment_comment

350450540

POST_comment_response

300400490

POST_thread

190240340

auto_auth

200200200


No. of clients = 162
req/s = 31.5
Methodsmedian response time95%99%

DELETE_comment

350500660

DELETE_thread

210300390

GET_comment_list

210350460

GET_thread

210320430

GET_thread_list

250620800

PATCH_comment

290440570

PATCH_thread

260430580

POST_comment_comment

380560710

POST_comment_response

330480610

POST_thread

210310430

auto_auth

210230230


No. of clients = 240
req/s = 44.6
Methodsmedian response time95%99%

DELETE_comment

400600700

DELETE_thread

240320530

GET_comment_list

240350480

GET_thread

240360470

GET_thread_list

270630800

PATCH_comment

330510600

PATCH_thread

290420530

POST_comment_comment

420640820

POST_comment_response

360560730

POST_thread

240310440

auto_auth

210210210


No. of clients = 240
req/s = 44.7
Methodsmedian response time95%99%

DELETE_comment

54010001800

DELETE_thread

3406802000

GET_comment_list

3407501500

GET_thread

3407101500

GET_thread_list

41010001700

PATCH_comment

4208001300

PATCH_thread

4007501300

POST_comment_comment

58011002500

POST_comment_response

5009401600

POST_thread

3407101000

auto_auth

490610610


No. of clients = 336
req/s = 53
Methodsmedian response time95%99%

DELETE_comment

250055006500

DELETE_thread

98023002900

GET_comment_list

80021002700

GET_thread

92027003300

GET_thread_list

85024003300

PATCH_comment

150039005000

PATCH_thread

120034004100

POST_comment_comment

230054007000

POST_comment_response

170044005900

POST_thread

74020002700

auto_auth

33013001300


No. of clients = 336
req/s = 48.9
Methodsmedian response time95%99%

DELETE_comment

230034004400

DELETE_thread

170026003200

GET_comment_list

190029003900

GET_thread

180028003700

GET_thread_list

230052006800

PATCH_comment

220032004200

PATCH_thread

190032005500

POST_comment_comment

240035004300

POST_comment_response

220032003900

POST_thread

180028003600

auto_auth

180021002100


No. of clients = 460
req/s = 44.5
Methodsmedian response time95%99%

DELETE_comment

5600980012000

DELETE_thread

370062007900

GET_comment_list

370061008100

GET_thread

5200980013000

GET_thread_list

370068008600

PATCH_comment

480079009500

PATCH_thread

460073009900

POST_comment_comment

5500930011000

POST_comment_response

5000830010000

POST_thread

350060009200

auto_auth

200026002600


No. of clients = 460
req/s = 32.4
Methodsmedian response time95%99%

DELETE_comment

76003800043000

DELETE_thread

67002600041000

GET_comment_list

72003800042000

GET_thread

100004200047000

GET_thread_list

76004000043000

PATCH_comment

73002800043000

PATCH_thread

73002700043000

POST_comment_comment

78004100043000

POST_comment_response

78003800042000

POST_thread

69002800042000

auto_auth

430048005400


No. of clients = 510
req/s = 32.4
Methodsmedian response time95%99%

DELETE_comment

77001500027000

DELETE_thread

54001200015000

GET_comment_list

63001400031000

GET_thread

86001900036000

GET_thread_list

64001400032000

PATCH_comment

70001400025000

PATCH_thread

69001600035000

POST_comment_comment

74001500036000

POST_comment_response

70001400026000

POST_thread

60001100032000

auto_auth

290040004400


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

OST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

OST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%
DELETE_comment   
DELETE_thread   
GET_comment_list   
GET_thread   
GET_thread_list   
PATCH_comment   
PATCH_thread   
POST_comment_comment   
POST_comment_response   
POST_thread   
auto_auth   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   


No. of clients = 
req/s =
Methodsmedian response time95%99%
DELETE_comment   
DELETE_thread   
GET_comment_list   
GET_thread   
GET_thread_list   
PATCH_comment   
PATCH_thread   
POST_comment_comment   
POST_comment_response   
POST_thread   
auto_auth   


No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
No. of clients = 
req/s =
Methodsmedian response time95%99%

DELETE_comment

   

DELETE_thread

   

GET_comment_list

   

GET_thread

   

GET_thread_list

   

PATCH_comment

   

PATCH_thread

   

POST_comment_comment

   

POST_comment_response

   

POST_thread

   

auto_auth

   
  • No labels