Page Comparison

With the addition of new query param (i.e. requested_fields=profile_image) to discussion API GET endpoints for thread and comment; the discussion APIs got exposed to the possibility of performance degradation because this new param fetches and prepares the profile image related data of all the users that are participating in thread or comment.

We have gathered and compared two sets of load test results. The former results (i.e. with profile_image) are run on the latest changes and the later are run on older implementation (i.e. without profile_image).Background:

In reference to 

1. Changed post/response/comment behaviour to update post's 'last_activity_at' only at time of creation of post and creation of response/comment on a post. Previously post's 'last_activity_at' was being updated for both creation and update.
2. To calculate 'read' status of a post, used 'last_acitvity_at' instead of 'updated_at'.
3. To calculate 'unread comment count' for a post, used 'created_at' instead of 'updated_at'.
   
   

'unread comment count' was being calculate as:

unread_comment_count = Comment.collection.find(:comment_thread_id => t._id, :author_id => {"$ne" => user.id}, :updated_at => {"$gte" => read_dates[thread_key]}).count

and had a compound index against it
index({_type: 1, comment_thread_id: 1, author_id: 1, updated_at: 1})
 

With new implementation:

unread_comment_count = Comment.collection.find(:comment_thread_id => t._id, :author_id => {"$ne" => user.id}, :created_at => {"$gte" => read_dates[thread_key]}).count

So, we removed index
index({_type: 1, comment_thread_id: 1, author_id: 1, updated_at: 1})

and added a new one

index({comment_thread_id: 1, author_id: 1, created_at: 1})

The results of load tests below show difference between the old and new implementation.

There appears to be anomaly in the results below (highlighted 99%) where the new implementation is faster than the old. Where as, for other results the percentile of new implementation increases significantly with the increase in load. For example, the first pair of results where no. of clients = 48, the difference is in tens or hundrend and for the last pair of results where no. of clients = 610, the difference is in thousands.

...