Forums Question Type and Frontend Redesign Testing

Owned by Jim Abramson (Deactivated)
Last updated: Aug 24, 2014
6 min read

Background

Significant user-facing changes have been made to the Forums product, impacting both the Discussion Tab and Inline Discussions.  This page describes the functionality that has changed and is intended to guide pre-deployment manual testing efforts, ensuring that functionality has been implemented correctly and that no regressions or new defects have been introduced.

References

TODO: link relevant JIRA cases here.

Specification

Please see the "Role, Action, Type Grid" section on this wiki https://edx-wiki.atlassian.net/wiki/display/FOR/Forum+Post+Action+Cleanup for details regarding the availability of actions for specific thread types and user roles.

Code

Implementation work is coalesced onto a long running branch in edx-platform: https://github.com/edx/edx-platform/tree/forums/new-post-type

The accompanying changes to cs_comments_service are already merged to master: https://github.com/edx/cs_comments_service/compare/c4f5a1f2b3bf64d92f2896123c1017ac21082b11...66482b544e541bbd004e03153f68ad91321cbd87

Deployment Notes

The migration script requires mongo client version 2.2 or later in order to work correctly.  Its syntax is not compatible with older versions.

After these two dependencies are met, the edx-platform code can be updated.

User-Facing Changes

  1. Revised UX
    1. The New Post form has been redesigned, and its positioning has changed (it now works as an overlay onto the thread viewing area, instead of sliding open and shifting other content downward).
    2. The New Post form has a toggle for selecting the type of thread to create (Discussion or Question).
    3. In the Discussion Tab sidebar, icons have been added to the left of each list item indicating the type of post: Discussion, Question (unanswered), Question (answered).
    4. At the top of the Discussion Tab sidebar, a dropdown control has been added to filter items by type: unread (by current user), unanswered questions, threads flagged as abuse (the latter is only available to users with the TA, Staff, or Admin role).
    5. In both tab and inline contexts, controls and labels for interacting with individual posts have been changed.
      1. threads:
        1. controls: vote, follow, report, pin, edit, delete, close
        2. labels: pinned, reported, closed
      2. responses:
        1. controls: vote, endorse (discussion threads only), mark as answer (question threads only), report, edit, delete
        2. labels: reported
      3. comments:
        1. controls: report, edit, delete
        2. labels: reported
  2. Server Interactions
    1. AJAX interactions on certain post-level operations have been subtly modified.
      1. affected: all controls related to voting, pinning, following, reporting (flagging), or closing.
      2. previously, the user interface would immediately update upon taking any of the above actions, but if the background server call encountered an error, the user would receive an alert message saying that the action did not succeed and must be retried.  At this point the UI would in most cases be in an inconsistent state with server-side data, until refreshing the page from the server.  (e.g. if the "follow" action failed, the followed icon would still show a "followed" state until the next page reload.)
      3. now, in the case of any of the above actions, the user interaction is generally the same, but the UI will revert to the prior state in case of failure (e.g. if the "follow" action failed, the icon indicating its state would revert to the "not followed" state). In addition, error messages have been implemented that are specific to each respective action, replacing generic messages which were previously used in some cases.
  3. Question Type
    1. When creating threads, users must now select between Discussion and Question types.  Discussion threads behave the same as all existing threads / data, while Question threads have a few differences:
      1. Responses to question threads may be marked as answers, either by TAs / Mods / Admins or by the author of the question thread.  Responses to Discussion threads may be endorsed, but only by TAs / Mods / Admins.
      2. the verbiage "mark as answer" only appears with question threads, and "endorse" only appears with discussion threads.
      3. When viewing question threads, those responses marked as answers appear in a separate area, below the original thread and above other responses.  Comments on responses marked as question answers are automatically collapsed (but may be expanded).  Discussion threads follow the same display behavior as before.  Responses to question threads not marked as answers behave the same as before.
      4. Responses marked as question answers will display name of the person who marked the answer, and the date/time it happened.

Non-User-Facing Changes

  1. Inline and Tab views of threads now use the same templates and javascript code (whereas before distinct code, styles, and templates were used for each)
  2. A good deal of unused frontend code, templates, and styles have been removed.

Data Model Changes

  1. CommentThread documents now have a required thread_type field which may either contain "discussion" or "question".  During initial backpopulation, all threads will be set to have type "discussion".
  2. Comment documents now have an optional endorsement field, which will be null by default, and will contain the id of the endorser and date-time of the endorsement if the Comment has been endorsed.  Note that "endorsement" refers to marking responses as answers in question threads.

Load Testing

The new version of the comments service has been load-tested for comparison against the previous baseline.  Results here https://docs.google.com/a/edx.org/spreadsheets/d/1YN9DLoRlYjDFGosw4w3Wzz1aMGzvamq9AdmfTRV8PJM/edit#gid=594971590

In general, we've observed a timing increase in certain write operations, and a timing decrease in read operations.  We're curious why these measures changed, but they haven't changed enough to warrant concerns about release.  In the large, these deltas balance out, and average / median timings and error rates are practically unchanged.

Areas For Manual Testing

  1. Appearance / availability / functionality of post actions and labels based on user role
    1. reporting (flagging) - always available to all users
      1. Students' reported status should only indicate whether or not they individually reported the post
      2. TAs / Mods / Admins see the reported status when any user has reported the post
      3. When TAs / Mods / Admins un-report reported posts, the changed status should take effect for all users
    2. voting - always available to all user types on threads and responses (not comments)
    3. pinning - only changeable by TAs / Mods / Admins
    4. closing - only changeable by TAs / Mods / Admins
    5. endorsing - only for discussion responses - only changeable by TAs / Mods / Admins
    6. marking as answer - only for question responses - only changeable by thread author and TAs / Mods / Admins
    7. following - available to all user types at thread level only
    8. editing and deleting - always available to TAs / Mods / Admins or the content author
  2. Correct behavior for Discussion threads vs Question threads
    1. endorsement rules differ for authors - they can only endorse (mark answers) on question threads
    2. responses marked as question answers are always clearly designated as such (appear in a separate section, include answer marker and date info)
    3. answered and unanswered questions are always clearly distinguished from discussion threads in the sidebar
  3. New sidebar thread list filters 
    1. Flagged thread filter is only available to TAs / Mods / Admins
    2. Unread thread filter accurately filters unread threads for the presently active user
    3. Filters correctly interact with category selection (discussion categories, "posts i'm following" filter)
    4. searches
  4. Tabs vs. inline: thread display should look and behave the same in both tab and inline mode
  5. AJAX - post actions (voting, pinning, following, reporting, closing) should be exercised to ensure they continue to function correctly.
    1. Errors can be deliberately triggered by killing the comments service and/or the LMS
  6. Closed threads: we should ensure we haven't regressed behavior with regard to closed threads.
    1. example: on recent inspection of our branch the "add a comment" box currently appears on closed threads, which may or may not be a regression.

Sandbox Info

http://forums.m.sandbox.edx.org/

in addition to the standard provisioned user accounts, there's a few extra set up for convenience

  • jsa+ta@edx.org / edx   (a community TA enrolled in edx 101)
  • jsa+moderator@edx.org / edx   (a moderator enrolled in edx 101)
  • also, staff@example.com has been granted the discussion admin role in edx 101

Code Version

https://github.com/edx/edx-platform/tree/gprice/forum-action-cleanup   / commit: afd9aa35e4dffe391b6915d386fe5a5fe46174f5

We have deployed this sandbox from the tip of a working branch because we have a few more minor changes to make but we don't want to block testing from starting.

In general, here is what we can expect to change in a subsequent update:

  • a handful of accessibility-related adjustments (making new controls available via keyboard / screen reader) - not expected to change present appearance or mouse interactions
  • a bunch of javascript test fixes (which should be entirely invisible)
  • a merge / rebase of this branch (not expected to impact functionality at all)

We will keep this section updated depending on any discussed / agreed-upon plans to redeploy.