Page Comparison

...

After more than a year of field testing this for (1) new courses and (2) reruns of pipelined courses, we have encountered various issues with the pipeline and its required workflow. 

Overview

• Current video pipeline is an inefficient, manual process. Process promotes waste, non-valued work, and ineffective employee/partner communications.
• Process relies on risky, ever-changing third-party hosting from YouTube that is not supported by a SLA.
• Process was created in early days of edX when org had < 5 partners. Given significant increase in partners, the current YouTube workflow is not sustainable or scalable.
• Current video process is fragile and produces ongoing fails for partners and learners.

Burden on edX Video Team

Some of which have been a time-consuming burden on the video team who have manually worked around the issues. For example:

...

• Creating YouTube CMS Content Owner (CO) accounts per partner
• Numerous handoffs between media team and partners to create YouTube channels in CMS for each course
• Manually enabling every course for video upload by adding HEX ID to studio
• Resolving frequent YouTube errors due to YT backend changes

Errors experienced by Mobile Learners

...

While a long-term plan to reassess the edX Asset Pipeline Strategy is under discussion, we plan to address the above issues through tactical short-term measures.

...

1. Consolidate and Validate Video Identifiers in Video Authoring UI 

...

 Addresses: Scenarios #6 and #7 above.

Related ticket: 

...

We want to simplify the Video Authoring block, so that it's clear to course authors that we support only 3 mutually exclusive cases:

1. YouTube only - with YouTube link supplied by the video author.
   
   
2. Video ID only - if the video was processed through the edX Video Pipeline workflow for this course (or processed through another course).
   
   Note: when processed through the Video Pipeline, a video will have multiple encodings, including YouTube, Web, Mobile, Audio-only, etc.  So it would be "nice" for the course author to see the links to the various encodings in the Video Authoring UI once the video is processed (related ticket:

   Jira Legacy
   server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key MA-150

   ).
   
   Note 2: Upon saving the Video block, the corresponding video in the Video pipeline database (VAL) should be updated to have a link to the course, if it doesn't already have so.
   
   
3. Externally Hosted Video only - with an external link supplied by the video author (e.g., video hosted on their own S3 site, if they are not using the edX Video Pipeline).
   Decouple Video Upload and Video Reporting and allow course authors to upload directly through the Video Authoring block (related ticket:
   

Additional Notes

• If the video pipeline is enabled for the course, display a warning if an option other than "Video ID" is used.
• When the "Video ID" option is chosen,
  • verify that the provided ID exists in the VAL database.
  • if the ID is not already associated with the course in the VAL database, create the association.
  • ONLY do this validation if Video Pipeline is enabled on the server.
• To handle existing video blocks that don't follow the mutual exclusivity, use the following preference order, which is used by the mobile apps:
  1. If Video ID is specified, use this field iff the ID is in VAL.
  2. Else if a Hosted URL is specified, use this field.
  3. Else, use the YouTube field.
• If there are any resulting content issues from the above preference order, course teams will need to fix them, since it affects the mobile apps.
• Consider removing old YouTube ID fields from UI as well - since they are no longer needed.

2. Simplify Video Linking

Addresses: Scenarios #1, #2, #5 above.

Related ticket: 

...

Allow course authors to search for their upload videos

...

within the Video Authoring block

...

by providing a type-ahead/searchable dropdown with the file display names and Video IDs of all the uploaded videos associated with the course.  This prevents unnecessary user errors (forgetting, not knowing, incorrect copy and pasting, etc).

...

Additional Notes

• Populate the drop-down by querying VAL for all videos associated with the course.
• Allow Course Authors to still enter or cut-and-paste the Video IDs directly into the video block, in case they didn't upload the video through this course.
• Follow through with the additional notes captured by #1 above.

3. Course Validation

Addresses: Scenarios #3 and #4 above.
Related ticket: 

...

Although Course Validation can be generalized to other requirements and other features, for the purpose of Video assets, adding a validation step in the Course Publishing/Importing workflow will allow course authors to understand all the video-related issues in their course.  In particular, for courses that are designated as "Mobile Available", reporting all the non-mobile available videos in the course will allow course authors to address and fix those issues.

Validation entry points:

1.    a. Course Import - Note that, a chicken-and-egg problem can arise if the course author is prevented from importing the course without a way to actually fix the course .
      b. - especially since (1) courses are automatically published upon import and (2) the video pipeline is not available outside of Prod.
2. Course Publish - Note that, today, the underlying database publishes courses at various times, without an explicit Publish action by the course author.   c.
3. Manual Validation UI - This may be the best option at this time, given the issues with the above choices.
4. Manual Script or Command - For a bulk validation report across courses, allow open edX operators to validate using a script or management command.

Validation results:

1.    a. Failed   b.
2. Warnings
3.    c. Success

It is a policy decision on what types of video issues are actual failures rather than simply warnings.  And it's also a matter of policy on whether we want to allow courses with video issues to be published to edx.org.

Additional Notes

• To minimize the scope for v1.5
  • go with options 3 and 4 for the validation entry points.
  • output all validation failures and warnings in a spreadsheet, so we don't have to implement a lot of UI features such as sorting, searching, paging, etc.
  • For Scott Dunn (Deactivated): Option 4 can be done first (to provide to Course teams to validate courses), then Option 3 (surface through UI), then Option 1 (to automatically validate on import).

4. Move Video Pipeline Settings into server-specific storage

Addresses: Scenario #4 above.

Related ticket: 

Store the video pipeline credentials for each course in a django-admin accessible VAL database.  This will allow the settings to remain on the server instance and not overridden when the course is imported from other servers.

5. Video Pipeline UI Improvements

 To provide a better user experience for course authors, the following UI improvements are highly recommended, in the short term, by the edX video team.

1. Deleting unneeded videos from the library (

   Jira Legacy
   server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key MA-326

   )
2. Better error states (

   Jira Legacy
   server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key MA-769

   ,

   Jira Legacy
   server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key MA-242

   )
3. Warning message when upload will be interrupted (

   Jira Legacy
   server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key MA-333

   )

...

Transferring a Course from Edge to Prod Workflow:

• On Prod, create your Course with Course Team (with negotiated org-course-run values)
• On Prod, specify Video Pipeline Settings, which remains on Prod (not in the OLX of the course) - assuming #4 above is implemented, fixing MA-1302.
• Import from Edge to Prod.
• Run the Validation Script - finding issues with missing VideoIDs, etc.
• Upload Videos on Prod.
• To Fix Video IDs, either
  
  1. Fix on Prod:
     
     1. Update all videos with issues on Prod to have correct information, or set OnlyOnWeb to disable VideoID.
     2. Export back to Edge if course teams want to continue to editing the course on Edge.
     3. Importing into Edge should bypass any validation issues since Video Pipeline is not enabled on Edge.
  2. Fix on Edge:

     1. Manually copy VideoIDs from Prod onto course on Edge.

     2. Update all videos with issues to have correct information, or set OnlyOnWeb to disable VideoID.

     3. Re-import back to Prod and verify that all issues have been fixed.