Video Pipeline 1.0 Successes

After more than a year of enabling version 1.0 of the video pipeline, we have successfully:

Video Pipeline 1.0 Issues

After more than a year of field testing this for (1) new courses and (2) reruns of pipelined courses, we have encountered various issues with the pipeline and its required workflow. 

Overview

Burden on edX Video Team

Some of which have been a time-consuming burden on the video team who have manually worked around the issues. For example:

Errors experienced by Mobile Learners

Some of which have manifested in learner-facing video issues on the released mobile apps.  For example, per :

Scenarios causing Course Teams to Misuse the Pipeline

After some analysis, the following set of scenarios summarize the root cause of the issues:

  1. Course author updates/adds videos after video migration.

    For courses that underwent a transformation using a manual video migration script run by the engineering team, the course staff were never trained to use the Video Upload tool.  So then, when they later updated or added videos in the course, they didn't know about the new process and didn't use the Video Upload tool.

  2. Course author updates/adds videos in a Course or in a Course rerun, bypassing the Video Upload tool.

    Similar to #1, except for courses that had been using the Video Upload tool.  In this case, the video was updated/added by a course author that was unaware or forgot about the process.

  3. Course updated/created in another environment (Edge, OLX, Open edX) and then imported into Prod.

    In this case, the course team was using an open edX environment to experiment and create their course content.  Once they were ready to start their course, they imported their course into studio.edx.org.  However, since there was no video pipeline service available at the time of course creation, the videos in the course were not processed through the pipeline.

  4. Video pipeline integration settings overridden on import on Prod.  ()

    When a course copy having empty video pipeline settings (e.g., when importing from an open edX environment) is imported onto its Production copy, it overrides any previously stored settings in Production.  In this case, video pipeline integration settings are reset and the Video Upload page disappears for that Production course.

  5. Manual entering of Video ID results in incorrect or missed value.

    Since the Video Upload page is separated from the Video Authoring content block, course authors are required to (somehow) find the correct corresponding video and then manually copy and paste the corresponding Video ID into the corresponding Video block.  This is understandably an error prone process and can lead to unwanted errors.

  6. Manual re-use of Video ID across courses.

    When course authors want to reuse the same video across courses, they may manually enter the Video ID of a video that appears in the Video Upload page of another course.  Although theoretically allowed, the video abstraction layer (VAL) isn't updated to link the video with the other course. This results in a performance issue since a bulk search for all videos in a course will not include those outliers.

    Note/Question: Should we just update the Video Authoring block to update VAL to link to the course whenever a legitimate Video ID is added?

  7. Licensed video not marked as only_on_web.

    In some legitimate cases where a video is not meant to have a downloadable reference (e.g., for licensed content), the intention is to mark the video as `only_on_web`.  However, this is not made clear in the Video Authoring block and so course teams forget to do so.

    Note/Question: Should these videos be marked as 'only_streaming' instead?  Now that we have YouTube support on the mobile apps, they can still be streamed using YouTube; just not downloaded.

Video Pipeline 1.5 Plan

While a long-term plan to reassess the edX Asset Pipeline Strategy is under discussion, we plan to address the above issues through tactical short-term measures.

1. Consolidate and Validate Video Identifiers in Video Authoring UI 

 Addresses: Scenarios #6 and #7 above.

Related ticket: 

We want to simplify the Video Authoring block, so that it's clear to course authors that we support only 3 mutually exclusive cases:

  1. YouTube - with YouTube link supplied by the video author.

  2. Video ID - if the video was processed through the edX Video Pipeline workflow for this course (or processed through another course).

    Note:     when processed through the Video Pipeline, a video will have multiple encodings, including YouTube, Web, Mobile, Audio-only, etc.  So it would be "nice" for the course author to see the links to the various encodings in the Video Authoring UI once the video is processed (related ticket: ).

    Note 2: Upon saving the Video block, the corresponding video in the Video pipeline database (VAL) should be updated to have a link to the course, if it doesn't already have so.

  3. Externally Hosted - with an external link supplied by the video author (e.g., video hosted on their own S3 site, if they are not using the edX Video Pipeline).

Additional Notes

2. Simplify Video Linking

Addresses: Scenarios #1, #2, #5 above.

Related ticket: 

Allow course authors to search for their upload videos within the Video Authoring block by providing a type-ahead/searchable dropdown with the file display names and Video IDs of all the uploaded videos associated with the course.  This prevents unnecessary user errors (forgetting, not knowing, incorrect copy and pasting, etc).

Additional Notes

3. Course Validation

Addresses: Scenarios #3 and #4 above.
Related ticket: 

Although Course Validation can be generalized to other requirements and other features, for the purpose of Video assets, adding a validation step in the Course Publishing/Importing workflow will allow course authors to understand all the video-related issues in their course.  In particular, for courses that are designated as "Mobile Available", reporting all the non-mobile available videos in the course will allow course authors to address and fix those issues.

Validation entry points:

  1. Course Import - Note that, a chicken-and-egg problem can arise if the course author is prevented from importing the course without a way to actually fix the course - especially since (1) courses are automatically published upon import and (2) the video pipeline is not available outside of Prod.
  2. Course Publish - Note that, today, the underlying database publishes courses at various times, without an explicit Publish action by the course author.
  3. Manual Validation UI - This may be the best option at this time, given the issues with the above choices.
  4. Manual Script or Command - For a bulk validation report across courses, allow open edX operators to validate using a script or management command.

Validation results:

  1. Failed
  2. Warnings
  3. Success

It is a policy decision on what types of video issues are actual failures rather than simply warnings.  And it's also a matter of policy on whether we want to allow courses with video issues to be published to edx.org.

Additional Notes

4. Move Video Pipeline Settings into server-specific storage

Addresses: Scenario #4 above.

Related ticket: 

Store the video pipeline credentials for each course in a django-admin accessible VAL database.  This will allow the settings to remain on the server instance and not overridden when the course is imported from other servers.

5. Video Pipeline UI Improvements

 To provide a better user experience for course authors, the following UI improvements are highly recommended, in the short term, by the edX video team.

  1. Deleting unneeded videos from the library ()
  2. Better error states (, )
  3. Warning message when upload will be interrupted ()


Transferring a Course from Edge to Prod Workflow: