...
After more than a year of field testing this for (1) new courses and (2) reruns of pipelined courses, we have encountered various issues with the pipeline and its required workflow.
Overview
- Current video pipeline is an inefficient, manual process. Process promotes waste, non-valued work, and ineffective employee/partner communications.
- Process relies on risky, ever-changing third-party hosting from YouTube that is not supported by a SLA.
- Process was created in early days of edX when org had < 5 partners. Given significant increase in partners, the current YouTube workflow is not sustainable or scalable.
- Current video process is fragile and produces ongoing fails for partners and learners.
Burden on edX Video Team
Some of which have been a time-consuming burden on the video team who have manually worked around the issues. For example:
...
- Creating YouTube CMS Content Owner (CO) accounts per partner
- Numerous handoffs between media team and partners to create YouTube channels in CMS for each course
- Manually enabling every course for video upload by adding HEX ID to studio
- Resolving frequent YouTube errors due to YT backend changes
Errors experienced by Mobile Learners
...
While a long-term plan to reassess the edX Asset Pipeline Strategy is under discussion, we plan to address the above issues through tactical short-term measures.
1. Simplify Consolidate and Validate Video Identifiers in Video Authoring UI
Addresses: Scenarios #6 and #7 above.
...
- If the video pipeline is enabled for the course, display a warning if an option other than "Video ID" is used.
- When the "Video ID" option is chosen,
- verify that the provided ID exists in the VAL database.
- if the ID is not already associated with the course in the VAL database, create the association.
- ONLY do this validation if Video Pipeline is enabled on the server.
- To handle existing video blocks that don't follow the mutual exclusivity, use the following preference order, which is used by the mobile apps:
- If Video ID is specified, use this field iff the ID is in VAL.
- Else if a Hosted URL is specified, use this field.
- Else, use the YouTube field.
- If there are any resulting content issues from the above preference order, course teams will need to fix them, since it affects the mobile apps.
- Consider removing old YouTube ID fields from UI as well - since they are no longer needed.
2. Simplify Video Linking
...
- Course Import - Note that, a chicken-and-egg problem can arise if the course author is prevented from importing the course without a way to actually fix the course - especially since (1) courses are automatically published upon import and (2) the video pipeline is not available outside of Prod.
- Course Publish - Note that, today, the underlying database publishes courses at various times, without an explicit Publish action by the course author.
- Manual Validation UI - This may be the best option at this time, given the issues with the above choices.
- Manual Script or Command - For a bulk validation report across courses, allow open edX operators to validate using a script or management command.
...
- To minimize the scope for v1.5
- go with options 3 and 4 for the validation entry points.
- output all validation failures and warnings in a spreadsheet, so we don't have to implement a lot of UI features such as sorting, searching, paging, etc.
- For Scott Dunn (Deactivated): Option 4 can be done first (to provide to Course teams to validate courses), then Option 3 (surface through UI), then Option 1 (to automatically validate on import).
4. Move Video Pipeline Settings into server-specific storage
...
- Deleting unneeded videos from the library (
)Jira Legacy server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key MA-326 - Better error states (
,Jira Legacy server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key MA-769
)Jira Legacy server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key MA-242 - Warning message when upload will be interrupted (
)Jira Legacy server JIRA (openedx.atlassian.net) serverId 13fd1930-5608-3aac-a5dd-21b934d3a4b4 key MA-333
Transferring a Course from Edge to Prod Workflow:
- On Prod, create your Course with Course Team (with negotiated org-course-run values)
- On Prod, specify Video Pipeline Settings, which remains on Prod (not in the OLX of the course)
...
- - assuming #4 above is implemented, fixing MA-1302.
- Import from Edge to Prod.
- Run the Validation Script - finding issues with missing VideoIDs, etc.
- Upload Videos on Prod.
- To Fix Video IDs, either
- Fix on Prod:
- Update all videos with issues on Prod to have correct information, or set OnlyOnWeb to disable VideoID.
- Export back to Edge if course teams want to continue to editing the course on Edge.
- Importing into Edge should bypass any validation issues since Video Pipeline is not enabled on Edge.
- Fix on Edge:
Manually copy VideoIDs from Prod onto course on Edge.
Update all videos with issues to have correct information, or set OnlyOnWeb to disable VideoID.
Re-import back to Prod and verify that all issues have been fixed.
- Fix on Prod: