Project Leads (Product): Brad Brown Jenna Makowski Jeremy Ristau Sarina Canelake
Date Proposed: 14-02-2023
Last Updated: 19-04-2023
Status: Community Review
Improving the Video Editor
Many edX partners feel that the Video Editor experience is lacking and needs improvement. We have outlined a three step process to improve the video editor. These steps are:
Update video editor settings (this work is complete). This has moved the Video Editor experience (settings) away from the older frontend and to the new(er) course authoring MFE. The feature parity replacement of settings has allowed us to move the existing experience from edx-platform/studio-frontend into the Course Authoring domain/MFE. This advances our architectural vision, as well as allows us to isolate Video Editor into its own deployable subsystem. New features like Video Uploads are now easier and faster to implement, and are done on a new tech stack so we do not incur more tech debt/increase the monolith.
Build a React based video player (the focus of this document), which is a requirement for video previews as well as the new video uploads. The video uploads path is planned to be incorporated into the Video Editor. The new Video Editor is react based, and the current player does not support that. We want to have an embedded video preview in the editor after upload, and cannot deliver that until we have a react-based player. The new video player will improve upon the user experience for additions and uploads to edX courses and reduces maintenance costs.
Construct the new video uploads (a future project), which is a faster and easier process and addresses one of the most common course author complaints (video management tools in general and uploads in particular are inefficient).
Additionally, Phase 2 of the Modular Content Initiative is reliant on the completion of BD-14 work. The video upload enhancement is the final piece of BD-14 (along with the currently ongoing visual problem editor work). Therefore, a new React based video player enables the video uploads enhancement, and the completion of BD-14 enables Phase 2 of modular content.
Problems
Current Video Player
Currently, many members of the Open edX community have embarked on creating their own video player implementations to overcome deficiencies with the player in the standard Open edX installation. The work represented in this document will result in an editor that the community can coalesce around. The benefits of this are a community-supported video editor that everyone can agree should be used. Our goal is to come to community consensus on the problems of the current video player and agree upon an implementation direction.
The current video player cannot be rendered directly inside of an MFE
The current video player does not work as a "video preview" player inside of the Video Settings Editor
It is not possible to have video preview on the video uploads page with the current player, as described above.
Platform Map
The “Platform Map” is a list of all features throughout the platform that 2U maintains. Periodically, 2U surveys their partners to get feedback on how the platform is working, and maps that feedback to Platform Map areas.
Video Block Platform Map Rating: The data in the chart below are from interviews with 17 partners of the edx.org website. They were asked to rate, for a variety of features, how critical a given feature is (“Criticality”) and how far behind the Open edX software is on delivering this feature at a satisfactory level (“Sentiment Behind”). The “gap rating” indicates features that are the most critical and the most behind, and therefore need the most attention.
The survey showed that the video block is rated as the fourth most important “feature gap” from all the features in courseware + Studio For TNL (behind import/export, text block, and problem block). [Note: TNL stands for Teaching and Learning, an engineering team and focus area at 2U] The map shows that 63% of edx.org partners feel that we are behind on video and 100% feel that the video block is a critical feature. After our work on video settings, React-based player upgrade, and video upload improvements, the team at 2U/edX we will conduct user interviews to test whether these ratings have improved.
Product Requirements
Basic Requirements
Unified video player experience across the platform (Studio, Content Libraries, LMS)
Will work in units with multiple types of components (text, problems, etc)
Player is FOSS and compatible with the Open edX platform’s software license
Parity with Current Video Player
Needs to support playing youtube videos as well as uploaded videos (e.g. mp4 and DASH)
Authors can set start and end times for videos
Authors can upload transcripts
Authors can download transcripts to edit and reupload
Closed captioning is visible in the video player view
Maintain the right-hand side timed transcript display that the current player has
Transcripts can be added in other languages, and can be selected by video viewers for display
Transcript shifts to left side for RTL languages
Accessibility features (i.e. keyboard based controls) must be considered requirements for anything that is going in LMS.
Download video and watch offline (with options to enable or disable)
Various replay speeds
Display SRT captions
Add contextual content such as license information or pdf handouts
Thumbnail upload and delete
Signed URLs: 2U/edx.org currently uses signed URLs (originally from AWS Cloudfront, now from Cloudflare) to ensure that the user is authorized to view a stream.
Mobile video support on parity with current support
Support for current in-video features
MFE Compatibility
Must work inside of an MFE directly
Specifically, the course-authoring MFE, Library Authoring MFE
Rework the Video component so that it directly renders in the Learning MFE with no iframe
Contingency: The legacy LMS frontend uses a way out-of-date version of React, so getting a new video player working within the iframe may present its own challenges. However, an upgrade of React in the legacy LMS frontend is on the Roadmap.
Enhancements from Current Player Capabilities
Ability for authors to upload multiple video files, and for video viewers to select from different video options
Support for DASH video stream
Support for Audio Descriptions - more detail in page comment https://openedx.atlassian.net/wiki/spaces/OEPM/pages/3674734593?focusedCommentId=3687514126
Need a player that will work as a "video preview" player inside of the Video Settings Editor (see Appendix for Video Preview Mock). A new video player is required to include a video preview on the Video Settings page. The existing video player is (according to previous discovery) a pre-React, pre-webpack player which is not viable for our use case of having a Video Preview capability in the Video Settings Editor.
Player supports the ability to preview video slices (e.g. a chapter) to an external source (e.g. having videos previewed inline for a search engine’s results)
Perform downscaling/play downscaled video to handle both high and low-bandwidth clients
Open questions to be solved in technical discovery: Should this be automatic, manual, or both? Should this be dynamic, or rely on multiple hosted file options?
We’re open to any set of options depending on outcome of discovery + feasibility.
Ability to track buffering issues that occur during video streaming
Support for additional in-video features, such as comments
Display VTT captions
Data Compatibility
We have a lot of analytic events that get triggered from the current video player. We MUST keep these requirements as-is, or supply an ADR regarding what would change.
https://github.com/openedx/event-routing-backends/blob/master/docs/event-mapping/Supported_events.rst#video-events
https://github.com/openedx/event-routing-backends/pull/245/files#diff-2974fab80c577a99efe25c1ee2aa50dfbb5496ff46468707dc40e1506652093c
Nice to Haves
Technical Discovery
Background
The current video player cannot be rendered directly inside of an MFE. A new video player is required to include a video preview on the Video Settings page. The current build of the video player has the video player with a custom control bar. The control bar is generated based on the values stored in the Video XBlock. All the features in the custom control bar are part of the regular HTML5 video component except transcripts. It is important to note that transcripts and captions are two different functions. The captions are played in the video view while the transcripts are played next to the video. These two functions do rely on the same file to generate content.
Note: Transcripts are saved as .srt files but in the backend are converted to .sjson
Initial Technical Discovery (edX)
Initial edX Technical Discovery (React video player)
Technical Requirements
It needs to be a Javascript-based component
Almost certainly, it needs an HTML5 video component base
There are a slew of legacy video player compatibility requirements, including (but possibly not limited to):
Discovery will need to be done on what fields currently exist in the XBlock, and as much as possible should go through the Deprecation process before beginning the process of migrating any over
Such as the multi-speed YouTube upload option, which we used in 2012 before YT started generating multiple speeds for you. Open questions: Is anyone using video bumpers? What features weren’t ported to the MFE, or are no longer exposed in the UI? etc
Discovery will also address the details of how we manage transcript files today
Transcripts aren’t standardized and may be in various locations - YouTube, local storage, or in edx-val - a system that edx.org uses, but unclear if anyone in the community does
What do we need to do with legacy Transcript utilities?
Ideally, this is a drop-and-replace player within the current XBlock, such that instances upgrading the version of the block would get the player across all courses with no intervention
Proposed Options
Option 1: https://www.npmjs.com/package/React-player
Option 2: https://www.npmjs.com/package/video-React (based on https://www.npmjs.com/package/video.js )
: https://www.npmjs.com/package/video.js (This comes highly recommended by edX’s accessibility specialist. Additionally, it is being used successfully by Prospectus, edX’s package for displaying the course catalog on edx.org )
Option 4: https://www.npmjs.com/package/plyr
Opinions and Further Questions
Frontend Input
For subtitles, moving to vtt is probably the right path forward. It's a well documented format.
For transcripts, vtt will also serve us well. There are plenty of ways to parse vtt and make it pretty.
The video player should support theme / skin customization. Ideally, we’d be able to associate CSS variables from Paragon to the styles for the video player so that things like colors are consistent between the video player and the rest of the application.
Consider the right home for the new video player (e.g., is it a new UI component in the Paragon design system?). If one of the goals of this project is to ensure consistent video player experience across the platform, it may make sense to live in Paragon.
Multiple Components
How will this play with units that have multiple types of components? For example: text followed by a video followed by a multi-choice problem.
At some point, I expect folks will want other types of components to run outside the IFrame as well. Can the Video component be reimplemented in a way that leaves the door open to a more general solution for React-based learning components?
Is there a reason that we shouldn’t migrate to using .vtt files?
At the very least, we need to get rid of .sjson files in favor of using SRT-only. The advantage of using SRT only is because they exist in the filesystem (and edx-val) already, so the migration path is simple. We’d be able to dump a lot of hacky sjson code. However, VTT comes with better accessibility options. So…
It mostly comes down to hammering out the migration path. We do a lot of hacky SRT/SRT-JSON conversion, and we store these files in a few different ways. Also some of the metadata we have that points to these files will be in OLX, meaning that changes to it will trigger course publishing. We can do things to work around this, but they all add to the complexity of the XBlock itself.
A possible way to migrate to VTT: Scan filesystem, make VTT equivalents of all sjson files - we can do this via file upload because that won’t trigger a course publish. Once we have that, we then have the player code preferentially look for the VTT files.
On edx.org, some of the subtitle stuff is managed via SRTs in edx-val. When that happens, the SRT files are loaded into S3 and that overrides everything else. So in that case, migration would be 2 steps; one migration of edx-val (making new VTT files) then updating db refs to srt & sjson files.
It would also be possible to convert Apparently, it is not rocket science.
Compatibility
The video player is layer upon layer upon layer of crufty compatibility code. We could probably make something a lot simpler overall if we created a new XBlock entirely and migrated video data over to it, though that would make gradual rollout/rollback more difficult.
It is believed that the way to do this while keeping some sense of sanity is to have an explicit transformation step in the code (even if it's not persisted).
Make it so that the new video player has a simple, sane data model to follow, and write a terrible shim layer to convert (up front and in one place) the layers of old terrible into the format the new system expects.
Some pieces of data can just be dropped entirely, like used ages ago to provide different video speeds (before YT provided this functionality natively).
New videos can just persist that new, sane data model, while old ones get converted as needed.
Most backwards compatibility is focused on not breaking content, even as we discover and implement better ways to do things.
Recommendation
edX engineering has concluded that the four options have many commonalities. edX advocates for option number three, video.js, in that this option is best suited to handle edX’s accessibility requirements (WCAG 2.1). This means that while edX currently favors option 3, we are interested in hearing other opinions as to which option best suits the needs of the Open edX community.
Axim’s recommendation is also for option three, video.js. It is not only the more mature project, but also the more consistently maintained one. In addition, the fact it is not written in React makes it the more flexible option: it will be possible to write an in-house React component based on video.js, but tailored specifically to the (many) project specifications without second-guessing the underlying project's React assumptions. There is also ample documentation on how to achieve this. Finally, video.js' plugin architecture makes it possible to implement virtually any needed feature that is not already exposed by video.js itself or one of the plethora of existing plugins. As an added bonus, DefinitelyTyped supports video.js as of a few months ago.
Appendix
Video Uploads
The project to improve video uploads is contingent on a React-based player. Video uploads is a separate workstream that can and will be prioritized after the new video player is built. With the newly enhanced video settings, video uploads are easier and faster to implement, and are also done on a new tech stack so we do not incur more tech debt.
User Problem: The process of uploading videos needs to be improved as can be seen from three comments from edX user interviews in regard to the video block
“Really struggling with this one right now. Process to upload a video in one place, copy the ID and figure out where you want it to be and then separately upload the captions, has been a real challenge for us. And then if we need to make a correction or update to the video, we have to re-upload the video again.”
“The video management tools in general are probably one of the most inefficient systems in edX. Makes her crazy that she has to:
upload videos via one tab
copy a code
go back
edit that component
paste that bit
delete the YouTube link
Put in a name
And then save it”
Video Preview Mockup
