Abstract

This proposal results from a collaborative discovery effort by Raccoon Gang and AXIM, building upon the earlier initiative, Proposal: Improve Common Cartridge Import to Open edX, published in April 2024 and approved by the community.

The document details the integration of the CC2OLX converter into the Open edX interface to streamline the import of Common Cartridge course archives. It highlights previous enhancements made to the CC2OLX converter, evaluates integration options within the Open edX ecosystem, and proposes a detailed approach to facilitate seamless migration of course archives from other LMS platforms.

Context & Background

Goals and Objectives

This proposal aims to outline the most effective method for integrating the CC2OLX converter into the Open edX interface, enabling the seamless import of course archives from LMS platforms that utilize the Common Cartridge format into Open edX.

What is a Common Cartridge?

A Common Cartridge (CC) is a set of open standards developed by the 1EdTech member community that enable interoperability between content and systems. Basically, it is a standard for creating and sharing digital learning content in the education sector. This standard is supported by major LMS systems on the market like Blackboard Learn, Moodle, Brightspace, Canvas. As a result, users of these systems can effortlessly export course data in .imscc archive from one system and seamlessly upload it to another, ensuring fast and easy migration between systems. A Common Cartridge file contains features such as an IMS manifest file, roles metadata, LTI links, weblinks, embedded files, and question data.

More information about common cartridge content can be found in Proposal: Improve Common Cartridge Import to Open edX #352

CC2OLX Current State

Open edX currently does not support the Common Cartridge standard, meaning tha courses exported in .imscc format cannot be seamlessly imported into the system. However, a cc2olx converter has been developed to transform Common Cartridge .imscc files into .tar.gz files, which can then be imported into Open edX Studio.

During the work on the previous scope outlined in Proposal: Improve Common Cartridge Import to Open edX #352, the converter was significantly improved.

CDATA nesting into CDATA was avoided as it is prohibited

During some OLX nodes creation (HTML, discussions) the entire content was wrapped into CDATA tag. But such content can already contain CDATA tags inside it, which causes errors. To fix it, the existed CDATA tag occurences were deleted (but its content is preserved) before such OLX nodes creation.

Application was dockerized

To make the converter platform-independent, Dockerfile was added to the converter repository. To use it, it’s needed to build the image and pass script arguments to Docker during container running.

Source specifying for relative links was allowed

It was possible that the .imscc course dump contains files with relative links to resources for some reason not included into it (e.g. some static files of LMS that are not processed by its exporting tool). It was a reason of broken images, not loaded styles etc. To handle such situations, it was allowed to specify the relative links source during converting.

QTI support was improved

During the cc2olx converter usage with correct Common Cartridge course .imscc dumps, some problems with CC XML processing were encountered. It was caused by handling of special cases by the converter while the actual CC XML can be much more complicated. So, the faced problems (default XML attribute values usage, some tags absence, tags nesting etc) were solved.

Also, it's found out that cc2olx completely ignores the fact that both OeX "Text Problem" and CC "Fill in the blank" support regexps, so their support was added.

Block types processing was refactored

There was a problem that when it was needed to add a new block type processing, it was needed to change the code in several places: update Cartridge.get_resource_content to define the new processed content type, add the OLX creation logic for this content type to OlxExport and call it in OlxExport._create_olx_nodes.

It was decided to create a separate class responsible for a block type processing. So, there are separate processors for HTML, LTI, QTI, Video etc. The list of block type processors are specified in settings, so we can control the processors to enable from settings. It will simlify a processor disabling if, for example, a third-party xblock created by the processor is not installed on the edX platform.

Assignment blocks processing was implemented

There is an assignment content type in Common Cartridge specification but it hasn't been supported by the cc2olx converter yet.
It was found out that Common Cartridge Assignment corresponds to oeX ORA xBlock. The corresponding content type processor was added to cc2olx converter.

Custom OeX xBlocks support was added

There are situations when some custom xBlocks not included into the default Open edX installation are installed on the target instance and they can display the Common Cartridge content. For example, PDF xBlock allows to display the PDF file directly on the course page; the xblock-google-drive xblock is used to render Google documents, spreadsheets, presentations, forms inside the unit without the need to follow the link.

So, --content_types_with_custom_blocks (or -c) CLI argument is added to the converter that allows to specify the xBlock types that can be created if the corresponding Common Cartridge content corresponds them. It is allowed to specify several supported xBlocks, but only pdf and google-document types are supported now.

Common Cartridge to OLX plugin for Tutor was created

The Tutor plugin simplifies the cc2olx converter usage. It allows eliminate the converter script and related Python version installation, makes the solution platform-independent. It also integrates the converter into a Tutor ecosystem.

The plugin is located in a new tutor-contrib-cc2olx repository.

Users Analysis

While developing this proposal, we conducted a survey within the Open edX community to gather insights about their experiences transferring courses from other LMS platforms and to understand their expectations for future functionality. The responses provided following insights:

• 17 individuals participated in the survey.

• Slightly more than half (52.9%) of participants reported transferring learning materials to Open edX from other platforms.

• Most respondents used the Open edX import feature in a course; however, only 43.8% used it in a content library.

• Similarly, most respondents used the Open edX export feature in a course, but only 26.7% used it in a content library.

• Courses were most frequently transferred to Open edX from Moodle (50%) and Canvas (25%), with 87.5% of users transferring entire courses.

• None of the participants reported transferring courses from other LMSs regularly (multiple times a month).

• Of those who transferred courses from other LMSs, 62.5% converted course archives to Open Learning XML (OLX) format, while 37.5% rebuilt courses manually in Open edX.

• 80% of respondents indicated they needed to modify content extensively or moderately after importing.

• Challenges:

  • 87.5% cited time and effort as significant challenges.

  • 75% faced issues with loss of content formatting and missing media (e.g., videos and images).

• User opinions on the most important issues to address were evenly distributed across the provided options, with no clear consensus.

Questions & Answers

Open

Approach & Alternatives Analyzes

The solution design process involved analyzing the needs of Open edX users through community feedback and assessing the technical constraints of the current CC2OLX converter and Open edX import functionality. This included reviewing the progress achieved during the enhancement of the Common Cartridge converter. The next steps will focus on validating the proposed scope and solution in collaboration with Open edX working groups.

During the discovery we discussed following strategies for the imscc files importing::

1. Create the page where a user can send the form with imscc file and converter-related flag values
   (link_file, passport-file). When he submits it, the data is sent to backend, backend saves the passed
   files in the temporary directory (or just as temporary files), calls the cc2olxscript to generate
   OLX and provides in the response .tar.gz archive. Temporary files are deleted at the end of the
   request processing. After this, the user can go to the default course import page and use this file
   to follow the default course import process.

2. to have 2 different course import page: the first is the default one, the second one has the form
   with imscc file and converter-related flag values (link_file, passport-file). Also, the
   converting to OLX status should be added in "Course import status" list widget. The form can be
   submitted to the existed endpoint (the view from the backend side will need to check the file format
   in _write_chunk from edx-platform/cms/djangoapps/contentstore/views/import_export.py) and if it's .tar.gz, convert it using cc2olx script to OLX archive, save it into settings.COURSE_IMPORT_EXPORT_STORAGE
   ('olx_import/' + filename) before import_olx task running. Delete imscc archive after the task running. Also, we can create a separate endpoint for imscc sending, but it will have the same logic for the chunk sending, so a lot of logic will be duplicated.

3. in the existed import page, after the archive uploading, if .tar.gz archive is received, the default
   importing flow is applied. If imscc archive is received, the form with converter-related flag values
   (link_file, passport-file) appears which user can fill and press "Submit" button. Here we need to
   think what if the user will re-upload the archive and it will be .tar.gz: maybe, the additional form
   will disappear and the importing process will start immediately. Or we will always have "Submit" button. After imscc-related form submitting, the converting to OLX status should be added in
   "Course import status" list widget. The backend logic will be the same as in the previous option.

4. in the existed import page, the radiobutton with imscc and .tar.gz options exists, it allows to switch between modes from options 2 and 3.

Based on the user feedback and technical analysis we diced to recommend the implementation of third option.

To enable the connection of the converter with Open edX interface, we can add cc2olx to the requirements (it will be installed during platform initialization). This way, it can be called from the container shell. In the Python code, we can use either the cc2olx module directly or the subprocess module and run the cc2olx script with the generated or user-specified parameters. After that, the converted archive from .imscc to .tar.gz will be imported by the existing OeX functionality.

The conversion logic, as described above, will remain in a separate cc2olx package, and the conversion run if an .imscc file is uploaded will be added to the core. Changes will also need to be added to the course-authoring MFE to extend the import page and implement the new modal window.

Scope

High-Level Scope

The scope of the proposal include the following items:

Backend development:

• Automate the migration of videos uploaded to third-party services to simplify the current migration flow.

• Enable automatic detection of archive formats.

• Generate detailed reports listing files that failed to import, along with the reasons for failure.

• Add a waffle flag in the admin panel to support importing the .imscc format.

Frontend Development

• Create a modal window with the configuration settings for

  • Source LMS Domain to ensure conversion of images and media

  • LTI mapping to automatically configure LTI (Learning Tools Interoperability) settings for the content.

• Modal window with the detailed report.

Additional:

• Documentation describing implemented improve.

Detailed Scope

To enable the import of Common Cartridge course archives into Open edX, we have broken down the task into the following work items. The designs of the proposed functionality be found by link.

The envisioned user flow is illustrated in the activity diagram.

Open

Activity diagram of .imscc archives import into Open edX

Value & Impact

The proposed solution will significantly decrease the efforts for course creators to transfer courses from other LMSs that support Common Cartridge format into Open edX.

As a result, the proposed functionality will improve following metrics:

1. Time needed to transfer the Common Cartridge course archive to Open edX;

2. The number of educational institutions who migrate to Open edX from systems that support Common Cartridge.

Further Potential Improvements

1. Export Functionality for Open edX Courses: Develop the capability to export Open edX courses in the .imscc format. This would facilitate the seamless transfer of Open edX courses to LMSs that support the Common Cartridge standard, enhancing interoperability.

2. Content Library Import and Export: Explore the possibility of enabling the import and export of content libraries between Open edX and other LMS platforms. This would streamline content migration and support a more integrated learning ecosystem.

Named Release

Teak

Proposed By

This proposal was prepared by @Mykhailo Marshalok, @Ivan Niedielnitsev and @Mykhailo Chernyshov as a part Raccoon Gang team initiatives.

Additional Info

• CC2OLX converter — GitHub - openedx/cc2olx: A library to convert Common Cartridge Courses to OLX.

• Proposal: Improve Common Cartridge Import to Open edX #352 — Proposal: Improve Common Cartridge Import to Open edX · Issue #352 · openedx/platform-roadmap

• Designs of the improve — Converter Integration to OeX