Summary

The "LTI Redesign Project" on Open edX aims to enhance usability and user satisfaction through user-centered design. Objectives include simplifying LTI addition and reuse for administrators and course creators, and accommodating users of all levels. The project involved competitor analysis, heuristic evaluations, and usability testing. Prototypes were developed and tested, resulting in several iterations to optimize interface and functionality.
You can see the final prototype here: LTI Prototype
And a presentation video here: https://drive.google.com/file/d/15YSrZzLy1soCN5A-ymggogBL4lRYugR1/view https://drive.google.com/file/d/1DY4-EbMwAVqv3JWgB91lfiVN-5mGXREm/view

Table of contents

Project Objectives

1. Facilitate the process of adding LTIs for administrators and course creators.

2. Create the flow of reusing LTIs in a friendly way

3. Adapt to make it easy to use for all types of users, from zero experts to the most experienced users.

Methodology

Research

• Competitor Analysis: For the creation of the moodboard, several competitor flows were analyzed, in addition to those from other platforms that provided a structure inspiring the creation process.

Research LTI Providers

• Heuristic Evaluation: Heuristic evaluations were performed to identify usability weak points in the interface.

• Usability tests: Interviews were conducted with 5 representative users to understand their needs, challenges, and expectations.

Design

• Prototyping: Interactive prototypes were developed based on research findings to test new solutions and improvements.

• UX Writing: Texts were crafted and refined with a focus on clarity, conciseness, and consistency to enhance user communication.

LTI Hierarchy

LTI hierarchy

Below we present a hierarchy to highlight the nomenclature we use: "Provider, LTI Tool, Implementations"

Open

Findings and Recommendations

Heuristic Evaluation

Heuristic Analysis - LTI project | Notion

Major areas for improvement identified were:

• Create recognition of errors. For users, it is difficult to know if there are errors.

• Evaluate the technical feasibility of standardizing LTI 1.3 preview functionality akin to LTI 1.1. in STUDIO

• Add tooltips with more information for users with different knowledge levels.

• View the technical feasibility of adding an option in LTI 1.3 only URL.

  This idea stems from my experience testing third-party tools in their free versions, where I was able to create interactive resources. To integrate them into my course, I simply used the URL Launch feature in version 1.3.

  During interviews, one participant mentioned attempting to use LTI but found it too complex to proceed. Thus, this new option to request a URL provides an easy avenue for experimenting with interactive content. While it would only serve as interactive content without data collection or scoring, given the lack of connection with external tools, this option would greatly simplify the process for some course creators.

Usability tests

LTI Project — Usability tests

1. Atentamente

2. Inclusive Conservation

3. Technical user / César Montilla

4. Zach Hancock

5. Naomi Maekawa

6. Interview - Scott Dunn

7. Laura Vence

8. Ana Gómez

Findings from the LTI tool addition process interviews:

Problems with the existing process:

• Lack of information about LTI providers

• Difficult to discover new tools

• Unclear configuration options

Desired improvements:

• Easier creation of new LTIs from existing ones

• Clear information about LTI providers and pricing

• Centralized repository for finding LTI tools

• App store-like system for discovering and installing tools

Usability testing revealed:

• The interface needs to be more user-friendly with clear explanations

• Less technical knowledge is required from users

Prototype testing showed:

• Improved interface and ability to choose LTI versions were well-received

• Areas for improvement include terminology, button functions, and documentation

Developer feedback highlighted the need for:

• Differentiating between one-time and reusable LTIs

• Sharing configured LTIs between instructors

• Permissions management for LTI tools

• Testing functionalities outside a course context

• Representing LTI tools at a platform level

Interviewee feedback emphasized the value of:

• Seeing all LTI tool instances within a course

• Reusing existing LTI tools

• Configuring tools at organization and course levels

Prototypes

LTI Prototype

We have developed two prototypes for our project: one designed specifically for course creators seeking to reuse content and tailored to anticipated technical constraints, and another that represents our ideal model. The latter includes a 'Learning Tools Center', where an administrator can create and manage usage permissions for each implemented tool.

1. Prototype Objectives:

   • Test the following flows: Create a new LTI, reuse the LTI, and edit configurations of a reused LTI.

   • Collect feedback from users regarding the process.

   • You recognize pain points during the process.

2. Functionalities and Features:

   • Create, reuse, edit, and view LTIs.

   • Interact with fields in Tool Settings that are in the hierarchy with others, too.

3. Iterations and Refinements:

   • 11 Iterations that were based on heuristic analysis, UX Writing, usability testing findings, post-test analysis.

      

First Iteration:

LTI Prototype

In the first iteration, we took as a base the wireframe that was already created, and we included the important settings that are already included in the process of creating a wireframe

First Version

 

Open

 

First Iteration

 

 

Second Iteration:

After testing with different LTIs in STUDIO, we noticed that in version 1.1 you get a preview of the LTI and in version 1.3 you don't. So we determined that it would be important for the administrators to see if their implementation works correctly, so we added a testing section (Technical feasibility needs to be validated) so that the administrators are 100% sure that their implementation works correctly and that the course creators do it. They can easily be used.

Third Iteration:

In this iteration, types of visualizations of the LTI cards added from the view of an admin and the view of a course creator were explored, keeping the rest of the flow the same.

Fourth Iteration:

The styles were changed according to the Open edX design system since it was designed with the edX style. Admin view

Fifth Iteration:

• We created a prototype in Spanish to adapt to the user's language and allow them to focus on the tasks rather than on translating everything.

• We proposed 2 types of modals, but the first modal made the user feel like they were completely leaving the screen they were on.

• Tabs were created that in a way filtered the course LTIs, the LTIs created by the course author, and the LTIs created by other course authors.

Findings of this iteration:

• The user did not fully understand the tabs.

• The texts were not clear enough.

Sixth and seventh Iteration:

This iteration addresses a use case identified during user interviews: administrators needing to monitor LTI usage beyond courses they've explicitly enabled. We added a list in the “Basic settings” section that details where the LTI is currently used. Clicking on an entry could previously redirect the administrator to that specific implementation. However, the original placement within the “Enable reuse” section with checkboxes is confusing. Users mistakenly thought unchecking a box would delete the LTI. To address this, we've introduced a dedicated “Usage” tab to display where the LTI is currently used.

Eighth Iteration:

LTI Prototype

After gathering information in the LTI Working Group, we have crafted a proposal that organizes options by provider. This will enable administrators to create LTI groups either collectively or individually.

Ninth Iteration:

LTI Prototype

In research, we found that it is difficult to automatically group LTIs by providers using the Launch URL since providers can handle URLs that are not identified with the provider's URL, and by keys it can also be not as effective. So we propose a manual, but quick process so that the process is not cumbersome.

Tenth Iteration:

LTI Prototype

In this iteration, the provider process was synthesized along with the basic settings, and the corresponding filters were added to facilitate the search for administrators and course creators. Additionally, we created a brand so that the administrator can easily identify which courses are enabled to be reused.

Eleventh Iteration:

The changes made in this latest iteration are based on findings obtained from the last two usability tests, as well as a post-test analysis.

The test results have allowed us to improve and add:

• More eye-catching tooltips, as the previous ones were not sufficiently visible.

• Some tooltips now provide information that addresses the question: Where can I find it? What is it for? and When should I use it?

• It is clarified for the course author that reusing an LTI "an implementation" does not alter the original template created by the admin, and for the admin editing their reusable LTI does not affect past implementations, only future ones.

After analysis, the question also arose of how we can ensure that the administrator has access to know which implementations share keys and Launch URLs to control expenses.

Through the provider added by the admin (This idea was in the tenth iteration)

• One option would be through the provider manually added by the administrator. Although this can be error-prone, as it is done manually.

• If the course author changes the provider, the administrator may have difficulty finding and managing that particular implementation.

• There is no parameter that facilitates the identification of the provider without the administrator adding it manually, as each provider has a particular structure.

Through the LTI setups (This was applied for the eleventh iteration)

• Through a set of unique credentials with which implementations will be grouped. In LTI 1.1, this would include Consumer Key, Secret Key, and LTI URL; while in LTI 1.3, they would be Tool Launch URL, Public Key, or Keyset URL.

• The provider would not be taken into account.

• In the administrator's Learning Tool Center, each implementation will be displayed, and only multiple implementations that share the same LTI setups will be grouped. "LTIs with the same credentials can occur in reruns or if the same component is used in different courses."

In this case, there is the template or the model that will be reused created by the administrator, this list of implementations appears when an LTI has been reused as is without making changes to the Launch URL and the Keys. The admin can duplicate any implementation created by a course author and enable it to be reused, that is, convert it into a template.

The admin can edit the templates he created, he can even delete them, however, he cannot delete the implementations reused by the course authors from the Learning tool center, to delete and edit you must go to the unit where it was implemented.

From the view of the implementation used by the course creator, you can redirect to the unit where the implementation is

UX Writing

https://docs.google.com/spreadsheets/d/1jO0VaA7E--uounfGYvBd6Z6h1DmQAYjxyGwaaAExmTs/edit?pli=1#gid=0

Significant changes were made to text to enhance user comprehension and interaction, including:

• In the development of the matrix for text evaluation, it was determined that some texts needed to be clearer and more precise.

• Descriptions were enhanced with valuable information to ensure understanding for users unfamiliar with the process, thereby reducing technical language.

• Additionally, links were added to certain descriptions, directing users straight to specific content to prevent the need for them to search through the entirety of the documentation

• Placeholders were created to give users an idea of the type of information they should add to the fields.

Throughout the process, the goal was to make it very easy to reuse an LTI, but it was also important to put ourselves in the shoes of the administrator and the need to have general control of the implementations within their organization.

Most of the iterations were focused on the administrator's uses because the process of reusing an LTI was successful in the tests, and more challenges arose from an administrator's point of view.