Open edX Adaptive Tools 

MS/Harvard VPAL 

• Designing Adaptive Learning and Assessment
  • Adaptive = dynamically change in response to student interactions within the MOOC, rather than on the basis of preexisting information such as a learner’s gender, age, or achievement test score.
  • The order of problems in a sequence is determined by a personalized learning progression, using learners’ real-time performance and statistical inferences on sub-topics they have mastered.
  • All problems in the course were manually tagged with one or several learning objectives.
  • Uses TutorGen's adaptive engine, SCALE®  - Student Centered Adaptive Learning Engine
    • Provides knowledge tracing, skill modeling, student modeling, adaptive problem selection, and automated hint generation for multi-step problems.
    • Knowledge components / skills (KCs) are tagged at the right level of granularity. Scale refines the tagging of these KCs after data has been collected from actual student interactions.
    • TutorGen extended SCALE algorithms to consider not only individual learning objectives (KCs), but also problem difficulty and problem selection within modules that group together various concepts and problems.
• The Adaptive Experiment : Implementation
  • VPAL LTI tool
    1. receives learner activity data from edX
    2. passes a sanitized version to SCALE
    3. receives updates from SCALE
    4. provides appropriate next activity to learners
  • LTI tool provides a pass-through frame with an "activity sequence" (sequence of problems) and iframes XBlock URLs.
  • Hiding assessments
    • Relies on XBlock URLs not enforcing content experiment groups.
    • All assessments must be available to the control group.
    • Experiment group sees ONLY the LTI tool.
  • Passing grades and data
    • https://adaptive-edx-v4.vpal.io/ called by https://studio.edx.org/asset-v1:HarvardX+SPU30x+3T2017+type@asset+block@EveryProblemScript.js
    • Data: username, problem's block_usage_key, score, max_score
    • Hack: waits 2secs after problem is submitted and page is reloaded
  • No one noticed: "Invisible implementation is a definite win."
• The Bridge for Adaptivity
  • 2 endpoints on SCALE
    • Transaction: submit student problem attempts with student, activity, and grade data.
    • Activity: get ID representing the next activity recommended by the engine for the student.
• Analyzing Data from an Adaptive MOOC
  
  • Performance (effectiveness)
  • Speed (efficiency)
  • Engagement (engaging)