Open edX Adaptive Tools 

MS/Harvard VPAL 

• Designing Adaptive Learning and Assessment
  • Adaptive = dynamically change in response to student interactions within the MOOC, rather than on the basis of preexisting information such as a learner’s gender, age, or achievement test score.
  • The order of problems in a sequence is determined by a personalized learning progression, using learners’ real-time performance and statistical inferences on sub-topics they have mastered.
  • All problems in the course were manually tagged with one or several learning objectives.
  • Uses TutorGen's adaptive engine, SCALE®  - Student Centered Adaptive Learning Engine
    • Provides knowledge tracing, skill modeling, student modeling, adaptive problem selection, and automated hint generation for multi-step problems.
    • Knowledge components / skills (KCs) are tagged at the right level of granularity. Scale refines the tagging of these KCs after data has been collected from actual student interactions.
    • TutorGen extended SCALE algorithms to consider not only individual learning objectives (KCs), but also problem difficulty and problem selection within modules that group together various concepts and problems.
• The Adaptive Experiment : Implementation
  • VPAL LTI tool
    1. receives learner activity data from edX
    2. passes a sanitized version to SCALE
    3. receives updates from SCALE
    4. provides appropriate next activity to learners
  • LTI tool provides a pass-through frame with an "activity sequence" (sequence of problems) and iframes XBlock URLs.
  • Hiding assessments
    • Relies on XBlock URLs not enforcing content experiment groups.
    • All assessments must be available to the control group.
    • Experiment group sees ONLY the LTI tool.
  • Passing grades and data
    • https://adaptive-edx-v4.vpal.io/ called by https://studio.edx.org/asset-v1:HarvardX+SPU30x+3T2017+type@asset+block@EveryProblemScript.js
    • Data: username, problem's block_usage_key, score, max_score
    • Hack: waits 2secs after problem is submitted and page is reloaded
  • No one noticed: "Invisible implementation is a definite win."
• The Bridge for Adaptivity
  • 2 endpoints on SCALE
    • Transaction: submit student problem attempts with student, activity, and grade data.
    • Activity: get ID representing the next activity recommended by the engine for the student.
• Analyzing Data from an Adaptive MOOC
  
  • Performance (effectiveness)
  • Speed (efficiency)
  • Engagement (engaging)

Adaptive Learning Engines / Algorithms

SCALE by TutorGen

• SCALE: Student Centered Adaptive Learning Engine
  • Unlike a pure machine learning solution:
    • SCALE is able to report to the developers exactly why the system behaves as it does.
    • Allows for human input to maximize improvements through refinement over time.
  • Does not require a priori expert-generated "student models".
  • Generates student models that build and improve as more data is collected.
  • Dynamically selects the students’ next problems to maximize student learning and minimize time needed to master a set of skills.
  • Knowledge Tracing and problem selection mechanisms use knowledge component (KC) modeling.

Hint generation algorithms used by TutorGen

• Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data
  • Primarily targets logic proofs in CS and philosophy.
  • Generates Markov Decision Processes that represent all student approaches to a particular problem, and uses the MDPs directly to automatically generate hints.
    • Reward for goal state (100)
    • Penalties for incorrect states (10)
    • Cost for taking an action (1) (to slightly favor shorter steps)
  • Comparison by both ordered and unordered matches with other students' responses.
  • One semester of data is sufficient to generate a significant amount of hints.
  • Alternatives
    • Constraint-based tutors can only provide condition violation feedback, not goal-oriented feedback.
    • Example-based authoring tools predict frequent correct and incorrect approaches.
    • Bootstrapping Novice Data also requires considerable authoring time.
    • ADVISOR predicts how long a student will take and provides (further) instructions accordingly.
    • Logic-ITA tutor warns students when they were likely to make mistakes.
• Automating the Generation of Production Rules for Intelligent Tutoring Systems

Student Models

• DataShop repository at Pittsburgh Science of Learning Center
  • Resource for educators and researchers to create, modify, and evaluate student models.
  • Data from thousands of students derived from interactions with on-line course materials and intelligent tutoring systems.
  • Categorized in terms of the hypothesized competencies or Knowledge Components (KCs), representing pieces of knowledge, concepts or skills that students need to solve problems.
• Managing the Educational Dataset Lifecycle with DataShop
  • DataShop is focused on becoming the premier repository for educational data.
  • Data logging API
  • Data import via text or XML
  • Custom fields for student logs
• Automated Student Model Improvement
  • CTA typically produces a symbolic representation of a student model, for instance, a rulebased production system of the skills in a domain.
  • An alternative is to use data and statistical inference to create a student model involving continuous parameters over latent variables with links to observed student performance variables.
  • When a specific set of KCs are mapped to a set of instructional tasks (usually steps in problems) they form a KC Model. A KC model is a specific kind of student model.

Sana Labs

• https://sanalabs.github.io/web-api-doc/

Knewton

• https://www.knewton.com/assets-v2/downloads/knewton-intro-2014.pdf
• https://www.knewton.com/wp-content/uploads/knewton-adaptive-learning-whitepaper.pdf

Aleks

• https://www.aleks.com/

Cerego

• https://www.cerego.com/