| Table of Contents |
|---|
...
- Designing Adaptive Learning and Assessment
- Adaptive = dynamically change in response to student interactions within the MOOC, rather than on the basis of preexisting information such as a learner’s gender, age, or achievement test score.
- The order of problems in a sequence is determined by a personalized learning progression, using learners’ real-time performance and statistical inferences on sub-topics they have mastered.
- All problems in the course were manually tagged with one or several learning objectives.
- Uses TutorGen's adaptive engine, SCALE® - Student Centered Adaptive Learning Engine
- Provides knowledge tracing, skill modeling, student modeling, adaptive problem selection, and automated hint generation for multi-step problems.
- Knowledge components / skills (KCs) are tagged at the right level of granularity. Scale refines the tagging of these KCs after data has been collected from actual student interactions.
- TutorGen extended SCALE algorithms to consider not only individual learning objectives (KCs), but also problem difficulty and problem selection within modules that group together various concepts and problems.
- The Adaptive Experiment : Implementation
- VPAL LTI tool
- receives learner activity data from edX
- passes a sanitized version to SCALE
- receives updates from SCALE
- provides appropriate next activity to learners
- LTI tool provides a pass-through frame with an "activity sequence" (sequence of problems) and iframes XBlock URLs.
- Hiding assessments
- Relies on XBlock URLs not enforcing content experiment groups.
- All assessments must be available to the control group.
- Experiment group sees ONLY the LTI tool.
- Passing grades and data
- https://adaptive-edx-v4.vpal.io/ called by https://studio.edx.org/asset-v1:HarvardX+SPU30x+3T2017+type@asset+block@EveryProblemScript.js
- Data: username, problem's block_usage_key, score, max_score
- Hack: waits 2secs after problem is submitted and page is reloaded
- No one noticed: "Invisible implementation is a definite win."
- VPAL LTI tool
- The Bridge for Adaptivity
- 2 endpoints on SCALE
- Transaction: submit student problem attempts with student, activity, and grade data.
- Activity: get ID representing the next activity recommended by the engine for the student.
- 2 endpoints on SCALE
- Analyzing Data from an Adaptive MOOC
- Performance (effectiveness)
- Speed (efficiency)
- Engagement (engaging)
...
- SCALE: Student Centered Adaptive Learning Engine
- Unlike a pure machine learning solution:
- SCALE is able to report to the developers exactly why the system behaves as it does.
- Allows for human input to maximize improvements through refinement over time.
- Does not require a priori expert-generated "student models".
- Generates student models that build and improve as more data is collected.
- Dynamically selects the students’ next problems to maximize student learning and minimize time needed to master a set of skills.
- Knowledge Tracing and problem selection mechanisms use knowledge component (KC) modeling.
- Unlike a pure machine learning solution:
Hint generation algorithms used by TutorGen
- Toward Automatic Hint Generation for Logic Proof Tutoring Using Historical Student Data
- Primarily targets logic proofs in CS and philosophy.
- Generates Markov Decision Processes that represent all student approaches to a particular problem, and uses the MDPs directly to automatically generate hints.
- Reward for goal state (100)
- Penalties for incorrect states (10)
- Cost for taking an action (1) (to slightly favor shorter steps)
- Comparison by both ordered and unordered matches with other students' responses.
- One semester of data is sufficient to generate a significant amount of hints.
- Alternatives
- Constraint-based tutors can only provide condition violation feedback, not goal-oriented feedback.
- Example-based authoring tools predict frequent correct and incorrect approaches.
- Bootstrapping Novice Data also requires considerable authoring time.
- ADVISOR predicts how long a student will take and provides (further) instructions accordingly.
- Logic-ITA tutor warns students when they were likely to make mistakes.
- Automating the Generation of Production Rules for Intelligent Tutoring Systems
...
- https://www.knewton.com/assets-v2/downloads/knewton-intro-2014.pdf
- https://www.knewton.com/wp-content/uploads/knewton-adaptive-learning-whitepaper.pdf