Scalable Open Response Grading

Overview

Scalable Open Response Grading covers the instructor-facing workflows for reviewing, grading, and managing open response assessments (ORA) at scale. While ORA configuration is an authoring concern, the grading workflow — staff review queues, peer grading oversight, rubric-based scoring, and grade finalization — is an instructor operations concern.

edx-ora2 implements the full ORA2 system including peer grading, self-assessment, and staff grading pathways. For large courses, staff grading interfaces allow instructors to review and score open-ended responses without requiring peer review at scale.

Current State (2026)

• Staff grading: Instructors can grade open responses directly via the staff grading interface in the ORA2 XBlock; bypasses or supplements peer review
• Peer review management: Instructors can monitor peer assessment completion, reassign submissions, and override grades
• Grade override: Staff can override peer-assigned grades for individual learners
• Rubric-based grading: All ORA assessments use instructor-defined rubrics with criterion scores; staff grading uses the same rubric
• Submission management: Instructors can view submission counts, completion status, and flag problematic submissions
• frontend-app-ora-grading: Dedicated MFE for staff grading of ORA submissions at scale

Architecture

• edx-ora2: XBlock-based ORA system; manages submission storage, peer matching, assessment workflows, and grade aggregation
• Staff grading workflow: Staff grading requests handled by edx-ora2 backend; submissions queued for review
• frontend-app-ora-grading: Standalone MFE for batch staff grading; communicates with edx-ora2 APIs
• Grade signals: ORA final grades written back to edx-platform gradebook via grade submission signals
• Submission storage: Learner text and file submissions stored in edx-ora2 database; files in S3

Relevant Repositories

Repository	Role in This Feature	Activity Level	Notes
openedx/edx-ora2	ORA2 XBlock: submission, peer review, staff grading	High	Core ORA system
openedx/frontend-app-ora-grading	Staff grading MFE for batch ORA review	Medium	Instructor grading interface
openedx/openedx-platform	Grade recording, gradebook integration	High	Receives ORA grade signals

Recent Changes

• frontend-app-ora-grading provides improved bulk staff grading UX
• Peer assessment improvements ongoing in edx-ora2

History

Origin

• Year introduced: ~2013–2014 (Open Response Assessments developed to handle essay-type grading at MOOC scale where instructor grading is infeasible)
• Initial implementation: edx-ora2 built as a standalone XBlock app; peer assessment was the primary grading mechanism at scale
• Context: MOOC-scale courses (10,000+ learners) cannot have instructors grade all open responses; peer review and self-assessment were designed to distribute the grading load

Key Milestones

Year	Milestone	Teams / People Involved
~2013–2014	ORA2 (edx-ora2) launched with peer and self-assessment	Unknown
~2015–2016	Staff grading pathway added for smaller courses or overrides	Unknown
~2019–2020	frontend-app-ora-grading MFE introduced for batch staff grading	Unknown
~2022–2023	Enhanced file submission support and improved grading UX	Unknown

People Who Shaped This Area

• Engineering: Unknown — open question for interview
• Product: Unknown — open question for interview
• Design: Unknown — open question for interview

Open Questions

• [ ] Who designed the original ORA2 peer review workflow and what were the key design decisions?
• [ ] How does the staff grading queue work — how are submissions prioritized and assigned?
• [ ] What happens when peer review pools are too small (not enough peers to complete reviews)?
• [ ] How does ORA grade aggregation interact with the gradebook when there are mixed peer + staff grades?
• [ ] What are the known limitations of frontend-app-ora-grading at very high submission volumes?