nbgrader_jupyterquiz.grader package

Grader subpackage — nbgrader preprocessor and quiz authoring utilities.

Submodules

nbgrader_jupyterquiz.grader._review module

Static HTML review rendering for graded quizzes.

The rendered output is scoped, self-contained HTML + inline CSS — suitable for display in any browser without a running Jupyter server or kernel. Emitted by display_review() from within the auto-generated hidden-tests block; nbgrader generate_feedback propagates it into the per-student feedback HTML.

nbgrader_jupyterquiz.grader._review.fmt_pts(value: float) → str[source]

Format a point value for display, collapsing binary-float noise.

Rounds to three decimal places and strips trailing zeros so that accumulation artefacts such as 0.3 + 0.3 + 0.4 == 0.9999999999999999 don’t bleed into the visible review text. Three decimals preserve common fractional weights exactly — halves (0.5), quarters (0.25), eighths (0.125), tenths (0.1) — which a two-decimal rounding would mangle (round(0.125, 2) == 0.12 under banker’s rounding).

Parameters

valuefloat: Point value to format.

Returns

str: Canonical short textual form, e.g. "1", "0.5", "0.125".

nbgrader_jupyterquiz.grader._review.render_review_html(result: QuizResult) → str[source]

Render a full quiz review as a scoped HTML string.

Parameters

resultQuizResult: Grading outcome to render.

Returns

str: Complete <style> + <div> block ready for IPython.display.HTML.

nbgrader_jupyterquiz.grader._scoring module

Per-question-type grading helpers for autograde.

Each grade_* function takes a parsed question dict plus the student’s recorded response payload and returns True iff the response satisfies that question’s correctness criteria. Grading is all-or-nothing per question — partial credit at the quiz level falls out of summing per-question outcomes in QuizResult.

nbgrader_jupyterquiz.grader._scoring.expected_answer(question: dict[str, Any]) → Any[source]

Derive a human-readable representation of a question’s expected answer.

Parameters

questiondict: Question dict as produced by parse.

Returns

list or None: A list of expected answer texts (choice/string) or values/ranges (numeric). None for unsupported question types.

nbgrader_jupyterquiz.grader._scoring.grade_many_choice(question: dict[str, Any], recorded: dict[str, Any]) → bool[source]

Grade a many-choice question (set equality).

Parameters

questiondict: Question dict with its answer list.
recordeddict: Student’s recorded response with a selected list.

Returns

bool: True iff the set of selected answers equals the set of correct answers.

nbgrader_jupyterquiz.grader._scoring.grade_multiple_choice(question: dict[str, Any], recorded: dict[str, Any]) → bool[source]

Grade a single-choice question.

Parameters

questiondict: Question dict with its answer list.
recordeddict: Student’s recorded response with a selected string.

Returns

bool: True iff selected matches a correct: true answer.

nbgrader_jupyterquiz.grader._scoring.grade_numeric(question: dict[str, Any], recorded: dict[str, Any]) → bool[source]

Grade a numeric question (value or range, optional precision).

Parameters

questiondict: Question dict with its answer list (values and/or ranges).
recordeddict: Student’s recorded response with a parsed numeric field.

Returns

bool: True iff parsed matches a correct value (at the configured precision) or falls inside a correct range.

nbgrader_jupyterquiz.grader._scoring.grade_string(question: dict[str, Any], recorded: dict[str, Any]) → bool[source]

Grade a string question (exact or fuzzy match).

Parameters

questiondict: Question dict with its answer list; each answer may set match_case and fuzzy_threshold.
recordeddict: Student’s recorded response with a value string.

Returns

bool: True iff value matches any correct answer (honouring match_case and fuzzy_threshold using Levenshtein similarity).

nbgrader_jupyterquiz.grader.autograde module

Autograder helper for graded quizzes (v0.4.0+).

Reads the responses.json sidecar file written by the JS recorder and grades the recorded responses against the quiz definition. Intended to be called from the auto-generated hidden-tests block of a graded-quiz cell — see CreateQuiz.

Usage (inside an autograder cell):

from nbgrader_jupyterquiz import grade_quiz

result = grade_quiz("quiz-1-autograded", questions=[...])
result.display_review()
_result.score  # bare expression → nbgrader partial credit

The sidecar path is resolved relative to the current working directory, which nbgrader autograde sets to the assignment directory. Per-question grading is all-or-nothing; partial credit at the quiz level falls out of summing QuestionResult.earned across the quiz’s questions.

exception nbgrader_jupyterquiz.grader.autograde.GradeQuizError[source]

Bases: Exception

Raised when the helper cannot locate or interpret required inputs.

class nbgrader_jupyterquiz.grader.autograde.QuestionResult(qnum: int, question: dict[str, Any], recorded: Any, correct: bool)[source]

Bases: object

Outcome of grading a single question.

correct: bool

property earned: float

Compute the points earned on this question (all-or-nothing per question).

Returns

float: self.points if correct, else 0.

property expected: Any

Return a human-readable representation of the expected answer.

Returns

Any: List of expected answer texts or numeric values/ranges.

property points: float

Return the maximum points for this question (defaults to 1).

Returns

float: Positive point weight. Integer values (from {N}) come through as int; fractional values ({0.5}) as float.

qnum: int

question: dict[str, Any]

property question_type: str

Return the canonical type string of the underlying question.

Returns

str: "multiple_choice", "many_choice", "numeric", "string", or "unknown".

recorded: Any

class nbgrader_jupyterquiz.grader.autograde.QuizResult(grade_id: str, details: list[QuestionResult])[source]

Bases: object

Outcome of grading every question in a quiz region.

details: list[QuestionResult]

display_review() → None[source]

Emit an HTML review of the quiz into the current cell output.

Intended to be called from the auto-generated hidden-tests block of a graded-quiz cell. When nbgrader generate_feedback converts the autograded notebook to HTML, the review appears inline with the score so students can see which answers were correct, which they picked, and which they missed.

The output is pure static HTML with scoped inline CSS — it renders correctly in any browser without a running Jupyter server or kernel.

grade_id: str

property max_score: float

Compute the sum of per-question point values.

Returns

float: Total points this quiz can yield.

property passed: bool

Return True when every question in the quiz was answered correctly.

Returns

bool: True iff score == max_score.

property report: str

Return a multi-line textual summary of the grade.

Returns

str: Human-readable breakdown of per-question outcomes.

property score: float

Compute the sum of per-question points earned.

Returns

float: Total points the student earned on this quiz.

nbgrader_jupyterquiz.grader.autograde.grade_quiz(grade_id: str, *, questions: list[dict[str, Any]] | None = None, notebook_path: str | Path | None = None) → QuizResult[source]

Grade the quiz identified by grade_id against recorded responses.

Parameters

grade_idstr: The sidecar key under which student responses are recorded. The auto-generated test cells use "<task_grade_id>-autograded".
questionslist of dict, optional: Answer key — the list of question dicts (as produced by the parser) for this quiz. When provided, the notebook is not read. This is the path taken by auto-generated test cells (see CreateQuiz), which embed the answer key directly in their ### BEGIN HIDDEN TESTS block.
notebook_pathstr or Path, optional: Path to the notebook containing the quiz task cell, for the fallback case where questions is None. Defaults to the only .ipynb file in the current working directory.

Returns

QuizResult: Grading result with per-question details. If the sidecar is missing or the quiz has no recorded responses, every question is reported as incorrect (score 0/max).

nbgrader_jupyterquiz.grader.encode module

Encode quiz questions for embedding in notebooks.

nbgrader_jupyterquiz.grader.encode.to_base64(payload: str) → str[source]

Base64-encode a string payload.

Parameters

payloadstr: JSON string to encode.

Returns

str: Base64-encoded string.

nbgrader_jupyterquiz.grader.parse module

Parse quiz question source from notebook cell markdown.

exception nbgrader_jupyterquiz.grader.parse.ParseError[source]

Bases: Exception

Raised when quiz source cannot be parsed.

class nbgrader_jupyterquiz.grader.parse.Quiz(options: dict[str, ~typing.Any], questions: list[dict[str, ~typing.Any]], warnings: list[str] = <factory>)[source]

Bases: object

A parsed quiz with options, a list of question dicts, and parse-time warnings.

warnings collects non-fatal issues the parser spotted (e.g. an MC question with 0 or 1 correct answers). Fatal issues raise ParseError instead. Callers such as the CreateQuiz preprocessor surface these through nbgrader’s UI logger.

options: dict[str, Any]

questions: list[dict[str, Any]]

warnings: list[str]

nbgrader_jupyterquiz.grader.parse.find_quiz_regions(source: str, begin_quiz_delimiter: str = '#### Quiz', end_quiz_delimiter: str = '#### End Quiz') → tuple[list[tuple[str, list[str]]], list[str]][source]

Extract regions within quiz delimiters.

Parameters

sourcestr: Full source text of the markdown cell.
begin_quiz_delimiterstr, optional: Marker that opens a quiz region.
end_quiz_delimiterstr, optional: Marker that closes a quiz region.

Returns

quizzeslist[tuple[str, list[str]]]: Each entry is (options_header, quiz_lines).
remaining_lineslist[str]: Lines that fall outside any quiz region.

nbgrader_jupyterquiz.grader.parse.line_to_mc_answer(line: str) → dict[str, Any][source]

Parse a multiple/many-choice answer line.

Parameters

linestr: Answer line starting with + (correct) or - (incorrect).

Returns

dict: Answer dict with correct, answer, and optional fields.

nbgrader_jupyterquiz.grader.parse.line_to_numeric_answer(line: str) → dict[str, Any][source]

Parse a numeric answer line.

Parameters

linestr: Answer line starting with + (correct) or - (incorrect).

Returns

dict: Answer dict with correct, type, and value/range/feedback.

nbgrader_jupyterquiz.grader.parse.line_to_question(line: str) → dict[str, Any][source]

Parse a question line into a partial question dict (without answers).

Parameters

linestr: Question line starting with *.

Returns

dict: Partial question dict with type, question, and optional fields.

nbgrader_jupyterquiz.grader.parse.parse_cell(source: str, begin_quiz_delimiter: str = '#### Quiz', end_quiz_delimiter: str = '#### End Quiz') → tuple[list[Quiz], list[str]][source]

Parse quiz regions from a notebook cell source string.

Parameters

sourcestr: Full source text of the markdown cell.
begin_quiz_delimiterstr, optional: Marker that opens a quiz region.
end_quiz_delimiterstr, optional: Marker that closes a quiz region.

Returns

quizzeslist[Quiz]: Parsed Quiz objects. Non-fatal parse-time warnings are attached to each quiz’s warnings field.
cell_contentslist[str]: Remaining cell lines with quiz regions removed.

nbgrader_jupyterquiz.grader.parse.parse_line(line: str, **components: tuple[str, str, Any]) → dict[str, Any][source]

Parse delimited components from a line and typecast them.

Parameters

linestr: Text to parse.
**componentstuple[str, str, Any]: Each keyword is a component name mapped to a (left_delim, right_delim, typecast) triple.

Returns

dict: Parsed components.

Raises

ParseError: If a duplicate component is found, the line ends with an unparsable segment, or a delimited field is not closed.

nbgrader_jupyterquiz.grader.parse.parse_question(lines: list[str]) → dict[str, Any][source]

Parse a question block into a question dict.

Parameters

lineslist[str]: First line is the question line; remaining lines are answer lines.

Returns

dict: Question dict matching the jupyterquiz schema.

nbgrader_jupyterquiz.grader.parse.parse_quiz_options(header: str) → dict[str, Any][source]

Parse quiz options from the header line following the begin delimiter.

Parameters

headerstr: Text on the same line as the begin delimiter, after the delimiter itself. Expected format: space-separated key=value pairs. Boolean values are true or false (case-insensitive). filename takes a string value. Unrecognised keys are ignored.

Returns

dict

Quiz options dict with keys encoded, inline, hidden, filename, hide_correctness, graded. Omitted keys retain their defaults.

hide_correctness=true propagates hide: true to every MC / many-choice answer so the display hides correctness feedback and shows a neutral Selected / Deselected state instead. Default None — the preprocessor treats None as “off unless the host cell is graded” and True/False as explicit opt-in/opt-out.
graded=false opts a single quiz out of auto-grading inside a task cell — the generated cell is a plain display_quiz(...) code cell with no nbgrader metadata, no hidden tests, and correctness feedback visible. Default None — the preprocessor treats None as “graded iff the host task cell has a grade_id and auto_generate_tests is on”.

nbgrader_jupyterquiz.grader.parse.redact_answer_key(questions: list[dict[str, Any]]) → list[dict[str, Any]][source]

Return a deep copy of questions with answer-key fields stripped.

The release notebook embeds question JSON into a hidden span the student’s browser loads. Without redaction, the student can read the answer key out of the DOM. Stripping the matching fields makes hide_correctness mode actually withhold the key, not just the visual feedback.

The redaction is keyed off question type:

multiple_choice / many_choice: drop correct from each answer. Keep answer, code, feedback, hide.
numeric: drop value, range, correct from each answer. Keep feedback and type=default entries so fall-through “Incorrect, try again” feedback still works.
string: replace answers with an empty list. String questions are server-graded; the JS path only runs in self-check mode (no hide-correctness), so an empty list is sufficient when this function is called.

Per-answer feedback strings are intentionally preserved — they are pedagogically valuable, and instructors who want them hidden can omit them from the question source.

Parameters

questionslist[dict]: The full parsed-question list (typically Quiz.questions). Not mutated.

Returns

list[dict]: Deep copy of questions with answer-key fields removed. Safe to serialise into the release notebook’s display JSON.

nbgrader_jupyterquiz.grader.parse.split_questions(quiz_source: list[str]) → list[list[str]][source]

Split lines of a quiz region into individual question blocks.

Each returned question is a list of logical lines — one for the question itself, followed by one per answer. A logical line may span multiple physical source lines when its content extends across them, joined with \n. Continuation is driven by markdown-list indentation: a physical line is a continuation of the active logical line iff its first-non-whitespace column is strictly greater than the opener’s column (0 for questions, 2 for answers). Code blocks delimited by triple-backticks are an exception — physical lines inside an open code block are appended regardless of indentation, matching markdown’s fenced-code semantics.

Parameters

quiz_sourcelist[str]: Physical lines within a quiz delimiter region.

Returns

list[list[str]]: Each inner list contains the question logical line followed by its answer logical lines. Each logical line is a single str (possibly containing \n).

Raises

ParseError: If a logical line ends with an unclosed delimited field — i.e. indentation drops back to the opener’s level (or beyond) while the field is still expecting its closer.

nbgrader_jupyterquiz.grader.preprocessor module

Nbgrader preprocessor that converts markdown quiz regions into interactive quizzes.

class nbgrader_jupyterquiz.grader.preprocessor.CreateQuiz(*args: t.Any, **kwargs: t.Any)[source]

Bases: NbGraderPreprocessor

Convert markdown quiz regions to interactive jupyterquiz cells.

Register in nbgrader_config.py at the front of the GenerateAssignment preprocessor list so that auto-generated autograder cells are picked up by SaveCells (into the gradebook) and ClearHiddenTests (to strip the grading body from the release):

c.GenerateAssignment.preprocessors.insert(0, "nbgrader_jupyterquiz.CreateQuiz")

Appending instead of inserting (.append(...)) will cause autograde to fail because our cells wouldn’t be in the gradebook and their checksums wouldn’t match the release.

auto_generate_tests

Whether to auto-generate autograder-cell content when the host task cell carries an nbgrader grade_id.

When True (the default), each generated display_quiz code cell also becomes the nbgrader-tracked graded cell: its source carries a ### BEGIN HIDDEN TESTS block that embeds the answer key and calls grade_quiz() — the block is stripped from the release by ClearHiddenTests and restored at autograde time by OverwriteCells. A bare _result.score expression at the end of the cell feeds nbgrader’s partial-credit scoring (utils.determine_grade). The task cell’s own points field is left untouched so it can still be manually graded alongside the auto-graded quiz score.

Disable to opt out of auto-grading and keep display_quiz as a plain (non-graded) code cell; in that mode instructors author their own autograded test cells.

begin_quiz_delimiter: The delimiter marking the beginning of quiz source.

end_quiz_delimiter: The delimiter marking the end of quiz source.

enforce_metadata: Whether to raise an error if cells containing quiz regions are not marked as ‘Manually Graded Task’ cells. Only disable this if you are using nbgrader generate_assignment without the full grading pipeline.

name = ''

preprocess(nb: NotebookNode, resources: ResourcesDict) → tuple[NotebookNode, ResourcesDict][source]

Process all cells in the notebook, expanding quiz regions.

Parameters

nbNotebookNode: Source notebook.
resourcesResourcesDict: Nbgrader resources dict (provides unique_key).

Returns

NotebookNode, ResourcesDict: Modified notebook with quiz cells appended after each quiz region.

preprocess_cell(cell: NotebookNode, resources: ResourcesDict, _index: int) → tuple[NotebookNode, ResourcesDict][source]

No-op — all processing happens in preprocess().

Parameters

cellNotebookNode: Current cell.
resourcesResourcesDict: Nbgrader resources dict.
_indexint: Cell index (unused).

Returns

NotebookNode, ResourcesDict: Cell unchanged.

quiz_cell_counter = count(0)

nbgrader_jupyterquiz.grader.validate module

Validate quiz questions against JSON Schema.

class nbgrader_jupyterquiz.grader.validate.Schema(*values)[source]

Bases: Enum

JSON Schema definitions for jupyterquiz question types.

MC = {'$id': 'https://github.com/jmshea/jupyterquiz/mc_schema.json', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'description': 'Schema for Multiple or Many Choice Questions in JupyterQuiz', 'properties': {'answers': {'items': {'properties': {'answer': {'type': 'string'}, 'answer_cols': {'type': 'number'}, 'correct': {'type': 'boolean'}, 'feedback': {'type': 'string'}, 'hide': {'type': 'boolean'}}, 'required': ['answer', 'correct'], 'type': 'object'}, 'type': 'array'}, 'code': {'type': 'string'}, 'points': {'exclusiveMinimum': 0, 'type': 'number'}, 'question': {'type': 'string'}, 'type': {'pattern': 'multiple_choice|many_choice', 'type': 'string'}}, 'required': ['type', 'question', 'answers'], 'title': 'JupyterQuiz Multiple or Many Choice Quiz', 'type': 'object'}

NUM = {'$id': 'https://github.com/jmshea/jupyterquiz/num_schema.json', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'description': 'Schema for Numeric Questions in JupyterQuiz', 'properties': {'answers': {'items': {'anyOf': [{'properties': {'correct': {'type': 'boolean'}, 'feedback': {'type': 'string'}, 'value': {'type': 'number'}}, 'required': ['value', 'correct'], 'type': 'object'}, {'properties': {'correct': {'type': 'boolean'}, 'feedback': {'type': 'string'}, 'range': {'maxItems': 2, 'minItems': 2, 'type': 'array'}}, 'required': ['range', 'correct'], 'type': 'object'}, {'properties': {'feedback': {'type': 'string'}, 'type': {'pattern': 'default', 'type': 'string'}}, 'required': ['type', 'feedback'], 'type': 'object'}]}, 'type': 'array'}, 'hide': {'type': 'boolean'}, 'points': {'exclusiveMinimum': 0, 'type': 'number'}, 'precision': {'type': 'integer'}, 'question': {'type': 'string'}, 'type': {'pattern': 'numeric', 'type': 'string'}}, 'title': 'JupyterQuiz Numeric Question', 'type': 'object'}

STR = {'$id': 'https://github.com/jmshea/jupyterquiz/str_schema.json', '$schema': 'https://json-schema.org/draft/2020-12/schema', 'description': 'Schema for String (free-text) Questions in JupyterQuiz', 'properties': {'answers': {'items': {'anyOf': [{'properties': {'answer': {'type': 'string'}, 'correct': {'type': 'boolean'}, 'feedback': {'type': 'string'}, 'fuzzy_threshold': {'maximum': 1, 'minimum': 0, 'type': 'number'}, 'match_case': {'type': 'boolean'}}, 'required': ['answer', 'correct'], 'type': 'object'}, {'properties': {'feedback': {'type': 'string'}, 'type': {'pattern': 'default', 'type': 'string'}}, 'required': ['type', 'feedback'], 'type': 'object'}]}, 'type': 'array'}, 'hide': {'type': 'boolean'}, 'points': {'exclusiveMinimum': 0, 'type': 'number'}, 'question': {'type': 'string'}, 'type': {'pattern': 'string', 'type': 'string'}}, 'required': ['type', 'question', 'answers'], 'title': 'JupyterQuiz String Question', 'type': 'object'}

nbgrader_jupyterquiz.grader.validate.validate_question(question: dict[str, Any]) → None[source]

Validate a question dict against the appropriate JSON Schema.

Parameters

questiondict: Parsed question dictionary with at least a 'type' key.

Raises

jsonschema.exceptions.ValidationError: If the question does not conform to its schema.