Masterytrace

MasteryTrace

Install

MasteryTrace ships as two independent, equally first-class packages that implement the same two models (BKT, 2PL IRT) and the same CLI contract.

npm (TypeScript CLI + library):

npm install -g masterytrace-cli

Requires Node.js 18 or later.

pip (Python CLI + library): a full, independent Python port of this repo's TypeScript source lives in python/ -- same two models, same CLI contract, its own 75-test pytest suite, built and verified end to end from a real wheel install.

pip install masterytrace-cli

This installs the same four subcommands (init, record, score, report) as a masterytrace console script, plus an importable masterytrace library, a genuine, independent port of this repo's TypeScript source, not a wrapper around the Node binary. See python/README.md for Python-specific usage.

[!NOTE]

The npm and pip distributions return equivalent data but with different JSON key casing (camelCase from the TypeScript CLI, snake_case from the Python CLI). Account for this if you parse output from both in the same pipeline.

Table of Contents

• Features
• Quickstart
• CLI command reference
• Library API reference
• How BKT and IRT work
• Benchmark
• Comparison
• FAQ
• Contributing
• License

Features

• Two named psychometric models, not a black-box score. Bayesian Knowledge Tracing outputs a posterior mastery probability per learner per skill; 2-parameter logistic IRT outputs a continuous ability estimate (theta) per learner and item difficulty/discrimination per skill. Run either or both with --model bkt|irt|both.
• Two independent, numerically matching distributions. The npm package (masterytrace-cli, TypeScript) and the PyPI package (masterytrace-cli, Python) are a real line-for-line port of each other, not a Python wrapper around the Node binary; this audit ran the same 58-event sample through both and got identical mastery scores.
• Agent-parseable by default. Every command accepts a global --json flag, report also takes --format markdown, and there's a real three-value exit code contract (0 success, 1 usage error, 2 bad event data) instead of a single generic failure code.
• 162 tests, 100% statement/line/function coverage. 87 TypeScript tests plus 75 Python tests, including a synthetic-data IRT recovery check that fits 4,000 responses with known ground-truth parameters and lands within 0.2 of the true theta, item difficulty, and item discrimination.
• No server, no database. State is two JSON files in a .masterytrace/ directory next to where you run the CLI. Scoring 100,000 events takes under a second on a single core.

Quickstart

masterytrace init

masterytrace record events.json masterytrace score masterytrace report

init scaffolds a sample events.json (3 learners, 3 skills, several responses each) and a default masterytrace.config.json in the current directory. Real output from that flow:

$ masterytrace init

Created: events.json, masterytrace.config.json Next: run 'masterytrace record events.json' to load it, then 'masterytrace score'.

$ masterytrace record events.json Stored 58 event(s) to /path/to/.masterytrace/events.json (record replaces any previously stored event log; see --help for details.)

$ masterytrace score Scored 58 event(s) with model(s): both Wrote /path/to/.masterytrace/scores.json

$ masterytrace report learner skill model metric value responses

---

learner-ada fractions bkt posterior_mastery_probability 0.9994 6 learner-ada fractions irt ability_theta 0.7349 6 learner-ada linear-equations bkt posterior_mastery_probability 0.9746 7 learner-brook fractions bkt posterior_mastery_probability 0.0612 6 learner-cyrus reading-comprehension bkt posterior_mastery_probability 0.9947 7 ...

[!WARNING]

masterytrace record always replaces the entire previously stored event log; there is no append mode. If you need to add new responses without losing existing ones, merge them into one file and re-run record with the full, combined log.

report also takes --format markdown or --format json, and every command accepts a global --json flag for machine-readable output on stdout, with a real exit code contract (0 success, 1 general/usage error, 2 bad event data) so a script or agent invoking this CLI can branch on the result without parsing text.

Your own event log is a JSON array of { learnerId, skillId, correct, timestamp } objects, or a CSV with header learner_id,skill_id,correct,timestamp. timestamp must be ISO 8601; correct is a boolean (JSON) or true/false/1/0 (CSV), and any other value in a CSV correct cell is rejected as a validation error rather than silently treated as false. Event log files over 100 MB are rejected up front with a clear error; event logs are small structured records and have no legitimate reason to approach that size.

CLI command reference

Global option: --json forces machine-readable JSON on stdout for any command, overriding --format on report.

Exit codes: 0 success, 1 general or usage error (bad flag, missing file), 2 validation error (the event log itself is malformed).

MCP Server

MasteryTrace ships a Model Context Protocol (MCP) server, so an agent (Claude Desktop, Claude Code, or any other MCP client) can invoke the CLI directly instead of shelling out itself.

Install the Python package with the mcp extra:

pip install "masterytrace-cli[mcp]"

Then point an MCP client at the masterytrace-mcp console script. Claude Desktop config example (claude_desktop_config.json):

{

"mcpServers": { "masterytrace": { "command": "masterytrace-mcp" } } }

The server exposes a single tool, run(args: list[str]) -> dict, which shells out to the installed masterytrace CLI with the given argument list and returns its parsed output -- any subcommand or flag the CLI supports is reachable through it. Example call: run(["score", "--model", "bkt", "--json"]) fits a BKT model against the stored event log and returns the parsed JSON mastery report.

Library API reference

Everything below is exported from masterytrace-cli's package entry point (src/index.ts, re-exporting src/core/* and src/models/*):

import {

// Event schema and validation ResponseEventSchema, parseResponseEvents, EventValidationError, type ResponseEvent,

// Shared model types type ScoringModel, type FittedModel, type MasteryReport, type MasteryLearnerEntry, type MasterySkillEntry,

// Engine: runs one or both models runScoring, type ModelSelector, type EngineConfig, type EngineResult,

// BKT BktModel, BKT_DEFAULT_PARAMS, runForwardRecursion, fitSkillParamsByGridSearch, type BktParams, type BktConfig, type BktFittedModel,

// IRT IrtModel, probabilityCorrect, type IrtItemParams, type IrtLearnerResult, type IrtConfig, type IrtFittedModel,

// Generic JSON/CSV event log adapter genericAdapter, parseCsv, type EventAdapter, } from 'masterytrace-cli';

A minimal library usage example:

import { runScoring, parseResponseEvents } from 'masterytrace-cli';

const events = parseResponseEvents([ { learnerId: 'l1', skillId: 'fractions', correct: true, timestamp: '2026-01-01T00:00:00Z' }, { learnerId: 'l1', skillId: 'fractions', correct: false, timestamp: '2026-01-02T00:00:00Z' }, ]);

const { reports } = runScoring(events, 'both'); // reports[0].model === 'bkt', reports[1].model === 'irt' // each learner's report.learners[i].skills[j].value is the mastery estimate

BktModel and IrtModel both implement the same ScoringModel interface (fit(events) then score(fittedModel)), so the engine, and your own code, can treat them interchangeably.

How BKT and IRT work

MasteryTrace implements two independent psychometric models. They answer different questions and produce different kinds of numbers, so masterytrace score --model both runs them side by side rather than picking one.

Bayesian Knowledge Tracing (BKT)

BKT models one learner's mastery of one skill as a hidden binary state (knows it / does not know it yet) and updates a probability of "knows it" after every response, using four parameters:

• pInit: probability the learner already knows the skill before any evidence.
• pTransit: probability of learning the skill between one attempt and the next.
• pSlip: probability of an incorrect answer despite knowing the skill.
• pGuess: probability of a correct answer despite not knowing the skill.

For each response, the forward recursion first updates the belief given the observed outcome (Bayes' rule), then advances it for possible learning before the next attempt:

after correct:   P(know | obs) = P(know) * (1 - pSlip) / [P(know) * (1 - pSlip) + (1 - P(know)) * pGuess]

after incorrect: P(know | obs) = P(know) * pSlip / [P(know) * pSlip + (1 - P(know)) * (1 - pGuess)] P(know)_next = P(know | obs) + (1 - P(know | obs)) * pTransit

MasteryTrace runs this recursion per learner per skill, in chronological order, and reports the final posterior as that learner's mastery probability for that skill. If you set "bkt": { "fit": true } in masterytrace.config.json, each skill's four parameters are fit from your own data by a coarse grid search (7 x 7 x 5 x 5 candidate combinations) that minimizes squared error between predicted and observed correctness, instead of using the textbook defaults (pInit=0.4, pTransit=0.3, pSlip=0.1, pGuess=0.2).

Item Response Theory (2PL IRT)

IRT models one continuous learner ability (`theta