Audit
What TheoremPath asserts is correct
Every educational claim on the site — page math, question solutions, hidden choice explanations, common-mistake notes, Lean theorems — has a stable id, a content hash, and a verification status. The full catalog is published as a machine-readable manifest so external auditors and agent runners do not need to scrape rendered HTML.
Current snapshot
Manifest generated
Question-bank claims (stems, solutions, hints, choice text and explanations, common mistakes, correct-answer labels)
13,748 total
1192
verified (9%)
12,556
unverified
0
stale
0
mismatch
Lean theorems (mathlib-verified or with formal Lean proofs)
33 total
33
verified (100%)
0
unverified
0
stale
0
mismatch
Author-tagged numerical claims on topic pages (explicit claimId with a verifiable formula)
4 total
4
verified (100%)
0
unverified
0
stale
0
mismatch
The author-tagged set starts small. A page claim only enters the audit graph when an author wraps a numerical statement with a stable id and a verifiable formula. Most page math today is verified through the Lean theorem track or appears as a derived value inside a question; both of those rows carry the bulk of the audit signal. The author-tagged row grows as we annotate more pages.
Fetch the manifest
/audit/manifest.json — UTF-8 JSON, ~4.5 MB. Compressible to ~700 KB gzip. Stable schema (versioned at the top). Updated on every commit that touches the audit graph.
What it contains: ids, content hashes, status, method, reviewer, sourceRefs, lastVerifiedAt. What it does not contain: the underlying content text. Auditors who already have the rendered pages can verify by hashing their scrape; the manifest is the verification surface, not a content dump.
Schema and methods
Statuses: verified · unverified · needs-review · disputed · stale
Methods: computation · human-review · source-check · lean · test-case · llm-assisted-review
computation verification reuses a small whitelisted parser (no eval, no shell-out): arithmetic (+, −, *, /, ^), square root, log / ln / log2 / log10, exp, abs, min, max, identifiers drawn from a per-claim inputs map, and the constants e and pi. Function calls require parentheses. Each computation claim ships with the formula text, the input map, the expected value, and a tolerance; the verifier re-evaluates and compares.
How to file an audit report
- Fetch the manifest. Find the claim id matching the surface you are reporting on.
- If the claim is
verifiedand you disagree, supply a verification artifact stronger than the recorded one (a Lean wrapper, a peer-reviewed source, or a counterexample with derivation). A bare prose disagreement against a verified claim is rejected by policy. - If the claim is
unverifiedorneeds-review, a normal report is welcome. Include the claim id, the proposed correction, and any sources. - Email [email protected] with subject prefix
[audit]. If you also publish the report at a public URL, include that.
What the manifest does not cover
- Subjective editorial choices (page tier, module placement, difficulty rating). Disagreement here is a normal feature request, not an audit issue.
- Comparative judgements that depend on context the page authors weighted differently (best textbook, canonical proof style).
- Sister-section content (LinguisticsPath, ComputationPath, PhilosophyPath) is published with its own per-section audit manifest. Each follows the same schema described here; visit the relevant sister section for its current snapshot.
Reports go to [email protected]. Include the claim id, the manifest snapshot date, and a minimum-viable counterexample or source citation. Reports that include both a content hash and a reproducible comparison get triaged first.