Two self-contained, offline HTML tools that train subject matter experts to give calibrated probability estimates before they contribute inputs to quantitative risk analysis, for example FAIR-style cyber risk quantification (CRQ), where overconfident inputs silently corrupt every Monte Carlo result downstream.
Inspired by the calibrated estimation training method popularised by Douglas Hubbard in How to Measure Anything: repeated testing with immediate feedback until a person's stated confidence matches their actual accuracy.
| File | What it is |
|---|---|
calibration-trainer.html |
The practice tool. 90% confidence-interval rounds and true/false confidence rounds, immediate feedback, hit-rate and Brier scoring, an SVG calibration curve, progress tracking, multiple named profiles, a workshop mode for projection, and JSON/CSV export. |
calibration-course.html |
A self-paced, nine-module course that drives the trainer. Teaches why calibration matters and the techniques that build it, verifies your practice rounds automatically, tracks a baseline-to-certified improvement arc, and issues a printable completion certificate. |
calibration-question-bank.md |
The full question bank and answer key, generated from the trainer, for independent verification. |
- Download
calibration-trainer.htmlandcalibration-course.htmlinto the same folder. - Open
calibration-course.htmlin Chrome or Edge. - Follow Module 0, which walks you through opening the trainer and linking a profile. Always open the trainer using the buttons inside the course, so the trainer can report your results back to the course automatically.
To practise without the course, just open calibration-trainer.html directly.
- Single-file tools. Each tool is one
.htmlfile: all CSS and JavaScript inline, no build step, no frameworks. - Fully offline. No CDN links, no external fonts, no network calls, no telemetry. Works opened straight from disk on a locked-down corporate laptop.
- No data leaves the browser. Everything is stored in your browser's localStorage. JSON export/import is the durable path for backup or moving between machines.
- Vanilla JavaScript, evergreen Chrome and Edge as targets.
The course certifies a learner when their trainer history shows sustained (not one-off) performance:
- two consecutive interval rounds capturing at least 8 of 10 true values, and
- two consecutive binary rounds with an average Brier score of 0.15 or better.
The certificate is a self-assessed training record, not an accredited qualification.
Profile, session, and course progress live in browser localStorage: per-browser, per-machine, and clearable by IT policy. Both tools provide JSON export and import; the trainer additionally exports session results as CSV. Nothing is transmitted anywhere.
The bank contains 240 general-knowledge questions (160 interval, 80 binary) chosen for stable, checkable facts, with reference years stated where facts are time-dependent. A wrong answer key mis-trains calibration, so the full answer key is published in calibration-question-bank.md for independent verification; corrections via issues or pull requests are welcome.
The training method implemented here was popularised by Douglas Hubbard (How to Measure Anything; with Richard Seiersen, How to Measure Anything in Cybersecurity Risk), building on the judgement-under-uncertainty research of Kahneman, Tversky, Lichtenstein, Fischhoff and others, and the forecasting-accuracy work of Glenn Brier and Philip Tetlock. This project is independent and is not affiliated with or endorsed by any of these authors. References to FAIR are descriptive; this project is not affiliated with or endorsed by the FAIR Institute or The Open Group.