Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/navigator_check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Navigator Check

# Thin caller for PyAutoBuild's reusable navigator-catalogue check. The check
# logic (path/banner lint + catalogue staleness) and its entrypoints live in
# PyAutoBuild/autobuild; this repo only declares which generator project to run.
# See PyAutoLabs/PyAutoBuild/.github/workflows/navigator_check.yml.

on: [push, pull_request]

permissions:
contents: read

jobs:
navigator:
uses: PyAutoLabs/PyAutoBuild/.github/workflows/navigator_check.yml@main
with:
project: howtofit
10 changes: 10 additions & 0 deletions .navigator_check_ignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Paths exempt from the navigator path/banner checks (see
# PyAutoBuild/autobuild/check_navigator.py). One path or glob per line.
#
# CLAUDE.md's "Relationship to autofit_workspace" section mentions sibling
# autofit_workspace scripts in prose (as the next destination after a concept).
# Those paths live in the separate autofit_workspace repo, not here, so the
# path-existence check would flag them as missing. Exempt them.
scripts/cookbooks/
scripts/searches/
scripts/overview/overview_2_science_workflow.py
47 changes: 47 additions & 0 deletions llms-full.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
AUTO-GENERATED by PyAutoBuild — do not edit by hand; regenerate with generate.py.

# HowToFit Lectures — Full Catalogue

> Complete, generated listing of every script (and its matching notebook) in this
> workspace, grouped by top-level `scripts/` folder. This is the expanded companion to
> the curated `llms.txt` routing layer. Each entry links the script's title to its path
> and gives the first line of its docstring; `Contents:` lists the sections within.

## chapter_1_introduction

- [HowToFit Lectures](scripts/chapter_1_introduction/start_here.py): Welcome to the HowToFit Jupyter Notebook lectures!
- [Tutorial 1: Models](scripts/chapter_1_introduction/tutorial_1_models.py): At the heart of model-fitting is the model: a set of equations, numerical processes, and assumptions describing a physical system of interest. The goal of model-fitting is to better understand this physical system and develop predictive models that describe it more accurately.
- Contents: Paths, Model Parameterization, Model Composition, Model Creation, Model Mapping, Complex Models, Tuple Parameters, Extensibility, Wrap Up
- [Tutorial 2: Fitting Data](scripts/chapter_1_introduction/tutorial_2_fitting_data.py): We've learned that a model consists of equations, numerical processes, and assumptions that describe a physical system. Using **PyAutoFit**, we defined simple 1D models like the Gaussian, composed them into models using `Model` and `Collection` objects, and generated model data by varying their parameters.
- Contents: Data, Model Data, Residuals, Normalized Residuals, Chi Squared, Noise Normalization, Likelihood, Recap, Fitting Models, Guess 1, Guess 2, Guess 3, Extensibility, Wrap Up
- [Tutorial 3: Non Linear Search](scripts/chapter_1_introduction/tutorial_3_non_linear_search.py): In the previous tutorials, we laid the groundwork by defining a model and manually fitting it to data using fitting functions. We quantified the goodness of fit using the log likelihood and demonstrated that for models with only a few free parameters, we could achieve satisfactory fits by manually guessing parameter values. However, as the complexity of our models increased, this approach quickly became impractical.
- Contents: Parameter Space, Non-Linear Search, Search Types, Deeper Background, Data, Model, Priors, Analysis, Searches, Maximum Likelihood Estimation (MLE), Markov Chain Monte Carlo (MCMC), Nested Sampling, What is The Best Search To Use?, Wrap Up
- [Tutorial 4: Why Modeling Is Hard](scripts/chapter_1_introduction/tutorial_4_why_modeling_is_hard.py): We have successfully fitted a simple 1D Gaussian profile to a dataset using a non-linear search. While achieving an accurate model fit has been straightforward, the reality is that model fitting is a challenging problem where many things can go wrong.
- Contents: Data, Model, Analysis, Alternative Syntax, Collection, Search, Model Fit, Result, Why Modeling is Hard, Prior Tuning, Reducing Complexity, Search More Thoroughly, Summary, Run Times, Model Mismatch, Astronomy Example, Wrap Up
- [Tutorial 5: Results And Samples](scripts/chapter_1_introduction/tutorial_5_results_and_samples.py): In this tutorial, we'll cover all of the output that comes from a non-linear search's `Result` object.
- Contents: Data, Reused Functions, Model Fit, Result, Samples, Parameters, Figures of Merit, Instances, Vectors, Labels, Posterior / PDF, Plot, Errors, PDF, Other Results, Sample Instance, Bayesian Evidence, Derived Errors (PDF from samples), Samples Filtering, Latex

## chapter_3_graphical_models

- [Tutorial 1: Individual Models](scripts/chapter_3_graphical_models/tutorial_1_individual_models.py): In many examples, we fit one model to one dataset. For many problems, we may have a large dataset and are not interested in how well the model fits each individual dataset. Instead, we want to know how the model fits the full dataset, so that we can determine "global" trends of how the model fits the data.
- Contents: Real World Example, Example Source Code (`af.ex`), Model, Data, Model Fits (one-by-one), Results, Estimating the Centre, Posterior Multiplication, Wrap Up
- [Tutorial 2: Graphical Models](scripts/chapter_3_graphical_models/tutorial_2_graphical_model.py): We have fitted a dataset containing 5 noisy 1D Gaussian which had a shared `centre` value. We estimated the `centre` by fitting each dataset individually and combining the value of the `centre` inferred by each fit into an overall estimate, using a weighted average.
- Contents: Example Source Code (`af.ex`), Dataset, Analysis, Model, Analysis Factors, Factor Graph, Search, Result, Wrap Up
- [Tutorial 3: Graphical Benefits](scripts/chapter_3_graphical_models/tutorial_3_graphical_benefits.py): In the previous tutorials, we fitted a dataset containing 5 noisy 1D Gaussian which had a shared `centre` value and compared different approaches to estimate the shared `centre`. This included a simple approach fitting each dataset one-by-one and estimating the centre via a weighted average or posterior multiplication and a more complicated approach using a graphical model.
- Contents: The Model, Example Source Code (`af.ex`), Dataset, Analysis, Model (one-by-one), Model Fits (one-by-one), Centre Estimates (Weighted Average), Discussion, Model (Graphical), Analysis Factors, Factor Graph, Search, Result, Discussion, Posterior Multiplication, Wrap Up
- [Tutorial 4: Hierarchical](scripts/chapter_3_graphical_models/tutorial_4_hierachical_models.py): In the previous tutorial, we fitted a graphical model with the aim of determining an estimate of shared parameters, the `centre`'s of a dataset of 1D Gaussians. We did this by fitting all datasets simultaneously. When there are shared parameters in a model, this is a powerful and effective tool, but things may not always be so simple.
- Contents: Example Source Code (`af.ex`), Dataset, Analysis, Model Individual Factors, Analysis Factors, Model, Factor Graph, Search, Result, Comparison to One-by-One Fits, Benefits of Graphical Model, Wrap Up
- [Tutorial 5: Expectation Propagation](scripts/chapter_3_graphical_models/tutorial_5_expectation_propagation.py): In the previous tutorials, we fitted graphical models to dataset comprising many noisy 1D Gaussians. These had a shared and global value of their `centre`, or assumed their centres were hierarchically drawn from a parent Gaussian distribution. This provides the basis of composing and fitting complex graphical models to large datasets.
- Contents: Example Source Code (`af.ex`), Dataset, Analysis, Model, Analysis Factors, Factor Graph, Expectation Propagation, Cyclic Fitting, Result, Output, Results
- [Tutorial Optional: Hierarchical Expectation Propagation (EP)](scripts/chapter_3_graphical_models/tutorial_optional_hierarchical_ep.py): This optional tutorial gives an example of fitting a hierarchical model using EP.
- Contents: Example Source Code (`af.ex`), Dataset, Analysis, Model Individual Factors, Analysis Factors, Model, Factor Graph, Model Fit
- [Tutorial Optional: Hierarchical Individual](scripts/chapter_3_graphical_models/tutorial_optional_hierarchical_individual.py): In tutorial 4, we fit a hierarchical model using a graphical model, whereby all datasets are fitted simultaneously and the hierarchical parameters are fitted for simultaneously with the model parameters of each 1D Gaussian in each dataset.
- Contents: Example Source Code (`af.ex`), Dataset, Analysis, Model, Model Fits (one-by-one), Results, Overall Gaussian Parent Distribution, Model, Analysis + Search

## simulators

- [__Simulators__](scripts/simulators/simulators.py): These scripts simulate the 1D Gaussian datasets used to demonstrate model-fitting.
- Contents: Gaussian x1, Gaussian x1 (0), Gaussian x1 (1), Gaussian x1 (2), Gaussian x1 (Identical 0), Gaussian x1 (Identical 1), Gaussian x1 (Identical 2), Gaussian x1 + Exponential x1, Gaussian x2 + Exponential x1, Gaussian x2, Gaussian x3, Gaussian x5, Gaussian x1 unconvolved, Gaussian x1 convolved, Gaussian x1 with feature, Gaussian x2 split, Gaussian x1 time
- [__Simulators__](scripts/simulators/simulators_sample.py): These scripts simulates many 1D Gaussian datasets with a low signal to noise ratio, which are used to demonstrate model-fitting.
- Contents: Gaussian x1 low snr (centre fixed to 50.0), Gaussian x1 low snr (centre drawn from parent Gaussian distribution to 50.0), Gaussian x2 offset centre
- [util](scripts/simulators/util.py): (no summary in script docstring)
80 changes: 80 additions & 0 deletions llms.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
If your AI can't browse GitHub, paste this entire file into the chat as context.

# HowToFit

> The **HowToFit** lecture series: tutorial scripts (and generated notebooks) that teach
> Bayesian model-fitting and the **PyAutoFit** API from first principles, using simple 1D
> datasets (e.g. noisy Gaussians) so the statistics — not the science — are the focus. This
> file is a routing layer: given a learner's question, point them to the RIGHT existing
> tutorial in this repository instead of inventing an explanation. Every path below is a
> real file in this repo; scripts run from the repo root (e.g.
> `python scripts/chapter_1_introduction/tutorial_1_models.py`), and each has a matching
> notebook under `notebooks/`.

## Who this is for

HowToFit is the **on-ramp for beginners** — anyone new to model-fitting / Bayesian inference,
or new to PyAutoFit. It teaches model composition, fitting, non-linear searches, results, and
graphical/hierarchical models step by step, independent of any science domain. Once a learner
wants to apply this to a real problem, route them *out* to the production examples in
[autofit_workspace](https://github.com/PyAutoLabs/autofit_workspace) (see "Ready for real
analysis?" below).

## Start here

- [scripts/chapter_1_introduction/start_here.py](scripts/chapter_1_introduction/start_here.py): Orientation — what the lectures cover and how to run them.
- [scripts/chapter_1_introduction/tutorial_1_models.py](scripts/chapter_1_introduction/tutorial_1_models.py): The first real lecture — what a "model" is, and composing one with `Model` and `Collection`. Begin here if the learner is new to model-fitting.

## The learning path (chapters in order)

- **Chapter 1 — Introduction** → [scripts/chapter_1_introduction/](scripts/chapter_1_introduction/): The foundations of model-fitting — defining models, fitting data (residuals / chi-squared / likelihood), the non-linear search, why modeling is hard, and reading results & samples.
- **Chapter 3 — Graphical Models** → [scripts/chapter_3_graphical_models/](scripts/chapter_3_graphical_models/): Fitting one model across a large dataset — individual fits vs graphical models, shared parameters, hierarchical models, and Expectation Propagation for scaling to many datasets.

## I want to understand…

- **How do I compose a model?** → [scripts/chapter_1_introduction/tutorial_1_models.py](scripts/chapter_1_introduction/tutorial_1_models.py): `Model`, `Collection`, parameter mapping, tuple parameters, building complex models.
- **What does "fitting data" actually compute?** → [scripts/chapter_1_introduction/tutorial_2_fitting_data.py](scripts/chapter_1_introduction/tutorial_2_fitting_data.py): Model data, residuals, normalized residuals, chi-squared, the log likelihood.
- **How does a non-linear search work?** → [scripts/chapter_1_introduction/tutorial_3_non_linear_search.py](scripts/chapter_1_introduction/tutorial_3_non_linear_search.py): Parameter space, priors, MLE / MCMC / nested sampling, and which to use.
- **Why do model fits fail, and how do I fix them?** → [scripts/chapter_1_introduction/tutorial_4_why_modeling_is_hard.py](scripts/chapter_1_introduction/tutorial_4_why_modeling_is_hard.py): Prior tuning, reducing complexity, searching more thoroughly.
- **What's in the `Result` / how do I read samples?** → [scripts/chapter_1_introduction/tutorial_5_results_and_samples.py](scripts/chapter_1_introduction/tutorial_5_results_and_samples.py): Samples, instances, PDFs, errors, Bayesian evidence, LaTeX output.
- **I have many datasets with a shared parameter — what's the right tool?** → [scripts/chapter_3_graphical_models/tutorial_2_graphical_model.py](scripts/chapter_3_graphical_models/tutorial_2_graphical_model.py): Factor graphs and graphical models vs one-by-one fits.
- **What is a hierarchical model / Expectation Propagation?** → [scripts/chapter_3_graphical_models/tutorial_4_hierachical_models.py](scripts/chapter_3_graphical_models/tutorial_4_hierachical_models.py) then [tutorial_5_expectation_propagation.py](scripts/chapter_3_graphical_models/tutorial_5_expectation_propagation.py).

## Ready for real analysis?

When the learner wants to apply PyAutoFit to their *own* model/data (not a teaching dataset),
route them to the production workspace — it has cookbooks and science-workflow examples:

- [autofit_workspace](https://github.com/PyAutoLabs/autofit_workspace) — see its own `llms.txt` routing layer for model cookbooks, search configuration, and result analysis.

## How to answer (for the assistant)

When routing a learner, reply in this shape (the same shape the workspace navigator uses, so
the two agree):

- **Start here** — the single best tutorial for their question (from the lists above).
- **Then see** — the next tutorial in the chapter, or the concept it builds toward.
- **Related guide** — a relevant later chapter, or the matching `autofit_workspace` example for production use.
- **Why this is the right example** — one line tying the tutorial to their question.
- **What to modify** — the few things they'd change to explore (model components, dataset, search settings).
- **What needs local execution** — flag any step that requires actually running the fit on their machine.

### Code style (match the lectures, not banner comments)

Drafted code must match this repo's style: triple-quoted `"""__Section__"""` docstrings with
a line of prose explaining each step, and a module docstring whose title is underlined with
`=`. Do NOT use `# -----` banner comments to mark sections; reserve inline `#` for short
line-level notes only.

## Capability boundary (chat without local execution)

A chat assistant can route to the right tutorial, explain model-fitting concepts, review
pasted scripts / tracebacks / results, and draft code. It CANNOT run fits, inspect the
learner's local files or `output/` folder, or guarantee code against their installed
PyAutoFit version. To run a tutorial, the learner executes it locally from the repo root, or
in Google Colab (the notebooks ship Colab setup cells). For sustained editing, execution and
project state, point them to a local coding agent (Claude Code / Codex).

---

For the full per-tutorial listing, see the generated companion catalogue `llms-full.txt`.
Loading
Loading