Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 80 additions & 48 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,100 @@
# PyAutoFit — Agent Instructions

**PyAutoFit** is a Python probabilistic programming language for model fitting and Bayesian inference.
Canonical, agent-agnostic instructions for this repo. `CLAUDE.md` imports this
file; any tool that does not process `@`-imports should read this directly.

## Setup
## What this repo is

```bash
pip install -e ".[dev]"
```
**PyAutoFit** (package `autofit`) is a probabilistic programming language for
model composition, non-linear search, and Bayesian inference: `af.Model` /
`af.Collection`, the `af.Analysis` interface, MCMC / nested-sampling / MLE
searches, samples, aggregator, and a SQLAlchemy results database.

Dependency direction: autofit depends on **autoconf** only. It does **not**
import `autoarray`, `autogalaxy`, or `autolens` — never add such an import.
Shared utilities (e.g. `test_mode`, `jax_wrapper`) belong in autoconf.

## Related repos

- **Source siblings:** PyAutoConf (upstream). PyAutoGalaxy / PyAutoLens are
downstream consumers (they build `Analysis` subclasses on autofit).
- **autofit_workspace** — runnable tutorials/examples (`../autofit_workspace`).
- **autofit_workspace_test** — integration + JAX/likelihood parity scripts.
- **HowToFit** — the lecture-style tutorial series (`../HowToFit`).
- **docs/** — Sphinx source; published to ReadTheDocs.

## Running Tests
## Quick commands

```bash
python -m pytest test_autofit/
python -m pytest test_autofit/non_linear/
python -m pytest test_autofit/mapper/
pip install -e ".[dev]" # install with dev/test extras
python -m pytest test_autofit/ # full test suite
python -m pytest test_autofit/non_linear/ # one focused subtree (add -s for output)
black autofit/ # formatter (advisory — not gated)
```

### Sandboxed / Codex runs
In a sandboxed / restricted environment, point numba and matplotlib at
writable caches:

```bash
NUMBA_CACHE_DIR=/tmp/numba_cache MPLCONFIGDIR=/tmp/matplotlib python -m pytest test_autofit/
```

## Key Architecture
## CI / definition of green

- **Non-linear searches** (`non_linear/search/`): MCMC (emcee), nested sampling (dynesty, nautilus), MLE (LBFGS, BFGS, drawer)
- **Model composition** (`mapper/`): `af.Model`, `af.Collection`, prior distributions
- **Analysis** (`non_linear/analysis/`): base `af.Analysis` class with `log_likelihood_function`
- **Aggregator** (`aggregator/`): results aggregation across runs
- **Database** (`database/`): SQLAlchemy backend for results storage
- **Graphical models** (`graphical/`): expectation propagation
PRs must pass `pytest --cov` on the CI matrix (Python 3.12 **and** 3.13). There
is no black/ruff/flake8 gate — formatting is advisory. (`requires-python` in
`pyproject.toml` is `>=3.9`.)

## Key Rules
## Configuration & defaults

- All files must use Unix line endings (LF)
- Format with `black autofit/`
autoconf supplies the packaged defaults under `autofit/config/`. Workspaces
override them via their own `config/` directory; the test suite pushes a local
config dir via `conf.instance.push(...)` in `test_autofit/conftest.py`. When a
change adds a new config key, mirror it into the packaged defaults so
downstream workspaces inherit it.

## Public API

The public surface is defined authoritatively in `autofit/__init__.py` — read
it rather than trusting a hand-maintained list. Canonical import:

```python
import autofit as af
```

## Working on Issues
Key subsystems: `non_linear/search/` (MCMC: emcee/zeus; nested: dynesty,
nautilus, NSS; MLE: LBFGS/BFGS/drawer), `mapper/` (model + priors),
`non_linear/analysis/` (`af.Analysis.log_likelihood_function`), `aggregator/`,
`database/` (SQLAlchemy), `graphical/` (expectation propagation),
`interpolator/`.

## Key rules / footguns

- Import direction: autoconf only — never `autoarray` / `autogalaxy` /
`autolens`.
- The `[nss]` extra (for `af.NSS`) needs a pinned **handley-lab/blackjax** fork
installed manually *after* the extras step; that fork conflicts with the
mainline `blackjax` pinned in `[optional]`. Do not naively combine or bump
them — see the install notes in `pyproject.toml`.
- All files use Unix line endings (LF, `\n`) — never `\r\n`.

## Working on issues

1. Read the issue description and any linked plan.
2. Identify affected files and write your changes.
3. Run the full test suite: `python -m pytest test_autofit/`
4. Ensure all tests pass before opening a PR.
5. If changing public API, note the change in your PR description — downstream packages (PyAutoArray, PyAutoGalaxy, PyAutoLens) and workspaces may need updates.
## Never rewrite history

NEVER perform these operations on any repo with a remote:

- `git init` in a directory already tracked by git
- `rm -rf .git && git init`
- Commit with subject "Initial commit", "Fresh start", "Start fresh", "Reset
for AI workflow", or any equivalent message on a branch with a remote
- `git push --force` to `main` (or any branch tracked as `origin/HEAD`)
- `git filter-repo` / `git filter-branch` on shared branches
- `git rebase -i` rewriting commits already pushed to a shared branch

If the working tree needs a clean state, the **only** correct sequence is:

git fetch origin
git reset --hard origin/main
git clean -fd

This applies equally to humans, local Claude Code, cloud Claude agents, Codex,
and any other agent. The "Initial commit — fresh start for AI workflow" pattern
that appeared independently on origin and local for three workspace repos is
exactly what this rule prevents — it costs ~40 commits of redundant local work
every time it happens.
2. Identify affected files and make the change.
3. Run the full suite: `python -m pytest test_autofit/`.
4. If you changed public API, say so explicitly — downstream packages
(PyAutoGalaxy, PyAutoLens) and the workspaces may need updates.
5. Ensure all tests pass before opening a PR.

## Clean state

Never rewrite history on a repo with a remote (no `git init` over a tracked
tree, no force-push to `main`, no rebasing pushed shared branches). To reset a
dirty tree the only correct sequence is:

```bash
git fetch origin
git reset --hard origin/main
git clean -fd
```
100 changes: 5 additions & 95 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,95 +1,5 @@
# PyAutoFit

**PyAutoFit** is a Python probabilistic programming language for model fitting and Bayesian inference.

- Authors: James Nightingale, Richard Hayes
- Requires Python >= 3.12
- Package name: `autofit`

## Dependency Graph

PyAutoFit depends on **autoconf** (shared configuration and utilities).
PyAutoFit does **NOT** depend on PyAutoArray, PyAutoGalaxy, or PyAutoLens.
Never import from `autoarray`, `autogalaxy`, or `autolens` in this repo.
Shared utilities (e.g. `test_mode`, `jax_wrapper`) belong in autoconf.

## Repository Structure

- `autofit/` - Main package
- `non_linear/` - Non-linear search algorithms
- `search/mcmc/` - MCMC (emcee, zeus)
- `search/mle/` - Maximum likelihood (LBFGS, BFGS, drawer)
- `search/nest/` - Nested sampling (dynesty, nautilus)
- `samples/` - Posterior samples handling
- `paths/` - Output path management
- `analysis/` - Analysis base classes
- `mapper/` - Model and prior machinery
- `prior/` - Prior distributions
- `prior_model/` - Prior model composition
- `model.py` - Core model class
- `graphical/` - Graphical models and expectation propagation
- `aggregator/` - Results aggregation across runs
- `database/` - SQLAlchemy-based results database
- `interpolator/` - Model interpolation
- `config/` - Default config files packaged with library
- `test_autofit/` - Test suite (pytest)
- `docs/` - Sphinx documentation

## Key Dependencies

- `dynesty==2.1.5` - Nested sampling
- `emcee>=3.1.6` - MCMC
- `scipy<=1.14.0` - Optimisation
- `SQLAlchemy==2.0.32` - Database backend
- `anesthetic==2.8.14` - Posterior analysis/plotting
- Optional: `nautilus-sampler`, `zeus-mcmc`, `getdist`

## Running Tests

```
pytest test_autofit
pytest test_autofit/non_linear
pytest test_autofit/mapper
```

## Codex / sandboxed runs

When running Python from Codex or any restricted environment, set writable cache directories so `numba` and `matplotlib` do not fail on unwritable home or source-tree paths:

```bash
NUMBA_CACHE_DIR=/tmp/numba_cache MPLCONFIGDIR=/tmp/matplotlib pytest test_autofit
```

This workspace is often imported from `/mnt/c/...` and Codex may not be able to write to module `__pycache__` directories or `/home/jammy/.cache`, which can cause import-time `numba` caching failures without this override.

## Shell Commands

- Prefer simple shell commands
- Avoid chaining with `&&` or pipes; run commands separately

## Related Repos

- **autofit_workspace** (tutorials/examples): `../autofit_workspace`
## Never rewrite history

NEVER perform these operations on any repo with a remote:

- `git init` in a directory already tracked by git
- `rm -rf .git && git init`
- Commit with subject "Initial commit", "Fresh start", "Start fresh", "Reset
for AI workflow", or any equivalent message on a branch with a remote
- `git push --force` to `main` (or any branch tracked as `origin/HEAD`)
- `git filter-repo` / `git filter-branch` on shared branches
- `git rebase -i` rewriting commits already pushed to a shared branch

If the working tree needs a clean state, the **only** correct sequence is:

git fetch origin
git reset --hard origin/main
git clean -fd

This applies equally to humans, local Claude Code, cloud Claude agents, Codex,
and any other agent. The "Initial commit — fresh start for AI workflow" pattern
that appeared independently on origin and local for three workspace repos is
exactly what this rule prevents — it costs ~40 commits of redundant local work
every time it happens.
# PyAutoFit — agent instructions
The canonical, agent-agnostic instructions live in `AGENTS.md`. Claude Code loads them
via the import below; if your tool does not process `@`-imports, open `AGENTS.md` in
this directory and read it directly.
@AGENTS.md
Loading