Skip to content

Stabilize discrete-state initial probability normalization#4375

Draft
FlorianPfaff wants to merge 2 commits into
mainfrom
agent/stabilize-discrete-state-probabilities-20260714
Draft

Stabilize discrete-state initial probability normalization#4375
FlorianPfaff wants to merge 2 commits into
mainfrom
agent/stabilize-discrete-state-probabilities-20260714

Conversation

@FlorianPfaff

Copy link
Copy Markdown
Owner

Summary

  • normalize finite-state initial probability vectors after scaling by their largest entry
  • preserve relative HMM and IMM priors when their direct floating-point sum would overflow
  • add regressions for both discrete_forward_backward() and imm_forward_backward()

Root cause

The public discrete-state compatibility layer validated each prior entry as finite and non-negative, then summed the raw values directly:

total = float(values.sum())
return values / total

Individually finite weights can still have an infinite sum. For example, [max_float, max_float / 2] has the well-defined ratio 2:1, but its direct sum overflows to inf. Dividing by that total produces [0, 0], so valid initial HMM or IMM probabilities lose all mass and the filter fails at the first emission.

Fix

After applying the optional valid-state mask, scale the vector by its largest entry before summing. The scaled values lie in [0, 1], retain the original ratios, and normalize to finite unit mass. Existing validation for shape, numeric type, finiteness, non-negativity, and positive mass is preserved.

Validation

  • reproduced the old normalization result: [max_float, max_float / 2] -> [0, 0]
  • verified the scale-first result is [2/3, 1/3] with unit mass
  • added end-to-end HMM and IMM regressions through the public API
  • python -m py_compile passes for the modified source and new test
  • branch is two commits ahead of current main and zero behind; production diff is 4 additions and 3 deletions

The full repository test and backend matrix is delegated to GitHub Actions.

@github-actions

Copy link
Copy Markdown
Contributor

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ COPYPASTE jscpd yes no no 21.63s
✅ JSON prettier 7 0 0 0 1.32s
✅ JSON v8r 7 0 0 3.88s
✅ MARKDOWN markdownlint 68 0 0 0 1.94s
✅ MARKDOWN markdown-table-formatter 68 0 0 0 0.62s
✅ PYTHON black 1490 182 0 0 86.61s
✅ PYTHON isort 1490 328 0 0 2.52s
✅ REPOSITORY betterleaks yes no no 2.7s
✅ REPOSITORY checkov yes no no 49.31s
✅ REPOSITORY gitleaks yes no no 13.61s
✅ REPOSITORY git_diff yes no no 0.38s
✅ REPOSITORY secretlint yes no no 59.3s
✅ REPOSITORY syft yes no no 6.39s
✅ REPOSITORY trivy-sbom yes no no 7.79s
✅ REPOSITORY trufflehog yes no no 30.88s
✅ YAML prettier 11 0 0 0 0.66s
✅ YAML v8r 11 0 0 9.96s
✅ YAML yamllint 11 0 0 0.52s

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts

Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining FLAVOR_SUGGESTIONS: false)

  • Documentation: Custom Flavors
  • Command: npx mega-linter-runner@9.6.0 --custom-flavor-setup --custom-flavor-linters PYTHON_BLACK,PYTHON_ISORT,COPYPASTE_JSCPD,JSON_V8R,JSON_PRETTIER,MARKDOWN_MARKDOWNLINT,MARKDOWN_MARKDOWN_TABLE_FORMATTER,REPOSITORY_CHECKOV,REPOSITORY_GIT_DIFF,REPOSITORY_GITLEAKS,REPOSITORY_BETTERLEAKS,REPOSITORY_SECRETLINT,REPOSITORY_SYFT,REPOSITORY_TRIVY_SBOM,REPOSITORY_TRUFFLEHOG,YAML_PRETTIER,YAML_YAMLLINT,YAML_V8R

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant