Skip to content

Preserve input dtype on perlin() dask backends#3471

Open
brendancol wants to merge 4 commits into
mainfrom
deep-sweep-accuracy-perlin-2026-06-23-01
Open

Preserve input dtype on perlin() dask backends#3471
brendancol wants to merge 4 commits into
mainfrom
deep-sweep-accuracy-perlin-2026-06-23-01

Conversation

@brendancol

Copy link
Copy Markdown
Contributor

perlin() returned a different output dtype depending on the backend. A float64 input kept float64 on numpy and cupy but came back as float32 on both dask backends, because _perlin_dask_numpy and _perlin_dask_cupy passed a hardcoded float32 to da.map_blocks.

(GitHub issues are disabled on this repo, so there is no linked issue.)

What changed

  • Both dask functions now derive the map_blocks output dtype from data.dtype instead of pinning float32.
  • Added cross-backend dtype regression tests so all four backends stay in sync.

Backend coverage

numpy / cupy already preserved the input dtype; this fixes dask+numpy and dask+cupy to match. Verified all four return float32 for float32 input and float64 for float64 input.

Test plan

  • pytest xrspatial/tests/test_perlin.py (18 passed, GPU paths included)
  • Existing cross-backend numerical parity tests still pass
  • New dtype tests cover numpy, cupy, dask+numpy, dask+cupy

The dask+numpy and dask+cupy paths hardcoded float32 in their
map_blocks calls, so a float64 input was silently downcast to float32
while the numpy and cupy backends kept float64. Derive the output
dtype from data.dtype in both dask functions and add cross-backend
dtype regression tests covering all four backends.

@brendancol brendancol left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: Preserve input dtype on perlin() dask backends

Blockers

None.

Suggestions

None.

Nits

  • test_perlin.py:127 test_perlin_dask_cpu_preserves_dtype checks only the output dtype. Since this bug was specifically about float64, you could also assert the float64 result is finite and in [0, 1], the way test_perlin_float64_input does for numpy. Optional; the existing parity tests already check the values.

What looks good

  • Both dask paths derive out_dtype from data.dtype now, so they line up with numpy and cupy, which already preserved it.
  • The partial import was unused once the inline closure replaced it, and it got removed.
  • New tests hit all four backends across float32 and float64.
  • perlin() rejects non-floating dtypes upfront, so out_dtype is always float32 or float64 and the cast is safe.

Note on precision

For float64 input the noise is still computed in float32 inside _perlin and the GPU kernel, then stored as float64. That is storage precision, not compute precision. It matches what numpy and cupy already do (float32 math written into a float64 buffer), so parity holds and nothing regresses here.

Checklist

  • Algorithm unchanged; only the output dtype derivation moved
  • All four backends agree (parity tests pass)
  • NaN handling untouched (perlin overwrites the input buffer)
  • dtype covered for both float32 and float64
  • Dask chunk boundaries unaffected (no shape change)
  • No premature materialization
  • No benchmark needed
  • README unchanged
  • Docstrings unchanged

@brendancol brendancol left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review (follow-up)

The one nit from the previous pass is fixed: test_perlin_dask_cpu_preserves_dtype now also asserts the result is finite and normalized to [0, 1] for both dtypes, matching test_perlin_float64_input.

No remaining findings. Blockers: none. Suggestions: none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant