Skip to content

Add cuTile RMSNorm and shared DTensor handling updates#1285

Draft
Charlesrizzly wants to merge 1 commit into
linkedin:mainfrom
Charlesrizzly:kenemuo/rmsnorm-pr1a-cutile
Draft

Add cuTile RMSNorm and shared DTensor handling updates#1285
Charlesrizzly wants to merge 1 commit into
linkedin:mainfrom
Charlesrizzly:kenemuo/rmsnorm-pr1a-cutile

Conversation

@Charlesrizzly

Copy link
Copy Markdown

Summary

This PR introduces the cuTile-side RMSNorm backend implementation and related core runtime updates, split out as the first part of the backend stack.

Main changes:

  • Added cuTile RMSNorm implementation and export wiring.
  • Updated shared DTensor handling in core ops paths to use a common detection helper.
  • Applied DTensor-path consistency updates in RMSNorm, ModulatedRMSNorm, and SwiGLU.
  • Updated cuTile backend routing coverage to include RMSNorm.

Testing Done

Updated backend routing test coverage for cuTile RMSNorm path.

  • Hardware Type: H100 and B200
  • run make test to ensure correctness
  • run make checkstyle to ensure code style
  • run make test-convergence to ensure convergence

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant