fix(isthmus): use Substrait type system when building Calcite RelBuilder#973
Merged
bestbeforetoday merged 1 commit intoJun 30, 2026
Conversation
When converting Substrait to Calcite, the RelBuilder was created via RelBuilder.create(Frameworks...) without specifying a type system, so its internal RexBuilder used Calcite's default type system (max decimal precision 19) while the converted expressions carried SubstraitTypeSystem types (precision 38). RelBuilder.project runs RexSimplify by default; with the mismatched type system it re-derived decimal arithmetic at precision 19 and then wrapped the result in a preserving CAST(... AS DECIMAL(38,0)) to keep the declared type. For TPC-H queries 8 and 14 this surfaced as the numerator SUM(CAST(CASE ... AS DECIMAL(19,0))) while the denominator was left uncast. In PostgreSQL, where the DECIMAL columns are arbitrary-precision NUMERIC, the cast truncated the fractional values in the numerator only, so the round-tripped SQL produced different results from the reference. This was latent until Calcite 1.42, which changed RelDataTypeSystemImpl so the max numeric precision/scale are derived from getMaxPrecision(DECIMAL)/getMaxScale(DECIMAL). Build the Substrait-to-Calcite RelBuilder with the Substrait type system, exposed via a new ConverterProvider.getTypeSystem() accessor (derived from the type factory so the two never disagree). SubstraitTypeSystem gains a public no-arg constructor because Calcite's Frameworks/Avatica machinery re-instantiates the type system from its class name when it is supplied to a FrameworkConfig. Removed queries 8 and 14 from the integration test's expected-failures list, and dropped the now-empty expected-failures code path. Closes substrait-io#954 Closes substrait-io#955
bestbeforetoday
approved these changes
Jun 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When converting Substrait to Calcite, the
RelBuilderwas created viaRelBuilder.create(Frameworks...)without specifying a type system, so its internalRexBuilderused Calcite's default type system (max decimal precision 19) while the converted expressions carriedSubstraitTypeSystemtypes (precision 38).RelBuilder.projectrunsRexSimplifyby default; with the mismatched type system it re-derived decimal arithmetic at precision 19 and then wrapped the result in a preservingCAST(... AS DECIMAL(38,0))to keep the declared type.For TPC-H queries 8 and 14 this surfaced as the numerator
SUM(CAST(CASE ... AS DECIMAL(19,0)))while the denominator was left uncast. In PostgreSQL, where theDECIMALcolumns are arbitrary-precisionNUMERIC, the cast truncated the fractional values in the numerator only, so the round-tripped SQL produced different results from the reference. This was latent until Calcite 1.42, which changedRelDataTypeSystemImplso the max numeric precision/scale are derived fromgetMaxPrecision(DECIMAL)/getMaxScale(DECIMAL).Changes
RelBuilderwith the Substrait type system, exposed via a newConverterProvider.getTypeSystem()accessor (derived from the type factory so the two never disagree).SubstraitTypeSystemgains a public no-arg constructor because Calcite'sFrameworks/Avatica machinery re-instantiates the type system from its class name when it is supplied to aFrameworkConfig.Testing
./gradlew :isthmus:test— all unit tests pass../gradlew :isthmus:integrationTest— all 22 TPC-H queries (including 8 and 14) now produce results matching the reference SQL in PostgreSQL.Closes #954
Closes #955
🤖 Generated with AI