Skip to content

feat(isthmus): migrate to PrecisionTime / PrecisionTimestamp#721

Open
nielspardon wants to merge 2 commits intosubstrait-io:mainfrom
nielspardon:par-isthmus-prectime
Open

feat(isthmus): migrate to PrecisionTime / PrecisionTimestamp#721
nielspardon wants to merge 2 commits intosubstrait-io:mainfrom
nielspardon:par-isthmus-prectime

Conversation

@nielspardon
Copy link
Member

This PR migrates isthmus to use PrecisionTime / PrecisionTimestamp instead of the deprecated Time / Timestamp types.

Signed-off-by: Niels Pardon <par@zurich.ibm.com>
@nielspardon nielspardon force-pushed the par-isthmus-prectime branch from d686f0d to a4d3319 Compare March 3, 2026 12:20
void extract() {
test(
"extract:req_ts",
"extract:req_pts",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need to test req_ts?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round tripping function calls with the deprecated Timestamp, TimestampTZ, Time was already broken in some cases before this change. We don't have tests that certify backwards compatibility of parsing Substrait plans using deprecated types / functions.

* @return a PrecisionTimeLiteral with nanosecond precision representing the given time
* @see #precisionTime(boolean, long, int) for creating precision time with custom precision
*/
public static Expression.PrecisionTimeLiteral precisionTime(boolean nullable, LocalTime value) {
Copy link
Member

@bestbeforetoday bestbeforetoday Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look to be used anywhere in the codebase. If it is worth adding, it should probably have a test driving it.

* @see #precisionTimestamp(boolean, long, int) for creating precision timestamp with custom
* precision
*/
public static Expression.PrecisionTimestampLiteral precisionTimestamp(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look to be used anywhere in the codebase. If it is worth adding, it should probably have a test driving it.

@nielspardon nielspardon force-pushed the par-isthmus-prectime branch from dc4841c to 4e580a0 Compare March 5, 2026 11:02
@bestbeforetoday
Copy link
Member

bestbeforetoday commented Mar 5, 2026

Thinking more on the new ExpressionCreator precisionTime and precisionTimestamp functions, it seems that these both create expressions that cannot then be converted to SQL since they exceed the maximum precision supported by Calcite and so will result in failure (in Isthmus) due to the precision checks this PR also adds. Assuming I have that correct, are we implementing the most useful behaviour? Should the precisionTime and precisionTimestamp functions create expressions at a precision that can be successfully converted to Calcite, or should we be more lenient on the conversion and truncate (or round) values during conversion to Calcite instead of failing? Maybe it is still fine since you might just be creating a Substrait plan that is not intended to be converted to Calcite using Isthmus? Worth some consideration anyway.

Signed-off-by: Niels Pardon <par@zurich.ibm.com>
@nielspardon nielspardon force-pushed the par-isthmus-prectime branch from 4e580a0 to 655a26b Compare March 5, 2026 11:09
@bestbeforetoday
Copy link
Member

I see the build failed because some test was expecting UnsupportedOperationException instead of IllegalArgumentException. If that is existing behaviour then it seems OK to me to preserve it, even if it also seems an unintuitive and potentially confusing exception type to me. Your choice.

@nielspardon
Copy link
Member Author

I see the build failed because some test was expecting UnsupportedOperationException instead of IllegalArgumentException. If that is existing behaviour then it seems OK to me to preserve it, even if it also seems an unintuitive and potentially confusing exception type to me. Your choice.

I fixed all occurences. We had the same pattern also in another class.

@nielspardon
Copy link
Member Author

Assuming I have that correct, are we implementing the most useful behaviour? Should the precisionTime and precisionTimestamp functions create expressions at a precision that can be successfully converted to Calcite, or should we be more lenient on the conversion and truncate (or round) values during conversion to Calcite instead of failing?

I think this a complex and separate topic.

  • ExpressionCreator is a component in core which means it can be used independently of isthmus / Calcite. - However the existing LocalDateTime, LocalDate, LocalTime method variants in the creator were originally created to make the interoperability with Calcite easier since you can get one of the above mentioned objects from a Calcite datetime expression.
  • I thought it might in general (independently of Calcite) be useful to create Substrait datetime objects from Java datetime objects without consumers of the substrait-java core having to do the conversion to milli, micro, nano, pico seconds themselves
  • How we handle compatibility between core / isthmus is yet another topic. We may even need to handle precision compatibility on the SQL dialect level. I expect the Substrait dialect to play a role here. I recently added support for specifying the maximum supported precision for datetime types to the Substrait dialect: feat(dialect): support specifying maximum supported subsecond precision substrait#961

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants