We sometimes come across systems with user defined types or less common types such as ST_GEOMETRY. These are a blocker.
It would be helpful to have an option to instruct Offload to apply a UDF to certain columns to offload them. For example:
bin/offload -t own.complex_table -x \
--udf-columns=own.my_udf1:col_stgeo1,col_stgeo2 \
--udf-columns=own2.my_udf2:col_custom_type
Or perhaps it could match what we do with precision/scale:
bin/offload -t own.complex_table -x \
--udf=own.my_udf1 --udf-columns=col_stgeo1,col_stgeo2 \
--udf=own2.my_udf2 --udf-columns=col_custom_type
We could have a limitation that UDFs must return a string, at least in order to get started. The onus would be on the user to format the data into a suitable string result.
This would have metadata implications because incremental offloads would need to know what function was used the original offload. The metadata would need to record this information on a per column basis.
We sometimes come across systems with user defined types or less common types such as ST_GEOMETRY. These are a blocker.
It would be helpful to have an option to instruct Offload to apply a UDF to certain columns to offload them. For example:
Or perhaps it could match what we do with precision/scale:
We could have a limitation that UDFs must return a string, at least in order to get started. The onus would be on the user to format the data into a suitable string result.
This would have metadata implications because incremental offloads would need to know what function was used the original offload. The metadata would need to record this information on a per column basis.