Fix the shred logic in parquet-variant

[klion26]

Describe the bug

Currently, we'll use the same VariantToArrowRowBuilder for shred_variant and variant_get, but shred and get are not always contain the same logic, even we have applied the cast-align with arrow-cast, such as

• bool -> int can't be casted in the code path of shred
• bool -> int can be casted in code path of get

To Reproduce

The following test should pass

#[test]
fn test_variant_shred() {  
       let mut array_builder = VariantArrayBuilder::new(30);

        array_builder.append_value(Variant::Int8(1));
        array_builder.append_value(Variant::BooleanTrue);

        let array = array_builder.build();
        let int8_shred_array = shred_variant(&array, &DataType::Int8).unwrap();
        let expected_array_builder = VariantArrayBuilder::new(30);
        let value = int8_shred_array.inner().column_by_name("value");
        let typed_value = int8_shred_array.inner().column_by_name("typed_value");

        // the Variant::BooleanTrue should not be shredded
        assert!(int8_shred_array.inner().column_by_name("value").unwrap().is_valid(1))
}

Expected behavior

We can't do cast logic in shred(like bool -> int*), but we can widen some types(like int8 -> int16)

Additional context

No response

[klion26]

take

[klion26]

cc @scovich this was found when I'm working on #10024

[klion26]

Variant::as_xx should not do the cast, but only can do the widen logic(etc int8 -> int16, and maybe int16 -> int8 if the value can fit in the range of int8), all the cast logic(for variant_get) we need to do in type_conversion.rs

[scovich]

Makes sense. Is the idea for variant_get to actually invoke casting logic if/when needed?

[sdf-jkl]

thanks @klion26 that makes sense.

[klion26]

Is the idea for variant_get to actually invoke casting logic if/when needed?

@scovich Yes, will always invoke casting logic in variant_get code path(int8 -> int16, and bool -> int8, etc), but not do cast in shred code path.