Skip to content

ORC range start index 18446744073709551615 out of range for slice of length 4 #1981

Description

@cxzl25

Describe the bug

Execution error: range start index 18446744073709551615 out of range for slice of length 4
	at org.apache.spark.sql.auron.JniBridge.nextBatch(Native Method)
	at org.apache.spark.sql.auron.AuronCallNativeWrapper$$anon$1.hasNext(AuronCallNativeWrapper.scala:86)
	at org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:31)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
	at org.apache.spark.sql.execution.UnsafeExternalRowSorter.sort(UnsafeExternalRowSorter.java:227)
	at org.apache.spark.sql.execution.SortExec.$anonfun$doExecute$1(SortExec.scala:119)
	at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:898)
	at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:898)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  19:     0x7f250ab0eed6 - orc_rust::encoding::integer::NInt::read_big_endian::hac8e9ec8cbefb5f8
  20:     0x7f250ab0e595 - <orc_rust::encoding::integer::rle_v2::RleV2Decoder<N,R,S> as orc_rust::encoding::rle::GenericRle<N>>::decode_batch::h4ef76493fdff5b65
  21:     0x7f250ab0d838 - orc_rust::encoding::rle::<impl orc_rust::encoding::PrimitiveValueDecoder<V> for G>::decode::h6ae96607824a2614
  22:     0x7f250ab0da14 - orc_rust::encoding::PrimitiveValueDecoder::decode_spaced::hd949ec753d4da0cf
  23:     0x7f250ab163f3 - <orc_rust::array_decoder::PrimitiveArrayDecoder<T> as orc_rust::array_decoder::ArrayBatchDecoder>::next_batch::h2128ebdb460e4d0b
  24:     0x7f250aaf3876 - <orc_rust::array_decoder::struct_decoder::StructArrayDecoder as orc_rust::array_decoder::ArrayBatchDecoder>::next_batch::h9a6c03fd2daf2965
  25:     0x7f250aaf6c2e - orc_rust::array_decoder::NaiveStripeDecoder::decode_next_batch::h00d5a2dcb9d60f0e
  26:     0x7f250aaf6713 - <orc_rust::array_decoder::NaiveStripeDecoder as core::iter::traits::iterator::Iterator>::next::h1eee41863c737bb3
  27:     0x7f25097c97f8 - <futures_util::stream::stream::map::Map<St,F> as futures_core::stream::Stream>::poll_next::h049b1093bbbbc33d
  28:     0x7f2508d7a559 - <datafusion_datasource::file_stream::FileStream as futures_core::stream::Stream>::poll_next::hb889cd02bfe5323d
  29:     0x7f25097bf6b9 - datafusion_ext_plans::common::execution_context::ExecutionContext::output_with_sender_impl::{{closure}}::h219066644ac66f16
  30:     0x7f25097c0569 - tokio::runtime::task::raw::poll::hd52de12d901d9858
  31:     0x7f250aedd367 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::hb4206a315ae080d6
  32:     0x7f250aee3e07 - tokio::runtime::task::raw::poll::h206608637a90eae9

datafusion-contrib/orc-rust#77

To Reproduce

Expected behavior

Screenshots

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions