Skip to content

[C++] Arrow test 'arrow-utility-test' contains container-overflow error #50244

Description

@PhoebeHui

Describe the bug, including details regarding any error messages, version, and platform.

When I verify arrow test after #48094 resolved, the test 'arrow-utility-test' failed with container-overflow error again, could you please take a look?

Steps to reproduce:

1. Open VS2022 x86 Native Command Prompt tools
2. Clone repo and checkout 0249a2e
3. set _CL_=/fsanitize=address /GS- /wd5072 & set _LINK_=/InferASanLibs /incremental:no /debug
4. cd to build folder: cd /d C:\gitP\apache\arrow\cpp\build_amd64
5. cmake -G "Visual Studio 17 2022" -A x64 -DCMAKE_SYSTEM_VERSION=10.0.26100.0  -DARROW_BUILD_TESTS=ON -DARROW_ACERO=ON -DARROW_FILESYSTEM=ON -DARROW_PARQUET=OFF -DARROW_SUBSTRAIT=ON -DARROW_DATASET=ON
6. msbuild /m /p:Platform=x64 /p:Configuration=Release arrow.sln /t:Rebuild
7.set ARROW_TEST_DATA=C:\gitP\apache\arrow\testing\data
8.ctest -C Release --output-on-failure -R "arrow-utility-test"

Memory safety issue reported by Address Sanitizer:

==6460==ERROR: AddressSanitizer: container-overflow on address 0x1298940cf031 at pc 0x7ffed5c28155 bp 0x00fb34d6e0c0 sp 0x00fb34d6d848
READ of size 42 at 0x1298940cf031 thread T0
    #0 0x7ffed5c28154 in MemcmpInterceptorCommon(void *, int (__cdecl *)(void const *, void const *, unsigned __int64), void const *, void const *, unsigned __int64) C:\repos\msvc\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:896
    #1 0x7ffed5c20600 in memcmp C:\repos\msvc\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:928
    #2 0x7ff72ef855c8 in testing::internal::CmpHelperEQ<class std::basic_string_view<char, struct std::char_traits<char>>, class std::basic_string_view<char, struct std::char_traits<char>>>(char const *, char const *, class std::basic_string_view<char, struct std::char_traits<char>> const &, class std::basic_string_view<char, struct std::char_traits<char>> const &) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x1405a55c8)
    #3 0x7ff72ef88189 in arrow::util::test::IsSecurelyCleared(class std::basic_string_view<char, struct std::char_traits<char>> const &) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x1405a8189)
    #4 0x7ff72ef72ffd in arrow::util::test::TestSecureString_AssertSecurelyCleared_Test::TestBody(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x140592ffd)
    #5 0x7fff05898986 in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test *, void (__cdecl testing::Test::*)(void), char const *) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180008986)
    #6 0x7fff058984db in testing::internal::HandleExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test *, void (__cdecl testing::Test::*)(void), char const *) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x1800084db)
    #7 0x7fff0590645c in testing::Test::Run(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x18007645c)
    #8 0x7fff05906721 in testing::TestInfo::Run(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180076721)
    #9 0x7fff05906b43 in testing::TestSuite::Run(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180076b43)
    #10 0x7fff05908491 in testing::internal::UnitTestImpl::RunAllTests(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180078491)
    #11 0x7fff058989e6 in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl *, bool (__cdecl testing::internal::UnitTestImpl::*)(void), char const *) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x1800089e6)
    #12 0x7fff058988d9 in testing::internal::HandleExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl *, bool (__cdecl testing::internal::UnitTestImpl::*)(void), char const *) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x1800088d9)
    #13 0x7fff05906fab in testing::UnitTest::Run(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180076fab)
    #14 0x7ff72eef1dce in main (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x140511dce)
    #15 0x7ff72f23ccae in invoke_main C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    #16 0x7ff72f23ccae in __scrt_common_main_seh C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    #17 0x7fff8bf5e956  (C:\WINDOWS\System32\KERNEL32.DLL+0x18002e956)
    #18 0x7fff8db47c1b  (C:\WINDOWS\SYSTEM32\ntdll.dll+0x180087c1b)

0x1298940cf040 is located 0 bytes after 48-byte region [0x1298940cf010,0x1298940cf040)
allocated by thread T0 here:
    #0 0x7ff72f23baf5 in operator new(unsigned __int64) C:\repos\msvc\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_win_new_scalar_thunk.cpp:40
    #1 0x7ff72ea65951 in std::_Allocate<16, struct std::_Default_allocate_traits>(unsigned __int64) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x140085951)
    #2 0x7ff72ea659e9 in std::basic_string<char, struct std::char_traits<char>, class std::allocator<char>>::_Allocate_for_capacity<0>(class std::allocator<char> &, unsigned __int64 &) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x1400859e9)
    #3 0x7ff72ea65f96 in std::basic_string<char, struct std::char_traits<char>, class std::allocator<char>>::_Construct<2, char const *>(char const *const, unsigned __int64) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x140085f96)
    #4 0x7ff72ef72f43 in arrow::util::test::TestSecureString_AssertSecurelyCleared_Test::TestBody(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x140592f43)
    #5 0x7fff05898986 in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test *, void (__cdecl testing::Test::*)(void), char const *) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180008986)
    #6 0x7fff058984db in testing::internal::HandleExceptionsInMethodIfSupported<class testing::Test, void>(class testing::Test *, void (__cdecl testing::Test::*)(void), char const *) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x1800084db)
    #7 0x7fff0590645c in testing::Test::Run(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x18007645c)
    #8 0x7fff05906721 in testing::TestInfo::Run(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180076721)
    #9 0x7fff05906b43 in testing::TestSuite::Run(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180076b43)
    #10 0x7fff05908491 in testing::internal::UnitTestImpl::RunAllTests(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180078491)
    #11 0x7fff058989e6 in testing::internal::HandleSehExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl *, bool (__cdecl testing::internal::UnitTestImpl::*)(void), char const *) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x1800089e6)
    #12 0x7fff058988d9 in testing::internal::HandleExceptionsInMethodIfSupported<class testing::internal::UnitTestImpl, bool>(class testing::internal::UnitTestImpl *, bool (__cdecl testing::internal::UnitTestImpl::*)(void), char const *) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x1800088d9)
    #13 0x7fff05906fab in testing::UnitTest::Run(void) (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow_gmock_main.dll+0x180076fab)
    #14 0x7ff72eef1dce in main (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x140511dce)
    #15 0x7ff72f23ccae in invoke_main C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
    #16 0x7ff72f23ccae in __scrt_common_main_seh C:\repos\msvc\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
    #17 0x7fff8bf5e956  (C:\WINDOWS\System32\KERNEL32.DLL+0x18002e956)
    #18 0x7fff8db47c1b  (C:\WINDOWS\SYSTEM32\ntdll.dll+0x180087c1b)

HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
Or if supported by the container library, pass -D__SANITIZER_DISABLE_CONTAINER_OVERFLOW__ to the compiler to disable  instrumentation.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
SUMMARY: AddressSanitizer: container-overflow (C:\gitP\apache\arrow\cpp\build_amd64\release\Release\arrow-utility-test.exe+0x1405a55c8) in testing::internal::CmpHelperEQ<class std::basic_string_view<char, struct std::char_traits<char>>, class std::basic_string_view<char, struct std::char_traits<char>>>(char const *, char const *, class std::basic_string_view<char, struct std::char_traits<char>> const &, class std::basic_string_view<char, struct std::char_traits<char>> const &)
Shadow bytes around the buggy address:
  0x1298940ced80: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
  0x1298940cee00: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
  0x1298940cee80: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
  0x1298940cef00: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fd
  0x1298940cef80: fa fa fd fd fd fd fd fd fa fa 00 00 00 00 00 03
=>0x1298940cf000: fa fa 00 00 00 00[01]fc fa fa 00 00 00 00 00 03
  0x1298940cf080: fa fa fd fd fd fd fd fd fa fa fd fd fd fd fd fa
  0x1298940cf100: fa fa fd fd fd fd fd fa fa fa fa fa fa fa fa fa
  0x1298940cf180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1298940cf200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1298940cf280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb

NOTE: the stack trace above identifies the code that *accessed* the poisoned memory.
HINT: To identify the code that *poisoned* the memory, try the experimental setting ASAN_OPTIONS=poison_history_size=<size>.
==6460==ABORTING

Component(s)

C++

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions