parse_markdown_v3_without_pre parser function causes a core dump due to corrupted UTF-8 bytes

[nazarenkomykyta]

Hi!

When converting UTF-8 bytes to UTF-16 offsets in the body of the parse_markdown_v3_without_pre() function, one of the messages from the RUWIKI ( https://t.me/ruwikiofficial ) telegram channel triggers an assert.

I suspect that the bot used to send messages on this telegram channel is most likely posting messages with very strange formatting, which is causing this behavior.

( gdb ) bt :

#0  thrkill () at /tmp/-:2
#1  0xb845c40d4bd51510 in ?? ()
#2  0x00000dbe54a2f97b in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51
#3  0x00000dbbbe34b6fd in td::process_fatal_error () from /usr/local/bin/nchat
#4  0x00000dbbbe34c038 in td::Logger::~Logger () from /usr/local/bin/nchat
#5  0x00000dbbbe32fc41 in td::detail::process_check_error ()
   from /usr/local/bin/nchat
#6  0x00000dbbbd101354 in _ZN2tdL29parse_markdown_v3_without_preENS_5SliceENSt3__16vectorINS_13MessageEntityENS1_9allocatorIS3_EEEE ()
   from /usr/local/bin/nchat
#7  0x00000dbbbd0ef584 in _ZN2tdL36parse_markdown_v3_without_blockquoteENS_13FormattedTextE () from /usr/local/bin/nchat
#8  0x00000dbbbd0efbc1 in td::parse_markdown_v3(td::FormattedText)::$_0::operator() () from /usr/local/bin/nchat
#9  0x00000dbbbd0ef053 in td::parse_markdown_v3 () from /usr/local/bin/nchat
#10 0x00000dbbbd0f07f4 in td::get_markdown_v3 () from /usr/local/bin/nchat
#11 0x00000dbbbd817805 in td::SynchronousRequests::do_request ()
   from /usr/local/bin/nchat
#12 0x00000dbbbd824302 in _ZZN2td19SynchronousRequests11run_requestEONS_2tl10unique_ptrINS_6td_api8FunctionEEEENK3$_1clINS3_15getMarkdownTextEEEDaRT_ ()
   from /usr/local/bin/nchat
#13 0x00000dbbbd80d1f0 in td::SynchronousRequests::run_request ()
   from /usr/local/bin/nchat
---Type <return> to continue, or q <return> to quit---
#14 0x00000dbbbcc0c2f1 in td::Client::execute () from /usr/local/bin/nchat
#15 0x00000dbbbcbde7c6 in TgChat::Impl::GetText () from /usr/local/bin/nchat
#16 0x00000dbbbcbdf7f0 in TgChat::Impl::TdMessageContentConvert ()
   from /usr/local/bin/nchat
#17 0x00000dbbbcbe09fa in TgChat::Impl::TdMessageConvert ()
   from /usr/local/bin/nchat
#18 0x00000dbbbcbf1b26 in std::__1::__function::__func<TgChat::Impl::GetChatHistory(long long, long long, int, int, bool)::$_0, std::__1::allocator<TgChat::Impl::GetChatHistory(long long, long long, int, int, bool)::$_0>, void ()(td::tl::unique_ptr<td::td_api::Object>)>::operator() () from /usr/local/bin/nchat
#19 0x00000dbbbcbc2c16 in TgChat::Impl::ProcessResponse ()
   from /usr/local/bin/nchat
#20 0x00000dbbbcbb99f6 in TgChat::Impl::ProcessService ()
   from /usr/local/bin/nchat
#21 0x00000dbbbcbf5af1 in _ZNSt3__114__thread_proxyB8ne190107INS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEMN6TgChat4ImplEFvvEPS8_EEEEEPvSD_ () from /usr/local/bin/nchat
#22 0x00000dbe0d73f7b2 in _rthread_start (v=Unhandled dwarf expression opcode 0xf3
)
    at /usr/src/lib/librthread/rthread.c:99
#23 0x00000dbe549e506a in __tfork_thread ()
    at /usr/src/lib/libc/arch/amd64/sys/tfork_thread.S:87
#24 0x00000dbe549e506a in __tfork_thread ()
    at /usr/src/lib/libc/arch/amd64/sys/tfork_thread.S:87
Previous frame identical to this frame (corrupt stack?)

( gdb ) thread apply all bt
( gdb ) bt :

Thread 8 (thread 458390):
#0  _thread_sys_sendto () at /tmp/-:2
#1  0x250b5859985ae367 in ?? ()
#2  0x00000dbe54a08892 in _libc_sendto_cancel (fd=Variable "fd" is not available.
)
    at /usr/src/lib/libc/sys/w_sendto.c:28
#3  0x00000dbbbe324bdd in td::detail::SocketFdImpl::write ()
   from /usr/local/bin/nchat
#4  0x00000dbbbe324b37 in td::SocketFd::write () from /usr/local/bin/nchat
#5  0x00000dbbbe326fbf in td::detail::EventFdBsd::release ()
   from /usr/local/bin/nchat
#6  0x00000dbbbe313d76 in td::Scheduler::send_to_other_scheduler ()
   from /usr/local/bin/nchat
#7  0x00000dbbbcc6a1fc in _ZN2td9Scheduler21send_immediately_implIZNS0_24send_closure_immediatelyINS_16ImmediateClosureINS_19FileDownloadManagerEMS4_FvyRKNS_22FullRemoteFileLocationERKNS_17LocalFileLocationExNSt3__112basic_stringIcNSB_11char_traitsIcEENSB_9allocatorIcEEEERKNS_17FileEncryptionKeyEbxxaEJRyRS5_RS8_RxOSH_RSI_RbSQ_SQ_RaEEEEEvNS_8ActorRefEOT_EUlPNS_9ActorInfoEE_ZNS2_ISV_EEvSW_SY_EUlvE_EEvRKNS_7ActorIdINS_5ActorEEERKSX_RKT0_ () from /usr/local/bin/nchat
#8  0x00000dbbbcc3b1f6 in _ZN2td12send_closureIRNS_8ActorOwnINS_19FileDownloadManagerEEEMS2_FvyRKNS_22FullRemoteFileLocationERKNS_17LocalFileLocationExNSt3__112basic_stringIcNSB_11char_traitsIcEENSB_9allocatorIcEEEERKNS_17FileEncryptionKeyEbxxaEJRyRS5_RS8_RxSH_RSI_RbSQ_SQ_RaEEEvOT_T0_DpOT1_ ()
---Type <return> to continue, or q <return> to quit---
   from /usr/local/bin/nchat
#9  0x00000dbbbcc31302 in td::FileManager::run_download ()
   from /usr/local/bin/nchat
#10 0x00000dbbbcc3a119 in td::FileManager::download_impl ()
   from /usr/local/bin/nchat
#11 0x00000dbbbcc38d90 in td::FileManager::download ()
   from /usr/local/bin/nchat
#12 0x00000dbbbcc38361 in td::FileManager::download_file ()
   from /usr/local/bin/nchat
#13 0x00000dbbbd89cf39 in _ZZN2td8Requests11run_requestEyONS_2tl10unique_ptrINS_6td_api8FunctionEEEENK3$_0clINS3_12downloadFileEEEDaRT_ ()
   from /usr/local/bin/nchat
#14 0x00000dbbbd834466 in td::Requests::run_request ()
   from /usr/local/bin/nchat
#15 0x00000dbbbd7c10d9 in td::Td::request () from /usr/local/bin/nchat
#16 0x00000dbbbcc0eb1c in _ZN2td9Scheduler21send_immediately_implIZNS0_24send_closure_immediatelyINS_16ImmediateClosureINS_2TdEMS4_FvyNS_2tl10unique_ptrINS_6td_api8FunctionEEEEJRyOS9_EEEEEvNS_8ActorRefEOT_EUlPNS_9ActorInfoEE_ZNS2_ISE_EEvSF_SH_EUlvE_EEvRKNS_7ActorIdINS_5ActorEEERKSG_RKT0_ () from /usr/local/bin/nchat
#17 0x00000dbbbcc0d943 in td::MultiTd::send () from /usr/local/bin/nchat
#18 0x00000dbbbe313532 in td::Scheduler::do_event () from /usr/local/bin/nchat
#19 0x00000dbbbcc0e91e in _ZN2td9Scheduler21send_immediately_implIZNS0_16send_immediatelyENS_8ActorRefEONS_5EventEEUlPNS_9ActorInfoEE_ZNS0_16send_immediatelyES2---Type <return> to continue, or q <return> to quit---
_S4_EUlvE_EEvRKNS_7ActorIdINS_5ActorEEERKT_RKT0_ () from /usr/local/bin/nchat
#20 0x00000dbbbe311758 in td::Scheduler::ServiceActor::loop ()
   from /usr/local/bin/nchat
#21 0x00000dbbbe313518 in td::Scheduler::do_event () from /usr/local/bin/nchat
#22 0x00000dbbbe3151b3 in td::Scheduler::flush_mailbox ()
   from /usr/local/bin/nchat
#23 0x00000dbbbe315417 in td::Scheduler::run_mailbox ()
   from /usr/local/bin/nchat
#24 0x00000dbbbe315828 in td::Scheduler::run_events ()
   from /usr/local/bin/nchat
#25 0x00000dbbbe315969 in td::Scheduler::run_no_guard ()
   from /usr/local/bin/nchat
#26 0x00000dbbbe30fa81 in td::ConcurrentScheduler::run_main ()
   from /usr/local/bin/nchat
#27 0x00000dbbbcc11cb9 in _ZN2td16LambdaDestructorIZNS_6detail13ThreadPthreadC1IZNS_9MultiImplC1ENSt3__110shared_ptrINS_13NetQueryStatsEEEEUlvE_JEEEOT_DpOT0_EUlvE_ED0Ev () from /usr/local/bin/nchat
#28 0x00000dbbbcc11bc5 in td::detail::ThreadPthread::run_thread ()
   from /usr/local/bin/nchat
#29 0x00000dbe0d73f7b2 in _rthread_start (v=Unhandled dwarf expression opcode 0xf3
)
    at /usr/src/lib/librthread/rthread.c:99
#30 0x00000dbe549e506a in __tfork_thread ()
    at /usr/src/lib/libc/arch/amd64/sys/tfork_thread.S:87
---Type <return> to continue, or q <return> to quit---
#31 0x00000dbe549e506a in __tfork_thread ()
    at /usr/src/lib/libc/arch/amd64/sys/tfork_thread.S:87
Previous frame identical to this frame (corrupt stack?)
2       in /tmp/-

TDLIB version : a8f21f5

Reported by : qypeqoiq

Reproduced on : OpenBSD ( amd64 / arm64 )

[levlam]

Thank you. I'll take a look.

[levlam]

This is fixed in the up-to-date TDLib version.