You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Apr 1, 2023. It is now read-only.
It seems the heartbeat for a txn is not cancelled when the txn is dropped in the HTTPProxy, causing the hb timer to attempt to use the RPCDispatcher to send a message. The dispatcher has shutdown at this point and asserts that it should not be used, e.g.
[0004:14:18:51.953.116]-http_proxy-[0]-(k2::skv_client) [DEBUG] [/build/src/k2/module/k23si/client/k23si_client.cpp:66 @operator()] send hb for mtr={timestamp={endCount=397131397989158, tsoId=100000, startDelta=100000}, priority=medium}
[0004:14:18:51.953.201]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:88 @partitionRequest] making partition request with deadline=0000:00:00:00.499.993
[0004:14:18:52.009.280]-http_proxy-[0]-(k2::transport) [ERROR] [/build/src/k2/transport/RPCDispatcher.h:189 @operator()] RPC send failed with uncaught exception: dispatcher has shut down
[0004:14:18:52.009.353]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:360 @operator()] partition call completed with status={code=500, message=unknown exception while sending request}
[0004:14:18:52.009.405]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:105 @operator()] call failed with retryable status: {code=503, message=unable to make call at this time}
[0004:14:18:52.009.423]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:110 @operator()] refreshing collection from CPO after status={code=503, message=unable to make call at this time}
[0004:14:18:52.010.636]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:247 @_getAssignedPartitionWithRetry] time remaining=0000:00:00:00.442.563, for coll=HTTPClient, key={schemaName=test_schema, partitionKey=^00^00^01, rangeKey=^00^00^01}, reverse=false, excludedKey=false, retries=1
[0004:14:18:52.013.215]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:273 @_getAssignedPartitionWithRetry] no existing waiter for name=HTTPClient. Creating new one
http_proxy: /usr/local/include/seastar/core/sharded.hh:755: Service& seastar::sharded::local() [with Service = k2::RPCDispatcher]: Assertion `local_is_initialized()' failed.
It seems the heartbeat for a txn is not cancelled when the txn is dropped in the HTTPProxy, causing the hb timer to attempt to use the RPCDispatcher to send a message. The dispatcher has shutdown at this point and asserts that it should not be used, e.g.
[0004:14:18:51.953.116]-http_proxy-[0]-(k2::skv_client) [DEBUG] [/build/src/k2/module/k23si/client/k23si_client.cpp:66 @operator()] send hb for mtr={timestamp={endCount=397131397989158, tsoId=100000, startDelta=100000}, priority=medium}
[0004:14:18:51.953.201]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:88 @partitionRequest] making partition request with deadline=0000:00:00:00.499.993
[0004:14:18:52.009.280]-http_proxy-[0]-(k2::transport) [ERROR] [/build/src/k2/transport/RPCDispatcher.h:189 @operator()] RPC send failed with uncaught exception: dispatcher has shut down
[0004:14:18:52.009.353]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:360 @operator()] partition call completed with status={code=500, message=unknown exception while sending request}
[0004:14:18:52.009.405]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:105 @operator()] call failed with retryable status: {code=503, message=unable to make call at this time}
[0004:14:18:52.009.423]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:110 @operator()] refreshing collection from CPO after status={code=503, message=unable to make call at this time}
[0004:14:18:52.010.636]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:247 @_getAssignedPartitionWithRetry] time remaining=0000:00:00:00.442.563, for coll=HTTPClient, key={schemaName=test_schema, partitionKey=^00^00^01, rangeKey=^00^00^01}, reverse=false, excludedKey=false, retries=1
[0004:14:18:52.013.215]-http_proxy-[0]-(k2::cpo_client) [DEBUG] [/build/src/k2/cpo/client/Client.h:273 @_getAssignedPartitionWithRetry] no existing waiter for name=HTTPClient. Creating new one
http_proxy: /usr/local/include/seastar/core/sharded.hh:755: Service& seastar::sharded::local() [with Service = k2::RPCDispatcher]: Assertion `local_is_initialized()' failed.