xpay: add a flag to indicate completion of initialization

[Lagrang3]

Add a "ready" flag that becomes true after the plugin has finished initializing. Usually RPC calls during initialization are done using synchronous communication but xpay uses hooks to bind itself to "pay". For some reason registering to hooks and using rcp_scan at init cannot be done. Therefore xpay's initialization is asynchronous which has the downside of race conditions like: trying to make a payment while xpay does not know yet the current node id. This is unlikely to happen in real life but it breaks our tests randomly.

Fixes flake tests/test_xpay.py:test_xpay_fake_channeld

Valgrind error file: valgrind-errors.356233
==356233== Conditional jump or move depends on uninitialised value(s)
==356233==    at 0x1116ED: handle_block_added (xpay.c:3159)
==356233==    by 0x11D7A8: ld_command_handle (libplugin.c:2144)
==356233==    by 0x11DB8A: ld_read_json (libplugin.c:2282)
==356233==    by 0x15DAC1: next_plan (io.c:60)
==356233==    by 0x15DF4C: do_plan (io.c:422)
==356233==    by 0x15E005: io_ready (io.c:439)
==356233==    by 0x15F99B: io_loop (poll.c:470)
==356233==    by 0x11DFD6: plugin_main (libplugin.c:2481)
==356233==    by 0x11876E: main (xpay.c:3412)
==356233==
{
   <insert_a_suppression_name_here>
   Memcheck:Cond
   fun:handle_block_added
   fun:ld_command_handle
   fun:ld_read_json
   fun:next_plan
   fun:do_plan
   fun:io_ready
   fun:io_loop
   fun:plugin_main
   fun:main
}

Fixes #9248

[Lagrang3]

I am not sure we should do this. We have never done this pattern before AFAIK.
Another workaround could be to wait until askrene has the xpay layer, or maybe do a lazy initialization (ie. don't initialize at init but instead initialize if we receive a request and we haven't initialized yet).
Or we could fix lightningd so that plugins that use command hooks can also use rpc_scan.

[Lagrang3]

Another way to solve this is to remove pay's backend and implement a json_pay into xpay.c that simply uses xpay
so that we can remove the hook. pay is in any case deprecated and we keep our promise to support the API.

[ddustin]

Rebasing on master to see if CI is getting further

[ddustin]

This looks like a net improvement 👍. Waiting for CI to pass before ack'ing though.

Looking forward, it would be great eventually to get this plugin operating synchronously. In other plugins the approach is to chain the needed RPC commands together, one at a time. Each _done RPC command chains off to the next one. This ensures things are synchronous and keeps the event chain cleaner / easier to understand.

For example, in the splice plugin: https://github.com/ElementsProject/lightning/blob/d573b58c65f5a275f23c77681d8b44b9ca182707/plugins/spender/splice.c

We have a call chain like so: continue_splice -> onchain_wallet_fund -> addpsbt_get_result -> continue_splice -> ....

Or another example in multi open:
https://github.com/ElementsProject/lightning/blob/d573b58c65f5a275f23c77681d8b44b9ca182707/plugins/spender/multifundchannel.c

perform_fundchannel_complete -> fundchannel_complete_ok -> fundchannel_complete_done -> after_fundchannel_complete -> perform_signpsbt -> ...

Perhaps xpay would benefit from a restructuring like this down the road