Open
Conversation
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
9 times, most recently
from
29f47f3 to
264aa7f
Compare
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
3 times, most recently
from
a30cbfb to
1e7bdbc
Compare
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
5 times, most recently
from
b5e980f to
67d47c2
Compare
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
2 times, most recently
from
95285b0 to
4b2d345
Compare
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
2 times, most recently
from
cba29a3 to
db1fe83
Compare
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
from
db1fe83 to
35f9ec2
Compare
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
3 times, most recently
from
e89ada5 to
3abd0a5
Compare
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
from
3abd0a5 to
4a75d67
Compare
Resolve Node.js 20 deprecation warnings by updating all GitHub Actions to their latest major versions supporting Node.js 24. Co-Authored-By: HAL 9000
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
3 times, most recently
from
5647237 to
04855d7
Compare
The parallel `JoinSet`-based batching loop was duplicated across `read_payments` and `read_pending_payments`. Extract it into a generic `read_all_objects<T: Readable>` helper that callers invoke directly with the relevant namespace constants. Per-type log messages are preserved via `std::any::type_name::<T>()`. Co-Authored-By: HAL 9000
Bump test dependencies, reorganize Docker files, and add interop integration test infrastructure with shared scenario runner for CLN, LND, and Eclair.
Update to use the new HeaderCache type instead of implementing the Cache trait, pass BestBlock instead of BlockHash to synchronize_listeners, and pass HeaderCache by value to SpvClient::new. Also adapt to BestBlock gaining a previous_blocks field and ChannelManager deserialization returning BestBlock instead of BlockHash. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
UniFFI cannot represent the fixed-size array that upstream's BestBlock carries via `previous_blocks`, so NodeStatus.current_best_block was unusable from Swift, Kotlin, and Python once upstream added that field. Introduce a small ldk-node BestBlock with just hash and height — the pieces bindings can handle — and expose it in place of the upstream type on the public API. Generated with assistance from Claude Code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DRY up batched `KVStore` reads utility methods
Add interop integration test harness for LND, CLN, and Eclair
…-api Adapt to `lightning-block-sync` API changes
…o-node-status Add a network field to NodeStatus
The max_inbound_htlc_value_in_flight_percent_of_channel config setting was used when acting as an LSPS2 service in order to forward the initial payment. However, upstream divided the config setting into two for announced and unannounced channels, the latter defaulting to 100%.
…-flight Drop `max_inbound_htlc_value_in_flight_percent_of_channel `
Note that we still don't expect to receive multiple outgoing HTLCs because trampoline has not yet been enabled, but we lay the groundwork here.
Update rust-lightning to use default_value_vec
This matches the logging done when setting up a VSS store.
In the mutual close case, before moving to check the expected balances of each node, we now assert that the mutual close transaction actually made it into a block. If the mutual close transaction got rejected for some reason, we now stop the test right there and fail instead of continuing onto balance checks.
…ll-request/patch Automated nightly rustfmt (2026-05-03)
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
from
dec4aa7 to
421f0cb
Compare
…le-mutual-close-coverage Make sure the mutual close gets confirmed in `do_channel_full_cycle`
.. we were erroneously logging the `NODE_METRICS` namespaces.
This internal method allows us to avoid instantiating the logger twice during the creation of a `Node`.
…persistence-namespaces Fix logged namespaces in case of persistence failure
…testore-setup-err Log the error returned from `SqliteStore::new` and `fs::create_dir_all`
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
from
421f0cb to
c1563e3
Compare
This commit: Adds `TierStore`, a tiered `KVStore`/`KVStoreSync` implementation that routes node persistence across three storage roles: - a primary store for durable, authoritative data - an optional backup store for a second durable copy of primary-backed data - an optional ephemeral store for rebuildable cached data such as the network graph and scorer TierStore routes ephemeral cache data to the ephemeral store when configured, while durable data remains primary+backup. Reads and lists do not consult the backup store during normal operation. For primary+backup writes and removals, this implementation treats the backup store as part of the persistence success path rather than as a best-effort background mirror. Earlier designs used asynchronous backup queueing to avoid blocking the primary path, but that weakens the durability contract by allowing primary success to be reported before backup persistence has completed. TierStore now issues primary and backup operations together and only returns success once both complete. This gives callers a clearer persistence guarantee when a backup store is configured: acknowledged primary+backup mutations have been attempted against both durable stores. The tradeoff is that dual-store operations are not atomic across stores, so an error may still be returned after one store has already been updated. TierStore also implements `KVStoreSync` in terms of dedicated synchronous helpers that call the wrapped stores' sync interfaces directly. This preserves the inner stores' synchronous semantics instead of routing sync operations through a previously held async runtime. Additionally, adds unit coverage for the current contract, including: - basic read/write/remove/list persistence - routing of ephemeral data away from the primary store - backup participation in the foreground success path for writes and removals
Add native builder support for tiered storage by introducing `TierStoreConfig` and builder methods for configuring backup and ephemeral stores. During node construction, wrap the configured primary store in `TierStore` and attach any configured secondary tiers: ephemeral storage for cache-like data and backup storage for mirrored durable writes.
Refactor backup storage to local SQLite Replaces the builder's BYO backup-store configuration with a path-based local SQLite backup mirror. The builder now constructs the backup store internally using a dedicated backup database file name and rejects configurations where the backup path conflicts with the primary storage path. Also adds test coverage for full-cycle backup mirroring and same-path rejection, as well as a `setup_node_with_builder` test helper to allow builder customization in integration tests.
enigbe
force-pushed
the
2025-10-tiered-data-storage
branch
from
c1563e3 to
a2458e4
Compare
- Make setup_builder! use a mutable binding for Builder under uniffi to preserve test helper compatibility for the FFI-backed builder - Add ArcedNodeBuilder forwarding methods set_backup_storage_dir_path and set_ephemeral_store Co-authored-by: Copilot <copilot@github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does:
We introduce
TierStore, aKVStoreimplementation that manages data acrossthree distinct storage layers.
The layers are:
(e.g., network graph). This tier aims to improve latency by leveraging a
local
KVStoredesigned for fast/local access.asynchronously/lazily to avoid blocking primary store operations.
We also permit the configuration of
Nodewith these stores allowingcallers to set exponential back-off parameters, as well as backup and ephemeral
stores, and to build the
Nodewith TierStore's primary store. These configurationoptions also extend to our foreign interface, allowing bindings target to build the
Node with their own
ffi::KVStoreimplementations.A sample Python implementation is added and tested.
Additionally, we add comprehensive testing for
TierStoreby introducingTierStorecore functionality.Nodebuilt with tiered storage.ffi::KVStoreimplementations.Concerns
It is worth considering the way retry logic is handled, especially because of nested
retries.
TierStorecomes with a basic one by default but there areKVStoreimplementationsthat come with them baked-in (e.g.
VssStore), and thus would have no need forthe wrapper-store's own logic.