Skip to content

feat: destructive reconciliation - #1822

Open
isekovanic wants to merge 14 commits into
release-v10from
feat/destructive-reconciliation
Open

feat: destructive reconciliation#1822
isekovanic wants to merge 14 commits into
release-v10from
feat/destructive-reconciliation

Conversation

@isekovanic

Copy link
Copy Markdown
Contributor

CLA

  • I have signed the Stream CLA (required).
  • Code changes are tested

Description of the changes, What, Why and How?

Adds destructive reconciliation of the channel message list. On reconnect/rehydrate, messages that were hard-deleted by anyone while the client was offline are removed from the list, for the channel main list and thread replies, with offline support enabled or disabled.

A hard delete emits no event to other clients, and the newest page merge is additive (it never removes), so a message deleted while you were offline lingers as a ghost forever after reconnect and with offline support on, it also survives in SQLite and returns on cold start.

Reconciliation is folded into the existing newest page merge, so every reseed path (channel.query, channel list hydrate, channel.reload, Thread.reload, recoverState) reconciles uniformly, without a full reload and without dropping already-loaded older pages. A loaded message that's absent from the authoritative page is pruned; a prefetch snapshot distinguishes a genuine hard-delete from a message that arrived live during the fetch, and local (optimistic/failed/error) messages are always kept. Reconciled ids are mirrored into the offline DB so a cold start doesn't reseed them. I've also added a billion unit tests that should capture this behaviour because it's an incredible pain to test all of the nooks and crannies here.

Changelog

  • Feature: hard deletes performed by other users while you were offline are now removed from the message list (and threads) on reconnect/re-hydrate instead of lingering as ghosts, so no full reload and mirrored into the offline DB so they don't return on cold start.

Comment thread src/channel.ts Outdated
Comment thread src/channel.ts Outdated
Comment thread src/channel.ts
Comment thread src/channel.ts
Comment thread src/channel.ts
const interval = this.ingestPage({ page, isHead: true, setActive: false });
if (!interval) return;

// Re-compute hasMoreTail from the FETCHED PAGE, not the merged interval. The interval's isTail is

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we miss tests for this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, why do you have that impression ?

const { requestedLimit } = options ?? {};
const reachedChannelStart =
typeof requestedLimit === 'number' &&
page.length < Math.min(requestedLimit, DEFAULT_CHANNEL_MESSAGE_LIST_PAGE_SIZE);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hard-coded channel value would be applied to Threads as well? I am wondering whether it is possible to take the configured value on the paginator instance.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would and it needs to apply to those as well. The point is not page size (100 is anyway too much for a page size at least for mobile), but rather the maximum BE can return. I'll try resolving this a different way and make sure pagination doesn't break. But in general we need to fetch as many as we can as there is no other way to reconcile updates without ditching the entire list altogether.

Comment thread src/pagination/paginators/MessageIntervalPaginator.ts
Comment thread src/pagination/paginators/MessageIntervalPaginator.ts Outdated
Comment thread src/pagination/paginators/MessageIntervalPaginator.ts Outdated
Comment thread src/pagination/paginators/MessageIntervalPaginator.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants