feat: Securejoin v3, encrypt all securejoin messages #7754

Hocuri · 2026-01-20T13:36:07Z

Close #7396. Before reviewing, you should read the issue description of #7396.
I recommend to review with hidden whitespace changes.

TODO:

Backwards compatibility:
Everything works seamlessly in my tests. If both devices are updated, then the new protocol is used; otherwise, the old protocol is used. If there is a not-yet-updated second device, it will correctly observe the protocol, and mark the chat partner as verified.

Note that I removed the Auto-Submitted: auto-replied header from securejoin messages. We don't need it ourselves, it's a cleartext header that leaks too much information, and I can't see any reason to have it.

src/mimeparser.rs

Hocuri · 2026-01-26T17:26:47Z

src/mimeparser.rs

                Ok(secret)
            })
            .await?;
+        secrets.extend(token::lookup_all(context, token::Namespace::Auth).await?);


There was the idea to use a new table rather than reusing the AUTH token.

I don't think this would be worth it:

It would increase complexity, because in decode_openpgp() and in "step 5+6" in handle_securejoin_handshake(), we would need to lookup tokens from both the old and new table, in order to keep backwards compatibility, in case someone scans a QR code that was generated before the upgrade, because in that case, the AUTH token would only be in the tokens table and not in the new table.

It would be another table to deprecate eventually. As it stands, we can just one day remove all the invitenumbers from tokens, and ignore the namespc column

All in all, we would need to juggle two tables rather than one.

...but it wouldn't be too much effort to switch to a new table, either.

…rt() and set_content_type_to_encrypted() functions

…ot to work yet)

Previously, AUTH tokens were only synchronized if an INVITE token was generated at the same time. This led to several user-visible bugs. Now, we always send a sync message after generating a new AUTH token. Since the INVITE token may have stayed the same, I added a UNIQUE bound to the `tokens` table, in order to prevent saving the same INVITE token many times.

I don't need them anymore, it seems unlikely they will be needed soon, and they made it hard to implement a performance improvement I'm planning

Hocuri · 2026-02-10T12:34:15Z

Benchmarking results:

Details

Loading 5000 auth tokens (a reeeally high number) slows down receiving a pk-encrypted message by ~7%

➜  deltachat-core-rust git:(hoc/securejoin-v3) ✗ NUM_BROADCAST_SECRETS=499 NUM_AUTH_TOKENS=2 c bench --bench decrypting --features="internals" -- 'Decrypt and parse';pling
firejail version 0.9.78

Parent pid 167445, child pid 167446
Base filesystem installed in 0.09 ms
Child process initialized in 19.40 ms
    Finished `bench` profile [optimized] target(s) in 0.30s
     Running benches/decrypting.rs (target/release/deps/decrypting-92a1aefcc2bf4270)
NUM_BROADCAST_SECRETS=499
NUM_AUTH_TOKENS=2
Decrypt/Decrypt and parse a symmetrically encrypted message
                        time:   [11.192 ms 11.198 ms 11.206 ms]
                        change: [−10.279% −10.136% −9.9857%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 10 measurements (20.00%)
  1 (10.00%) low mild
  1 (10.00%) high mild
Decrypt/Decrypt and parse a public-key encrypted message
                        time:   [12.466 ms 12.469 ms 12.475 ms]
                        change: [−7.0396% −6.9896% −6.9373%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild


Parent is shutting down, bye...
➜  deltachat-core-rust git:(hoc/securejoin-v3) ✗ NUM_BROADCAST_SECRETS=499 NUM_AUTH_TOKENS=4999 c bench --bench decrypting --features="internals" -- 'Decrypt and parse';pling
firejail version 0.9.78

Parent pid 167569, child pid 167570
Base filesystem installed in 0.11 ms
Child process initialized in 20.70 ms
    Finished `bench` profile [optimized] target(s) in 0.30s
     Running benches/decrypting.rs (target/release/deps/decrypting-92a1aefcc2bf4270)
NUM_BROADCAST_SECRETS=499
NUM_AUTH_TOKENS=4999
Decrypt/Decrypt and parse a symmetrically encrypted message
                        time:   [12.435 ms 12.439 ms 12.444 ms]
                        change: [+10.951% +11.106% +11.267%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
Decrypt/Decrypt and parse a public-key encrypted message
                        time:   [13.384 ms 13.395 ms 13.408 ms]
                        change: [+7.3096% +7.4043% +7.5107%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild


Parent is shutting down, bye...

We don't need to load the secrets for messages that are asymmetrically encrypted see dbd6360.

Replacing ORDER BY id DESC with calling rev() in Rust doesn't improve performance:

NUM_BROADCAST_SECRETS=500
NUM_AUTH_TOKENS=5000
Decrypt/Decrypt and parse a symmetrically encrypted message
                        time:   [12.420 ms 12.432 ms 12.446 ms]
                        change: [+0.4867% +0.5933% +0.7196%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe
Decrypt/Decrypt and parse a public-key encrypted message
                        time:   [12.263 ms 12.280 ms 12.302 ms]
                        change: [−0.1650% +0.0110% +0.2077%] (p = 0.92 > 0.05)
                        No change in performance detected.
Found 16 outliers among 100 measurements (16.00%)
  4 (4.00%) high mild
  12 (12.00%) high severe

This improves performance, though not a lot - 8% performance improvement if there is a huge number of shared secrets. Anyways, I think it's worth it. ``` NUM_BROADCAST_SECRETS=500 NUM_AUTH_TOKENS=5000 Decrypt/Decrypt and parse a symmetrically encrypted message time: [12.354 ms 12.364 ms 12.376 ms] change: [−0.5375% −0.3815% −0.2345%] (p = 0.00 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 3 (3.00%) high mild 4 (4.00%) high severe Decrypt/Decrypt and parse a public-key encrypted message time: [12.269 ms 12.273 ms 12.278 ms] change: [−7.9163% −7.8439% −7.7724%] (p = 0.00 < 0.05) Performance has improved. Found 13 outliers among 100 measurements (13.00%) 8 (8.00%) high mild 5 (5.00%) high severe ```

…hared_secrets()

…n v3 When we finally get rid of legacy Securejoin, we can just leave them out, then.

…o invitenumber

…succeeds with 1:1 chats and broadcasts

…her we know this QR code

…ages

Hocuri · 2026-02-10T18:31:07Z

benches/decrypting.rs

 //! ```
 //!
-//! or, if you want to only run e.g. the 'Decrypt a symmetrically encrypted message' benchmark:
+//! or, if you want to only run e.g. the 'Decrypt and parse a symmetrically encrypted message' benchmark:


I removed the "Decrypt a symmetrically/public-key encrypted message" benchmarks, because I don't need them anymore, it seems unlikely they will be needed soon, and they made it hard to implement the performance improvement in 61ae743

Hocuri · 2026-02-10T18:33:26Z

src/chat/chat_tests.rs


        let request = charlie.pop_sent_msg().await;
-        assert_eq!(request.recipients, "alice@example.org charlie@example.net");
+        assert_eq!(request.recipients, "alice@example.org");


The first two messages of the protocol aren't sent to self anymore, because in many cases, one's own other devices wouldn't be able to decrypt them, anyways.

Hocuri · 2026-02-10T20:06:46Z

src/mimefactory.rs

+                    .into(),
+            ));
+        }
+


I moved this code up, in order to be able to extract the group_headers_by_confidentiality() function.

Hocuri · 2026-02-10T20:44:17Z

src/message.rs

+/// to prevent downloading the same message in the future.
+///
+/// Returns tombstone database row ID.
+pub(crate) async fn insert_tombstone(context: &Context, rfc724_mid: &str) -> Result<MsgId> {


I moved this function from receive_imf.rs to here, so that it can be used from the securejoin code

Hocuri · 2026-02-10T20:45:22Z

src/securejoin.rs

+        context
+            .sync_qr_code_tokens(Some(chat.grpid.as_str()))
+            .await?;
+        context.scheduler.interrupt_smtp().await;


This actually fixes an existing bug:

We now generate a new AUTH token every time a QR code is shown. Therefore, we always need to sync it.

Hocuri changed the title ~~[WIP] Securejoin v3: Encrypt all securejoin messages~~ [WIP] feat: Securejoin v3, encrypt all securejoin messages Jan 20, 2026

Hocuri force-pushed the hoc/securejoin-v3 branch 2 times, most recently from 59957ed to ef97a7e Compare January 21, 2026 15:44

Hocuri commented Jan 22, 2026

View reviewed changes

src/mimeparser.rs Outdated Show resolved Hide resolved

Hocuri force-pushed the hoc/securejoin-v3 branch from 807b8c4 to afb7e2a Compare January 22, 2026 17:06

Hocuri commented Jan 26, 2026

View reviewed changes

Hocuri force-pushed the hoc/securejoin-v3 branch from afb7e2a to 81f79a9 Compare January 26, 2026 18:18

Hocuri added 19 commits January 27, 2026 12:31

Send v=3 in QR codes

c0b86f5

[wip, not compiling] Add render_symm_encrypted_securejoin_message()

ad584d2

[wip, not compiling] extract function group_headers_by_confidentiality()

3b66611

[wip, doesn't compile] refactor: exctract add_headers_to_encrypted_pa…

2025e9b

…rt() and set_content_type_to_encrypted() functions

[wip, doesn't compile] fix: Set subject for securejoin message

2eba552

[wip, doesn't compile] Extract render_outer_message() functions

2566c5b

It compiles

1014bf1

feat: Decrypt and answer on Alice's side (sending the message seems n…

c91b349

…ot to work yet)

Fix some bugs when sending vc-pubkey, decrypt vc-pubkey on Bob's side

4c00bc1

Handle vc-pubkey on Bob's side

d979938

Adapt some tests

c9bd5d0

fix: Don't leak cryptographic identity by signing vc-request-pubkey

c3d4c43

fix: Don't limit number of auth tokens to 1

7ff5cd6

modify test

297c6d8

Update golden tests

2358c9d

Make the remaining Rust tests pass

90b0f94

Update src/mimeparser.rs

aceb75b

Add failing test test_auth_token_is_synchronized()

8ee3c2a

Hocuri force-pushed the hoc/securejoin-v3 branch from 81f79a9 to 6b658a0 Compare January 30, 2026 16:46

Hocuri added 3 commits February 10, 2026 10:52

Merge remote-tracking branch 'origin/main' into hoc/securejoin-v3

db7e667

bench: Also add auth tokens into benchmark

20ef54b

bench: Add black_box

50eb923

bench: Remove benchmarks that only decrypt without parsing

1b25853

I don't need them anymore, it seems unlikely they will be needed soon, and they made it hard to implement a performance improvement I'm planning

Hocuri force-pushed the hoc/securejoin-v3 branch from db0d9d9 to dbd6360 Compare February 10, 2026 12:32

Hocuri force-pushed the hoc/securejoin-v3 branch from dbd6360 to 61ae743 Compare February 10, 2026 13:56

Hocuri added 5 commits February 10, 2026 15:13

perf: Load BobState shared secrets first; refactor: extract fn load_s…

7a81411

…hared_secrets()

Add explanatory performance comment

a442077

Remove TODO

cca6bd4

feat: Make it possible to omit the invitenumber and v=3 for Securejoi…

642ba1d

…n v3 When we finally get rid of legacy Securejoin, we can just leave them out, then.

test: Test that securejoin v3 runs even without v=3 when there is n…

b5ae93c

…o invitenumber

Hocuri force-pushed the hoc/securejoin-v3 branch from 1832269 to b5ae93c Compare February 10, 2026 16:00

Hocuri added 3 commits February 10, 2026 17:15

test: Test that Securejoin without v=3 and without INVITENUMBER also …

80db501

…succeeds with 1:1 chats and broadcasts

fix: For future-compatibility, don't rely on Invite for checking whet…

81d9fa7

…her we know this QR code

fix: Don't save empty tokens

e773468

Hocuri force-pushed the hoc/securejoin-v3 branch from be7c5da to 55d6919 Compare February 10, 2026 18:36

Hocuri added 4 commits February 10, 2026 19:47

Improve comment

0e359e9

feat: Remove Auto-Submitted: auto-replied header from securejoin mess…

46c9fc3

…ages

test: Improve test

10933cc

Some more comments improvements

fd61423

Hocuri force-pushed the hoc/securejoin-v3 branch from 55d6919 to fd61423 Compare February 10, 2026 19:21

Hocuri commented Feb 10, 2026

View reviewed changes

More comments and renamings

0d47a52

Hocuri changed the title ~~[WIP] feat: Securejoin v3, encrypt all securejoin messages~~ feat: Securejoin v3, encrypt all securejoin messages Feb 10, 2026

Hocuri requested review from iequidoo and link2xt February 10, 2026 20:52

Merge remote-tracking branch 'origin/main' into hoc/securejoin-v3

3b0f798

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Securejoin v3, encrypt all securejoin messages #7754

feat: Securejoin v3, encrypt all securejoin messages #7754

Uh oh!

Hocuri commented Jan 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Hocuri Jan 26, 2026 •

edited

Loading

Uh oh!

Hocuri commented Feb 10, 2026 •

edited

Loading

Uh oh!

Hocuri Feb 10, 2026

Uh oh!

Hocuri Feb 10, 2026

Uh oh!

Hocuri Feb 10, 2026

Uh oh!

Hocuri Feb 10, 2026

Uh oh!

Hocuri Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

feat: Securejoin v3, encrypt all securejoin messages #7754

Are you sure you want to change the base?

feat: Securejoin v3, encrypt all securejoin messages #7754

Uh oh!

Conversation

Hocuri commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Hocuri Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hocuri commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Hocuri Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Hocuri Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Hocuri Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Hocuri Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Hocuri Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Hocuri commented Jan 20, 2026 •

edited

Loading

Hocuri Jan 26, 2026 •

edited

Loading

Hocuri commented Feb 10, 2026 •

edited

Loading