Switching organizations breaks Olm hole punching for the same client installation

[Serph91P]

Describe the Bug

This was originally reported as fosrl/pangolin#3455. A maintainer asked for it to be moved to the Android repository.

Switching organizations in the Android client can leave Olm registration broken for both organizations.

The same Android installation is registered as a separate client in two organizations. One organization initially connected successfully while the other consistently failed with HOLEPUNCH_MISSING. After switching back and forth, the organization that had worked before also stopped connecting.

The app reaches Pangolin over HTTPS and WebSocket, but registration is rejected with Client last hole punch is too old. This happens even though UDP traffic from the client reaches Gerbil on port 21820.

The two organization-specific client records share the same olmId. They have different clientId values and different orgId values. In the affected state, one client has a valid lastHolePunch, while the other remains NULL.

The global olms row can only reference one clientId for that olmId. Switching organizations changes this mapping. This may be relevant because updateHolePunch.ts resolves the Olm record by olmId and updates clients.lastHolePunch through olm.clientId. The Gerbil update payload does not contain an orgId.

Machine clients running in separate Docker containers continue to work. Each of those installations has its own credentials and a fixed organization. The issue has only been reproduced with clients that switch the same Olm installation between organizations.

Environment

• Server OS: Ubuntu 24.04.4 LTS
• Docker Engine: 29.4.1
• Docker Compose: 5.1.3
• Pangolin Version: 1.20.0
• Edition: Enterprise
• Gerbil Version: 1.4.2
• Traefik Version: 3.7.8
• Newt: Multiple online sites using both Docker and binary deployments
• Android Client Version: 0.32.0
• Android Device: Pixel 10 Pro XL
• Windows Client: Current installed version, same HOLEPUNCH_MISSING symptom
• Database: SQLite
• Docker mode: Rootful

The device clock and timezone are set automatically. The Android app is current and has also been reinstalled during testing.

To Reproduce

1. Create two organizations on the same Pangolin instance.
2. Add different sites and private resources to each organization.
3. Sign in to the Android client and connect to Organization A.
4. Switch to Organization B in the client.
5. Attempt to connect to Organization B.
6. Switch back to Organization A and attempt to connect again.

Observed sequence in this environment:

1. Organization A connected successfully after a full Pangolin stack restart.
2. Organization B failed with HOLEPUNCH_MISSING.
3. Switching back to Organization A caused Organization A to fail as well.
4. Repeated attempts and an app reinstall did not restore reliable switching.

Expected Behavior

The client should be able to switch between organizations and connect to the sites assigned in the selected organization. Each organization-specific client record should receive the current hole punch state for the active organization.

Actual Behavior

The client remains on Registering and eventually shows:

Unable to coordinate client P2P connection.
Please ensure your client can reach the server on UDP port 21820
and try registering again.

Pangolin logs repeat:

Public key mismatch. Updating public key and clearing session info...
Client last hole punch is too old and we have sites to send; skipping this register.
The client is failing to hole punch and identify its network address with the server.
Can the client reach the server on UDP port 21820?

The database contains two records for the same Android installation:

Organization A client:
  clientId: <redacted>
  olmId: <same value as Organization B>
  online: false
  lastHolePunch: <valid timestamp after the successful connection>

Organization B client:
  clientId: <different value>
  olmId: <same value as Organization A>
  online: false
  lastHolePunch: NULL

The corresponding global Olm record has one mapping:

olms.id: <shared olmId>
olms.clientId: <one of the two organization-specific client IDs>
agent: Pangolin Android
version: 0.32.0

Network checks already performed

• UDP 21820 is published on IPv4 and IPv6.
• The host firewall accepts UDP 21820.
• Packet counters show external UDP traffic on the port.
• Tcpdump confirmed bidirectional forwarding through Docker to Gerbil and replies back to external clients.
• UDP packets from the affected client were observed reaching the VPS during registration attempts.
• HTTPS and WebSocket registration traffic works.
• DNS resolves to the correct server.
• There is no CrowdSec decision for the affected client address.
• The same phone and network can reach the same Pangolin server when the other organization is selected.
• The full Pangolin stack was restarted. This temporarily allowed one organization to connect, but organization switching broke registration again.

This looks different from a general port, timezone, Rootless Docker, or mobile network problem because the failure follows the organization/client mapping on the same Pangolin instance.

Related reports with the same final error, but without the multi-organization reproduction:

• #2318
• #2400