Edge not marked active when two instances start simultaneously

[ILLISIS]

Bug

When two instances sharing an edge start simultaneously on the same host, one instance ends up with the edge marked inactive (red) while the other shows active (green). Restarting the inactive instance resolves the issue.

Code

// controller.ts:66-110
async onInstanceStatusChanged(instance: InstanceInfo, prev?: lib.InstanceStatus): Promise<void> {
    const edges = [...this.edgeDatastore.values()].filter(edge => edge.source.instanceId === instance.id
        || edge.target.instanceId === instance.id
    );
    edges.forEach(edge => {
        let newStatus = this.isEdgeActive(edge);
        if (edge.active !== newStatus) {
            edge.updatedAtMs = Date.now();
        }
        edge.active = newStatus;
    });
    // ...
    for (let instanceId of instanceEdgeMap.keys()) {
        if (this.controller.instances.get(instanceId)?.status === "running") {
            this.controller.sendTo({ instanceId }, new messages.EdgeUpdate(edgesToSend));
        }
    }
}

Impact

When the first instance goes "running", the controller computes active=false (partner not running yet) and sends EdgeUpdate(active=false). When the second instance goes "running", the controller computes active=true and should send EdgeUpdate(active=true) to both. However, one instance does not end up with the correct active state — likely due to a race condition in the message delivery pipeline when two InstanceStatusChangedEvent messages are dispatched in rapid succession (fire-and-forget event handlers in Clusterio can interleave at await points).

Additionally, the sendTo calls in the loop are not wrapped in try-catch, so a failure sending to one instance would prevent sending to the remaining instances.

Expected behaviour

Both instances should show the edge as active (green) shortly after both are running, regardless of startup timing.

[ILLISIS]

This issue is very difficult to reproduce as the occurrence is semi-frequent and cause unknown.