network: add cleanupOrphanTaps() to remove orphan TAPs by carrier state

[sidneychang]

• Call new function cleanupOrphanTaps() at the start of DynamicNetwork.NetworkSetup().
• Add cleanupOrphanTaps() (new): scan current netns for interfaces matching ^tap.*_urunc$ and use kernel carrier/operational state as the sole criterion:
  • NO-CARRIER => delete (orphan) after removing TC/qdisc
  • LOWER_UP / OperUp / FlagRunning => consider in-use and abort with error
• Preserve existing networkSetup() create-only semantics and ensure TC/qdisc cleanup before link deletion. This resolves an issue observed on Kubernetes where restarting/exiting urunc left orphan TAP devices in the pod's network namespace, causing subsequent network setup to fail. The new cleanup removes such orphan TAPs so a new urunc instance can create and configure a fresh TAP.

Description

In kubernetes setups when a pod is getting restarted, the network namespace (created by the pause container) remains active and hence the tap0_urunc device still exists. Therefore, when urunc (re)creates the container it identifies the tap0_urunc device and it does not recreates it.

Related issues

• Fixes Orphan tap*_urunc left after urunc restart in Kubernetes, preventing NetworkSetup #406

How was this tested?

1. Deploy the test Deployment/Service and observe the Pod status

kubectl apply -f nginx-urunc.yaml
kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-urunc-67f8694dd6-ntvgg 1/1 Running 0 8s

2. Find the QEMU process on the host (record the PID)

ps aux | grep qemu

Example output (initial QEMU PID = 1166356):

root      377624  0.0  0.0 838184 84172 ?        Ssl  Jan23   2:52 /usr/bin/qemu-system-x86_64 ...
root     1166356  4.7  0.0 840108 84624 ?        Ssl  18:42   0:00 /usr/bin/qemu-system-x86_64 ...
root     1166481  0.0  0.0   9212  2308 pts/1    S+   18:42   0:00 grep --color=auto qemu

3. Inspect the tap device inside that QEMU netns (should be LOWER_UP)

sudo nsenter -t 1166356 -n ip link show tap0_urunc
3: tap0_urunc: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 96:27:d0:99:cd:9a brd ff:ff:ff:ff:ff:ff

4. Force-kill QEMU (simulate a crash)

sudo kill -9 1166356

Then observe the Pod status and restart:

kubectl get pods
NAME                           READY   STATUS   RESTARTS   AGE
nginx-urunc-67f8694dd6-ntvgg   0/1     Error    0          46s
# then the Pod restarts successfully
NAME                           READY   STATUS    RESTARTS     AGE
nginx-urunc-67f8694dd6-ntvgg   1/1     Running   1 (6s ago)   48s

5. Find the new QEMU process and verify the new tap (new PID = 1166964)

ps aux | grep qemu
root 1166964 9.0 0.0 840108 84456 ? Ssl 18:43 0:00 /usr/bin/qemu-system-x86_64 ...

sudo nsenter -t 1166964 -n ip link show tap0_urunc
3: tap0_urunc: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether d2:b6:52:78:bf:0e brd ff:ff:ff:ff:ff:ff

6. Verify the Pod IP and network connectivity

kubectl get pods -o wide
ping -c 3 <POD_IP>

NAME                           READY   STATUS    RESTARTS      AGE   IP            NODE   ...
nginx-urunc-67f8694dd6-ntvgg   1/1     Running   1 (30s ago)   72s   10.88.0.104   test

PING 10.88.0.104 (10.88.0.104) 56(84) bytes of data.
64 bytes from 10.88.0.104: icmp_seq=1 ttl=255 time=0.513 ms
64 bytes from 10.88.0.104: icmp_seq=2 ttl=255 time=0.245 ms
--- 10.88.0.104 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss

7. Optional: List all interfaces in the netns to confirm both eth0 and tap are UP

sudo nsenter -t 1166964 -n ip link
# old tap interface has been removed
1: lo: <LOOPBACK,UP,LOWER_UP> ...
2: eth0@if281: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
4: tap0_urunc: <BROADCAST,MULTICAST,UP,LOWER_UP> ...

LLM usage

I use LLM to disscuss how to identify a orphan tap device.

Checklist

• I have read the contribution guide.
• The linter passes locally (make lint).
• The e2e tests of at least one tool pass locally (make test_ctr, make test_nerdctl, make test_docker, make test_crictl).
• If LLMs were used: I have read the llm policy.

[netlify]

✅ Deploy Preview for urunc ready!

Name	Link
🔨 Latest commit	dbb9c93
🔍 Latest deploy log	https://app.netlify.com/projects/urunc/deploys/6978272d95f0a00008c59aef
😎 Deploy Preview	https://deploy-preview-407--urunc.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[cmainas]

Hello @sidneychang ,

thank you for this fix. This proposal though affects all deployments with urunc adding some additional overhead, even in deployments which are not affected from the issue. Therefore, I would prefer to dig a bit further and identify why reusing the tap device does not work. I will come up here with more info as soon as I find time to investigate this in more details.

[sidneychang]

Hello @sidneychang ,

thank you for this fix. This proposal though affects all deployments with urunc adding some additional overhead, even in deployments which are not affected from the issue. Therefore, I would prefer to dig a bit further and identify why reusing the tap device does not work. I will come up here with more info as soon as I find time to investigate this in more details.

Hello, I’ve verified locally that reusing an existing TAP device is feasible and works well in Pod restart scenarios, avoiding failures caused by leftover TAP devices.
To address this, I’m planning to resolve the issue with the following approach.

Currently, NetworkSetup blocks startup based solely on the presence of a TAP:

tapIndex, err := getTapIndex()
if err != nil {
    return nil, fmt.Errorf("getTapIndex failed: %w", err)
}
if tapIndex > 0 {
    return nil, fmt.Errorf("unsupported operation: can't spawn multiple unikernels in the same network namespace")
}

This approach infers unikernel liveness from device existence, which can lead to false positives after crashes or restarts.

My proposal is to decouple these concerns by checking for an active hypervisor process in the same network namespace to determine whether a unikernel is actually running. If no hypervisor is present and exactly one TAP exists, we treat this as a restart path and reuse the TAP; only when a hypervisor is still active do we reject the setup to prevent multiple concurrent unikernels.

From an implementation perspective, this would mainly involve:

• relaxing the current tapIndex > 0 hard failure into a state check rather than an error condition;

• adding a liveness check for an existing hypervisor process associated with the same network namespace;

• introducing a clear “restart path” in NetworkSetup where an existing TAP is reused instead of creating a new one, with appropriate debug logging;

• keeping the original safety guarantee by still rejecting startup when a running hypervisor is detected.

I’d appreciate your feedback on this approach, or if you see a better way to detect an already-running unikernel in the same namespace.