fix(perf): TRex ASTFIPGenDist needs a [start,end] IP range by gspivey · Pull Request #105 · gspivey/dpdk-stdlib-rust

gspivey · 2026-07-01T13:08:48Z

Follow-up to #104. That fix worked — run 28518413002 got TRex ASTF fully up (no LRO death, RPC serving, preflight passed) and tcp-echo bound on the real NIC. Next layer:

run_tcp_benchmark.py:52  ASTFIPGenDist(ip_range=src_ip, ...)
→ Bad IP range "10.0.1.246" ... Range should contain two IPs

TRex's ASTFIPGenDist requires a 2-element [start, end] range even for a single host. Every config failed at profile-build. Fix: pass [ip, ip] for the client/server/glob generators. Adds a regression check (grep-based, no TRex needed) asserting every ASTFIPGenDist(ip_range=...) is a list.

py_compile clean, harness tests green. Merge → re-dispatch the TCP perf suite. This is the last known blocker before real DPDK-TCP numbers.

🤖 Generated with Claude Code

…gle IP Run 28518413002 (post-#104) got TRex ASTF fully up (LRO fix worked, RPC serving, DUT tcp-echo bound) but every config failed at profile-build: Bad IP range "10.0.1.246" for parameter ip_range to function ASTFIPGenDist - Range should contain two IPs ASTFIPGenDist requires a 2-element [start, end] range even for a single host. Pass [ip, ip] for client/server/glob gens. Add a regression check (no TRex needed) asserting every ASTFIPGenDist ip_range is a list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

github-actions · 2026-07-01T13:09:53Z

Synthetic Performance Results (run)

Commit: 76ab6756

✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)

Synthetic UDP Performance Results

Measures framework overhead: sync dpdk_udp::UdpSocket vs async (std::sync::Mutex + try_recv_from).

IPv4 Baseline

Test	Payload	Sync PPS	Async PPS	Ratio (sync/async)	Async ns/op
TX send_to	64B	14.8M	14.3M	1.0x	69
RX recv_from	64B	4.5M	5.4M	0.8x	184
TX send_to	1400B	3.0M	3.0M	1.0x	334
RX recv_from	1400B	1.6M	1.7M	0.9x	586

IPv6

Test	Payload	Sync PPS	Async PPS	Ratio (sync/async)	Async ns/op
TX send_to (IPv6)	64B	12.1M	11.4M	1.1x	87
RX recv_from (IPv6)	64B	4.9M	6.1M	0.8x	164
TX send_to (IPv6)	1400B	4.2M	4.2M	1.0x	238
RX recv_from (IPv6)	1400B	1.6M	1.5M	1.0x	664

IPv6 vs IPv4 Comparison (sync path)

Test	Payload	IPv4 PPS	IPv6 PPS	IPv4/IPv6 Ratio
TX send_to (sync)	64B	14.8M	12.1M	1.22x
RX recv_from (sync)	64B	4.5M	4.9M	0.91x
TX send_to (sync)	1400B	3.0M	4.2M	0.72x
RX recv_from (sync)	1400B	1.6M	1.6M	1.03x

IPv4 avg sync/async ratio: 1.0x, worst: 1.0x | IPv6 vs IPv4 worst ratio: 1.22x (OK)

OK: IPv6 is 22.2% slower than IPv4 — within acceptable threshold (<50%). Expected due to larger headers (40B vs 20B) and mandatory UDP checksum.

Good: Async wrapper is within 1.0x of sync — minimal framework overhead.

github-actions · 2026-07-01T13:10:03Z

Synthetic Performance Results — Graviton (run)

Commit: 76ab6756

✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to [2001:db8::1]:9000 (MAC: 02:00:00:00:00:01)

Synthetic UDP Performance Results

Measures framework overhead: sync dpdk_udp::UdpSocket vs async (std::sync::Mutex + try_recv_from).

IPv4 Baseline

Test	Payload	Sync PPS	Async PPS	Ratio (sync/async)	Async ns/op
TX send_to	64B	11.7M	11.2M	1.0x	89
RX recv_from	64B	3.5M	4.7M	0.8x	212
TX send_to	1400B	1.8M	1.8M	1.0x	555
RX recv_from	1400B	1.1M	1.2M	0.9x	822

IPv6

Test	Payload	Sync PPS	Async PPS	Ratio (sync/async)	Async ns/op
TX send_to (IPv6)	64B	9.0M	8.9M	1.0x	112
RX recv_from (IPv6)	64B	3.9M	5.5M	0.7x	183
TX send_to (IPv6)	1400B	3.0M	3.0M	1.0x	329
RX recv_from (IPv6)	1400B	1.2M	1.3M	0.9x	795

IPv6 vs IPv4 Comparison (sync path)

Test	Payload	IPv4 PPS	IPv6 PPS	IPv4/IPv6 Ratio
TX send_to (sync)	64B	11.7M	9.0M	1.30x
RX recv_from (sync)	64B	3.5M	3.9M	0.90x
TX send_to (sync)	1400B	1.8M	3.0M	0.59x
RX recv_from (sync)	1400B	1.1M	1.2M	0.95x

IPv4 avg sync/async ratio: 0.9x, worst: 1.0x | IPv6 vs IPv4 worst ratio: 1.30x (OK)

OK: IPv6 is 29.8% slower than IPv4 — within acceptable threshold (<50%). Expected due to larger headers (40B vs 20B) and mandatory UDP checksum.

Good: Async wrapper is within 1.0x of sync — minimal framework overhead.

github-actions · 2026-07-01T13:20:11Z

[CI] Stage: Deploy

Infrastructure ready.

Sender: i-012389062f8b7419f (DPDK ENI: 10.0.1.224)
Receiver: i-04d03430698fa662e (DPDK ENI: 10.0.1.208)
Both instances SSM-ready.

github-actions · 2026-07-01T13:20:36Z

[CI] Stage: Deploy

Infrastructure ready.

Sender: i-0e5133368345fc270 (DPDK ENI: 10.0.1.104)
Receiver: i-053aa2f901a287b5b (DPDK ENI: 10.0.1.91)
Both instances SSM-ready.

github-actions · 2026-07-01T13:37:57Z

[CI] Stage: Summary

All tests PASSED.

ARP seeding: kernel /proc/net/arp (automatic)

tier1-dpdk-echo: 6 tests, 0 failures
tier2-kernel-interop: 4 tests, 0 failures
tier3-iperf-interop: 1 tests, 0 failures
tier3-iperf-interop: 1 tests, 0 failures

github-actions · 2026-07-01T13:38:00Z

[CI] Stage: Summary

All tests PASSED.

ARP seeding: kernel /proc/net/arp (automatic)

tier1-dpdk-echo: 6 tests, 0 failures
tier2-kernel-interop: 4 tests, 0 failures
tier3-iperf-interop: 1 tests, 0 failures
tier3-iperf-interop: 1 tests, 0 failures

github-actions · 2026-07-01T13:41:01Z

✅ Integration Tests Passed — Graviton (run)

Branch: 105/merge | Commit: 76ab6756

Test Results

tier1-dpdk-echo.xml: 6 tests, 0 failures
tier2-kernel-interop.xml: 4 tests, 0 failures
tier3-iperf-sends.xml: 1 tests, 0 failures
tier3-our-app-sends.xml: 1 tests, 0 failures

Application Logs (last 20 lines)

receiver-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.208:9000 (MAC: 02:60:30:f1:2d:b3)
echo listening on 10.0.1.208:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.224:9000 (MAC: 02:62:6a:a9:fd:2b)
echo listening on 10.0.1.224:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-test-client.log

EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
DPDK bind failed (Port init failed: Invalid port ID: 0), falling back to tokio
Backend: tokio
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.208:9000: 'arp-probe #1'
Test complete
[2026-07-01T13:25:39Z] INFO: ARP resolution succeeded (got response from peer)
[2026-07-01T13:25:39Z] INFO: Test: udp_send_receive
[2026-07-01T13:25:40Z] INFO: UDP send/receive succeeded
[2026-07-01T13:25:40Z] INFO: Test: echo_roundtrip
[2026-07-01T13:25:40Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-07-01T13:25:40Z] INFO: Test: payload_integrity
[2026-07-01T13:25:40Z] INFO: Response received, checking payload match...
[2026-07-01T13:25:40Z] INFO: Payload integrity verified (found in response)
[2026-07-01T13:25:40Z] INFO: JUnit XML written to: /tmp/test-results/tier2-kernel-interop.xml
[2026-07-01T13:25:40Z] INFO: Tier 2 sender tests complete. Results: /tmp/test-results/tier2-kernel-interop.xml

github-actions · 2026-07-01T13:41:04Z

✅ Integration Tests Passed (Run 28519883925)

Branch: 105/merge | Commit: 76ab6756

Test Results

tier1-dpdk-echo.xml: 6 tests, 0 failures, skipped
tier2-kernel-interop.xml: 4 tests, 0 failures, skipped
tier3-iperf-sends.xml: 1 tests, 0 failures, skipped
tier3-our-app-sends.xml: 1 tests, 0 failures, skipped

Application Logs (last 20 lines)

receiver-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.91:9000 (MAC: 02:c5:19:cf:0e:c3)
echo listening on 10.0.1.91:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.104:9000 (MAC: 02:1b:2c:c1:4f:9f)
echo listening on 10.0.1.104:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-test-client.log

EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
DPDK bind failed (Port init failed: Invalid port ID: 0), falling back to tokio
Backend: tokio
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.91:9000: 'arp-probe #1'
Test complete
[2026-07-01T13:25:48Z] INFO: ARP resolution succeeded (got response from peer)
[2026-07-01T13:25:48Z] INFO: Test: udp_send_receive
[2026-07-01T13:25:49Z] INFO: UDP send/receive succeeded
[2026-07-01T13:25:49Z] INFO: Test: echo_roundtrip
[2026-07-01T13:25:50Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-07-01T13:25:50Z] INFO: Test: payload_integrity
[2026-07-01T13:25:50Z] INFO: Response received, checking payload match...
[2026-07-01T13:25:50Z] INFO: Payload integrity verified (found in response)
[2026-07-01T13:25:50Z] INFO: JUnit XML written to: /tmp/test-results/tier2-kernel-interop.xml
[2026-07-01T13:25:50Z] INFO: Tier 2 sender tests complete. Results: /tmp/test-results/tier2-kernel-interop.xml

receiver-test-client-iperf.log

[2026-07-01T13:32:48Z] INFO: iperf-sends: sent 10 packets, received 10 responses
[2026-07-01T13:32:48Z] INFO: iperf-sends: PASS (sent >= 5 packets)
[2026-07-01T13:32:48Z] INFO: JUnit XML written to: /tmp/test-results/tier3-iperf-sends.xml
[2026-07-01T13:32:48Z] INFO: iperf-sends test complete

sender-test-client-iperf.log

Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #3'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #4'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #4'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #5'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #5'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #6'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #6'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #7'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #7'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #8'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #8'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #9'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #9'
Sent 31 bytes: 'dpdk-to-kernel-test-payload #10'
Received 31 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #10'
Test complete
[2026-07-01T13:31:48Z] INFO: our-app-sends: sent 10 packets, received 10 responses
[2026-07-01T13:31:48Z] INFO: our-app-sends: PASS (sent >= 5 packets)
[2026-07-01T13:31:48Z] INFO: JUnit XML written to: /tmp/test-results/tier3-our-app-sends.xml
[2026-07-01T13:31:48Z] INFO: our-app-sends test complete

Full Application Logs (last 200 lines each)

receiver-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.91:9000 (MAC: 02:c5:19:cf:0e:c3)
echo listening on 10.0.1.91:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.104:9000 (MAC: 02:1b:2c:c1:4f:9f)
echo listening on 10.0.1.104:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-test-client.log

[2026-07-01T13:21:54Z] INFO: Test: arp_resolution
UDP Test Client
Target: 10.0.1.91:9000
Bind address: 10.0.1.104:0
Message: 'arp-probe'
Count: 1
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.104:32768 (MAC: 02:1b:2c:c1:4f:9f)
Backend: dpdk
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.91:9000: 'arp-probe #1'
Test complete
[2026-07-01T13:21:55Z] INFO: ARP resolution succeeded (got response from peer)
[2026-07-01T13:21:55Z] INFO: Test: udp_send_receive
[2026-07-01T13:21:57Z] INFO: UDP send/receive succeeded
[2026-07-01T13:21:57Z] INFO: Test: echo_roundtrip
[2026-07-01T13:21:58Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-07-01T13:21:58Z] INFO: Test: payload_integrity
[2026-07-01T13:21:59Z] INFO: Response received, checking payload match...
[2026-07-01T13:21:59Z] INFO: Payload integrity verified (found in response)
[2026-07-01T13:21:59Z] INFO: Test: jumbo_diagnostics
[2026-07-01T13:21:59Z] INFO: === JUMBO FRAME DIAGNOSTICS ===
[2026-07-01T13:21:59Z] INFO: Interface MTU:
  9001
  65536
[2026-07-01T13:21:59Z] INFO:   ens5: MTU=9001
[2026-07-01T13:21:59Z] INFO:   lo: MTU=65536
[2026-07-01T13:21:59Z] INFO: Routing table MTU column:
Iface	Destination	Gateway 	Flags	RefCnt	Use	Metric	Mask		MTU	Window	IRTT                                                       
ens5	00000000	0101000A	0003	0	0	512	00000000	0	0	0                                                                             
ens5	0200000A	0101000A	0007	0	0	512	FFFFFFFF	0	0	0                                                                             
ens5	0001000A	00000000	0001	0	0	512	00FFFFFF	0	0	0                                                                             
ens5	0101000A	00000000	0005	0	0	512	FFFFFFFF	0	0	0                                                                             
[2026-07-01T13:21:59Z] INFO: DPDK port config (from echo server log):
[2026-07-01T13:21:59Z] INFO:   (no MTU info in echo log)
[2026-07-01T13:21:59Z] INFO: === END JUMBO DIAGNOSTICS ===
[2026-07-01T13:21:59Z] INFO: Test: jumbo_echo_8000
[2026-07-01T13:22:01Z] INFO: Jumbo output: UDP Test Client
Target: 10.0.1.91:9000
Bind address: 10.0.1.104:0
Payload size: 8000 bytes
Count: 3
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.104:32768 (MAC: 02:1b:2c:c1:4f:9f)
Backend: dpdk
Sending packets...
Sent 8000 bytes (binary payload)
Received 8000 bytes from 10.0.1.91:9000 (expected 8000, OK)
Sent 8000 bytes (binary payload)
Received 8000 bytes from 10.0.1.91:9000 (expected 8000, OK)
Sent 8000 bytes (binary payload)
Received 8000 bytes from 10.0.1.91:9000 (expected 8000, OK)
Test complete
[2026-07-01T13:22:01Z] INFO: Jumbo frame echo succeeded: 3/3 responses with correct size
[2026-07-01T13:22:01Z] INFO: JUnit XML written to: /tmp/test-results/tier1-dpdk-echo.xml
[2026-07-01T13:22:01Z] INFO: Tier 1 sender tests complete. Results: /tmp/test-results/tier1-dpdk-echo.xml
[2026-07-01T13:25:48Z] INFO: Test: arp_resolution
UDP Test Client
Target: 10.0.1.91:9000
Bind address: 0.0.0.0:0
Message: 'arp-probe'
Count: 1
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
DPDK bind failed (Port init failed: Invalid port ID: 0), falling back to tokio
Backend: tokio
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.91:9000: 'arp-probe #1'
Test complete
[2026-07-01T13:25:48Z] INFO: ARP resolution succeeded (got response from peer)
[2026-07-01T13:25:48Z] INFO: Test: udp_send_receive
[2026-07-01T13:25:49Z] INFO: UDP send/receive succeeded
[2026-07-01T13:25:49Z] INFO: Test: echo_roundtrip
[2026-07-01T13:25:50Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-07-01T13:25:50Z] INFO: Test: payload_integrity
[2026-07-01T13:25:50Z] INFO: Response received, checking payload match...
[2026-07-01T13:25:50Z] INFO: Payload integrity verified (found in response)
[2026-07-01T13:25:50Z] INFO: JUnit XML written to: /tmp/test-results/tier2-kernel-interop.xml
[2026-07-01T13:25:50Z] INFO: Tier 2 sender tests complete. Results: /tmp/test-results/tier2-kernel-interop.xml

receiver-test-client-iperf.log

[2026-07-01T13:32:48Z] INFO: iperf-sends: sent 10 packets, received 10 responses
[2026-07-01T13:32:48Z] INFO: iperf-sends: PASS (sent >= 5 packets)
[2026-07-01T13:32:48Z] INFO: JUnit XML written to: /tmp/test-results/tier3-iperf-sends.xml
[2026-07-01T13:32:48Z] INFO: iperf-sends test complete

sender-test-client-iperf.log

[2026-07-01T13:31:46Z] INFO: Pre-flight: checking DPDK state and ARP cache...
[2026-07-01T13:31:46Z] INFO: Local IP: 10.0.1.104, Peer IP: 10.0.1.91, Port: 9000
[2026-07-01T13:31:46Z] INFO: /proc/net/arp contents:
IP address       HW type     Flags       HW address            Mask     Device
10.0.1.156       0x1         0x2         02:b1:6b:66:5c:61     *        ens5
10.0.1.104       0x1         0x2         02:1b:2c:c1:4f:9f     *        ens5
10.0.1.139       0x1         0x2         02:4b:31:f2:b5:0b     *        ens5
10.0.1.91        0x1         0x2         02:c5:19:cf:0e:c3     *        ens5
10.0.1.1         0x1         0x2         02:da:68:5d:70:ef     *        ens5
[2026-07-01T13:31:46Z] INFO: DPDK runtime state:
No /var/run/dpdk/ directory
[2026-07-01T13:31:46Z] INFO: vfio-pci bindings:
0000:00:06.0
bind
module
new_id
remove_id
uevent
unbind
[2026-07-01T13:31:46Z] INFO: Test binary: /opt/dpdk-stdlib/target/release/test-client
-rwxr-xr-x. 2 root root 1950584 Jul  1 13:18 /opt/dpdk-stdlib/target/release/test-client
[2026-07-01T13:31:46Z] INFO: Launching test-client: /opt/dpdk-stdlib/target/release/test-client --target 10.0.1.91 --port 9000 --bind-ip 10.0.1.104 --count 10 --delay 200
[2026-07-01T13:31:48Z] INFO: Test client output: UDP Test Client
Target: 10.0.1.91:9000
Bind address: 10.0.1.104:0
Message: 'dpdk-to-kernel-test-payload'
Count: 10
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.104:32768 (MAC: 02:1b:2c:c1:4f:9f)
Backend: dpdk
Sending packets...
Sent 30 bytes: 'dpdk-to-kernel-test-payload #1'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #1'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #2'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #2'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #3'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #3'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #4'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #4'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #5'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #5'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #6'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #6'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #7'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #7'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #8'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #8'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #9'
Received 30 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #9'
Sent 31 bytes: 'dpdk-to-kernel-test-payload #10'
Received 31 bytes from 10.0.1.91:9000: 'dpdk-to-kernel-test-payload #10'
Test complete
[2026-07-01T13:31:48Z] INFO: our-app-sends: sent 10 packets, received 10 responses
[2026-07-01T13:31:48Z] INFO: our-app-sends: PASS (sent >= 5 packets)
[2026-07-01T13:31:48Z] INFO: JUnit XML written to: /tmp/test-results/tier3-our-app-sends.xml
[2026-07-01T13:31:48Z] INFO: our-app-sends test complete

⚠️ SSM Command Failures (receiver-ssm-failure.log)

=== Polling timeout after 30s ===
Status: InProgress
Instance: i-053aa2f901a287b5b (receiver)
Command ID: 785f2dc5-6c1a-4e25-9ef3-8501bf9e8cf7

=== STDOUT ===


=== STDERR ===


=== Polling timeout after 30s ===
Status: InProgress
Instance: i-053aa2f901a287b5b (receiver)
Command ID: 55ae78d8-c9a5-48c3-9fd2-c9b271d48b30

=== STDOUT ===


=== STDERR ===


=== Polling timeout after 30s ===
Status: InProgress
Instance: i-053aa2f901a287b5b (receiver)
Command ID: 7c980d34-962d-4f5b-887a-53bd59324768

=== STDOUT ===


=== STDERR ===


=== Polling timeout after 30s ===
Status: InProgress
Instance: i-053aa2f901a287b5b (receiver)
Command ID: cd9610a9-910d-4516-85ec-dc96ddbb9e4c

=== STDOUT ===


=== STDERR ===

⚠️ SSM Command Failures (sender-ssm-failure.log)

=== Polling timeout after 30s ===
Status: InProgress
Instance: i-0e5133368345fc270 (sender)
Command ID: 8704b674-99ca-4ab9-8c45-d8dd4039264a

=== STDOUT ===


=== STDERR ===

Network & PCI State

receiver-network-interfaces.log

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:4f:2c:f0:ef:e3 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-0d7f7b8e4ad188f92
    altname device-number-0.0
    inet 10.0.1.217/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 2128sec preferred_lft 2128sec
    inet6 fe80::4f:2cff:fef0:efe3/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever

sender-network-interfaces.log

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:36:c4:7f:71:41 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-0345131794acf693f
    altname device-number-0.0
    inet 10.0.1.74/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 2150sec preferred_lft 2150sec
    inet6 fe80::36:c4ff:fe7f:7141/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever

receiver-networking-diag-baseline.txt

=== NETWORKING DIAGNOSTICS ===
timestamp: 2026-07-01T13:21:15Z
hostname: ip-10-0-1-217.ec2.internal
kernel: 6.18.35-68.129.amzn2023.x86_64

=== DPDK PORT STATUS ===

Network devices using DPDK-compatible driver
============================================
0000:00:06.0 'Elastic Network Adapter (ENA) ec20' drv=vfio-pci unused=ena

Network devices using kernel driver
===================================
0000:00:05.0 'Elastic Network Adapter (ENA) ec20' if=ens5 drv=ena unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'DMA' devices detected
=========================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

=== IP ADDRESSES ===
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:4f:2c:f0:ef:e3 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-0d7f7b8e4ad188f92
    altname device-number-0.0
    inet 10.0.1.217/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 3119sec preferred_lft 3119sec
    inet6 fe80::4f:2cff:fef0:efe3/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever

=== ARP TABLE ===
10.0.1.139 dev ens5 lladdr 02:4b:31:f2:b5:0b REACHABLE 
10.0.1.1 dev ens5 lladdr 02:da:68:5d:70:ef REACHABLE 

=== ROUTE TABLE ===
default via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.217 metric 512 
10.0.0.2 via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.217 metric 512 
10.0.1.0/24 dev ens5 proto kernel scope link src 10.0.1.217 metric 512 
10.0.1.1 dev ens5 proto dhcp scope link src 10.0.1.217 metric 512 

=== IMDS: ENI INFORMATION ===
ENI MACs found: 02:4f:2c:f0:ef:e3/ 02:c5:19:cf:0e:c3/ 

--- ENI: 02:4f:2c:f0:ef:e3/ ---
  device-number: 0
  local-ipv4s: 10.0.1.217
  subnet-id: subnet-0b818407ff498a772
  subnet-cidr: 10.0.1.0/24

--- ENI: 02:c5:19:cf:0e:c3/ ---
  device-number: 1
  local-ipv4s: 10.0.1.91
  subnet-id: subnet-0b818407ff498a772
  subnet-cidr: 10.0.1.0/24


=== GATEWAY ARP TEST ===
Gateway IP: 10.0.1.1
Gateway ARP entry:
10.0.1.1 dev ens5 lladdr 02:da:68:5d:70:ef REACHABLE 

arping result:
ARPING 10.0.1.1 from 10.0.1.217 ens5
Unicast reply from 10.0.1.1 [02:DA:68:5D:70:EF]  0.544ms
Sent 1 probes (1 broadcast(s))
Received 1 response(s)

=== HUGEPAGE STATUS ===
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:     51200 kB
HugePages_Total:    1024
HugePages_Free:     1024
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:         2097152 kB

=== VFIO STATUS ===
total 0
drwxr-xr-x.  2 root root       80 Jul  1 13:21 .
drwxr-xr-x. 14 root root     3100 Jul  1 13:19 ..
crw-------.  1 root root 243,   0 Jul  1 13:21 noiommu-0
crw-rw-rw-.  1 root root  10, 196 Jul  1 13:13 vfio

noiommu mode:
Y

=== DPDK SHARED MEMORY ===
no /var/run/dpdk/ directory (clean state)

=== DPDK-RELATED DMESG (last 30 lines) ===
[    0.053518] printk: legacy console [ttyS0] enabled
[    0.054526] x2apic enabled
[    0.058692] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto
[    0.058801] x86/fpu: Enabled xstate features 0x2ff, context size is 2568 bytes, using 'compacted' format.
[    0.068598] audit: type=2000 audit(1782911587.930:1): state=initialized audit_enabled=0 res=1
[    0.088062] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    0.140602] ACPI: Interpreter enabled
[    0.140602] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.159255] pci 0000:00:05.0: enabling Extended Tags
[    0.254046] SGI XFS with ACLs, security attributes, quota, no debug enabled
[    0.269213] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[    0.370551] IPI shorthand broadcast: enabled
[    3.507935] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[    3.546402] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[    3.699205] VFIO - User Level meta-driver version: 0.3
[    4.282005] ena 0000:00:05.0: Elastic Network Adapter (ENA) v2.17.1g
[    4.293174] ena 0000:00:05.0: ENA device version: 0.10
[    4.293950] ena 0000:00:05.0: ENA controller version: 0.0.1 implementation version 1
[    4.394614] ena 0000:00:05.0: ENA Large LLQ is disabled
[    4.407074] ena 0000:00:05.0: Elastic Network Adapter (ENA) found at mem c0500000, mac addr 02:4f:2c:f0:ef:e3
[    4.446752] ena 0000:00:05.0 ens5: renamed from eth0
[  391.435061] pci 0000:00:06.0: enabling Extended Tags
[  391.438956] ena 0000:00:06.0: enabling device (0000 -> 0002)
[  391.451671] ena 0000:00:06.0: ENA device version: 0.10
[  391.452386] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  391.552111] ena 0000:00:06.0: ENA Large LLQ is disabled
[  391.565658] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:c5:19:cf:0e:c3
[  391.571358] ena 0000:00:06.0 ens6: renamed from eth0
[  473.545310] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  473.546716] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device

=== DPDK-RELATED PROCESSES ===
no DPDK processes running

=== END DIAGNOSTICS ===

sender-networking-diag-baseline.txt

=== NETWORKING DIAGNOSTICS ===
timestamp: 2026-07-01T13:21:07Z
hostname: ip-10-0-1-74.ec2.internal
kernel: 6.18.35-68.129.amzn2023.x86_64

=== DPDK PORT STATUS ===

Network devices using DPDK-compatible driver
============================================
0000:00:06.0 'Elastic Network Adapter (ENA) ec20' drv=vfio-pci unused=ena

Network devices using kernel driver
===================================
0000:00:05.0 'Elastic Network Adapter (ENA) ec20' if=ens5 drv=ena unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'DMA' devices detected
=========================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

=== IP ADDRESSES ===
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:36:c4:7f:71:41 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-0345131794acf693f
    altname device-number-0.0
    inet 10.0.1.74/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 3127sec preferred_lft 3127sec
    inet6 fe80::36:c4ff:fe7f:7141/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever

=== ARP TABLE ===
10.0.1.139 dev ens5 lladdr 02:4b:31:f2:b5:0b REACHABLE 
10.0.1.1 dev ens5 lladdr 02:da:68:5d:70:ef REACHABLE 

=== ROUTE TABLE ===
default via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.74 metric 512 
10.0.0.2 via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.74 metric 512 
10.0.1.0/24 dev ens5 proto kernel scope link src 10.0.1.74 metric 512 
10.0.1.1 dev ens5 proto dhcp scope link src 10.0.1.74 metric 512 

=== IMDS: ENI INFORMATION ===
ENI MACs found: 02:1b:2c:c1:4f:9f/ 02:36:c4:7f:71:41/ 

--- ENI: 02:1b:2c:c1:4f:9f/ ---
  device-number: 1
  local-ipv4s: 10.0.1.104
  subnet-id: subnet-0b818407ff498a772
  subnet-cidr: 10.0.1.0/24

--- ENI: 02:36:c4:7f:71:41/ ---
  device-number: 0
  local-ipv4s: 10.0.1.74
  subnet-id: subnet-0b818407ff498a772
  subnet-cidr: 10.0.1.0/24


=== GATEWAY ARP TEST ===
Gateway IP: 10.0.1.1
Gateway ARP entry:
10.0.1.1 dev ens5 lladdr 02:da:68:5d:70:ef REACHABLE 

arping result:
ARPING 10.0.1.1 from 10.0.1.74 ens5
Unicast reply from 10.0.1.1 [02:DA:68:5D:70:EF]  0.539ms
Sent 1 probes (1 broadcast(s))
Received 1 response(s)

=== HUGEPAGE STATUS ===
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:     51200 kB
HugePages_Total:    1024
HugePages_Free:     1024
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:         2097152 kB

=== VFIO STATUS ===
total 0
drwxr-xr-x.  2 root root       80 Jul  1 13:20 .
drwxr-xr-x. 14 root root     3100 Jul  1 13:20 ..
crw-------.  1 root root 243,   0 Jul  1 13:20 noiommu-0
crw-rw-rw-.  1 root root  10, 196 Jul  1 13:13 vfio

noiommu mode:
Y

=== DPDK SHARED MEMORY ===
no /var/run/dpdk/ directory (clean state)

=== DPDK-RELATED DMESG (last 30 lines) ===
[    0.052909] printk: legacy console [ttyS0] enabled
[    0.053923] x2apic enabled
[    0.058051] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto
[    0.058160] x86/fpu: Enabled xstate features 0x2ff, context size is 2568 bytes, using 'compacted' format.
[    0.067898] audit: type=2000 audit(1782911588.586:1): state=initialized audit_enabled=0 res=1
[    0.087398] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    0.138730] ACPI: Interpreter enabled
[    0.138730] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.158500] pci 0000:00:05.0: enabling Extended Tags
[    0.232208] SGI XFS with ACLs, security attributes, quota, no debug enabled
[    0.239463] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[    0.357754] IPI shorthand broadcast: enabled
[    3.176858] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[    3.212384] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[    3.295778] VFIO - User Level meta-driver version: 0.3
[    3.623199] ena 0000:00:05.0: Elastic Network Adapter (ENA) v2.17.1g
[    3.647010] ena 0000:00:05.0: ENA device version: 0.10
[    3.647742] ena 0000:00:05.0: ENA controller version: 0.0.1 implementation version 1
[    3.764567] ena 0000:00:05.0: ENA Large LLQ is disabled
[    3.776758] ena 0000:00:05.0: Elastic Network Adapter (ENA) found at mem c0500000, mac addr 02:36:c4:7f:71:41
[    3.793557] ena 0000:00:05.0 ens5: renamed from eth0
[  431.960946] pci 0000:00:06.0: enabling Extended Tags
[  431.964937] ena 0000:00:06.0: enabling device (0000 -> 0002)
[  431.979087] ena 0000:00:06.0: ENA device version: 0.10
[  431.979853] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  432.080994] ena 0000:00:06.0: ENA Large LLQ is disabled
[  432.094734] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:1b:2c:c1:4f:9f
[  432.101442] ena 0000:00:06.0 ens6: renamed from eth0
[  465.977054] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  465.978498] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device

=== DPDK-RELATED PROCESSES ===
no DPDK processes running

=== END DIAGNOSTICS ===

⚠️ Crash Diagnostics

receiver-dmesg-crashes.log

[    0.067959] pid_max: default: 32768 minimum: 301
[    0.163108] iommu: Default domain type: Translated
[    0.163139] NetLabel:  unlabeled traffic allowed by default
[    0.192209] PCI: CLS 0 bytes, default 64
[    0.286228] nvme nvme0: 2/0/0 default/read/poll queues
[    0.449665] systemd[1]: systemd 252.23-12.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    0.449790] systemd[1]: No hostname configured, using default hostname.
[    0.514206] systemd[1]: Queued start job for default target initrd.target.
[    2.531127] systemd[1]: systemd 252.23-12.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[  511.214567] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:8400)
[  744.793227] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:8839)
[ 1338.129458] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:10010)

sender-dmesg-crashes.log

[    0.067304] pid_max: default: 32768 minimum: 301
[    0.162328] iommu: Default domain type: Translated
[    0.162372] NetLabel:  unlabeled traffic allowed by default
[    0.188952] PCI: CLS 0 bytes, default 64
[    0.254908] nvme nvme0: 2/0/0 default/read/poll queues
[    0.429290] systemd[1]: systemd 252.23-12.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    0.429417] systemd[1]: No hostname configured, using default hostname.
[    0.497239] systemd[1]: Queued start job for default target initrd.target.
[    2.550002] systemd[1]: systemd 252.23-12.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 1162.633802] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:9621)

Kernel Console (dmesg)

receiver-console-output.log (PCI/driver events only)

[  511.213005] vfio-pci 0000:00:06.0: reset done
[  511.214567] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:8400)
[  511.215970] vfio-pci 0000:00:06.0: resetting
[  511.432872] vfio-pci 0000:00:06.0: reset done
[  592.093364] vfio-pci 0000:00:06.0: Removing from iommu group 0
[  593.106265] ena 0000:00:06.0: ENA device version: 0.10
[  593.107041] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  593.208197] ena 0000:00:06.0: ENA Large LLQ is disabled
[  593.222100] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:c5:19:cf:0e:c3
[  593.229424] ena 0000:00:06.0 ens6: renamed from eth0
[  613.054510] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  613.055898] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[  744.565424] vfio-pci 0000:00:06.0: resetting
[  744.791676] vfio-pci 0000:00:06.0: reset done
[  744.793227] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:8839)
[  744.794583] vfio-pci 0000:00:06.0: resetting
[  745.011565] vfio-pci 0000:00:06.0: reset done
[  820.061289] vfio-pci 0000:00:06.0: Removing from iommu group 0
[  821.069595] ena 0000:00:06.0: ENA device version: 0.10
[  821.070361] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  821.169578] ena 0000:00:06.0: ENA Large LLQ is disabled
[  821.181722] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:c5:19:cf:0e:c3
[  821.192281] ena 0000:00:06.0 ens6: renamed from eth0 (while UP)
[ 1263.470849] vfio-pci 0000:00:06.0: Adding to iommu group 0
[ 1263.472214] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[ 1337.910659] vfio-pci 0000:00:06.0: resetting
[ 1338.127909] vfio-pci 0000:00:06.0: reset done
[ 1338.129458] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:10010)
[ 1338.130863] vfio-pci 0000:00:06.0: resetting
[ 1338.347750] vfio-pci 0000:00:06.0: reset done

sender-console-output.log (PCI/driver events only)

[ 1162.635208] vfio-pci 0000:00:06.0: resetting
[ 1162.852131] vfio-pci 0000:00:06.0: reset done
[ 1225.177753] vfio-pci 0000:00:06.0: Removing from iommu group 0
[ 1226.190384] ena 0000:00:06.0: ENA device version: 0.10
[ 1226.191143] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[ 1226.290364] ena 0000:00:06.0: ENA Large LLQ is disabled
[ 1226.302785] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:1b:2c:c1:4f:9f
[ 1226.310597] ena 0000:00:06.0 ens6: renamed from eth0
[ 1255.976308] vfio-pci 0000:00:06.0: Adding to iommu group 0
[ 1255.977744] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[ 1353.146846] vfio-pci 0000:00:06.0: resetting
[ 1353.361578] vfio-pci 0000:00:06.0: reset done
[ 1353.363110] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:10065)
[ 1353.364669] vfio-pci 0000:00:06.0: resetting
[ 1353.581435] vfio-pci 0000:00:06.0: reset done
[ 1353.869009] vfio-pci 0000:00:06.0: resetting
[ 1354.091454] vfio-pci 0000:00:06.0: reset done
[ 1354.092989] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:10085)
[ 1354.094598] vfio-pci 0000:00:06.0: resetting
[ 1354.311408] vfio-pci 0000:00:06.0: reset done
[ 1355.601132] vfio-pci 0000:00:06.0: resetting
[ 1355.831470] vfio-pci 0000:00:06.0: reset done
[ 1355.833009] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:10108)
[ 1355.834606] vfio-pci 0000:00:06.0: resetting
[ 1356.051409] vfio-pci 0000:00:06.0: reset done
[ 1357.330927] vfio-pci 0000:00:06.0: resetting
[ 1357.551471] vfio-pci 0000:00:06.0: reset done
[ 1357.553007] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:10131)
[ 1357.554606] vfio-pci 0000:00:06.0: resetting
[ 1357.771408] vfio-pci 0000:00:06.0: reset done

gspivey merged commit 0f2f4f3 into development Jul 1, 2026
7 checks passed

gspivey deleted the agent/trex-astf-iprange-fix branch July 1, 2026 18:51

gspivey mentioned this pull request Jul 1, 2026

fix(perf): TRex ASTFIPGen.glob type + client ip_gen (profile build) #106

Merged

Conversation

gspivey commented Jul 1, 2026

Uh oh!

github-actions Bot commented Jul 1, 2026

Synthetic Performance Results (run)

Synthetic UDP Performance Results

IPv4 Baseline

IPv6

IPv6 vs IPv4 Comparison (sync path)

Uh oh!

github-actions Bot commented Jul 1, 2026

Synthetic Performance Results — Graviton (run)

Synthetic UDP Performance Results

IPv4 Baseline

IPv6

IPv6 vs IPv4 Comparison (sync path)

Uh oh!

github-actions Bot commented Jul 1, 2026

[CI] Stage: Deploy

Uh oh!

github-actions Bot commented Jul 1, 2026

[CI] Stage: Deploy

Uh oh!

github-actions Bot commented Jul 1, 2026

[CI] Stage: Summary

Uh oh!

github-actions Bot commented Jul 1, 2026

[CI] Stage: Summary

Uh oh!

github-actions Bot commented Jul 1, 2026

✅ Integration Tests Passed — Graviton (run)

Test Results

Application Logs (last 20 lines)

Uh oh!

github-actions Bot commented Jul 1, 2026

✅ Integration Tests Passed (Run 28519883925)

Test Results

Application Logs (last 20 lines)

receiver-echo-server.log

sender-echo-server.log

sender-test-client.log

receiver-test-client-iperf.log

sender-test-client-iperf.log

receiver-network-interfaces.log

sender-network-interfaces.log

receiver-networking-diag-baseline.txt

sender-networking-diag-baseline.txt

receiver-dmesg-crashes.log

sender-dmesg-crashes.log

receiver-console-output.log (PCI/driver events only)

sender-console-output.log (PCI/driver events only)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant