Skip to content

[Bugfix][TE] Reject MC_IB_PORT=0 instead of disabling every RNIC - #3381

Merged
staryxchen merged 1 commit into
kvcache-ai:mainfrom
SongOf:fix/rdma-port-selection
Aug 11, 2026
Merged

[Bugfix][TE] Reject MC_IB_PORT=0 instead of disabling every RNIC#3381
staryxchen merged 1 commit into
kvcache-ai:mainfrom
SongOf:fix/rdma-port-selection

Conversation

@SongOf

@SongOf SongOf commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Description

loadGlobalConfig validated MC_IB_PORT with val >= 0, so 0 passed and became globalConfig().port. IB port numbers are 1-based: RdmaContext::openRdmaDevice then calls ibv_query_port(ctx, 0, ...) on every device, which fails, so
construct() returns ERR_CONTEXT and each RNIC is disableDeviced. With the topology empty, TransferEngine::init() fails with ERR_DEVICE_NOT_FOUND — one mistyped environment variable takes down all RDMA on the node, and no log
line names the value as the cause.

atoi() returning 0 for unparseable input made this reachable by typo as well: MC_IB_PORT=abc selected port 0 instead of being ignored.

Fix: require val > 0, matching the other bounded knobs in the same function. Out-of-range and unparseable values now keep the default port 1 and log the existing warning.

Module

  • Transfer Engine (mooncake-transfer-engine)
  • Mooncake Store (mooncake-store)
  • Mooncake EP (mooncake-ep)
  • Mooncake PG (mooncake-pg)
  • Integration (mooncake-integration)
  • P2P Store (mooncake-p2p-store)
  • Python Wheel (mooncake-wheel)
  • Common (mooncake-common)
  • Mooncake RL (mooncake-rl)
  • CI/CD
  • Docs
  • Other

Type of Change

  • Bug fix
  • New feature
  • Refactor
  • Breaking change
  • Documentation update
  • Performance improvement
  • Other

How Has This Been Tested?

Test commands:
cmake .. -DBUILD_UNIT_TESTS=ON && cmake --build . -j$(nproc)
ctest -R config_test --output-on-failure

Test results:

  • Unit tests pass
  • Integration tests pass (if applicable)
  • Manual testing done (describe below)

config_test gains 6 IbPortEnvTest cases: unset default, valid override, and rejection of 0 / 256 / -1 / non-numeric. All pass.

Verified the tests are not inert: with val >= 0 restored, ZeroIsRejected and NonNumericIsRejected fail; with the fix they pass.

Full ctest: 28/31 pass. The 3 failures (tcp_transport_test, transfer_metadata_test, memory_location_test) are pre-existing environment issues in the build container — no etcd metadata server, and no CAP_SYS_NICE for mbind —
and are unrelated to this change.

Checklist

  • I have performed a self-review of my own code
  • I have formatted my code using ./scripts/code_format.sh
  • I have run pre-commit run --all-files and all hooks pass
  • I have updated the documentation (if applicable)
  • I have added tests to prove my changes are effective
  • For changes >500 LOC: I have filed an RFC issue

pre-commit was run on the two touched files only (all hooks pass); --all-files was not run because cmake-format rewrites unrelated CMakeLists.txt files. Docs need no change — MC_IB_PORT is documented as "default value 1" and
never advertised 0 as valid.

AI Assistance Disclosure

  • No AI tools were used
  • AI tools were used (specify below)

@staryxchen staryxchen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! LGTM.

@staryxchen
staryxchen merged commit da2f5be into kvcache-ai:main Aug 11, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants