CNF-23565: Dedicate CPU resources for DPDK-based vSwitch/vRouter by Tal-or · Pull Request #2001 · openshift/enhancements

Tal-or · 2026-05-07T11:37:49Z

Adds enhancement proposal for dedicating CPUs exclusively for infrastructure networking workloads (OVS-DPDK, OpenPErouter). Introduces two new PerformanceProfile API fields: spec.cpu.dedicated and spec.net.disableOvsDynamicPinning.

Tracking: CNF-22582, RFE-8921

AIA Human-AI blend, New content, Human-initiated, Reviewed, Claude Opus 4.6 v1.0
Signed-off-by: Talor Itzhak titzhak@redhat.com

openshift-ci-robot · 2026-05-07T11:37:54Z

@Tal-or: This pull request references CNF-23565 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Adds enhancement proposal for dedicating CPUs exclusively for infrastructure networking workloads (OVS-DPDK, OpenPErouter). Introduces two new PerformanceProfile API fields: spec.cpu.dedicated and spec.net.disableOvsDynamicPinning.

Tracking: CNF-22582, RFE-8921

AIA Human-AI blend, New content, Human-initiated, Reviewed, Claude Opus 4.6 v1.0
Signed-off-by: Talor Itzhak titzhak@redhat.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2026-05-07T11:39:52Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lmzuccarelli for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Tal-or · 2026-05-07T11:48:16Z

@MarSik @jmencak @yanirq @mcoquelin

Tal-or · 2026-05-07T11:48:49Z

@JM1

…Router Adds enhancement proposal for dedicating CPUs exclusively for infrastructure networking workloads (OVS-DPDK, OpenPErouter). Introduces two new PerformanceProfile API fields: spec.cpu.dedicated and spec.net.disableOvsDynamicPinning. Tracking: CNF-22582, RFE-8921 AIA Human-AI blend, New content, Human-initiated, Reviewed, Claude Opus 4.6 v1.0 Signed-off-by: Talor Itzhak <titzhak@redhat.com>

jmencak

Looks good to me overall. Have a couple of questions to improve my understanding of the problem and found a few nits.

jmencak · 2026-05-14T10:56:30Z

+
+### Non-Goals
+
+- Managing the lifecycle of OVS-DPDK processes themselves (PMD thread creation, DPDK EAL


I understand this is a non-goal, however, I'd like to understand how the OVS-DPDK processes run in OpenShift. So, they're not managed by kubelet at all? Do they run as regular userspace processes outside of OpenShift control?

So, they're not managed by kubelet at all?

They are not.

Do they run as regular userspace processes outside of OpenShift control?

AFAIU, yes.

The user will by pass the kernel networking stack and all infra-communication will be done through the DPDK talking directly to the NICs

I believe @MarSik can add more details and clarity about that

That is how I understand the feature as well. The whole idea is that this stack is a regular OS systemd controlled service and that is why they need a dedicated set of cpus isolated from from the OS and from OCP.

Add on different commit for clarity. Will squash once it'll be ready to merge. Signed-off-by: Talor Itzhak <titzhak@redhat.com>

openshift-ci · 2026-05-20T10:05:33Z

@Tal-or: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

jmencak · 2026-06-01T07:11:10Z

Thank you for the changes.
/lgtm

bartwensley · 2026-06-05T14:49:44Z

/cc @browsell

bartwensley

A few questions and comments - mostly regarding isolcpus=domain

bartwensley · 2026-06-05T14:17:23Z

+`spec.net.disableOvsDynamicPinning` to prevent OVN-Kubernetes from dynamically changing `ovs-vswitchd` and
+`ovsdb-server` processes' CPU affinity.
+When `dedicated` is set, the operator automatically configures full kernel-level
+isolation (`isolcpus=domain,managed_irq`, `nohz_full`, `rcu_nocbs`), adds the dedicated CPUs to


OpenShift doesn't use "domain" AFAIK so I don't think that should be part of the proposed enhancement. That affects several places in the document.

bartwensley · 2026-06-05T14:24:19Z

+
+- Provide a `dedicated` CPU set in the PerformanceProfile API that is fully excluded from Kubelet
+  scheduling (all QoS classes), OS daemons, and kernel housekeeping.
+- Automatically ban dedicated CPUs from irqbalance and configure `isolcpus=domain,managed_irq`


Isn't adding domain here going to disable scheduling for ALL isolated CPUs which will break application pods?

bartwensley · 2026-06-05T14:38:04Z

+   - A **MachineConfig** that:
+     - Does NOT include the OVS dynamic pinning trigger file (because `disableOvsDynamicPinning`
+       is `true`).
+     - Configures the irqbalance service with `IRQBALANCE_BANNED_CPUS` set to the hex mask of


The cri-o service updates IRQBALANCE_BANNED_CPUS to exclude isolated containers. Will it require any updates to account for the new dedicated CPUs?

bartwensley · 2026-06-05T14:39:18Z

+     and interrupt handling.
+   - CPUs 2-3,6-7 are isolated for application workloads (Guaranteed QoS pods).
+
+6. OVN-Kubernetes (or the network operator) starts OVS-DPDK and pins PMD threads to the


What component will be changed to do this pinning and how will it determine which CPUs to use?

bartwensley · 2026-06-05T14:43:52Z

+
+## Open Questions
+
+1. Should the API field be named `dedicated` or something more descriptive like


I think using dedicated is going to be confusing it sounds very similar to isolated (or reserved). I'd prefer more specific naming.

bartwensley · 2026-06-05T14:44:39Z

+   `infrastructureNetworking` or `dpdkCpus`? The current name is generic enough to support
+   future use cases beyond DPDK but may be too vague.
+
+2. Should `isolcpus=domain,managed_irq` be used together (both domain isolation and managed


As commented above, I don't see how it would be possible to use "domain" as it would apply to the isolated (i.e. application) CPUs as well.

bartwensley · 2026-06-05T14:46:56Z

+  - Hardware interrupts are not routed to dedicated CPUs (`/proc/interrupts` verification).
+  - No host processes are running on dedicated CPUs (`ps -eo pid,psr,comm` verification).
+
+### Integration Tests


I think a test is required to ensure that the IRQBALANCE_BANNED_CPUs is handled correctly when isolated containers (with the irq-load-balancing.crio.io annotation) are created and deleted.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 7, 2026

openshift-ci Bot requested review from Miciah and jmguzik May 7, 2026 11:39

Tal-or force-pushed the dedicate_cpus_for_dpdk_vswitch branch from 192566c to c941e7f Compare May 7, 2026 13:07

MarSik reviewed May 13, 2026

View reviewed changes

Comment thread enhancements/node-tuning/dedicated-cpus-for-dpdk-vswitch.md Outdated

MarSik reviewed May 13, 2026

View reviewed changes

Comment thread enhancements/node-tuning/dedicated-cpus-for-dpdk-vswitch.md Outdated

jmencak reviewed May 14, 2026

View reviewed changes

Tal-or force-pushed the dedicate_cpus_for_dpdk_vswitch branch 5 times, most recently from 9342e11 to 60e437d Compare May 19, 2026 17:08

05192026: address reviewers comments

b632c5c

Add on different commit for clarity. Will squash once it'll be ready to merge. Signed-off-by: Talor Itzhak <titzhak@redhat.com>

Tal-or force-pushed the dedicate_cpus_for_dpdk_vswitch branch from 60e437d to b632c5c Compare May 20, 2026 09:44

openshift-ci Bot assigned jmencak Jun 1, 2026

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 1, 2026

openshift-ci Bot requested a review from browsell June 5, 2026 14:49

bartwensley suggested changes Jun 5, 2026

View reviewed changes


		### Non-Goals

		- Managing the lifecycle of OVS-DPDK processes themselves (PMD thread creation, DPDK EAL


		## Open Questions

		1. Should the API field be named `dedicated` or something more descriptive like

Conversation

Tal-or commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented May 7, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci Bot commented May 7, 2026

Uh oh!

Tal-or commented May 7, 2026

Uh oh!

Tal-or commented May 7, 2026

Uh oh!

Uh oh!

Uh oh!

jmencak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Tal-or May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openshift-ci Bot commented May 20, 2026

Uh oh!

jmencak commented Jun 1, 2026

Uh oh!

bartwensley commented Jun 5, 2026

Uh oh!

bartwensley left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Tal-or commented May 7, 2026 •

edited

Loading

openshift-ci-robot commented May 7, 2026 •

edited by openshift-ci Bot

Loading

Tal-or May 19, 2026 •

edited

Loading