Allow post-install `kargs.d` updates

[RoyalOughtness]

If /usr/lib/bootc/kargs.d is updated in the image, kargs should also be modified by bootc. Currently this only happens at install time.

This would require some kind of mechanism to keep track of which kargs are set at runtime by the user (rpm-ostree kargs) and which are set by kargs.d (so that they can be adjusted when kargs.d files change).

[LukeShortCloud]

I was surprised to hear this. I assumed those would be applied after any bootc upgrade or bootc switch.

This would be a very useful feature to automatically roll out kargs changes (remove no longer needed kargs thanks to kernel updates, add new kargs for workarounds, add new kargs for performance tweaks, etc.).

[andrewdunndev]

We use kargs.d on CentOS Stream 10 bootc images for console configuration (console=tty0 console=ttyS0,115200n8, order matters when NVIDIA takes the framebuffer) and kernel module parameters. For context, we build NVIDIA drivers from source into our bootc images:
https://gitlab.com/dunn.dev/immutable/basec/-/blob/main/images/zfs-nvidia/Containerfile

It looks like the core ask here, applying kargs.d during upgrades, was implemented in #1783. We've been testing with that behavior and it works as expected for our use case.

Is this issue resolved by #1783, or is there remaining scope around /etc/bootc/kargs.d or other aspects that still need work?