frontend: Disable hotkeys when typing or in modal dialog

[cg2121]

Description

This disables hotkeys when an user is typing in a editable widget or a modal dialog is open.

Motivation and Context

Simpler version of #3072 (was reverted) and it automatically works with all widgets.

Fixes #7824

How Has This Been Tested?

Focused a editable widget and made sure the hotkeys didn't trigger. Also unfocused the same widget and made sure the hotkeys worked again. Also made sure hotkeys didn't trigger when opening a modal dialog.

Tested on Ubuntu 22.04 and Windows 10.

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[RytoEX]

How did you test this? What operating systems did you test on? IIRC, focus events don't work the same on each OS, so it would be good to confirm that this behaves as intended on each.

[cg2121]

This has now been tested on Linux and Windows without any problems. It just needs testing on macOS.

[cg2121]

Updated PR to also disable hotkeys when a modal dialog is open.

[PatTheMav]

So what is the underlying root cause of the issue? Because it doesn't seem like this was documented/explained anywhere (neither in this PR nor the original one).

When OBS is in focus, hotkey management should be handled by Qt's own event loop and keyboard event handling, so are the key events not consumed by the editable widget? Are we intercepting the event before that and trigger the hotkey action before it's consumed? Wouldn't it be easier to only consume key events that have not been consumed by a widget ("leftover" events) and pass those to the hotkey system then?

[cg2121]

The main issue is that a user doesn't want a hotkey to trigger when they are typing in a editable widget.

This PR, in my opinion, is simpler than using the event filter because it works for every widget. We would need to install an event filter to every single dialog and wouldn't work for plugins that have their own dialogs. At least that is my understanding of how the event filter works.

[cg2121]

Updated to latest master

[RytoEX]

This has merge conflicts.

[PatTheMav]

While this potentially prohibits hotkey handling while a QFileDialog is open on Windows, it might not do so on macOS where QFileDialog uses a native dialog window and thus no editable QWidget's receive focus (indeed WidgetFocusChanged is called, but now is a nullptr).

In that case the hotkey would still be handled by libobs and a scene change event triggered (which updates the context bar and thus destroys the widgetry the file dialog belongs to from within its nested event loop).

[Warchamp7]

@cg2121 I think it would be better to revive #9981 and instead make that class a QObject that takes a parent. Then it can be tied to either function scope or another widget's lifetime.