[BUG] Browser interaction can not support inputting Chinese/Korean

[walker-WSH]

Platform

Operating system and version: WIN10 (64bit)
OBS Studio version: 25.0.4 (64bit)

Expected Behavior

Can input Chinese/Korean by browser interaction

Current Behavior

Cann't input Chinese/Korean by browser interaction.
Even input method of system has been switched to Chinese, we can only input English in browser. There is no IME UI appears when I try to input Chinese.

Steps to Reproduce

1. add a browser source whose URL is www.google.com
2. open interaction window
3. switch input method to Chinese, try to input Chinese character in google.com

Additional information

Currently version of CEF used in obs is "75.1.16+g16a67c4+chromium-75.0.3770.100"。
I checked the matched CefClient in CEF solution, its interaction work normally while osr and shared handle are enable.
By the way, the input method I am using is provided by OS. Besides, this bug also reproduce with other third-party input methods, such as QQ input method, Sogou input method.

[WizardCM]

Hi there, I've moved the issue over from the obs-studio repository, as this is purely based on how we translate obs keys to Cef keys.

Could you please verify with the latest browser build as this PR #206 should've fixed the issue.

Edit: It looks like 25.0.8 should also contain the fix.

[walker-WSH]

Hi there, I've moved the issue over from the obs-studio repository, as this is purely based on how we translate obs keys to Cef keys.

Could you please verify with the latest browser build as this PR #206 should've fixed the issue.

Edit: It looks like 25.0.8 should also contain the fix.

Hi, it still exists in 25.0.8.
It seems that your commit link was fixing bug on Linux, not Windows.

[WizardCM]

Good to know. So, in this case, you're specifically looking for the IME UI to appear? It'll most likely require some custom triggers, so may not be easy to do. I'm not sure how well CEF & Qt will interact in this particular use case.

[walker-WSH]

Good to know. So, in this case, you're specifically looking for the IME UI to appear? It'll most likely require some custom triggers, so may not be easy to do. I'm not sure how well CEF & Qt will interact in this particular use case.

I just want to input non-English string.

Yes, there is trigger for IME window, but I have confirmed trigger about IME is open in OBS by ImmGetOpenStatus() or GetSystemMetrics()
https://docs.microsoft.com/zh-cn/windows/win32/intl/about-input-method-manager

For CefClient in osr mode, there are code for handling IME, such as OsrImeHandlerWin, OnImeCompositionRangeChanged, and WM_IME_SETCONTEXT/WM_IME_STARTCOMPOSITION...

But that code seems to be handling position of IME window, and inputting non-English still works normally even I remove these code from CefClient.

I have not found out the reason which causes difference between OBS and CefClient. Maybe we lost any code in OBS to enable this feature ~~

[walker-WSH]

Good to know. So, in this case, you're specifically looking for the IME UI to appear? It'll most likely require some custom triggers, so may not be easy to do. I'm not sure how well CEF & Qt will interact in this particular use case.

Hi, currently interaction window is not created in main thread of CEF, I guess this case causes user cannot input Korean/Chinese/Janpanese.

I have tested to create interaction window in CEF's main thread, in which we run CefRunMessageLoop(), then I can input Chinese/Korean/Janpanese normally.
Of course the interaction window is created by win32 API.

[WizardCM]

Hi @walker-WSH do you have some demo code of launching the interaction window in CEF's main thread that I could use to verify? Thanks!

[walker-WSH]

Hi @walker-WSH do you have some demo code of launching the interaction window in CEF's main thread that I could use to verify? Thanks!

Hi, the code can be found here:
https://github.com/naver/prismlivestudio/tree/master/src/obs/plugins/obs-browser/interaction
To check all code about interaction, you can search key word with "#3784"
Most code related with IME and offscreen-input are integrated from CefClient.
https://github.com/chromiumembedded/cef/tree/master/tests/cefclient

The interaction UI is developped in obs_browser plugin by WIN32 API, instead of by QT in UI project.
Interaction UI is running in main thread of CEF : INTERACTION_PTR interaction_ui

This can fix bug about inputting Chinese/Korean/Japan, but I don't known why.
I am just referring to CEF's demo to make sure the input events are inside CEF's main thread. That is the big difference between demo and OBS.

2021-08-13 10-30-57.zip