Controlling CEF with OSC

[marcusstenbeck]

I'm trying to use OSC to trigger browser events in CEF. Think of OSC as similar to a MIDI signal, but it's usually transferred over UDP. So, basically, I want to create a UDP socket that listens on a particular port, and then can pass the signal on to CEF.

I have gotten the UDP socket connection to work outside of CEF, but it does block everything else while listening. I assume that running this on one of the existing CEF threads is a bad idea (or is it?), so I've been researching yesterday and today how I would create a thread within CEF where the socket listener could live.

So far, looking into Chromiums own thread creation facilities seemed like a viable option, but I'm inclined to think that there may be a better option already built into CEF that I'm not aware of. Any tips would be appreciated!

[magreenblatt]

I have gotten the UDP socket connection to work outside of CEF, but it does block everything else while listening. I assume that running this on one of the existing CEF threads is a bad idea (or is it?)

Correct, blocking an existing CEF thread would be a bad idea. Have you considered using a non-blocking socket API?

So far, looking into Chromiums own thread creation facilities seemed like a viable option, but I'm inclined to think that there may be a better option already built into CEF that I'm not aware of. Any tips would be appreciated!

CEF does not currently expose Chromium's thread creation routines. You can instead use OS functions or a library like boost to create threads.

[marcusstenbeck]

Have you considered using a non-blocking socket API?

This is probably the best option going forward. Is there a good place to integrate this message checking into CEF, e.g. where can I put it to check it in-between every frame "tick" (or often enough)?

I'm new to the CEF world, and I've been away from C++ dev for a while, so please feel free to course-correct me if I seem like I'm going down strange paths.

You can instead use OS functions or a library like boost to create threads.

This is how I've solved it currently (with pthread).

[magreenblatt]

If the API is non-blocking and completes quickly then you can run it on the browser process IO thread.

[marcusstenbeck]

I've done some more work, and I've gotten non-blocking working with the library I'm using. Now to integrate it into CEF.

After poking around I see that the CefBrowserHost is responsible for "triggering" stuff in the browser, like mouse movement through SendMouseMoveEvent(). What I'd like to do is have one socket connection per browser, so this leads me to believe I should first extend a class CefBrowser to have that socket information and/or connection as some type of member data. Then, to send messages I'm thinking I'll extend a class CefBrowserHost that supports a method that sends the data, e.g. something along the lines of SendOSCMessage(). SendOSCMessage() will then post this into the IO thread as you suggested.

For the polling of the sockets … where would I put the code that should be run once every tick? Is there somewhere in the CEF code I could see how that kind of thing has been implemented for other things that need to happen each tick?

Thanks!

[magreenblatt]

What specifically are you trying to do? If your goal is to control the browser from an external source (navigate, send mouse/keyboard events, etc) then there should be no need to modify the libcef source code. Just use the existing APIs to control browser behavior.

[marcusstenbeck]

I'd like to expose a JavaScript function that can be used by the page to send messages, e.g. something along the lines of sendOSC("message"). I guess this is technically extension of what I was trying to initially do.

[magreenblatt]

You can do that using JavaScript bindings and then send IPC messages to/from the browser process. See the JavaScriptIntegration and GeneralUsage Wiki pages.

[marcusstenbeck]

Thank you. The JavaScript bindings solved what I was trying to do with outbound messages. Super excited to see these OSC messages flying out from JavaScript. Thanks a lot!

As for inbound messages ... I don't know if I'm asking the right question. I still haven't grasped from where I would call a pollSocket() type of function. The solution may very well be in the answers you've already given me, but I would appreciate it very much if you could clarify where to hook that in. Would I register some task somewhere and then once it's done register a new task of the same type? In the cefclient app ... would that possibly be in some type of OnCreated/OnInitialized/OnReady?

[magreenblatt]

What is the content of the incoming OSC message? What do you want to do with it?