CefBrowserHost::Invalidate() only part of screen??

[chanman]

We have an application using CEF 3.2883.1542, and we are using off-screen rendering.

The performance of the application is suffering because there doesn't seem to be a way to invalidate part of the screen. The only API to invalidate the screen is CefBrowserHost::Invalidate():

Code: Select all

virtual void Invalidate(PaintElementType type) =0;

This causes OnPaint to be called with the dirtyRects parameter set to a single rectangle occupying the entire screen, forcing us to repaint the whole thing.

There used to be a way to invalidate only part of the screen, possibly in an earlier CEF version. For example, the 2011 code at http://magpcss.org/ceforum/apidocs/projects/(default)/cef_browser.h.html shows a prototype as follows:

Code: Select all

virtual void Invalidate(const CefRect& dirtyRect) =0;

How are we supposed to do a partial invalidate in CEF 2883 and onwards?

[magreenblatt]

Why are you trying to do a partial update? OnPaint will be called if the browser content changes. Keep a copy of the buffer passed to OnPaint and perform partial updates from that if necessary.

[chanman]

Because our application has an HTML user interface which has a custom, non-HTML video feed that we get from an external source. This video may have HTML elements overlaid on top of it, so that is why we need to use off-screen rendering to do our compositing. Whenever frames from this video feed arrive, we only want to invalidate the video area, not the entire page, because repainting the entire page is incredibly wasteful (like, 10x the necessary pixels).

The ability to invalidate only part of the screen used to exist, before it was removed in commit f6bc617bc5b58ed3cf2b84e9a5673f6baa7477e7. When off-screen rendering was re-implemented with commit dea4daffd71be5262cf42fa8f3ba6a93d0465c9f that ability was lost. Are there any plans to add it back in or is it something we would have to write?

Most people who use off-screen rendering are doing so because they need special control over the painting process. It seems like a pretty critical feature to be able to paint only the minimum number of pixels required. Your proposal to save the buffer passed to OnPaint would not work, because we need some way to drive the OnPaint thread, which causes painting to the screen. The only way to do that right now is Invalidate().

[amaitland]

Why are you calling Invalidate manually?

[chanman]

Because we have video frames coming in that are not HTML. The video frames are composited with the HTML UI in OnPaint. Therefore, if a new video frame comes in, we need to force CEF to invalidate because nothing else will have changed and no paint would have been triggered normally.

[HarmlessDave]

If this is Windows, what about using hWnd = browser->GetHost()->GetWindowHandle(); and WIn32 functions like Invalidate() or RedrawWindow()?

Oops, I missed the off-screen rendering part. We don't use OSR so the above works for us but is probably useless for you. See amaitland's suggestion below.

[amaitland]

Have you considered refactoring your code? What's stopping you from keeping a copy of the last frame CEF rendered and simply using that to composite your video onto?

[magreenblatt]

Have you considered refactoring your code? What's stopping you from keeping a copy of the last frame CEF rendered and simply using that to composite your video onto?

As amaitland suggests, this is what you should do. Asking the browser to call OnPaint again when nothing has changed is also wasteful.

[chanman]

Have you considered refactoring your code? What's stopping you from keeping a copy of the last frame CEF rendered and simply using that to composite your video onto?

Our off-screen rendering is based on the Windows CefClient example which uses OpenGL. In this code, CEF creates a window (hWnd), attaches OpenGL to the drawing context for that hWnd, and CEF's OnPaint is the sole function responsible for drawing on that window. Because I can't invalidate only part of the screen, you're suggesting I have some separate Invalidate/OnPaint on the browser's parent window, right? A Windows-native Invalidate/Onpaint (as opposed to CEF Invalidate/Onpaint)? Doesn't this cause all sorts of mutual exclusion problems because now you have a CEF thread trying to do its Onpaint and a Windows one trying to do the same thing at different intervals?

The most confusing part is that CEF used to be able to invalidate part of the screen in prior releases. Why is the functionality gone now? Is it just because you didn't have time to do it and it's coming back in a future release, or was it a deliberate design decision?

[chanman]

Have you considered refactoring your code? What's stopping you from keeping a copy of the last frame CEF rendered and simply using that to composite your video onto?

As amaitland suggests, this is what you should do. Asking the browser to call OnPaint again when nothing has changed is also wasteful.

In this case, something has changed. Just imagine a browser with a non-HTML video window occupying a subset of that 2D space. And that video window has HTML elements on top of it. What happens if the video frame changes? CEF still needs to redraw that portion because there are HTML elements on it. amaitland suggests keeping the last successful CEF (HTML) buffer somewhere else and do our compositing separately, but there is only one Invalidate/Paint loop owned by CEF. Is the suggestion really to have a separate Windows thread doing another Invalidate/Paint loop calling the same code? There is a lot of literature out there discouraging multithreaded OpenGL rendering!