Hardware offscreen render support

Made a cool mod to CEF that you'd like to share with others? Discuss CEF modifications here, and view modifications that others have made.

Hardware offscreen render support

Postby Assert » Mon Apr 07, 2014 1:36 am

Main target of this patch, is ability to use a result of hardware offscreen render in CEF client applications without any transfer to system memory. By default, cromium use 3 processes, browser, render and GPU. Hardware painting happens in GPU process. If we want to use a result of this painting in our application, we must use some context\surface sharing mechanism or run GPU command processing in browser process. Sharing is possible for D3D from Windows Vista and for OpenGL in platforms with glXImportContextEXT. Cromium implementation supports natively only D3D sharing, but my patch can be expanded to add support of full sharing on other platforms. Cromium can run GPU command processing in browser process if --in-process-gpu flag was specified, and it is enough stable to use it, opposite to --single-process flag. I am implemented path for D3D in separate GPU process, and OpenGL path for browser GPU command processing. By default, on windows, D3D through EGL and ANGLE were used, but this can be switched to OpenGL by --use-gl=desktop. I think, that most comfortable way, is to use platform specific handle to some object in CEF API, shared texture handle for D3D, HDC of PBUFFER on windows, etc. because Cromium code is strongly platform specific in accelerated drawing functions, and this can give us some freedom for modifications and adding new platforms. Main idea of implementation, is to add new surface type and handle it like regular draw to window, but connect it to texture or PBUFFER at end point. This can be achieved using CreateOffscreenGLSurface instead of CreateViewGLSurface in ImageTransportSurface::CreateNativeSurface.

Here is a brief description of my patch:

Modifications in CEF:
- Added OnAcceleratedPaint in CefRenderHandler.
- Added support of OnAcceleratedPaint in CefRenderWidgetHostViewOSR.
- Removed force accelerated_compositing=STATE_DISABLED in CEF offscreen mode.
- OSR rendering path in cefclient modified for OpenGL hardware acceleration support. Just run with -off-screen-rendering-enabled -in-process-gpu -use-gl=desktop.

Modifications in cromium:
- Added gfx::NATIVE_SURFACE to SurfaceType enum.
- GpuProcessHost modified for gfx::NATIVE_SURFACE and AcceleratedSurfacePostSubBuffer support.
- GpuSurfaceTracker modified for gfx::NATIVE_SURFACE support.
- PassThroughImageTransportSurface modified for gfx::NATIVE_SURFACE support. Some special handling for gfx::NATIVE_SURFACE were introduced (gfx::GLSurfaceAdapter::SwapBuffers skip, share handle transition, glFinish call to prevent flicker and scheduling disable during surface processing in browser).
- ImageTransportSurface::CreateNativeSurface modified on win and linux for gfx::NATIVE_SURFACE support. MAC support must be handled in more special way, I will add this handling, and support of separate GPU process on linux later.
- AddGenericPolicy in sandbox_win modified for --in-process-gpu flag support by default configuration.
- Added resizing support and ShareHandle return in PbufferGLSurfaceWGL.

TEST:
cefclient -off-screen-rendering-enabled -in-process-gpu -use-gl=desktop
Some WebGL demos for performance test:
http://alteredqualia.com/three/examples ... namic.html
http://carvisualizer.plus360degrees.com/threejs/
This is a path for Desktop OpenGL rendering on windows. Everything works OK, and very fast for me including Flash and WebGL.
D3D path is not tested but should work. OpenGL path for Linux is not tested but should work. MAC is not supported for now at all.

TODO:
Test transparency.
Test on compatibility issues.
Test D3D on Windows and OpenGL on Linux.
Add MAC OpenGL support.
Add support for GPU in separate process (MAC and Linux).
Add unittest for OnAcceleratedPaint.

It will be great, if someone help me to finish and test all features.
Attachments
OnAcceleratedPaint.zip
(3.36 KiB) Downloaded 743 times
Assert
Techie
 
Posts: 11
Joined: Mon Apr 07, 2014 1:10 am

Re: Hardware offscreen render support

Postby Assert » Mon Apr 07, 2014 10:17 am

Hardware popup drawing is not compatible with existing cefclient infrastructure, but should work if handled as separate texture.

Latest version with patch for cromium and CEF.
Attachments
OnAcceleratedPaint_CEF.zip
(4.71 KiB) Downloaded 1054 times
OnAcceleratedPaint_cromium.zip
(3.37 KiB) Downloaded 924 times
Assert
Techie
 
Posts: 11
Joined: Mon Apr 07, 2014 1:10 am

Re: Hardware offscreen render support

Postby magreenblatt » Mon Apr 07, 2014 10:25 am

The legacy rendering path represented by RenderWidgetHostView::Paint and AcceleratedSurfaceBuffersSwapped will be going away in the near future. The recommended approach currently is to use the software compositor. Quoting from https://groups.google.com/a/chromium.or ... SJ7xTl3c0J:
The software compositor is currently in a bit of flux. Once https://codereview.chromium.org/25942002/ is checked in you'll be able to hook into the SoftwareFramebuffer. However, for aura we'll probably be switching over to delegated rendering/ubercompositor by default, so that will change how all this behaves. It might be best to create a RenderWidgetHostViewFrameSubscriber or similar to capture the frame from the RenderWidgetHostViewAura; that abstracts out many of the details and gives back a media::VideoFrame.

Today the the Aura implementation is no longer using AcceleratedSurfaceBuffersSwapped -- see render_widget_host_view_aura.cc. We should explore how to use the software compositor and/or delegated rendering/ubercompositor in CEF instead of adding more functionality that requires the legacy rendering path.
magreenblatt
Site Admin
 
Posts: 9365
Joined: Fri May 29, 2009 6:57 pm

Re: Hardware offscreen render support

Postby Assert » Mon Apr 07, 2014 11:34 pm

I read this, and play with aura code before doing my implementation. I hope that I have understanding how rendering engine works, and what will going in near future... but i need HW implementation even yesterday. As simple as possible. At synced old revision of cromium in CEF, because i can't play with new cromium. I have no idea when new sync will be made and how croimum will handle painting at this point. I have no idea about "how to use the software compositor and/or delegated rendering/ubercompositor in CEF". Offscreen rendering and composition of whole page is not a simple feature of cromium. But I know, if we can draw to window, we always can draw to PBUFFER in place of that window. If this is multiplatform system, we always can add new platform with this virtual window. This is exactly what i am doing, AcceleratedSurfaceBuffersSwapped and other things, just only existing infrastructure to send shared handle of that PBUFFER back to browser. We can define own IPC instead of using AcceleratedSurfaceBuffersSwapped, redefine ImageTransport and make own gfx::GLSurface... i think this is stable solution that can be ported to any infrastructure changes. Attach to endpoint and send result back in 'some way'. Above all, we need raw system-dependent handle of surface in GPU to integrate cromium and other hardware accelerated applications, this always will be dirty hack of crossplatform abstraction... and i think it is no matter where we do this exactly.
Assert
Techie
 
Posts: 11
Joined: Mon Apr 07, 2014 1:10 am

Re: Hardware offscreen render support

Postby magreenblatt » Tue Apr 08, 2014 11:58 am

What Chromium revision are you using? While I sympathize with your needs we won't be adding this functionality to older versions of CEF. It needs to work with the current CEF trunk which uses Aura on Windows and (soon) on Linux. Maintaining custom changes to Chromium is a lot of work, so any such changes need to be minimized as much as possible. For example, if using the software compositor supports all necessary functionality and requires minimal or no Chromium changes we should explore that approach first.
magreenblatt
Site Admin
 
Posts: 9365
Joined: Fri May 29, 2009 6:57 pm

Re: Hardware offscreen render support

Postby Assert » Wed Apr 09, 2014 5:39 am

This patch is for current CEF trunk rev 1640, and cromium 251746 as in CHROMIUM_BUILD_COMPATIBILITY.txt
I do not find a way how software compositor can be used without any significant changes in CEF or cromium at this revision. Without any changes to cromium, we can't create shareable GPU offscreen surface. Composition engine in Aura uses unshareable FBO binded textures (links on textures with some virtual id's, that can't be used directly) that lives in GPU process behind IPC all the time. Composition engine issue commands to GPU process for drawing this textures on the native window surface. No other options, and i do not know how to use this.
Assert
Techie
 
Posts: 11
Joined: Mon Apr 07, 2014 1:10 am

Re: Hardware offscreen render support

Postby jarrednicholls » Wed Apr 09, 2014 8:50 am

As Marshall has said, this definitely should be forward compatible with the latest rendering path in Chromium trunk. I know all too well how painful it is to maintain differences with Chromium trunk. Touching as little of that surface area as possible goes a long way. Any changes to trunk should be good enough to be upstreamed, and we can't upstream modifications to the old rendering path.

This is really excellent though, and I'd like to help in any way I can.
jarrednicholls
Newbie
 
Posts: 6
Joined: Wed Dec 08, 2010 2:51 pm

Re: Hardware offscreen render support

Postby Assert » Wed Apr 09, 2014 12:43 pm

As i see diff with latest chromium trunk, there is no changes or conflicts in used infrastructure at all. Great revolution in low level rendering was completed at 247312. I do not think that gfx::GLSurface will change interface in near or far future, and i think that NATIVE_DIRECT will be there for long time. It means that mimics of NATIVE_DIRECT (this is my solution) will work in future. Transport functions for gfx::GLSurface::SwapBuffer, such as AcceleratedSurfaceBuffersSwapped, can be rewrited on own IPC in 30 minutes, if it thrown away... as soon as CEF will be synced at new cromium revision, i will make new patch. Maybe more compatable for upstream modifications (but i realy can't see any problems now for maintaining differences), and of course compatible with new CEF software OSR. I do not ask some one to integrate my patch in trunk, i just want to share my results because of lack ANY solution at this time.
Assert
Techie
 
Posts: 11
Joined: Mon Apr 07, 2014 1:10 am

Re: Hardware offscreen render support

Postby magreenblatt » Wed Apr 09, 2014 12:53 pm

Assert wrote:as soon as CEF will be synced at new cromium revision, i will make new patch. Maybe more compatable for upstream modifications (but i realy can't see any problems now for maintaining differences), and of course compatible with new CEF software OSR. I do not ask some one to integrate my patch in trunk, i just want to share my results because of lack ANY solution at this time.

OK, that's fair, and thank you for sharing your work :). I'd like to hold off on integrating functionality like this into the main CEF repository until after CEF has moved to the new OSR implementation. Other methods like RenderWidgetHostView::Paint are going away as well, which means we will need to re-work a lot of the existing code.
magreenblatt
Site Admin
 
Posts: 9365
Joined: Fri May 29, 2009 6:57 pm

Re: Hardware offscreen render support

Postby Assert » Wed Apr 16, 2014 8:39 am

D3D and OpenGL path tested on windows with real complex application. Sharing and transparency works as expected. Intermediate GL rendering context created with PBUFFER HDC may be required in client side for proper sharing due to different pixel formats. Be careful about syncronization with main render thread, do not return OnAcceleratedPaint until data from shared resource was copied, and do not try to copy this data while rendering something on main context\device. Just suspend OnAcceleratedPaint call while main rendering thread reach point where data can be safely copied.

I think it is better to wait new CEF sync point and only then continue work on hardware offscreen render support. Current patch is stable and can give all functionality for windows and may be linux. Please share information about new offscreen painting system for CEF here (when it will be in development), if this is not very difficult, and may be ideas how hardware support must be integrated with it.
Assert
Techie
 
Posts: 11
Joined: Mon Apr 07, 2014 1:10 am

Next

Return to Modifications Forum

Who is online

Users browsing this forum: No registered users and 7 guests