CEF offscreen rendering with shared textures using skia

Having problems with building or using CEF's C/C++ APIs? This forum is here to help. Please do not post bug reports or feature requests here.

CEF offscreen rendering with shared textures using skia

Postby cmp4694 » Wed Mar 15, 2023 1:37 pm

Hi,

We have been using GL Renderer in our app in the company for "offscreen rendering to a hardware GL/D3D texture/surface provided by the client" on Windows and Mac OS. (https://github.com/chromiumembedded/cef/issues/1006).

Since the GL Renderer support is now deprecated from Chromium and in turn, in CEF from version 104, I have been trying to explore ways to use different renderer for the same. I explored the code and now confused on which path to take for latest CEF versions to achieve the similar things.

I have been exploring skia to achieve the same, but I am not able to find similarities on how to create and pass the surfaces as there is different handling in skia for creating surfaces. I am able to create a single surface to render the content and share the same to the client, but this has major performance impact and synchronisation issues with client app.

Could someone provide any ideas on how it could be achieved using different renderer on Windows and Mac both? any head start would be appreciated.
cmp4694
Newbie
 
Posts: 6
Joined: Tue Oct 01, 2019 4:59 am

Re: CEF offscreen rendering with shared textures using skia

Postby cmp4694 » Sat Mar 25, 2023 12:06 pm

@margreenblatt
Could you provide any update over this? What could be the best approach as GL renderer is now not available anymore?
cmp4694
Newbie
 
Posts: 6
Joined: Tue Oct 01, 2019 4:59 am

Re: CEF offscreen rendering with shared textures using skia

Postby doubleaa93 » Wed Mar 29, 2023 10:44 am

Im also looking into this.

Im using https://bitbucket.org/rrjksn/cefsharedt ... /src/5060/ as a starting point to then port it to latest. I have not tested this yet but in theory it should work.

Build that branch and, enable offscreen, shared texture, and skia rendering, and run it in debug and set some breakpoints

When using offscreen with shared texture and skia rendering chromium will use
Code: Select all
/ components / viz / service / display_embedder / skia_output_device_offscreen.cc


Look at
Code: Select all
void SkiaOutputDeviceOffscreen::EnsureBackbuffer()


In there it creates a backendTexture on every frame for skia to use in SkSurface* SkiaOutputDeviceOffscreen::BeginPaint which it will then draw to it

Since skia uses opengl only at this point ( uses angle for windows) then we must use opengl back texture.

Couple of possible solutions.
Use sharedimage to create a texture backed by dxgi (not sure how to do this one)
Use GpuMemoryBuffer, similar to how the current shared does.
Create the texture yourself using dxgi and pass a handle to the gpu processs.

I think if we go with GpuMemoryBuffer, we can create a gpumemorybuffer that is backed by dxgi texture and then create a gl texture through angle that uses our shared handle with eglCreatePbufferFromClientBuffer.
Once we have a created a GL texure we can then create a backend texture with SKIA that is backed with the gl texture we created with gl ( check https://github.com/microsoft/angle/wiki ... rectX-code on how to interop, there also some examples in chromium where they use this ( video decoding I believe).
After that then SKIA can render to that texture and we can signal it using MOJOM similar how the currrent OSR works (https://bitbucket.org/rrjksn/cefsharedt ... #lines-242)

Have not tested this, and all of this in theory. I will be trying this when I get some time, but let me know if you make some progress. I will share my progress here as well.
doubleaa93
Newbie
 
Posts: 4
Joined: Wed May 08, 2019 1:26 pm

Re: CEF offscreen rendering with shared textures using skia

Postby doubleaa93 » Wed Mar 29, 2023 12:08 pm

Also check for sharedimages
https://source.chromium.org/chromium/ch ... pv=1;bpt=1
This here creates a sharedimage for skia. If wired correctly if you follow the call path it should use the d3d11 image factory to create a shared image then call the produce skia which in then will create a passthrough GL image with backing

https://source.chromium.org/chromium/ch ... pv=1;bpt=1

Using the sharedImageFactory you can create a sharedimage that will pretty much do everything I mention in previous reply. This will create a ID3D11Texture2D and wire everything else

Look at https://source.chromium.org/chromium/ch ... pv=1;bpt=1 on how its implemented.
It creates a sharedImage
Code: Select all
auto representation = CreateSharedImageRepresentationSkia(
          ResourceFormat::RGBA_8888,
          gfx::Size(geometry.result_bounds.width(),
                    geometry.result_bounds.height()),
          color_space);


Then it does a beginaccess which internally it will try to aquire a shared handle through dxgi
Code: Select all
SkSurfaceProps surface_props{0, kUnknown_SkPixelGeometry};
      std::vector<GrBackendSemaphore> begin_semaphores;
      std::vector<GrBackendSemaphore> end_semaphores;

      auto scoped_write = representation->BeginScopedWriteAccess(
          /*final_msaa_count=*/1, surface_props, &begin_semaphores,
          &end_semaphores,
          gpu::SharedImageRepresentation::AllowUnclearedAccess::kYes);


You can then get the skia surfcace with scoped_write->surface() and pass use it in the skia offscreen renderer beginpaint method.
After that we need to flush the commands
After that it just a matter of notifying the main process that our texture is ready to be used on our own renderer and we can retrieve the handle send from IPC from mojo like it does right now. https://bitbucket.org/rrjksn/cefsharedt ... #lines-223

Let me know if this helps
doubleaa93
Newbie
 
Posts: 4
Joined: Wed May 08, 2019 1:26 pm

Re: CEF offscreen rendering with shared textures using skia

Postby cmp4694 » Mon Apr 10, 2023 2:46 am

Hi,
I don't have any major progress to give, but I am able to find the similar flow which you have mentioned.

    1. Create a gpumemorybuffer/shared image (using shared iamge factory) with dxgi shared handle or iosurface handle. For skia renderer, preferred is shared iamge as it is used with some of skia devices
    2. Use the shared image/texture to create backendTexture and backEndRenderTarget for skia. We might use the skia representation factory for this. This will be used to create skia surface on which we can render
    3. In the begin paint method, try to get the surface from the representation factory and draw on that surface.
    4. We can try to share the surface when after the rendering is done and SwapBuffers() function is called. (Here we need to get the texture information from the surface).

There are some points which I am trying to understand and solve:
    1. The skia_output_device_buffer_queue.h is not allowed to be used on windows. Trying to find a way it can be used for windows. [My progress: I don't know why it cannot be used on windows as there might be some dependency issues which I am not getting right now. So not able to solve that.]
    2. Need to create a shared image/texture for offscreen skia rendering approach i.e. Create a shared iamge in skia_output_device_offscreen.h and use it to create a skia surface and draw on the same surface. Later use the same image to share with client. [My progress: I am able to create shared image, but trying to understand how can I use skia representation class to get the surface out of it.]
    3. I am not sure how I would be able to share the same image to the client using the approach we mentioned using external_renderer_updater. Still trying to find a way to do the same. I am able to find that somehow output_presenter can be used, but not sure how as there are many other things associated with it. I am trying to check the following old CLs for the same. Link: https://chromium-review.googlesource.co ... /+/2236232 (This CL for the GL surface connection with shared image), https://chromium-review.googlesource.co ... /+/1647438 (This CL is to understand the dependencies)

Let me know if you have got any answers for the same. I will post here if I get answers to my own questions.
cmp4694
Newbie
 
Posts: 6
Joined: Tue Oct 01, 2019 4:59 am

Re: CEF offscreen rendering with shared textures using skia

Postby cmp4694 » Wed Apr 26, 2023 2:40 am

Hi @doubleaa93,

Were you able to make progress? I am still stuck in trying to identify the resolution to the issues I mentioned in previous post.
cmp4694
Newbie
 
Posts: 6
Joined: Tue Oct 01, 2019 4:59 am

Re: CEF offscreen rendering with shared textures using skia

Postby cmp4694 » Mon Jun 19, 2023 1:29 am

Hi,

Does anyone has any update to share on this? I am still stuck and not able to find a way to get this working. Any help is apprciated.

Thanks in advance!
cmp4694
Newbie
 
Posts: 6
Joined: Tue Oct 01, 2019 4:59 am

Re: CEF offscreen rendering with shared textures using skia

Postby aSurgingRiver » Tue Jul 04, 2023 8:03 am

Based on the above discussion ideas, I have found a method for generating SKCanvas on specific platforms such as DXGI. And through testing and verification, it can operate normally. The attachment is the complete code.I hope this code can reduce your workload.
Attachments
SkiaStudy.zip
Complete engineering
(91.11 KiB) Downloaded 467 times
微信图片_20230704205837.png
Related Codes
微信图片_20230704205837.png (109.23 KiB) Viewed 12746 times
aSurgingRiver
Newbie
 
Posts: 3
Joined: Fri Nov 25, 2022 10:02 pm

Re: CEF offscreen rendering with shared textures using skia

Postby aSurgingRiver » Tue Jul 11, 2023 10:15 pm

The code related to Opengl has been removed and shared handles cannot be obtained through Opengl. If we re add the previous opengl code, the workload will be significant. We need to find another way to obtain the shared handle
Last edited by aSurgingRiver on Wed Jul 19, 2023 8:50 pm, edited 1 time in total.
aSurgingRiver
Newbie
 
Posts: 3
Joined: Fri Nov 25, 2022 10:02 pm

Re: CEF offscreen rendering with shared textures using skia

Postby cmp4694 » Mon Sep 11, 2023 1:20 am

Hi aSurgingRiver,

Yes, you are right. we want to achieve the shared textures support using the skia renderer in offscreen mode, but it is a lot of work. We can use shared image factory to generate the textures, but using skia to render to it and sharing it with the main process using mojom is very complex to design/implement.

It would be great if we can find some solution for this. Waiting for a positive update while I am trying for the same.
cmp4694
Newbie
 
Posts: 6
Joined: Tue Oct 01, 2019 4:59 am

Next

Return to Support Forum

Who is online

Users browsing this forum: Google [Bot] and 36 guests