CEF offscreen rendering with shared textures using skia

[reito]

Why not create a mojo like FrameSinkVideoConsumerStub and name it FrameSinkTextureConsumerStub, and use the same design structure? The final texture is encoded into video anyway.

[cybersight]

Could Dawn / WebGPU help at all with this?

[reito]

Update: I'm goint to implement a method that supports macOS (via IOSurface) and Windows (via DXGI) using GMB buffers. My idea is implement another FrameSink that is dedicated to pop copyable resource up to the CEF layer, in parallel with existing VideoCapture method

[reito]

Update: I managed to pass the DXGI handle out. I'm working on the patch now. The codebase of chromium changes a lot in viz::service.

[reito]

It turns out the existing VideoCapturer can finish this job already. We just need to patch the RenderableGpuMemoryBufferVideoFramePoolImpl to support ARGB NativeTexture so that the GMB on Windows will use BufferFactoryDXGI as backend and it also has SHARED_KEYEDMUTEX | NTHANDLE attribute already.

[reito]

So after investigation the minimum patch will be:

video_capturer_->SetFormat(media::PIXEL_FORMAT_ARGB);
video_capturer_->Start(this, viz::mojom::BufferFormatPreference::kDefault);
changes to
video_capturer_->Start(this, viz::mojom::BufferFormatPreference::kPreferGpuMemoryBuffer);

and at the other side if the format is ARGB and preferGpuMemoryBuffer, we pass out the DXGI handle.

[reito]

I finished the implementation.

I also create a CL to chromium: https://chromium-review.googlesource.co ... /+/5265077
(But I got no hope it will get merged)

There're also some changes to the CEF: https://bitbucket.org/reitovovo/cef/src/master-osr/

My test shows it runs like expected.

[magreenblatt]

It looks like you linked the Chromium changes twice. What is the correct URL for the CEF changes?

[reito]

It looks like you linked the Chromium changes twice. What is the correct URL for the CEF changes?

Sorry, link updated.

Also, this method could be easily port to macOS because on mac the backing factory use IOSurface. We only need to add some mac specific code at CEF (OnFrameCaptured) handler

[magreenblatt]

Thanks, looks like the specific CEF commit is https://bitbucket.org/reitovovo/cef/com ... 570dba346e