Sudden crash after days of running

[StanB123]

I am not sure if running a CEF application for days without restarts is among the goals of this project. But I seem to have some problems doing it.

I am running CEF 91.1.21+g9dd45fe+chromium-91.0.4472.114 on Windows 10. My web application runs pretty smoothly in my CEF wrapper for days on end, and then suddenly both the GPU and network processes crash, causing a white screen. Here's some of the logging that occurred after 2 days and 2 hours of running:

Code: Select all

[0814/084020.722:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084020.800:ERROR:gpu_process_host.cc(1002)] GPU process exited unexpectedly: exit_code=1073807364
[0814/084020.800:WARNING:gpu_process_host.cc(1316)] The GPU process has crashed 1 time(s)
[0814/084021.316:ERROR:gpu_process_host.cc(1002)] GPU process exited unexpectedly: exit_code=-1073741205
[0814/084021.316:WARNING:gpu_process_host.cc(1316)] The GPU process has crashed 2 time(s)
[0814/084021.316:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084021.394:ERROR:gpu_process_host.cc(1002)] GPU process exited unexpectedly: exit_code=-1073741205
[0814/084021.394:WARNING:gpu_process_host.cc(1316)] The GPU process has crashed 3 time(s)
[0814/084021.410:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084021.488:ERROR:gpu_process_host.cc(1002)] GPU process exited unexpectedly: exit_code=-1073741205
[0814/084021.488:WARNING:gpu_process_host.cc(1316)] The GPU process has crashed 4 time(s)
[0814/084021.503:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084021.566:ERROR:gpu_process_host.cc(1002)] GPU process exited unexpectedly: exit_code=-1073741205
[0814/084021.566:WARNING:gpu_process_host.cc(1316)] The GPU process has crashed 5 time(s)
[0814/084021.582:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084021.660:ERROR:gpu_process_host.cc(1002)] GPU process exited unexpectedly: exit_code=-1073741205
[0814/084021.660:WARNING:gpu_process_host.cc(1316)] The GPU process has crashed 6 time(s)
[0814/084021.688:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084021.763:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084021.827:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084021.890:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084021.952:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.015:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.077:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.124:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.187:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.249:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.312:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.374:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.437:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.499:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.562:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.624:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.687:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.749:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.812:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.874:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.937:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084022.999:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084023.062:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084023.124:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084023.187:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084023.249:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084023.312:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084023.374:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084023.437:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084023.484:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.
[0814/084023.546:ERROR:network_service_instance_impl.cc(331)] Network service crashed, restarting service.

The application also reports a problem with the GPU when it starts, but I doubt it is related because it takes multiple days before this crash occurs (between 2 and 5). Just to be complete, this is the logging for that problem.

Code: Select all

[0812/095626.796:ERROR:angle_platform_impl.cc(44)] SwapChain11.cpp:627 (reset): Could not create additional swap chains or offscreen surfaces, HRESULT: 0x887A0001
[0812/095626.797:ERROR:gl_surface_egl.cc(780)] EGL Driver message (Critical) eglCreateWindowSurface: Bad allocation.
[0812/095626.797:ERROR:gl_surface_egl.cc(1508)] eglCreateWindowSurface failed with error EGL_BAD_ALLOC

I have allready investigated if my web app might be leaking memory, but this not seem to be the case. Memory usage stays well within the expected window so I don't think this will be the cause of the problem. But I understand that it's difficult to take my word for it, and I could be wrong.

My current solution is to reboot the system every night, but I would prefer not to if it all possible. Could anyone give me some pointers as to what I am doing wrong? It would be much appreciated.

I am also considering turning off GPU acceleration if it helps, but having it was one of the reasons for me to move to CEF. However, my application does not depend upon it, so it is definitely an option.

[magreenblatt]

Are you using off-screen or windowed rendering? Have you tested with Google Chrome at the same version?

[StanB123]

I am using Windowed rendering. I have not run the application in Chromium for an extended period yet. But I will give it a try. Thanks.

[StanB123]

Okay, so I have been running the Chromium version that matches my CEF version for almost a week now, and no crash has occurred.

Some observations:

• hardware acceleration is turned on according to the Chromium chrome://gpu page
• the chrome-debug.log mentions just one error: gpu_init.cc(440)] Passthrough is not supported, GL is disabled
  
  • I do not see this occur in my CEF application, instead I see some GL bad alloc error (see the opening post)
• the chrome://gpu page mentions the workaround: "Applied Workarounds: disable(GL_KHR_blend_equation_advanced), disable(GL_KHR_blend_equation_advanced_coherent)"

I think, this problem could very well be related to my issue. I am not quite sure though.

I am considering turning off hardware acceleration all together, but if I could just get away with using the workaround mentioned above that might be better. Can anyone tell me if they think I am on the right path, and if there is a way just to use the workaround?

[magreenblatt]

How does the chrome://gpu output compare in CEF and Chrome?

[StanB123]

The outputs are much more different than I expected. I have attached them. Both tests were ran on the same Windows 10 machine. If you could take a look that would be much appreciated!

For some reason the CEF app does not use hardware acceleration for compositing. Which seems to have effect on other settings being disabled, and workarounds being triggered.

There is also something about anglebug:3682, where my CEF app seems to think I am not using Windows 10, but Chromium does. Chromium is correct here, I don't understand why CEF is not. Maybe I am not targetting Windows 10 properly.

I am also doubting whether or not maybe my Chromium version is slightly different than my CEF version, because it shows some very minor version differences for Angle (CEF's version is slightly newer, 2.1.15371 vs 2.1.15375).

[magreenblatt]

Is your manifest file correct? See https://stackoverflow.com/questions/229 ... upportedos

[StanB123]

Is your manifest file correct? See https://stackoverflow.com/questions/229 ... upportedos

Correcting it seems to get rid of the GL Bad Aloc error. And the outputs of chrome://gpu appear to be nearly identical now so that appears to be good news.

Thanks for that. I am going to try and give it a couple of days of uptime to see what happens now .

[StanB123]

I am no longer having the problems I had before. And in general things seem to be running a lot more smoothly.

One day this week I ended up with a blank screen early in the morning - when no one was using the machine - with the following log fragments repeating constantly for about a minute:

Code: Select all

[0829/064016.908:ERROR:shared_image_representation.cc(207)] Attempt to read from an uninitialized SharedImage
[0829/064016.959:ERROR:shared_image_manager.cc(214)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.

But I think this is probably much more Chromium related than CEF related. I am probably just going to restart the program once a day or use --disable-gpu, unless anyone has a brilliant idea on how to solve this one.

Thanks.