Crash during CefShutdown in WebRequestAPI::Shutdown()

[dam4rus]

Hi!

Cef is randomly crashing during CefShutdown. We couldn't really tell what the cause could be. It crashes on some machines, but works fine on others. It doesn't matter which Windows version we try it on. Any help is appreciated.
Cef version: 3.3464.1794.g669e867_windows32

Call stack:

Code: Select all

    libcef.dll!extensions::EventRouter::UnregisterObserver(extensions::EventRouter::Observer * observer) Line 238   C++
   libcef.dll!extensions::WebRequestAPI::Shutdown() Line 473   C++
    libcef.dll!KeyedServiceFactory::ContextShutdown(base::SupportsUserData * context) Line 114   C++
    libcef.dll!DependencyManager::DestroyContextServices(base::SupportsUserData * context) Line 91   C++
    libcef.dll!CefBrowserContext::Shutdown() Line 91   C++
    libcef.dll!CefBrowserContextImpl::~CefBrowserContextImpl() Line 227   C++
    libcef.dll!CefBrowserContextImpl::~CefBrowserContextImpl() Line 216   C++
    libcef.dll!CefBrowserContextImpl::RemoveCefRequestContext(CefRequestContextImpl * context) Line 336   C++
    libcef.dll!CefRequestContextImpl::~CefRequestContextImpl() Line 149   C++
    [External Code]   
    libcef.dll!content::BrowserThread::DeleteOnThread<content::BrowserThread::UI>::Destruct<CefImageImpl>(const CefImageImpl * x) Line 209   C++
    [External Code]   
    libcef.dll!CefBrowserMainParts::PostMainMessageLoopRun() Line 225   C++
    libcef.dll!content::BrowserMainLoop::ShutdownThreadsAndCleanUp() Line 1009   C++
    libcef.dll!content::BrowserMainRunnerImpl::Shutdown() Line 226   C++
    libcef.dll!CefMainDelegate::ShutdownBrowser() Line 655   C++
    libcef.dll!CefContext::FinalizeShutdown() Line 541   C++
    libcef.dll!CefShutdown() Line 257   C++

Exception: Access violation reading location 0x00000124

So we suspect that EventRouter is null

Thanks in advance!

[ndesktop]

If is EventRouter, it cannot be null, since AV reports 0x00000124. Most likely EventRouter (this) is destroyed. But without a crash dump I can't tell.

[dam4rus]

I can confirm that EventRouter is null. 0x0124 is the offset of this->observers_ which it tries to access inside the method. The function i'm referring to is `void EventRouter::UnregisterObserver(Observer* observer)` in file `extensions/browser/event_router.cc`

[ndesktop]

Ah, yes. Then is a classic error of Unregister called after delete. So the EventRouter object might be destroyed in one thread and Unregister called on another (or at least I imagine so, I don't know this area).
Does this appear in cefclient? It might be something in your specific implementation.

[dam4rus]

Sadly, it's really hard to reproduce on our machines. So i can't really confirm that the crash appears in cefclient. We collect crash dumps from our users and we get around 10-11 crashes/day with a user base of around 3000. Not all of the uses our browser client mind you. If it's something that can be caused by multithreading is it possible that our machines are just too fast?

The only big difference from the previous build we used, which worked fine, is that we enabled proprietary codecs. If you can give me a pointer of what can cause this problem, i will gladly help.

[ndesktop]

Not sure how codecs are related to EventRouter (but it might).
Without a crash dump with full symbols, best bet would be a search on Chromium issues list (although the lifetime and loop are probably totally different).

But looking on the stack trace it seems all the action is on UI thread, so I'd say the EventRouter object is deleted on another thread.

[ndesktop]

This one looks similar, although it is quite old, I see it is still in the Assigned status.

[dam4rus]

Yeah, this looks really similar. So this is a bug in Chromium and not CEF? Should we just wait for them to fix it or is there a workaround? Maybe disabling extensions?

[ndesktop]

Looks like a Chromium bug, but I cannot be sure.

If there is no other solution, you might need a modified libcef to handle this error.
I know I had to do this to avoid shutdown leaks, for example (URL requests DCHECK on shutdown plagued me for years).

[magreenblatt]

Looks like a Chromium bug, but I cannot be sure.

If there is no other solution, you might need a modified libcef to handle this error.
I know I had to do this to avoid shutdown leaks, for example (URL requests DCHECK on shutdown plagued me for years).

Did you fix the leaks, or just remove the DCHECK? If you fixed the leaks that would be a welcome PR