Page 1 of 1

Troubleshoot renderer crash on facebook

PostPosted: Tue Sep 08, 2020 10:14 am
by edgardog
Hi, I have an issue uploading a picture to facebook, where the renderer process dies (on cef client it basically opens google again).
I'm using linux64 (Ubuntu 16) and downloaded the latest CEF automated build to test ( cef_binary_85.3.1+g1306235+chromium-85.0.4183.83_linux32.tar.bz2 )
I also have this problem on my own build from source and also on an older branch.
I ran cefclient, enabled trace, tried to make a facebook post (with a picture) and as soon as the picture loads browser goes back to google.com.. so I stopped the trace and I notice a message saying :

Code: Select all
{"args":{},"cat":"renderer.scheduler","id2":{"local":"0x24a904a5cb40"},"name":"ShortIdlePeriod","ph":"b","pid":25607,"scope":"renderer.scheduler","tid":1,"ts":146847453424},
{"args":{},"cat":"cc","id2":{"local":"0x24a904b7f870"},"name":"ScheduledTasks","ph":"b","pid":25607,"scope":"cc","tid":7,"ts":146847480038},
{"args":{"state":{"tasks_pending":{"activate_tile_tasks_completed":false,"all_tile_tasks_completed":false,"draw_tile_tasks_completed":false}}},"cat":"cc","id2":{"local":"0x24a904b7f870"},"name":"ScheduledTasksState","ph":"n","pid":25607,"scope":"cc","tid":7,"ts":146847480062},
{"args":{"state":{"tasks_pending":{"activate_tile_tasks_completed":false,"all_tile_tasks_completed":false,"draw_tile_tasks_completed":false}}},"cat":"cc","id2":{"local":"0x24a904b7f870"},"name":"ScheduledTasksState","ph":"n","pid":25607,"scope":"cc","tid":7,"ts":146847480301},
{"args":{"state":{"tasks_pending":{"activate_tile_tasks_completed":true,"all_tile_tasks_completed":false,"draw_tile_tasks_completed":false}}},"cat":"cc","id2":{"local":"0x24a904b7f870"},"name":"ScheduledTasksState","ph":"n","pid":25607,"scope":"cc","tid":7,"ts":146847480318},
{"args":{},"cat":"cc","id2":{"local":"0x24a904b7f870"},"name":"ScheduledTasks","ph":"e","pid":25607,"scope":"cc","tid":7,"ts":146847480333},
{"args":{"params":{"byte_count":30},"source_type":"SOCKET"},"cat":"netlog","id":"0x3d1","name":"SOCKET_BYTES_SENT","ph":"n","pid":25606,"scope":"netlog","tid":25621,"ts":146847662657},
{"args":{"params":{"byte_count":8},"source_type":"SOCKET"},"cat":"netlog","id":"0x3d1","name":"SSL_SOCKET_BYTES_SENT","ph":"n","pid":25606,"scope":"netlog","tid":25621,"ts":146847662670},
{"args":{"browser_context":"0x387bf2f9cb00","render_process_host":"0x387bf37ed600"},"cat":"shutdown","id2":{"local":"0x387bf37ed600"},"name":"Cleanup in progress","ph":"b","pid":25573,"scope":"shutdown","tid":25573,"ts":146847693507},
{"args":{"Initial URL":"http://www.google.com/","Net Error Code":0,"URL":"https://www.google.com/"},"cat":"navigation","id2":{"local":"0x387bf3aaa800"},"name":"Navigation StartToCommit","ph":"b","pid":25573,"scope":"navigation:StartToCommit","tid":25573,"ts":146847700788},



Then I tried running cefclient with --single-process , and it generates a segfault.. so I ran it with gdb and printed the backtrace and got:

Code: Select all
[New Thread 0x7fff7159e700 (LWP 28708)]
[New Thread 0x7fff70d9d700 (LWP 28709)]
[New Thread 0x7fff7059c700 (LWP 28710)]
[Thread 0x7fff70d9d700 (LWP 28709) exited]
[Thread 0x7fff7159e700 (LWP 28708) exited]

Thread 24 "Chrome_InProcRe" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffa51b2700 (LWP 28687)]
0x00002b8b1934b1cb in ?? ()
(gdb) bt
#0  0x00002b8b1934b1cb in ?? ()
#1  0x00000d9500000000 in ?? ()
#2  0x00007fffb00ddb68 in ?? ()
#3  0xffffffff00000000 in ?? ()
#4  0x000000003bec7267 in ?? ()
#5  0x00000000000812d4 in ?? ()
#6  0x00000d950a83f355 in ?? ()
#7  0x000000000000000a in ?? ()
#8  0x00000d9508041da9 in ?? ()
#9  0x00000d9508dbb7cd in ?? ()
#10 0x00000d9508dbb175 in ?? ()
#11 0x000000000000000a in ?? ()
#12 0x00007fffa51b0360 in ?? ()
#13 0x00000d95000be233 in ?? ()
#14 0x00000d9508040329 in ?? ()
#15 0x00000d9508dbb175 in ?? ()
#16 0x00000d9508dbb7cd in ?? ()
#17 0x00000d950a7b4bf9 in ?? ()
#18 0x00007fffa51b03c0 in ?? ()
#19 0x00007ffff2136c35 in Builtins_InterpreterEntryTrampoline () from /home/v3/Code/scratchpad/official/cef_binary_85.3.1+g1306235+chromium-85.0.4183.83_linux64/build/tests/cefclient/Release/libcef.so
#20 0x00000d9500a55fb0 in ?? ()
#21 0x00000d9509780f3d in ?? ()
#22 0x00000d9508040171 in ?? ()
#23 0x00000d9508dbb7cd in ?? ()
#24 0x00000d950a7b526d in ?? ()
#25 0x00000d9509b616dd in ?? ()
#26 0x000000000000007a in ?? ()
#27 0x00000d95086ce2ed in ?? ()
#28 0x00000d950a1d403d in ?? ()
#29 0x00000d950a83f355 in ?? ()
#30 0x00007fffa51b03f8 in ?? ()
#31 0x00007ffff212e01f in Builtins_ArgumentsAdaptorTrampoline () from /home/v3/Code/scratchpad/official/cef_binary_85.3.1+g1306235+chromium-85.0.4183.83_linux64/build/tests/cefclient/Release/libcef.so
#32 0x00000d950a82a259 in ?? ()
#33 0x0000000000000000 in ?? ()
(gdb)



Should I ran a debug build in order to track the problem further?
Any pointers in how can I trace the problem?

Thanks

Re: Troubleshoot renderer crash on facebook

PostPosted: Tue Sep 08, 2020 12:29 pm
by magreenblatt
Running a Debug build with gdb be a good start. I'm a bit surprised that your stack trace doesn't include symbols, since the libcef binaries should include symbol information. You can also attach gdb to the renderer process as described here.

Re: Troubleshoot renderer crash on facebook

PostPosted: Tue Sep 08, 2020 2:11 pm
by edgardog
Thanks.
I tried the steps to run GDB on the renderer process and worked.. but gave me the same dump

Code: Select all
[New Thread 0x7fffb2557700 (LWP 29753)]

Thread 1 "cefclient" received signal SIGSEGV, Segmentation fault.
0x00002ca922ddd0eb in ?? ()
(gdb) bt
#0  0x00002ca922ddd0eb in ?? ()
#1  0x00000ba300000000 in ?? ()
#2  0x00000000008c3bc0 in ?? ()
#3  0xffffffff00000000 in ?? ()
#4  0x000000003bec7267 in ?? ()
#5  0x00000000000806d4 in ?? ()
#6  0x00000ba30b21bd79 in ?? ()
#7  0x000000000000000a in ?? ()
#8  0x00000ba308041da9 in ?? ()
#9  0x00000ba3082f3ab5 in ?? ()
#10 0x00000ba3082f345d in ?? ()
#11 0x000000000000000a in ?? ()
#12 0x00007fffffffbb70 in ?? ()
#13 0x00000ba3001e06f3 in ?? ()
#14 0x00000ba308040329 in ?? ()
#15 0x00000ba3082f345d in ?? ()
#16 0x00000ba3082f3ab5 in ?? ()
#17 0x00000ba308cfd635 in ?? ()
#18 0x00007fffffffbbd0 in ?? ()
#19 0x00007ffff2136c35 in Builtins_InterpreterEntryTrampoline ()
   from /home/v3/Code/scratchpad/official/cef_binary_85.3.1+g1306235+chromium-85.0.4183.83_linux64/build/tests/cefclient/Release/libcef.so
#20 0x00000ba300a55fb0 in ?? ()
---Type <return> to continue, or q <return> to quit---
#21 0x00000ba3087816f1 in ?? ()
#22 0x00000ba308040171 in ?? ()
#23 0x00000ba3082f3ab5 in ?? ()
#24 0x00000ba308cfdca9 in ?? ()
#25 0x00000ba30a1e76b9 in ?? ()
#26 0x000000000000007a in ?? ()
#27 0x00000ba30b365d85 in ?? ()
#28 0x00000ba308a45ab1 in ?? ()
#29 0x00000ba30b21bd79 in ?? ()
#30 0x00007fffffffbc08 in ?? ()
#31 0x00007ffff212e01f in Builtins_ArgumentsAdaptorTrampoline ()
   from /home/v3/Code/scratchpad/official/cef_binary_85.3.1+g1306235+chromium-85.0.4183.83_linux64/build/tests/cefclient/Release/libcef.so
#32 0x00000ba308ee715d in ?? ()
#33 0x0000000000000000 in ?? ()



I'll start building a debug version and test with that and probably be back here with more questions... Thanks!

Re: Troubleshoot renderer crash on facebook

PostPosted: Thu Sep 10, 2020 9:38 am
by edgardog
I was doing something stupid with the standard distrib.. I ran Cmake to create the Release and not the Debug.
I realized this after building with Debug enabled (I usually build from source only release).
Now on both versions (the one that I built from source and the standard binary distrib) when I try to run it single-process it fails on some setlocale "C" check for wtf_string.cc(173)....

But I tried the xterm GDB debugger attach for multiprocess and that one starts fine.. when I try to upload a picture on a facebook story one of the renderer process dies, but on the Debug build I get only this:

Code: Select all
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/v3/Code/scratchpad/official/cef_binary_85.3.1+g1306235+chromium-85.0.4183.83_linux64/build/tests/cefclient/Debug/cefclient...done.
Starting program: /home/v3/Code/scratchpad/official/cef_binary_85.3.1+g1306235+chromium-85.0.4183.83_linux64/build/tests/cefclient/Debug/cefclient --type=renderer --no-sandbox --log-file=/home/v3/Code/scratchpad/official/cef_binary_85.3.1+g1306235+chromium-85.0.4183.83_linux64/build/tests/cefclient/Debug/debug.log --field-trial-handle=14485173593974179704,9614135995170544178,131072 --enable-features=CastMediaRouteProvider --lang=en-US --no-zygote --log-file=/home/v3/Code/scratchpad/official/cef_binary_85.3.1+g1306235+chromium-85.0.4183.83_linux64/build/tests/cefclient/Debug/debug.log --num-raster-threads=1 --renderer-client-id=5 --no-v8-untrusted-code-mitigations --shared-files=v8_snapshot_data:100
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffdbc38700 (LWP 4573)]
[New Thread 0x7fffdb437700 (LWP 4574)]
[New Thread 0x7fffdac36700 (LWP 4575)]
[New Thread 0x7fffda435700 (LWP 4576)]
[New Thread 0x7fffd9c34700 (LWP 4577)]
[New Thread 0x7fffd9433700 (LWP 4583)]
[New Thread 0x7fffd8c32700 (LWP 4584)]
[New Thread 0x7fffbbfff700 (LWP 4585)]
[New Thread 0x7fffbb7fe700 (LWP 4586)]
[New Thread 0x7fffbaffd700 (LWP 4596)]
[New Thread 0x7fffb8dd3700 (LWP 4601)]
---Type <return> to continue, or q <return> to quit---

Thread 1 "cefclient" received signal SIGSEGV, Segmentation fault.
0x000021f09f204cf0 in ?? ()
(gdb) bt
#0  0x000021f09f204cf0 in ?? ()
#1  0x00001c5509e705d9 in ?? ()
#2  0x0000000000000000 in ?? ()
(gdb)



I built it using symbol_level=1 .. should I try using =2 ?

I would appreciate any pointers to try to troubleshoot this problem.

Best Regards

Re: Troubleshoot renderer crash on facebook

PostPosted: Thu Sep 10, 2020 9:49 am
by magreenblatt
Single-process mode is not supported, so you'll have to debug the issue with multiprocess. Building with symbol_level=2 might fix the stack trace symbols issue.

Re: Troubleshoot renderer crash on facebook

PostPosted: Fri Sep 11, 2020 8:31 am
by edgardog
Need some guidance in here on how to continue...
I made a build (debug with symbol_level=2). I can tell the binary distrib is now quite big compared to symbol_level=1 (2.7GB now so definitively the files are bigger).
But when I run with GDB and do the facebook post I get again:
Code: Select all
Thread 1 "cefclient" received signal SIGSEGV, Segmentation fault.
0x000029dec93b5770 in ?? ()
(gdb) bt
#0  0x000029dec93b5770 in ?? ()
#1  0x00001c8f0a88adf1 in ?? ()
#2  0x0000000000000000 in ?? ()
(gdb)



I'm not quite sure what else to try.
The renderer seems to die but I can't figure out why.
I would appreciante some guidance on how to keep troubleshooting this problem,
Thanks!

Re: Troubleshoot renderer crash on facebook

PostPosted: Mon Sep 14, 2020 8:23 am
by edgardog
I seem to be getting the same problem using Chromium as well, so its not a specific CEF issue but an upstream problem.
There is no issue using chrome (same branch for Chrome, Chromium and CEF). I'll continue troubleshooting this with Chromium.

Thanks for the help.