Running/Debugging CEF in a docker container

[bam4d]

I have an open source project at [url]roxxy.io[/url].

It's a headless browser wrapped with facebook's proxygen server libraries.

I can get it to build and run on ubuntu 14.04 no problem, tried this on 3 different machines and its fine.

Code: Select all

./deps.sh && build.sh


Hoever, I want to be able to run this in a docker container but I am getting segmentation faults whenever cefInitialize is being called when running within docker.

I am running with

Code: Select all

./Roxxy -v=1


And the only debugging information from within cef before the segfault I get is the following:

Code: Select all

[0716/162345:VERBOSE1:zygote_main_linux.cc(600)] ZygoteMain: initializing 0 fork delegates
[0716/162345:INFO:systeminfo.cc(82)] Available number of cores: 8
Segmentation fault (core dumped)


The code can be found here:
https://github.com/Bam4d/Roxxy

to build the docker image

Code: Select all

docker build -t Roxxy .


To run it:

Code: Select all

docker run Roxxy


I am using the latest linux-64 spotify build.

What is the best way to debug this? is -v=1 the most verbose level of logging?

I have tried running docker with --ipc=host and --privileged thinking it would be an IPC issue, but the issue is the same every time.

[Czarek]

You need to debug with GDB to get stack trace.

[bam4d]

so running with gdb and debug libcef and debug of my own code i get the following:

Code: Select all

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0723 08:51:45.120519   110 Roxxy.cc:56] browser process
██████╗  ██████╗ ██╗  ██╗██╗  ██╗██╗   ██╗
██╔══██╗██╔═══██╗╚██╗██╔╝╚██╗██╔╝╚██╗ ██╔╝
██████╔╝██║   ██║ ╚███╔╝  ╚███╔╝  ╚████╔╝
██╔══██╗██║   ██║ ██╔██╗  ██╔██╗   ╚██╔╝ 
██║  ██║╚██████╔╝██╔╝ ██╗██╔╝ ██╗   ██║   
╚═╝  ╚═╝ ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝   ╚═╝   



I0723 08:51:45.120872   110 Roxxy.cc:81] Initializing cef
[New Thread 0x7fffd86cd700 (LWP 111)]
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0723 08:51:45.175024     1 Roxxy.cc:61] zygote process
[New Thread 0x7fffd7ecc700 (LWP 116)]
[0723/085145:FATAL:thread_task_runner_handle.cc(27)] Check failed: current.
warning: Could not find DWO CU obj/base/debug/base.debugger_posix.dwo(0x4735ac92f5c5cfad) referenced by CU at offset 0x9891 [in module ./libcef.so]

Program received signal SIGTRAP, Trace/breakpoint trap.
base::debug::(anonymous namespace)::DebugBreak() () at ../../base/debug/debugger_posix.cc:230
230   ../../base/debug/debugger_posix.cc: No such file or directory.


if i then continue at this point I hit the segfault:

Code: Select all

Continuing.
warning: Could not find DWO CU obj/cef/libcef/browser/libcef_static.browser_urlrequest_impl.dwo(0x4131807823c65d8b) referenced by CU at offset 0x1bf4 [in module ./libcef.so]
warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a700 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)


Program received signal SIGSEGV, Segmentation fault.
warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a700 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

0x00007fffe7f9a718 in scoped_refptr<scheduler::TaskQueueManagerDelegate>::scoped_refptr(scoped_refptr<scheduler::TaskQueueManagerDelegate> const&) ()
    at ../../base/memory/ref_counted.h:283
283   ../../base/memory/ref_counted.h: No such file or directory.
warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x7fffe7f9a718 in read in psymtab, but not in symtab.)


Doesn't seem particularly informative, anything else i can try (not sure what all these warnings mean... maybe im not producing the right debugging symbols)

without gdb i do seem to get a trace:

Code: Select all

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0723 08:55:58.081043   121 Roxxy.cc:56] browser process
██████╗  ██████╗ ██╗  ██╗██╗  ██╗██╗   ██╗
██╔══██╗██╔═══██╗╚██╗██╔╝╚██╗██╔╝╚██╗ ██╔╝
██████╔╝██║   ██║ ╚███╔╝  ╚███╔╝  ╚████╔╝
██╔══██╗██║   ██║ ██╔██╗  ██╔██╗   ╚██╔╝ 
██║  ██║╚██████╔╝██╔╝ ██╗██╔╝ ██╗   ██║   
╚═╝  ╚═╝ ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝   ╚═╝   



I0723 08:55:58.081270   121 Roxxy.cc:81] Initializing cef
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0723 08:55:58.118614     1 Roxxy.cc:61] zygote process
[0723/085558:FATAL:thread_task_runner_handle.cc(27)] Check failed: current.
#0 0x7f17940bf22e base::debug::StackTrace::StackTrace()
#1 0x7f179411000f logging::LogMessage::~LogMessage()
#2 0x7f1794244732 base::ThreadTaskRunnerHandle::Get()
#3 0x7f179422dec5 base::SequencedWorkerPool::SequencedWorkerPool()
#4 0x7f179b79b0ab content::(anonymous namespace)::BrowserThreadGlobals::BrowserThreadGlobals()
#5 0x7f179b79b015 base::DefaultLazyInstanceTraits<>::New()
#6 0x7f179b79aef5 base::internal::LeakyLazyInstanceTraits<>::New()
#7 0x7f179b79aead base::LazyInstance<>::Pointer()
#8 0x7f179b798c25 base::LazyInstance<>::Get()
#9 0x7f179b79a33d content::BrowserThread::CurrentlyOn()
#10 0x7f1793d7b6c3 CefContext::Initialize()
#11 0x7f1793d7b2fa CefInitialize()
#12 0x7f1793bb7bd3 cef_initialize
#13 0x0000004861e5 CefInitialize()
#14 0x000000450fdd Roxxy::Run()
#15 0x00000045e50a main
#16 0x7f17927adf45 __libc_start_main
#17 0x00000042a13e <unknown>

Aborted (core dumped)


Anything useful in there?

Also ran it with strace because docker has some intricacies around IPC that i'm not 100% familiar with that might be an issue.

Code: Select all

recvmsg(17, WARNING: Logging before InitGoogleLogging() is written to STDERR
I0723 08:57:34.036738     1 Roxxy.cc:61] zygote process
{msg_name(0)=NULL, msg_iov(1)=[{"ZYGOTE_BOOT\0", 13}], msg_controllen=32, {cmsg_len=28, cmsg_level=SOL_SOCKET, cmsg_type=SCM_CREDENTIALS{pid=170, uid=0, gid=0}}, msg_flags=0}, 0) = 12
recvmsg(17, {msg_name(0)=NULL, msg_iov(1)=[{"ZYGOTE_OK\0", 11}], msg_controllen=32, {cmsg_len=28, cmsg_level=SOL_SOCKET, cmsg_type=SCM_CREDENTIALS{pid=172, uid=0, gid=0}}, msg_flags=0}, 0) = 10
wait4(170, NULL, WNOHANG, NULL)         = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7fc2916cc000
mprotect(0x7fc2916cc000, 4096, PROT_NONE) = 0
clone(child_stack=0x7fc291ecbaf0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fc291ecc9d0, tls=0x7fc291ecc700, child_tidptr=0x7fc291ecc9d0) = 173
close(18)                               = 0
sendmsg(17, {msg_name(0)=NULL, msg_iov(1)=[{"\4\0\0\0\3\0\0\0", 8}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 8
read(17, "~\0\0\0", 4)                  = 4
getrlimit(RLIMIT_NOFILE, {rlim_cur=512*1024, rlim_max=1024*1024}) = 0
setrlimit(RLIMIT_NOFILE, {rlim_cur=8*1024, rlim_max=1024*1024}) = 0
gettid()                                = 167
rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER, 0x7fc2b0970330}, NULL, 8) = 0
open("/proc/self/status", O_RDONLY)     = 18
read(18, "Name:\tRoxxy\nState:\tR (running)\nT"..., 1024) = 778
close(18)                               = 0
write(2, "[0723/085734:FATAL:thread_task_r"..., 77[0723/085734:FATAL:thread_task_runner_handle.cc(27)] Check failed: current.
) = 77
write(3, "[0723/085734:FATAL:thread_task_r"..., 77) = 77
open("/proc/self/status", O_RDONLY)     = 18
read(18, "Name:\tRoxxy\nState:\tR (running)\nT"..., 1024) = 778
close(18)                               = 0
open("/proc/self/status", O_RDONLY)     = 18
read(18, "Name:\tRoxxy\nState:\tR (running)\nT"..., 1024) = 778
close(18)                               = 0
--- SIGTRAP {si_signo=SIGTRAP, si_code=SI_KERNEL} ---
+++ killed by SIGTRAP (core dumped) +++
Trace/breakpoint trap (core dumped)


Any ideas?

Thanks for your help

[Czarek]

Stack trace shows that CefInitialize fails, because thread check fails. CefInitialize must be called on the UI thread. CEF depends on X11 and requires a window manager. Is this a desktop environment you're running CEF on?

[bam4d]

It's running in a docker container which has no X11 installed or a window manager. (I'm hoping to run it on a server).

I guess if i install X11 libs etc i might be able to trick it into being totally headless.

why does it require X11 and a window manager if there is headless mode?

I guess i could also use something like xvfb?

Thanks loads for your help.

[magreenblatt]

Yes, you can use xvfb.