LLVM out of memory Win10 Release 5735

Having problems with building or using CEF's C/C++ APIs? This forum is here to help. Please do not post bug reports or feature requests here.

LLVM out of memory Win10 Release 5735

Postby oleksii » Mon Jun 05, 2023 9:35 am

I'm trying to build stable CEF dll with proprietary codecs for the C++ project. I'm a newbie so maybe I've done something stupid))) but actually I've spent the whole weekend on this and the same result always: LLVM ERROR: out of memory

PC:
AMD Threadripper 3960X 24-Core (48 logical proccessors)
32 GB RAM (25 GB should be available)
Screenshot 2023-06-05 at 17.19.36.png
VS setup
Screenshot 2023-06-05 at 17.19.36.png (40.71 KiB) Viewed 3586 times


Build commands:
$env:GN_DEFINES="is_official_build=true proprietary_codecs=true ffmpeg_branding=Chrome"
set GYP_MSVS_VERSION=2022
set CEF_ARCHIVE_FORMAT=tar.bz2
py.exe automate-git.py --no-debug-build --download-dir=cef_auto --branch=5735 --minimal-distrib --client-distrib --force-clean --x64-build --with-pgo-profiles

Some of the output:

args.gn:
Code: Select all
--> D:\cef\cef_auto2\chromium\src\out\Release_GN_x64\args.gn contents:
alternate_cdm_storage_id_key="968b476909da4373b08903c28e859454"
clang_use_chrome_plugins=false
enable_background_mode=false
enable_basic_printing=true
enable_cdm_host_verification=true
enable_cdm_storage_id=true
enable_nacl=false
enable_print_preview=true
enable_resource_allowlist_generation=false
enable_rlz=true
enable_widevine=true
ffmpeg_branding="Chrome"
is_component_build=false
is_debug=false
is_official_build=true
optimize_webui=true
proprietary_codecs=true
target_cpu="x64"
use_perfetto_client_library=false
v8_enable_sandbox=false


First crashed thread:
Code: Select all
[44656/57427] CXX obj/content/browser/browser/network_download_pending_url_loader_factory.obj
FAILED: obj/content/browser/browser/network_download_pending_url_loader_factory.obj
..\..\third_party\llvm-build\Release+Asserts\bin\clang-cl.exe /c ../../content/browser/download/network_download_pending_url_loader_factory.cc /Foobj/content/browser/browser/network_download_pending_url_loader_factory.obj /nologo /showIncludes:user "-imsvcC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\include" "-imsvcC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\ATLMFC\include" "-imsvcC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-imsvcC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-imsvcC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-imsvcC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-imsvcC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-imsvcC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" -D__ATLHOST_H__ -DUSE_AURA=1 -DOFFICIAL_BUILD -D_HAS_NODISCARD -D_CRT_NONSTDC_NO_WARNINGS -D_WINSOCK_DEPRECATED_NO_WARNINGS "-DCR_CLANG_REVISION=\"llvmorg-17-init-8029-g27f27d15-3\"" -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -DCR_LIBCXX_REVISION=bff81b702ff4b7f74b1c0ed02a4bcf6c2744a90b -D_LIBCPP_ENABLE_ASSERTIONS=1 -D__STD_C -D_CRT_RAND_S -D_CRT_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_DEPRECATE -D_ATL_NO_OPENGL -D_WINDOWS -DCERT_CHAIN_PARA_HAS_EXTRA_FIELDS -DPSAPI_VERSION=2 -DWIN32 -D_SECURE_ATL -DWINAPI_FAMILY=WINAPI_FAMILY_DESKTOP_APP -DWIN32_LEAN_AND_MEAN -DNOMINMAX -D_UNICODE -DUNICODE -DNTDDI_VERSION=NTDDI_WIN10_NI -D_WIN32_WINNT=0x0A00 -DWINVER=0x0A00 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DCONTENT_IMPLEMENTATION -DV8_USE_EXTERNAL_STARTUP_DATA -DWEBP_EXTERN=extern -DUSE_EGL -DVK_USE_PLATFORM_WIN32_KHR -DLIBYUV_DISABLE_NEON -D_WTL_NO_AUTOMATIC_NAMESPACE -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DUSE_CHROMIUM_ICU=1 -DU_ENABLE_TRACING=1 -DU_ENABLE_RESOURCE_TRACING=0 -DU_STATIC_IMPLEMENTATION -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DSK_CODEC_DECODES_PNG -DSK_CODEC_DECODES_WEBP -DSK_ENCODE_PNG -DSK_ENCODE_WEBP -DSK_ENABLE_SKSL -DSK_UNTIL_CRBUG_1187654_IS_FIXED "-DSK_USER_CONFIG_HEADER=\"../../skia/config/SkUserConfig.h\"" -DSK_WIN_FONTMGR_NO_SIMULATIONS -DSK_DISABLE_LEGACY_IMAGE_FACTORIES -DSK_DISABLE_LEGACY_MAKE_TEXTURE_IMAGE -DSK_DISABLE_LEGACY_IMAGE_FLUSH -DSK_DISABLE_LEGACY_GET_BACKEND_TEXTURE -DSK_DISABLE_LEGACY_IMAGE_ENCODE_METHODS -DSK_DISABLE_LEGACY_IMAGE_ENCODER -DSK_CODEC_DECODES_JPEG -DSK_ENCODE_JPEG -DSK_HAS_WUFFS_LIBRARY -DGR_GL_FUNCTION_TYPE=__stdcall -DSK_GANESH "-DSK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\"" -DSK_GL -DSK_VULKAN=1 -DCRASHPAD_ZLIB_SOURCE_EXTERNAL -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DGOOGLE_PROTOBUF_INTERNAL_DONATE_STEAL_INLINE=0 -DLEVELDB_PLATFORM_CHROMIUM=1 -DWEBRTC_ENABLE_AVX2 -DRTC_ENABLE_WIN_WGC -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0 -DWEBRTC_CHROMIUM_BUILD -DWEBRTC_WIN -DABSL_ALLOCATOR_NOTHROW=1 -DLOGGING_INSIDE_WEBRTC -DV8_COMPRESS_POINTERS -DV8_COMPRESS_POINTERS_IN_SHARED_CAGE -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_DEPRECATION_WARNINGS -DCPPGC_CAGED_HEAP -DCPPGC_YOUNG_GENERATION -DCPPGC_POINTER_COMPRESSION -DCPPGC_SLIM_WRITE_BARRIER -DLIBGAV1_MAX_BITDEPTH=10 -DLIBGAV1_THREADPOOL_USE_STD_MUTEX -DLIBGAV1_ENABLE_LOGGING=0 -DLIBGAV1_PUBLIC= -DSQLITE_OMIT_ANALYZE -DSQLITE_OMIT_AUTOINIT -DSQLITE_OMIT_AUTOMATIC_INDEX -DSQLITE_OMIT_AUTORESET -DSQLITE_OMIT_COMPILEOPTION_DIAGS -DSQLITE_OMIT_EXPLAIN -DSQLITE_OMIT_GET_TABLE -DSQLITE_OMIT_INTROSPECTION_PRAGMAS -DSQLITE_DEFAULT_LOOKASIDE=0,0 -DSQLITE_OMIT_LOOKASIDE -DSQLITE_OMIT_TCL_VARIABLE -DSQLITE_OMIT_REINDEX -DSQLITE_OMIT_TRACE -DSQLITE_OMIT_UPSERT -DSQLITE_OMIT_WINDOWFUNC -DSQLITE_ENABLE_FTS3 -DSQLITE_DISABLE_FTS3_UNICODE -DSQLITE_DISABLE_FTS4_DEFERRED -DSQLITE_ENABLE_ICU -DSQLITE_SECURE_DELETE -DSQLITE_THREADSAFE=1 -DSQLITE_MAX_WORKER_THREADS=0 -DSQLITE_MAX_MMAP_SIZE=268435456 -DSQLITE_DEFAULT_FILE_PERMISSIONS=0600 -DSQLITE_DEFAULT_LOCKING_MODE=1 -DSQLITE_DEFAULT_MEMSTATUS=1 -DSQLITE_DEFAULT_PAGE_SIZE=4096 -DSQLITE_DEFAULT_PCACHE_INITSZ=0 -DSQLITE_LIKE_DOESNT_MATCH_BLOBS -DSQLITE_OMIT_DEPRECATED -DSQLITE_OMIT_PROGRESS_CALLBACK -DSQLITE_OMIT_SHARED_CACHE -DSQLITE_USE_ALLOCA -DSQLITE_OMIT_DECLTYPE -DSQLITE_OMIT_JSON -DSQLITE_OMIT_LOAD_EXTENSION -DSQLITE_HAVE_ISNAN -DSQLITE_HAVE_SQLITE3R -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_BATCH_ATOMIC_WRITE -DSQLITE_TEMP_STORE=3 -DSQLITE_ENABLE_LOCKING_STYLE=0 -I../.. -Igen -I../../buildtools/third_party/libc++ -I../../third_party/perfetto/include -Igen/third_party/perfetto/build_config -Igen/third_party/perfetto -I../../third_party/libwebp/src/src -I../../third_party/khronos -I../../gpu -I../../third_party/vulkan-deps/vulkan-headers/src/include -Igen/third_party/dawn/include -I../../third_party/dawn/include -I../../third_party/libyuv/include -I../../third_party/jsoncpp/source/include -Igen/third_party/private_membership/src -Igen/third_party/shell-encryption/src -Igen/components/policy/proto -I../../third_party/wtl/include -I../../third_party/abseil-cpp -I../../third_party/boringssl/src/include -I../../third_party/protobuf/src -Igen/protoc_out -I../../third_party/ipcz/include -I../../third_party/ced/src -I../../third_party/icu/source/common -I../../third_party/icu/source/i18n -I../../third_party/libwebm/source -I../../third_party/skia -I../../third_party/wuffs/src/release/c -I../../third_party/vulkan/include -I../../third_party/mesa_headers -I../../third_party/crashpad/crashpad -I../../third_party/crashpad/crashpad/compat/win -I../../third_party/zlib -I../../third_party/leveldatabase -I../../third_party/leveldatabase/src -I../../third_party/leveldatabase/src/include -I../../third_party/libaom/source/libaom -I../../net/third_party/quiche/overrides -I../../net/third_party/quiche/src/quiche/common/platform/default -I../../net/third_party/quiche/src -Igen/net/third_party/quiche/src -I../../third_party/webrtc_overrides -I../../third_party/webrtc -Igen/third_party/webrtc -Igen/third_party/metrics_proto -I../../v8/include -Igen/v8/include -I../../third_party/libgav1/src -I../../third_party/libgav1/src/src -I../../third_party/brotli/include -I../../third_party/distributed_point_functions -I../../third_party/distributed_point_functions/code -Igen/third_party/distributed_point_functions -I../../third_party/highway/src -I../../third_party/re2/src /W4 -Wimplicit-fallthrough -Wextra-semi -Wunreachable-code-aggressive -Wthread-safety /WX -Wno-missing-field-initializers -Wno-unused-parameter -Wno-psabi -Wloop-analysis -Wno-unneeded-internal-declaration -Wno-nonportable-include-path -Wenum-compare-conditional -Wno-ignored-pragma-optimize -Wno-deprecated-builtins -Wno-bitfield-constant-conversion -Wno-deprecated-this-capture -Wshadow -fno-delete-null-pointer-checks -fno-ident -fcolor-diagnostics -fmerge-all-constants -fcrash-diagnostics-dir=../../tools/clang/crashreports -mllvm -instcombine-lower-dbg-declare=0 /clang:-ffp-contract=off -flto=thin -fsplit-lto-unit -fwhole-program-vtables -fcomplete-member-pointers /Gy /FS /bigobj /utf-8 /Zc:twoPhase -ffile-reproducible /Zc:sizedDealloc- /D__WRL_ENABLE_FUNCTION_STATICS__ -fmsc-version=1916 -m64 -msse3 -ffile-compilation-dir=. -no-canonical-prefixes -ftrivial-auto-var-init=pattern /O2 /Ob2 /Oy- /Zc:inline /Gw /Oi /Z7 -gno-codeview-command-line -gcodeview-ghash -Xclang -fuse-ctor-homing -fprofile-use=../../chrome/build/pgo_profiles/chrome-win64-5735-1684841082-12c77b463d1b2fef445b58d98ab6f7cbe0fd5b81.profdata -Wno-profile-instr-unprofiled -Wno-profile-instr-out-of-date -Wno-backend-plugin /guard:cf /MT -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Wexit-time-destructors -Wno-redundant-parens -Wno-null-pointer-subtraction -DPROTOBUF_ALLOW_DEPRECATED=1 -Wno-redundant-parens /std:c++20 -Wno-trigraphs /TP /GR- -I../../buildtools/third_party/libc++/trunk/include /Fd"obj/content/browser/browser_cc.pdb"
LLVM ERROR: out of memory
Allocation failed


I've attache a bigger snapshot of the console output. I can also attach crash reports from chromium\src\tools\clang\crashreports but zip is more than 100 MB)))

I will be very glad to have any help with this :roll: :roll: :roll:
Attachments
cef_out_of_memory_PS.txt
more logs from Power Shell
(28.43 KiB) Downloaded 236 times
oleksii
Newbie
 
Posts: 3
Joined: Mon Jun 05, 2023 8:18 am

Re: LLVM out of memory Win10 Release 5735

Postby magreenblatt » Mon Jun 05, 2023 10:47 am

32 GB RAM (25 GB should be available)

Add more RAM, or increase swap space?
magreenblatt
Site Admin
 
Posts: 12379
Joined: Fri May 29, 2009 6:57 pm

Re: LLVM out of memory Win10 Release 5735

Postby ndesktop » Mon Jun 05, 2023 12:04 pm

Building (for some years) on 32 GB machine bumped linker into 29-30 GB zone, so bad that keyboard starts emitting sounds instead of WM_CHARs. But in the end it finishes after great pains.
With the new machine with 64 GB RAM it all become a thng of the past, also the linking seems way faster (but still it takes some time, it's not like a 4.6+ GB .pdb will generate instantly).
ndesktop
Master
 
Posts: 748
Joined: Thu Dec 03, 2015 10:10 am

Re: LLVM out of memory Win10 Release 5735

Postby oleksii » Mon Jun 05, 2023 1:05 pm

Thanks for the fast replies)
Tomorrow I will try to reassemble my server and add 32 GB. But I'm not sure if it will work (different memory modules).
All disks on the server mapped to RAID so I can't add swap((((
Maybe if I can limit in some way threads count linker will be okay with 32 GB?
oleksii
Newbie
 
Posts: 3
Joined: Mon Jun 05, 2023 8:18 am

Re: LLVM out of memory Win10 Release 5735

Postby ndesktop » Mon Jun 05, 2023 2:41 pm

From my experience it's not the linked, but the pdb that consumes the most resources. Your experience might be different, though.
ndesktop
Master
 
Posts: 748
Joined: Thu Dec 03, 2015 10:10 am

Re: LLVM out of memory Win10 Release 5735

Postby oleksii » Mon Jun 05, 2023 2:54 pm

Hooray, I've added 32 GB (64 GB total) and it worked. Thanks, everyone)

PS I suppose I should update all my machines to 64 GB now)))))
oleksii
Newbie
 
Posts: 3
Joined: Mon Jun 05, 2023 8:18 am


Return to Support Forum

Who is online

Users browsing this forum: No registered users and 22 guests