JavaScript Heap Size Limit

Having problems with building or using CEF's C/C++ APIs? This forum is here to help. Please do not post bug reports or feature requests here.

JavaScript Heap Size Limit

Postby jogy » Mon Mar 22, 2021 7:38 pm

Hello,

When running CefSharp + Cef 75 on a Windows machine with 32G RAM, we were able to provide a lot of memory to Cef by using --js-flags : --max_old_space_size=32000.
This allowed us to load a very large page in memory and manipulate it.

Now we are trying to upgrade to CefSharp + Cef 88, and the same page cannot fully load, but the browser process crashes.

Invoking
Code: Select all
window.performance.memory.jsHeapSizeLimit
on Cef75 reports a value around 32G.
Invoking the same code on Cef 88 reports a value of just 4G.

How can we provide more memory to the Cef engine, when enough memory is available? Our tests indicate that the memory limits were roughly the same on a 32G and on a 16G machine.

We tried other flags like --max_semi_space_size, but while that increased the reported jsHeapSizeLimit value, the browser still crashed on the large page.

Regards,
Jogy
jogy
Newbie
 
Posts: 7
Joined: Mon Mar 22, 2021 7:19 pm

Re: JavaScript Heap Size Limit

Postby magreenblatt » Mon Mar 22, 2021 7:52 pm

Are you using 64-bit binaries?
magreenblatt
Site Admin
 
Posts: 12382
Joined: Fri May 29, 2009 6:57 pm

Re: JavaScript Heap Size Limit

Postby ndesktop » Tue Mar 23, 2021 6:12 am

I saw on several ocassions crashes due to extremely large JS accumulated in a page, and Increasing stack size (from 1MB to 1.25MB) on linker settings was the solution for me.
ndesktop
Master
 
Posts: 750
Joined: Thu Dec 03, 2015 10:10 am

Re: JavaScript Heap Size Limit

Postby fddima » Tue Mar 23, 2021 6:19 am

ndesktop wrote:I saw on several ocassions crashes due to extremely large JS accumulated in a page, and Increasing stack size (from 1MB to 1.25MB) on linker settings was the solution for me.


Just for FYI: Chrome uses whopping 8MB stack size.
fddima
Master
 
Posts: 788
Joined: Tue Dec 07, 2010 6:10 am

Re: JavaScript Heap Size Limit

Postby ndesktop » Tue Mar 23, 2021 10:15 am

So it seems I had the right idea at the time (2013).
ndesktop
Master
 
Posts: 750
Joined: Thu Dec 03, 2015 10:10 am

Re: JavaScript Heap Size Limit

Postby jogy » Tue Mar 23, 2021 10:36 am

magreenblatt wrote:Are you using 64-bit binaries?


Yes.
jogy
Newbie
 
Posts: 7
Joined: Mon Mar 22, 2021 7:19 pm

Re: JavaScript Heap Size Limit

Postby jogy » Tue Mar 30, 2021 12:05 am

For repro steps:
If I open an infinite scroll page like https://www.facebook.com/Google and execute this simple script that tries to scroll down the page:
Code: Select all
async function wait(intervalInMilliseconds) {
  return new Promise((resolve) => setInterval(resolve, intervalInMilliseconds));
}

async function unlimitedScroll() {
  for (let i = 0; ; ++i) {
      await window.scrollTo(0, document.body.scrollHeight);
      await wait(2000);
      await console.log(`Scroll: ${i}, total: ${window.performance.memory.totalJSHeapSize.toLocaleString()}, used: ${window.performance.memory.usedJSHeapSize.toLocaleString()}, limit: ${window.performance.memory.jsHeapSizeLimit.toLocaleString()}`);
  }
}

unlimitedScroll();


With Cef 75, I am able to scroll very far down, reaching posts from 2010.
With Cef 88/90, the browser crashes when reaching 2016. The totalJSHeapSize reported just before the crash is in the vicinity of 4G, and I have not been able to pass that barrier.
With Cef 75, when I set the memory with the flag "--max_old_space_size=16000", the reported totalJSHeapSize reaches much higher numbers.

My question is, how can I enable latest versions of Cef to use more memory, so it can load such large page?
jogy
Newbie
 
Posts: 7
Joined: Mon Mar 22, 2021 7:19 pm

Re: JavaScript Heap Size Limit

Postby HarmlessDave » Mon Apr 05, 2021 11:50 am

Are you trying to load pages to display to the user or to screen scrape (parse) the contents?

If you are scraping, you might change your logic to do data fetches and parse the text without ever loading it into the browser window.
HarmlessDave
Expert
 
Posts: 370
Joined: Fri Jul 11, 2014 2:02 pm

Re: JavaScript Heap Size Limit

Postby jogy » Mon Apr 05, 2021 12:03 pm

HarmlessDave wrote:Are you trying to load pages to display to the user or to screen scrape (parse) the contents?

If you are scraping, you might change your logic to do data fetches and parse the text without ever loading it into the browser window.


That won't be easy to do with pages like Facebook, where there is so much going on in the background to show and expand posts, comments, etc.
jogy
Newbie
 
Posts: 7
Joined: Mon Mar 22, 2021 7:19 pm

Re: JavaScript Heap Size Limit

Postby jogy » Tue Apr 06, 2021 10:54 am

I am trying to find out the differences between CefSharp/Cef 79 (last version that could load large page) and CefSharp/Cef 81 (The version that could no longer load the page), and one thing that strikes out is the update of the V8 JavaScript engine, which has introduced pointer compression: https://v8.dev/blog/pointer-compression
Is it possible that this pointer compression is resulting in the 4G memory limit that we see?
jogy
Newbie
 
Posts: 7
Joined: Mon Mar 22, 2021 7:19 pm

Next

Return to Support Forum

Who is online

Users browsing this forum: No registered users and 42 guests