Large file causes crashes using CefURLRequestClient

[meflynn]

I'm working on project where I make a HTTP Get request to retrieve video files. I'm making the request via a CefURLRequest object and implementing the CefURLRequestClient to read and save the video file as i receive it to a local file. For most file sizes this works fine, but for larger videos (>170 MB) this is causing malloc to fail within cef. Debugging the issue we found that there is an object in the call stack that appends all the data from the download as it is received to a STL string. The string is doing the STL exponential reallocation thing so it is trying to malloc a 90meg buffer and malloc is not finding a free range of memory big enough. So clearly it seems that memory fragmentation is causing us issues here.

We tried setting the flag on the request that tells it to skip the cache, but then it just made it a little bit farther before failing when trying to reallocate to 150meg.

This seems to be a limitation of the API CEF provides for doing arbitrary network requests. The underlying chromium class that handles HTTP transactions allows for GET data to be routed to a file or an in memory string. CEF only gives you the option of the in memory string. That is fine for small resources that would go in a web page, but it is causing me major headaches for larger pieces of data.

My question is, can anyone point me to a better way to do this? I'm not going to be able to change away from using HTTP GET to retrieve these files so I need to figure something out. I know there is the CefDownloadHandler but it isn't clear to me when and where this can be used.

Any help would be appreciated.

Note: I'm using CEF3

[magreenblatt]

You can use the HTTP range header to download the video using multiple smaller requests. Please also add an issue to the issue tracker for fixing the underlying memory allocation problem.

[meflynn]

Thanks marshall, we will look in to that.

Another question we have is that it looks like the http response data we are receiving is being added to the in memory cache. Is there anyway to disable this for certain requests?

[meflynn]

Issue added to issue tracker : http://code.google.com/p/chromiumembedd ... ail?id=872

[magreenblatt]

Another question we have is that it looks like the http response data we are receiving is being added to the in memory cache. Is there anyway to disable this for certain requests?

You can set the UR_FLAG_SKIP_CACHE flag on the CefRequest used to create the CefURLRequest.