I'm working on project where I make a HTTP Get request to retrieve video files. I'm making the request via a CefURLRequest object and implementing the CefURLRequestClient to read and save the video file as i receive it to a local file. For most file sizes this works fine, but for larger videos (>170 MB) this is causing malloc to fail within cef. Debugging the issue we found that there is an object in the call stack that appends all the data from the download as it is received to a STL string. The string is doing the STL exponential reallocation thing so it is trying to malloc a 90meg buffer and malloc is not finding a free range of memory big enough. So clearly it seems that memory fragmentation is causing us issues here.
We tried setting the flag on the request that tells it to skip the cache, but then it just made it a little bit farther before failing when trying to reallocate to 150meg.
This seems to be a limitation of the API CEF provides for doing arbitrary network requests. The underlying chromium class that handles HTTP transactions allows for GET data to be routed to a file or an in memory string. CEF only gives you the option of the in memory string. That is fine for small resources that would go in a web page, but it is causing me major headaches for larger pieces of data.
My question is, can anyone point me to a better way to do this? I'm not going to be able to change away from using HTTP GET to retrieve these files so I need to figure something out. I know there is the CefDownloadHandler but it isn't clear to me when and where this can be used.
Any help would be appreciated.
Note: I'm using CEF3