Sending and reading POST requests in UTF8

Having problems with building or using CEF's C/C++ APIs? This forum is here to help. Please do not post bug reports or feature requests here.

Sending and reading POST requests in UTF8

Postby me12 » Mon Oct 06, 2014 6:48 am

Does anyone know how to send an XMLHttpRequest with post data encoded as UTF8 and read that in CEF3?

This is how I send the request:

Code: Select all
   function sendRequest(data) {      
      var xhr = new XMLHttpRequest();
      xhr.open("POST", "http://request", false);
      xhr.setRequestHeader("Content-type", "application/x-www-form-urlencoded; charset=utf-8");
      xhr.onload = function (e) {
        if (xhr.readyState === 4) {
         if (xhr.status === 200) {
            console.log(xhr.responseText);
         } else {
            console.error(xhr.statusText);
         }
        }
      };
      
      xhr.onerror = function (e) {
        console.error(xhr.statusText);
      };
      xhr.send(data);
   }
    sendRequest('data=somethingöüß&param2=lsdjf');


And this is how I try to red that request in CEF3:

Code: Select all
   CefRefPtr<CefPostData> postData = request->GetPostData();
   
   if (postData != NULL) {
      CefPostData::ElementVector elements;
      postData->GetElements(elements);

      if (elements.size() > 0) {

         std::wstring queryString;

         // it looks like the whole query string is in this first post data element?
         CefRefPtr<CefPostDataElement> query = elements[0];
         

         if (query->GetType() == PDE_TYPE_BYTES) {
            wchar_t *buff = new wchar_t[query->GetBytesCount() + 1];
            query->GetBytes(query->GetBytesCount(), buff);  // buff contains a string which is not correctly utf8 encoded.
                                                     // this is how it looks like: 慤慴猽浯瑥楨杮뛃볃鿃瀦牡浡㴲獬橤쵦췍
                                                     // if I cast it to (char*) it looks like: data=somethingöüß&param2=lsdjf
            queryString = std::wstring(buff, query->GetBytesCount());
            delete[] buff;
         }
      }
   }


I could not find any example for this. Any help is much appreciated. :)
me12
Techie
 
Posts: 31
Joined: Thu Jul 10, 2014 12:46 pm

Re: Sending and reading POST requests in UTF8

Postby magreenblatt » Mon Oct 06, 2014 11:45 am

me12 wrote:wchar_t *buff = new wchar_t[query->GetBytesCount() + 1];
query->GetBytes(query->GetBytesCount(), buff); // buff contains a string which is not correctly utf8 encoded.

UTF8 is a 1-to-multi-byte encoding. You can't just read it into a 2- or 4-byte wchar_t and expect the value to be correct.
magreenblatt
Site Admin
 
Posts: 12383
Joined: Fri May 29, 2009 6:57 pm

Re: Sending and reading POST requests in UTF8

Postby Mayhew » Mon Oct 06, 2014 12:48 pm

I think you will need to use the Windows MultiByteToWideChar() api to convert this from UTF8 to UTF16 to work with the string as a wchar_t.
Mayhew
Expert
 
Posts: 303
Joined: Mon Apr 18, 2011 8:02 pm

Re: Sending and reading POST requests in UTF8

Postby me12 » Tue Oct 07, 2014 2:31 am

Thank you for your help!

For everyone who will run into this problem in the future. Here is the simplest solution I found (using C++11).

Code: Select all
         if (query->GetType() == PDE_TYPE_BYTES) {

            std::wstring_convert<std::codecvt_utf8<wchar_t>, wchar_t> convert;

            char *buff = new char[query->GetBytesCount()];
            query->GetBytes(query->GetBytesCount(), buff);
            queryString = convert.from_bytes(std::string(buff, query->GetBytesCount()));
            delete[] buff;

            params = ParseQueryString(queryString);
         }
me12
Techie
 
Posts: 31
Joined: Thu Jul 10, 2014 12:46 pm

Re: Sending and reading POST requests in UTF8

Postby Mayhew » Tue Oct 07, 2014 3:51 pm

Also, be sure to handle the PDE_TYPE_FILE case. In CEF 3.2062.xxxx I started seeing requests come in with that type of post data when they didn't previously. If you aren't handling that case your code will mess some request body data.
Mayhew
Expert
 
Posts: 303
Joined: Mon Apr 18, 2011 8:02 pm


Return to Support Forum

Who is online

Users browsing this forum: Google [Bot] and 50 guests