CEF Custom Scheme Handler

[heshiming]

Updated on 8-15-2009.

I've written some code to provide custom scheme handler to CEF. The concept is similar to Internet Explorer's Asynchronous Pluggable Protocol, but with CEF it's much easier. The implementation is quite straightforward thanks to the existing CEF classes.

In a nutshell, the scheme handler interface is as below:

Code: Select all

#ifndef _CEF_SCHEME_H
#define _CEF_SCHEME_H

#include <string>
#include "include/cef.h"
#include "include/cef_ptr.h"
#include "include/cef_types.h"

class CefSchemeHandler;
class CefSchemeHandlerFactory;

// Register a custom scheme handler using the CefSchemeHandler interface
// Custom scheme in the format of scheme_name://host_name/ will be handled through
// the specified handler interface.
// All URLs beginning with scheme_name://host_name/ will go through this handler
/*--cef()--*/
bool CefRegisterScheme(   const std::wstring& scheme_name,
                                    const std::wstring& host_name,
                                    CefRefPtr<CefSchemeHandlerFactory> factory);

// Class that creates CefSchemeHandler instances.
/*--cef(source=client)--*/
class CefSchemeHandlerFactory : public CefBase
{
public:
   // Return a new scheme handler instance to handle the request.
   /*--cef()--*/
   virtual CefRefPtr<CefSchemeHandler> Create() =0;
};

// Class used to represent a custom scheme handler interface.
/*--cef(source=client)--*/
class CefSchemeHandler : public CefBase
{
public:
   // Process the request. All response generation should take place in this
   // method.
   /*--cef()--*/
   virtual bool ProcessRequest(CefRefPtr<CefRequest> request) =0;

   // Cancel processing of the request.
   /*--cef()--*/
   virtual void Cancel() =0;

   // Return the mime type for the request.
   /*--cef()--*/
   virtual std::wstring GetMimeType() =0;

   // Return the response length for the request. If there is no response return
   // 0 and ReadResponse() will not be called. If the response length is not
   // known return -1 and ReadResponse() will be called until it returns false or
   // until the value of |bytes_read| is set to 0. If the response length is
   // known return a positive value and ReadResponse() will be called until it
   // returns false, the value of |bytes_read| is set to 0 or the specified
   // number of bytes have been read.
   /*--cef()--*/
   virtual size_t GetResponseLength() =0;

   // Copy up to |bytes_to_read| bytes into |data_out|. If the copy succeeds
   // set |bytes_read| to the number of bytes copied and return true. If the
   // copy fails return false and ReadResponse() will not be called again.
   /*--cef()--*/
   virtual bool ReadResponse(void* data_out, int bytes_to_read,
      int* bytes_read) =0;
};

#endif

In the attachment package, source files that implements this mechanism are provided. Files have been put outside the CEF source tree. The above interface is defined in libcef/scheme/cef_scheme.h , and the exported function CefRegisterScheme is implemented in libcef/scheme/scheme_impl.cc . It talks to URLRequestFilter to let chromium use custom scheme handlers.

libcef_dll/scheme/cef_scheme_capi.h contains the translated C interface using tools/translator_scheme.bat . And the rest files:
libcef_dll/cpptoc/scheme_handler_cpptoc.cc
libcef_dll/cpptoc/scheme_handler_cpptoc.h
libcef_dll/cpptoc/scheme_handler_factory_cpptoc.cc
libcef_dll/cpptoc/scheme_handler_factory_cpptoc.h
libcef_dll/ctocpp/scheme_handler_ctocpp.cc
libcef_dll/ctocpp/scheme_handler_ctocpp.h
libcef_dll/ctocpp/scheme_handler_factory_ctocpp.cc
libcef_dll/ctocpp/scheme_handler_factory_ctocpp.h
libcef_dll/scheme/cef_scheme_capi.cc
libcef_dll/wrapper/scheme/scheme_wrapper.cc
are all binding/wrapper implementations.

tests/cefclient/sample_scheme.h contains a sample scheme handler, and some small modifications in cefclient.cpp . To test out this handler, click here to download a 8MB zip file (compiled with chromium R22728). It contains a patched version of chromium to make XMLHttpRequest work with custom schemes (see notes below). The test client looks like this:
.

* Note:
It is discovered that although an XMLHttpRequest may generate a POST request that goes through our custom scheme handler like a form post, the upload data (XML data posted) will be lost. The cause is that WebKit's XMLHttpRequest checks the URL and see if it's HTTP or HTTPS protocol before setting post data. This is done inside XMLHttpRequest::send of third_party/WebKit/WebCore/xml/XMLHttpRequest.cpp. The method calls KURL::protocolInHTTPFamily to determine whether post data should be created for the request.

I'm not really sure where to patch. I picked KURLGooglePrivate::initProtocolInHTTPFamily in third_party/WebKit/WebCore/platform/KURLGoogle.cpp , and changed:

Code: Select all

     if (m_parsed.scheme.len == 4)
         m_protocolInHTTPFamily = lowerCaseEqualsASCII(scheme, scheme + 4, "http");
     else if (m_parsed.scheme.len == 5)
         m_protocolInHTTPFamily = lowerCaseEqualsASCII(scheme, scheme + 5, "https");
     else
         m_protocolInHTTPFamily = false;


to:

Code: Select all

      switch (m_parsed.scheme.len) {
      case 3:
         m_protocolInHTTPFamily = !lowerCaseEqualsASCII(scheme, scheme + 3, "ftp");   break;
      case 4:
         m_protocolInHTTPFamily = !lowerCaseEqualsASCII(scheme, scheme + 4, "file");   break;
      case 5:
         m_protocolInHTTPFamily = !lowerCaseEqualsASCII(scheme, scheme + 5, "about");   break;
      case 10:
         m_protocolInHTTPFamily = !lowerCaseEqualsASCII(scheme, scheme + 10, "view-cache");   break;
      default:
         m_protocolInHTTPFamily = true;   break;
      }


This solved the problem. Hopefully the chromium team will know about this.

Any comments or new feature requests are welcomed.

[magreenblatt]

Awesome work! I think this should be part of CEF. I'll try to make that happen next week and give you an educated opinion on how to handle the XMLHttpRequest problem.

[magreenblatt]

I'm reviewing your code and I have a question. Is it possible for more than one CefUrlRequestJob instance to be actively handling a request at the same time? If so, how does the CefSchemeHandler instance keep track of which request it's returning results for?

[magreenblatt]

The approach that you're using to look up the handler in CefUrlRequestJob::Factory() loses the association to the host name. If you're only interested in associating a handler to a scheme then you can call URLRequest::RegisterProtocolFactory() directly. If you want to keep the association to both the scheme and the host name then you need to implement a map lookup similar to what's done in URLRequestFilter. In either case, you probably don't need to use the URLRequestFilter class.

[magreenblatt]

To address the above issues I propose the following interface:

Code: Select all

#ifndef _CEF_SCHEME_H
#define _CEF_SCHEME_H

#include "cef.h"


class CefSchemeHandler;
class CefSchemeHandlerFactory;

// Register a custom scheme handler factory for the specified |scheme_name| and
// |host_name|. All URLs beginning with scheme_name://host_name/ will be passed
// to the CefSchemeHandler instance returned by the CefSchemeHandlerFactory.
/*--cef()--*/
bool CefRegisterScheme(const std::wstring& scheme_name,
                       const std::wstring& host_name,
                       CefRefPtr<CefSchemeHandlerFactory> factory);


// Class that creates CefSchemeHandler instances.
/*--cef(source=client)--*/
class CefSchemeHandlerFactory : public CefBase
{
public:
  // Return a new scheme handler instance to handle the request.
  /*--cef()--*/
  virtual CefRefPtr<CefSchemeHandler> Create() =0;
};

// Class used to represent a custom scheme handler interface.
/*--cef(source=client)--*/
class CefSchemeHandler : public CefBase
{
public:
  // Process the request. All response generation should take place in this
  // method.
  /*--cef()--*/
  virtual bool ProcessRequest(CefRefPtr<CefRequest> request) =0;

  // Cancel processing of the request.
  /*--cef()--*/
  virtual void Cancel() =0;

  // Return the mime type for the request.
  /*--cef()--*/
  virtual std::wstring GetMimeType() =0;

  // Return the response length for the request. If there is no response return
  // 0 and ReadResponse() will not be called. If the response length is not
  // known return -1 and ReadResponse() will be called until it returns false or
  // until the value of |bytes_read| is set to 0. If the response length is
  // known return a positive value and ReadResponse() will be called until it
  // returns false, the value of |bytes_read| is set to 0 or the specified
  // number of bytes have been read.
  /*--cef()--*/
  virtual size_t GetResponseLength() =0;

  // Copy up to |bytes_to_read| bytes into |data_out|. If the copy succeeds
  // set |bytes_read| to the number of bytes copied and return true. If the
  // copy fails return false and ReadResponse() will not be called again.
  /*--cef()--*/
  virtual bool ReadResponse(void* data_out, int bytes_to_read,
                            int* bytes_read) =0;
};

#endif // _CEF_SCHEME_H

The new interface will require the following changes to the existing implementation:

1. Implement a clone of the URLRequestFilter class that stores a reference to a CefSchemeFactory instance instead of a function pointer. This change allows CEF to maintain the mapping to both the scheme and the host name.
2. Have the client create and return a new CefSchemeHandler instance every time the framework calls CefSchemeHandlerFactory::Create(). This way multiple simultaneous requests can be handled correctly and any serialization complexity is offloaded to the client.

[heshiming]

I agree with you. I also had the doubt about simultaneous processing. I've actually seen conflicts during later test with things such as loading multiple image tags at the same time.

I'll take your suggestion and change the corresponding code. This will single out each request to their own instance of handler. It appears that no locking is even required. It should be prepared within a couple of days. I'll post questions here if I've got any. Thanks.

Incidentally, have you had any information regarding XMLHttpRequest (the problem I had earlier) ?

[magreenblatt]

Incidentally, have you had any information regarding XMLHttpRequest (the problem I had earlier) ?

Do you have a simple JavaScript example that I can use to test this?

[heshiming]

Sure, it's actually included in my test client sample (mentioned in the first post).

Code: Select all

<script language="javascript">
function xmlpost(cmd)
{
var xhr = new XMLHttpRequest();
xhr.open("POST", "custom://host/xml", false);
xhr.send("<node>"+cmd+"</node>");
cmdresponse.innerHTML = xhr.responseText;
}
</script>


Just make sure you have a <div id="cmdresponse"></div> to see the response. And use a button to trigger it.

[heshiming]

Hi again. I've updated the first post to include my latest work. I've implemented the factory method. And it's working fine now under multi-thread. I've also updated the test client executable with a screenshot.

Some notes: I followed your suggestion about URLRequestFilter. A new CefUrlRequestFilter class is created. Nothing much was changed, I added a map to track factory method pointer, and a method for URLRequestJob to lookup this pointer. It's in scheme_impl.cc, I'm not sure if it's 100% appropriate. And I haven't updated to R34 of CEF yet. This is still R33 with Chromium R22728.

Thanks for the help. Comments are welcomed.

p.s. If it's okay, you can try putting it into the CEF tree. Also if you compile my sample handler with the official chromium tree, the XMLHttpRequest test in the sample client won't work. It should be easier to see what I'm talking about then.

[magreenblatt]

After thinking about this further, I wonder if there's any advantage to having GetMimeType() and GetResponseLength() as separate methods. What do you think about returning these from the ProcessRequest() method instead?

Code: Select all

virtual bool ProcessRequest(CefRefPtr<CefRequest> request, std::wstring& mime_type, int* response_length) =0;