AJAX with custom scheme

[JoeAndrieu]

I have been strugging to use a custom scheme for an AJAX call.

I am using the scheme "px:", e.g., "px://blahblah" and it refers to a local SPARQL endpoint I'm accessing over COM. I am not using the chrome .exe method for custom scheme handlers; instead, I am using CEF's CefRegisterScheme. For testing I use two different approaches. One actually makes it to my custom handler, the other doesn't.

Testing this in Chrome gives me this error, which led me to the underlying bug:

XMLHttpRequest cannot load px://blahblah. Cross origin requests are only supported for HTTP.

But with an unknown scheme, the meaning of the scheme-specific part is indeterminate. There is no way to actually know if it is cross origin. Which makes it a bug, but whether it's because of WebCore or a bug in integration, I don't know.

It turns out that if I use "px:///blahblah" it works just fine. So the scheme-specific part is being parsed as a domain and a path, which, it isn't.

So, is this something we should/could fix in Cef's scheme handling or is it buried in WebCore's assumptions about URLs?

[magreenblatt]

Are you saying that px:///foo can talk to px:///bar but px://foo can't talk to px://bar ? If so this makes sense because in the former case the domain is the same (though empty) and in the latter case the domain is different ('foo' and 'bar').

If you want to allow cross-site scripting you can set CefSettings.web_security_disabled to true (not recommended if the browser will access untrusted content). Otherwise, follow URL structure requirements because they're baked into the code pretty much everywhere.

Regards,
Marshall

[JoeAndrieu]

Actually, the page is file://pathtomyhtml.html, which can access px:///foo but not px://foo. Then, once it makes it to my SchemeHandler, the URL has been mangled, moving one of the / to the end.

For example,
px:///blamblam becomes px://blamblam/

And the following URL fails completely, with no reported error or exception:
px:///http%3A%2F%2Fswb.switchbook.com%2F7F31084B-7184-4546-9586-42D15424FDBA/sparql


It seems like there are two errors, either of which could be fixed and this might work.

First, px gets registered as standard, but it isn't standard. In c:\chromium\src\googleurl\src\url_util.cc, in DoCanonicalize, px: returns true from DoIsStandard (line 191). But you'll note that "weird" URLs (like data: and javascript:) are handled differently (lin 205) and do not return true from DoIsStandard. This registration is in c:\chromium\src\cef\libcef\scheme_impl.cc

For a "weird" url ParsePathURL and CanonicalisePathURL are called, which leave the host null and put the entire scheme-specific part in the path. This is what I think should happen for "non-standard" URLs like my custom scheme.

Second, webkit's standard URL handling can't handle empty hosts. (As per url_canon_stdurl.cc : // Host must not be empty for standard URLs.) It handles file URLs as a special case. As a result, px:///blamblam, "blamblam" gets set as the host.

Unfortunately, the standard canonicalization of the "host" gives me
"http:%2F%2Fswb.switchbook.com%2F7f31084b-7184-4546-9586-42d15424fdba"

Note the colon that got unescaped /and/ the other escape characters trigger a failure during the parsing.

This weekend, I'll try bypassing the scheme registration as standard in scheme_impl and see what happens. If that does the trick, maybe we just need to add a flag to CefRegisterScheme() to indicate whether or not it is a "standard scheme".

-j

[JoeAndrieu]

Ok. That worked. If we don't register the scheme as standard, it gets treated properly and makes it to my scheme handler without mangling, in both the short and long URLs.

We should probably fix this.

One option would be to add a "CefRegisterNonStandardScheme" or add an optional flag to CefRegisterScheme.

However, that would still allow the improper handling of "standard" custom schemes with null hosts. A null host allows local files to bypass the same-origin check for custom schemes. A URL like "px:///abc.com/index.html" currently gets treated as null origin (which passes the check against the "file:///" URL of the page) making it to my handler as "px://abc.com/index.html" after being processed by webkit as a standard URL. At the same time a URL like "px://abc.com/index.html" fails to reach my handler at all (presumably because of the same-origin check).

Is there a good reason for registering a custom scheme as standard? If so, we should probably either disallow null origin "standard" custom schemes or fix webkit to process them properly. Note that If we disallow null origin for custom schemes entirely (without adding a CefRegisterNonStandardScheme), it would prevent exactly the kind of CEF use I need, namely distributing local files for use in my app and using a custom scheme for IMGs and AJAX queries.

If there isn't a compelling reason, we could stop registering custom schemes as standard. After all, they aren't standard by their nature... they are custom. This would be the simplest fix.

-j

[magreenblatt]

If we don't register the scheme as standard, it gets treated properly and makes it to my scheme handler without mangling, in both the short and long URLs.

One possible solution which side steps the null host issues would be using a placeholder domain like px://app/ for all content instead of file://. Your application can translate px://app/path/file.html to c:\some\path\file.html and load the contents in CefHandler::HandleBeforeResourceLoad().

Is there a good reason for registering a custom scheme as standard?

The call to AddStandardScheme was added due to GURL changes in googleurl revision 123 (commit message below). The previous behavior was to treat all :// URLs as standard URLs.

Remove the rule that "://" means a standard URL. This fixes a number of bugs
and layout tests. The only thing we use now to determine whether a scheme is
standard scheme is whether it is on a known list.

Fix some bugs around setting protocols. Previously, we wouldn't reparse the
entire URL, which would give weird results in some cases, since which part
actually gets replaced changes if you change the protocol. This change has us
replace the protocol, then completely reparse the URL which ensures we don't
miss any edge cases. It is a bit slower, but changing the protocol is very
unusual.

This differs from KURL in several cases. On the
fast/dom/HTMLAnchorElement/set-href-attribute-hostname.html
layout test, we fail setting a hostname on a URL with "foo" protocol because we
no longer treat a "foo" URL as a standard protocol. However, this matches
Firefox's behavior, so I think there should not be compat problems.

Other differences in that test are that we allow setting hosts with spaces in
them for IE compat, and that when we generate the URL "c:/path/testurl.html" on
Windows, we reinterpret that as a file URL to file:///C:/path/testurl.html.
This is how "c:/path/testurl.html" would be interpreted if we just found it on
a page, so I think this behavior is correct, even though no other browser does
this (our handling of these generally matches IE, but it fails to set a "c"
protocol with an exception in this case).

http://crbug.com/160
Review URL: http://codereview.chromium.org/564011

[JoeAndrieu]

The workaround you suggest would function correctly, but it would be distorting the semantics of the scheme. The scheme means "access this local RDF file and perform a particular operation, using the permissions embedded in the file itself." It shouldn't be used to access resources outside the file. I do use it to load images from within the RDF, but the file-specific apps are part of the application, not part of a user file (which would be like having the standard menus for MS Excel loaded from within a .xls file).

After reading through the commit message, I checked to see what would happen with "px:foo".

Answer: it gets translated to "px://foo/".

From what I can tell, we actually went in the wrong direction responding to the GURL changes. The GURL fixes were to avoid treating too many URLs as "standard". The current code actually forces ALL custom schemes to be standard. This forces a canonicalization that isn't appropriate and exposes a security bug where custom schemes can be abused to access third-party origins when acting from a null origin context.

Given that commit comment, Marshall, I think we should make the default CefRegisterScheme assume the scheme is NOT "standard". If developers want to load a scheme that they claim /is/ standard, that should be the exception and can be supported with a CefRegisterStandardScheme.

Did we have unit tests that failed when GURL changed? Or was this proactive?

In the comment you quote, the only "failure" mode they mentioned is that they fail setting a hostname on a URL with a "foo" scheme. They note that is also Firefox's behavior. It also seems correct to me... unless a scheme /is/ standard, there's no reason to assume it has a hostname. Again, data: and javascript: urls are spot-on examples. And Cef fails to allow those kinds of schemes currently.

In my case, px:foo would work just fine... but it isn't handled properly today. Also, px://escapedCharacters/ fails.

[magreenblatt]

Did we have unit tests that failed when GURL changed? Or was this proactive?

The "Scheme Handler" test in cefclient fails with non-standard schemes. The call to AddStandardScheme was added to restore the previous behavior (treating all :// URLs as standard). The test fails for the following reasons:

1. Non-standard schemes don't support a host name concept (must pass an empty |host_name| argument to CefRegisterScheme).
2. Form submission doesn't work with non-standard schemes. I don't recall exactly where in the code this problem originates from.

I think we can add a boolean argument to CefRegisterScheme for specifying whether the scheme is standard or non-standard. We just need to document it very carefully so the expected difference in behavior is clear to the user. Please create an issue for this in the issue tracker and link to this thread (and provide a patch if you're feeling so inclined ).

[JoeAndrieu]

Cool.

I'll create an issue and figure out how to make a patch.

FWIW, the current code (1) doesn't treat all :// URLS as standard, it treats *all* URLs as standard and (2) ignores the underlying reason that GURL was modified, namely that treating all :// URLs as standard isn't the best option.

That's an Interesting issue with the form submissions. I have a feeling that is similar to my AJAX issue, namely the non-standard scheme gets interpreted as having a host and that fails either at the same-origin check or at the canonicalization. I'll try to look at the tests to see if I can find it.

I'll also make a pass at documenting the consequences of standard/nonstandard in the patch. Is there also a good place to add documentation for the custom schemes in general?

[magreenblatt]

Is there also a good place to add documentation for the custom schemes in general?

At a minimum we should document it in the cef.h comments. We can also add a page to the CEF Wiki if necessary.