How to send mouse events in Java-CEF browser component?

Having problems with building or using the JCEF Java binding? Ask your questions here.

How to send mouse events in Java-CEF browser component?

Postby rivolt80 » Sat Dec 08, 2018 3:40 am

I am trying to click a button programmatically on a webpage. For some reason , I cannot do it directly by executing JavaScript on my CefBrowser instance when the page is loaded. I tried the same JS code in Chrome dev console and it does the click without any problem. Due to this limitation, I am planning to simulate the click from within Java in a "native" way so I could bypass any browser security restrictions, if that's what preventing the click in JS.

My question is, how to do this in Window/non-osr mode?

I can see that CefBrowser_N defines methods like sendMouseEvent, but I am unable to make effective use of them. Here is what I am trying.

Because methods like sendMouseEvent are protected and my code which needs to call this resides outside org.cef.browser, I have changed the specifier to public. Again, because the documentation says we should not create instances of CefBrowser_N directly and instead opt to use CefBrowserFactory.create, I am defining a method CefBrowserFactory.createWrBrowser which returns an instance of CefBrowserWr, as the CefBrowser interface doesn't expose sendMouseEvent. The new method is implemented like this:

Code: Select all
public static CefBrowserWr createWrBrowser (
        CefClient client, String url, CefRequestContext context) {
    return new CefBrowserWr(client, url, context);
}

In a separate class, MyBrowserHandler, I am creating an instance of CefBrowserWr like this:

Code: Select all
CefClient client = MainFrame.getCefInstance().createClient();
CefBrowserWr browser = CefBrowserFactory.createWrBrowser(client, Properties.URL, null);


It might be of interest that I am passing null for context. Not sure what that does but I did it the way it is done in CefClient class. Check the source of CefClient.createBrowser and notify if I should be doing it some other way. Now that I am able to use sendMouseEvent, I am doing this when the page (and the element to be clicked) is loaded:

Code: Select all
 MouseEvent mouseEvent = new MouseEvent(
      browser.getUIComponent(),
      MouseEvent.MOUSE_CLICKED,
      System.currentTimeMillis() + 1000, // wait 1 second
      0,
      300, 60, // x, y co-ordinates
      1, false
);

browser.sendMouseEvent(mouseEvent);


I am unsure about the co-ordinates being accurate, but anyways, I am listening to document.onclick in JavaScript and logging "Clicked" when a click happens in the document. When I run my program, no click event is fired. When I manually click anywhere in the document, I see the message "Clicked". What am I doing wrong? Also, is there a cleaner way to send mouse clicks. Like an alternative to CefSharp's Browser.GetBrowser().GetHost().SendMouseClickEvent in Jcef?

I probably should mention that clicking the button should open a file dialog. When trying the same script in Jcef's dev console, file dialog opens as expected. How can I open the dialog without opening dev toolbar? Or if there is no way to do this without dev toolbar, how can I run my JS automatically from within devtoolbar?
rivolt80
Newbie
 
Posts: 9
Joined: Fri Apr 13, 2018 5:16 am

Re: How to send mouse events in Java-CEF browser component?

Postby rivolt80 » Sat Dec 08, 2018 9:30 am

Is runFileDialog going to help in any way? I have tried and it did not work for me. In JavaScript, I fire the click event and at the same time, signal Java through MessageRouter that the button has been clicked. After that, I runFileDialog on cefBrowser like this:

Code: Select all
System.out.println("Try opening file dialog here!");
Vector <String> filters = new Vector<>();
filters.add("video/*");
browser.runFileDialog(CefDialogHandler.FileDialogMode.FILE_DIALOG_OPEN,
"Choose File", "C://App/temp/sample.mkv", filters, 0, null);


This does not open file dialog. What else can I try?
rivolt80
Newbie
 
Posts: 9
Joined: Fri Apr 13, 2018 5:16 am

Re: How to send mouse events in Java-CEF browser component?

Postby Czarek » Sat Dec 08, 2018 3:54 pm

See below an example in Python for how to send mouse clicks programmatically using CEF API only. It works in windowed mode:
https://github.com/cztomczak/cefpython/ ... cks.py#L22

You have to perform 3 actions: move mouse, press button, release button.
Last edited by Czarek on Sun Dec 09, 2018 1:43 am, edited 2 times in total.
Maintainer of the CEF Python, PHP Desktop and CEF C API projects. My LinkedIn.
User avatar
Czarek
Virtuoso
 
Posts: 1927
Joined: Sun Nov 06, 2011 2:12 am

Re: How to send mouse events in Java-CEF browser component?

Postby rivolt80 » Sat Dec 08, 2018 11:19 pm

Yes. Worked! I was missing modifiers. Turns out, I have to add MouseEvent.BUTTON1_MASK modifier to make this work. Sending MOUSE_PRESSED and MOUSE_RELEASED work as expected now. Thanks!
rivolt80
Newbie
 
Posts: 9
Joined: Fri Apr 13, 2018 5:16 am


Return to JCEF Forum

Who is online

Users browsing this forum: No registered users and 2 guests