Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PoC] Edge: Mouse events on DOM -> SWT mouse events #1545

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
+90 −8
Open

[PoC] Edge: Mouse events on DOM -> SWT mouse events #1545

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 90 additions & 8 deletions bundles/org.eclipse.swt/Eclipse SWT Browser/win32/org/eclipse/swt/browser/Edge.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -641,6 +641,76 @@ void setupBrowser(int hr, long pv) {
webView.AddHostObjectToScript("swt\0".toCharArray(), hostObj);
hostDisp.Release();

class MouseNavigationFunction extends BrowserFunction {
public MouseNavigationFunction (Browser browser, String name) {
super (browser, name);
}
@Override
public Object function (Object[] arguments) {
System.out.println(Arrays.toString(arguments));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I receive the mouse events just fine, e.g. in the "internal web browser".

But for some reason I do not receive any events in the element info popup or the variable hover.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mouse events only seem to be processed when then content is retrieved from an "ordinary" URL. Whenever you set contents via setText (such as in the Javadoc view, in the Java element info popup, in the help) it does not work. It does not depend on how the widget is embedded: in the help view, the first page is custom content (also with a custom context menu), which does not report mouse events, while when navigating to other entries in the help (like from the "see also" list), mouse events will be reported.

So it seems to be some problem of registering the browser function for "custom content" pages. Is it even possible to register browser functions to them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what I thought as well at first, but it works perfectly fine with other setText()-based browser usages, so does the following example:

public class Test {

	public static void main(String[] args) {
		Display display = Display.getDefault();
		Shell shell = new Shell ();
		shell.setLayout(new FillLayout());

		new Browser(shell, SWT.EDGE).setText("hi");

		shell.open ();
		while (!shell.isDisposed()) {
			if (!display.readAndDispatch ()) display.sleep ();
		}
		display.dispose ();
	}
}

I don't understand what the element info / hover does differently.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, easy one... It's simply that those use Browser.setJavascriptEnabled(false)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://learn.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/iwebview2settings#get_isscriptenabled:

This only affects scripts in the document; scripts injected with ExecuteScript will run even if script is disabled. It is true by default.

So we need to register those listeners on the window, not the document.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, no dice. Doesn't make a difference... I guess this script-based approach won't be possible universally :(

Event newEvent = new Event ();
newEvent.widget = browser;
String eventType = (String) arguments[0];
int x = (int) Math.round((Double) arguments[1]);
int y= (int) Math.round((Double) arguments[2]);
int button = (int) Math.round((Double) arguments[3]);

if (OS.GetKeyState (OS.VK_MENU) < 0) newEvent.stateMask |= SWT.ALT;
if (OS.GetKeyState (OS.VK_SHIFT) < 0) newEvent.stateMask |= SWT.SHIFT;
if (OS.GetKeyState (OS.VK_CONTROL) < 0) newEvent.stateMask |= SWT.CONTROL;

x = DPIUtil.scaleUp(x, DPIUtil.getNativeDeviceZoom());
y = DPIUtil.scaleUp(y, DPIUtil.getNativeDeviceZoom());
x = DPIUtil.scaleDown(x, DPIUtil.getZoomForAutoscaleProperty(browser.getShell().nativeZoom)); //
y = DPIUtil.scaleDown(y, DPIUtil.getZoomForAutoscaleProperty(browser.getShell().nativeZoom));
newEvent.x = x;
newEvent.y = y;

switch (button) {
case 0: button = 1; break;
case 1: button = 3; break;
case 2: button = 2;
break;
}
if (eventType.equals("mousedown")) {
newEvent.type = SWT.MouseDown;
newEvent.button = button;
newEvent.count = 1;
} else if (eventType.equals("mouseup")/* || eventType.equals(EVENT_DRAGEND)*/) {
newEvent.type = SWT.MouseUp;
newEvent.button = button != 0 ? button : 1; /* button assumed to be 1 for dragends */
newEvent.count = 1;
switch (newEvent.button) {
case 1: newEvent.stateMask |= SWT.BUTTON1; break;
case 2: newEvent.stateMask |= SWT.BUTTON2; break;
case 3: newEvent.stateMask |= SWT.BUTTON3; break;
case 4: newEvent.stateMask |= SWT.BUTTON4; break;
case 5: newEvent.stateMask |= SWT.BUTTON5; break;
}
/*} else if (eventType.equals("mousewhell")) {
newEvent.type = SWT.MouseWheel;
rgdispid = newEvent.getIDsOfNames(new String[] { PROPERTY_WHEELDELTA });
dispIdMember = rgdispid[0];
pVarResult = newEvent.getProperty(dispIdMember);
newEvent.count = pVarResult.getInt () / 120 * 3;
pVarResult.dispose();*/
} else if (eventType.equals("mousemove")) {
newEvent.type = SWT.MouseMove;
} else if (eventType.equals("mouseover")) {
newEvent.type = SWT.MouseEnter;
} else if (eventType.equals("mouseout")) {
newEvent.type = SWT.MouseExit;
/*} else if (eventType.equals(EVENT_DRAGSTART)) {
newEvent.type = SWT.DragDetect;
newEvent.button = 1; // button assumed to be 1 for dragstarts
newEvent.stateMask |= SWT.BUTTON1;*/
}
browser.notifyListeners(newEvent.type, newEvent);
return null;
}
}
new MouseNavigationFunction(this.browser, "swtHandleMouse");

browser.addListener(SWT.Dispose, this::browserDispose);
browser.addListener(SWT.FocusIn, this::browserFocusIn);
browser.addListener(SWT.Resize, this::browserResize);
Expand Down Expand Up @@ -840,14 +910,6 @@ int handleNavigationStarting(long pView, long pArgs, boolean top) {
if (event.doit) {
jsEnabled = jsEnabledOnNextPage;
settings.put_IsScriptEnabled(jsEnabled);
// Register browser functions in the new document.
if (!functions.isEmpty()) {
StringBuilder sb = new StringBuilder();
for (BrowserFunction function : functions.values()) {
sb.append(function.functionString);
}
execute(sb.toString());
}
} else {
args.put_Cancel(true);
}
Expand Down Expand Up @@ -890,6 +952,7 @@ int handleDOMContentLoaded(long pView, long pArgs) {
if (startEvent != null && startEvent.top) {
if (lastCustomText != null && getUrl().equals(ABOUT_BLANK)) {
IUnknown postExecute = newCallback((long result, long json) -> {
registerBrowserFunctionsAndEventListenersInDocument();
sendProgressCompleted();
return COM.S_OK;
});
Expand All @@ -899,12 +962,31 @@ int handleDOMContentLoaded(long pView, long pArgs) {
postExecute.Release();
this.lastCustomText = null;
} else {
registerBrowserFunctionsAndEventListenersInDocument();
sendProgressCompleted();
}
}
return COM.S_OK;
}

private void registerBrowserFunctionsAndEventListenersInDocument() {
StringBuilder sb = new StringBuilder();
// Register browser functions in the new document.
if (!functions.isEmpty()) {
for (BrowserFunction function : functions.values()) {
sb.append(function.functionString);
}
execute(sb.toString());
}
sb.append("""
"mousedown mouseup mousewheel mouseover mouseout mousemove".split(" ").forEach(function(e){
window.document.addEventListener(e, (event)=>swtHandleMouse(e, event.x, event.y, event.button))
})""");
IUnknown newCallback = newCallback((r, j) -> COM.S_OK);
webViewProvider.getWebView(true).ExecuteScript(stringToWstr(sb.toString()), newCallback);
newCallback.Release();
}

private static String escapeForSingleQuotedJSString(String str) {
return str.replace("\\", "\\\\") //
.replace("'", "\\'") //
Expand Down
Loading