Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run dashboard on non-standard port and set dashboard link #25

Merged
merged 9 commits into from
Nov 8, 2023

Conversation

jacobtomlinson
Copy link
Collaborator

@jacobtomlinson jacobtomlinson commented Nov 8, 2023

It looks like when running Ray clusters on Databricks there is a way to access the dashboard.

https://docs.databricks.com/en/machine-learning/ray-integration.html

It seems that on Databricks you can proxy arbitrary ports via /driver-proxy/o/{orgId}/{clusterId}/{port}/. But when we try it with the Dask dashboard we see a 403 error.

This PR starts the dashboard on a few additional ports, including 8265 which is used by Ray's dashboard to see if 8787 is being explicitly blocked. It looks like Databricks just blocks 8787 for some reason, so this PR starts the Dask dashboard on 8087 too in case users want to run both Dask and Ray on the same cluster.

Also, set the dashboard_link to the correct URL.

Closes #2

@jacobtomlinson jacobtomlinson changed the title Run dashboard on Ray port and set Dashboard link Run dashboard on non-standard port and set dashboard link Nov 8, 2023
@jacobtomlinson
Copy link
Collaborator Author

I've been trying this out and the dashboard page loads, but not correctly. It looks like it's because of some absolute URLs in the Bokeh resources. This only seems to happen with the latest bokeh==3.3.0 and downgrading to 3.2.2 works around it.

@jacobtomlinson
Copy link
Collaborator Author

It works!

Screenshot 2023-11-08 at 14 45 21 image

@jacobtomlinson jacobtomlinson marked this pull request as ready for review November 8, 2023 14:48
@jacobtomlinson jacobtomlinson merged commit 371e2f6 into main Nov 8, 2023
7 checks passed
@jacobtomlinson jacobtomlinson deleted the dashboard-proxy branch November 8, 2023 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Access Dask dashboard through Databricks proxy
1 participant