-
Notifications
You must be signed in to change notification settings - Fork 44.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
browse_website is broken on Apple M1 & M1 Max due to chromedriver mismatch #2600
Comments
I have been hitting a very similar issue, albeit a different error code and @Pwuts suggested I add my logs to this Issue thread: I am running AutoGPT within a Docker container in continuous mode and everything works great until it tries to access a website and I receive the following error:
This results in a near-iterative loop as it works through the array of websites it initially finds in the Google API search. I am on a MacBook Pro with an M1 Max chip and I have configured my Docker container to use Pinecone, Google, Hugging Face, and my OpenAI GPT-4 API Key but the issue seems to be solely related to the browse_website function. I have tested on master, stable, and some other branches that claimed to solve similar issues to no avail. It sounds like another user has ran into the same issue and we are pretty certain it is due to the Mac M1 architecture but we are unclear on how to work around. Any help would be much appreciated! |
Let me know if anymore information would be helpful! |
I was the guy having issues with @camolechowski.. It seems like it's that particular error is happening on ╰─ docker-compose exec auto-gpt python -it
Python 3.10.11 (main, Apr 12 2023, 11:49:01) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from selenium import webdriver
>>> from webdriver_manager.chrome import ChromeDriverManager
>>>
>>> driver = webdriver.Chrome(ChromeDriverManager().install())
[WDM] - Downloading: 100%|███████| 6.75M/6.75M [00:00<00:00, 24.8MB/s]
<stdin>:1: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/appuser/.local/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 80, in __init__
super().__init__(
File "/home/appuser/.local/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 101, in __init__
self.service.start()
File "/home/appuser/.local/lib/python3.10/site-packages/selenium/webdriver/common/service.py", line 106, in start
self.assert_process_still_running()
File "/home/appuser/.local/lib/python3.10/site-packages/selenium/webdriver/common/service.py", line 119, in assert_process_still_running
raise WebDriverException(f"Service {self.path} unexpectedly exited. Status code was: {return_code}")
selenium.common.exceptions.WebDriverException: Message: Service /home/appuser/.wdm/drivers/chromedriver/linux64/112.0.5615.49/chromedriver unexpectedly exited. Status code was: 255
I just made removed and made sure I had no images built, and rebuilt using SYSTEM: Command browse_website returned: Error: Message: Service /home/appuser/.wdm/drivers/chromedriver/linux64/112.0.5615.49/chromedriver unexpectedly exited. Status code was: 255 |
Getting closer! appuser@9945be5e2d08:~/.wdm/drivers/chromedriver/linux64/112.0.5615.49$ ls -al .
total 20692
drwxr-xr-x 2 appuser appuser 4096 Apr 20 17:36 .
drwxr-xr-x 3 appuser appuser 4096 Apr 20 17:36 ..
-rw-r--r-- 1 appuser appuser 326400 Apr 20 17:36 LICENSE.chromedriver
-rwxr-xr-x 1 appuser appuser 13768328 Apr 20 17:36 chromedriver
-rw-r--r-- 1 appuser appuser 7079001 Apr 20 17:36 driver.zip
appuser@9945be5e2d08:~/.wdm/drivers/chromedriver/linux64/112.0.5615.49$ ./chromedriver
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
appuser@9945be5e2d08:~/.wdm/drivers/chromedriver/linux64/112.0.5615.49$ I do have... |
I am having same issues and logs: git commit hash : 14d3eca
|
I get that error If i change the build arch to emulate amd64
|
Please try adding these packages to the end of line 16 in the Dockerfile: Then rebuild and try again. Does that help? |
@vanities @Pwuts: I found a pretty ridiculous workaround that solves the issue:
Just manually added a step to install unzip and the correct driver in the Dockerfile and reference the directory directly. I know that doesn't help you much @Pwuts but @vanities I figured you were just trying to get it working, so I figured I'd share. UPDATE: That's embarrassing. I might have spoken too soon. Just a moment. |
hey I'm a noob feeling concerned about that here on a M1 max chip I get
So i'm subscribing to this issue |
@Pwuts: I tried your suggestion and unfortunately it did not work. Same error code:
|
I'm encountering the same error, in the same context, albeit on a Windows machine. Set my browser to Firefox, if that has any bearing on this.
|
The only thing I can do that makes it work is changing the
the build time 5x's though |
Tried within devcontainer but failed as follows:
|
Adding those libs isn't going to create a |
Joining late to the party. I had the same issue and this is what I did that solved the issue for me in my M1
it seems that the original line that installs the chrome driver is installing a wrong version while the version installed i the docker file is the proper one because:
Hope this helps others. |
This didn't resolve the issue for me. It produces a new error instead. Is this the only change you made? |
You're right, the error is captured here #2600 (comment) |
Did you not receive the same error I did that claims it cannot find the driver located at /usr/local/bin/chromedriver? |
Extended test results: Environment : macbook pro with M2 : on Test result on
Test result in
|
Just tried the latest stable, and here are results from my M1:
Status code was: 255 when enabled Dockers Rosetta for x86/amd64 emulation on Apple Silicon:
Status code was: -5 |
You are using /usr/local/bin/chromedriver, but you need to use /usr/bin/chromedriver because that is the location of the chromedriver installed in the Dockerfile definition. |
Just tried it, and it works! Even when Rosetta emulation is disabled in Docker. Have a stable branch, just changed one line in web_selenium.py This could be a solution, detect the platform, then use this setup (from README.md of webdriver_manager repo):
|
Editing This is the complete function I'm using in case it's easier to copy and paste:
Thank you so much for the fix! |
@jcalderonzumba Thank you so much. This one works for me as well! 🎉 |
@jcalderonzumba thanks a lot! We'll try matrix testing this and pushing a fix asap. |
Chipping in to say that @jcalderonzumba 's fix worked for me too. browse_website didn't work previously, it just errored out with different error messages like I am running AutoGPT locally on macOS M1. |
This will be fixed for Docker users and users who have already installed |
Docker + Chromedriver on M2 works with #1843 |
#2600 (comment) |
I tried this on my Macbook Pro M1 Pro but get this
|
It worked with me, Thanks a lot! |
Worked for me, I'm on M1, Thanks a lot! |
I did the above edit of the web_selenium.py on my M1 macbook, and have had no success. (using visual studio code) Command browse_website returned: Error: Message: unknown error: Chrome failed to start: exited abnormally. (chrome not reachable) (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.) Stacktrace: #0 0x0040006e6fe3 #1 0x004000425d36 #2 0x00400044eb20 #3 0x00400044aa9b #4 0x00400048caf7 #5 0x00400048c11f #6 0x004000483693 #7 0x00400045603a #8 0x00400045717e #9 0x0040006a8dbd #10 0x0040006acc6c #11 0x0040006b64b0 #12 0x0040006add63 #13 0x004000680c35 #14 0x0040006d1138 #15 0x0040006d12c7 #16 0x0040006df093 #17 0x004002585ea7 start_thread Additionally, this is my entire file, if it helps figure out why its not working. """Selenium web scraping module.""" import logging from bs4 import BeautifulSoup import autogpt.processing.text as summary FILE_DIR = Path(file).parent.parent def browse_website(url: str, question: str) -> tuple[str, WebDriver]:
def scrape_text_with_selenium(url: str) -> tuple[WebDriver, str]:
def scrape_links_with_selenium(driver: WebDriver, url: str) -> list[str]:
def close_browser(driver: WebDriver) -> None:
def add_header(driver: WebDriver) -> None:
|
Even if you try all suggested solutions to get it working on the latest release
There are two main issues:
you can verify this is the docker container path for
and then once in the container
not Doing both of the changes above fixed it |
@danielvoigt I tried what you recommended and it's still not working |
i changed the first part as you recommended, but im not sure where to go in order to type in "FROM --platform=linux/arm64 python:3.10-slim" |
@CharlieCappn you go to the
@0xG3ckers I don't see how that's possible. Are you running with Docker? What error are you getting? I would do the following steps.
It should work if you are pulling the correct image for M1 and referencing the correct path as the executable. Also, I think it would make sense to provide these directions in the README for M1 users. Apple M1 is highly used by engineers and it shouldn't be expected that everyone that uses AutoGPT knows how to debug Docker / platform dependency errors. |
@danielvoigt damn I really don't know what I'm doing wrong. I tried fresh installs before following your instructions too. I ran the Just checked through the file system in docker and found the chromedriver in the right location but the file was illegible. Maybe thats where the problem is? edit: finally got the |
Which Operating System are you using?
MacOS
GPT-3 or GPT-4?
GPT-4
Steps to reproduce 🕹
Let the GPT execute a
browse_website
commandCurrent behavior 😯
Browsing fails because Selenium can't find an appropriate chromedriver for Apple M1 arch. It tries to download
chromedriver_mac64_m1.zip
, while onlychromedriver_mac64.zip
exists (see repo).EDIT: apparently a similar issue occurs on Apple M1 Max which should use
mac_arm64
arch.Your Logs 📒
The text was updated successfully, but these errors were encountered: