browse_website is broken on RPi

[talvasconcelos]

Duplicates

• I have searched the existing issues

Summary 💡

On latest, using selenium, on an arm device, browse_website doesn't work, as it doesn't get the correct (if any) drivers.

Also, using chromium, as there's no chrome on the RPI, needs some selenium love, I'm trying to add it locally but still testing.

Please guys, don't start assuming everyone is on a desktop, or on windows, or any assumptions!

Examples 🌈

No response

Motivation 🔦

No response

[2tYfX8sgnrx]

I tried adding mozilla geckodriver in attempt to get Firefox working but it still defaults to chrome which is not installed on the M1 Macbook I'm running.

Changing the .env file settings to point towards Safari also isn't working at the moment, defaulting to chrome still.

Selenium does need love.

[k-boikov]

Might be fixed with #1473

[Pwuts]

Please guys, don't start assuming everyone is on a desktop, or on windows, or any assumptions!

We learned that the hard way over the past week :')

[sadmuphin]

Chromium is supported, and with #2066 you should be able to specify the chromium binary (I'm not sure if that will fix as I think you would still selenium drivers). However I do agree, non-selenium web-scraping should still be supported.

[Pwuts]

@talvasconcelos can you post the output of the WebDriver manager? (like in #2600)

[Pwuts]

SeleniumHQ/docker-selenium#1076

[talvasconcelos]

@talvasconcelos can you post the output of the WebDriver manager? (like in #2600)

i don't have an output, webdriver just installs the wrong driver (i assume it's because of the settings in user-agent)

[Pwuts]

Which one does it install, and which one should it install?

[talvasconcelos]

Which one does it install, and which one should it install?

this is the error i get on the RPI: SYSTEM: Command browse_website returned: Error: [Errno 8] Exec format error: '~/.wdm/drivers/chromedriver/linux64/112.0.5615.49/chromedriver'

I believe the issue is in this line, that "hardcodes" a platform and a driver: https://github.com/Significant-Gravitas/Auto-GPT/blob/20ef130341981e7500039e31d87e566e432acdd7/autogpt/commands/web_selenium.py#L76-L78

Since there's no chrome in the RPI, only chromium, this fails not only because it's getting a driver for the wrong architecture, as it insists on chrome! I found this, if it helps making it work: https://ivanderevianko.com/2020/01/selenium-chromedriver-for-raspberrypi

[k-boikov]

The user-agent is used only for the http request header, that could not be the issue.

[github-actions]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[github-actions]

This issue was closed automatically because it has been stale for 10 days with no activity.