Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to try sample codes on Windows saying OSError #61

Closed
Atmos16 opened this issue Jul 26, 2024 · 6 comments
Closed

Failed to try sample codes on Windows saying OSError #61

Atmos16 opened this issue Jul 26, 2024 · 6 comments
Assignees

Comments

@Atmos16
Copy link

Atmos16 commented Jul 26, 2024

When I tried sample codes on Windows 11, it failed with OSError.
All commands are executed under venv environment, python 3.9.10.
How can I do with this?

Commands for installiing library

pip install "crawl4ai[all] @ git+https://github.com/unclecode/crawl4ai.git"

Executed Codes

Codes are almost same as ones on Quick Start

On PowerShell via VSCode,

from crawl4ai import WebCrawler
import os

os.environ["REQUESTS_CA_BUNDLE"] = "C:\\pathtocertificate"
os.environ["SSL_CERT_FILE"] = "C:\\pathtocertificate"

def create_crawler():
    crawler = WebCrawler(verbose=False)
    crawler.warmup()
    return crawler

crawler = create_crawler()

result = crawler.run(url="https://www.nbcnews.com/business", screenshot=True)
with open("screenshot.png", "wb") as f:
    f.write(base64.b64decode(result.screenshot))
print("Screenshot saved to 'screenshot.png'!")

Error message occurred during Executing codes

(c4a) PS C:\pathtodir> python crawl4aiSample.py
[LOG] 🚀 Initializing LocalSeleniumCrawlerStrategy
Traceback (most recent call last):
  File "C:\pathtodir\crawl4aiSample.py", line 13, in <module>
    crawler = create_crawler()
  File "C:\pathtodir\crawl4aiSample.py", line 9, in create_crawler
    crawler = WebCrawler(verbose=False)
  File "C:\pathtodir\c4a\lib\site-packages\crawl4ai\web_crawler.py", line 27, in __init__
    self.crawler_strategy = crawler_strategy or LocalSeleniumCrawlerStrategy(verbose=verbose)
  File "C:\pathtodir\c4a\lib\site-packages\crawl4ai\crawler_strategy.py", line 143, in __init__
    self.driver = webdriver.Chrome(service=self.service, options=self.options)
  File "C:\pathtodir\c4a\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 45, in __init__    
    super().__init__(
  File "C:\pathtodir\c4a\lib\site-packages\selenium\webdriver\chromium\webdriver.py", line 55, in __init__  
    self.service.start()
  File "C:\pathtodir\c4a\lib\site-packages\selenium\webdriver\common\service.py", line 98, in start
    self._start_process(self._path)
  File "C:\pathtodir\c4a\lib\site-packages\selenium\webdriver\common\service.py", line 208, in _start_process
    self.process = subprocess.Popen(
  File "C:\pathtouser\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\pathtouser\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 1420, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
OSError: [WinError 193] %1 は有効な Win32 アプリケーションではありません。
@aravindkarnam
Copy link
Contributor

@unclecode could be same as #62 🤔. I got similar error on linux, related to chrome driver.

@Atmos16 Can you install the latest version 4.0.2 of webdriver_manager with pip install --force-reinstall -v "webdriver_manager==4.0.2" and see if that solves this error.

@Atmos16
Copy link
Author

Atmos16 commented Jul 26, 2024

@aravindkarnam Thanks for your advise! I updated the library with pip install --force-reinstall -v "webdriver_manager==4.0.2" . However, the tip didn't work for me.

I found the issue @aravindkarnam mensioned in #62 has the environment with Selenium==4.22.0. I also tried that version, but it didn't change my situation, too.

@unclecode
Copy link
Owner

@aravindkarnam You are right, and @Atmos16 I will fix the docker file in 0.2.76, which will be available today or tomorrow

@unclecode unclecode self-assigned this Jul 27, 2024
@FractalMind
Copy link

@aravindkarnam You are right, and @Atmos16 I will fix the docker file in 0.2.76, which will be available today or tomorrow

I'm waiting for 0.2.76 to deploy on prod and stress tests the system. Do you have an estimation when it would be pushed?

@unclecode
Copy link
Owner

@FractalMind You can pull it from branch "main-75", above that there r a few more changes and all will be ready by tomorrow or dat after tomorrow.

@unclecode
Copy link
Owner

This issue has been resolved in v0.2.77 and later, so I will close it. Please let me know if you encounter any difficulties.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants