Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] incompatible architecture loading model for M1 Pro #259

Open
jgilewski-kone opened this issue Nov 20, 2024 · 2 comments
Open

[BUG] incompatible architecture loading model for M1 Pro #259

jgilewski-kone opened this issue Nov 20, 2024 · 2 comments
Assignees
@Davidqian123
Labels
🐞 bug Something isn't working

Comments

@jgilewski-kone
Copy link

Issue Description

I did a fresh installation of Nexa (v0.0.9.2) on MacOs Montery with Apple M1 Pro chip with the command from the documentation:
CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir

Everything went smoothly without error with the following logs:

Looking in indexes: https://nexaai.github.io/nexa-sdk/whl/metal, https://pypi.org/simple
Collecting nexaai
  Downloading https://github.com/NexaAI/nexa-sdk/releases/download/v0.0.9.2-metal/nexaai-0.0.9.2-cp311-cp311-macosx_12_0_arm64.whl (5.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 35.5 MB/s eta 0:00:00
Collecting cmake (from nexaai)
  Downloading cmake-3.31.0.1-py3-none-macosx_10_10_universal2.whl.metadata (6.4 kB)
Collecting faster_whisper (from nexaai)
  Downloading faster_whisper-1.0.3-py3-none-any.whl.metadata (15 kB)
Collecting typing-extensions>=4.5.0 (from nexaai)
  Downloading typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Collecting numpy>=1.20.0 (from nexaai)
  Downloading numpy-2.1.3-cp311-cp311-macosx_11_0_arm64.whl.metadata (116 kB)
Collecting diskcache>=5.6.1 (from nexaai)
  Downloading diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)
Collecting jinja2>=2.11.3 (from nexaai)
  Downloading jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting librosa>=0.8.0 (from nexaai)
  Downloading librosa-0.10.2.post1-py3-none-any.whl.metadata (8.6 kB)
Collecting fastapi (from nexaai)
  Downloading fastapi-0.115.5-py3-none-any.whl.metadata (27 kB)
Collecting uvicorn (from nexaai)
  Downloading uvicorn-0.32.0-py3-none-any.whl.metadata (6.6 kB)
Collecting pydantic (from nexaai)
  Downloading pydantic-2.9.2-py3-none-any.whl.metadata (149 kB)
Collecting pillow (from nexaai)
  Downloading pillow-11.0.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (9.1 kB)
Collecting huggingface_hub (from nexaai)
  Downloading huggingface_hub-0.26.2-py3-none-any.whl.metadata (13 kB)
Collecting prompt_toolkit (from nexaai)
  Downloading prompt_toolkit-3.0.48-py3-none-any.whl.metadata (6.4 kB)
Collecting tqdm (from nexaai)
  Downloading tqdm-4.67.0-py3-none-any.whl.metadata (57 kB)
Collecting tabulate (from nexaai)
  Downloading tabulate-0.9.0-py3-none-any.whl.metadata (34 kB)
Collecting streamlit>=1.37.1 (from nexaai)
  Downloading streamlit-1.40.1-py2.py3-none-any.whl.metadata (8.5 kB)
Collecting streamlit-audiorec (from nexaai)
  Downloading streamlit_audiorec-0.1.3-py3-none-any.whl.metadata (2.4 kB)
Collecting python-multipart (from nexaai)
  Downloading python_multipart-0.0.17-py3-none-any.whl.metadata (1.8 kB)
Collecting MarkupSafe>=2.0 (from jinja2>=2.11.3->nexaai)
  Downloading MarkupSafe-3.0.2-cp311-cp311-macosx_11_0_arm64.whl.metadata (4.0 kB)
Collecting audioread>=2.1.9 (from librosa>=0.8.0->nexaai)
  Downloading audioread-3.0.1-py3-none-any.whl.metadata (8.4 kB)
Collecting scipy>=1.2.0 (from librosa>=0.8.0->nexaai)
  Downloading scipy-1.14.1-cp311-cp311-macosx_12_0_arm64.whl.metadata (60 kB)
Collecting scikit-learn>=0.20.0 (from librosa>=0.8.0->nexaai)
  Downloading scikit_learn-1.5.2-cp311-cp311-macosx_12_0_arm64.whl.metadata (13 kB)
Collecting joblib>=0.14 (from librosa>=0.8.0->nexaai)
  Downloading joblib-1.4.2-py3-none-any.whl.metadata (5.4 kB)
Collecting decorator>=4.3.0 (from librosa>=0.8.0->nexaai)
  Downloading decorator-5.1.1-py3-none-any.whl.metadata (4.0 kB)
Collecting numba>=0.51.0 (from librosa>=0.8.0->nexaai)
  Downloading numba-0.60.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (2.7 kB)
Collecting soundfile>=0.12.1 (from librosa>=0.8.0->nexaai)
  Downloading soundfile-0.12.1-py2.py3-none-macosx_11_0_arm64.whl.metadata (14 kB)
Collecting pooch>=1.1 (from librosa>=0.8.0->nexaai)
  Downloading pooch-1.8.2-py3-none-any.whl.metadata (10 kB)
Collecting soxr>=0.3.2 (from librosa>=0.8.0->nexaai)
  Downloading soxr-0.5.0.post1-cp311-cp311-macosx_11_0_arm64.whl.metadata (5.6 kB)
Collecting lazy-loader>=0.1 (from librosa>=0.8.0->nexaai)
  Downloading lazy_loader-0.4-py3-none-any.whl.metadata (7.6 kB)
Collecting msgpack>=1.0 (from librosa>=0.8.0->nexaai)
  Downloading msgpack-1.1.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (8.4 kB)
Collecting altair<6,>=4.0 (from streamlit>=1.37.1->nexaai)
  Downloading altair-5.4.1-py3-none-any.whl.metadata (9.4 kB)
Collecting blinker<2,>=1.0.0 (from streamlit>=1.37.1->nexaai)
  Downloading blinker-1.9.0-py3-none-any.whl.metadata (1.6 kB)
Collecting cachetools<6,>=4.0 (from streamlit>=1.37.1->nexaai)
  Downloading cachetools-5.5.0-py3-none-any.whl.metadata (5.3 kB)
Collecting click<9,>=7.0 (from streamlit>=1.37.1->nexaai)
  Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting packaging<25,>=20 (from streamlit>=1.37.1->nexaai)
  Downloading packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
Collecting pandas<3,>=1.4.0 (from streamlit>=1.37.1->nexaai)
  Downloading pandas-2.2.3-cp311-cp311-macosx_11_0_arm64.whl.metadata (89 kB)
Collecting protobuf<6,>=3.20 (from streamlit>=1.37.1->nexaai)
  Downloading protobuf-5.28.3-cp38-abi3-macosx_10_9_universal2.whl.metadata (592 bytes)
Collecting pyarrow>=7.0 (from streamlit>=1.37.1->nexaai)
  Downloading pyarrow-18.0.0-cp311-cp311-macosx_12_0_arm64.whl.metadata (3.3 kB)
Collecting requests<3,>=2.27 (from streamlit>=1.37.1->nexaai)
  Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting rich<14,>=10.14.0 (from streamlit>=1.37.1->nexaai)
  Downloading rich-13.9.4-py3-none-any.whl.metadata (18 kB)
Collecting tenacity<10,>=8.1.0 (from streamlit>=1.37.1->nexaai)
  Downloading tenacity-9.0.0-py3-none-any.whl.metadata (1.2 kB)
Collecting toml<2,>=0.10.1 (from streamlit>=1.37.1->nexaai)
  Downloading toml-0.10.2-py2.py3-none-any.whl.metadata (7.1 kB)
Collecting gitpython!=3.1.19,<4,>=3.0.7 (from streamlit>=1.37.1->nexaai)
  Downloading GitPython-3.1.43-py3-none-any.whl.metadata (13 kB)
Collecting pydeck<1,>=0.8.0b4 (from streamlit>=1.37.1->nexaai)
  Downloading pydeck-0.9.1-py2.py3-none-any.whl.metadata (4.1 kB)
Collecting tornado<7,>=6.0.3 (from streamlit>=1.37.1->nexaai)
  Downloading tornado-6.4.1-cp38-abi3-macosx_10_9_universal2.whl.metadata (2.5 kB)
Collecting starlette<0.42.0,>=0.40.0 (from fastapi->nexaai)
  Downloading starlette-0.41.3-py3-none-any.whl.metadata (6.0 kB)
Collecting annotated-types>=0.6.0 (from pydantic->nexaai)
  Downloading annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)
Collecting pydantic-core==2.23.4 (from pydantic->nexaai)
  Downloading pydantic_core-2.23.4-cp311-cp311-macosx_11_0_arm64.whl.metadata (6.6 kB)
Collecting av<13,>=11.0 (from faster_whisper->nexaai)
  Downloading av-12.3.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (4.6 kB)
Collecting ctranslate2<5,>=4.0 (from faster_whisper->nexaai)
  Downloading ctranslate2-4.5.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (10 kB)
Collecting tokenizers<1,>=0.13 (from faster_whisper->nexaai)
  Downloading tokenizers-0.20.3-cp311-cp311-macosx_11_0_arm64.whl.metadata (6.7 kB)
Collecting onnxruntime<2,>=1.14 (from faster_whisper->nexaai)
  Downloading onnxruntime-1.19.2-cp311-cp311-macosx_11_0_universal2.whl.metadata (4.5 kB)
Collecting filelock (from huggingface_hub->nexaai)
  Downloading filelock-3.16.1-py3-none-any.whl.metadata (2.9 kB)
Collecting fsspec>=2023.5.0 (from huggingface_hub->nexaai)
  Downloading fsspec-2024.10.0-py3-none-any.whl.metadata (11 kB)
Collecting pyyaml>=5.1 (from huggingface_hub->nexaai)
  Downloading PyYAML-6.0.2-cp311-cp311-macosx_11_0_arm64.whl.metadata (2.1 kB)
Collecting wcwidth (from prompt_toolkit->nexaai)
  Downloading wcwidth-0.2.13-py2.py3-none-any.whl.metadata (14 kB)
Collecting h11>=0.8 (from uvicorn->nexaai)
  Downloading h11-0.14.0-py3-none-any.whl.metadata (8.2 kB)
Collecting jsonschema>=3.0 (from altair<6,>=4.0->streamlit>=1.37.1->nexaai)
  Downloading jsonschema-4.23.0-py3-none-any.whl.metadata (7.9 kB)
Collecting narwhals>=1.5.2 (from altair<6,>=4.0->streamlit>=1.37.1->nexaai)
  Downloading narwhals-1.14.1-py3-none-any.whl.metadata (7.4 kB)
Requirement already satisfied: setuptools in ./.venv/lib/python3.11/site-packages (from ctranslate2<5,>=4.0->faster_whisper->nexaai) (74.1.2)
Collecting gitdb<5,>=4.0.1 (from gitpython!=3.1.19,<4,>=3.0.7->streamlit>=1.37.1->nexaai)
  Downloading gitdb-4.0.11-py3-none-any.whl.metadata (1.2 kB)
Collecting llvmlite<0.44,>=0.43.0dev0 (from numba>=0.51.0->librosa>=0.8.0->nexaai)
  Downloading llvmlite-0.43.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (4.8 kB)
Collecting numpy>=1.20.0 (from nexaai)
  Downloading numpy-2.0.2-cp311-cp311-macosx_11_0_arm64.whl.metadata (60 kB)
Collecting coloredlogs (from onnxruntime<2,>=1.14->faster_whisper->nexaai)
  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl.metadata (12 kB)
Collecting flatbuffers (from onnxruntime<2,>=1.14->faster_whisper->nexaai)
  Downloading flatbuffers-24.3.25-py2.py3-none-any.whl.metadata (850 bytes)
Collecting sympy (from onnxruntime<2,>=1.14->faster_whisper->nexaai)
  Downloading sympy-1.13.3-py3-none-any.whl.metadata (12 kB)
Collecting python-dateutil>=2.8.2 (from pandas<3,>=1.4.0->streamlit>=1.37.1->nexaai)
  Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytz>=2020.1 (from pandas<3,>=1.4.0->streamlit>=1.37.1->nexaai)
  Downloading pytz-2024.2-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.7 (from pandas<3,>=1.4.0->streamlit>=1.37.1->nexaai)
  Downloading tzdata-2024.2-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting platformdirs>=2.5.0 (from pooch>=1.1->librosa>=0.8.0->nexaai)
  Downloading platformdirs-4.3.6-py3-none-any.whl.metadata (11 kB)
Collecting charset-normalizer<4,>=2 (from requests<3,>=2.27->streamlit>=1.37.1->nexaai)
  Downloading charset_normalizer-3.4.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests<3,>=2.27->streamlit>=1.37.1->nexaai)
  Downloading idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3,>=2.27->streamlit>=1.37.1->nexaai)
  Downloading urllib3-2.2.3-py3-none-any.whl.metadata (6.5 kB)
Collecting certifi>=2017.4.17 (from requests<3,>=2.27->streamlit>=1.37.1->nexaai)
  Downloading certifi-2024.8.30-py3-none-any.whl.metadata (2.2 kB)
Collecting markdown-it-py>=2.2.0 (from rich<14,>=10.14.0->streamlit>=1.37.1->nexaai)
  Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich<14,>=10.14.0->streamlit>=1.37.1->nexaai)
  Downloading pygments-2.18.0-py3-none-any.whl.metadata (2.5 kB)
Collecting threadpoolctl>=3.1.0 (from scikit-learn>=0.20.0->librosa>=0.8.0->nexaai)
  Downloading threadpoolctl-3.5.0-py3-none-any.whl.metadata (13 kB)
Collecting cffi>=1.0 (from soundfile>=0.12.1->librosa>=0.8.0->nexaai)
  Downloading cffi-1.17.1-cp311-cp311-macosx_11_0_arm64.whl.metadata (1.5 kB)
Collecting anyio<5,>=3.4.0 (from starlette<0.42.0,>=0.40.0->fastapi->nexaai)
  Downloading anyio-4.6.2.post1-py3-none-any.whl.metadata (4.7 kB)
Collecting sniffio>=1.1 (from anyio<5,>=3.4.0->starlette<0.42.0,>=0.40.0->fastapi->nexaai)
  Downloading sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)
Collecting pycparser (from cffi>=1.0->soundfile>=0.12.1->librosa>=0.8.0->nexaai)
  Downloading pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython!=3.1.19,<4,>=3.0.7->streamlit>=1.37.1->nexaai)
  Downloading smmap-5.0.1-py3-none-any.whl.metadata (4.3 kB)
Collecting attrs>=22.2.0 (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=1.37.1->nexaai)
  Downloading attrs-24.2.0-py3-none-any.whl.metadata (11 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=1.37.1->nexaai)
  Downloading jsonschema_specifications-2024.10.1-py3-none-any.whl.metadata (3.0 kB)
Collecting referencing>=0.28.4 (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=1.37.1->nexaai)
  Downloading referencing-0.35.1-py3-none-any.whl.metadata (2.8 kB)
Collecting rpds-py>=0.7.1 (from jsonschema>=3.0->altair<6,>=4.0->streamlit>=1.37.1->nexaai)
  Downloading rpds_py-0.21.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (4.2 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich<14,>=10.14.0->streamlit>=1.37.1->nexaai)
  Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Collecting six>=1.5 (from python-dateutil>=2.8.2->pandas<3,>=1.4.0->streamlit>=1.37.1->nexaai)
  Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime<2,>=1.14->faster_whisper->nexaai)
  Downloading humanfriendly-10.0-py2.py3-none-any.whl.metadata (9.2 kB)
Collecting mpmath<1.4,>=1.1.0 (from sympy->onnxruntime<2,>=1.14->faster_whisper->nexaai)
  Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Downloading diskcache-5.6.3-py3-none-any.whl (45 kB)
Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
Downloading librosa-0.10.2.post1-py3-none-any.whl (260 kB)
Downloading streamlit-1.40.1-py2.py3-none-any.whl (8.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.6/8.6 MB 50.3 MB/s eta 0:00:00
Downloading pillow-11.0.0-cp311-cp311-macosx_11_0_arm64.whl (3.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 142.4 MB/s eta 0:00:00
Downloading typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Downloading cmake-3.31.0.1-py3-none-macosx_10_10_universal2.whl (47.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 47.2/47.2 MB 108.3 MB/s eta 0:00:00
Downloading fastapi-0.115.5-py3-none-any.whl (94 kB)
Downloading pydantic-2.9.2-py3-none-any.whl (434 kB)
Downloading pydantic_core-2.23.4-cp311-cp311-macosx_11_0_arm64.whl (1.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 119.6 MB/s eta 0:00:00
Downloading faster_whisper-1.0.3-py3-none-any.whl (2.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 127.5 MB/s eta 0:00:00
Downloading huggingface_hub-0.26.2-py3-none-any.whl (447 kB)
Downloading tqdm-4.67.0-py3-none-any.whl (78 kB)
Downloading prompt_toolkit-3.0.48-py3-none-any.whl (386 kB)
Downloading python_multipart-0.0.17-py3-none-any.whl (24 kB)
Downloading streamlit_audiorec-0.1.3-py3-none-any.whl (872 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 872.8/872.8 kB 360.6 MB/s eta 0:00:00
Downloading tabulate-0.9.0-py3-none-any.whl (35 kB)
Downloading uvicorn-0.32.0-py3-none-any.whl (63 kB)
Downloading altair-5.4.1-py3-none-any.whl (658 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 658.1/658.1 kB 467.2 MB/s eta 0:00:00
Downloading annotated_types-0.7.0-py3-none-any.whl (13 kB)
Downloading audioread-3.0.1-py3-none-any.whl (23 kB)
Downloading av-12.3.0-cp311-cp311-macosx_11_0_arm64.whl (19.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.9/19.9 MB 106.0 MB/s eta 0:00:00
Downloading blinker-1.9.0-py3-none-any.whl (8.5 kB)
Downloading cachetools-5.5.0-py3-none-any.whl (9.5 kB)
Downloading click-8.1.7-py3-none-any.whl (97 kB)
Downloading ctranslate2-4.5.0-cp311-cp311-macosx_11_0_arm64.whl (1.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 26.6 MB/s eta 0:00:00
Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB)
Downloading fsspec-2024.10.0-py3-none-any.whl (179 kB)
Downloading GitPython-3.1.43-py3-none-any.whl (207 kB)
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
Downloading lazy_loader-0.4-py3-none-any.whl (12 kB)
Downloading MarkupSafe-3.0.2-cp311-cp311-macosx_11_0_arm64.whl (12 kB)
Downloading msgpack-1.1.0-cp311-cp311-macosx_11_0_arm64.whl (81 kB)
Downloading numba-0.60.0-cp311-cp311-macosx_11_0_arm64.whl (2.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 1.5 GB/s eta 0:00:00
Downloading numpy-2.0.2-cp311-cp311-macosx_11_0_arm64.whl (13.7 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.7/13.7 MB 5.2 MB/s eta 0:00:00
Downloading onnxruntime-1.19.2-cp311-cp311-macosx_11_0_universal2.whl (16.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.8/16.8 MB 8.6 MB/s eta 0:00:00
Downloading packaging-24.2-py3-none-any.whl (65 kB)
Downloading pandas-2.2.3-cp311-cp311-macosx_11_0_arm64.whl (11.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.3/11.3 MB 105.1 MB/s eta 0:00:00
Downloading pooch-1.8.2-py3-none-any.whl (64 kB)
Downloading protobuf-5.28.3-cp38-abi3-macosx_10_9_universal2.whl (414 kB)
Downloading pyarrow-18.0.0-cp311-cp311-macosx_12_0_arm64.whl (29.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 29.5/29.5 MB 107.5 MB/s eta 0:00:00
Downloading pydeck-0.9.1-py2.py3-none-any.whl (6.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.9/6.9 MB 113.1 MB/s eta 0:00:00
Downloading PyYAML-6.0.2-cp311-cp311-macosx_11_0_arm64.whl (172 kB)
Downloading requests-2.32.3-py3-none-any.whl (64 kB)
Downloading rich-13.9.4-py3-none-any.whl (242 kB)
Downloading scikit_learn-1.5.2-cp311-cp311-macosx_12_0_arm64.whl (11.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.0/11.0 MB 113.1 MB/s eta 0:00:00
Downloading scipy-1.14.1-cp311-cp311-macosx_12_0_arm64.whl (29.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 29.9/29.9 MB 110.1 MB/s eta 0:00:00
Downloading soundfile-0.12.1-py2.py3-none-macosx_11_0_arm64.whl (1.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 203.8 MB/s eta 0:00:00
Downloading soxr-0.5.0.post1-cp311-cp311-macosx_11_0_arm64.whl (159 kB)
Downloading starlette-0.41.3-py3-none-any.whl (73 kB)
Downloading tenacity-9.0.0-py3-none-any.whl (28 kB)
Downloading tokenizers-0.20.3-cp311-cp311-macosx_11_0_arm64.whl (2.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 206.4 MB/s eta 0:00:00
Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Downloading tornado-6.4.1-cp38-abi3-macosx_10_9_universal2.whl (435 kB)
Downloading filelock-3.16.1-py3-none-any.whl (16 kB)
Downloading wcwidth-0.2.13-py2.py3-none-any.whl (34 kB)
Downloading anyio-4.6.2.post1-py3-none-any.whl (90 kB)
Downloading certifi-2024.8.30-py3-none-any.whl (167 kB)
Downloading cffi-1.17.1-cp311-cp311-macosx_11_0_arm64.whl (178 kB)
Downloading charset_normalizer-3.4.0-cp311-cp311-macosx_11_0_arm64.whl (118 kB)
Downloading gitdb-4.0.11-py3-none-any.whl (62 kB)
Downloading idna-3.10-py3-none-any.whl (70 kB)
Downloading jsonschema-4.23.0-py3-none-any.whl (88 kB)
Downloading llvmlite-0.43.0-cp311-cp311-macosx_11_0_arm64.whl (28.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 28.8/28.8 MB 104.7 MB/s eta 0:00:00
Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Downloading narwhals-1.14.1-py3-none-any.whl (220 kB)
Downloading platformdirs-4.3.6-py3-none-any.whl (18 kB)
Downloading pygments-2.18.0-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 320.5 MB/s eta 0:00:00
Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Downloading pytz-2024.2-py2.py3-none-any.whl (508 kB)
Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Downloading tzdata-2024.2-py2.py3-none-any.whl (346 kB)
Downloading urllib3-2.2.3-py3-none-any.whl (126 kB)
Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
Downloading flatbuffers-24.3.25-py2.py3-none-any.whl (26 kB)
Downloading sympy-1.13.3-py3-none-any.whl (6.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.2/6.2 MB 110.8 MB/s eta 0:00:00
Downloading attrs-24.2.0-py3-none-any.whl (63 kB)
Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
Downloading jsonschema_specifications-2024.10.1-py3-none-any.whl (18 kB)
Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 1.1 GB/s eta 0:00:00
Downloading referencing-0.35.1-py3-none-any.whl (26 kB)
Downloading rpds_py-0.21.0-cp311-cp311-macosx_11_0_arm64.whl (318 kB)
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Downloading smmap-5.0.1-py3-none-any.whl (24 kB)
Downloading sniffio-1.3.1-py3-none-any.whl (10 kB)
Downloading pycparser-2.22-py3-none-any.whl (117 kB)
Installing collected packages: wcwidth, pytz, mpmath, flatbuffers, urllib3, tzdata, typing-extensions, tqdm, tornado, toml, threadpoolctl, tenacity, tabulate, sympy, sniffio, smmap, six, rpds-py, pyyaml, python-multipart, pygments, pycparser, pyarrow, protobuf, prompt_toolkit, platformdirs, pillow, packaging, numpy, narwhals, msgpack, mdurl, MarkupSafe, llvmlite, joblib, idna, humanfriendly, h11, fsspec, filelock, diskcache, decorator, cmake, click, charset-normalizer, certifi, cachetools, blinker, av, audioread, attrs, annotated-types, uvicorn, soxr, scipy, requests, referencing, python-dateutil, pydantic-core, numba, markdown-it-py, lazy-loader, jinja2, gitdb, ctranslate2, coloredlogs, cffi, anyio, starlette, soundfile, scikit-learn, rich, pydeck, pydantic, pooch, pandas, onnxruntime, jsonschema-specifications, huggingface_hub, gitpython, tokenizers, librosa, jsonschema, fastapi, faster_whisper, altair, streamlit, streamlit-audiorec, nexaai
Successfully installed MarkupSafe-3.0.2 altair-5.4.1 annotated-types-0.7.0 anyio-4.6.2.post1 attrs-24.2.0 audioread-3.0.1 av-12.3.0 blinker-1.9.0 cachetools-5.5.0 certifi-2024.8.30 cffi-1.17.1 charset-normalizer-3.4.0 click-8.1.7 cmake-3.31.0.1 coloredlogs-15.0.1 ctranslate2-4.5.0 decorator-5.1.1 diskcache-5.6.3 fastapi-0.115.5 faster_whisper-1.0.3 filelock-3.16.1 flatbuffers-24.3.25 fsspec-2024.10.0 gitdb-4.0.11 gitpython-3.1.43 h11-0.14.0 huggingface_hub-0.26.2 humanfriendly-10.0 idna-3.10 jinja2-3.1.4 joblib-1.4.2 jsonschema-4.23.0 jsonschema-specifications-2024.10.1 lazy-loader-0.4 librosa-0.10.2.post1 llvmlite-0.43.0 markdown-it-py-3.0.0 mdurl-0.1.2 mpmath-1.3.0 msgpack-1.1.0 narwhals-1.14.1 nexaai-0.0.9.2 numba-0.60.0 numpy-2.0.2 onnxruntime-1.19.2 packaging-24.2 pandas-2.2.3 pillow-11.0.0 platformdirs-4.3.6 pooch-1.8.2 prompt_toolkit-3.0.48 protobuf-5.28.3 pyarrow-18.0.0 pycparser-2.22 pydantic-2.9.2 pydantic-core-2.23.4 pydeck-0.9.1 pygments-2.18.0 python-dateutil-2.9.0.post0 python-multipart-0.0.17 pytz-2024.2 pyyaml-6.0.2 referencing-0.35.1 requests-2.32.3 rich-13.9.4 rpds-py-0.21.0 scikit-learn-1.5.2 scipy-1.14.1 six-1.16.0 smmap-5.0.1 sniffio-1.3.1 soundfile-0.12.1 soxr-0.5.0.post1 starlette-0.41.3 streamlit-1.40.1 streamlit-audiorec-0.1.3 sympy-1.13.3 tabulate-0.9.0 tenacity-9.0.0 threadpoolctl-3.5.0 tokenizers-0.20.3 toml-0.10.2 tornado-6.4.1 tqdm-4.67.0 typing-extensions-4.12.2 tzdata-2024.2 urllib3-2.2.3 uvicorn-0.32.0 wcwidth-0.2.13

Trying to load a model like nexa run llama2 gives me the following error:

Downloading Llama-2-7b-chat/q4_0.gguf...
q4_0.gguf: 100%|████████████████████████████████████████████████████████████████████| 3.56G/3.56G [00:36<00:00, 104MB/s]
Successfully downloaded Llama-2-7b-chat/q4_0.gguf to /Users/jgi/.cache/nexa/hub/official/Llama-2-7b-chat/q4_0.gguf
Successfully pulled model Llama-2-7b-chat:q4_0 to /Users/jgi/.cache/nexa/hub/official/Llama-2-7b-chat/q4_0.gguf, run_type: NLP
Error running ggml inference: Failed to load shared library '/Users/jgi/Development/nexaai/.venv/lib/python3.11/site-packages/nexa/gguf/lib/libllama.dylib': dlopen(/Users/jgi/Development/nexaai/.venv/lib/python3.11/site-packages/nexa/gguf/lib/libllama.dylib, 0x0006): tried: '/Users/jgi/Development/nexaai/.venv/lib/python3.11/site-packages/nexa/gguf/lib/libllama.dylib' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e)))
Please refer to our docs to install nexaai package: https://docs.nexaai.com/getting-started/installation

Why but is an incompatible architecture (have (x86_64), need (arm64e))) as I installed the version for M1 arm ?

The similar error is for other model nexa run sd1-4:

Downloading stable-diffusion-v1-4/q4_0.gguf...
q4_0.gguf: 100%|████████████████████████████████████████████████████████████████████| 1.46G/1.46G [00:15<00:00, 101MB/s]
Successfully downloaded stable-diffusion-v1-4/q4_0.gguf to /Users/jgi/.cache/nexa/hub/official/stable-diffusion-v1-4/q4_0.gguf
Successfully pulled model stable-diffusion-v1-4:q4_0 to /Users/jgi/.cache/nexa/hub/official/stable-diffusion-v1-4/q4_0.gguf, run_type: Computer Vision
Error running ggml inference: Failed to load shared library '/Users/jgi/Development/nexaai/.venv/lib/python3.11/site-packages/nexa/gguf/lib/libllama.dylib': dlopen(/Users/jgi/Development/nexaai/.venv/lib/python3.11/site-packages/nexa/gguf/lib/libllama.dylib, 0x0006): tried: '/Users/jgi/Development/nexaai/.venv/lib/python3.11/site-packages/nexa/gguf/lib/libllama.dylib' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e)))
Please refer to our docs to install nexaai package: https://docs.nexaai.com/getting-started/installation

Steps to Reproduce

$ python3.11 -m venv .venv
$ source .venv/bin/activate
$ CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir
$ nexa run llama2

OS

macOS Monterey

Python Version

3.11

Nexa SDK Version

0.0.9.2

GPU (if using one)

Apple M1 Pro
@jgilewski-kone jgilewski-kone added the 🐞 bug Something isn't working label Nov 20, 2024
@Davidqian123
Copy link
Collaborator

Hi, this problem is weird actually. You can try:

  1. pip uninstall and reinstall the latest version of nexa-sdk
  2. local build https://github.com/NexaAI/nexa-sdk?tab=readme-ov-file#install-option-2-python-package

@jagin
Copy link

jagin commented Nov 25, 2024

@Davidqian123 it doesn't help :(

@Davidqian123 Davidqian123 self-assigned this Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Davidqian123 Davidqian123

Labels
🐞 bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jagin @Davidqian123 @jgilewski-kone