Molmo-7B-D 4bit suddenly stopped working

[mkagenius]

It was working yesterday (25/12/24) and now without any underlying change (I think) it starts complaining about missing params.

ValueError: Missing parameters: vision_tower.image_vit.patch_embedding.biases vision_tower.image_vit.patch_embedding.scales.

Full command -
(even without image, it gives same error)

(myenv) (miniconda3) ~/clickclickclick$ python3.12 -m mlx_vlm.generate --model mlx-community/Molmo-7B-D-0924-4bit --max-tokens 100 --temp 0.0 --image http://images.cocodataset.org/val2017/000000039769.jpg
Fetching 15 files: 100%|███████████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 246723.76it/s]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/manish/clickclickclick/myenv/lib/python3.12/site-packages/mlx_vlm/generate.py", line 111, in <module>
    main()
  File "/Users/manish/clickclickclick/myenv/lib/python3.12/site-packages/mlx_vlm/generate.py", line 80, in main
    model, processor, image_processor, config = get_model_and_processors(
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/manish/clickclickclick/myenv/lib/python3.12/site-packages/mlx_vlm/generate.py", line 68, in get_model_and_processors
    model, processor = load(
                       ^^^^^
  File "/Users/manish/clickclickclick/myenv/lib/python3.12/site-packages/mlx_vlm/utils.py", line 292, in load
    model = load_model(model_path, lazy)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/manish/clickclickclick/myenv/lib/python3.12/site-packages/mlx_vlm/utils.py", line 257, in load_model
    model.load_weights(list(weights.items()))
  File "/Users/manish/clickclickclick/myenv/lib/python3.12/site-packages/mlx/nn/layers/base.py", line 181, in load_weights
    raise ValueError(f"Missing parameters: {missing}.")
ValueError: Missing parameters: vision_tower.image_vit.patch_embedding.biases vision_tower.image_vit.patch_embedding.scales.
(myenv) (miniconda3) ~/clickclickclick$ 
(myenv) (miniconda3) ~/clickclickclick$ 
(myenv) (miniconda3) ~/clickclickclick$ pip3.12 freeze | grep torch
torch==2.5.1
torchvision==0.20.1
(myenv) (miniconda3) ~/clickclickclick$ pip3.12 freeze | grep mlx
mlx==0.21.1
mlx-vlm==0.1.6
(myenv) (miniconda3) ~/clickclickclick$ 

Its a little weird, since neither mlx, nor mlx-vlm has changed yesterday (nor the weights from mlx-community on huggingface)

[mkagenius]

temp.txt
All the keys attached above. Those two keys indeed seem missing but maybe present with some other name, as some sanitization(transformation) was happening to couple of other keys too.

(BTW, same issue with python3.13)

[mkagenius]

Oh, there was this change in the hugging-face 3 hours ago - https://huggingface.co/mlx-community/Molmo-7B-D-0924-4bit/commit/826ce5935256778523bfee774845f8387a2d6e3e

And exactly those keys are missing.

[Blaizzy]

My bad!

I'm refactoring utile and because of it I had change the model configs.

I will make a new release fixing it in a few.

[Blaizzy]

Bear with me :)

[Blaizzy]

You can use this branch for now
https://github.com/Blaizzy/mlx-vlm/tree/pc/refactor-utils-1

[mkagenius]

I tried the branch but it has the same issue of missing keys.

No issues, I will wait for the refactor to finish.

[mkagenius]

Oh, there was this change in the hugging-face 3 hours ago - https://huggingface.co/mlx-community/Molmo-7B-D-0924-4bit/commit/826ce5935256778523bfee774845f8387a2d6e3e

And exactly those keys are missing.

Seems the keys were removed a month ago, not 3 hours ago. 3 hours ago there was a minor change not related to keys:

So, I am confused now what caused the issue then. I have tried 0.1.4 and 0.1.6 (and the refactor utils branch)

[Blaizzy]

It's fixed ✅

Please use the branch in this PR #161

[Blaizzy]

What caused the issue is a change in the config file that is necessary for the upcoming changes.

[mkagenius]

Works now. The released 0.1.6 itself worked (I havent tested the branch).

[Blaizzy]

Thanks!

I found a better solution that is backwards compatible