This is a Repository so that I may share and back up my jupiter notebooks, All were ran on Runpod, So you will have to change some stuff if you run it locally or with a different service.
Quantize GPTQ is ONLY for LLAMA 2 and LLAMA based models with the CasualLMforLLama architecture.
The merge.ipynb is just a merge for me, and is basically just a copy of this person's work but with some modifications: https://github.com/DocShotgun/LLM-notebooks/ Same with re-shard.ipynb