Whisper updates to allow HF models

[awni]

For example:

python convert.py --torch-name-or-path distil-whisper/distil-large-v2

And then use the model in the regular way.

Closes #865
Closes #927

[OKHand-Zy]

I tried to use that commit to transform the model, but the script only run downloads the model to convert.
I used the same model (downloaded from Hugging Face) in localhost , but it did not run.
Upon examining the source code, I determined that it is exclusively configured to search for models within the '~/.cache/whisper' directory. Thank you !
I'll try using this code to fix it and convert the local model.

[OKHand-Zy]

I tried to use that commit to transform the model, but the script only run downloads the model to convert. I used the same model (downloaded from Hugging Face) in localhost , but it did not run. Upon examining the source code, I determined that it is exclusively configured to search for models within the '~/.cache/whisper' directory. Thank you ! I'll try using this code to fix it and convert the local model.

I've adjusted the source code to enable the use of local models. Hopefully, this will be helpful.

def load_torch_weights_and_config(
    name_or_path: str,
    download_root: str = None,
):
    
    if download_root is None:
        download_root = os.path.join(os.path.expanduser("~"), ".cache/whisper")
    # todo: accept alignment_heads of local Pytorch checkpoint
    alignment_heads = None
    if os.path.isdir(name_or_path) :
        # check folder, config.json, pytorch_model.bin
        config_file = Path(name_or_path+"/config.json").is_file()
        pytorch_file = Path(name_or_path+"/pytorch_model.bin").is_file()
        if not config_file and not pytorch_file :
            raise RuntimeError(
                f"Local Model Path：{name_or_path} is not found config.json and pytorch_model.bin files."
            )
        elif not config_file:
            raise RuntimeError(
                f"Local Model Path：{name_or_path} is not found config.json file."
            )
        elif not pytorch_file:
            raise RuntimeError(
                f"Local Model Path：{name_or_path} is not found pytorch_model.bin file."
            )
    elif name_or_path in _MODELS:
        # Check input model in _MODELS
        alignment_heads = _ALIGNMENT_HEADS[name_or_path]
        name_or_path = _download(_MODELS[name_or_path], download_root)
    elif not Path(name_or_path).is_file():
        # Try downloading from HF
        from huggingface_hub import snapshot_download
        name_or_path = snapshot_download(
            repo_id=name_or_path,
            allow_patterns=["*.json", "pytorch_model.bin", "*.txt"],
        )
    else:
        raise RuntimeError(
            f"Model {name_or_path} is not found in {available_models()},"
            "on Hugging Face or as a local path."
        )

    if name_or_path.endswith(".pt"):
        checkpoint = torch.load(name_or_path, map_location="cpu")
        weights, config = checkpoint["model_state_dict"], checkpoint["dims"]
    else:
        name_or_path = Path(name_or_path)
        weights = torch.load(
            name_or_path / "pytorch_model.bin",
            map_location="cpu",
        )
        with open(name_or_path / "config.json", "r") as fp:
            config = json.load(fp)
        weights, config = hf_to_pt(weights, config)

    return weights, config, alignment_heads

[angeloskath]

🙏