Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to keep model in high alert state for faster inference? #243

Open
SuperMaximus1984 opened this issue Aug 25, 2024 · 1 comment
Open

Comments

@SuperMaximus1984
Copy link

I'm running insanely-fast-whisper in environment where low latency is crucial. Once .wav file is created, it needs to be transcribed immediately. Everytime I run:

D:\InsanelyFastWhisper>insanely-fast-whisper --file-name myoutputfile_17245952653c006.wav --device-id 0
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
🤗 Transcribing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:00
Voila!✨ Your file has been transcribed go check it out over here 👉 output.json

I see some pause in this part: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

And then Transcribing runs pretty fast. I assume the model is being loaded first, so it takes time.
How do I keep it in the high alert state so that transcription would run faster upon feeding of the new .wav file?
Thanks!

@ArmykOliva
Copy link

Don't use the CLI and use the transformers python way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants