Live microphone inference

[Hannes1]

Hi, I've been playing around and love the size and performance of your models. Was wondering if there's any examples we're I can use the mic

[snakers4]

Hi,

(i)

In theory the fastest way to get started is to use ipywebrtc + notebooks.
In the past the below snippet worked for me (now it does not for some reason).
But as with may OSS projects and front-end-related stuff, it may not work on your environment for no apparent reason.

!pip install ipywebrtc
from ipywebrtc import CameraStream, AudioRecorder

camera = CameraStream(constraints={'audio': True,
                                   'video': False})
recorder = AudioRecorder(stream=camera)

# turn the recorder on
recorder.recording = True
display(recorder)

# the container and codec depends on your browser and OS
recorder.save('test.ogg')
wav, sr = read_audio('test.ogg', sr=16000)

(ii)
As a workaround some time ago I used a combination of Voila + ipywebrtc + ipyvuetify to create this demo for Russian. This is very cool, but technologically it is a dead-end. The magic is that Voila just supports all of the notebook widgets by default. The widgets natively received and Audio player widget, but no audio recorder yet.

(iii)
A bit later we created this demo, that works on all major mobile and desktop browsers.
We are actually planning to publish an updated cleaned-up version of this demo using tf.js in future.

(iv)
If you are on a laptop / desktop (not in a headless environment), you may try something like this.

[tugstugi]

@Hannes1 Until the Silero guys provide a demo with microphone, you can use this colab notebook with microphone: https://colab.research.google.com/github/tugstugi/dl-colab-notebooks/blob/master/notebooks/SileroSTT.ipynb

@snakers4 Has your mobile demo a KenLM decoding? It is really fast.

[Hannes1]

@tugstugi Tank you I'll try it out (was busy converting deepspeech vad example to work with their model😂)
@snakers4 thanks for the detailed response

[snakers4]

Hello to everyone,

@tugstugi

Has your mobile demo a KenLM decoding? It is really fast.

This demo is running a EE version behind. We choose not to comment on what is powering it.
But yes, it is really fast xD
I have also not decided you whether we will expose an EE demo with languages other that Russian publicly, most likely we will host a tf-js client-side demo of our CE-models to showcase their speed.

It is a bit misleading because the CE-model has hard time producing visually pleasing transcriptions sometimes, but I am afraid of traffic spikes - we have not yet came public with this repo and it is already gaining some traction.

@Hannes1 @tugstugi

Until the Silero guys provide a demo with microphone, you can use this colab notebook with microphone:

Many thanks for the demo.
I took my time rewriting our current demos, cleaning up / polishing some util code and I have integrated a client-side recording demo into our example as well.

[snakers4]

Some further examples with microphone - https://github.com/snakers4/silero-vad/tree/master/examples

[AshutoshDongare]

Check out my repo that uses selaro models and rast to create a conversationalist https://github.com/AshutoshDongare/convo