02b - Speech.ipynb

{"cells":[{"cell_type":"markdown","source":["# Speech\n","\n","Increasingly, we expect to be able to communicate with artificial intelligence (AI) systems by talking to them, often with the expectation of a spoken response.\n","\n","![A robot speaking](./images/speech.jpg)\n","\n","*Speech recognition* (an AI system interpreting spoken language) and *speech synthesis* (an AI system generating a spoken response) are the key components of a speech-enabled AI solution.\n","\n","## Create a Cognitive Services resource\n","\n","To build software that can interpret audible speech and respond verbally, you can use the **Speech** cognitive service, which provides a simple way to transcribe spoken language into text and vice-versa.\n","\n","If you don't already have one, use the following steps to create a **Cognitive Services** resource in your Azure subscription:\n","\n","1. In another browser tab, open the Azure portal at https://portal.azure.com, signing in with your Microsoft account.\n","2. Click the **&#65291;Create a resource** button, search for *Cognitive Services*, and create a **Cognitive Services** resource with the following settings:\n","    - **Name**: *Enter a unique name*.\n","    - **Subscription**: *Your Azure subscription*.\n","    - **Location**: *Any available location*.\n","    - **Pricing tier**: S0\n","    - **Resource group**: *Create a resource group with a unique name*.\n","3. Wait for deployment to complete. Then go to your cognitive services resource, and on the **Overview** page, click the link to manage the keys for the service. You will need the endpoint and keys to connect to your cognitive services resource from client applications.\n","\n","### Get the Key and Endpoint for your Cognitive Services resource\n","\n","To use your cognitive services resource, client applications need its  endpoint and authentication key:\n","\n","1. In the Azure portal, on the **Keys and Endpoint** page for your cognitive service resource, copy the **Key1** for your resource and paste it in the code below, replacing **YOUR_COG_KEY**.\n","2. Copy the **Endpoint** for your resource and and paste it in the code below, replacing **YOUR_COG_ENDPOINT**.\n","3. Copy the **Location** for your resource and and paste it in the code below, replacing **YOUR_COG_REGION**.\n","4. Run the code below by clicking the **Run cell** (&#9655;) button to the left of the cell."],"metadata":{}},{"cell_type":"code","source":["cog_key = 'YOUR_COG_KEY'\n","cog_endpoint = 'YOUR_COG_ENDPOINT'\n","cog_region = 'YOUR_COG_REGION'\n","\n","print('Ready to use cognitive services in {} using key {}'.format(cog_region, cog_key))"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599695240794}}},{"cell_type":"markdown","source":["To use the Speech service in your Cognitive Services resource, you'll need to install the Azure Cognitive Services Speech SDK"],"metadata":{}},{"cell_type":"code","source":["! pip install azure.cognitiveservices.speech"],"outputs":[],"execution_count":null,"metadata":{}},{"cell_type":"markdown","source":["## Speech recognition\n","\n","Suppose you want to build a home automation system that accepts spoken instructions, such as \"turn the light on\" or \"turn the light off\". Your application needs to be able to take the audio-based input (your spoken instruction), and interpret it by transcribing it to text that it can then parse and analyze.\n","\n","Now you're ready to transcribe some speech. The input can be a microphone or an audio file. In this case, you'll use an audio file.\n","\n","Run the cell below to use the speech-to-text capabilities of the Speech service to transcribe the audio."],"metadata":{}},{"cell_type":"code","source":["import os\n","import IPython\n","from azure.cognitiveservices.speech import SpeechConfig, SpeechRecognizer, AudioConfig\n","\n","# Get spoken command from audio file\n","file_name = 'light-on.wav'\n","audio_file = os.path.join('data', 'speech', file_name)\n","\n","# Configure speech recognizer\n","speech_config = SpeechConfig(cog_key, cog_region)\n","audio_config = AudioConfig(filename=audio_file) # Use file instead of default (microphone)\n","speech_recognizer = SpeechRecognizer(speech_config, audio_config)\n","\n","# Use a one-time, synchronous call to transcribe the speech\n","speech = speech_recognizer.recognize_once()\n","\n","# Play audio and show transcribed text\n","IPython.display.display(IPython.display.Audio(audio_file, autoplay=True),\n","                        IPython.display.HTML(speech.text))\n"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599695250434}}},{"cell_type":"markdown","source":["Try changing the **file_name** variable to *light-off.wav*, and run the cell again. The service should be able to transcribe both files correctly to text.\n","\n","## Speech synthesis\n","\n","So now you've seen how the Speech service can be used to transcribe speech into text; but what about the opposite? How can you convert text into speech?\n","\n","Well, let's assume your home automation system has interpreted a command to turn the light on. An appropriate response might be to acknowledge the command verbally (as well as actually performing the commanded task!)"],"metadata":{}},{"cell_type":"code","source":["import os\n","import IPython\n","from azure.cognitiveservices.speech import SpeechConfig, SpeechSynthesizer, AudioConfig\n","\n","# Get text to be spoken\n","response_text = 'Turning the light on.'\n","\n","# Configure speech synthesis\n","speech_config = SpeechConfig(cog_key, cog_region)\n","output_file = os.path.join('data', 'speech', 'response.wav')\n","audio_output = AudioConfig(filename=output_file) # Use a file instead of default (speakers)\n","speech_synthesizer = SpeechSynthesizer(speech_config, audio_output)\n","\n","# Transcribe text into speech\n","result = speech_synthesizer.speak_text(response_text)\n","\n","# Play the output audio file\n","IPython.display.display(IPython.display.Audio(output_file, autoplay=True),\n","                        IPython.display.Image(data=os.path.join(\"data\", \"speech\" , response_text.lower() + 'jpg')))"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599695261170}}},{"cell_type":"markdown","source":["Try changing the **response_text** variable to *Turning the light off.* (including the period at the end) and run the cell again to hear the result.\n","\n","## Learn more\n","\n","You've seen a very simple example of using the Speech cognitive service in this notebook. You can learn more about [speech-to-text](https://docs.microsoft.com/azure/cognitive-services/speech-service/index-speech-to-text) and [text-to-speech](https://docs.microsoft.com/azure/cognitive-services/speech-service/index-text-to-speech) in the Speech service documentation."],"metadata":{}}],"metadata":{"language_info":{"name":"python","version":"3.6.9","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kernelspec":{"name":"python3-azureml","language":"python","display_name":"Python 3.6 - AzureML"},"kernel_info":{"name":"python3-azureml"},"nteract":{"version":"nteract-front-end@1.0.0"}},"nbformat":4,"nbformat_minor":2}