Sunday, May 31
Shadow

Build your First Voice Bot with Rasa

Voice… Voice… Voice… 🔊

As we all are using the technology for automating the resources, we are also advancing the way it will do the automation. Like till now we were using the chatbots for automation but as we are advancing the way of implementation, our chatbots are also getting converted to the voice bots. I am back with this advancing technique for you to apply it in real time.

In this post you will learn how to build your first voice bot by integrating it with Rasa chatbot. The integration of rasa chatbot for it conversion to a voice bot have a journey that starts from voice and ends with voice that makes it user friendly and user interactive. Steps that are included here for voice bot are:

1. Speech to text conversion

2. Externally sending an input to rasa chatbot

3. Text to speech conversion

Steps mentioned above are the most important one that will help you make an effective and user interactive bot. In the first step, the user will give an input message in the form of speech that is supposed to be to be sent to the rasa chatbot so that it can give response as per the phrase said. But rasa chatbot only accepts the textual input so we need to convert speech to text and then to further send it to rasa chatbot.

In the second step, the converted text is passed to the rasa chatbot so that your trained rasa chatbot will respond properly as per the stories created. But here the input is not send directly by simply running rasa x or rasa shell instead we need to setup few things and we will use a seperate command that will run rasa server in backend and using this server your chabtbot will be able to respond according and you will get the output as the textual data.

Now, when you will get this textual output from rasa chatbot you need to now convert it back to the speech as the user has provide the input in the form of speech so for this step to make it more interactive and attractive we will use this text and convert it to speech.

If you are new here and python programming is new for you then check this link and get started with python and start your journey from Zero to Hero.

Now let’s start one by one and understand how to do it.

Step 1: Speech to Text

Speech to text conversion is the process of converting spoken words into written texts. This process is also often called speech recognition. Although these terms are almost synonymous, Speech recognition is sometimes used to describe the wider process of extracting meaning from speech, i.e. speech understanding. The term voice recognition should be avoided as it is often associated to the process of identifying a person from their voice, i.e. speaker recognition.

In this step, we will ask the user to say a message and that message will be recording using the microphone connected to your system and the this recording audio will be send to the object created using SpeechRecognition module that we will use here for speech to text conversion. The python module SpeechRecognition and PyAudio are used here where SpeechRecognition is created by google, which makes it easier for speech to text conversion.First install the required module for the conversion by typing the following command in the terminal and installing it in the virtual environment,

$ pip install SpeechRecognition PyAudio

Here is the program for this conversion:

import speech_recognition as sr     # import the library
r = sr.Recognizer()                 # initialize recognizer
with sr.Microphone() as source:     # mention source it will be  either Microphone or audio files.
    print("Speak Anything :")
    audio = r.listen(source)        # listen to the source
    try:
        text = r.recognize_google(audio)    # use recognizer to convert  our audio into text part.
        print("You said : {}".format(text))
    except:
        print("Sorry could not recognize your voice")    # In case of  voice not recognized  clearly

To understand it with practical implementation check the video,

Step 2: Externally sending an input to rasa chatbot

In this step, we will use our trained rasa chatbot that we have already built in the previous sessions and we will setup few easy things that are required for calling the rasa chatbot externally and to send an input message t it that we have received in step 1 and futher the chatbot will reply you accordingly. Here we need to make changes in the credentials file, endpoints file and to call it seperately to call rasa chatbot in backend.

If this is your first post that are reading on my website and if you have skipped how to build your own rasa chatbot the do checout this playlist and get the understanding of Rasa chatbot to become and expert in this field,

In credentials file, add these lines and uncomment it if those are commented:

rest:
## you don't need to provide anything here

rasa:
  url: "http://localhost:5002/api"
In endpoints file, 
action_endpoint:
  url: "http://localhost:5055/webhook"

When these files has been setup then run the following command in your project directory in terminal,

$ rasa run -m models --endpoints endpoints.yml --port 5002 --credentials credentials.yml

This will start your rasa chatbot in backend now add the following lines of code to send the received text (from step 1) to rasa chatbot externally and to get the corresponding out for it.

import requests
sender = input("What is your name?\n")
bot_message = ""
while bot_message != "Bye":
    message = input("What's your message?\n")
    print("Sending message now...")
    r = requests.post('http://localhost:5002/webhooks/rest/webhook', json={"sender": sender, "message": message})
    print("Bot says, ",end=' ')
    for i in r.json():
        bot_message = i['text']
        print(f"{i['text']}")

after the execution of this code you will get the output from rasa chatbot(in textual form) as per your trained model. Now make it more interactive and attractive by converting the output to speech again. To understand it with practical implementation check the video

Step 3: Text to speech conversion
Text to speech, abbreviated as TTS, is a form of speech synthesis that converts text into spoken voice output. Text to speech systems were first developed to aid the visually impaired by offering a computer-generated spoken voice that would “read” text to the user
In this step, we will learn how to convert the text to speech using the most useful and easiest way of conversion using gTTS module that is google Text to Speech which is designed by google for the needy one like the blind person or those who can’t view properly. For this we will use the gTTs class that is use to convert the text to speech and we will further save it to any of the audio format and then we will play it with the inbuilt module named subprocess that will help play and exit the audio using any of the installed media player.
Firstly we will install the gTTs module by simple typing the following command in the terminal and to install it in the virtual environment.


$ pip install gtts

and also install the media player for playing the audio after conversion,

$ sudo apt-get install mpg321

When everything is installed as per the requirements the add the following code to make it happen,

import subprocess
from gtts import gTTS
# The text that you want to convert to audio
mytext = 'Welcome to Innovate Yourself!'
# Language in which you want to convert
language = 'en'

myobj = gTTS(text=mytext, lang=language)
# Saving the converted audio in a mp3 file named
# welcome
myobj.save("welcome.mp3")
# Playing the converted file
subprocess.call(['mpg321', "welcome.mp3", '--play-and-exit'])

When everything is working fine we need to combine the program of all three steps and to make an effective voice bot that can handle automation. To understand it with practical implementation check the video

To get the full code of the working of this chatbot then check this link: https://github.com/ashus3868/RasaCustomerService.git
Now to see your voice bot in action run the Voice_bot.py file and also run the action server using

$ rasa run actions

Note: Before testing your bot make suure you have an active internet connection on your system. Now, You can see your Voice bot in action. To get the full code understanding and to learn how to merge the code to see the final working and it’s practical implementation check this video, https://youtu.be/fyidAFHz05Y

For any doubts or queries related to the topic, feel free to leave a comment below in the comment section. Also do comment your valuable feedback in the feedback so as to appreciate or improve ourselves. For any further information or queries you can also WhatsApp me at +91 8209829808. Till then stay tuned and happy learning ? 🙂

2 Comments

  • Bijal

    Hi @Innovate Yourself
    I stuck on error part like ->
    Hi,
    I’m getting error over here. could you please suggest me anything if you know.

    Speak Anything :
    You said : hello hello hello
    Sending message now…
    Bot says 1, inside loop 2
    Hello! I am calling from Innovate Yourself. How may I assist you?
    saved2
    Speak Anything :
    You said : I am facing network issue
    Sending message now…
    Bot says 1, Speak Anything :
    You said : I am facing network issue
    Sending message now…
    Bot says 1, inside loop 2
    Thanks for confirming, tell me what issues you are facing?
    Traceback (most recent call last):
    File “C:/Users/DELL/PycharmProjects/rasa_innovate/RasaCustomerService-master/Voice_bot.py”, line 56, in
    myobj2.save(“welcome.mp3”)
    File “C:\Users\DELL\Anaconda3\envs\py36\lib\site-packages\gtts\tts.py”, line 294, in save
    with open(str(savefile), ‘wb’) as f:
    PermissionError: [Errno 13] Permission denied: ‘welcome.mp3’
    Process finished with exit code 1
    And
    Line 56 is in Voioice_bot.py -> myobj2.save(“welcome.mp3”)
    And
    Line 294 is in tts.py -> with open(str(savefile), ‘wb’) as f:
    It means file is not saved and so it not loaded to generate the new mp3 that is where I’m getting permission error.

Leave a Reply

Your email address will not be published. Required fields are marked *