Today’s landscape

The state of the technology is vastly different now. The wide adoption of AI has changed the way human speech is interpreted by machines in a substantial way, making the task to develop Speech-to-Text systems much easier and performance much better – meaning that transcription precision has improved significantly. Speech-to-Text offers have exploded in number and dozens of companies now provide the service, either directly from the public Cloud or integrated more strictly with speech applications.

However, speech is not the same for all people and applications. The variations are staggering. People speak in different ways depending on what they want, what is being asked of them, where they are in a conversation, and of course in dozens of different languages. Providing a Speech-to-Text service that covers effectively all the variations and parts of a conversation is exceedingly hard. So, inevitably some services are better than other for specific tasks and languages.

Interactive Media’s approach to Speech-to-Text

Since Speech-to-Text is still integral to Interactive Media’s offer, we are constantly monitoring its advances and testing different services on a day-to-day basis. We have developed metrics and standardized test suites to inform the decision of what service to use for the benefit of our customers, depending on the use case which dictates the task at hand, the settings, and the language.

What’s the benefit? We have found that the main general-purpose Speech-to-Text services have some weak points, for instance when the task is to fill in a form with numbers or alphanumeric strings. In this case the field of results is limited, but some services don’t seem to use this to their advantage and retain the same percentage of correct recognition as the general speech. But while a 95% recognition accuracy is usually enough to find out an intent (for instance), when you need to take in a string of 10 digits, you’ll get it wrong roughly 40% of the times.

However, other Speech-to-Text engines are optimized for recognizing digits or allow the user to define tight grammars that can help with the task. Using these engines, you can get an accuracy up to 99%, which over 10 digits results in a 90% probability to get the whole string right.

Similarly, there are more common tasks that need optimization for the Virtual Agent to be effective. Maybe the most challenging one is transcribing an email address. Human agents have a hard time with it, and the percentage of errors is exceedingly high. Again, some Speech-to-Text services do better than others and even a 5% difference makes it worth it to switch to a better performing service in mid-call if the volume of traffic is high enough.

So, we engineered our platform to use several of the best Speech-to-Text services, constantly testing the connected services and adding new ones as they become available. It’s a big task, but (we think) we are being fairly smart about it: we model conversations by defining categories of tasks that Virtual Agents must accomplish, and continuously test each of the services we integrate with using sample atomic interactions belonging to each category. This way, we derive scores for the various services for each task, in several languages.

This would be academic without a way for the Virtual Agent application to tell us what to expect. So, we added this feature to all our services, provided by the PhoneMyBot and OMNIA platforms. The API allows to specify the expected category of utterance coming from the user, based on the question being asked. So for instance, if the system prompts the user to provide a numerical code, the service knows that the next utterance is most likely composed of numbers, and will use the Speech-to-Text engine with the best performance recognizing them.​

The difference in performance is substantial – if even 10% less calls have to be forwarded to human agents, especially when the task is simply collecting data from the customer, the customer experience is better and the ROI for our customers soars, which is the promise of Virtual Agents, delivered.

There are good reasons for companies to reach users proactively and immediately: for instance, to remind them of an appointment and give them the ability to reschedule. Or to confirm an order before it ships. Text messages can be used for that but there’s no guarantee that the answer will be fast – or there will be an answer at all. The main way to reach people quickly and with real-time feedback is a phone call: the phone will ring and if the user answers the unfolding conversation allows to go over the matter completely and with a high degree of certainty. So, this is now done with automatic dialers that are backed up by live agents, which is also expensive and not pleasant for the agents themselves. Too bad that chatbots cannot use the phone.

Or can they?

PhoneMyBot by Interactive Media provides services that allow chatbots to seamlessly operate on voice channels, starting with the telephone. It is a Cloud-based environment with connectivity to the telephone network, APIs to connect with the chatbots, and using multiple speech-to-text and text-to-speech services to “translate” between the voice-based and text-based ends.

PhoneMyBot uses a layer of software adaptors to natively talk with several common conversational AI frameworks. Chatbots based on these frameworks don’t have to do anything to interact with voice users: they see the endpoint as just another website-based client. But of course, this is for incoming calls.

But PhoneMyBot also exposes a standard cloud API that the chatbots can use, and it supports placing calls to telephones. Once the call is established, the chatbot interacts with the user like in any other chat conversation, leaving to PhoneMyBot the task of converting between text and voice. If the call cannot be connected, or it goes to voicemail, the chatbot receives a message from PhoneMyBot and can continue to the next call.

The applications are numerous, all resulting in better customer experience and lower costs for the company:

• Reservations and scheduling
• Order confirmations, delivery alerts
• Reminders or appointment confirmation
• Service renewal
• Upselling
• …

With its outbound service, PhoneMyBot allows to use chatbots in a completely new way, giving voice to their chat and opening new perspectives. To learn more please visit https://www.phonemybot.com or contact us at info@phonemybot.com.

Virtual agents: what are they and why are they important in service?

Increasingly common in corporate daily life, virtual agents are computer applications that use artificial intelligence (AI) and machine learning to optimize service processes. Virtual agents use Natural Language Processing (NLP) and Natural Language Understanding (NLU) as the basis for conducting a conversation with people.

By incorporating virtual agents into workflows, companies have a double gain: while leveraging service team productivity, speeding up problem solving through technology, they also optimize important resources – such as labor, time and money. The result of the equation can be extremely positive and, therefore, very attractive to high performance companies.

NLP, in turn, plays a decisive role in the effectiveness of virtual agents, especially when the user experience is a central theme. “Generally speaking, Natural Language Processing is the ability of a computer system to interact with people using speech, adapting to understand what they say and to respond to them in a natural way,” says Livio Pugliese, CEO of PhoneMyBot, an Interactive Media company.

Technically speaking, NLP is at the intersection of linguistics and information technology, benefiting from the advances of both. According to a Gartner report, it is estimated that by 2021, 15% of all customer service interactions will be fully handled by artificial intelligence mechanisms. In Brazil, the virtual agent market also continues to be heated: in 2019 alone, 60 thousand bots were launched, a number almost 353% higher than the previous year.

Natural language: what are the biggest challenges and main advantages?

The rapid advance of technology, especially of artificial intelligence, has in recent years led to a substantial increase in the quality of comprehension and language generation. As a result, virtual agents also improved and gained more and more space in business departments – from sales to technical support.

In practice, NLP needs written text to function. Therefore, it is necessary that machines accurately transcribe what people say, providing a coherent and precise interpretation – which undoubtedly emerges as one of the main challenges of Natural Language Processing.

“Even more challenging, however, is the mission of assigning a faithful meaning to the transcript, since people use many different phrases to say the same thing and, in some cases, the same word can mean different things in specific contexts”, comments Livio Pugliese.

However, overcoming this obstacle brings a substantial reward: adopting automated service solutions brings many advantages and can represent significant gains in the short, medium and long term.

“When we are talking about voice, NLP can help whenever people need to communicate with machines without using their hands,” explains the CEO of PhoneMyBot. The executive reinforces that these days it is not just questions and answers, but conversations. The biggest advantage lies in the ability to understand the user’s intention to, if possible, provide the services most appropriate to the question or complaint.

“Often, the virtual agent gets only some of the meaning during the first conversation exchange” according to Livio Pugliese. But with AI and machine learning resources, this is no longer a problem: the technology can continue to ask complementary questions to single out the intention until it is completely understood, and the most relevant answer is sent to the customer.

The operational impact of virtual agents is important for the bottom line as well. When bots are in charge of leading the initial interaction with customers, often containing the operation into self-service, professionals in the field can dedicate themselves to more analytical and strategic tasks, which helps the overall company performance.

If you still have doubts about the efficiency of virtual agents in generating savings for corporations, it is worth remembering that a survey by Juniper Research predicts that, by 2022, companies will save 8 billion dollars a year with the application of conversational technologies. In other words: it is worth investing now.

Experience and technology: why is Interactive Media a specialist in NLP?

Interactive Media develops, deploys, and continuously improves conversational virtual customer service agents across multiple channels. With great expertise in artificial intelligence tools and machine learning, the company helps its customers to optimize their interaction flows with users.

With more than 20 years of experience in voice applications, Interactive Media has implemented many successful use cases in organizations of the most diverse sizes and segments. “Based on the history, we know that virtual agents can solve up to 80% of the problems in the call center, freeing human agents from countless telephone contacts”, points out Pugliese.

Interactive Media’s platform uses a carefully tailored approach to build technologies that fosters high-performance understanding, allowing for more accurate interactions – which, in turn, improves both ends of the chain. For the company, it is about maximizing resources and reducing costs; for the user, it means that problem solving is more agile and efficient.

“At Interactive Media, we cover the complete lifecycle of virtual NLP agents and also of all integrations, which ensures that we can provide the most appropriate and intelligent solution to the problem presented by the customer”, concludes the PhoneMyBot CEO .

We want to end with two complementary conclusions. The first is that Natural Language Processing emerges as the most assertive way to enable a more human-like automated service, capable of really understand the user’s intention. The second solidifies Interactive Media’s position as a reference in the area; after all, even before the chatbots boom, the company already offered AI-based conversational services and they have only become smarter and more focused with time.

To improve the service flow in your company you need an effective technological mechanism. Contact us and find out how we can help you implement more complete solutions, in line with the demands of an evolving market.

And so, people send and enjoy messages at an ever-increasing rate. Of course, chatbots are also in the mix, following their audience to the channels that they use. This way, people can get services from chatbots on their favorite messaging app, just like they were messaging with friends.

Chatbots work on text and all messaging applications are based on text. They all support pictures and videos, which are transferred as text-based links that the app follows to retrieve the content or attachments. Chatbots can connect to any messaging platform with an API that allows it, simulating a mobile device or implementing a business endpoint.

All good then? Not completely. A functionality offered by some messaging platforms is to record a voice message instead of typing and send it instead of (or together with) a text message. This is becoming more and more common – people on the move may not want to stop and type, while recording a brief message is fast and easy. It is also more personal: you can say a lot more with your tone of voice than sending text and emojis.  Humans also appreciate to hear their friends voice more than just reading what they write.

But not chatbots. For them, a recorded voice message in a text exchange means the end of the conversation: they are not (in general) equipped for receiving a voice file and transcribing it into text to feed to the conversational AI engine that propels the conversation. The alternative, that can be used in high-value conversations like sales or customer support ones, is to transition the interaction to a human agent who will listen to the voice message and reply back, taking over the exchange with the user. But this is expensive as it requires the organization to staff humans in a sufficient number to pick up failed bot conversations in addition to conducting their normal business.

Even worse would be for human agents to simply listen and transcribe the message to pass it back to the chatbot: this would be an impossibly dull and menial job and likely to lead to massive turnover.

What is needed is a service to transcribe voice recordings and get them back to chatbots accurately and quickly.  

PhoneMyBot from Interactive Media provides such a service. PhoneMyBot is dedicated to expanding the chatbots realm to voice, be it from the telephone network or any other channel. For the telephone channel, PhoneMyBot must transform live voice from a user into text and text from the chatbot into voice. All of this, in several languages and with a selection of the best speech-to-text service for the job. This also enables PhoneMyBot to spot-transcribe recorded messages.

A crucial point is to make it very easy for chatbots to submit a recorded voice message to transcribe. PhoneMyBot exposes a RESTful API for this, supporting numerous encodings and formats for the voice file. Considering that most users are on WhatsApp and so chatbots also use this channel, PhoneMyBot also provides a WhatsApp enabled number for access. Chatbots can send a message to PhoneMyBot with the voice file and receive back the transcription as the response.

With this feature, we of PhoneMyBot believe that we gave a definitive answer to the recorded voice messages dilemma.