Using  Customer Speech Analysis  With AI To Improve Call Center Performance

What are your customers really feeling? Artificial intelligence can detect frustrations and analyze problems to improve your employee call center performance. 

I was recently reading an article about how to inject humanity into health care performance, particularly in call centers. The AI was much like the one we discussed on Tuesday, that could analyze the emotional state of representatives for performance review, but this one was talking about offering real-time coaching.

Cogito's algorithms parse each nanosecond of a caller's speech over the phone and simultaneously flash recommendations and feedback to the customer service center worker mid-call. As in, "you're speaking too much," "frequent overlaps," "breathe, pause between phrases," or "you're tense."

This is really interesting because the AI is actually listening to an analyzing the unique rhythm of each individual call, and is able to discern whether or not there are interruptions, and how the tone of the representative sounds. But it isn’t just focused on the rep. It does so by focusing on the caller as well.

Not all callers are the same. And AI which can detect the moods and emotions of each caller can judge how a contact should go, can judge what the customer might want, and can understand how to guide each interaction. It’s another way that AI will improve, and not replace, your customer service.

What the Customer Knows

The Harvard Business Review recently did a survey about customer service and the frustrations people had with it. A few of the statistics might stand out.

  1. 56% of people have had to re-explain their problems to customer service
  2. 59% have to use “moderate-to-high” effort to resolve a problem
  3. 35% say that they have “lost their temper” when speaking to customer service, presumably at least in part because of the above numbers. (Source)
  4. 84% of people said expectations had not been exceeded in their last customer service interaction.

That last number is pretty stark, because, if they had a bad interaction in the past, or even a middling one, chance are their expectations were already pretty low. Barely meeting poor expectations is not something of which to be proud.

Most of this confusion and anger comes from mixed signals and a lack of comprehension. Sometimes that is on the part of the CSR, who has a million things to balance. It’s a very difficult job. You have to be fluent in the company, have to know how to solve problems, and have to try to judge what kind of response the person on the other end needs to hear. Even being emotionally disconnected by a few degrees can lead to frustration and anger.

That’s where AI can come in. Artificial intelligence has the capacity to guage a customer’s emotions, and can give employees the advice they need in real-time. A representative trying to figure out the problem might not be able to pick up on subtle cues that signify building frustration.

But imagine, as in the example above, that the AI detects slight tremors in the customer’s voice as the CSR unwittingly cuts them off or steps on the end of their sentences, in an understandable rush to ask questions or move toward a solution. That can turn even a positive experience into a deep negative.

But AI can fix that, by picking up on tiny cues a person might miss. AI can use its base of knowledge from millions and millions of microinteractions to judge how it instantly analyzes a customer is feeling.

AI Understands What The Customer Doesn’t

What’s more, artificial intelligence can help to solve problems that the customer can’t quite articulate. Imagine that you sell lawnmowers, and there is a problem with the recoil motor. It’s just not starting when customers pull it. Now, they don’t know it is called a recoil motor. And if you sell dozens of appliances and machinery, your CSRs might not, off the top of their head, be able to ascertain exactly what a customer is talking about.

You can picture it: the customer struggling to say that “the part with the string!” is broken. After a few questions, the CSR might understand. The customer might have to repeat themselves a few times, growing increasingly angry, revving up like a lawnmower being pulled over and over.

But the AI has heard customers describe it in the exact same way before, because it can monitor and analyze literally every conversation. It knows, instantly, that someone saying “the stringer puller thinger!” means the recoil motor. Heck, it can probably understand what it meant when someone just imitates the recoil motor because someone, on some call, has tried to do the same thing. So the CSR gets a flash notice that says “recoil motor malfunction,” with information on how to handle the call. No repetition, no over-explaining, no frustration.

Is AI perfect? Of course not. There will be mistakes. Someone could sound very agitated but be totally placid, or cool as a cucumber but boiling and about to leave the world’s worst Yelp review. And maybe it isn’t a recoil motor. But it probably is.

The point is that AI has the capacity to learn and to grow, with every call. And that knowledge can be transferred to the highly-trained customer service reps you have in house or with a respected, award-winning, third-party call center partner. With that knowledge, your call center can improve, every day.


RDI-Connect is your full-service call center and customer care partner. We combine cutting-edge technology with superior hiring and training procedures to give your organization the support it needs. Your success is ours. To learn more about how Artificial Intelligence will impact your business, please download our white paper, “Artificial Intelligence, Real Results: How AI Will Improve Your Call Center Experience”. To learn more about how we can partner with you, please connect with us today.

Share this post...

Share your thoughts about this post.

Better Together: How to Align Sales and Service With the Customer Experience


RDI Corporation was founded in 1978 and is headquartered in Blue Ash, Ohio. We provide precise business solutions through a fully integrated outsourcing model and our clients ranged from mid-sized corporations to distinguished Fortune 500 companies.