Voice is doing to human-machine interface what touch did years ago. The primary driver behind voice winning over touch is digital assistants. These have become parts of our digital lives. Amazon Alexa, Google Assistant and Apple Siri help you access information via your phone, smart speakers, laptops and other devices.
I spend a few minutes talking to Alexa everyday. My interaction ranges from requests to play music, have news updates read out to me and tracking sports. Alexa always answers those questions with poise and when she doesn’t know something, her reply “Sorry, I didn’t understand the question” strikes as one from a robot butler. Is Alexa or Google Assistant useful? Well the answer is yes. With home automation, you can walk into your house and say Alexa, turn on lights or you can make her lock the door. There is endless scenario where they can be useful but one area, where they need work is understanding human emotion.
Every time you talk to Alexa or Google Assistant or Bixby or Siri or Cortana, they understand the context but not the semantics of it. For instance, whether you softly ask or yell at these digital assistants, they will only know that you have requested music. They are not capable of understanding the emotion behind that sound. If these digital assistants hope to replace our current set of devices powered by touch mainly then they need to understand emotion.
Emotion, as defined by Wikipedia, is “any conscious experience characterized by intense mental activity and a certain degree of pleasure or displeasure.” One key reason why smart assistants cannot understand the mental activity or tone of your request is because they lack the conscious experience. But it is set to change very soon. “AI – whether consumer-centric or business-centric – need to emotionally aware in the next five years,” said Ranjan Kumar, Founder & CEO, Entropik Tech, a company working on Emotion AI.
Amazon and Google are both working on adding an emotion layer to their digital assistants. At I/O 2018 early this year, Google showed a duplex conversation carried out by its digital assistant where the machine interface was able to produce human emotions like “mmhmmm” in the conversation. This is the first step towards making these assistants as human-like as possible. The lack of emotional quotient with digital assistants predominently used right now has also led to rise of smaller companies like Entropik Tech, who are essentially filling the gap.
How is EmotionAI different from traditional AI?
“Any sort of Artificial Intelligence is meant to provide intelligent transaction between human and machine,” explains Ranjan. “What it does not have today is emotional aspect of transaction between human and machine. For instance, the AI today cannot understand whether you are saying something happily or angrily. Can AI be adaptive to those human emotions, that’s what EmotionAI is all about,” he adds.
A very simplistic example of this can be seen in a conversation between a user and a digital assistant. An user might be asking something angrily to Alexa, Amazon’s digital assistant but the reality is that Alexa is not able to understand that context itself. Ranjan says Alexa or Google Assistant are not capable of understanding emotional context in their current form and his company aims to add that layer of emotional activity and are calling it as EmotionAI.
How can Emotion be added to current form of AI?
Emotion is an integral element to humans and it can be detected via facial expression, voice synthesis and neural response. Ranjan tells BGR India that his company uses these traditional elements via brain wave mapping, facial recognition, eye tracking and voice-based emotional tracking to send the emotional context to AI interface.
Before deploying the test data on AIs used by consumers, Entropik Tech is using its EmotionAI in other areas such as retail and product development. “Our voice-based emotional tracking is used to analyse calls and understand whether the customer was happy or disappointed with the experience,” Ranjan explains. He adds that the emotional tracking feature is also used to understand consumer response for Accenture in the retail space, Vodafone Spain in the telecom space and Xiaomi in developing truly consumer-centric devices.
How facial recognition and brain wave mapping are used to build emotional layer?
For brain wave mapping, Entropik Tech uses a special hardware, which works similar to wearing a headphone. The hardware helps track how neurons of the person wearing it fire at second level. The corresponding data can be mapped to attention level of the user and can also judge the mental effort applied towards a context. “The hardware provides us with raw data and we have built the software part to convert these raw data into meaningful information,” Ranjan confirms.
The facial recognition system developed by the team at Entropik can track 58 actionable facial codes from a face. This translated into data ranging from the way you grin, the way you roll your eyes to way you smirk. All of these data can be fed to the machine learning algorithm and deep neural networks those power the modern AI system to become emotionally aware.
What about privacy?
There is no denying the fact that AI cannot improve if we don’t feed it with data. The best example is the Portrait Mode effect on Google’s newest flagship smartphones, the Pixel 3 and Pixel 3 XL. Instead of using dual camera sensors like its rivals, Google is using a single sensor and filling the gap by understanding the depth of an image and deep neural network.
When asked about his emotionAI affecting privacy, Ranjan says there is “technology to ensure that privacy is not affected.” He says data collection and machine learning should happen on the device and not on a cloud service and companies should develop less intrusive ways to track emotion on devices like smartphones. He adds that brain wave mapping will be limited to certain test environments since it requires additional hardware.
“With EmotionAI, the interaction between AI assistant and a human could be as good as one between two humans, contextually aware as well as emotionally aware,” says Ranjan Kumar. The big question for the industry would be lay down ethics for such implementation. The progress of AI could be good for the human society but it should not come at a cost of data privacy and malpractice.
