Emotions are a new type of big data

Artificial intelligence is rapidly maturing, with new applications appearing from artwork-generators to deepfake detectors. But can emotions be recognised with an AI tool? A Luxembourg company thinks so.

It’s the World Cup. Regulation play has ended and the teams are tied. The coach has mere moments to choose penalty shooters from among the 11 players and additional substitutes… a decision that, since 120 minutes of game time haven’t done it, will determine the outcome of the game.

But this is several years in the future. The coach consults some information on a tablet, namely the on-the-fly results of an analysis of each player’s present emotional state–done by artificial intelligence (AI). After reviewing the data, the coach makes the call.

This is just an example of the type of future that MoodMe CEO Chandra De Keyser foresees. MoodMe, founded in Luxembourg ten years ago, is a deep-tech company that started out with an augmented reality (AR) product to alter faces. Having won clients such as Qatar Airways, FIFA, Gucci and others, it has expanded into artificial intelligence and is seeking to monetise emotions.

“AI is the massive tidal wave, I believe,” says De Keyser in an interview, referring to technology trends. “AR is fun. It’s cool. But it’s not a massive tidal wave.”

Reading emotions

“The emotional state of the athlete is so critical to his or her performance,” says De Keyser, stressing the importance of also respecting the individual’s privacy.

This particular use-case–calculating athletes’ emotional states–would also involve analysing tone of voice and body language, the CEO adds.

The possibilities for such an AI tool are certainly not limited to the field of sportstech. For instance, MoodMe has already provided emotion analytic tools to researchers at Georgia State University who are looking at how players react to various elements of video games (in a bid to design better games). De Keyser also mentions three other potential targets: shoppers, employees and students. “Emotions are a new type of big data,” he sums up.

Also in the race to monetise emotions is video-chat service Zoom, which launched its own “Zoom IQ for Sales” product earlier this year. Aimed at salespeople, the AI tool gathers and interprets data from sales calls and measures metrics like how customers react to questions and the conversation speed. Israeli company Emotion Logic (part of Nemesysco), meanwhile, is taking another route by working on detecting human emotions from voice. Its website suggests use-cases in HR, call centres, medicine and bots.

Unlike these companies, MoodMe currently has a B2B2C model in which it offers software developer kits to companies that want to embed AR or AI technology in their apps, the selling point being that the product is stored locally and not in the cloud (which promotes privacy and control over the data). However, De Keyser says that if the company does decide to pursue an emotion-detecting tool for use on athletes they would probably want to build the end-product too, i.e. become B2C. “We’re at a crossroads right now,” he says.

Emotions versus gestures

Some in the academic community have called into question the idea of reading emotions with technology. In an interview with Tech Monitor from earlier this year, Rosalind Picard–founder of MIT’s Affective Computing Research Group–stressed the difference between using AI to recognise facial gestures and using those gestures to determine an inner state of mind. What’s missing from the latter, according to Picard, is an understanding of context. The article does suggest that emotion recognition technology could be useful in limited, controlled contexts, where cultural and personal biases are taken into account.

Earlier, in 2019, a study found that there was no scientific evidence supporting the claim that an emotional state could be determined based on external indicators.

“It is true,” admits De Keyser, “that there is no mathematical certainty that, given some facial expression’s accurate detection, we can be 100% sure of the inner emotional state.”

Still, the CEO claims that MoodMe’s technology can get pretty close, by providing objective data and plenty of it. The tool analyses faces every second or even every tenth-of-a-second. “Over long periods, the odds that the facial expressions reflect the true state of mind of a participant in a video call… are high.”

He also agrees that deciphering the results is important, stressing: “We provide a new layer of big data that has to be interpreted carefully.”

New world

Regardless of this debate, emotion detection seems to have caught the interest of investors. Besides MoodMe and the aforementioned examples of Zoom and Emotion Logic, other players include Hume AI (founded by a scientist formerly with Google) and the startup Emotiva, which raised half a million euros last year for its real-time emotion analysis product.

In concrete terms, MoodMe already has 20 customers in AI (on top of 130 in AR).

De Keyser has also positioned himself and MoodMe in opposition to huge tech companies, emphasising the importance of privacy and keeping data off the cloud. “Should we give up on AI because it has been done the wrong way by Big Tech? Certainly not! AI is the only way to process huge amounts of data. Human emotions are important in many scenarios.”