Speakers who use purposeful hand gestures to illustrate their spoken content appear more competent and persuasive to audiences, with illustrative movements making speakers seem more knowledgeable and improving audience understanding, according to research analyzing thousands of presentations using artificial intelligence.
Researchers at the UBC Sauder School of Business analysed 2,184 TED Talks using AI and automated video analysis, isolating more than 200,000 hand gestures into 10-second clips and comparing them against audience engagement metrics, such as likes on social media, while controlling for factors like gender, occupation, language, and video length.
The team also ran randomised experiments in which participants watched videos of sales pitches where speakers delivered identical scripts but varied their hand movements. Viewers then rated the speakers and the products being pitched.
Researchers categorised gestures into types, including “illustrators,” which visually depict spoken content such as demonstrating the size of a fish whilst describing it, and “highlighters,” such as pointing to an object mentioned in the speech. They also examined random, unrelated movements and the absence of gestures.
Illustrators have impact
Illustrators had the strongest effect, making speakers seem more knowledgeable and improving audience understanding. Highlighters and random gestures, however, showed little to no impact.
“Illustrators can help make the content easier to understand because we’re delivering the same information in two modes: visual and verbal,” explains Dr Mi Zhou, study co-author and UBC Sauder assistant professor. “When people use illustrators, it increases viewers’ perception of the speaker’s competence.”
According to Dr Zhou, audiences interpret illustrative gestures as a sign of mastery. The team used 21 key points on the hands to calculate hand movement in videos, classified gestures. Then it linked them to spoken content using multimodal AI, a type of artificial intelligence that can simultaneously analyse multiple types of data.
“One of the key takeaways for marketers is that you can use the same content, but if you pay more attention to how that content is delivered, it could have a big impact on persuasiveness,” said Dr Zhou.
The findings have implications for marketers, influencers and anyone trying to persuade an audience. The research could also help companies design more lifelike virtual assistants and AI-generated characters by pairing verbal communication with natural hand movements.
The study was co-authored by Dr Giovanni Luca Cascio Rizzo of the University of Southern California and Dr Jonah Berger of the University of Pennsylvania.
