Artificial intelligence is significantly worse at detecting human deception than people are, showing extreme bias toward assuming lies and achieving only 19.5 per cent accuracy for truths compared to 85.8 per cent for lies in interrogation settings, according to research from Michigan State University.
The study, published in the Journal of Communication, conducted 12 experiments with over 19,000 AI participants to examine how well AI personas were able to detect deception and truth from human subjects. The findings suggest that AI’s results do not match human results or accuracy and that significant progress is needed before generative AI can be used for deception detection.
“This research aims to understand how well AI can aid in deception detection and simulate human data in social scientific research, as well as caution professionals when using large language models for lie detection,” said David Markowitz, associate professor of communication in the MSU College of Communication Arts and Sciences and lead author of the study.
To evaluate AI in comparison to human deception detection, the researchers pulled from Truth-Default Theory, which suggests that people are mostly honest most of the time and are inclined to believe that others are telling the truth. This theory helped the researchers compare how AI acts to how people act in the same kinds of situations.
A natural truth bias
“Humans have a natural truth bias — we generally assume others are being honest, regardless of whether they actually are,” Markowitz said. “This tendency is thought to be evolutionarily useful, since constantly doubting everyone would take much effort, make everyday life difficult, and be a strain on relationships.”
The researchers used the Viewpoints AI research platform to assign audiovisual or audio-only media of humans for AI to judge. The AI judges were asked to determine if the human subject was lying or telling the truth and provide a rationale. Different variables were evaluated, such as media type, contextual background, lie-truth base-rates and the persona of the AI to see how AI’s detection accuracy was impacted.
One of the studies found that AI was lie-biased, as AI was much more accurate for lies compared to truths in short interrogation settings. However, in a non-interrogation setting, such as when evaluating statements about friends, AI displayed a truth-bias, aligning more accurately to human performance. Generally, the results found that AI is more lie-biased and much less accurate than humans.
“It’s easy to see why people might want to use AI to spot lies — it seems like a high-tech, potentially fair, and possibly unbiased solution,” said Markowitz. “But our research shows that we’re not there yet. Both researchers and professionals need to make major improvements before AI can truly handle deception detection.”
