Companies using “gotcha” phishing tests that provide instant feedback to employees who click on suspicious links may be undermining their own cybersecurity defences, as new research shows that the widely used training method can trigger defensiveness and limit learning.
Embedded training, where employees who fall for simulated phishing emails receive immediate on-the-spot lessons, is considered a best practice in the cybersecurity anti-phishing industry. However, researchers at the University of South Florida’s Muma College of Business found the approach has two critical shortcomings: only those who were duped receive training, whilst catching employees at the exact moment of failure can lead to adverse reactions.
The study employed three large-scale experiments using a real phishing simulation platform, with thousands of students receiving realistic but simulated phishing emails. Some received immediate feedback after clicking, whilst others received follow-up messages days later. The team tracked how likely participants were to fall for future simulated scams over the following weeks and months.
“Giving feedback only to the people who clicked the ‘fake’ phishing email misses a big opportunity,” said Dezhi Yin, co-author at USF’s Muma College of Business. “We found that employees learn better when everyone — even those who didn’t fall for it — gets a follow-up message explaining the phishing test.”
Recognising scams
Researchers discovered that sharing lessons with the entire group, not just those who were duped, helped people recognise scams more effectively and stay alert for months afterwards. Training does not need to be delivered at the point of failure to be effective, with a time-delayed but more inclusive approach, ultimately building better defence against real attacks.
“Phishing training companies can directly make use of our key insights in designing more effective software tools, and we heard that KnowBe4 is already doing that,” said Matthew Mullarkey, co-author at USF’s Muma College of Business.
The project began with support from KnowBe4, a Clearwater, Florida-based cybersecurity company that donated software licences for more than 12,000 users and provided technical expertise and research funding. The study’s findings could help companies strengthen their cybersecurity defences as phishing scams grow more sophisticated and increasingly use artificial intelligence.
The paper is co-authored by Yin and Mullarkey of USF’s Muma College of Business, Gert-Jan de Vreede of Stevens Institute of Technology, and Moez Limayem of the University of North Florida. The findings were published in MIS Quarterly.
