What Works in Phishing Awareness Training? Evidence on Warnings, Feedback, and Reporting

Short phishing warnings reduced clicks in a 15-month field study

Lain, Kostiainen and Čapkun studied simulated phishing in a company over 15 months, from July 2019 to October 2020. The study included 14,733 employees and 117,864 simulated phishing emails.

Warnings shown with suspicious emails were associated with fewer clicks. The researchers recorded 3,964 clicks in the no-warning condition, compared with 1,427 with a short warning and 1,289 with a detailed warning. These are raw click counts, not click-through percentages, so they should not be read as a direct comparison of each employee's risk. The study's statistical analysis found warnings effective, but a detailed warning was not significantly more effective than a short one.

For security teams, the practical takeaway is to make a warning useful at a glance. It should tell the employee what to check and where to verify a request safely. The study does not establish the ideal wording for every organization, or show that an email warning works equally well for a phone call, text or chat message.

A training page shown after a failed simulation did not improve resilience

The same study tested voluntary embedded training: after an employee failed a simulated phish, they could be shown a training page. The researchers did not find that this intervention improved later phishing resilience; their results raised concern that this specific format could make employees more susceptible.

Among respondents who remembered the page, 43% selected the statement that seeing it made them feel safe. That finding is about one intervention, a voluntary page shown after a simulated failure. It is not evidence that all security awareness training is ineffective, or that training adapted to a person's behavior has the same result.

The distinction matters. A lesson shown after the click may not be enough on its own. Teams should evaluate whether the message is understood, whether it leads to a safer next action, and whether behavior changes in a later test.

Immediate phishing feedback lowered failures on a second test

Bender, Horn, Loewenstein and Roberts studied just-in-time feedback in a large U.S. health solutions organization. They randomly selected 11,802 employees, and the study's participant flow included 10,844 people. After an initial simulated phishing email, employees received feedback about their response. A second simulated email was sent two weeks later.

Among employees who fell for the first test, 40% of those who received feedback fell for the second one, compared with 50% in the no-feedback group. Among employees who ignored the first email, the second-test failure rates were 20% with feedback and 22% without it.

For the employees who ignored the first email, the intervention also included a follow-up message one week later that encouraged reporting. When the second test arrived, 10% of that group reported it, compared with 7% in the control group.

These are results from simulated emails in one organization. They support testing timely feedback and positive reinforcement, but they do not establish the same effect in every workforce or prove a reduction in real incidents.

Reporting rate and time to report show what employees did

In the ETH Zurich study, about 10% of reports of simulated phishing emails arrived within five minutes, and 30% to 40% arrived within 30 minutes. The researchers also found that positive feedback encouraged later reporting.

For a security team, the reporting rate and time to report can help show whether employees use the reporting route and how quickly a suspicious message reaches the team. They are operational measures, not proof by themselves that an organization is protected. Read them alongside delivery, clicks, investigation and response, and compare like-for-like simulations over time.

Completion rates still describe whether people finished a course. They do not tell a team whether employees recognized a suspicious request, reported it, or followed a verification process when a decision had to be made.

What these studies mean for a human risk program

The evidence supports a focused approach to phishing awareness training:

• Use concise, actionable warnings where the system detects a suspicious email. Lain et al. found no significant advantage for a detailed warning over a short one.
• Give feedback close to the simulated decision. In Bender et al., just-in-time feedback was followed by fewer failures on a second simulated email among people who initially clicked.
• Make reporting easy and reinforce it. Bender et al. found a higher reporting rate after a follow-up message encouraged reporting. Lain et al. also found that positive feedback encouraged later reports.
• Measure behavior as well as course completion. Track reporting rate, time to report and repeat behavior in comparable tests, while keeping each metric's limits clear.
• Test the channels employees actually use. Both intervention studies discussed here tested email. Their results do not establish how the same techniques perform in voice, text or collaboration tools.
• Treat simulations as evidence about simulations. Confirm the effect in your own environment and do not describe a better test result as proof that real attacks have been stopped.

At Zepo, we see an operational implication: connect real threat signals to relevant simulations, feedback, reporting and measurement over time. Our platform links detection with adaptive simulations and training. For a related discussion of why single-channel exercises leave part of the risk untested, read our analysis of multi-channel phishing simulations.

FAQ

Does phishing awareness training reduce phishing clicks?It depends on the intervention. In a 15-month field study, warnings shown with suspicious simulated emails were associated with fewer clicks. A voluntary training page shown after a simulated failure did not improve later resilience in that study. These findings do not establish that every training format has the same effect.

What is just-in-time phishing feedback?Just-in-time phishing feedback is an explanation shown close to an employee's response to a simulated phishing email. In a study of 10,844 participants, employees who fell for a first simulated email and received feedback failed less often on a second test two weeks later. The comparison group received no feedback.

Should security teams measure phishing reporting?Yes. Reporting rate and time to report show whether employees use the organization's reporting route and how quickly suspicious messages reach security staff. They complement click and course-completion measures, but do not prove by themselves that real attacks were prevented.

Do these studies prove that training stops real phishing attacks?No. Both studies tested simulated phishing emails. They provide evidence about behavior in those tests, not direct proof that the same interventions prevent real-world breaches or work equally well across email, voice, text and chat.