Your staff versus the deepfake: how AI is changing the human layer

Avatar photo
By Chris Arceo / May 17, 2026 / AI & Automation

You can deploy every control in this series, Zero Trust identity, managed EDR, application control, machine-speed patching, continuous vulnerability management, and still lose everything the moment one staff member approves a payment for a voice they trusted. The human layer has always been the softest target. AI has turned it into the actively hunted front line. Attackers no longer send clumsy emails riddled with typos. They clone your CEO’s voice from a podcast clip and join a video call wearing your CFO’s face. The good news, and there is good news, is that the human layer is also the one you can measurably strengthen.

This is the sixth post in our series on how AI is reshaping each layer of your security stack, following Zero Trust, EDR, application control, patching, and vulnerability management. The full ecosystem overview ties the whole series together.

The attack got an AI upgrade

For years, the advice was to watch for poor grammar, generic greetings, and odd phrasing. That advice is now actively dangerous, because it teaches people to trust messages that look polished. Analysis from KnowBe4 and SlashNext found that 82.6 percent of phishing emails now contain some AI-generated content, which strips out exactly the tells that legacy filters and old-style awareness training relied on. A perfectly written, personalised email referencing your real projects and colleagues is no longer a reassuring sign. It is the new normal for an attack.

From the inbox to the voice and the face

The bigger shift is that social engineering has jumped channels. Voice phishing, or vishing, rose by around 442 percent between 2023 and 2024, and reported deepfake incidents have grown roughly 680 percent year on year. The most infamous case is engineering firm Arup, where a finance employee paid out around 25 million US dollars after joining a video conference in which the CFO and other colleagues were all AI-generated deepfakes. Enterprises now report average losses near 680,000 US dollars per voice fraud attack. The uncomfortable part for leaders: anyone with public audio, a conference talk, a podcast, a webinar, a LinkedIn video, is now a viable impersonation target.

This is a business-process problem, not just an IT one

AI-powered business email compromise drove 2.77 billion US dollars in losses across more than 21,000 incidents in a single year, according to the FBI’s Internet Crime Complaint Centre. The single most effective defence against the deepfake voice and video version costs nothing to implement: a firm policy that no payment, transfer, or change of bank details is ever authorised on the strength of a call or video alone, and must always be verified through a second, independent channel. That one rule removes the entire attack surface that deepfake voice and video fraud depends on. Technology cannot fix a process gap, and this is a process gap.

Training still works, but it has to evolve

Here is the number that justifies the whole effort. KnowBe4’s 2025 Phishing by Industry Benchmarking Report, the largest dataset of its kind, found the global average Phish-prone Percentage sits at 33.1 percent, rising to 37.1 percent in North America. In plain terms, roughly one in three untrained employees will interact with a phishing simulation. Training drives that number down sharply, but only if it matches the modern threat.

We run KnowBe4 for our managed clients, and it now includes dedicated deepfake training built specifically for AI-driven social engineering. Two things make the difference. First, multi-vector simulations: attackers coordinate an email, then a text, then a follow-up phone call on the same pretext, and employees who breeze through email-only tests routinely fail when a convincing call follows. Training has to rehearse that combination, not each channel in isolation. Second, repetition: structured vishing simulations have been shown to improve employee verification behaviour by around 65 percent, and continuous simulation-based training cut successful compromises by nearly half over twelve months. This is the core of our cyber security awareness training.

Dimension Old-style phishing AI-era social engineering
Tell-tale signs Typos, odd grammar, generic greeting Flawless, personalised, references real context
Channel Email only Email, voice, video, and SMS combined
Target Mass, untargeted Specific named executives and finance staff
Effective defence Spot-the-typo training Multi-vector simulations, verification policy, phishing-resistant MFA

The human layer does not stand alone

Even the best-trained employee will occasionally be fooled, which is why the human layer has to sit behind the technical ones. Phishing-resistant multi-factor authentication, covered in our Zero Trust post, means that even a credential handed to a convincing fake is far harder to reuse, and identity monitoring catches the compromise that does slip through. People, process, and technology reinforcing each other is the whole idea behind our managed cyber security.

What you should do now

Put a verification policy in writing today. No payment or bank-detail change authorised on a call or video alone, ever, with mandatory out-of-band confirmation. It is free, it is fast, and it defeats the most expensive class of attack.

Upgrade your training to multi-vector and deepfake-aware. An annual email-only phishing test no longer reflects how staff are attacked. Simulations need to combine email, voice, and SMS, and specifically rehearse deepfake scenarios.

Measure your baseline. You cannot improve what you have not measured. Contact Epic IT for a free phishing risk assessment and we will show you your real Phish-prone starting point and how to bring it down.

Frequently asked questions

How is AI changing phishing and social engineering?
AI removes the grammar mistakes and generic wording that people were trained to spot, and extends attacks beyond email into cloned voice and deepfake video. Analysis suggests 82.6 percent of phishing emails now contain AI-generated content, so a polished, personalised message is no longer a sign of safety.
What is a deepfake CEO or CFO fraud?
It is an attack where criminals use AI-cloned voice or video to impersonate a senior executive and pressure staff into transferring money or changing payment details. In the Arup case, an employee paid out around 25 million US dollars after a video call in which every colleague shown was an AI-generated deepfake.
How can businesses defend against deepfake fraud?
The most effective single step is a process rule: never authorise a payment or bank-detail change on the basis of a call or video alone, and always verify through a separate, independent channel. Pair that with multi-vector awareness training and phishing-resistant MFA for defence in depth.
Does security awareness training actually work against AI threats?
Yes, when it is continuous and matches the modern threat. Roughly one in three untrained employees fail a phishing simulation, but structured, repeated training, including voice and deepfake scenarios, has been shown to improve verification behaviour by around 65 percent and cut successful compromises by nearly half over a year.
What is multi-vector phishing simulation?
It is training that tests employees across more than one channel at once, for example an email followed by a text and then a phone call built on the same story. Attackers coordinate across channels, and staff who pass email-only tests often fail a coordinated approach, so simulations need to mirror that.

Next in the series: business email compromise, and how AI has upgraded the most expensive attack in cybercrime.

Would your team spot a deepfake of your CEO?

Our Perth-based team can run a free phishing risk assessment, giving you a real baseline and a plan to reduce it. Contact us on 1300 EPIC IT.

Book a Free Phishing Risk Assessment

About the Author
Written by Chris Arceo, Cyber Security Officer at Epic IT, a CRN Fast50-recognised managed IT services provider in Perth. Chris holds a Bachelor of Science in Information Technology (Network Administration) and over a dozen active certifications including CompTIA Security+, Cisco CCNA, and specialist qualifications across Datto, Sophos, Kaseya, and ConnectWise platforms.

Further Reading

Previous

Vulnerability management when attackers scan faster than you can patch

Return to News
Back to News
Next

Microsoft 365 packaging changes June 2026: what AU SMBs actually need to know