Deepfake Technology Issues

This is a significant move in consumer deepfake protection: Chinese smartphone brand Honor introduces native deepfake detection for video calls. Announced last year but globally available from April, Honor claims they can identify suspected synthetic content in live video calls within six seconds. Using continuous frame-by-frame monitoring, Honor's detection analyses discrepancies in "eye contact, lighting, image clarity, and video playback". If suspected synthetic content is detected, users automatically receive a pop-up warning, like anti-virus software or on web browsers when accessing a website without a valid SSL certificate. The anti-virus framing for detection is understandably appealing- a seamless (but not infallible) protective layer between users and content on social media, video calls, or even suspected AI-generated emails. It's encouraging to see big consumer tech companies taking the risk of deepfakes seriously and looking to protect users with this integrated approach, but caveats do still apply: 🔎 It's unclear how the increasing use of filters or other benign synthetic effects may impact the triggering of alerts/detection. 🔎 A reliability benchmark hasn't been shared, nor has any red teaming/robustness testing. As usual, unreliable and unevolving detection often does more harm than good... 🔎 Research is still needed to understand if these notifications are meaningful interventions in a live conversational context. Too many false positives and the 'crying wolf' effect may also feed notification fatigue. Still, I'm confident Honor won't be the last smartphone company to introduce these native detection capabilities. Deepfake fraud numbers have skyrocketed (one study found a 2137% increase in the last three years), and AI-generated content continues to grow more pervasive and sophisticated. I wouldn't be surprised if these features become key product differentiators moving forward, particularly for corporate customers where security is the ultimate priority.

There’s a pretty good chance that the shocking rate at which AI is advancing is out-pacing your cyber security training, policies and maybe even technologies. Have you addressed the use of AI and deep fakes in your cyber security policies? In a recent and alarming development that seems to have leapt straight from the pages of a science fiction novel, a Hong Kong based finance worker at a multinational firm was defrauded of $25 million, falling victim to an elaborate scam that employed deepfake technology to impersonate the company's CFO. This incident, which unfolded during a video conference call, marks a disturbing milestone in the intersection of cybercrime and AI, underscoring the urgent imperative for companies to bolster their cybersecurity frameworks, particularly against the backdrop of deepfake technology. The mechanics of the scam were deceptively simple yet devastatingly effective. The finance employee was lured into a video call with several participants, believed to be colleagues and the CFO, only to discover later that each participant was a digital fabrication. The deepfake avatars, mirroring the appearance and voice of real company personnel, instructed the employee to initiate a "secret transaction", leading to the unauthorised transfer of $25.6 million. This incident is not an isolated event but rather a harbinger of the potential threats posed by AI-driven disinformation and fraud. The use of deepfake technology to bypass facial recognition software, impersonate individuals for fraudulent purposes, and undermine the integrity of personal and corporate identities presents a clear and present danger. The case in Hong Kong, where fraudsters successfully manipulated digital identities to orchestrate financial theft, exemplifies the sophistication of contemporary cybercrime. The implications of this event extend far beyond the immediate financial loss. It serves as a stark reminder of the vulnerabilities inherent in digital communication platforms and the necessity for robust verification processes. The reliance on video conferencing and digital communication, accelerated by the global pandemic, has exposed systemic weaknesses ripe for exploitation. In response to this escalating threat, it is incumbent upon companies to adopt comprehensive cybersecurity strategies that address the unique challenges posed by deepfake technology. This includes implementing advanced authentication protocols, raising awareness and training employees on the potential risks of deepfakes, and deploying AI-driven security measures capable of detecting and neutralising synthetic media. As AI output become increasingly indistinguishable from reality, the line between authentic and artificial communication will blur, challenging individuals and organisations to navigate a new frontier of digital authenticity. It compels a reevaluation of the assumptions underpinning digital trust and identity verification, urging a proactive approach to cyber defence.

BSE has now warned investors about a deepfake of their own CEO 4 times in 4 months. The 4th warning came 3 weeks ago. Each time, a fabricated video of BSE MD & CEO Sundararaman Ramamurthy circulates on social media and WhatsApp, supposedly offering stock tips and promising extraordinary returns. Each time, BSE has to put out a public clarification, file complaints, and ask platforms to take the content down. Four times. Four months. The same man's face. The same institution's credibility. Used as a weapon, repeatedly, to steal money from ordinary investors who trusted what they saw. I want to explain what this actually is, because calling it "deepfake fraud" undersells the problem. Because BSE has spent decades building credibility as the institution Indian retail investors rely on. So every deepfake of Ramamurthy isn't just a fraud, it is a direct assault on the reputation of the oldest stock exchange in Asia, weaponised against the very people it exists to protect. In 3 decades of cybersecurity, I've watched every kind of digital attack evolve from technical exploits into social engineering. But right now, the attacker no longer needs to break through your firewall. They just need to make their content look legitimate enough that a retired schoolteacher in Nagpur believes the CEO of BSE is personally recommending a stock to her. That's the attack surface that's wide open right now. And not just BSE. Every recognisable face in Indian finance, business, and government is a potential weapon in someone's fraud campaign. The Pi-Labs data puts it plainly: 550% rise in deepfake-related banking and financial fraud cases since 2019. The platforms that host this content are moving slowly. The legal process is moving slowly. But the technology generating the fakes is moving fast. Which is why cybersecurity can no longer remain limited to protecting systems inside the organisation. Institutions now need visibility into how their brand, executives, and public trust are being exploited outside their own perimeter across social platforms, fake domains, messaging apps, and the dark web. At Seqrite, we are seeing this shift accelerate rapidly. What is your experience around these challenges? #SeqriteDRPS #CyberSecurity #Deepfake #DigitalTrust #AI #FraudPrevention #BrandProtection #Seqrite

AI PR Nightmares Part 2: When AI Clones Voices, Faces, and Authority. What Happened: Last week, a sophisticated AI-driven impersonation targeted White House Chief of Staff Susie Wiles. An unknown actor, using advanced AI-generated voice cloning, began contacting high-profile Republicans and business leaders, posing as Wiles. The impersonator requested sensitive information, including lists of potential presidential pardon candidates and even cash transfers. The messages were convincing enough that some recipients engaged before realizing the deception. Wiles’ personal cellphone contacts were reportedly compromised, giving the impersonator access to a network of influential individuals. This incident underscores a huge growing threat: AI-generated deepfakes are becoming increasingly realistic and accessible, enabling malicious actors to impersonate individuals with frightening accuracy. From cloned voices to authentic looking fabricated videos, the potential for misuse spans politics, finance, and way beyond. And it needs your attention now. 🔍 The Implications for PR and Issues Management: As AI-generated impersonations become more prevalent, organizations must proactively address the associated risks as part of their ongoing crisis planning. Here are key considerations: 1. Implement New Verification Protocols: Establish multi-factor authentication for communications, especially those involving sensitive requests. Encourage stakeholders to verify unusual requests through secondary channels. 2. Educate Constituents: Conduct training sessions to raise awareness about deepfake technologies and the signs of AI-generated impersonations. An informed network is a critical defense. 3. Develop a Deepfakes Crisis Plan: Prepare for potential deepfake incidents with a clear action plan, including communication strategies to address stakeholders and the public promptly. 4. Monitor Digital Channels: Utilize your monitoring tools to detect unauthorized use of your organization’s or executives’ likenesses online. Early detection and action can mitigate damage. 5. Collaborate with Authorities: In the event of an impersonation, work closely with law enforcement and cybersecurity experts to investigate and respond effectively. ———————————————————— The rise of AI-driven impersonations is not a distant threat, it’s a current reality and only going to get worse as the tech becomes more sophisticated. If you want to think and talk more about how to prepare for this and other AI related PR and issues management topics, follow along here with my series or DM if I can help your organization prepare or respond.

The New Corporate Threat: Deepfakes That Even Experts Can't Detect Welcome to the new reality where AI doesn’t just generate content, it manufactures convincing lies. You’ve probably seen it: - A CEO announces a fake acquisition. - A politician "says" something they never did. - A voice note "from your boss" requests a fund transfer. It all looks real. But it’s not. It’s a deepfake AI-generated audio, video, or images designed to deceive. Why it matters: Deepfakes are no longer just internet tricks or entertainment. They’re now: - Financial fraud enablers (voice clones used to scam employees) - Corporate risk vectors (fake news impacting stock prices) - Political weapons (manipulated clips used to sway public opinion) - Personal threats (identity misuse, blackmail, defamation) How to spot a deepfake  Look for: - Unnatural blinking or awkward lip sync - Plastic skin or weird lighting - Robotic tone or emotionless speech - Out-of-character statements - No credible source backing the video If it feels off, it probably is. What you can do: - Pause before sharing - Use tools like Deep ware, Microsoft Video Authenticator, or Adobe Verify - Train your teams especially PR, legal, and finance - Push for content provenance in your organization In the GenAI era, trust is currency. Don’t spend it on content you didn’t verify. #artificialintelligence

Microsoft's case against illicit AI developers confirms what we at Reality Defender have tracked for years: deepfake impersonation has evolved from theoretical concern to sophisticated criminal enterprise targeting vulnerable individuals daily and much more frequently than last year. While those of us with good BS detectors (and, yes, inference-based deepfake detection) are able to spot celebrity deepfakes from a mile away, these deceptive creations continue to be remarkably effective at defrauding everyday people. The financial impact is substantial, to say the least, and the aftermath of these scams extends beyond financial loss. Most importantly, when someone transfers retirement savings to a deepfaked "Elon Musk" investment scheme or sends money to an AI-generated "Brad Pitt," the profound shame often prevents victims from reporting these incidents — creating a dangerous gap in our understanding of the true scale of this crisis. What makes this trend particularly concerning is the organizational sophistication behind these operations. We're seeing structured criminal networks with specialized roles: technical developers creating the AI tools, others perfecting impersonation techniques, and frontline operators executing the financial fraud with increasing effectiveness. At Reality Defender, we partner with financial institutions to implement proactive protection against a related threat — deepfake impersonations of legitimate account holders attempting to breach security systems and conduct unauthorized transactions. These attacks threaten both individual finances and institutional reputational integrity, and like the victims of celebrity deepfake impersonations, are far more common than reported. As generative AI technology becomes even more accessible, we remain committed to sharing our insights while respecting victim privacy. Chances are high that your organization faces AI impersonation risks you haven't yet considered. Reality Defender's proactive detection measures can help you identify these vulnerabilities and implement robust safeguards before your customers or employees become victims. 

Deepfake Dominance in Cybercrime. We’ve crossed a tipping point: 40% of phishing campaigns are now AI-powered. Threat actors are extracting as much as $81,000 from a single victim using deepfake-enhanced tactics. Emails, calls, and even video conferences can now be convincingly AI-generated. This means traditional “spot the red flag” awareness training is no longer enough. Trusting your eyes or ears alone is no longer safe in a world where fraudsters can impersonate anyone. Zero Trust must extend to human identity verification. Confirm unexpected requests for money, credentials, or sensitive data through an out-of-band channel. Layer your controls. MFA, identity verification callbacks, and vendor authentication into daily workflows. Reinforce to employees that hesitation and validation are strengths, not weaknesses. At AdvisorDefense, we’re preparing RIAs for a reality where cybercrime isn’t just about malware, it’s about manipulation. If 40% of phishing is already AI-driven, the question is: how will your firm adapt before the other 60% gets there too? #AdvisorDefense #RIA #Cybersecurity #ZeroTrust

Fraud no longer hides in the shadows. It might show up disguised as someone you know. Like when the CEO calls and her voice on the phone sounds exactly right. Her urgency feels real, and the wire transfer request to a new bank account seems legitimate, so accounting releases the funds. And just like that, the company loses $20k to a fraudster who weaponized AI. This isn't science fiction. It's happening right now to individuals and organizations alike. Fraudsters are creating disturbingly real AI deepfakes that can fool even the most cautious people. And companies need strategies to combat them. Because those audio and visual cues we've relied on for decades are no longer reliable indicators of authenticity when it comes to AI deepfakes. Organizations can fight back with these defense strategies: ✔ Stay cautious and be wary of anyone requesting money or personal information, even if they look or sound like someone you trust. ✔ Don’t send money or share sensitive data in response to a single phone or video call. Phone numbers can be spoofed, so always verify a person’s identity by contacting them separately at a number you trust. ✔ Use small action requests, like asking a person to turn their head, blink repeatedly, or hum a song while on a video or phone call. If they decline, freeze up, or go silent, it could be a fraudster. ✔ Establish a safe word that only your inner circle knows to confirm the identity of someone claiming to be a colleague, family member, or friend.   ✔ Use strong passwords. Enable multifactor authentication (MFA) on all company devices and accounts whenever possible. And don’t forget to report AI deepfakes to law enforcement and any relevant social media channels, websites, and other platforms where the encounter took place. All of these tips ALSO work for individuals too because hackers like causing havoc with anyone they can. The question isn't whether AI deepfakes will target your organization. It's whether your organization will be ready when it does.   Food for thought as we kick off Cybersecurity Awareness Month.   ♻ Share our infographic to help companies combat AI deepfakes. 

A finance employee just wired $25 million to criminals. After a video call with her CFO. She could see him. Hear him. See her colleagues. All of them were AI. This happened to Arup, a major UK engineering firm, in 2024. And it's happening RIGHT NOW everywhere. Here's how the scam worked Finance employee gets email from "CFO" requesting urgent transfers. She's suspicious, so she demands a video call to verify. Joins conference with "CFO" and multiple "colleagues." Everyone looks real. Sounds real. She makes 15 transfers over several days. $25.6 million gone. The criminals? Downloaded public videos of these executives. Fed them into AI. Created perfect deepfakes in real-time on a live video call. Here's what terrifies me Q1 2025 numbers just dropped → $200 million stolen via deepfake fraud in 3 MONTHS → AI clones any voice with 3 seconds of audio → 68% of deepfake videos are indistinguishable from real → Deepfake incidents up 1,700% in North America → 51% of companies have ALREADY been targeted This isn't phishing emails anymore. This is your CEO on video asking for a wire transfer. And you can't tell it's fake. Ferrari almost fell for it too Executive received WhatsApp call from "CEO Benedetto Vigna." Voice perfect. Accent perfect. But the executive asked a personal question only the real CEO would know. The fake CEO hung up immediately. Here's what keeps me up at night As a cybersecurity recruiter placing SOC Analysts and CISOs, I can tell you Most companies are NOT prepared. They're focused on firewalls while criminals are → Scraping executive speeches from YouTube → Pulling voices from earnings calls → Grabbing faces from LinkedIn videos → Training AI models in hours Your security? Useless. The attack isn't against your systems. It's against your people's ability to trust their own eyes and ears. What companies need RIGHT NOW • Verify ALL financial requests through different channels... even video calls • Create "safe word" systems only real executives know • Multi-person approval for large transfers • Train employees: "I can see them" is NO LONGER PROOF But most companies won't act until AFTER they get hit. The Arup CFO said, "If cyberattacks were bullets, we would all be crawling around on the floor because they would be coming through the window, thousands of rounds a second." To every finance professional Next time your CEO asks you to wire money, even on video, verify through a DIFFERENT channel. Call their cell. Walk to their office. Text a personal question. Because seeing is no longer believing. To every CEO Your face and voice are weapons now. Every video you post trains the AI that will rob your own company. Sunday question If your CEO called you RIGHT NOW on video asking for an urgent wire transfer, what would you do? Be honest. Because criminals are betting you'll just do it. #CyberSecurity #Deepfake #AIFraud #InfoSec #AIScams

‼️ 🚨 AI Didn't Invent New Cyber Attacks - It Made Them Faster, Smarter, and Harder to Stop. The same AI models helping developers, security teams, and businesses boost productivity are also being weaponized by attackers. From deepfake executives to AI-generated malware, today's threats aren't science fiction they're already happening. This playbook breaks down 8 real ways adversaries are using AI and the defensive strategies every security team should know. Here's what stands out: 🎭 Deepfake CEO Fraud Attackers are using cloned faces and voices during live video calls to authorize fraudulent wire transfers. 🧠 Malicious LLMs Underground AI models remove safety restrictions, making phishing campaigns, malware generation, and social engineering more scalable than ever. 🔍 AI-Accelerated Reconnaissance LLMs can process massive amounts of public information in minutes, helping attackers build highly targeted phishing campaigns. 📞 Deepfake Voice Vishing A few seconds of publicly available audio can be enough to clone someone's voice and bypass help desk verification. 💀 AI-Generated Malware AI is helping attackers create, modify, and obfuscate malware faster, making traditional detection increasingly difficult. ⚡ Automated Exploitation AI agents can analyze vulnerabilities, generate proof-of-concept exploits, and dramatically reduce the time between disclosure and exploitation. 🔑 AI-Powered Password Attacks Machine learning models understand how humans create passwords, making password guessing far more effective than traditional brute force. 🛡️ Adversarial Machine Learning Attackers are beginning to target AI systems directly through adversarial inputs, data poisoning, and model manipulation. The biggest takeaway? AI didn't replace traditional attack techniques. It supercharged them. That's why strong security fundamentals matter more than ever: ✅ Verify sensitive requests through out-of-band communication. ✅ Deploy phishing-resistant MFA (FIDO2/Passkeys). ✅ Prioritize behavioral detection over static signatures. ✅ Reduce your organization's public exposure. ✅ Patch internet-facing assets quickly. ✅ Train employees to recognize AI-powered social engineering. AI is changing the threat landscape but organizations that combine modern security controls with disciplined security practices will remain ahead of the curve. 💬 Which AI-powered attack concerns you the most over the next few years? #CyberSecurity #ArtificialIntelligence #AISecurity #ThreatIntelligence #ThreatHunting #Deepfake #Phishing #Malware #RedTeam #BlueTeam #ZeroTrust #MachineLearning #SecurityAwareness #InfoSec #CyberDefense