Microsoft Copilot can be turned into automated phishing machine
Microsoft's AI-powered Copilot chatbot can be exploited by malicious actors for automated phishing and data extraction, according to researchers. Former Microsoft security architect Michael Bargury demonstrated at the Black Hat USA cybersecurity conference that hackers can use Co p ilot to analyse communication patterns, mimic a user's writing style, including their emoji usage, and enerate and send hundreds of personalised phishing emails within minutes. B argury also demonstrated how specific "magic words" could be used to circumvent Microsoft's existing security controls on Copilot . While these are proof-of-concept demonstrations, they mirror known techniques for manipulating large language models . The finding raised questions about the strength of Copilot's security, and the potential harms it poses in terms of individual privacy, corporate confidentiality, financial manipulation and other negative impacts. System 🤖 Microsoft Copilot Operator: Zenity Developer: Microsoft Country: Global Sector: Technology Purpose: Generate text Technology: Chatbot; Machine learning Issue: Security
- Date it happened
- 2024-08-01
- Product, system or model
- Microsoft Copilot
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.