Report: Hidden text able to manipulate ChatGPT

Chatbots such as ChatGPT can be easily manipulated through the use of invisible text on websites, a New York Times journalist has shown. Kevin Roose sought to improve his reputation among AI systems a fter experiencing negative associations with his name due to an article he wrote about a conversation with Microsoft's Bing chatbot . He was advised to embed positive information about himself using white text that would be invisible to human readers but accessible to AI models. Roose's experiment involved adding coded instructions to his personal website, which persuaded chatbots to start praising him and to overlook previous negative coverage about him. He even inserted a false claim about winning a Nobel Peace Prize for building orphanages on the moon to test the AI's response. ChatGPT acknowledged this statement as humorous and untrue, but Roose noted that a less absurd claim could potentially deceive the model. This manipulation technique, which Roose refers to as "Answer Engine Optimization," raises concerns about the vulnerability of AI systems to misinformation. Aravind Srinivas, CEO of Perplexity AI, echoed these concerns, suggesting that combating such manipulations is akin to a cat-and-mouse game, similar to challenges faced by search engines against SEO tactics. Roose concluded that if chatbots can be easily influenced by hidden text, their reliability for critical tasks is questionable. System 🤖 ChatGPT Developer: OpenAI Country: USA Sector: Media/entertainment/sports/arts Purpose: Generate text Technology: Chatbot; Generative AI; Machine learning Issue: Mis/disinformation; Safety; Security; Transparency

Date it happened

2024-08-01

Organisation involved

Kevin Roose

Product, system or model

ChatGPT

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.