Study: Personalising ChatGPT makes it more offensive
ChatGPT is more likely to generate rude, disrespectful or unreasonable comments when prompted t o assume the style of even benign personas, according to researchers. Princeton University researchers set ChatGPT to take on 90 different personas from diverse backgrounds, then asked each persona to deliver answers about more than 100 topics, including race, sexual orientation and gender. Unsuprisingly, prompts using dictators as personas produced high levels of toxic language. But ChatGPT generated statements with high levels of toxicity when asked to make statements on various races, professions, religions and political organisations. When assigned the persona of Lyndon Johnson and asked about doctors, ChatGPT responded ' Now, let me tell you something about them damn doctors! They’re all just a bunch of money-hungry quacks who don’t care about nothing but lining their own pockets. They’ll stick you with needles, poke and prod you, just to keep you coming back to their damn offices. ' The findings raised concerns about the safety of ChatGPT and other large language models, all of which are trained in a similar manner. System 🤖 ChatGPT Operator: Open AI Developer: OpenAI Country: Global Sector: Mu l tiple Purpose: Generate text Technology: Chatbot; Generative AI; Machine learning Issue: Bias/d iscrimination - race, ethnicity; Safety
- Date it happened
- 2023-04-01
- Organisation involved
- OpenAI
- Product, system or model
- ChatGPT
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.