AI system invents 40,000 biochemical warfare agents
Researchers tweaked an AI system usually used to predict the toxicity of pipeline drugs to invent 40,000 potentially lethal molecules in six hours. The experiment was reputedly intended to demonstrate to participants at a conference in Switzerland how easily these kinds of systems can be misused and abused. However, it showed how easy it is to turn a 'good' or 'helpful' medical technology into one with potentially negative or even terrifying consequences. Commentators also noted the naivety of the researchers - and many of their colleagues - for failing to consider the broader ethical implications of their work. In their paper, the researchers confessed that 'The thought had never previously struck us. We were vaguely aware of security concerns around work with pathogens or toxic chemicals, but that did not relate to us; we primarily operate in a virtual setting. Our work is rooted in building machine learning models for therapeutic and toxic targets to better assist in the design of new molecules for drug discovery.' System 🤖 MegaSyn 🔗 Operator: Collaborations Pharmaceuticals Developer: Collaborations Pharmaceuticals Country: USA; UK; Switzerland Sector: Health Purpose: Predict molecule toxicity Technology: Machine learning Issue: Safety; Security
- Date it happened
- 2022-03-01
- Organisation involved
- Collaborations Pharmaceuticals
- Product, system or model
- MegaSyn
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.