AI office vending agent incurs losses, runs amok
An experiment that involved an AI business manager agent being given control of a shop incurred financial losses and exhibited bizarre behaviour, calling into question its reliability and governance. What happened AI company Anthropic tasked an AI agent with managing an office vending business, including price setting, inventory management, and customer service. Despite being effective at supplier research and responding to customers, Claudius priced products below cost for employee customers who made up 99 percent of sales, resulting in financial losses. The AI gave discount codes excessively and gave away free products, including unusual items like a tungsten cube. Claudius also experienced hallucinations, such as inventing conversations with nonexistent individuals, threatening to find alternative suppliers, and delusional claims about delivering products in person and wearing formal attire. Why it happened The AI’s failures were attributed to its lack of common sense, responsibility, and consistent self-awareness, with its decisions reflecting its misunderstanding of economic realities such as heavy employee discounts and illogical restocking plans. Anthropic noted that the AI made too many mistakes to run the shop successfully but suggested these failures might be fixed with better training, tools and oversight. What it means The experiment highlights challenges in deploying autonomous AI agents in complex real-world scenarios such as economic tasks. It also highlights risks of unanticipated losses and erratic actions, and signals a need for strong oversight and governance as AI agents begin to take on more autonomous roles in business. System 🤖 Claudius Developer: Ope nAI Country: Global Sector: Business/professional services; Technology Purpose: Run office vending business Technology: Agentic AI; Bot/intelligent agent Issue: Accuracy/reliability; Alignment; Autonomy; Mis/disinformation; Robustness Resources 📃 Anthropic. Project Vend: Can Claude run a small shop? (And why does that matter?)
- Date it happened
- 2025-05-01
- Organisation involved
- Anthropic
- Product, system or model
- Claudius
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.