IBM accused of secretly using NYPD images to train object recognition tool

IBM developed a surveillance tool for the New York Police Department (NYPD) that could identify individuals based on physical characteristics, including skin colour, raising serious privacy and racial profiling concerns. What happened IBM secretly used video footage collected by the NYPD to develop software capable of identifying individuals based on features such as hair colour, clothing colour, facial features and skin tone. The system could recognise more than 16,000 data points, making it highly accurate for recognising faces. While the NYPD claimed the skin tone search feature was only used for evaluation purposes, the technology's development and potential use raised significant ethical concerns about privacy and racial profiling. Why it happened The NYPD began collecting images of New Yorkers following the 9/11 attacks to monitor potential terrorist activity and other suspicious behaviour. IBM partnered with the NYPD to develop object identification technology, gaining access to the department's camera footage to create new search features for law enforcement use. What it means IBM's activities represent a significant breach of privacy, as thousands of unknowing New Yorkers had their images used to develop surveillance technology without their knowledge or consent. Furthermore, the potential for misuse and racial profiling is substantial, given the system's ability to search based on skin tone and ethnicity. For society, this case highlights the urgent need for regulation and transparency in the development and use of facial recognition and surveillance technologies. It raises critical questions about the balance between public safety and individual privacy rights, as well as the potential for these technologies to exacerbate existing racial biases in law enforcement. The incident underscores the importance of public oversight and ethical considerations in the deployment of AI-powered surveillance tools in urban environments. System 🤖 Unknown Developer: IBM Country: USA Sector: Govt - police Purpose: Identify facial features, skin tone, age, etc Technology: Object identification Issue: Privacy/surveillance; Transparency Investigations, assessments, audits 👁️ The Interce pt. IBM Used NYPD Surveillance Footage to Develop Technology That Lets Police Search by Skin Color

Date it happened

2018-09-01

Product, system or model

IBM Intelligent Video Analytics 2.0

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.