Chinese AI criminality prediction study criticised as unethical

A system reportedly able to accurately predict whether someone is a criminal by analysing a few of their facial features was criticised as unlikely and unethical. Shanghai Jiao Tong University researchers Xiaolin Wu and Xi Zhang took ID photographs of 1,856 Chinese men between the ages of 18 and 55 with no facial hair, scars or other markings, half of which were criminals. They then used 90 percent of these images to train a convolutional neural network to recognise the difference and tested the neural net on the remaining 10 percent of the images. The claimed result that the neural network was correctly able to identify criminals and non-criminals with an accuracy of 89.5 percent was questioned by critics, who took issue with the research supposition, approach and methodology, especially with regard to potential data bias. Critics also expressed concerns about how data of this kind could be misused and abused, including in an authoritarian Chinese context. The researchers responded by saying "Our work is only intended for pure academic discussions; how it has become a media consumption is a total surprise to us." "Although in agreement with our critics on the need and importance of policing AI research for the general good of the society, we are deeply baffled by the ways some of them mispresented our work, in particular the motive and objective of our research," they continued. System 🤖 Unknown Operator: Shanghai Jiao Tong University Developer: Xiaolin Wu; Xi Zhang Country: China Sector: Politics; Research/academia Purpose: Predict criminality Technology: Computer vision; Neural network; Deep learning; Machine learning Issue: Accuracy/reliability; Dual/multi-use; Privacy/surveillance

Date it happened

2016-11-01

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.