Data ethics: an evolving challenge to policing legitimacy?Giles Herdaleco-chair IDEPP
Themes Peel’s principles: 7. To maintain at all times a relationship with the public that gives reality to the historic tradition that the police are the public and that the public are the police; the police being only members of the public who are paid to give full-time attention to duties which are incumbent on every citizen, in the interests of community welfare and existence. • Policing by consent in an age of big data and ‘surveillance capitalism’ • Police legitimacy has always been contested • Data adds a significant new dynamic to landscape requiring new approaches ‘I think we face two grave risks: In 10-20 years will we look back and say that the police service failed to embrace the opportunities presented by technology to keep people safe, or conversely will we be facing a scandal over use and abuse of data that puts the current undercover policing inquiry in the shade’ Cressida Dick, February 2019
‘The Das Kapital of data?’ Concern about ethics of new technology is not new, but the speed and pervasiveness of the data driven economy takes this to another level Zuboff describes the “rogue mutation of capitalism" that has created a "surveillance-based economic order" ‘Now we have markets of business customers that are selling and buying predictions of human futures.’
There is a clear conflict between the state’s drive to make use of an enormous range of data for policing purposes and our individual privacy and identity. The level of data required to operate algorithms presents a real risk to our privacy. Liberty ‘Policing by Machine’ 2019 ‘Justice is fairness. Public confidence in the police depends on their being conspicuously fair to everyone’ HMIC State of Policing 2017
Technology and the Law Policy Commission - Algorithms in the Justice System • There is widespread use of algorithms in the criminal justice system, and significant variety in this use • There is a lack of explicit standards and a lawful basis for the use of algorithms in the criminal justice system • Algorithms are not being critically assessed, and are creating risks to the justice system and the rule of law.
‘While AFR can enable police to identify persons of interest and suspects where they would probably not otherwise have been able to do so, considerable investment and changes to police operating procedures are required to generate consistent results.’ Professor Martin Innes Evaluating the Use of Automated Facial Recognition Technology in Major Policing Operations Nov 18
Can robots assess risk of re-offending better than humans? ‘I initially thought this was a project about technology, and later realised it was actually all about ethics’ Sheena Urwin Durham Police HART project Section 49 of the Data Protection Act 2018 outlines the restrictions on automatic decision-making, whereby a significant decision cannot be based solely on automatic computing. “There is a difference between an algorithm advising a human who then acts on a prediction, and that action happening automatically without human intervention,” From the beginning of the development of this tool, and since its validation, Durham Constabulary has been open about its use of HART, the tool’s internal workings and the results of the first validation exercise, attracting considerable attention (and sometimes criticism). The purpose of being so open was to acknowledge that this approach is new to policing and is therefore also new to communities.
‘Important that police use of new technology is grounded in proper ethical and legal considerations of what the technology actually achieves in practice’
‘For too long, ethics has been seen by many as separate from the day-to-day work of data science and AI research. As the pace of technological change increases, the stakes for society become higher.’ Data ethics is an emerging branch of applied ethics which describes the value judgements and approaches we make when generating, analysing and disseminating data.
A new dawn? Advisory body established to drive up oversight and transparency of AI and machine learning Part of the industrial strategy – UK being world leader in AI including ethics Review of Bias in Algorithmic Decision Making in Crime and Justice Developing a code of practice on Predictive Policing during 2019 Announced partnership with RUSI to map landscape and engage with stakeholders across policing
Outstanding issues Lack of a framework for new capabilities to be critically evaluated against – CDEI may provide this Chaotic and uncoordinated development approaches across policing – where is NPCC? Role of PTF – HO asleep at the wheel? Variable access to internal and external expertise to test and advise – who is building/maintaining the network ‘Doomed to success’ culture – innovation involves failure