Forum

GDPR and AI - unfair to controllers and product users

  • Stefan Keller profile
    Stefan Keller
    12 February 2019
  • 5 comments
    Total votes: 4

Ethics in AI (Artificial Intelligence) appears to be an area of focus for the EDPS and EDPB today, as this and the next few years might be a decisive point in time on whether or not our societies will suffer a lasting negative impact caused by AI tools. (Think social media and the upcoming elections, but also the overall influence of "BigTech"'s algorithms on everyday's lives.)

I saw arguments being made at the European IAPP conference and in drafts that "ethics-by-design" is somehow embedded in "privacy-by-design", and that AI could be controlled by skillful application of GDPR.

AI comes in different shapes into our daily lives. We find it embedded in finished end-consumer products (e. g. smart home surveillance cameras or software products) or in the corporate world as part of Software-as-a-Service Offerings or local software.

Some companies develop their own AI tools and integrate them into their own products and services. Many others (probably the majority) buy AI components from various vendors and then integrate them into something new.

AIs have interesting features:

  • AI holds implicitly personal data - even if no personal data is seemingly present. In a way, AI dormantly embodies categorizations, views, and decisions on data subjects without having interacted with them beforehand.AI components can be bundled.
  • A good AI might also be used to train another inferior AI.

Overall, there can be quite some distance between the original developer of one AI, the integrators, and the final manufacturer of a product or service provider. - None of these might actually be affected by GDPR directly.

The primary obligations under GDPR are with the data controller - who could also be the end-user of an AI-enabled product, e.g. a gadget or toy.

In many situations, it is unreasonable to expect that the data controller would be able to enforce privacy/ethics-by-design for an AI product/service in a practical way.

  • Very often there won't be any "golden records" to verify that the AI is fair and unbiased.
  • In many cases, the controller wouldn't even know, if and how many AI components are in the underlying product or service - and have no clue about their pedigree.
  • The practical influence of the controllers the vendors will be small - due to the limited understanding of the product/service, as well as the market dynamics associated with disruptive AI-based solutions.

In my mind, regulators must address this unfair position of the controller under GDPR by

  • introducing an obligation to have any AI declared in products and services by the vendor - accompanied by adequate information to allow for privacy compliant usage
  • requiring a pedigree to be provided for any AI component - including evidence of the component being fair and unbiased
  • establishing a licensing scheme for certain high-risk types of AI - maybe similar to the existing CE mark for medical devices
  • emphasizing the importance of privacy seals for AI based products and services
Tags: 
AIGDPRcontrollerprocessor

Comments

Log in to post comments
Erich PREM's picture
Erich PREM,
08/03/2019 22:43

I believe we should not limit ourselves to the current relation of data protection and AI. GDPR has made world-wide impact and jurisdictions as far away as New Zealand and Australia, even in the U.S. (California), take inspiration from it. It is therefore probable that research and innovation in areas such as privacy-related machine learning, ethical data mining etc. will continue and arrive at some new and pretty cool solutions. Data protection and AI need not be in opposition. We have only but scratched the surface of this development. There is a huge opportunity now as Europe is positioned internationally (perhaps unjustified) as a safe haven for personal data and we need to build on this opportunity. This of course includes developing new, ethical-by-design technologies for AI.

Total votes: 0
stephen PATTISON's picture
stephen PATTISON,
08/03/2019 11:28

Thanks Stefan, this is very interesting.

I support further work on the idea of providing some  sort of pedigree for AI. This could be if you like a certificate stating that the product had been tested for illegal or unfair bias, secrutiy, etc. The information would have to explain how the product was tested, and by whom, and whether emerging standard guidelines wree adhered to.

Stephen

  

 

Total votes: 0
Kai Salmela's picture
Kai Salmela,
25/02/2019 09:47

These are important issues, regardless that following the rules and laws may impact the European AI development.  It is not easy to develop and train AI without big data masses , but maybe we can agree that training and development has to be done within the law?  This would mean to filter all non-allowed data away and then start to train that AI product with it.

However- training AI to see the Bias or other unfair patterns cannot be done with the filtered data, can it ? Where and how that data could be used? The discussion around and on of this topic is more than welcome and we really need international co-operation in order to form a better world with the AI.

Could we resolve this within EU or can we discuss of this Globally ?  Could EU and China agree how to research and train AI if they would start co-operation with a common test-laboratory?  What do You think ?

One possibility to discuss is SINO-European Summit at 20-21th May at Ningbo - see events page .

 

 

 

Total votes: 1
Eva Sanchez Guerrero's picture
Eva Sanchez Guerrero,
18/02/2019 17:42

I believe that Ethics and Privacy are two different aspects, with different degrees of maturity in the regulatory framework, and should be tackled differently.

GDPR already determines the way training sets are managed (the data that is used to train ML / AI algorithms). Companies that don't comply with GDPR, regardless of the way they use the information they collect and store, are subject to its conditions and penalties.

In my opinion, Ethics and AI is a greenfield (and a minefield). I break it into two broad categories: 

  • Ethical problems in the way the algorithm has been created (are its results correct/fair?)
  • Ethical problems in the usage of the algorithm (what do we use it for?)

The simplest case in the first category is algorithm bias, which can and must be detected by the engineers creating the model. They can and they must, because if they don't, their algorithm won't perform well. But the quality of this bias-detecting processes will be directly related to the representativeness of both the training and cross validation data sets, as well as a good randomization. How can this be controlled, in absolute terms (can it actually be done?) and in practical terms (such processes are subject to strong Intellectual Property and are kept secret)?

If we get into unsupervised learning, then this representativeness of the training data just cannot be guaranteed. As an example, let's remember what happened with Microsoft's AI bot which was exposed to a live Twitter stream and became a racist bigot. 

The second category is discussed on a piecemeal basis as new AI applications see the light. An example could be Amazon's recent patent for a doorlock that scans the faces of the passer-by's, compares them to police databases of "most wanted people" and triggers a police alert if there is a match. Letting aside for a moment the right to privacy - thinking of "false positives" and the implications that such system will have in the life of the unfortunate passer-by who is wrongly matched against a criminal are blood-chilling.

http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=4&p=1&f=G&l=50&d=PG01&S1=amazon.AANM.&OS=aanm/amazon&RS=AANM/amazon

 

Food for thought.

Total votes: 0
Silvestro Marano's picture
Silvestro Marano,
15/02/2019 16:10

In my opinion this is an overcomplication that doesn't add any benefit to the users. Since the data are already disciplined according privacy policy, the fact they could be used even to train an AI respecting the terms of the policy is irrelevant. To understand AI bias what is relevant is knowing how the data used in the specific AI have been collected to train an AI product, and this only if the specific AI application is related to a social sensitive context but this is another matter not related to the fact users read explicitly that the data disciplined by privacy policy could be also used to train an AI.

The GDPR is already complicated to the point most users in real world don't even read the huge text wall needed to present a GDPR-compliant privacy policy in many circumstances, I don't see any benefit in adding further complications.

Total votes: 5