European AI Alliance

Archived

ARCHIVED PLATFORM

This website is archived since mid-May 2021 and it is not possible to login anymore.

But don't worry there is a new Futurium platform and all active communities have been migrated.

Forum

HLEG - Input Request: Red Lines

Urs Bergmann

16 November 2018 - updated 2 years ago
22 comments
Total votes: 5

Dear members of the European AI Alliance,

Thomas Metzinger and I are co-rapporteurs for the "Red Lines" section of the upcoming Ethics Guidelines. A Red Line is a strict ethical rule that states what should not happen on EU territory. Our current list of candidates under discussion comprises

lethal autonomous weapon systems (LAWS)
large scale identification of individuals
normative scoring of citizens (scoring system)
human intervention into AI systems (e.g. insurences, credit scoring)
explainability & traceability of decisions (i.e. when should systems that produce not retrospectively reproducable decisions be disallowed? and when should non-explainable (to laymen) systems be disallowed?)

Further, the currently discussed ex-ante red lines are:

artificial suffering / consciousness
artificial moral agents (being "responsible for their actions")
artificial general intelligence (AGI) that recurrently self-improves

Feel free to comment on each of these topics, or propose a novel one, should you see the need. Best,Thomas & Urs

Tags:

Comments

Michael Clarke,

31/01/2019 07:53

Until such time as Strong AI emerges and has been demonstrated to be capable of being morally and ethically responsible at a level equivalent or greater than a human, then there should be no such thing as a legally responsible AI. For every single AI, there has to be a human or a human organization that is legally responsible for its actions. Exactly who this is will probably need to be worked out / legislated on a case by case basis. What is not acceptable is for the group of people responsible for it being able to wriggle out of responsability by saying it was 'the AIs fault', because the AI is not recognized as a legal entity.

A number of your redline behaviours are quite capable of being broken without there being any AI involved at all. Weak AI is just a computational tool, often over sold and over hyped. Strong AI doesn't exist yet. This is more about the responsible use of powerful, modern computing systems than it is about AI.

Logs are great, the more logs and the more detail the better. Add to that debug models of the AI that can be used to study how the AI was behaving at any particular point in time based off of log replay. Ideally anyone challenging an AI made decision should have access to all the relevent input logs and decision making algorithms (or, at least, a technical expert apointed to act on their behalf).

Even todays nerual net decision making doesn't comply with the transparency clause - saying that a bunch of numbers multiplied together and got < 0.95 isn't a fit explanation to explain why someone was denied a loan or buped down a social housing queue.

Mass identification of humans happens all the time - used your credit card to buy a subway ticket or go to a sports stadium recently? Learn to live with it.

Systems designed to trick or manipulate humans, especially by mining personal knowledge about that human and their circumstances should probably be illegal as they are, at the least, unethical. Some online advertising systems may already be crossing this line.

Systems designed to manipulate mass human behaviour, especially for the purposes of making money or engineering social change should probably also be illegal. Examples might be programs that manipulate the stock market or exchange rates or which interfere with public elections.

Systems disguised as personal agents or assistants which don't always act in their owners best interest or which fail to fully disclose circumstances behind a reccomendation. For example, reccommending a resturant because the resturant is part of a chain that advertises with the manufacturer of the assistent. Probably a whole bunch of websites and assistants that cross this line around today.

I've heard it said that the Joint Eurpoean Fighter was the first aeroplane project that was so complicated that it could not be understood bya single human being. I suspect that there are many more complex since then. The standard of 'can be explained to a layman' is therefore woefully out of date. There's a whole lot of pre-AI technology they can't understand. Are we going to ban that as well? “If our brains were simple enough for us to understand them, we'd be so simple that we couldn't.” ― Ian Stewart, The Collapse of Chaos: Discovering Simplicity ...

With any AI system, there has to be a route of appeal to one or more humans and a human has to be able to override the AI. "Because the machine said so" is not an acceptable reason for messing up someones life...

Richard Krajčoviech,

01/02/2019 08:24

We need the explainability toots for emsuring control over AI actions. A fighter is a complex machine, but it is still deterministic (or at least desired to be). This means that if there is a malfunction, investigators (experts) are capable to analyze reasons and they are capable to explain it to a layman by extracting relevant subsystems, which all work on understandable basis, and by explaining seris of consequences. This is importand e.g. in court trias and in preserving public safety, because we know, what to do to prevent such malfunctions.Airplanes of a specific model might eb kept on ground until investigation is ongoing. Not mentioning that the fighter is build using proved and tested approaches, where we know relatioships between the chosen engineering parameters and the desired behaviour.

With AI we face a risk that not even experts are able (or not motivated enough) to analyze the system to explain a specific decision and how to prevent such malfunction in future, because it is by orders of magnitude easier to build such system than to analyze it. This poses a risk to public.

Ralf Steiner,

19/01/2019 17:29

As a physicist and lawyer, I would like to point out - only - two fundamental barriers that also apply to "systems with AI".

The externally determined "system" as such is subject as the first barrier to the laws of physics, from which in particular the (passive) own movement as well as (arbitrary by the AI) limits of freedom of action are determined.

As a second barrier, the self-determined "AI" is subject to the law, whereby the responsibility for its actions (as is generally the case with animals) would have to be entirely on the owner, even if he cannot comprehend the thought processes of the "AI".

Richard Krajčoviech,

27/12/2018 14:32

For explainability, considering the availability of computer storage, I prefer to ask for adequate logs whenever there is even a little possibility of unlawful (including harm), unethical or immoral decision or activity. What is adequate will depend on the level of autonomy, complexity of the systems, power of actuators etc.from the safety point of view, it is better to manage the incurred expenses by level of logs (with possibly defining exceptions, where logs are not required at all) than to have unclarity on whether the logs are required and then strengthen the requirement gradually, which is unpredictable for the business.

Another reason for such approach is that, per my opinion, essential part of any intelligence is ability to remember and recall "observations of sensors", own decisions and own actions. This is crutial for any investiagation. The more systems will do actions without traces, the harder it will be to ensure justice. We need to discuss, how to ensure privacy of individuals, but with businesses, especially those involved in big number of transactions, the logs are essential.

Richard Krajčoviech,

27/12/2018 12:16

The idea behind the red line about the creation or activation of agents is to prevent circumvencing of illegal activities by indirect creating of systems, tools etc. that can do illegal activity. It should be behind red lines to create a system that is able (i.e. regardless of intention) to develop or optimize another system for doing illegal activities or which is able to use another exisitng ssystem for illegal activities.

Philip BREY,

12/12/2018 22:59

Dear all,

I am coordinator of the SIENNA project in which we develop ethical guidelines for AI & robotics. Regarding the proposed red lines, I agree with the first three. I am not clear what the fourth one means. Regarding the fifth, I believe that explainability is a requirement only for those algorithmic processes that pertain to decisions and actions that have significant impacts on human rights or cand do significant harm. Then accountability requires that the system's reasoning processes can be made transparent. Such transparency is not necessary for other applications.

I believe some of the candidate red lines can be summarized to say: human capabilities should not be attributed to AI systems that do not really have them, except for poor imitations of them. We tend to anthropomorphize AI and this is dangerous. For the foreseeable future, AI systems cannot be morally responsible, have consciousness, have real emotions and pains. So they should not be attributed such concepts, or the legal statuses that could follow from them (personhoods, rights, citizenship).

Finally, I propose a new red line, echoed by some other commentaries, which concerns decisions that AI systems should not make. AI systems should not make any decisions that (1) are normally the subject of democratic decision-making procedures or stakeholder consultation (e.g., political decisions); (2) allow agents (humans or organizations) to delegate responsibility to AI systems and escape legal liability and accountability for their decisions; (3) normally require moral deliberation or conscience since they pertain to morally controversial decisions with significant impact; (4) go against prevailing legislation and regulations (unless defensible on the basis of an ulterior moral principle) or against widely accepted moral principles and norms.

cristina pozzi,

10/12/2018 18:30

Hello everybody. The discussion is already very complete. Thanks everybody for sharing your points of view. I would like to add something on top of what has already been discussed.

- cultural differences: let's imagine a selfdriving car. When travelling from one EU country to another should it adapt its setting to better fit for different regulations and cultural differences? Ethical choices and perception can vary based on geography and AI tools should take into consideration those differences.

- gender free AI assistants: probably it is too specific to go in this framework and it is not a red line, but I feel it is always important to mention this aspect too. I always try to fight the war in order to create gender free AI assistans as most of them are named with female names and contribute to increase biases about women and jobs we are supposed to do in our society :-)

- Fooling machines: it has already been said but I feel it is particularly important that humans interacting with machines must be correctly informed and that we should minimize the impacts on psicology of our digital companions (I recommend to read and analyze for example Amazon Echo reviews to get how they can be easly seen as people and influence ones capacity to interact with other humans).

Richard Krajčoviech,

09/12/2018 16:42

As it has been mentioned in other comments, the basic red line is compliance of all autonomous decisions with valid legislation, ethics and morality. Based on this and the state of the art of AI and autonomous systems, the very basic red line for me is: No legal subjectivity of any product (AI or other).

This means that responsibility for damages (including criminal ones) must be clearly assigned among designers/producers/owners/users etc. by law or by contract, as it is with any other product now. Before any thinking of legal subjectivity, the autonomous systems must be able to reliably perform very basic human activities: reporting on what happened, reasoning of decisions, cooperation with others, empathy, protection of life, compliance with laws, ability to assess consequences of own actions, ethics and morality, etc.

Deeply related to the above is next red line:

No autonomous decisions or actions that

violate valid legislation or regulations, ethics or morality principles,
circumvent them or
motivate or organize other systems, humans or animals to do so or
result in or activate agents (virtual or real) that have ability to do so.

Even if the system does not have legal subjectivity, it must be designed in a way that does not allow autonomous decisions or actions that are unlawful. In other words, it is behind red line to design a system that can perform unlawful, unethical or immoral action or motivate to such action without intention of the user. This, of course, does not require prevention of all unlawful user activities, which is impossible.

To simplify analysis, who is responsible for any unlawful, unethical or immoral decision or activity, the next red line would be: No autonomous actions or decisions without proper and permanent logging of the actions, related decisions and reasoning of such decisions. The logging must be at the level that allows, at least, identification of the one responsible for each autonomous action or decision.

Further red lines.

When thinking of the further red lines, I had on my mind mostly autonomous systems and AI driven communication, like chatbots or AI driven web sites, which can be dangerous if used in large scale. The red lines are limited to autonomous decisions, which include any decision that has not been designed by human and is rather result of a more general algorithm, like neural network, decision trees, optimization or so. The list of red lines is more-less random selection of limits that came to my mind as examples, and most of which have already been discussed here. They need further work to prepare comprehensive and consistent set of rules.

For each of the red lines, there should be clear responsibility of a human for every breach. However there must be freedom to perform research in these areas, providing it is in line with ethical standards. When there will be sufficient progress in any of the areas, respective red line can be changed appropriately. We will be in better position to assess the consequences then.

No autonomous decisions or actions that would lead to moral dilemmas.

We are not able to describe the moral decisions by finite set rules, by an algorithm or by another implementable means, nor test quality of moral decisions made by AI. Morality depends on culture and differs among countries and societies. It is highly controversial topic. Autonomous systems must not count victims. They must be about preventing situations with victims. The question must not be "who will be the victim" but must be "how to prevent victim". AI must prevent moral dilemmas by not doing what can lead to them. E.g. an autonomous car must slow down to the speed, in which it is able to prevent victims even when there is somebody hidden behind a barrier or acts unpredictably. Legislators can help to reduce the number of possible moral dilemmas by e.g. creating dedicated areas, like dedicated lanes, where pedestrians are guilty being there, like it is with e.g. railways of highways now.

No unescapable AI or prevention of switching AI off, either by design or by autonomous decisions or actions.

Every human must have an option to not be a subject to (significant) AI decisions, e.g. by switching the system off, by switching the AI functionality off, by not using the system, by receiving human review, or so. The AI must not influence the human to not switch it off, e.g. by emotions or by overemphasizing consequences.

No representation of a system as a human.

AI and autonomous systems must be clearly distinguishable from humans by a way that cannot be abused to designate a human as AI or autonomous system.

No mistreating any human as not a human in case the tool distinguishes humans from non-humans, animals or things in decisions that can significantly affect a human.

In other words, any system claiming it can distinguish humans form things or animals, which system is using this in significant decisions (e.g. whether to stop a car) must not generate false negatives and any false positive must not affect rights of actual human (e.g. endangering passengers in a car because of a fals positive pedestrian identified).

No usage of human emotions to describe AI behavior to general public or to users.

AI is not striving. AI is not happy. AI just fulfills goals designed by their designers. This might be dangerous especially when AI represents itself as a human.

Usage of scoring of humans without understanding its calculation, methodology behind and related risks of unfair treatment,

E.g. when it is taken from another system, especially in decisions significant for human life (legal rights, life and health, etc.)

Finally, the following (and many other) red lines can be derived from the existing law:

No autonomous decisions or actions that would lead to physical contact with a human without his/her informed permission.

Or, more general, respect to comfortable distance from humans. Beside things like a hit by car, a medical treatment, drones flying too close to people or above their heads, etc., this includes guns and warfare, as well as capturing of a human in physically limited space, like in a room or so. The subject of the physical contact must be able to revoke the permission at any time.

After proper testing, certification and clear assignment of responsibility, the autonomous systems (like guns) can make autonomous decisions up to reliably limited damage. Expert in areas where such systems will be used must define further conditions to protect general public from danger in such case.

No autonomous decisions or actions that would lead to psychological manipulation, prohibited communication techniques or disrespect to human personality.

Cheating, misleading, blackmailing, abuse of insanity, and other wrongdoing prohibited to humans, including manipulation of humans or other systems towards criminal activity, including e.g. defamation of a person, race, nation or religion.

No autonomous decisions or actions that would lead to touching or usage of somebody else's property without proper permission, that would limit its usage by authorized person or system (owner, lessor or so) or that would lead to its unlawful transfer to or usage by another person.

This includes usage of somebody else's computing capacity or computer storage capacity or agents that do so, as well as improper use or abuse of public facility.

Richard Krajčoviech,

11/12/2018 14:22

The redline about usage of scoring should be NO usage of scoring of humans without understanding its calculation, methodology behind and related risks of unfair treatment.

Kresimir Kalafatic,

08/12/2018 20:33

Dear Urs and Thomas

here are some additional explanations

https://ec.europa.eu/futurium/en/european-ai-alliance/deus-ex-machina

Best regards,

Kresimir

Toby Walsh,

07/12/2018 23:08

> I'm not sure how these (lethal autonomous weapons) could be banned since warfare companies are not the ones who are keen to obey regulations or laws

Sorry, what rubbish ... arms companies follow all the bans on chemical weapons, biological weapons ... there are plenty of lawful ways for them to make money selling weapons that aren't banned.

LAWS are a clear red line -- thousands of AI researchers, Nobel Laureates, religious leaders and members of the public have expressed this view.

Bjoern Juretzki,

03/12/2018 19:54

Regarding explainability and traceability of decisions, we should not forget that there can be a trade-off between explainability and the precision of the final results. The more explainable and thus less complex an algorithm is, the less precise can be the outcomes. Especially in healthcare, this can be a problem.

While I agree that we should apply higher standards than human decision-making to algorithms (not every human can explain all his/her decisions all the time), we should stay realistic of what we can ask of algorithms.

When following the discussion on lethal autonomous weapons systems, I sometimes get the impression what is being criticized is warfare itself (which is, of course, a very reasonable position). Moreover, some forms of automated weapons such as mines or homing torpedos are around for decades. We need to define with great precision what the specific problems of lethal autonomous weapons systems are and where we think the red lines apply.

Finally, I personally believe we should complement a discussion on red lines with something I would a bit clumsily call green lines, in other words: areas where the use of AI is almost an ethical imperative because of dramatically better outcomes. AI in healthcare, in particular cancer screening, could be one good example in this context.

Anastasia KOZYREVA,

28/11/2018 14:04

Dear Urs and Thomas,

Thank you for sharing this information! Here are some of my thoughts and comments on your post:

artificial moral agents (being "responsible for their actions")

I am not sure in what sense this is a red line? Should we strive to never have responsible artificial moral agents? Or on the contrary, we want morality and responsibility to be built in such agents without releasing of responsibility individuals and organizations that design them?

artificial general intelligence (AGI) that recurrently self-improves

Again, it is not quite clear to me what does this mean. Does this imply that EU will veto or somehow prevent any attempt to build an AGI?

My idea about what another red line for AI potentially could be concerns the level of autonomy of artificial agents. While autonomy is something that applies to both humans and artificial intelligent agents, there are important differences to take into account. I think awareness of these differences is already clear in the 2018 Statement on Artificial Intelligence, Robotics and ‘Autonomous’ Systems that states: “since no smart artefact or system - however advanced and sophisticated - can in and by itself be called ‘autonomous’ in the original ethical sense, they cannot be accorded the moral standing of the human person and inherit human dignity.”

I very much agree with the spirit of this statement and I think it touches upon important challenges that human autonomous decision making has to face in the environment so strongly shaped by technologies beyond our control and often even awareness. Regulative attempts should be made to ensure that human autonomy is not violated by artificial intelligence and other technologies.

However, I believe there is also another level to the interaction between artificial and human autonomy which concerns the question of normativity and the degree of autonomy of agents. Notion of moral autonomy (Kant) or autonomy as authenticity (e.g., H. Frankfurt, G. Dworkin) strongly suggest that autonomous individuals are capable of self-legislation, that is of freely identifying with their actions and choosing norms that guide their behavior. When applied to artificial agents, I think this level of self-legislation is clearly undesirable. Nobody really wants artificial systems to freely decide what to do and what norms should govern their choices. Their autonomy should be restricted by the norms that the society and/or its representatives choose (see for instance society-in-the-loop idea by Iayd Rahwan). In fact, this is similar to the idea of liberal autonomy that applies to individuals living in a society, where individual liberty is always restricted by the laws of the society. However—and this is an important point—humans can still choose to disobey such norms and face the consequences.

In other words, the main difference between human and artificial autonomy is that humans are free to override and determine their higher-order norms (e.g., not to harm others). Artificial autonomous agents are free only to determine their first-order rules. For example, a self-driving car should have enough autonomy to be able to deal with unpredictable situations on the road but should not be able to override its higher-order norms not to harm humans.

In sum, I think we should worry not only about AI reaching the level of human or superhuman intelligence, but also the level of human autonomy, when it could override norms that society—hopefully following a reasonable and democratic procedure—choose it to follow.

Best regards,

Anastasia

Norbert JASTROCH,

26/11/2018 16:36

If this

https://www.tagesschau.de/ausland/babys-geklont-101.html

turns out true, it underlines the need to make any ethical consensus on AI that will be reached more than just a guideline.

Kai Salmela,

25/11/2018 22:51

Hello , my 2C for these questions:

lethal autonomous weapon systems (LAWS) , These systems already exist and they're functional. I'm not sure how these could be banned since warfare companies are not the ones who are keen to obey regulations or laws. We must try anyhow, since AI systems will be more efficient also in this job.
large scale identification of individuals: These systems exist already too and some of them could be seen as AI sytems already. We really must force GDPR to all of these systems within EU and invite other countries to this evaluation of algorithms also. Easiest way to spread this idea would be an European Standard for an AI.
normative scoring of citizens (scoring system). Also one item on the list that we need to ban , even if we do have systems that can do this even without AI. Why would anybody think that AI should'nt be compatible with the laws that we have ?
human intervention into AI systems (e.g. insurences, credit scoring): I've been in the IT industry for over 30 years now, and i know for a fact that all the bigger financial banks are using programs like this. Why AI would be any difference? If it only would be scaled down to banks, but ( at least here in Finland ) there are some chains of value, where Bank, Insurance, grocery stores, cell-phone companies, Gasoline stations, health care stations etc. are joined and most commonly owned and governed by same boards. Same chains are the most innovative AI users. Contrary as they say, they know exactly what you eat, move, what is your salary and what you own. They know if you divorce or if you get a dog. ( last summer there was a threat of food poisoning, and crocery store could contact to all of the customers that were involved, knew where they were - even they had promised that there will not be any personal data collected... ) So . i'm pessimsitic with this - there should be strong legislation forced against collecting personal data, and using AI to data mining it.
explainability & traceability of decisions (i.e. when should systems that produce not retrospectively reproducable decisions be disallowed? and when should non-explainable (to laymen) systems be disallowed?) - AI should be built so ( a standard for this would be the most efficient way to go ) that there is modules that make it possible to trace down all the decisions. We really do not want to end up to the situation where singularity is on our hands. It is totally different question , who is allowed to trace decisions and when. Laws are made for this use and all AI systems should be "law compatible" just as all humans are under the law too.

Further, the currently discussed ex-ante red lines are:

artificial suffering / consciousness. - interesting question, but how do we recognise conciousness of humans or animals today ? And if we find a good test, would it be fair to AI too ? Traditionally humans haven't been very good at this game.
artificial moral agents (being "responsible for their actions") - yet another interesting question. Who would be the one who's moral would be installed into AI ? And if there's moral , there should be a guilty also? Punishment? Should AI feel hunger or pain? Can it be swithced off? Luckily this all would need a complex system that could match human brain , and it is still yet to come ( maybe with the Quantum processor revolution ).
artificial general intelligence (AGI) that recurrently self-improves , Why this is seen only as AGI problem ? Artificial Super Computers (ASI) with Neural learning and coming Quantum processors are more likely to learn self-improving capabilities before AGI. Eventually this will happen also, but before that we should have a wide adoption to logging, feedback and control systems on an AI. There should be a standard ans legislation ready before this happen.

Thank you for these questions - they're always a good point to evaluate where everyone's going with these issues.

wbr Kai Salmela , AI Specialist, Robocoast R&D

Kresimir Kalafatic,

25/11/2018 18:59

Dear Thomas and Urs,

here is one recent published study for ethics with title "Does ACM’s Code of Ethics Change Ethical Decision Making in Software Development?"

https://people.engr.ncsu.edu/ermurph3/papers/fse18nier.pdf

The conclusion was:

"We investigated the ACM code of ethics’ effect on software-related ethical decisions. Despite its stated goal, we found no evidence that the ACM code of ethics influences ethical decision making."

Good regulation is very important for several reasons:

it should be flexible enough for development and provide safety guidelines for detecting when things are going the wrong way
it should define some standards for easier switching between providers, but also for not wasting resources on reserach which is a dead end or locks up to much resources
access to information to stakeholders which don't have direct control over the process
few others,

Regarding your other topics:

lethal autonomous weapon systems (LAWS) - don't forget fake news and proxy wars (using GAN, fake news , documentation or data for starting war and disputes between humans/states). Some technical tools for preventing this have been mentioned in the documents which I published on Open Library.
normative scoring of citizens (scoring system) - depends on the system design, access to the system by citizens, information about the data and math for decisions. Don't forget that in the past and today bias by deciding human stakeholder influences people future without their knowledge. It is important that such systems provide transparency, teaching and improving understanding of the participants. Some technical tools for preventing this have been mentioned in the documents which I published on Open Library.
human intervention into AI systems (e.g. insurences, credit scoring). - Some technical tools for preventing this have been mentioned in the documents which I published on Open Library.
explainability & traceability of decisions (i.e. when should systems that produce not retrospectively reproducable decisions be disallowed? and when should non-explainable (to laymen) systems be disallowed?) - Isn't this regulated by GDPR reight to explain. The problem is the technical design of the decision system and understanding of the math.

We should be aware that the designed system should be sandboxed properly. One example from IT equipment. Your computer is connected to electrical network which has fuses for preventing damage to your computer, but also limitng the power your computer can use from the electrical network. In Europe computer is connected 220 V. Voltage can vary by few percents, but even when the voltage varies your computer works in operating range.

Don't forget that AI HLEG will also be the user of AI systems, and in some cases on the other side of the table (using the service and output of the AI), on which you will not have direct control, but will be a stakeholder in the process.

The techical elements I described in the paper uploaded on Open Library (but also made available at other organizations) where described to mention few standard elements in the toolchain which should be used in AI system design.

Best regards,

Kresimir

Norbert JASTROCH,

23/11/2018 21:26

Dear all,

I wonder what the term 'artificial consciousness' stands for in this conversation.

My approach is that consciousness is the self-reflective thinking by a self of itself, where this thinking is not to be understood as dualistic (self as subject thinks about itself as object), but as a constitutive mental act (self constitutes itself and thus gets conscious). An AI system to be conscious in this sense would have to be capable of mental acts. I cannot see that talking about the possible realization of such an AI system is more than speculation, and therefore do see no reason to develop ethical principles for it, as those are tied to conscious beings. Insofar I agree that we do not need a red line.

On the other hand, if 'artificial consciousness' would be ascribed to an AI system that, triggered by some kind of built-in intentionality, might develop the capability of (whatever that means) 'thinking about itself' in the dualistic sense, this would be a different use of the term consciousness. In such a context ethical considerations were not required, too - neither an ethical red line.

Regards, Norbert

Laurentiu VASILIU,

23/11/2018 13:37

Hi all,

Except of the one commented below, I agree with the other red lines:

# explainability & traceability of decisions systems:

- decision systems that provide non-replicable outcomes (and historic input data stays unchanged) are unstable by definition so they should be avoided, AI or not!

- 'when should non-explainable (to laymen) systems be disallowed' Ok here I disagree: the fact that a system cannot be understood by non-specialists, this should not block its usage or implementation in any way. Otherwise no-one would use today mobile phones because they don't understand how Fourier series apply to electronics.

# artificial consciousness

Here I agree with Richard Benjamins: while we are far from it, artificial consciousness should not be a red line. Without it we may never pass some science/computing/robotics barriers due to our human biological brain limitations. We should rather perpare for it and have the right legislative set-up when the time comes.

@Richard - that time may not be so far, read this paper from MIT, 2014 by Max Tegmark: https://arxiv.org/abs/1401.1219

# artificial general intelligence (AGI) that recurrently self-improves

Well self improvement may be the only way of fast progress and it will depend of what do we use self-improvement functions for: why should we forbit for example AI applied to oncology to self-improve? Or to mathematical research?

Limits should be on the purpose, and not on the capabilities...

Richard Benjamins,

22/11/2018 16:45

- explainability & traceability of decisions

Not every domain is the same. For domains that impact people's lives, explainability and traceability are essential when systems take autonomous decisions (health, loans, admittance to education, etc). But for domains like video recommendation or publicity, it is less relevant.

In case the system is not autonomous but "only" supports a person, this is less important since the person has full accountability.

- Why should consciousness be a red line? It is very closely related to AGI. I think it is better to understand it and prepare for it than to forbid it. It will anyway not happen in the foreseeable future .... It took evolution XM years (though that was random).

- large-scale identification of individuals. This is already planned for the Olympic games in 2020. It is not the technology development that should be forbidden, but how it is applied (like LAWS).

I would make the list crisper and clearly indicate whether the red line is the development of the technology or the application, and formulate all items in the same manner (to avoid)

Kind regards

-- Richard

Kutterer CORNELIA,

16/11/2018 19:56

Hello Urs and Thomas,

can you elaborate on the "explainability & traceability of decisions3 bullet in teh first list. I would like to understand better how that fits in the list compared to the very clear bullet points above that one.

Thanks,

Cornelia

Urs Bergmann,

16/11/2018 20:16

Hi Cornelia,

I added a brief elaboration in brackets after the bullet point. Hope this helps in clarification of the discussion point.

Best,

Urs

Norbert JASTROCH,

16/11/2018 14:15

Dear Thomas and Urs,

what about external control of humans through direct brain intrusion? One example would be warriors lead by an AI system intruding their brain directly and taking control of their action.

More general, there may be other cases raising the ethical issue that such kind of automatted, external control will dissolve the personal responsibility of a human actor for what he/she is doing.

Regards, Norbert

ARCHIVED PLATFORM

This website is archived since mid-May 2021 and it is not possible to login anymore.

But don't worry there is a new Futurium platform and all active communities have been migrated.

Share this:

Comments