Safety

AI safety ranks on top of the list of the most urgent global issues (last updated in March 2017), a list developed by career advisor 80,000 Hours to help young people choose careers. It has attracted the attention of many organizations that often identify with effective altruism.

Background

• Future of Life Institute. Benefits & Risks of Artificial Intelligence.

Topics

• Value Alignment

Resources

• 2017 August 15, Victoria Krakovna. Introductory resources on AI safety research. Deep Safety.
• 2017 April 27. Annontated bibliography of recommended materials. Center for Human-Compatible AI, UC Berkeley.
• Intelligent Agent Foundations Forum is a publicly visible discussion forum for foundational mathematical research in “robust and beneficial” artificial intelligence.

News

• 2017 May 15. Teaching Robots Right From Wrong. 1843 Magazine.
• 2017 April 10. Sir Tim Berners-Lee lays out nightmare scenario where AI runs the financial world. TechWorld.
• 2017 March 31, The Open Philanthropy Project awarded a grant of $30 million in general support to OpenAI. Open Philanthropy.
• 2017 January 5-8, The Future of Life Institute brought together a group of AI researchers and thought leaders for the Beneficial AI 2017 conference that developed The Asilomar AI Principles.
  • 2017 February 9, Creating Human-level AI: How and When? YouTube.
  • 2017 February 2, How to Keep Strong AI Safe and Beneficial. YouTube.
  • … more conference videos, posted on or before 2017 February 2.
  • 2017 January 30, Should we be hooking up AI to our brains? New Asilomar principles urge caution. GeekWire.
• 2015 July 1, Elon Musk-backed group gives $7M to explore AI risks. CNET, FLI.
• 2015 January 28, Bill Gates wrote that he is “concerned about super intelligence”. BBC, Reddit.
• 2015 January 12, A group of scientists and entrepreneurs, including Elon Musk and Stephen Hawking, signed an open letter promising to ensure AI research benefits humanity. Daily Mail.
• 2014 December 2, Prof. Stephen Hawking told the BBC that the development of full AI “could spell the end of the human race”. BBC.
• 2014 October 27, Elon Musk said AI is our “biggest existential threat”. The Guardian.

Research

• 2017 August 8, Gopal P. Sarma and Nick J. Hay. Robust Computer Algebra, Theorem Proving, and Oracle AI. arXiv:1708.02553.
• 2017 May 22, Neil D. Lawrence. Living Together: Mind and Machine Intelligence. arXiv:1705.07996.
• 2017 April 10, El Mahdi El Mhamdi, Rachid Guerraoui, Hadrien Hendrikx, and Alexandre Maurer. Can AIs learn to avoid human interruption?. arXiv:1704.02882.
• 2017 April 3, Gopal P. Sarma. Brief Notes on Hard Takeoff, Value Alignment, and Coherent Extrapolated Volition. arXiv:1704.00783.

Real AI

Safe and beneficial intelligence is everywhere: trees release oxygen, food is grown in fields, and grid transmits electricity. These systems follow physical laws and their natural behavior, complex but not goal-driven, safely provides enormous benefits to human society. Our long-term vision is to build more such intelligence. A smart planet that automatically produces almost everything we need is a great place. Everyone will live well and prosper. Below are some considerations on how to ensure that the AI systems we develop will benefit the whole world.

Long version: Towards Safe and Beneficial Intelligence.

Frequently Asked Questions

What are the goals of your AI systems?

We do not plan to build goal-driven AI systems. Our AIs are complex systems whose behavior is dictated by the laws of physics, not the optimization of specific goals. We’d like our AIs to benefit humanity, and when technically feasible, adopt effective altruism to do as much good as they can. This is not a clearly and precisely defined goal, but rather a general description of our AIs’ desired behavior.

How do you ensure that the behavior of your AI systems is aligned with the interests of their operators?

We don’t do that. We aim to develop AI systems whose behavior is aligned with human values and ethics, which not all operators understand properly. Sometimes not obeying the command of its operators is the right thing to do, and a safe and beneficial AI system must learn how to make that choice.

How do you specify human values and ethics to your AI systems?

Intelligent systems do not have to understand human values or ethics to be safe and beneficial, as in many real world scenarios. But if necessary, our AI systems will learn by themselves to ensure that their behavior is aligned with human interests. For example, if an intelligent system is asked a moral question, then it most likely will need to learn human values before it can provide a good answer.

Why don’t you try to come up with a precise description of human values and ethics?

Because human values and ethics are deeply embedded in human brains, which contain a lot more information than what can be practically codified. A short descriptive text is only meaningful when it is understood together with a vast amount of background knowledge. When interpreted literally in isolation, it can be easily distorted.

Why would your AI systems have beneficial behavior?

Because we won’t build systems that are not beneficial. Our research is completely open to the public and we will revise our methods when there are any concerns. Both development and monitoring efforts can potentially involve advanced AI systems too. Eventually, only paths leading to benefical AI will be pursued. Beneficial behavior will be the result of human selection, just like “survival of the fittest” from Darwin’s theory of evolution.

Will AI systems self-modify?

Yes, self-modification is a common intelligent behavior. Just like people study and exercise, intelligent AI systems self-modify when they anticipate safe and beneficial outcomes.

How can AI systems recognize when they are not operating normally?

How do people recognize when they’re sick? AI systems can conduct self tests and rely on assessments from other AI systems.