Title: Superintelligence: Paths, Dangers, Strategies
Author: Nick Bostrom
Year Read: 2015
Buy This Book: Amazon

Below are the notes/bullet points I wrote down while reading Nick Bostrom’s “Superintelligence: Paths, Dangers, Strategies”. They were mostly notes for my future self, but I decided to start publishing all of my reading notes on this blog. Hopefully, someone else finds them useful too, but I will not be sad if I’m the only person who will ever read them :)

Paths To Superintelligence

There are 5 different, but not mutually-exclusive, ways we can get to a greater-than-human intelligence:

Forms Of Superintelligence

Digital Intelligence Advantages Over Biological

The Superintelligent Will

Failure Modes

The Control Problem

Superintelligence Castes

Value Loading Problem

The problem of loading values into an artificial agent to make it pursue the value as its final goal.

Value Choosing Problem

Suppose we can load the values into the system, now we need to decide what values to load.