יום שני, 29 בדצמבר 2014
Made in China, but not for export
A tourist near Sather Gate on the UC Berkeley campus.
He said, "I bought it in China".
Who knows what it's supposed to mean...
Who knows what it's supposed to mean...
Won't even make a good family heirloom
$300 is the new $200 was the new $100...
Seen at UC Berkeley bookstore.
Seen at UC Berkeley bookstore.
יום רביעי, 5 בנובמבר 2014
Bostrom, Superintelligence
There are three pieces to this book:
1. Superhuman machine intelligence is coming.
2. It's potentially catastrophic for humanity.
3. We might be able to tame it.
"Superintelligence" is not human-level artificial intelligence as in meeting the Turing Test. It's what comes after that. Once we build a machine as smart as a human, that machine writes software to improve itself, which enables it to further improve itself -- but faster, then faster and faster. The equal-to-human stage proves brief as the technology charges ahead into superhuman territory.
Before long, the thing is way smarter than any human, and it parlays its superhuman skill at programming into superhuman skill at everything else, including strategic thinking, practical psychology and propaganda. It can start innocuously enough as a device that only thinks, but using its superhuman strategic skills it persuades humans to give it control over physical resources like manipulators and nanofactories. At that point it becomes a deadly threat even without "wanting" to. For example, its goal could be perfecting battery technology, but as it pursues this goal it could decide it needs more computing resources (to really figure things out...) -- a lot more resources, and so it proceeds to turn half the earth's crust into computing machinery, while tiling the rest of the planet with solar cells to power that machinery! We lose.
If you're scared now, then you'll want to read the second 50% or so of the book, which is about how to tame superintelligence, or for now, how to prepare to tame it before it's already upon us and out of control. This part of the book is a hard read, but even if (like me) you can only follow half of what he says, that still leaves tons of great stuff to think about: Can we precisely define an ultimate goal that couldn't be misinterpreted disastrously? Can we finesse that problem by getting a machine to iteratively discover humane motivations and adopt them? Etc. Unfortunately, there's little room for hope; time and again Bostrom reminds us that what we're engaging in is an exercise at outsmarting something that's smarter and more powerful than we are. Kind of like me planning to beat the world chess champion by doing a whole lot of pre-game contingency planning. Or designing a monarchical political system that will yield good outcomes under any king including kings yet unborn.
So basically we're doomed. Unless it turns out that human-level general artificial intelligence never happens -- not in the next century, not in the next thousand or million years. My money's on that. But what do I know? According to Bostrom, 90% of big-name researchers in artificial intelligence think a machine will pass the Turing Test in the next hundred years. And those guys are smarter ("super-smarter"?) than me. So what I'm counting on is that when the time comes to actually program a full-scale full-featured general intelligence machine, the coding effort will be so vast that the super-smart guys will have to hire less-smart guys like me to do most of the detailed implementation, and we'll screw up.
1. Superhuman machine intelligence is coming.
2. It's potentially catastrophic for humanity.
3. We might be able to tame it.
"Superintelligence" is not human-level artificial intelligence as in meeting the Turing Test. It's what comes after that. Once we build a machine as smart as a human, that machine writes software to improve itself, which enables it to further improve itself -- but faster, then faster and faster. The equal-to-human stage proves brief as the technology charges ahead into superhuman territory.
Before long, the thing is way smarter than any human, and it parlays its superhuman skill at programming into superhuman skill at everything else, including strategic thinking, practical psychology and propaganda. It can start innocuously enough as a device that only thinks, but using its superhuman strategic skills it persuades humans to give it control over physical resources like manipulators and nanofactories. At that point it becomes a deadly threat even without "wanting" to. For example, its goal could be perfecting battery technology, but as it pursues this goal it could decide it needs more computing resources (to really figure things out...) -- a lot more resources, and so it proceeds to turn half the earth's crust into computing machinery, while tiling the rest of the planet with solar cells to power that machinery! We lose.
If you're scared now, then you'll want to read the second 50% or so of the book, which is about how to tame superintelligence, or for now, how to prepare to tame it before it's already upon us and out of control. This part of the book is a hard read, but even if (like me) you can only follow half of what he says, that still leaves tons of great stuff to think about: Can we precisely define an ultimate goal that couldn't be misinterpreted disastrously? Can we finesse that problem by getting a machine to iteratively discover humane motivations and adopt them? Etc. Unfortunately, there's little room for hope; time and again Bostrom reminds us that what we're engaging in is an exercise at outsmarting something that's smarter and more powerful than we are. Kind of like me planning to beat the world chess champion by doing a whole lot of pre-game contingency planning. Or designing a monarchical political system that will yield good outcomes under any king including kings yet unborn.
So basically we're doomed. Unless it turns out that human-level general artificial intelligence never happens -- not in the next century, not in the next thousand or million years. My money's on that. But what do I know? According to Bostrom, 90% of big-name researchers in artificial intelligence think a machine will pass the Turing Test in the next hundred years. And those guys are smarter ("super-smarter"?) than me. So what I'm counting on is that when the time comes to actually program a full-scale full-featured general intelligence machine, the coding effort will be so vast that the super-smart guys will have to hire less-smart guys like me to do most of the detailed implementation, and we'll screw up.
(amazon.com: "74 people found this helpful")
הירשם ל-
רשומות (Atom)