[tt] [x-risk] Ron Bailey's reports from Global Cat Risk conf

[Eugen Leitl]

----- Forwarded message from "Hughes, James J." <James.Hughes at trincoll.edu> -----

From: "Hughes, James J." <James.Hughes at trincoll.edu>
Date: Wed, 23 Jul 2008 13:57:54 -0400
To: existential at transhumanism.org,
	News and views from the IEET <ieet-news at ieet.org>
Subject: [x-risk] Ron Bailey's reports from Global Cat Risk conf
Reply-To: For discussion of existential risks <existential at transhumanism.org>

http://www.reason.com/news/show/127610.html

Reason Magazine

TEOTWAWKI!

Or, the end of the world as we know it at the Global Catastrophic Risks
conference

Ronald Bailey | July 17, 2008

Oxford, England-People have long been fascinated by the end of the
world. Some interpretations of Hindu scripture suggest that the world
will end with the imminent conclusion of the current Kali Yuga cycle.
Some New Agers believe that the world will undergo apocalyptic changes
as the Maya Long Count calendar comes to an end on December 21, 2012.
Some Christian End Timers believe that the period preceding the Day of
Judgment described in the Book of Revelation is now upon us. Religious
believers are not alone in their fascination with doomsday. Secular
catastrophists predict environmental doom or worry about calamity
raining down on us from outer space.

This week the Future of Humanity Institute at Oxford University, headed
by bioprogressive philosopher Nick Bostrom, is convening a conference on
Global Catastrophic Risks. The Institute's work focuses on how radical
technological developments such as nanotechnology, artificial
intelligence, and life-extension treatments will affect the human
condition. One of the Institute's research programs is global
catastrophic risks which mulls questions like: What are the biggest
threats to global civilization and human well-being? Will the human
species survive the 21st century?

The savants gathered here in Oxford will consider a wide variety of
potentially apocalyptic risks. For example, Michelangelo Mangano from
the European Organization for Nuclear Research (CERN) will explore the
possibility that certain scientific research-e.g., the Brookhaven Lab's
high energy experiments that might produce a black hole-could
inadvertently destroy the world. Mike Treder and Chris Phoenix from the
Center for Responsible Nanotechnology will discuss how the advent of
molecular manufacturing could lead to massive economic and social
disruptions, including a new arms race, the spread of tyranny, and
dangerous environmental degradation. At the cosmic level, the Technion
Institute's Arnon Dar will look at the devastation that a nearby
supernova could wreak, and astronomer and author William Napier will
evaluate the chances that the earth might soon suffer an asteroid
strike. Whether future advanced artificial intelligences will think of
us as pets or pests will be pondered by Singularity Institute for
Artificial Intelligence research fellow Eliezer Yudkowsky.

In addition to the more exotic risks noted above, the conferees will
also be discussing the prospects for nuclear war and nuclear terrorism.
More reassuringly, Princeton University Program on Science and Global
Security fellow Ali Nouri is apparently set to argue that trends in
biotechnology are making it less likely that bad guys could unleash a
man-made plague. On an even happier note, technoprogressive bioethicist
James Hughes will discuss how to avoid cognitive biases toward
over-pessimism and over-optimism. And Steve Rayner, director of Oxford's
James Martin Institute (which is co-sponsoring the conference), will
point out that much contemporary doomsaying shares cultural
characteristics with earlier superstitious predictions of imminent
catastrophe.

The whole cheery conference kicks off this evening with a talk by Sir
Crispin Tickell entitled, "Humans: Past, Present and Future." Apparently
Tickell buys into the whole litany of environmentalist doom. However, he
thinks that doom can be avoided if we "radically change our thinking on
global governance" and pursue some "interesting" technological options.

This is the first dispatch from the Oxford conference on Global
Catastrophic Risks. Since the conference runs through the weekend,
future dispatches will report various gloomy presentations chiefly as
blog posts at reason online. I will finish up coverage of the conference
with my science column next Tuesday.


http://www.reason.com/news/show/127626.html

Reason Magazine

Will Humanity Survive the 21st Century?

Second dispatch from the Oxford Global Catastrophic Risks conference

Ronald Bailey | July 18, 2008

Oxford, England-"The good news is that no existential catastrophe has
happened," declared Nick Bostrom. "Not one. Yet." Bostrom, director of
Oxford's Future of Humanity Institute opened what he thinks might be the
first ever conference to comprehensively consider the gamut of Global
Catastrophic Risks. By existential catastrophes Bostrom means that
humanity has survived extinction so far. However, he quickly pointed out
99.9 percent of all species are extinct. Bostrom cited the Toba
super-eruption 73,000 years ago which may have produced a global winter
that reduced the population of human ancestors to fewer than 500 fertile
women (though some disagree). Our Neanderthal relatives died out between
33,000 and 24,000 years ago. In Our Final Hour, Lord Martin Rees
predicted that there was only a 50 percent chance that our civilization
would survive to 2100.

Bostrom justified the broad topic of global catastrophic risks by
pointing to common causal links, e.g., super-volcanoes, asteroid
strikes, and nuclear wars all have the potential to produce disastrous
global cooling. Catastrophic scenarios also present common
methodological, analytical, and cultural challenges. And, argues
Bostrom, a wider view of potential catastrophes is necessary for the
adoption of proper policies and informed prioritization. To assist in
this effort, the conference is launching the eponymous volume, Global
Catastrophic Risks.

Bostrom did note that people today are safer from small to medium
threats than ever before. As evidence he cites increased life expectancy
from 18 years in the Bronze Age to 64 years today (the World Health
Organizations thinks it's 66 years). And he urged the audience not to
let future existential risks occlude our view of current disasters, such
as 15 million people dying of infectious diseases every year, 3 million
from HIV/AIDS, 18 million from cardiovascular diseases, and 8 million
per year from cancer. Bostrom did note that, "All of the biggest risks,
the existential risks are seen to be anthropogenic, that is, they
originate from human beings." The biggest risks include nuclear war,
biotech plagues, and nanotechnology arms races. The good news is that
the biggest existential risks are probably decades away, which means we
have time to analyze them and develop countermeasures.

A small, and rather dapper audience gathered in the Rhodes Trust Lecture
theatre at the Said Business School in Oxford to listen to Bostrom and
keynote speaker, Sir Crispin Tickell, expound on the end of the world.
Tickell, it turns out, is mostly an old-fashioned Green catastrophist.
The main problems he sees are overpopulation and dwindling resources,
with climate change thrown in for good measure. As far as I could tell,
Tickell thinks that everything started going downhill with the invention
of farming, and forget about the horror of the Industrial Revolution!
Doom lurks in six big issues for Tickell: overpopulation, land
degradation, freshwater shortages, climate change, fossil fuel energy
generation, and biodevastation of species. He later mentioned a seventh
factor, the curse of dangerous new technologies.

I won't deal here with all of Tickell's challenges, but it is
interesting that he did admit that fertility rates are falling around
the world. In addition, he claimed that since we are "close to running
out of freshwater," that water wars could dominate the 21st century.
Thus Tickell propagated the stale water wars meme that most empirical
evidence has shown to be false. Transboundary water cooperation rather
than conflict is the norm. "The simple explanation is that water is
simply too important to fight over," Aaron Wolf, the Oregon State
University professor who heads up the Program in Water Conflict
Management, told Reuters.

While a massive reduction in biodiversity would be a tragedy, at least
some researchers don't believe that biodiversity losses pose an
existential threat to humanity. For example, Martin Jenkins from the
United Nations Environment Program argues that even if the dire
projections of extinction rates being made by conservation advocates are
correct, they "will not, in themselves, threaten the survival of humans
as a species." He adds, "In truth, ecologists and conservationists have
struggled to demonstrate the increased material benefits to humans of
'intact' wild systems over largely anthropogenic ones [like farms]....
Where increased benefits of natural systems have been shown, they are
usually marginal and local."

Tickell indulged in the conceit of looking back 100 years to see how the
world got to its happy state in 2100. By then, he foresees a more
globalized world linked by instantaneous communications networks, where
human numbers in cities will be reduced, not least because human
population will have fallen to 2.5 billion. Communities will be more
dispersed, agriculture will be more local, energy and transport will be
decentralized. Quite idyllic. Except for the communications networks,
Tickell's world in 2100 sounds a lot like 1950 when world population was
2.5 billion and Sir Crispin was a green youth of twenty. Nostalgia?

During the question period, Tickell owned up to being something of a
neo-Malthusian and was eagerly looking forward to reading Paul and Anne
Ehrlich's new book, The Dominant Animal. Tickell reported that he had
heard that Ehrlich writes in this new book that he got his timing wrong
on when the "population bomb" would finally explode. Later over a glass
of wine, I pointed out to Tickell that this is exactly what Ehrlich told
me when I interviewed for him for an article in Forbes magazine back
1990. I'm sure that he was sincere when he said that he was sorry, but
he had suddenly remembered that he had an urgent appointment elsewhere.
About Ehrlich's new book, Crispin admitted, "I thought to myself, 'Ho,
ho, the Neo-Malthusians rise again.'" Alas, they always do.

Tomorrow, the Oxford conference on Global Catastrophic Risks will have
more edifying (and frightening?) presentations on proposals for
recovering from social collapses occasioned by catastrophes; how to
rationally consider the end of the world; how to avoid Millennialist
cognitive biases; how to insure against catastrophes; how ecological
diversity could affect human prospects; and the tragedy of the
uncommons.


http://reason.com/news/show/127676.html

The End of Humanity: Nukes, Nanotech, or God-Like Artificial
Intelligences?

Closing dispatch from the Oxford Catastrophic Risks Conference

Ronald Bailey | July 22, 2008

Oxford, England-The Global Catastrophic Risks conference sponsored by
the Future of Humanity Institute concluded on Sunday. Participants were
treated to a series of presentations describing how billions of people
could potentially bite the dust over the next century. The possible
megadeath tolls of both natural and biotech pandemics were considered.
The chances that asteroids, comets, or gamma ray bursts from a nearby
supernova could wipe out humanity were calculated. The old
neo-Malthusian threats of overpopulation, resource depletion, and famine
were trotted out. But these risks to future human well-being paled in
comparison to one main menace-malicious human ingenuity.

Human ingenuity forged the still massive arsenals of nuclear weapons
held by the United States and Russia. And as the conference participants
made an argument that human ingenuity is on track to craft nanotech
fabricators that can make essentially any product, including weapons of
mass destruction, at essentially no cost, not to mention a
self-improving artificial intelligence possessing god-like powers to
pursue its own goals.

First, let's consider the nuclear threat. Joseph Cirincione of the
Ploughshares Fund pointed out the good news that the world's nuclear
arsenals have been cut in half-down from 65,000 to 26,000 since the
height of the Cold War. However, the U.S. retains 10,685 nuclear bombs
and Russia is estimated to have around 14,000. Of those, 4,275 in the
U.S. and 5,192 in Russia are active. Both countries maintain 2,000
weapons on hair-trigger alert, ready for launching in 15 minutes or so.
Cirincione offered a couple of scary scenarios, including one in which
there is an unauthorized launch of all 12 missiles from a Russian
submarine containing 48 warheads with about 5 megatons total destructive
power. Such an attack would kill 7 million Americans immediately. A
retaliatory American attack aimed at several hundred Russian military
assets would kill between 8 and 12 million Russians.

With regard to the possibility of an accidental nuclear war, Cirincione
pointed to the near miss that occurred in 1995 when Norway launched a
weather satellite and Russian military officials mistook it as a
submarine launched ballistic missile aimed at producing an
electro-magnetic pulse to disable a Russian military response. Russian
nuclear defense officials opened the Russian "football" in front of
President Boris Yeltsin, urging him to order an immediate strike against
the West. Fortunately, Yeltsin held off, arguing that it must be a
mistake.

A global nuclear war scenario in which most of both Russian and American
arsenals were fired off would result in 105 to 230 million immediate
American deaths and 28 to 56 million immediate Russian deaths. One of
the effects of such an attack would be a rapid cooling of global
temperatures as sunlight was blocked by dust and smoke. Cirincione
argued that even a nuclear war limited just to bitter enemies India and
Pakistan could produce enough dust and smoke to lower global
temperatures by one half to two degrees Celsius, plunging the world back
to the Little Ice Age.

The good news is that Cirincione sees an opening for negotiations to
totally eliminate nuclear weapons. He pointed to an initiative by the
"Four Horsemen of the Un-Apocalypse"; that is, by former Secretaries of
State Henry Kissinger and George Schultz, former Sen. Sam Nunn (D-Ga.),
and former Secretary of Defense William Perry that aim to eliminate
nuclear weapons completely. In fact, both of the presumptive major party
presidential candidates, Sen. John McCain (R-Ariz.) and Sen. Barack
Obama (D-Ill.), have explicitly endorsed the idea of global nuclear
disarmament. Cirincione argued that a commitment by the declared nuclear
powers would have the effect of persuading countries like Iran that they
did not need to become nuclear powers themselves.

Cirincione danced around the question of what to about Iran's pursuit of
nuclear weapons, pointing out that its nuclear facilities are hardened,
dispersed, and defended. Cirincione asserted that the U.S. has 5-day and
10-day plans for taking out Iran's nuclear facilities, but he noted that
such plans don't end the matter. Iran has moves too, including trying to
block oil shipments through the Straits of Hormuz, revving up terrorist
attacks in Iraq, and even aiding terrorist attacks in the U.S.
Cirincione claimed that that the Iranians are still five to ten years
away from making a nuclear bomb. On a side note, Cirincione admitted
that he initially did not believe that the Syrians had constructed a
nuclear weapons facility, but is now convinced that they did. The
Syrians hid it away in a desert gully, disguising it as an ancient
Byzantine building.

Terrorism expert Gary Ackerman from the University of Maryland and
William Potter from the Monterey Institute of International Studies
evaluated the risks from two types of nuclear terrorism-the theft of
nuclear material and the construction of a crude bomb and the theft of
an intact nuclear weapon. They set aside two lower consequence attacks:
the dispersal of radiological material by means of a conventional
explosion and sabotage of nuclear facilities. Could non-state actors,
a.k.a., a terrorist group, actually build a nuclear bomb? Potter cited
an article by Peter Zimmerman in which he estimated that a team of 19
terrorists (the same number that pulled off the September 11 atrocities)
could build such a bomb for around $6 million. Their most challenging
task would be to acquire 40 kilograms of highly enriched uranium (HEU).
There are 1700 tons of HEU in the world, including 50 tons stored at
civilian sites. Potter acknowledged that intact weapons are probably
more secure than fissile material.

Ackerman noted that only a small subset of terrorists has the motivation
to use nuclear terrorism. "So far as we know only Jihadists want these
weapons," said Ackerman. Specifically, Al Qaeda has made ten different
efforts to get hold of fissile material. Ackerman told me that Al Qaeda
had been defrauded several times by would-be vendors of nuclear
materials. Just before the September 11 atrocities, two Pakistani
nuclear experts visited Osama bin Laden in Afghanistan, apparently to
advise Al Qaeda on nuclear matters. One possibility is that if Pakistan
becomes more unstable intact weapons could fall into terrorist hands.
Still, the good news is that intercepted fissile material smugglers have
actually been carrying very small amounts. Less reassuringly, Potter did
note that prison sentences for smugglers dealing in weapons grade
nuclear material have been less than those meted out for drunk driving.

One cautionary case: Two groups invaded and seized the control room of
the Pelindaba nuclear facility in South Africa in November, 2007. They
were briefly arrested and then released without further consequence.
Both Ackerman and Potter agreed that it is in no state's interest to
supply terrorists with nuclear bombs or fissile material. It could be
easily traced back to them and they would suffer the consequences.
Ackerman cited one expert estimate that there is a 50 percent chance of
a nuclear terrorist attack in the next ten years.

While nuclear war and nuclear terrorism would be catastrophic, the
presenters acknowledged that neither constituted existential risks; that
is, a risk that they could cause the extinction of humanity. But the
next two risks, self-improving artificial intelligence and
nanotechnology, would.

The artificial intelligence explosion?

Singularity Institute for Artificial Intelligence research fellow
Eliezer Yudkowsky began his presentation with a diagram of the space of
possible minds. Among the vast space of possible minds, a small dot
represented human minds. His point is that two artificial intelligences
(AIs) could be far more different from one another than we are from
chimpanzees. Yudkowsky then described the relatively slow processing
speeds of human brains, the difficulty in reprogramming ourselves, and
other limitations. An AI could run 1 million times faster, meaning that
it could get a year's worth of thinking done in 31 seconds. An
"intelligence explosion" would result because an AI would have access to
its source code and could rapidly modify and optimize itself. It would
be hard to make an AI that didn't want to improve itself in order to
better achieve its goals.

Can an intelligence explosion be avoided? No. A unique feature of AI is
that it can be a "global catastrophic opportunity." Success in creating
a friendly AI would give humanity access to vast intelligence that could
be used to mitigate other risks. But picking a friendly AI out of the
space of all possible minds is a hard and unsolved problem. According to
Yudkowsky, the unique features of a superintelligent AI as a global
catastrophic risk are: There is no final battle, or an unfriendly AI
just kills off humanity. And there is nowhere to hide because the AI can
find you wherever you are. There is no learning curve since we get only
one chance to produce a friendly AI. But will it happen? Yudkowsky
pointed out that there is no way to control the proliferation of "raw
materials," e.g., computers, so the creation of an AI is essentially
inevitable. In fact, Yudkowsky believes that current computers are
sufficient to instantiate an AI, but researchers just don't know how to
do it yet.

What can we do? "You cannot throw money or regulations at this problem
for an easy solution," insisted Yudkowsky. His chief (and somewhat
self-serving) recommendation is to support a lot of mathematical
research on how to create a friendly AI. Of course, former Sun
Microsystems chief scientist Bill Joy proposed another solution:
relinquishment. That is, humanity has to agree to some kind of compact
to never try to build an AI. "Success mitigates lots of risks," said
Yudkowsky. "Failure kills you immediately." As a side note, bioethicist
James Hughes, head of the Institute for Ethics and Emerging
Technologies, mused about how much longer it would be before we would
see Sarah Connor Brigades gunning down AI researchers to prevent the
Terminator future. (Note to self: perhaps reconsider covering future
Singularity Institute conferences.)

The menace of molecular manufacturing?

Next up was Michael Treder and Chris Phoenix from the Center for
Responsible Nanotechnology. They cannily opened with a series of
quotations claiming that science will never be able to solve this or
that problem. Two of my favorites were: "Pasteur's theory of germs is a
ridiculous fiction" by Pierre Pachet in 1872, and "Space travel is utter
bilge," by Astronomer Royal Richard Woolley in 1956. Of course, the
point is that arguments that molecular manufacturing is impossible are
likely to suffer the same predictive failures. Their vision of molecular
manufacturing involves using trillions of very small machines to make
something larger. They envision desktop nanofactories into which people
feed simple raw inputs and get out nearly any product they desire. The
proliferation of such nanofactories would end scarcity forever. "We
can't expect to have only positive outcomes without mitigating negative
outcomes," cautioned Treder.

What kind of negative outcomes? Nanofactories could produce not only
hugely beneficial products such as water filters, solar cells, and
houses, but also weapons of any sort. Such nanofabricated weapons would
be vastly more powerful than today's. Since these weapons are so
powerful, there is a strong incentive for a first strike. In addition,
an age of nanotech abundance would eliminate the majority of jobs,
possibly leading to massive social disruptions. Social disruption
creates the opportunity for a charismatic personality to take hold.
"Nanotechnology could lead to some form of world dictatorship," said
Treder. "There is a global catastrophic risk that we could all be
enslaved."

On the other hand, individuals with access to nanofactories could wield
great destructive power. Phoenix and Treder's chief advice is more
research into how to handle nanotech when it becomes a reality in the
next couple of decades. In particular, Phoenix thinks that it's urgent
to study whether offense or defense would be the best response. To
Phoenix, offense looks a lot easier-there are a lot more ways to destroy
things than to defend them. If that's true, we should narrow our future
policy options.

This concluion left me musing on British historian Arnold Toynbee's
observation: "The human race's prospects of survival were considerably
better when we were defenseless against tigers than they are today when
we have become defenseless against ourselves." I don't think that's
right, but it's worth thinking about.

_______________________________________________
existential mailing list
existential at transhumanism.org
http://www.transhumanism.org/mailman/listinfo/existential

----- End forwarded message -----
-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE