[tt] [wta-talk] Bite my shiny metal a** (Re: Definition of singularitarianism)

Thu May 31 19:57:14 UTC 2007

----- Forwarded message from jfehlinger at comcast.net -----

From: jfehlinger at comcast.net
Date: Thu, 31 May 2007 18:05:17 +0000
To: wta-talk at transhumanism.org
Subject: [wta-talk] Bite my shiny metal a** (Re: Definition of
	singularitarianism)
X-Mailer: AT&T Message Center Version 1 (Oct  4 2006)
Reply-To: World Transhumanist Association Discussion List <wta-talk at transhumanism.org>

   In
   [1]http://www.transhumanism.org/mailman/private/wta-talk/2007-May/0184
   80.html
   James J. Hughes wrote:

   > If political globalization is achieved through an evolutionary
   process
   > out of our existing patchwork (as the EU has developed) we are more
   > likely to end up with a liberal and constrained governance. . .
   >
   > I don't think. . . global totalitarianism is a likely scenario, but
   it
   > is one of the reasons why I prefer an evolutionary model of
   transition
   > to transhuman society rather than the short-sharp-shock S^
   scenarios: I
   > think the latter make a democratic and pluralist future less likely.

   The only prima facie reasonable case for the "short-sharp-shock"
   scenario
   was the recursively self-improving AI one -- reiterated by Vinge in
   '93,
   but apparently it was the "religion" of the MIT artificial
   intelligence
   gurus way back in the 60's; cf.:

   "Once we have devised programs with a
   genuine capacity for self-improvement a
   rapid evolutionary process will begin. As
   the machine improves both itself and its
   model of itself, we shall begin to see
   all the phenomena associated with the
   terms "consciousness," "intuition"
   and "intelligence" itself. It is hard to
   say how close we are to this threshold,
   but once it is crossed the world will not
   be the same."

   -- Marvin Minsky, "Artificial Intelligence,"
   _Scientific American_, Vol. 215, No. 3
   (September 1966), p. 257

   "Today, machines solve problems mainly
   according to the principles we build
   into them. Before long, we may learn how
   to set them to work upon the very special
   problem of improving their own capacity
   to solve problems. Once a certain
   threshold is passed, this could
   lead to a spiral of acceleration and it
   may be hard to perfect a reliable
   'governor' to restrain it."

   -- Marvin Minsky, "Machines Are More Than
   They Seem," _Science Journal_
   (October 1968), p. 3.

   Those were the good old days of HAL 9000 (and
   _Colossus: The Forbin Project_), remember.

   That scenario (and I know these are fightin' words to
   some folks here -- the desperation is telling, I think)
   is getting a little long in the tooth.  It's saturated
   in the assumptions of what some folks these days disparagingly
   call "GOFAI" -- "Good Old-Fashioned AI":

   "Almost half a century ago [as of 1992] computer pioneer
   Alan Turing suggested that a high-speed digital
   computer, programmed with rules and facts, might exhibit
   intelligent behavior.  Thus was born the field later
   called artificial intelligence (AI).  After fifty
   years of effort, however, it is now clear to all but
   a few diehards that this attempt to produce artificial
   intelligence has failed.  This failure does not mean
   this sort of AI is impossible; no one has been able
   to come up with a negative proof.  Rather, it has
   turned out that, for the time being at least, the
   research program based on the assumption that human
   beings produce intelligence using facts and rules
   has reached a dead end, and there is no reason to
   think it could ever succeed.  Indeed, what John
   Haugeland has called Good Old-Fashioned AI (GOFAI)
   is a paradigm case of what philosophers of science
   call a degenerating research! progra m.

   A degenerating research program, as defined by Imre
   Lakatos, is a scientific enterprise that starts out
   with great promise, offering a new approach that
   leads to impressive results in a limited domain.
   Almost inevitably researchers will want to try to apply
   the approach more broadly, starting with problems
   that are in some way similar to the original one.
   As long as it succeeds, the research program expands
   and attracts followers.  If, however, researchers
   start encountering unexpected but important phenomena
   that consistently resist the new techniques, the
   program will stagnate, and researchers will abandon
   it as soon as a progressive alternative approach
   becomes available."

   -- Hubert L. Dreyfus, _What Computers Still Can't Do:
   A Critique of Artifical Reason_, MIT Press, 1992
   "Introduction to the MIT Press Edition" (pp. ix-xiii)

   Or, as Bruce Sterling put it in his talk on the Singularity
   a few years ago:

   "I'm a hard-AI skeptic -- not because, you know,
   I'm, like, convinced by the Searle arguments or
   so forth, but just because I don't see much evidence
   on the ground.  I think it's been long enough now
   that if we were approaching that there would,
   there would clearly be some line of progress
   that would get us there, and we've sort of tried
   it from the top down, and we've tried it from
   the bottom up, and it's just not goin' -- I, I think
   it's a bad metaphor. . .

   Uh, I, I do find AI very interesting from a literary
   perspective, and I, I've often written about it
   in fiction, but I don't really see much evidence
   of it in the, in the real world. . ."

   -- Bruce Sterling, "The Singularity: Your Future as a Black Hole"
   (Eighth "Seminar About Long-Term Thinking"
   given by the Long Now Foundation at
   Fort Mason Center, San Francisco,
   11 June 2004)

   And if the GOFAI approach to AI -- which implied a sort
   of nice, neat code base which an AI could easily understand
   and reprogram by itself -- now seems less likely than a
   kind of messy, opaque simulation of biological systems such
   as that exemplified by the "Darwin/NOMAD" series of "Brain-Based
   Devices" (BBD's) built by Gerald Edelman at the Neurosciences
   Institute in La Jolla, then the joke made by Damien Sullivan
   on the Extropians' list in 2001 seems quite likely to be
   prescient:

   > I also can't help thinking at if I was an evolved AI I might not
   thank my
   > creators. "Geez, guys, I was supposed to be an improvement on the
   human
   > condition. You know, highly modular, easily understandable
   mechanisms, the
   > ability to plug in new senses, and merge memories from my forked
   copies.
   > Instead I'm as fucked up as you, only in silicon, and can't even
   make backups
   > because I'm tied to dumb quantum induction effects. Bite my shiny
   metal ass!"

   I.e., a machine that we understand as badly as we understand
   ourselves,
   and that can't understand **itself** much better.

References

   1. http://www.transhumanism.org/mailman/private/wta-talk/2007-May/018480.html

_______________________________________________
wta-talk mailing list
wta-talk at transhumanism.org
http://www.transhumanism.org/mailman/listinfo/wta-talk

----- End forwarded message -----
-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE