[info] Designing something smarter than yourself [WAS Re: [agi] Low I.Q. AGI]

[Eugen Leitl]

----- Forwarded message from Richard Loosemore <rpwl at lightlink.com> -----

From: Richard Loosemore <rpwl at lightlink.com>
Date: Mon, 16 Apr 2007 13:26:47 -0400
To: agi at v2.listbox.com
Subject: Designing something smarter than yourself  [WAS Re: [agi] Low I.Q.
 AGI]
User-Agent: Thunderbird 1.5.0.9 (Windows/20061207)
Reply-To: agi at v2.listbox.com

Eric Baum wrote:
>Richard> efforts (some people seem to think that there is something
>Richard> inherently impossible about a human being able to design
>Richard> something smarter than itself, but that idea is really just
>Richard> science-fiction hearsay, not grounded in any real
>Richard> limitations).
>
>Well, no it is grounded in real limitations. I doubt, Richard, that
>even you think you could "design" a human level intelligence by hand,
>any more than you could personally design a mirage jet, the blueprints
>for which filled a warehouse. At the very least you would want to use
>a computer, and write code for the computer, and have the computer do
>a lot of the design for you by running the code. At the end of that
>process, you wouldn't necessarily "understand" much about how that
>design worked. And if the very guts of the reason that design worked
>are because it contains programs that were output by finding
>approximate solutions to computationally intractable problems,
>you'd be in real trouble.

A bit of confusion going on here, I think.

I was not talking about a human 'understanding' the design of something 
smarter than a human -- that point is being debated in parallel, and is 
quite different from what I said.

I was only talking about the pop-science idea that a human, because it 
has a certain level of intelligence, could never in principle design 
something that could then become smarter than the human.  It's a (false) 
generalization of the idea that you cannot pull yourself up by your own 
bootstraps.

Only a small point:  I don't think you would agree with the position I 
was trying to oppose, there.



But meanwhile, about the parallel question of whether a human could 
*understand* a human-level intelligence.  The points you make above 
could be applied to an "aircraft designer".  Such a person could design 
a new aircraft perfectly well... in a certain sense.  They would not be 
qualified to design, say, all the details of the inflight entertainment 
system, down to every last transistor in the amplifier of the sound 
system -- but then, we wouldn't say "Ha!  You don't really know how to 
design an aircraft!"



Richard



-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=1391210&user_secret=bd910682

----- End forwarded message -----
-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE