Chapter-04
"Attack Of The Assemblers"

Bill Joy has the kind of résumé that would get the attention of Benjamin Franklin’s headhunter. Cofounder and, until early 2004, chief scientist of Sun Microsystems; cochair of the Presidential Commission on the Future of IT Research; coauthor of the Java language specification; and creator of the Jini pervasive computing technology—Joy is a renowned thinker about the effects of technology upon people. I mention all this so that you’ll keep in mind that the following notions do not come from some space shot.
In April 2000, Bill Joy published a much-noted article entitled, “Why the Future Doesn’t Need Us.” The subhead to the article warned, “Our most powerful 21st-century technologies—robotics, genetic engineering, and nanotech—are threatening to make humans an endangered species.”

The article made a big impact because of its frightening premise: there may be no place for our species in a future that is dominated by our creations. Most notable of those creations will be something called an “assembler,” a device that springs from the intersection of nanotechnology, biotechnology, and information technology. The article cited other works with similar messages. All of them reflect an understanding that if we create things that have the capacity to rule us, then we will let them rule us.
Could that happen?

Nobody has come up with a good argument to suggest that it can’t.
Then again, it implies that human beings will voluntarily hand over their prerogatives to their creations. What sort of mentality accepts such an inevitability?
In fact, that mentality is commonplace among Internet technologists. It comes from an assumption underlying the writings of Joy and others that must be challenged. It’s the fundamental assumption of something I call the “open Internet mindset.”

The assumption goes like this: since the information highway is essential to the deployment of new developments and is the essential information and communication medium of the future, and since activity on that highway is ungovernable, then everything to which the highway connects is beyond the reach of governance.

That assumption is wholly without basis. The Internet is governable, as any highway is governable. Standards bodies decide what top-level domains and transport protocols may be used, just as the highway departments of municipalities, provinces, and nations decide upon traffic signals, signage, and vehicle registration standards. As long as you are not carrying hazardous cargo, it is not the highway department’s business what you use the highway for. But obviously, governments do care if you are using a highway to transport illegal drugs. The highway department or the department of motor vehicles may not care, but the law enforcement branches of government care very much and will make it their business to stop you.
And other authorities concern themselves with stopping noncriminal activities. If the highway takes you to a meeting where you are about to disclose company secrets to a competitor, the highway department will not care nor will the statutory government; but those who govern your company will care a lot. They will take steps to prevent the trip if they know about it. If necessary, they will appeal to judicial authorities (i.e., the statutory government) to issue an injunction to prevent the trip.

The highway system called the Internet is indeed open; it is owned by no one—just as the world’s physical highway system is owned by no one. Even if you own equipment and communication lines that transport Internet traffic, you do not own equity in the Internet any more than ownership of the roadways in your office park gives you ownership interest in the world’s system of highways.
Given the usefulness of the highway metaphor, let’s consider a couple of things about the way highways work:
The openness of the highway does not in the least change our right to govern activity that may involve that highway.

The openness of the highway does not prevent our using it for transport to spaces that are not so open.
The governance of those not-so-open spaces and the governance of activity that takes place on and off highways is not the business of the highway department, except as it affects the operation of the highway itself.

Many companies have their own networks that are built on top of the public Internet but at the same time are apart from it. The information and communication spaces they provide are not open to the rest of the Internet. Those networks are obviously owned by the companies that built them. They are bounded spaces—buildings, if you will—that are used for private communication among employees, suppliers, distributors, and whomever else the company invites in.

Such bounded, manageable networks are not now provided to affinity groups among Internet users. Instead, the Internet offers “communities” that present themselves as gathering points for people with common interests. But such spaces are no more bounded than the Internet itself—offering, in effect, roadside hangouts where anyone with time on their hands may drop in, hang out with others, and adopt any identity that suits their fancy. Is it any wonder that people are reluctant to communicate anything of substance in those spaces?
We will go into more detail about the construction of bounded spaces in Parts 3 and 4.

Mere Jelly

As Bill Joy sounds the alarm about our creations taking over, a truly scary book by Hans Moravec openly celebrates the possibility. Moravec believes that if we manage to get all the information from a person’s central nervous system into software and files, then the software and files are a complete substitute for the person. What is left behind is a useless carcass or, in Moravec‘s truly memorable expression, “mere jelly.”
Moravec is a leading researcher in the field of robotics. But his vision of robots of the future is far removed from the quaint R2-D2 kind of image most of us associate with robots:


Some of us humans have quite egocentric world views. We anticipate the discovery, within our lifetimes, of methods to extend human life, and we look forward to a few eons of exploring the universe. The thought of being grandly upstaged by our artificial progeny is disappointing. Long life loses much of its point if we are fated to spend it staring stupidly at our ultra-intelligent machines as they try to describe their ever more spectacular discoveries in baby-talk that we can understand. We want to become full, unfettered players in this new superintelligent game. What are the possibilities for doing that?
Genetic engineering may seem an easy option. Successive generations of human beings could be designed by mathematics, computer simulations, and experimentation, like airplanes, computers, and robots are now. They could have better brains and improved metabolisms that would allow them to live comfortably in space. But, presumably, they would still be made of protein, and their brains would be made of neurons. Away from earth, protein is not an ideal material. It is stable only in a narrow temperature and pressure range, is very sensitive to radiation, and rules out many construction techniques and components. And it is unlikely that neurons, which can now switch less than a thousand times per second, will ever be boosted to the billions-per-second speed of even today’s computer components. Before long, conventional technologies, miniaturized down to the atomic scale, and biotechnology, its molecular interactions understood in detailed mechanical terms, will have merged into a seamless array of techniques encompassing all materials, sizes, and complexities. Robots will then be made of a mix of fabulous substances, including, where appropriate, living biological materials. At that time a genetically engineered superhuman would be just a second-rate kind of robot, designed under the handicap that its construction can only be by DNA-guided protein synthesis. Only in the eyes of human chauvinists would it have an advantage—because it retains more of the original human limitations than other roots.

Robots, first or second rate, leave our question unanswered. Is there any chance that we—you and I, personally—can fully share in the magical world to come? This would call for a process that endows an individual with all the advantages of the machines, without loss of personal identity. Many people today are alive because of a growing arsenal of artificial organs and other body parts. In time, especially as robotic techniques improve, such replacement parts will be better than any originals. So what about replacing everything, that is, transplanting a human brain into a specially designed robot body? Unfortunately, while this solution might overcome most of our physical limitations, it would leave untouched our biggest handicap, the limited and fixed intelligence of the human brain. This transplant scenario gets our brain out of our body. Is there a way to get our mind out of our brain?

You’ve just been wheeled into the operating room. A robot brain surgeon is in attendance. By your side is a computer waiting to become a human equivalent, lacking only a program to run. Your skull, but not your brain, is anaesthetized. You are fully conscious. The robot surgeon opens your brain case and places a hand on the brain’s surface. This unusual hand bristles with microscopic machinery, and a cable connects it to the mobile computer at your side. Instruments in the hand scan the first few millimeters of brain surface. High-resolution magnetic resonance measurements build a three-dimensional chemical map, while arrays of magnetic and electric antennas collect signals that are rapidly unraveled to reveal, moment to moment, the pulses flashing among the neurons . . .


 . . . to further assure you of the simulation’s correctness, you are given a pushbutton that allows you to momentarily “test drive” the simulation, to compare it with the functioning of the original tissue . . .


 . . . As soon as you are satisfied, the simulation connection is established permanently. The brain tissue is now impotent—it receives inputs and reacts as before but its output is ignored. Microscopic manipulators on the hand’s surface excise the cells in this superfluous tissue and pass them to an aspirator, where they are drawn away.


The surgeon’s hand sinks a fraction of a millimeter deeper into your brain, instantly compensating its measurements and signals for the changed position. The process is repeated for the next layer . . . Layer after layer the brain is simulated, then excavated. Eventually your skull is empty, and the surgeon’s hand rests deep in your brainstem. Though you have not lost consciousness, or even your train of thought, your mind has been removed from the brain and transferred to a machine. In a final, disorienting step the surgeon lifts out his hand. Your suddenly abandoned body goes into spasms and dies. For a moment you experience only quiet and dark. Then, once again, you can open your eyes. Your perspective has shifted. The computer simulation has been disconnected from the cable leading to the surgeon’s hand and reconnected to a shiny new body of the style, color, and material of your choice. Your metamorphosis is complete.

Moravec then describes less invasive ways to do the same thing, “for the squeamish.” The result is still the replacement of your body—“mere jelly”—with a robot of “your” choice. (“Your” is in quotes, because the pronoun has just become ambiguous.)

Mind Children
was recommended to me by my Delphi colleague Kip Bryan, as we were implementing a means of providing artificial opponents for players of Delphi‘s games when no human opponent was available or desired. The idea had come from a legendary MIT computer program called Eliza, which simulated a psychotherapist—you would tell Eliza something, and “she” would ask you a question in the context of your comment.

The question of disclosure had to be dealt with: how do we ensure that the Delphi game player knows that his or her opponent is not a human being? I wanted to make it clear but humorous rather than pedantic—avoiding the style of those idiotic warnings that were starting to appear on wine bottles. We thought we had accomplished that, but then a competitor—General Electric’s GEnie online service—started “revealing” to the market of online users that Delphi was conning them with fake game players. Our reaction: Oh please, is anyone so naïve that they can’t tell? Answer: Yes indeed, there were a few. Perhaps there were many more too embarrassed to admit they’d been fooled!

Effectively we had created robots that were participating in human society. When Kip Bryan suggested reading the Moravec book and thinking about the larger implications, I was thoroughly amused. I got a copy of the book not so much to humor him as to humor myself with some off-the-wall science fiction. Kip’s concerns seemed to me to be in the same category as those of a compulsive conspiracy theorist.

In the intervening decade and a half, however, I have come to see that Kip’s concerns were valid. What is more alarming than the scenarios offered by Bill Joy and Hans Moravec is the belief that at every step of the way we must yield our prerogatives to anything that seems to be an advancement in intelligence.
What is it that makes intelligence the highest ideal of our age? Which of the following intelligent minds is closest to the ideal of the intelligence supremacists:

Josef Goebbels
Slobodan Milosevic
Dennis Kozlowski
Ivan Boesky
Joseph Stalin
Saddam Hussein
Pol Pot
Osama bin Laden

Is this what we’re after, the pursuit of super intelligence to the exclusion of all other values? Is that really what will advance humanity toward Utopia 0.6?

If my children had a choice between living a fulfilling and responsible life and graduating from MIT at age 16, I would obviously encourage them to seek the former. Wouldn’t you? I hope so, as long as we both inhabit the same planet. The position advocated here is that intelligence is a tool for implementation of something that’s essentially a matter of arbitrary choice: the desire to improve the lives of everybody by providing a means for encouraging people to be more responsible to one another and to the world.

It’s ridiculous to live 100 years and only be able to remember 30 million bytes. You know less than a compact disc. The human condition is really becoming more obsolete every minute.
Marvin Minsky

I am fortunate in having had to deal with real artificial intelligence early, in the encounter with game-bots. The real artificial intelligence question isn’t about applying some neural network technique to solving a problem, it’s about software participating in society. Soon it will become a real issue. It is essentially ideological and political; there is no “correct” answer to the question of whether a robot or program with superior intelligence should take over the prerogatives of humans. If you believe that an object with a superior ability to process incoming signals and act on them quickly in a manner that suggests intelligence should always assume control over slower carbon-based objects, then for you the Internet is as it should be. Human identities shouldn’t get in the way of the progress of digital objects. Without such encumbrances the most intelligent objects on the Net will gain control, and any human casualties along the way are of not much consequence as the new order is built. The new collection of intelligent objects may coalesce into one big global or intergalactic organism or, who knows, they may form nations that go to war with each other. The outcome will be of no consequence to us. If we humans are permitted to live as flesh-and-blood physical specimens, it will be in zoos or alongside the squirrels in places like Colonial Williamsburg, where robots can take their children to see life as it used to be, complete with the now endangered human species.
If one who favors members of his own race is a racist, then is one who favors his own species a speciesist? If so—forgive me, but I am a speciesist. And I hope that the speciesists will always prevail.
Call me a human supremacist.

As a human supremacist, I want and need the digital identity tools that will allow me and those I care about to assert our humanness over the various non-human objects found in networks. For our children’s sake, I hope you agree with me.

Wired, April 2000.

Hans Moravec, Mind Children (Cambridge: Harvard University Press, 1988).