Giant brains; or, Machines that think

It is often easier for men to create a device than to guide it well afterwards: it is often easier for a scientist to study his science than to study the results for good or evil that his discoveries may lead to. But it is not right nor proper for a scientist, a man who is loyal to truth as an ideal, to have no regard for what his discoveries may lead to.

This principle is now being widely recognized. Many scientists today—both as individuals and as groups, and especially the atomic scientists—are considering the results of their scientific discoveries; and they are sharing in the effort to render those results truly useful to humanity.

It would be easy to leave out of this book any discussion of how machines that think may be controlled, any consideration of how they may be made truly useful to humanity. But that would be hardly right or proper. In concluding a book such as this one, that touches on many aspects of machines that think, we need to consider what can and should be done to make such machines of true benefit to all of humanity.

So, we come to the most important of all our questions: What sort of control over machines that think do we need in human society?

MACHINE THAT BOTH THINKS AND ACTS

From a narrow point of view, a machine that only thinks produces only information. It takes in information in one state, and it puts out information in another state. From this viewpoint, information in itself is harmless; it is just an arrangement of marks; and accordingly, a machine that thinks is harmless, and no control is necessary.

Although it is true that the information produced only becomes good or evil after other machinery or human beings act on the information, in reality a machine with the power to produce information is constructed only for the reason of its use. We want to know what such machines can tell us only because we can then proceed to act much more efficiently than before. For example, a guided missile needs a mechanical brain only because then it can reach its target. In all cases mechanical brains are inseparable from their uses.

For the purposes of this chapter, the narrow view will be rejected because it dodges the issue. We shall be much concerned with the combination of a machine that thinks with another machine that acts; and we shall often call this combination the robot machine.

READING THIS CHAPTER

Now, before launching further into the discussion, we need to say that the conclusions suggested in this chapter are not final. Even if they are expressed a little positively in places, they are nevertheless subject to change as more information is discovered and as the appraisal of information changes with time. Also, almost any conclusions about social control—including, certainly, the conclusions in this chapter—are subject to controversy. But controversy is good: it leads to thought. The more minds that go to work on solving the problem of social control over robot machines and other products of the new technology—which is rushing upon us from the discoveries of the scientists—the better off we all will be. If, while stimulating disagreement, the ideas expressed in this chapter should succeed in stimulating thought and deliberation, the purpose of this chapter will be well fulfilled.

Up to this point in this book, the emphasis has been on possibilities of benefits to humanity that may arise from machines that think. In this chapter, devoted as it is to the subject of control, the emphasis is on possibilities for harm. Both possibilities are valid, and the happening of either depends upon the actions of men. In much the same way, atomic energy is a great possibility for benefit and for harm. It is the nature of control to put a fence around danger; and so it is natural in this chapter that the weight of attention should shift to the dangerous aspects of machines that think.

Perhaps a reader may feel that a chapter of this kind is rather out of place in a book, such as this one, that seeks to be scientific. If so, he is reminded that, in accordance with the general suggestions for reading this book stated in the preface, he should omit this chapter.

FRANKENSTEIN

Perhaps the first study of the consequences of a machine that thinks is a prophetic novel called Frankenstein, written more than a hundred years ago, in 1818. The author, then only 21 years old, was Mary W. Godwin, who became the wife of the poet Percy Bysshe Shelley.

According to the story, a young Swiss, an ardent student of physiology and chemistry, Victor Frankenstein, finds the secret of life. He makes an extremely ugly, clever, and powerful monster, with human desires. Frankenstein promptly flees from his laboratory and handiwork. The monster, after seeking under great hardships for a year or two to earn fair treatment among men, finds himself continually attacked and harmed on account of his ugliness, and he becomes embittered. He begins to search for his creator for either revenge or a bargain. When they meet:

Frankenstein starts to comply with the main condition, which is to make a mate for the monster; but Frankenstein cannot bring himself to do it. So the monster causes the death one after another of all Frankenstein’s family and closest friends; and the tale finally ends with the death of Frankenstein and the disappearance of the monster.

As the dictionary says about Frankenstein, “The name has become a synonym for one destroyed by his own works.”

ROSSUM’S UNIVERSAL ROBOTS

Perhaps the next study of the consequences of a machine that thinks is a remarkable play called R.U.R. (for Rossum’s Universal Robots), first produced in Prague in 1921. Karel Čapek, the Czech dramatist who wrote it, was then only 31. The word “robot” comes from the Czech word “robota,” meaning compulsory service.

According to the play, Rossum the elder, a scientist, discovered a “method of organizing living matter” that was “more simple, flexible, and rapid” than the method used by nature. Rossum the younger, an engineer, founded a factory for the mass production of artificial workmen, robots. They had the form of human beings, intelligence, memory, and strength; but they were without feelings.

In the first act, the factory under Harry Domin, General Manager, is busy supplying robots to purchasers all over the world—for work, for fighting, for any purpose at all, to anyone who could pay for them. Domin declares:

In the second act, ten years later, it turns out that Domin and the others in charge of the factory have been making some robots with additional human characteristics, such as the capacity to feel pain. The newer types of robots, however, have united all the robots against man, for the robots declare that they are “more highly developed than man, stronger, and more intelligent, and man is their parasite.”

In the last act, the robots conquer and slay all men except one—an architect, Alquist, who in the epilogue provides a final quirk to the plot.

FACT AND FANCY

Now what is fact and what is fancy in these two warnings given to us a hundred years apart?

Of course, it is very doubtful that a Frankenstein monster or a Rossum robot will soon be constructed with nerves, flesh, and blood like an animal body. But we know that many types of robot machines can even now be constructed out of hardware—wheels, motors, wires, electronic tubes, etc. They can handle many kinds of information and are able to perform many kinds of actions, and they are stronger and swifter than man.

Of course, it is doubtful that the robot machines, by themselves and of their own “free will,” will be dangerous to human beings. But as soon as antisocial human beings have access to the controls over robot machines, the danger to society becomes great. We want to escape that danger.

Escape from Danger

A natural longing of many of us is to escape to an earlier, simpler life on this earth. Victor Frankenstein longed to undo the past. He said:

“Learn from me, if not by my precepts, at least by my example, how dangerous is the acquirement of knowledge, and how much happier that man is who believes his native town to be the world, than he who aspires to become greater than his nature will allow.”

Any sort of return to the past is, of course, impossible. It is doubtful that men could, even if they wanted to, stop the great flood of technical knowledge that science is now producing. We all must now face the fact that the kind of world we used to live in, even so recently as 1939, is gone. There now exist weapons and machines so powerful and dangerous in the wrong hands that in a day or two most of the people of the earth could be put to death. Giant brains are closely related to at least two of these weapons: scientists have already used mechanical brains for solving problems about atomic explosives and guided missiles. In addition, thinking mechanisms designed for the automatic control of gunfire were an important part of the winning of World War II. They will be a still more important part of the fighting of any future war.

Nor can we escape to another part of the earth which the new weapons will not reach. At 300 miles an hour, any spot on earth can be reached from any other in less than 48 hours. A modern plane exceeds this speed; a rocket or guided missile doubles or trebles it.

Nor can we trust that some kind of good luck will pull us through and help men to escape the consequences of what men do. Both Frankenstein and Domin reaped in full the consequences of what they did. The history of life on this earth that is recorded in the rocks is full of evidence of races of living things that have populated the earth for a time and then become extinct, such as the dinosaurs. In that long history, rarely does a race survive. In our own day, insects and fungi rather than men have shown fitness to survive and spread over the earth: witness the blight that destroyed the chestnut trees of North America, in spite of the best efforts of scientists to stop it.

There seems to be no kind of escape possible. It is necessary to grapple with the problem: How can we be safe against the threat of physical harm from robot machines?

UNEMPLOYMENT

The other chief threat from robot machines is against our economic life. Harry Domin, in R.U.R., you remember, prophesied: “All work will be done by living machines.” As an example, in the magazine Modern Industry for Feb. 15, 1947, appeared a picture of a machine for selling books, and under the picture were the words:

Another new product in robot salesmen—Latest in the parade of mechanical vending machines is this book salesman.... It is designed for use in hospitals, rail terminals, and stores. It offers 15 different titles, selected manually, and obtained by dropping quarter in slot. Cabinet stores 96 books.

Can you feel the breath of the robot salesman, workman, engineer,——, on the back of your neck?

At the moment when we combine automatic producing machinery and automatic controlling machinery, we get a vast saving in labor and a great increase in technological unemployment. In extreme cases, perhaps, the effect of robot machines will be the disappearance of men from a factory. Such a factory will be like a modern power plant that turns a waterfall into electricity: once the machinery is installed, only one watchman is ordinarily needed. But, in most cases, this will be the effect: in a great number of factories, mines, farms, etc., the labor force needed will be cut by a great proportion. The effect is not different in quality, because the new development is robot machinery; but the amount of technological unemployment coming from robot machines is likely to be considerably greater than previously.

The robot machine raises the two questions that hang like swords over a great many of us these days. The first one is for any employee: What shall I do when a robot machine renders worthless all the skill I have spent years in developing? The second question is for any businessman: How shall I sell what I make if half the people to whom I sell lose their jobs to robot machines?

SOCIAL CONTROL
AND ITS TWO SIDES

The two chief harmful effects upon humanity which are to be expected from robot machines are physical danger and unemployment. These are serious risks, and some degree of social control is needed to guard against them.

There will also be very great advantages from robot machines. The monster in Frankenstein is right when he says, “Do your duty towards me, and I will do mine towards you and the rest of mankind.” And Harry Domin in R.U.R. is right as to possibility when he says, “There will be no poverty.... Everybody will be free from worry.” Social control must also be concerned with how the advantages from robot machines are to be shared.

The problem of social control over men and their devices has always had two sides. The first side deals with what we might plan for control if men were reasonable and tolerant. This part of the problem seems relatively easy. The other side deals with what we must ordinarily arrange, since most men are often unreasonable and prejudiced and, as a result, often act in antisocial ways. This part of the problem is hard. Let us begin with the easier side first.

TYPES OF CONTROL—
IF MEN WERE REASONABLE

In seeking to fulfill wants and achieve safety, men have used hundreds of types of control. The main types are usually called political and economic systems, but there are always great quantities of exceptions. The more mature and freer the society, the greater the variety of types of control that can be found in it.

Probably the most widely used type of control in this country is private and public control working together, as private ownership and public regulation—for example, railroads, banks, airlines, life insurance companies, telephone systems, and many others. It would be reasonable to expect private ownership and public regulation of a great many classes of robot machines, to the end that they would never threaten the safety of people.

Another common type of control is public ownership and operation; examples are toll bridges, airports, city transit systems, and water-supply systems. Atomic energy was so clearly fraught with serious implications that in 1946 the Congress of the United States placed it entirely under public control expressed as the Atomic Energy Commission. There is a class of robot machinery which has already reached the stage of acute public concern: guided missiles and automatic fire-control. It would be reasonable that in this country all activity in this subdivision should be under close control by the Department of Defense.

In the international arena, again, the problem becomes soluble if we assume men to be reasonable. An international agency, such as an organ of the United Nations, would take over inspection and control of robot machine activities closely affecting the public safety anywhere in the world. Particularly, this agency would concern itself with guided missiles, robot pilots for planes, automatic gunfire control, etc. Much manufacturing skill is needed to make such products as these: the factories where they could be manufactured would thereby be determined. Also, a giant brain is a useful device for solving scientific problems about weapons of mass destruction. So the agency would need to inspect the problems being solved on such machines. This agency would be responsible to a legislature or an executive body representing all the people in the world—if men were reasonable.

In regard to the effects of robot machines on unemployment, again, if men were reasonable, the problem would be soluble. The problem is equivalent to the problem of abundance: how should men distribute the advantages of a vast increase in production among all the members of society in a fair and sensible way? A vast increase in production is not so impossible as it may seem. For example, in 1939, with 45 million employed, the United States index of industrial production was at 109, and, in 1943, with 52½ million employed, the index of production was at 239.

If men were reasonable, the net profits from robot machinery would be divided among (1) those who had most to do with devising the new machinery, and (2) all of society. A rule would be adopted (probably it could be less complicated than some existing tax rules) which would take into account various factors such as rewards to the inventors, incentives to continue inventing, adequate assistance to those made unemployed by the robot machines, reduction of prices to benefit consumers, and contributions to basic and applied scientific research.

In fact, under the assumption “if men were reasonable,” it would hardly be necessary to devote a chapter to the problem of social control over robot machines!

OBSTACLES

The discussion above of how robot machines could be controlled supposing that men were reasonable, seems, of course, to be glaringly impractical. Men are not reasonable on most occasions most of the time. If we stopped at this point, again we would be dodging the issue. What are the obstacles to reasonable control?

There are, it seems, two big obstacles and one smaller one to reasonable types of social control over robot machines. The smaller one is ignorance, and the two big obstacles are prejudice and a narrow point of view.

Ignorance

By ignorance we mean lack of knowledge and information. Now mechanical brains are a new and intricate subject. A great many people will, through no fault of their own, naturally remain uninformed about mechanical brains and robot machines for a long time. However, there is a widespread thirst for knowledge these days: witness in magazines, for example, the growth of the article and the decline of the essay. There is also a fairly steady surge of knowledge from the austere scientific fountain of new technology. We can thus see both a demand and a supply for information in such fields as mechanical brains and robot machines. We can expect, therefore, a fairly steady decline in ignorance.

Prejudice

Prejudice is a much more serious obstacle to reasonable control over robot machines. It will be worth our while to examine it at length.

Prejudice is frequent in human affairs. For example, in some countries, but not in all, there is conflict among men, based on their religious differences. Again, in other countries, but not in all, there is wide discrimination among men, based on the color of their skin. Over the whole world today, there is a sharp lack of understanding between conservatives, grading over to reactionaries, on the one hand, and liberals, grading over to radicals, on the other hand. All these differences are based on men’s attitudes, on strongly held sets of beliefs. These attitudes are not affected by “information”; the “information” is not believed. The attitudes are not subject to “judgment”; they come “before judgment”: they are prejudices. Even in the midst of all the science of today, prejudice is widespread. In Germany, from 1933 to 1939, we saw one of the most scientific of countries become one of the most prejudiced.

Prejudice is often difficult to detect. We find it hard to recognize even in ourselves. For a prejudice always seems, to the person who has it, the most natural attitude in the world. As we listen to other people, we are often uncertain how to separate information, guesses, humor, prejudice, etc. Circumstances compel us to accept provisionally quantities of statements just on other people’s say-so. A good test of a statement for prejudice, however, is to compare it with the scientific view.

Prejudice is most dangerous for society. Its more extreme manifestations are aggressive war, intolerance (especially of strange people and customs), violence, race hatred, etc. In the consuming hatred that a prejudiced man has towards the object of his prejudice, he is likely to destroy himself and destroy many more people besides. In former days, the handy weapon was a sword or a pistol; not too much damage could be done when one man ran amuck. But nowadays a single use of a single weapon has slain 70,000 people (the atom bomb dropped at Hiroshima), and so a great many people live anxious and afraid.

What is prejudice? How does it arise? How can it be cured, and thus removed from obstructing reasonable control over robot machines and the rest of today’s amazing scientific developments?

Prejudice is a disease of men’s minds. It is infectious. The cause and development of the disease are about as follows: Deprive someone of something he deeply needs, such as affection, food, or opportunity. In this way hurt him, make him resentful, hostile; but prevent him from expressing his resentments in a reasonable way, giving him instead false outlets, such as other people to hurt, myths to believe, hostile behavior patterns to imitate. He will then break out with prejudices as if they were measles. The process of curing the disease of prejudice is about as follows: Make friends with the patient; win his trust. Encourage him to pour out his half-forgotten hates. Help him to talk them over freely, by means of questions but not criticisms, until finally the patient achieves insight, sees through his former prejudices, and drops them.

In these days prejudice is a cardinal problem of society. It is perhaps conservative to say that a chief present requirement for the survival of human society—with the atom bomb, bacterial warfare, guided missiles, etc., near at hand—is cure of prejudice and its consequences, irrational and unrestrained hate.

Narrow Point of View

A narrow point of view regarding what is desirable or good is the third obstacle to rational control over robot machines. What do we mean by this?

Our point of view as a two-year-old is based on pure self-interest. If we see a toy, we grab it. There is no prejudice about this; it is entirely natural—for two-year-olds. As we grow older, our point of view concerning what is good or desirable rapidly broadens: we think of others and their advantage besides our own. For example, we may become interested in a conservation program to conserve birds, or soil, or forests, and our point of view expands, embraces these objectives, which become part of our personality and loyalties.

Unfortunately, it seems to be true that the expanding point of view, the expanding loyalties, of most people as they grow up are arrested somewhere along the line of: self, family, neighborhood, community, section of country, nation. An honorable exception is the scientists’ old and fine tradition of world-wide unity and loyalty in the search for objective truth.

Now the problem of rational control over robot machines and other parts of the new technology is no respecter of national boundaries. To be solved it requires a world-wide point of view, a loyalty to human society and its best interests, a social point of view.

Almost all that you and I have and do and think is the result of a long history of human society on this earth. All men on the earth today are descendants of other men who lived 1000, 2000, 3000 ... years ago, whether they were Romans or Chinese or Babylonians or Mayas or members of any other race. To ride in a subway or an airplane, to talk on the telephone, to speak a language, to calculate, to survive smallpox or the black death, etc.—all these privileges are our inheritance from countless thousands of other human beings, of many countries, and nearly all of whom are now dead. During our lives we pass on to our own children an inheritance in which our own contribution is remarkably small. Since each person is the child of two others, the number of our forefathers is huge, and we are all undoubtedly blood cousins. Because of this relationship, and because we owe to the rest of society nearly all that we are, we have a social responsibility—we need to hold a social point of view. Each of us needs to accept and welcome a world-wide social responsibility, as a member of human society, as a beneficiary and trustee of our human inheritance. Otherwise we are drones, part of the hive without earning our keep. The social point of view is equitable, it is inspiring, and it is probably required now in order for human beings to survive. We need to let go of a narrow point of view.

CONCLUSION

We have now outlined the problem of social control over robot machines, supposing that human beings were reasonable. We have also discussed the practical obstacles that obstruct reasonable control.

It is not easy to think of any yet organized group of people anywhere that would have both the strength and the vision needed to solve this problem through its own efforts. For example, a part of the United Nations might have some of the vision needed, but it does not have the power. Consequently, it is necessary and desirable for individuals and groups everywhere to take upon themselves an added load of social responsibility—just as they tend to do in time of war. People often “want to do their share.” Through encouragement and education, the basic attitude of a number of people can contain more of “This is our business; we have a responsibility for helping to solve this problem.” We also need public responsibility; we need a public body responsible for study, education, advice, and some measure of control. It might be something like an Atomic Energy Commission, Bacterial Defense Commission, Mental Health Commission, and Robot Machine Commission, all rolled into one.

When, at last, there is an effective guarantee of the two elements physical safety and adequate employment, then at last we shall all be free from the threat of the robot machine. We can then welcome the robot machine as our deliverer from the long hard chores of many centuries.