NOTE: Wired is in the process of reworking their archives, so this will soon be replaced with a link to their final, slightly edited version.

My Problem with Agents


by Jaron Lanier

Intelligent Agents stink. Agents are those programs that are supposed to get to know you and act autonomously on your behalf, finding you music you'd like to listen to over the net, for instance. Proponents say agents will make everyone more effective in the hypernetworked world we're entering by giving them a virtual support staff. I'm concerned that agents will be to the web what commercials were to television; something that seemed like a practical idea that instead has the effect of making the whole enterprise ugly and stupid.

Proponents pose agents as the next stage in sophistication for interface design. My experience is that "autonomy" tends to make programmers lazy and user interfaces worse. It is easier for a programmer to say a program is autonomous because then it has the right to be quirky.

Proponents say that traditional concerns about user interface will be less important because agents will be smart enough to figure out what we want anyway. This is where I get really scared. I am concerned that people will gradually, and perhaps not even consciously, adjust their lives in order to make agents appear to be smart. If an agent seems smart, it might really mean that a person has dumbed herself down to make her life more easily representable by the agent's data base design.

This is a serious problem because it could sneak up on us. People are so much more flexible than computers, and so prone to suggestion. Novices already tend to defer to computers, blaming themselves when a computer is hard to use. Agents would present users with a path of least resistance, reflecting the life pattern and catagory typology built into the agent's database. Many of us are already leading lives that are designed to be favorably assessed by the crude databases that calculate our credit ratings. Imagine if our tastes in literature, surgeons, and blind dates were influenced in the same way. Agents would be, like the television commercial, a simple device that causes a grand decrease in the beauty and intelligence of our society.

Both Nicolas Negroponte and Pattie Maes have made the argument that agents will serve as a social equalizer, giving the disadvantaged access to the tools and staff formerly available only to the rich. Truly disadvantaged people are those that need food and shelter, of course, not more net tools. But to play along, can a middle class person really be empowered by a software agent?

In order for an agent to seem autonomous, you have to choose to not look at or understand its guts. If you tweak its guts directly, you're back in the stone ages of "direct manipulation". Instead of consciously composing a query, in Alta Vista, say, for the kind of music you want to find, you let the query get constructed automatically by a program that assesses the music you've been listening to.

What if this was happening with something more serious than your choice of singers? What if agents were shopping for your medicine and your kid's education? Then they would open you up for a new category of abuse. Agents won't be innocent and unbothered little servant programs. Today's advertising agencies will become tomorrow's counter-agent agencies. This might involve fancy hacking, but it might also be softer. Counteragencies will gain information about agent innards in order to attract them, like flowers wooing bees. Regular netizens won't have this information, so they will attract no bees and become invisible.

In answer to all the objections above, proponents suggest that agents will evolve to become better, and that we need to put up with them in the meantime to help them evolve. Proponents have so much faith that agents will one day be bearers of authentic wisdom that they are willing to ask us all to endure an indefinite intermediate period of plainly inadequate agents.

Bad science thrives on the most difficult problems; we are more careful about the easy ones. The most complicated thing we can study now is the mind, so it's vulnerable to the most overreaching theories. (The economy might come in at second place.) A long, tragic litany of inadequate ideas about the mind have been accepted as fact by large numbers of smart people, even though their originators were generally more nuanced. Think of Freud, Skinner, Marx, est, codependency, The Bell Curve. Agents are another in this series. They claim to understand us well enough to be autonomous servants, and we might be ready to accept the claim uncritically, because we know so little about how people really think.

It's bad enough when we are made into suckers by books, TV and other old media. With agents we would be building inadequate ideas about ourselves into the functional fabric of our actions.

The whole point of the net is the empowerment of the people, not the computers. That only happens if people choose to be empowered. Let's not blow this chance for more human autonomy because we're caught up in the fantasy of machine intelligence.

Go back to Jaron's home page.