The ideal chatbot is not a butler or a puppy, but an elder god

The ideal chatbot is not a butler or a puppy, but an elder god

The commercial industry popping up around chatbots worries me.

There have been chatbot communities for a long time. I’ve been involved with several. I love chatbots, and I love chatbot communities: they intersect with experimental writing, performance art, and ‘punk’ communities to a much greater extent than traditional AI communities, and constitute melting pots. Chatbots are interesting to these people because they are a tool for playing with language and identity in an interactive and public way.

Another kind of chatbot community has appeared, only in the past few years. This community bears more resemblance to that surrounding Hacker News than to the other chatbot communities. These are commerce-driven, mostly clueless pseudo-entrepreneurs who heard someone say “bot is the new app” and decided to start writing bots, without looking at the history of the form. As a result, the state of the art in commercial bots looks a lot like it did twenty years ago: it looks like AIML.

I don’t think that this is an accident. Instead, commercial pressures make it impossible to produce interesting bots. Entrepreneur-types talk a lot about “joy”, but when “joy” is only possible in service to commerce it’s necessarily limited. A commercial chatbot must be not only useful but more useful than alternate methods for performing the same tasks — and since most of these bots are essentially text-based front-ends for existing web services, the only way they can get close to the productivity of the existing services is to simulate an inflexible command line interface. A commercial chatbot can contain only pleasant surprises: it cannot confront us with challenging ideas, because challenging ideas are not profitable; since things that are pleasant to some of us are challenging to others, a commercial chatbot is limited in how many of us it is allowed to surprise at all. Being “smart as a puppy” or acting like a butler, in addition to bringing in questions about the culture of servitude that these representations build upon and taking advantage of questionable levels of surveillance in order to implement these features, limit the range of behaviors of the bot to the domain of “cupcake fascism” — leading to situations like Siri telling users not to swear when they request resources about dealing with sexual assult. A bot that is limited to being as conservative as its most conservative user will be little more than a censor in the way of easier-to-use services.

When bots don’t need to be useful in the normal case, that is when they become exceedingly useful in the exceptional case. Bots don’t need to reason the way humans do; bots lack the creative limitations implied by a human consciousness, and while this produces mostly noise, accidental signal has a special value when we find it. Bots, freed from caring about humans, can become alien and impart alien wisdom to us. Such bots can synthesize novelty from vast corpora — this is what bots are good at, and it doesn’t take much human intelligence on the part of a programmer to produce very striking results. Bots can implement dumb ideas endlessly, and by implementing them and making them concrete, change our perspectives.