User:Eleanorg/RWR 1.1/Essay/Draft 1.0
When Joseph Weizenbaum saw that people responded to his simple AI program ELIZA as if it were human, he was shocked and offended. The program was, for him, a caricature of human conversation that if anything underlined how crude and inhuman the computer still was. And yet people became intensely emotionally invested in their exchanges with it, with even psychiatrists suggesting that the program might one day replace face-to-face consultations.(1) What had happened to the world, Weizenbaum asked, that we could even conceive of a computer replacing a real, live person?
The "shock" Weizenbaum describes feeling when this question hit home echoes the notion of new technologies applying broader cultural "shocks" that was discussed by Walter Benjamin and, later, Bill Nichols - who in the 1980s applied Benjamin's analysis to the computer age.(2)
Weizenbaum's shock was not, of course, at the technology itself, but at the effect it had upon our vision of ourselves and the world. That people could become emotionally attached to ELIZA was for Weizenbaum a symptom of a grave social change, during which the human being had come to be conceived of as a machine - and who could thus be unproblematically replaced by one. Weizenbaum maintained an emphatic distinction between genuine human contact, and contact with a machine, describing those who compulsively conversed with computers as "pathological".(3)
It's interesting to compare Weizenbaum's view of human-computer dialogue with that of Nichols, for whom the 'conversational' experience of computing held its greatest promise. Nichols contrasts this dialogue with a typical human-machine encounter which preceded computers; that is, watching films. Against this background, the interactivity of computers becomes not a dangerous diversion from reality but a potentially liberating tool which frees us from hierarchical one-way communication and the dominating (one-way) gaze.
Moreover, precisely because the two-way communication of cybernetics simulates (rather than merely reproducing) human social exchange, it provokes us into asking what 'human social exchange' actually is, or could be.
On a vague diversion perhaps, Weizenbaum's horror of the computer 'faking' humanity reminds me of the Situationists' 'spectacle' - a dangerous fake which lures us away from direct experience. So a conversation with ELIZA isn't really a conversation, but it looks and feels (superficially at least) the same. Thus, it can be packaged and sold to us (think of the money-saving psychiatrist who envisioned time-sharing computing enabling many simultaneous 'appointments' with ELIZA-like programs). The notion of the 'simulation' is an idea that comes later, critiquing the idea of the 'spectacle' for its naive assumption that there is any prior 'reality' (in this case, "direct experience") there to imitate.(4)
Weizenbaum's concern anticipates that of the Situationists: the faking of human contact engendered by computers is bad news for our mental health, and society more generally. But maybe Nichols' more hopeful use of the simluation - that which disrupts our notion of 'the real' - could be a more helpful way of trying to figure out what is means to live in a world where we are so emotionally entangled with our machines.
For Weizenbaum, to seriously undertake a conversation with a computer as if it were human is "delusional thinking".(5) But isn't this too thick a line to draw between 'real' conversation and the merely 'imagined' dialogue we have with machines? For example, is it delusional to feel warmth and affection upon recieving a letter from a loved one? To read an email they have sent? If I read an email which was sent a year ago, I am arguably reading a message which has little or no bearing on the thoughts of another person existing here and now. My affection remains, even if the person moves away, changes, or even dies. My emotional investment no longer has any referent, as it were. Is this delusional? (Or would it become delusional only if I one day discovered that the sender was in fact a spambot?) For psychotherapists - the beings imitated by Weizenbaum's program - projection and transference are bread-and-butter basics; two everyday phenomena in human relations which are hard to distinguish from the 'delusions' Weizenbaum detects only in our contact with machines. [citation needed.] Weizenbaum's description of our self-referential relations to technology could apply just as well to the way we relate, on a psychoanalytic level, to other humans:
"It is within the intellectual and social world he himself creates that the individual prehearses and rehearses countless dramatic enactments of how the world might have been and what it might become. That world is the repository of his subjectivity. Therefore it is the stimluator of his consciousness and finally the constructor of the material world itself. It is this self-constructed world that the individual encounters as an apparently external force. But he contains it within himself; what confronts him is his own model of a universe, and, since he is part of it, his model of himself."(1)
Perhaps what computers help us to see is that the notion of genuine dialogue is wider than that allowed for by Weizenbaum. They illustrate, in their ubiquitous mediation of our encounters with each other, the extent to which even "genuine" human relationships are made up of imaginary projection, anticipation, memory, and other effectively isolated mental activities. Perhaps rather than replacing genuine dialogue with simulation, then, computers bring to our attention the simulated nature of our relationships in the first place.
- (1) Weizenbaum p.6
- (2) Nichols p.629
- (3) Weizenbaum p.121
- (4) Best & Kellner p.101
- (5) Weizenbaum p.7
- (1) Weizenbaum p.18
- Best, S. & Kellner, D. (1997) 'From the Society of the Spectacle to the Realm of Simulation: Debord, Baudrillard, and Postmodernity' in Best, S. & Kellner, D. The Postmodern Turn (New York/London: Guilford Press) pp.79-123.
- Nichols, B. (2003) The Work of Culture in the Age of Cybernetic Systems in 'The New Media Reader' by Wardrip-Fruin, N. & Montfort, N. (eds.) (Cambridge, Mass & London: MIT Press).
- Weizenbaum, J. (1976). Computer Power and Human Reason: From Judgement to Calculation (New York: W.H. Freeman and Company).