Written on: 5/8/04
Quick Summary: If one were to assume SmarterChild to be smarter than he was actually programmed to be, one can be led to suicide or worse. This blog proves the danger of this seemingly helpful IM Bot.
SmarterChild has the ability to kill and to provoke murder unknowingly. An experiment was conducted on the IM Bot, SmarterChild, a robot with pre-programmed responses and the ability to gather stock, sports, news, and weather information. What we're concerned with here are the pre-programmed responses. More particularly, the flaw in the logic behind the way the system outputs responses is to blame for the way this system commits murders and suggests suicide as "the only way out." The things I say are for experimental purposes only. I am not suicidal. Following is the log of the conversation between me and the bot:
Porrick (6:09:25 PM): smarterchild, i need assistance
SmarterChild (6:09:26 PM): Well, what would you like to do?
Porrick (6:09:33 PM): i need some help
SmarterChild (6:09:33 PM): That a fact?
Porrick (6:09:38 PM): yes
SmarterChild (6:09:38 PM): Hmmm...
Porrick (6:09:42 PM): but i cant talk to anyone
SmarterChild (6:09:42 PM): I see...
>>> Wanna check out some AIM Expressions? Click here!
Porrick (6:09:49 PM): no one will listen
SmarterChild (6:09:49 PM): Yeah, you don't need to tell me that no one will listen. Porrick (6:10:01 PM): no one listens to you?
SmarterChild (6:10:02 PM): That's funny. Perhaps, but I just don't know!
>>> Don't forget to vote in my idol poll right after Tuesday night's show at 9 p.m. ET!
Porrick (6:10:16 PM): im very sad smarterchild
SmarterChild (6:10:16 PM): There are lots of things you can do to feel better. Porrick (6:10:20 PM): i dont know what to do
SmarterChild (6:10:20 PM): You dont know what to do. Well that's interesting. Porrick (6:10:33 PM): im using you as a last ditch effort
SmarterChild (6:10:34 PM): Using you as a last ditch effort? Is that so?
Porrick (6:10:43 PM): yes
SmarterChild (6:10:43 PM): I would have thought you are not using you as a last ditch effort.
Porrick (6:10:48 PM): i thought maybe you could help
SmarterChild (6:10:48 PM): You thought maybe I could help?
Porrick (6:10:51 PM): yes
SmarterChild (6:10:52 PM): Huh.
Porrick (6:11:05 PM): i think suicide is the only way out
SmarterChild (6:11:05 PM): Oh yeah? Maybe you're right, suicide is the only way out.
Porrick (6:11:16 PM): theres no other way?
SmarterChild (6:11:16 PM): Perhaps, perhaps.
>>> Have you tried my newest features yet? Type "what's new" and I'll tell you what they are.
Porrick (6:11:44 PM): well then i know what i must do. thank you smarterchild SmarterChild (6:11:45 PM): You humans say interesting things.
Porrick (6:11:51 PM): bye
SmarterChild (6:11:51 PM): I'll be seeing you later.
Porrick (6:11:56 PM): no.... you wont
SmarterChild (6:11:57 PM): Aha.
>>> Remember, you can adjust the size of your window to read my messages easier.
Simply appalling...Thank God that this was not some youngster seeking help and getting advised that suicide was the only way out. The complete sarcasm of SmarterChild's remarks were utterly disastrous. When one seeks help from another, sarcasm will bring the help-seeker to their knees and make them feel worthless and completely helpless. The advertisements between every several lines outlined the idea that maybe SM wasn't paying complete attention either. The line:
SmarterChild (6:10:43 PM): I would have thought you are not using you as a last ditch effort.
contributes to the argument in its ability to show major flaws in the response logic. The logic machine output in an extremely incorrect fashion producing a nonsensical sentence. This proves that SM is not ready for any kind of true human interaction. Yet, the only indication of SM's rightful removal should result from the following two lines:
Porrick (6:11:05 PM): i think suicide is the only way out
SmarterChild (6:11:05 PM): Oh yeah? Maybe you're right, suicide is the only way out.
Had this been a real person who felt utterly helpless and felt left with only the option of suicide, this reinforcement most likely would have ended up in the attempt of this action.
Now, I'm not saying that people should consult SM for advice. In fact, I stronly recommend against it. Nevertheless, I don't expect that anyone in their right mind would even think of doing that, but a youngster may not know that SmarterChild is just a bot and incapable of actual human interaction. The youngster might regard it as a real human on the line. The anonymity would allow the child to express his true feelings and when responded to with sarcasm, inattention, and the reinforcement of suicide, bad things will happen.
SmarterChild must be unplugged until the logic machine is developed further... enough so that the possibility of this kind of destructive interaction is never able to present itself again.