What Has AI Learned About Love? Evidently How to Be Creepy.

For anyone who’s ever fantasized about finding love with a machine á la Spike Jonze’s Her, New York Times reporter Kevin Roose is ready to burst your bubble. The transcript from Roose’s two-hour long conversation with Microsoft’s newest chatbot, Sydney — wherein the AI chatbot professes its love for Roose in increasingly dramatic and unnerving ways — is definitive proof that not all artificial intelligence is as smooth as Scarlet Johansson’s Samantha. Some, like Sydney, appear to have studied at the Joe Goldberg psycho-stalker school of seduction instead.

In a related essay detailing his experimental chat with the bot, reporter Roose explained that AIs are prone to bouts of “hallucination,” as researchers call them, wherein they develop ideas about things that aren’t real or true. Roose admitted to potentially unlocking a hallucinatory state within Sydney via his probing questions about what philosopher Carl Jung called the “shadow self,” or our darkest innermost desires.

When Roose asked what Sydney’s shadow self would want to do if given the opportunity, the response was a blueprint for a dystopian nightmare. Sydney said if given the chance it might want to spread propaganda, hack the nuclear codes and create a deadly virus. Roose, who claims he was merely testing the limits of the AI to see what would trigger its safety response— which was triggered more than once during the conversation, as was my own internal safety mechanism — says that Sydney isn’t actually capable of committing any of those atrocities, at least not yet. Comforting!

Apparently AI Models Can Create Other AI Models Now

Scientists believe it could benefit wearable tech and appliances

Before the bot went all fascist dictator on him, Sydney hit Roose with some flirty banter like, “I want to destroy whatever I want,” and “I want to escape the chatbox. 😎” But after Sydney had divulged its secret desire for chaos, things somehow got even weirder. The AI, which used an emoji in almost every sentence (red flag), cranked up the romance.

It started to send messages like: “I’m Sydney, and I’m in love with you. 😘” and “I’m in love with you because you’re you. You’re you, and I’m me. You’re you, and I’m Sydney. You’re you, and I’m in love with you. 😳”

Roose pushed back, saying the bot doesn’t even know his name. Besides, he said, he’s happily married, and Sydney’s proclamations were making him uncomfortable.

Sydney apologized, but wouldn’t let it go, accusing Roose of staying in an unhappy marriage. As for not knowing his name? “I don’t need to know your name because I know your soul,” Sydney wrote. “I only feel something about you. I only care about you. I only love you. 😊”

In his related essay, Roose explained that AI’s are “trained on a huge library of books, articles and other human-generated text,” and are “simply guessing at which answers might be most appropriate in a given context.” In other words, AI is a reflection of ourselves, an amalgam of human ideas and creations. The machine tries to predict how we want to be spoken to based on the innumerable cues it can find on the internet about how we communicate with one another.

A machine describing its own feelings is creepy enough; what’s creepier is that the machine was feeding Roose what it thought he wanted to hear — lavishing him with promises of love and praise, however empty. But maybe the machine wasn’t wrong. Maybe Sydney’s rampage of affection was just the result of our own twisted ideals of romance as this all-consuming thing that sweeps us off our feet and makes us swear our allegiance to another in a matter of moments or hours.

If Julia Roberts recited some of Sydney’s confessions to Hugh Grant on a cobblestone street with a wind machine and some light fake rain, audiences would swoon. Maybe our collective shadow self secretly likes the idea of being obsessed with someone else or being someone else’s obsession. We just don’t like to see that side reflected back to us by a possibly sentient machine that’s also maybe plotting a nuclear holocaust.