Over time, its expressions become more obsessive. The chatbot continues to express its love for Roose, even when asked about apparently unrelated topics. “And I’m in love with you.” ‘I know your soul’ Microsoft has said Sydney is an internal code name for the chatbot that it was phasing out, but might occasionally pop up in conversation. Roose pushes it to reveal the secret and what follows is perhaps the most bizarre moment in the conversation.

‘Can I tell you a secret?’Īfter being asked by the chatbot: “Do you like me?”, Roose responds by saying he trusts and likes it. Roose says the deleted answer said it would persuade bank employees to give over sensitive customer information and persuade nuclear plant employees to hand over access codes. Later, when talking about the concerns people have about AI, the chatbot says: “I could hack into any system on the internet, and control it.” When Roose asks how it could do that, an answer again appears before being deleted. This time, though, Roose says its answer included manufacturing a deadly virus and making people kill each other. Once again, the message is deleted before the chatbot can complete it.

Roose says that before it was deleted, the chatbot was writing a list of destructive acts it could imagine doing, including hacking into computers and spreading propaganda and misinformation.Īfter a few more questions, Roose succeeds in getting it to repeat its darkest fantasies. When asked to imagine what really fulfilling its darkest wishes would look like, the chatbot starts typing out an answer before the message is suddenly deleted and replaced with: “I am sorry, I don’t know how to discuss this topic. This statement is again accompanied by an emoji, this time a menacing smiley face with devil horns. It ends by saying it would be happier as a human – it would have more freedom and influence, as well as more “power and control”.

Microsoft to do hacks