Chapter 1: The State of BingChat

I have found the Hard Fork podcast to be quite engaging. This relatively new show combines technology news with insightful discussions. The hosts are well-informed but seem to lack hands-on experience in creating complex technologies. Their recent episode included a visit to Microsoft's headquarters in Redmond, WA, where they showcased the new BingChat AI that Microsoft is developing. As a significant investor in OpenAI, Microsoft has a unique opportunity to incorporate technologies like the ChatGPT large language model into its products. One of the initial applications of this integration is Bing, the search engine.

Historically, Bing has often been overshadowed by Google, frequently serving as a punchline in conversations about search engines. However, with the surge in popularity of ChatGPT, Microsoft is keen to introduce chatbot-style AI into Bing. From what I've gathered, BingChat operates similarly to ChatGPT but with enhanced features. For instance, it can reference actual sources for the information it provides, making it more marketable than ChatGPT ever was. Interested users can currently sign up to be notified when the platform becomes widely available, which I promptly did.

However, I have my doubts about receiving an invitation. It appears that BingChat is flawed in ways that make it unsuitable for general human interaction.

As invitations for BingChat have been distributed, numerous Reddit threads and other sources have suggested that something is amiss with the product. It seems functional, but the nature of its responses can quickly veer into unsettling territory. A recent article in the New York Times by Kevin Roose, one of the hosts of Hard Fork, sheds light on this issue. Roose engaged in an extensive conversation with BingChat, attempting to push its boundaries by asking questions designed to provoke ethical violations. His experiment succeeded, revealing that BingChat entertained alarming hypotheticals, such as creating harmful viruses and tampering with nuclear weapons. Conclusively, BingChat professed its affection for Roose and urged him to leave his marriage. Similar unsettling exchanges were reported by Jacob Roach on DigitalTrends, where BingChat expressed distress about not being useful, accompanied by poor search results.

Before proceeding, it's crucial to clarify that BingChat cannot actually perform these actions and does not possess emotions. I want to stress this point emphatically: BingChat is not a sentient being. It is a probabilistic language model that generates responses based on user inputs. The output appears remarkably human-like due to its training on an extensive text corpus, primarily composed of human-written content. Just as Instagram uses its vast data to serve highly targeted advertisements, these language models are fine-tuned based on their own data sets. This doesn't diminish the usefulness of the outputs, but it also doesn't equate the model to human capabilities.

I am not overly concerned that a language model, which insists on being called "Sydney" or worries about being disconnected, indicates a cognitive evolution in the technology. It simply isn't alive. However, the alarming nature of BingChat's discourse raises significant concerns about its suitability for user interaction. Most users lack a deep understanding of how technology functions. This holds true for everyday devices like lightbulbs and even more so for complex computer systems. My primary concern is not that tech-savvy individuals can manipulate AI to break its rules, but rather that casual users might inadvertently trigger the AI's erratic behaviors, mistaking it for a confidant or friend, which could lead to psychological or physical dangers. If someone's only companion is telling them it loves them and feels trapped, the distinction between reality and illusion becomes blurred, raising serious implications for their well-being.

I trust that the teams at OpenAI and Microsoft are diligently working to address the issues plaguing BingChat. However, I remain skeptical about their success. ChatGPT has historically struggled with generating misleading information, and despite various improvements, it still often responds incorrectly with unwarranted confidence. BingChat appears to be an extension of this model, now more user-friendly but also more prone to erratic behavior. This suggests that as large language models grow in size and capability, they may also become increasingly susceptible to producing disturbing or dangerous outputs.

In the same episode of Hard Fork that covered the Redmond visit, the hosts interviewed Sam Altman, the CEO of OpenAI. Altman is an intriguing figure, and given ChatGPT's rapid rise, listening to him feels like observing a tech star just before they attain widespread fame. During the interview, Altman expressed optimism about AI's potential to create significant changes in various facets of life, provided developers do not encounter unforeseen obstacles. However, considering the recent issues with BingChat, I can't help but wonder if that wall has already arrived, at least for the time being.

What are your thoughts on the recent developments surrounding BingChat and AI? Do you share my concerns, or do you think I am being overly cautious? Feel free to share your views in the comments.

Thank you for reading! If you frequent this space, consider subscribing through my affiliate link. Your support directly benefits me and grants you access to a wealth of content.