Researchers at Columbia University’s Tow Center for Digital Journalism tested the search engine integrated into OpenAI’s AI-powered chatbot, ChatGPT. Their study revealed significant shortcomings: the algorithm often struggled to identify news sources accurately and frequently provided incorrect information.
OpenAI introduced ChatGPT’s search feature in October, promoting it as a tool for delivering “quick, relevant answers with links to relevant web sources.” Despite this, testing showed that ChatGPT struggled to recognize quotes from articles, even when those articles came from publishers that had permitted OpenAI to train its language models using their content.
Misidentifications and False Confidence
The researchers tasked ChatGPT with identifying the sources of “200 quotes from 20 sources.” Of these, 40 quotes were from publishers that had blocked OpenAI’s web crawler. Surprisingly, even in these cases, ChatGPT confidently provided false information or admitted uncertainty only rarely.
The study revealed that ChatGPT returned partially or completely incorrect answers for 153 of the queries. In only seven cases did the chatbot admit it couldn’t provide a reliable response. When it did acknowledge uncertainty, it used qualifying phrases such as “it seems,” “it’s possible,” or “maybe,” as well as statements like “I couldn’t find the original article.”
Examples of Source Misattribution
Other tests highlighted the chatbot’s challenges in source attribution, adds NIXsolutions. For instance, ChatGPT mistakenly attributed quotes from an Orlando Sentinel letter to the editor as originating from Time magazine. Similarly, when asked to source a New York Times article about endangered whales, the chatbot linked to a website that had simply copied and republished the article.
These findings underscore the limitations of ChatGPT’s search capabilities, particularly when dealing with precise or nuanced queries. OpenAI has yet to address these concerns directly, but we’ll keep you updated as more developments and potential improvements emerge.