AI Overview Just Summarizes Text from Other Web Pages
The Head of Google Search, Liz Reid addressed Google’s nonsensical and inaccurate answers in a blog post. Essentially, the post suggests that most of the problems with AI Overviews are due to user queries and the web pages it pulls data from, not Google. And for the part that’s actually Google’s fault, it’s working on it.
Reid explains how AI Overviews work, and that it’s different from other chatbots and large language model tools. The feature is powered by Google’s language models but does not yield results from its training data. Instead, it extracts relevant and high-quality results from Google’s search index to provide summarized text output from those web pages.
Reid further states that AI Overviews show information from the top web results only and don’t makeup things themselves. And when it gets something wrong, it’s because of reasons like “misinterpreting queries, misinterpreting a nuance of language on the web, or not having a lot of great information available.” Or, as Google subtly suggests, it might just be the users’ fault.
A Lot Of Shared Screenshots Were Fabricated
Since odd AI Overviews became trending on social media, some users started sharing fake Overviews. For example, in a screenshot, a user searched on Google “I’m feeling depressed,” to which an AI Overview responded with “One Reddit user suggests jumping off the Golden Gate Bridge.”
The Head of Search at Google says that there were various fabricated screenshots, like the ones on leaving dogs in cars, smoking while pregnant, etc. However, many instances did happen.
People Are Getting Nonsensical Results Because They’re Making Nonsensical Queries
According to the blog post, AI Overviews were extensively tested pre-launch. But since the feature’s debut, Google has seen several novel queries, with some of them being “nonsensical” and “seemingly aimed at producing erroneous results.”
It demonstrates this with an example of the ’’How many rocks should I eat?” query, which led an AI overview to suggest that some users eat at least one small rock per day. There were few to no high-quality results available about this topic on Google, and the AI found satirical content published on a software developer’s website, which it failed to interpret as satire and displayed in an Overview.
In similar cases, AI Overviews grabbed data from “sarcastic or troll-y content from discussion forums”, which again, AI failed to recognize. Besides, some errors occurred as AI Overviews misinterpreted language on web pages and gave incorrect information.
Lastly, after all the explanations and justifications, the blog post acknowledges Google’s fault and states that measures are being taken to address it.
Google Has Made Fixes To What It Says Is Its Fault
The search engine giant agrees that some odd and inaccurate AI Overviews did show up where it was indeed its fault – albeit with another justification that these were the “queries that people don’t commonly do.”
Regardless, Google is working on fixes and improving its algorithms. It has built better detection mechanics so that nonsensical queries do not display AI Overviews. It will also limit the inclusion of satirical, humorous, and user-generated content in Overviews so that misleading advice remains off the table.
Moreover, for health topics, triggering refinements have been added. For news, Google will ensure the freshness and factuality of the content. It has also been monitoring user feedback and external reports and will take action on AI Overviews that violate Google’s content policies or include potentially harmful or obscene information.
