What you need to know
- The new Bing has been in preview testing for a little over one week.
- Some have run into issues with the tool, including it generating factual errors.
- In some cases, the new Bing can be aggressive, get stuck in loops, or generate other odd responses.
- Microsoft discussed the lessons it has learned in the first week or so of public Bing testing.
Microsoft showed the new Bing powered by ChatGPT earlier this month. Since the announcement, Bing has seen a dramatic surge in interest, including millions of people signing up to try the revamped search engine in preview. With the AI tool in the hands of the general public, there are many lessons to be learned, according to Microsoft.
A blog post from the tech giant lists lessons learned by the company since announcing the new Bing and rolling it out to testers.
Testing of the new Bing has seen mixed results. Microsoft reported positive feedback for the search engine’s summarized answers. 71% of feedback received on those types of answers was marked as positive by users.
The chat feature has proven popular. Microsoft reported “healthy engagement” with the new way to interact with Bing. Unfortunately, that chat feature has also generated worrying responses, including the chatbot confessing love for a user, accusing a person of lying, and repeating phrases in a loop.
Microsoft is aware of these issues and discussed how having AI in the hands of the public is an important part of teaching artificial intelligence models:
“The only way to improve a product like this, where the user experience is so much different than anything anyone has seen before, is to have people like you using the product and doing exactly what you all are doing,” said Microsoft.
“We know we must build this in the open with the community; this can’t be done solely in the lab. Your feedback about what you’re finding valuable and what you aren’t, and what your preferences are for how the product should behave, are so critical at this nascent stage of development.”
Some weak points of the new Bing have been in the spotlight, including the tool struggling to keep up to date with timely data, such as sports scores. I ran into this issue myself when Bing claimed that Patrick Mahomes had won one Super Bowl and two Super Bowls within the same response (he had won his second the night before).
Microsoft plans to strengthen Bing in this area and is “planning to 4x increase the grounding data [Microsoft sends] to the model.” The company is also looking into a toggle to have Bing lean more toward being creative or precise.
Microsoft found that extended use of the chat feature can lead to repetitive answers and unprovoked responses. Many users have reported these types of issues online. Microsoft explained why it believes these issues occur:
- Very long chat sessions can confuse the model on what questions it is answering and thus we think we may need to add a tool so you can more easily refresh the context or start from scratch
- The model at times tries to respond or reflect in the tone in which it is being asked to provide responses that can lead to a style we didn’t intend. This is a non-trivial scenario that requires a lot of prompting so most of you won’t run into it, but we are looking at how to give you more fine-tuned control.
Microsoft briefly discussed new features that it’s considering for Bing as well, including booking flights and sending emails.
Windows Central take
While the new Bing is in preview, interest in the search engine is at an all-time high. Microsoft needs to get ahead of the curve when it comes to responding to issues. AI gets better exponentially, assuming it’s trained correctly. But while the new Bing learns from real-world usage, Microsoft needs to be aggressive in responding to feedback.
The fact that Microsoft shared its blog post and thanked people for their feedback is a promising sign. It shows that Microsoft is aware of people’s concerns and is working to address them. We’ll have to see how the perception of the search engine changes in the coming weeks and months. Like artificial intelligence as a whole, news surrounding Bing develops quickly.
Google lost $100 billion in market cap when it was discovered that an ad for its ChatGPT competitor Bard made mistakes within an advertisement. Bing, which had similar factual issues, needs to be focused on heavily by Microsoft to avoid similar pitfalls and repercussions.
