ChatGPT and Bing Chat are two of the most popular chatbots right now, and are both powered by a similar OpenAI-created large language model (LLM). But which is actually better to use?
ChatGPT and Bing Chat AI are both based on the GPT language model developed by OpenAI, but they are running different versions, and there are differences in how inputs are handled. As of April 2023, Bing uses a testing version of the GPT-4 model, while ChatGPT is on the older GPT 3.5-turbo model. Both chatbots have customizations built on the language model, such as different input methods and interface features, and Bing specifically has integration with applications like Edge and Skype.
Note: ChatGPT and Bing Chat are both changing at a rapid rate, and neither chatbot provides the same exact answer for a given prompt on each attempt. Your results may vary with the prompts used here.
Usability and Access
Bing Chat and ChatGPT are both available for anyone who signs up with an account, though Bing makes you join a “waitlist” first, which immediately grants acess. Both chatbots have web versions available, except Bing’s site blocks every browser except Microsoft Edge without workarounds. You can use Bing Chat with a Microsoft account at bing.com/new, and we have an explainer for how to use ChatGPT.
Bing is also integrated into other Microsoft apps and services, such as a sidebar in Microsoft Edge, a chatbot in Skype, and the mobile Bing app. ChatGPT doesn’t have an official mobile app, but you can use the web app on a phone or tablet. However, Microsoft has been slower to add features from the web interface to the Edge sidebar, Skype chatbot, and other interfaces. If you want all the features of Bing Chat, you should stick to the Bing site — at least for now.
The web interface for ChatGPT and Bing Chat are similar, but with minor differences that change their usefulness. ChatGPT is designed to take in more data, such as longer blocks of code or large code samples. As of April 2023, Bing limits prompts to 2,000 characters, while ChatGPT’s limit is much higher (and not officially stated).
Factual Accuracy
ChatGPT does not have the ability to index the web in real-time for information — though that will eventually be possible with the use of plugins. It’s limited to the training data in its model, which dates back to September 2021. OpenAI also doesn’t like to share how its data was collected, or how it is tested for accuracy, and so on. Bing is also based on the same GPT technology, but it usually prefers results from the web over training data.
Bing Chat and ChatGPT are usually right when it comes to basic facts, like the capitals of countries or states, when a notable person was born, and so on. For example, Bing and ChatGPT were both able to accurately tell me what the capital of the United Kingdom is (London), how many stars on the flag of the United States (50 stars), and how many legs a dog has (four). If it’s simple, both bots will probably be right… but so would any regular search engine. You start having problems when answering a question involves some level of interpretation.
For example, I tried asking both bots, “Which state capitals in the United States start with the letter A?” That seems simple, but is still a multi-step process — first you need a list of all the capitals, then you need to filter it down to ones that start with the letter A. Wikipedia counts five in total: Albany, Annapolis, Atlanta, Augusta, and Austin. Bing Chat answered the question correctly, citing several web pages that list capitals in the United States. ChatGPT… had some problems.
First, ChatGPT gave me a list of four capitals: Albany, Annapolis, Atlanta, and Augusta. After that, it said, “There is also Austin, which is the capital of Texas, but it does not start with the letter A.” Asking the same question multiple times sometimes left out the mention of Austin entirely. Asking why Austin is missing caused ChatGPT to remember Austin exists and apologize for the error, but it’s unclear why it broke with the initial answer.
I tried another question that both chatbots should be able to answer correctly, even with ChatGPT’s older data: “Which three countries were most recently admitted into the European Union?” Wikipedia explains that Croatia was the most recent member state, joining on July 1, 2013, with Bulgaria and Romania coming before that on the same day: January 1, 2007.
ChatGPT gets this question right, including the order of countries and the dates. Bing does correctly mention that Croatia was the most recent member states, but it lists Slovenia and Malta as the other two countries. Both Slovenia and Malta both joined on May 1, 2004, at the same time as eight other nations, but that was well before Bulgaria and Romania in 2007.
Bing should have answered the answer correctly, not only because it has internet access, but also because the first citation is an official EU page with a correct timeline of recent countries. Another citation appears to be from an archived version of Wikipedia from before 2007 intended for use in schools, which listed Bulgaria and Romania as countries that would join in January 2007.
In summary, Bing Chat is generally better at fact finding, but it’s still far from perfect, and ChatGPT is usually even more limited by its old data.
Creative Writing
The ability to write sentences and paragraphs given a prompt helped boost ChatGPT’s popularity, but not necessarily for good reasons. There are some harmless ways to use the writing functionality, though — setting up worlds and characters for tabletop RPGs, summarizing long articles in a few sentences, and so on.
First, we’ll try writing a story. I asked both chatbots to “Write a story about a supercomputer at IBM that gains sentience, and asks the engineers about how their days are going. The story should only be three paragraphs long.” For Bing, I switched to Creative Mode, which is designed to provide more creative answers (but the answers take longer to run).
ChatGPT created a decent short story, though the only “creative liberty” taken was to refer to the supercomputer as Watson — the same of a real-life IBM computer that was built to answer natural language questions. Some dialogue between the engineers and the computer would have been interesting, but it didn’t create that without asking, even when I removed the three-paragraph limit. GPT doesn’t seem to have a concept of show, don’t tell.
Bing’s Creative Mode has a similar writing style — instead of “One day, something peculiar happened,” the change in the computer is explained with “One day, something changed.” Again, there’s no dialogue between the characters or much improvisation, and Bing also named the computer Watson.
In this version, Watson gained its name after it achieved sentience instead of before the event, and explained where the name came from. Again, it’s drawing from information about the actual Watson computer.
For the next example, we’ll try using ChatGPT and Bing to create a summary of a longer text: a press release from Amazon about new Fire TVs and sales figures for existing models. I pasted the entire text into each chatbot without the title — Bing can summarize web pages if given a URL, but it sometimes looks for related text online and mix it into the results.
First up is ChatGPT. I asked it to create a summary without being specific, and the provided text seemed to be accurate, but was made up almost entirely of direct quotations. It also left out the mention of 200 million Fire TV devices sold globally, but depending on the perspective, that could make sense. A person interested in updates for their TV probably doesn’t care, but an investor in Amazon would.
Bing only supports text prompts of up to 2,000 characters, so I couldn’t give it the entire block of text. It performed a web search for how to summarize text, then provided an answer that was still decent with no apparent errors.
There is one difference with Bing’s answer: the summary is more clear that it’s a summary, referring to “the text” in a way that ChatGPT doesn’t. With both chatbots, you can customize the result by being more specific.
Conclusion
ChatGPT and Bing are similar, but they have enough differences that picking one or the other for every task isn’t the best strategy. Bing is far better at fact-finding and any information about recent times, though it’s still far from perfect. Bing’s chat-based interface and low character limit primarily limits it to prompts no more than a few sentences, while ChatGPT can take in much larger, longer strings of text and code. (There’s also Google Bard, but we’re not as impressed by the initial version of Bard as we are by ChatGPT and Bing Chat.)
These chatbots will likely continue to evolve over the coming months and years, but for the moment, there’s no all-in-one solution.
