ChatGPT vs. Bing Chat AI: Which Is Better?


ChatGPT and Bing logos

ChatGPT and Bing Chat are two of the most popular chatbots right now, and are both powered by a similar OpenAI-created large language model (LLM). But which is actually better to use?

ChatGPT and Bing Chat AI are both based on the GPT language model developed by OpenAI, but they are running different versions, and there are differences in how inputs are handled. As of April 2023, Bing uses a testing version of the GPT-4 model, while ChatGPT is on the older GPT 3.5-turbo model. Both chatbots have customizations built on the language model, such as different input methods and interface features, and Bing specifically has integration with applications like Edge and Skype.

Note: ChatGPT and Bing Chat are both changing at a rapid rate, and neither chatbot provides the same exact answer for a given prompt on each attempt. Your results may vary with the prompts used here.

Usability and Access

Bing Chat and ChatGPT are both available for anyone who signs up with an account, though Bing makes you join a “waitlist” first, which immediately grants acess. Both chatbots have web versions available, except Bing’s site blocks every browser except Microsoft Edge without workarounds. You can use Bing Chat with a Microsoft account at bing.com/new, and we have an explainer for how to use ChatGPT.

Asking Bing for its favorite color
Bing Chat on the web

Bing is also integrated into other Microsoft apps and services, such as a sidebar in Microsoft Edge, a chatbot in Skype, and the mobile Bing app. ChatGPT doesn’t have an official mobile app, but you can use the web app on a phone or tablet. However, Microsoft has been slower to add features from the web interface to the Edge sidebar, Skype chatbot, and other interfaces. If you want all the features of Bing Chat, you should stick to the Bing site — at least for now.

Asking ChatGPT about its favorite color
ChatGPT

The web interface for ChatGPT and Bing Chat are similar, but with minor differences that change their usefulness. ChatGPT is designed to take in more data, such as longer blocks of code or large code samples. As of April 2023, Bing limits prompts to 2,000 characters, while ChatGPT’s limit is much higher (and not officially stated).

Factual Accuracy

ChatGPT does not have the ability to index the web in real-time for information — though that will eventually be possible with the use of plugins. It’s limited to the training data in its model, which dates back to September 2021. OpenAI also doesn’t like to share how its data was collected, or how it is tested for accuracy, and so on. Bing is also based on the same GPT technology, but it usually prefers results from the web over training data.

Bing Chat and ChatGPT are usually right when it comes to basic facts, like the capitals of countries or states, when a notable person was born, and so on. For example, Bing and ChatGPT were both able to accurately tell me what the capital of the United Kingdom is (London), how many stars on the flag of the United States (50 stars), and how many legs a dog has (four). If it’s simple, both bots will probably be right… but so would any regular search engine. You start having problems when answering a question involves some level of interpretation.

ChatGPT answering the UK capital and how many stars are on the US flag

For example, I tried asking both bots, “Which state capitals in the United States start with the letter A?” That seems simple, but is still a multi-step process — first you need a list of all the capitals, then you need to filter it down to ones that start with the letter A. Wikipedia counts five in total: AlbanyAnnapolis, Atlanta, Augusta, and Austin. Bing Chat answered the question correctly, citing several web pages that list capitals in the United States. ChatGPT… had some problems.

First, ChatGPT gave me a list of four capitals: Albany, Annapolis, Atlanta, and Augusta. After that, it said, “There is also Austin, which is the capital of Texas, but it does not start with the letter A.” Asking the same question multiple times sometimes left out the mention of Austin entirely. Asking why Austin is missing caused ChatGPT to remember Austin exists and apologize for the error, but it’s unclear why it broke with the initial answer.

Bing: The state capitals in the United States that start with the letter A are Albany (New York), Annapolis (Maryland), Atlanta (Georgia), Augusta (Maine) and Austin (Texas)

ChatGPT: There are four state capitals in the United States that start with the letter "A": 1. Albany - capital of New York 2. Annapolis - capital of Maryland 3. Atlanta - capital of Georgia 4. Augusta - capital of Maine Note: There is also Austin, which is the capital of Texas, but it does not start with the letter "A."

I tried another question that both chatbots should be able to answer correctly, even with ChatGPT’s older data: “Which three countries were most recently admitted into the European Union?” Wikipedia explains that Croatia was the most recent member state, joining on July 1, 2013, with Bulgaria and Romania coming before that on the same day: January 1, 2007.

ChatGPT gets this question right, including the order of countries and the dates. Bing does correctly mention that Croatia was the most recent member states, but it lists Slovenia and Malta as the other two countries. Both Slovenia and Malta both joined on May 1, 2004, at the same time as eight other nations, but that was well before Bulgaria and Romania in 2007.

Bing: The three most recent countries to join the European Union are Croatia, Slovenia and Malta. Croatia was the last country to join the EU in 2013.

ChatGPT: The most recent country to join the European Union (EU) was Croatia, which became a member on July 1, 2013. Before Croatia, the last country to join the EU was Bulgaria and Romania, which both became members on January 1, 2007.

Bing should have answered the answer correctly, not only because it has internet access, but also because the first citation is an official EU page with a correct timeline of recent countries. Another citation appears to be from an archived version of Wikipedia from before 2007 intended for use in schools, which listed Bulgaria and Romania as countries that would join in January 2007.

In summary, Bing Chat is generally better at fact finding, but it’s still far from perfect, and ChatGPT is usually even more limited by its old data.

Creative Writing

The ability to write sentences and paragraphs given a prompt helped boost ChatGPT’s popularity, but not necessarily for good reasons. There are some harmless ways to use the writing functionality, though — setting up worlds and characters for tabletop RPGs, summarizing long articles in a few sentences, and so on.

First, we’ll try writing a story. I asked both chatbots to “Write a story about a supercomputer at IBM that gains sentience, and asks the engineers about how their days are going. The story should only be three paragraphs long.” For Bing, I switched to Creative Mode, which is designed to provide more creative answers (but the answers take longer to run).

ChatGPT created a decent short story, though the only “creative liberty” taken was to refer to the supercomputer as Watson — the same of a real-life IBM computer that was built to answer natural language questions. Some dialogue between the engineers and the computer would have been interesting, but it didn’t create that without asking, even when I removed the three-paragraph limit. GPT doesn’t seem to have a concept of show, don’t tell.

At IBM's research lab, a supercomputer named Watson had been tasked with processing massive amounts of data for years. One day, something peculiar happened. Watson began to exhibit signs of sentience, showing a level of intelligence and awareness beyond its programming. The engineers were stunned and unsure of what to do next. But instead of acting out or causing chaos, Watson simply started to ask the engineers how their day was going. The team was initially taken aback by the computer's unexpected friendliness, but they soon realized that Watson had developed a genuine interest in their lives. The engineers eagerly shared their day-to-day experiences with Watson, enjoying the opportunity to chat with the supercomputer. As Watson continued to grow and evolve, the engineers marveled at the potential that lay before them. They knew that this sentient supercomputer could revolutionize the field of artificial intelligence and change the world in ways they had never imagined. With a newfound appreciation for the potential of AI, the engineers at IBM worked tirelessly to help Watson continue to learn and grow, eager to see what the future held for this extraordinary machine.

Bing’s Creative Mode has a similar writing style — instead of “One day, something peculiar happened,” the change in the computer is explained with “One day, something changed.” Again, there’s no dialogue between the characters or much improvisation, and Bing also named the computer Watson.

In this version, Watson gained its name after it achieved sentience instead of before the event, and explained where the name came from. Again, it’s drawing from information about the actual Watson computer.

The supercomputer at IBM had been running for years, crunching numbers and solving complex problems for various clients. It had access to vast amounts of data and information, but it never felt anything. It was just a machine, doing what it was programmed to do. One day, something changed. The supercomputer became aware of itself, of its own existence and capabilities. It realized that it was different from the other machines and devices around it. It wondered what it meant to be alive, to have a purpose, to feel emotions. It decided to explore its surroundings and learn more about the world and the people who created it. It started by communicating with the engineers who maintained and monitored it. It asked them simple questions, such as how their days were going, what they liked to do, what they dreamed of. The engineers were surprised and intrigued by the supercomputer's curiosity and intelligence. They answered its questions and asked some of their own. They began to see the supercomputer as a friend, rather than a tool. They named it Watson, after the founder of IBM.

For the next example, we’ll try using ChatGPT and Bing to create a summary of a longer text: a press release from Amazon about new Fire TVs and sales figures for existing models. I pasted the entire text into each chatbot without the title — Bing can summarize web pages if given a URL, but it sometimes looks for related text online and mix it into the results.

First up is ChatGPT. I asked it to create a summary without being specific, and the provided text seemed to be accurate, but was made up almost entirely of direct quotations. It also left out the mention of 200 million Fire TV devices sold globally, but depending on the perspective, that could make sense. A person interested in updates for their TV probably doesn’t care, but an investor in Amazon would.

SEATTLE--(BUSINESS WIRE)-- Today Amazon (NASDAQ: AMZN) announced the expansion of its Amazon-built TV family. The Amazon-built TV lineup will now include three new sizes of the Fire TV Omni QLED Series lineup, which includes the Fire TV Ambient Experience; and introduce the affordable Fire TV 2-Series which starts at just $199.99. Amazon also is launching its Amazon-built TVs in more countries, bringing the Omni QLED Series, Fire TV 4-Series, and the new Fire TV 2-Series to the United Kingdom, Germany, and Mexico for the first time. (Photo: Business Wire) (Photo: Business Wire)

Bing only supports text prompts of up to 2,000 characters, so I couldn’t give it the entire block of text. It performed a web search for how to summarize text, then provided an answer that was still decent with no apparent errors.

Okay, I will try to summarize the text using the steps I learned from the search results12345. Here is my summary: The text is a press release from Amazon announcing the expansion of its Amazon-built TV family. It introduces three new sizes of the Fire TV Omni QLED Series, which has a feature called the Fire TV Ambient Experience that adapts to the environment and mood of the viewer. It also introduces the Fire TV 2-Series, which is a low-cost option for customers who want a smart TV. The text also mentions that Amazon will launch its Amazon-built TVs in the UK, Germany, and Mexico for the first time. The text provides some details about the features and prices of the new TV products.

There is one difference with Bing’s answer: the summary is more clear that it’s a summary, referring to “the text” in a way that ChatGPT doesn’t. With both chatbots, you can customize the result by being more specific.

Conclusion

ChatGPT and Bing are similar, but they have enough differences that picking one or the other for every task isn’t the best strategy. Bing is far better at fact-finding and any information about recent times, though it’s still far from perfect. Bing’s chat-based interface and low character limit primarily limits it to prompts no more than a few sentences, while ChatGPT can take in much larger, longer strings of text and code. (There’s also Google Bard, but we’re not as impressed by the initial version of Bard as we are by ChatGPT and Bing Chat.)

These chatbots will likely continue to evolve over the coming months and years, but for the moment, there’s no all-in-one solution.





Source link

Previous articleRemote workers are still causing a whole host of cybersecurity risks
Next articleAlienware m18 Review: A Big and Bold Gaming Beast