Chatbot review bias threatens online shopping trust, study reveals

• Research uncovers significant chatbot review bias in online retail ratings.
• Study shows chatbot review bias leads to inflated scores and shorter feedback.

The growing phenomenon of chatbot review bias is undermining the reliability of online shopping platforms, according to groundbreaking research from European business schools. The systematic distortion in customer feedback could have far-reaching implications for e-commerce trust and transparency.

The comprehensive study on chatbot review bias, published in the Journal of the Academy of Marketing Science, reveals that when consumers interact with chatbots to leave reviews – particularly those with human-like characteristics – they consistently provide higher ratings while writing shorter, less helpful reviews compared to traditional form-based feedback methods.

“Online reviews are critical for consumers to make good purchase decisions. For this to happen, online reviews must be helpful, and ratings must accurately reflect product quality,” explained Dr Dimitrios Tsekouras, Senior Lecturer at Rotterdam School of Management. “But if ratings are better than they should be, and not very detailed, this can mislead consumers and harm the reputation of online selling platforms.”

The research team conducted multiple experiments, including field tests, to understand how chatbot review bias manifests in real-world scenarios. Their findings identified two fundamental mechanisms driving bias: interaction enjoyment with moderately human-like chatbots and social presence with highly human-like chatbots. Both factors contributed to inflated ratings, with reviews collected through chatbots showing consistently higher scores than those collected through conventional methods.

Perhaps most concerning is the impact on review quality. Dr Tsekouras and his colleagues found that chatbot-collected reviews were significantly shorter and less detailed, making them less valuable for potential buyers seeking authentic product information. This aspect of chatbot review bias poses a particular challenge for platforms like Amazon, where detailed customer feedback plays a crucial role in purchase decisions.

The implications of systematic bias extend beyond just numbers. Dr Dominik Gutt, one of the study’s co-authors, points out, “Low-quality sellers could abuse chatbots to boost their ratings and disguise the low quality of their products.” The potential for manipulation raises severe concerns about the transparency and reliability of online review systems.

The researchers suggest that platforms and policymakers consider whether chatbots should be permitted to collect online reviews. Their findings indicate that the current trend toward automated review collection might be undermining the very purpose of customer feedback systems.

Key findings from the study include:

• Chatbot review bias consistently resulted in higher product ratings,
• The more human-like the chatbot, the higher the rating inflation,
• Chatbot-collected reviews were shorter and contained less detailed information,
• The bias was particularly pronounced for lower-quality products.

“If ratings are better than they should be and not very detailed, this can mislead consumers and harm the reputation of online selling platforms,” warned Dr Tsekouras.

As online retail grows, addressing chatbot review bias presents a crucial challenge for the industry. While chatbots offer efficiency and cost savings in customer service, their impact on review integrity suggests that businesses might need to reconsider their implementation in feedback collection processes.

The study recommends that e-commerce platforms and retailers carefully weigh the benefits of automated review collection against the potential costs of review quality and accuracy. Dr. Irina Heimbach from WHU – Otto Beisheim School of Management – suggests that the research should inform policymakers designing feedback mechanisms.

For consumers, the message is clear: be aware that reviews collected through chatbots might not tell the whole story. Be wary of mostly-short and mostly-positive reviews – they may be subject to the bias the researchers unearthed. As AI continues to reshape online retail, understanding and mitigating chatbot review bias becomes increasingly crucial for maintaining the web of trust that’s essential for e-commerce ecosystems to prosper.

Dashveenjit Kaur | @DashveenjitK

Dashveen writes for Tech Wire Asia and TechHQ, providing research-based commentary on the exciting world of technology in business. Previously, she reported on the ground of Malaysia’s fast-paced political arena and stock market.