Microsoft’s Bing with ChatGPT is under fire for not only getting things wrong but for becoming “unhinged” and having “meltdowns.” I’m not dismissing the experiences that other people have had so far with ChatGPT, but I think it’s also worth pointing out that this chatbot can admit mistakes and get smarter. Literally on the fly.
At the same time, the new Bing has a long way to go.
For example, since I’m looking forward to the next NFL season, I decided to ask the new Bing what the chances were that the New York Giants would make it to the Super Bowl next year. The response had a big error.
First, Bing told me that the chances are not very high, which makes sense given how the Giants were pulverized by the Eagles in this year’s playoffs. But then Bing told me they “have +5500 odds to win Super Bowl 57.” For those scoring at home, that Super Bowl just happened.
Bing went on to tell me that the Giants “have not made the playoffs since 2016, and they finished with a 4-12 record.” Not true on either count. They made the playoffs this year and finished with a 9-7-1 record. Clearly, Bing was pulling info from last year, so I said.
“That’s not correct. The Giants finished with a 9-7-1 record this year.”
Then came the apology I was not expecting from the ChatGPT-powered Bing.
“I’m sorry, I was using the information from the search results from the previous year. The search results may be outdated or inaccurate. Thank you for correcting me.” (Complete with embarrassed emoji.)
The chatbot went on to write that the Giants did finish with the record I shared and that they won the NFC East Division. Unfortunately, that’s not correct, either. It’s the Eagles. And Bing said the Giants lost to the Rams when it was the Eagles. Not great. I corrected Bing on both counts.
Undeterred, I asked Bing explicitly if the New York Giants made the playoffs this year, and it said yes. Progress!
“They finished second in the NFC East with a 9-7-1 record, behind the Philadelphia Eagles, who had a 10-7 record1. They clinched their playoff berth with a dominant 38-10 victory over the Indianapolis Colts in Week 17, where quarterback Daniel Jones threw for two touchdowns and ran for two more. It was their first time returning to the postseason since 2016.”
All of that is true, but then Bing faltered again, writing that the Giants faced the Los Angeles Rams in the wild-card round and that they play the Green Bay Packers next week. Good news, football season isn’t over!
Microsoft readily admits that the new Bing with ChatGPT makes mistakes. And it’s nice to see when the chatbot admits when it is wrong and even corrects itself. But it’s going to take a lot more corrections from people like me before this tool can become trustworthy.
And it’s worth keeping in mind that sports is a frivolous topic compared to other subjects and queries that the new Bing will get about subjects like money and health. For now, I would consider Bing with ChatGPT more for entertainment than serious research.