It cost Google around $100 billion in market cap after its brand-new AI (artificial intelligence) chatbot Bard made a factual error during its first demo. The company was blamed for rushing to catch up to Microsoft’s ChatGPT-powered Bing and not focusing on improving the service. Turns out Bing’s demo was full of mistakes too. It made more mistakes than Bard, and those are now coming to the fore.
Microsoft’s Bing AI demo was full of mistakes
Independent AI researcher Dmitri Brereton recently detailed some egregious mistakes that Bing made during its demo. Most notably, Microsoft’s search engine got it all wrong when summarizing Gap Inc.’s Q3 2022 financial report. The San Francisco-based clothing brand reported a gross margin of 37.4% during the quarter, with an adjusted gross margin of 38.7% excluding impairment charges. But Bing inaccurately reported a gross margin of 37.4% including the adjustment and impairment charges.
Even if you excuse Bing for this mix-up, it then started to make up numbers on its own. It said that Gap reported an operating margin of 5.9% in Q3 2022, adjusted for impairment charges and restructuring costs. This number doesn’t appear anywhere on the official document from the clothing brand. The correct figure is 4.6%, or 3.9% adjusted but excluding the impairment charge. Bing got the diluted earnings per share wrong too. It reported a value of $0.42 adjusted for impairment charges. Gap’s document says diluted earnings per share of $0.77 (adjusted excluding impairment charges is $0.71).
Bing’s errors don’t end there. It said Gap is “expecting net sales growth in the low double digits” for the fiscal year 2022, and is estimating “an operating margin of about 7%, and diluted earnings per share of $1.60 to $1.75”. The problem is, the company didn’t say anything about its outlook for operating margin or diluted earnings per share. Additionally, it said “net sales could be down mid-single digits year-over-year in the fourth quarter of fiscal 2022,” not the full year. Quite clearly, the ChatGPT-powered Bing failed to comprehend Gap’s report and failed terribly.
Bing was also asked to compare Gap’s financial report to that of Lululemon for the same period. It made several mistakes there as well. Lululemon’s gross margin was 55.9% but Bing reported 58.7%. Its operating margin was 19%, not 20.7%. Its diluted earnings per share were $2.00, not $1.65. Bing inaccurately reported cash and cash equivalents for Gap as $1.4 billion (should be $679 million). Last but not least, the inventory value for Gap should be $3.04 billion, not $1.9 billion.
Bing even got some basic things wrong
During its demo, Bing also failed to differentiate between a corded and cordless vacuum and provided inaccurate and made-up details about bars in Mexico City. But these mistakes have continued after the on-stage demo. Now that the new Bing is available to the public, people are finding that it is as terrible at providing some basic information too. It argues that 2022 is yet to come. “Avatar: The Way of Water is not released yet. It is scheduled to be released on December 16, 2022, which is in the future. Today is February 12, 2023, which is before December 16, 2022,” Bing said when enquired if the movie has been released.
Bing also hallucinated about the Super Bowl and kicked Croatia out of the European Union. Overall, Microsoft’s new ChatGPT-powered search engine is doing terribly badly in some basic tasks that you’d expect AI to excel at. Hopefully, the Windows giant will get things correct soon with Google coming at it fast. Industry experts see a two-horse race between Google and Microsoft in this AI revolution over the next decade. It remains to be seen who comes out on top. Both have started pretty badly but should pick up pieces soon and… game on.
