Even more sophisticated AI systems have issues of reliability and cost. McDonald’s scrapped an AI ordering system it had been piloting in the United States last month that occasionally added bacon to soft serve cones or piled a football team’s worth of chicken nuggets onto a solo customer’s order. Goldman Sachs’ head of global equity research, Jim Covello, who is much more bearish on AI than his employer, said in one of the bank’s reports that generative tools were failing to complete even basic tasks efficiently. “For example, we’ve found that AI can update historical data in our company models more quickly than doing so manually, but at six times the cost,” Covello said.
AI bulls’ rejoinder is that the technology is in its infancy. Costs will come down and performance will improve with more investment, they say. Covello said one common analogue was the rapid reduction in server costs in the late 1990s as the internet grew. “But,” he said, “the number of $64,000 Sun Microsystems servers required to power the internet technology transition in the late 1990s pales in comparison to the number of expensive chips required to power the AI transition today, even without including the replacement of the power grid and other costs necessary to support this transition that on their own are enormously expensive.”
If anything, costs are going up. The chief executive of Anthropic, an $US18.4 billion ($27 billion) generative AI start-up backed by Amazon, has forecast an increase in the order of magnitude. “Right now, [models cost] $US100 million,” Dario Amodei told a podcast in April. “There are models in training today that are more like $US1 billion,” Amodei said. Generative AI models costing $US10 billion or $US100 billion could be required as soon as next year if the industry was to match or better humans’ performance at many tasks, he predicted.
It’s not clear where more high-quality data will come from and whether it will be easily and cheaply available to AI models.
— Daron Acemoglu, Massachusetts Institute of Technology
Whether AI reaches that milestone is key to understanding its potential. Massachusetts Institute of Technology economist Daron Acemoglu thinks that will happen slowly, if at all, underpinning his forecast of a 1 per cent GDP boost and 0.5 per cent productivity improvement from AI over the next decade. He argues that the number of purely mental tasks, which are best-suited to AI, are only a small subset of Western economies and sees limits to how much models can improve.
“It’s not clear where more high-quality data will come from and whether it will be easily and cheaply available to AI models,” Professor Acemoglu said in the Goldman Sachs report. Even if that does arrive, AI tools cannot reason like a human. “A big leap of faith is still required to believe that the architecture of predicting the next word in a sentence will achieve capabilities as smart as HAL 9000 in 2001: A Space Odyssey,” Professor Acemoglu said.
Productivity Commissioner Stephen King said he largely agreed with Professor Acemoglu that the benefits of AI may not show up in formal productivity numbers. But he argued that did not diminish its utility to the country on intangible measures such as wellbeing or clearer statistics such as GDP over the long term. “Anybody who’s coming along two years after ChatGPT was released and saying, ‘I can’t see the productivity benefits … therefore, it’s a lost technology’ … [is] far too impatient in not looking back at history and seeing these things take time.
“I suspect they’ll be the same people who will be sitting there in 30 years, probably saying, ‘Damn, we missed that wave. Everybody’s using the stuff now.’”
Even Australia Post announced a renewed partnership with Microsoft last month to experiment with genuine generative AI tools. It did not explicitly mention a new chatbot.