According to data released by research firm Epoch AI, the cost of training AI models has exploded over the past year. This development aptly illustrates how complex and capable AI models have become in a short space of time. In 2023, OpenAI released the latest version of ChatGPT in March, sparking a global AI boom. Google followed suit with its advanced AI model Gemini in December.
According to Epoch AI’s announcement, training both systems will be much more expensive than previous AI models, and they could cost hundreds of millions of dollars to develop. Training costs for Gemini, a large-scale language model that can be input by text, voice commands, and images, are reported to be between $30 million and $191 million, not even taking into account staff salaries. These could account for 29% to 49% of the final price, according to Epoch AI. The latest version, ChatGPT-4, cost between $41 million and $78 million to develop the technology, sources said. OpenAI CEO Sam Altman previously said the model cost more than $100 million, backing up this calculation.
Looking back, previous AI models were much less expensive: ChatGPT-3 cost only around $2-4 million in 2020, while Gemini’s predecessor, PaLM, cost $3-12 million to train in 2022, just looking at compute costs. Even at these price points, it may have been difficult for academic institutions and other public organizations that have traditionally been active in AI research to keep up with cutting-edge AI developments.
Epoch AI noted that cost estimates for 2023 make it essentially impossible, and cited the National AI Research Resource, which the Biden administration will create in late 2023, as a potential solution. The resource would grant researchers and students access to relevant AI tools and provide grant funding. However, it is still in the pilot phase. The executive order that created the resource focuses primarily on setting standards for AI safety and privacy, for example, strengthening consumer rights over algorithms and employee rights in the face of workplace changes.
AI for consumption?
ChatGPT-4 will be updated to support voice and images in Fall 2023, but as its name suggests, it started out with a core text input, whereas Gemini and its app were designed from the beginning as a multimodal LLM. This explains why ChatGPT’s initial training costs were lower, whereas Gemini’s general focus on app distribution, for example by having users take photos with their smartphones and then select features from them to analyze, may have led to higher costs.
Gemini also includes e-commerce-related features, such as showing where to buy something in a photo, in a manner similar to Google (shopping) search. This shows that Google is applying its brand identity as a search engine to its AI models, whereas OpenAI, an AI-first company, has had to build its identity and strength in the AI field from scratch. It also raises the question of whether the future of AI will move towards commerce support functions, as predicted by the Biden administration, rather than original text creation, ChatGPT’s most touted feature.
DALL-E, OpenAI’s text-to-image model, was much less expensive as of 2021 than LLMs created during the same period, including ChatGPT version 3 in 2020. According to Epoch AI, its creation cost was only $118,000 to $335,000. Its latest follow-up version, DALL-E 3, is now part of the extended ChatGPT version for paying customers. This calculation uses amortized costs, so the price of your own hardware is always lower than a cloud computing approach, meaning that only the percentage of the total lifetime cost of the hardware components relative to the time used to train the respective AI model is taken into account.
—
Author Statista