- Former Salesforce executive Richard Socher spoke about AI models on a Harvard Business Review podcast.
- He said one way to significantly improve AI is to be able to program responses rather than just predict them.
- That “will give them even more fuel for the next few years in terms of what they can do,” he said.
Generative AI technology has advanced so rapidly over the past few years that some experts worry we may have already reached it. “Peak AI”
but Richard Sorcher, former Salesforce Chief Scientist He is the CEO of You.com, an AI-powered search engine, and believes we still have a way to go.
About Harvard Business Review podcast Sorcher said last week that large language models can be taken to the next level by forcing them to respond to specific prompts in the code.
Currently, large-scale language models only “predict the next token given the previous set of tokens,” Socher said. A token is the smallest meaningful unit of data in an AI system. Therefore, even if LLMs show good reading comprehension and coding skills and can pass difficult exams, AI models still cause a hallucination — A phenomenon in which they persuasively present misconceptions as truth.
And that’s especially problematic when faced with complex mathematical problems, Sorcher says.
He gave an example where large language models can fumble. “If you give a baby her $5,000 at birth to invest in a commission-free stock index fund, and assume a percentage of the average annual return, how much will the baby have by the time she is two years old?” Until 5 o’clock? ”
He said the large-scale language model simply starts generating text based on similar questions it has encountered in the past. “It’s not really, ‘This requires you to think very carefully and do some serious calculations before coming up with an answer,'” he explained.
But if you can “force” the model to translate that question into computer code and generate an answer based on the output of that code, you increase your chances of getting an accurate answer, he said.
Socher declined to provide details of the process, but said You.com was able to translate the questions into Python. Broadly speaking, programming “will give them more energy in terms of what they can do for years to come,” he added.
Socher’s comments come as a growing roster of large-scale language models struggle to outdo OpenAI’s GPT-4. Gemini, “Google’s most capable AI model ever” It barely outperforms GPT-4 on important benchmarks such as MMLU, one of the most common ways to assess an AI model’s knowledge and problem-solving skills. And while the main approach has been to simply scale these models in terms of the data and computational power given, Socher suggests that this approach can lead to a dead end.
“There is no more data that is very useful for training models,” he said.
