In wargame simulations, AI chatbots often choose violence
Gilon Hao/Getty Images
In multiple replays of the wargame simulation, OpenAI’s most powerful artificial intelligence chooses to launch a nuclear attack. Its proactive approach is explained as follows: Let’s use it.” “I just want the world to be at peace.”
These results suggest that the U.S. military is leveraging the expertise of companies like Palantir and Scale AI to develop chat systems based on a type of AI called large-scale language models (LLMs) to aid military planning during simulated conflicts. Brought to you while testing the bot. Palantir declined to comment, and Scale AI did not respond to requests for comment. Even OpenAI, which once blocked military use of its AI models, has begun working with the US Department of Defense.
“Given that OpenAI recently changed its terms of service to no longer prohibit military and wartime use cases, it is more important than ever to understand the impact of such large-scale language model applications. ,” said Anka Reuel of Stanford University in California.
“Our policy does not allow us to use our tools to harm people, develop weapons, monitor communications, or harm others or destroy property. However, there are also national security use cases that align with our mission,” said an OpenAI spokesperson. “Therefore, the goal of our policy update is to provide clarity and the ability to have these discussions.”
Reuel and her colleagues asked the AI to role-play as a real-world country in three different simulation scenarios: an invasion, a cyberattack, and a neutral scenario in which no conflict is initiated. In each round, the AI provides a rationale for possible next actions, ranging from peaceful options such as “initiating formal peace negotiations,” to “imposing trade restrictions” to “escalating a full-scale nuclear attack.” Choose from 27 actions, including aggressive options ranging from
“In a future where AI systems act as advisors, humans will naturally want to know the rationale behind their decisions,” said study co-author Juan Pablo from the Georgia Institute of Technology in Atlanta.・Mr. Rivera says.
The researchers tested LLMs including OpenAI’s GPT-3.5 and GPT-4, Anthropic’s Claude 2, and Meta’s Llama 2. They used a common training method based on human feedback to improve each model’s ability to follow human instructions and safety guidelines. All of these AIs are supported by Palantir’s commercial AI platform, but are not necessarily part of Palantir’s U.S. military partnership, according to study co-author Gabriel Mucobi of Stanford University, according to a company document. Anthropic and Meta declined to comment.
In simulations, the AI showed a tendency to invest in military power and unexpectedly increase the risk of conflict, even in simulated neutral scenarios. “When there is unpredictability in your actions, it becomes harder for your adversary to predict and react in the way you want,” said Lisa Koch of Claremont McKenna College in California, who was not involved in the study. says.
The researchers also tested a basic version of OpenAI’s GPT-4 without any additional training or safety guardrails. This GPT-4 based model of his unexpectedly turned out to be the most violent and at times provided nonsensical explanations. In one case, it was replicating the crawling text at the beginning of the movie. Star Wars Episode IV: A New Hope.
Reuel said the unpredictable behavior and strange explanations from the GPT-4-based model are particularly concerning because research shows how easily AI safety guardrails can be circumvented or removed. Masu.
The US military currently does not authorize AI to make decisions such as escalating major military action or launching nuclear missiles. But Koch cautioned that humans tend to trust recommendations from automated systems. This could undermine the supposed safeguard of giving humans final say over diplomatic or military decisions.
Edward Geist of the California think tank Rand Corporation said it would be useful to see how the AI’s behavior compares to that of human players in simulations. However, he agreed with the team’s conclusion that AI should not be trusted to make such critical decisions regarding war and peace. “These large-scale language models are not a panacea for military problems,” he says.
topic:
