We train AI, and it may be training us, University of Washington study finds

And not only will humans change, but those changes may last into the future, creating new habits in their human trainers, and potentially even changing their behavioral tendencies and biases, even if they are not aware of them.

So, who’s training who here?

“This study clearly demonstrates that we need to understand the behavior of people interacting with AI, especially those helping to train the tools, so that we can measure and mitigate bias,” said Dr. Philip R. O. Payne, director of the University of Washington’s Information Sciences Institute and professor of medicine.

But if human behavior is used to train an AI, there may be downsides to improving it. For example, when humans help train an AI for a self-driving car, they might drive extra carefully, says lead researcher Lauren Treiman, a graduate student at the University of Washington. That could make the AI ​​the perfect driver.

But in a place like St. Louis, where running yellow lights is common, it could be dangerous for self-driving cars to try to be perfect drivers.

“If people are inclined to do that, then AI may need to learn how to handle yellow cards,” she said.

Treiman said he first started thinking about how AI is trained and the influence of the algorithms that determine what we see online while scrolling through his social media feeds.

“The algorithms are very sensitive when it comes to ‘recommended’ videos,” she says. “You sit on something for a few seconds and then you see the same content over and over again.”

She said she purposely swipes past something quickly or doesn’t click at all so that social media algorithms can learn to show them less of that kind of content.

The experience of changing one’s own behavior in response to AI was what inspired the experiment.

The researchers used a “classic academic approach” called the “Ultimatum Game,” according to Wouter Kuhl, an assistant professor of psychological and brain sciences at the University of Washington.

The Ultimatum Game has two players. One player decides how to split $10 between them, and the other makes an offer that can be accepted or rejected. If the offer is accepted, each player gets the amount they agreed to split. However, if the offer is rejected, neither player gets any money.

In the study, some players were told that the way they played the game would be used to teach an AI how to play. They were informed of this by a small webcam icon in the corner of their screen and the words “Offers will be used to train AI.”

The researchers found that people who play to train an AI tend to reject offers that are unfair. Although rejecting an offer reduces the player’s monetary gain at the end of the game, people who play to train an AI rejected offers much more frequently than those who don’t. It appears that people want to train the AI ​​to play fairly, and they change their behavior to satisfy that desire.

But participants were told they would later play an ultimatum game with the AI. Was they training the AI ​​fairly for the sake of fairness, or was they training it to get better luck in the next round?

To answer this question, the researchers set up the same experiment, but this time told participants that they would be training an AI that would play against someone else in the next round. The result was the same: people trained the AI ​​to play fairly, even if it cost them money and even if they would never play against that AI in the future.

Finally, the researchers investigated whether the feeling of being observed encouraged participants to play fairly.

“This is an old problem in research,” Payne says: “People know they’re being observed, and that influences what[researchers]observe.”

So the researchers removed the webcam icon from the AI ​​training group’s screens to reduce any sense of being watched.

Still, “people were willing to sacrifice rewards to make the AI ​​fairer,” says Chengju Ho, a researcher and assistant professor of computer science and engineering at the University of Washington.

All these experiments provide strong evidence that behavior changes when humans train AI, but all three researchers agreed that what’s most interesting is how long those behavioral changes last.

The first experiment only took about five minutes, Kuhl said, and participants returned a few days later, playing just like they had in the first session, even after being explicitly told they were no longer training the AI.

“When we looked at the behavior in the second session, we saw really beautiful and clear patterns of behavior persistence,” Kuhl said.

Treiman said the findings could have implications for how AI is trained and also raise other questions.

“In the Ultimatum game, the definition of fairness is clear. But in many other contexts, the definition isn’t clear,” Treiman says. “You always want an AI that’s as fair and honest as possible. But in other contexts, what is fair? Especially since people will inject their own preferences, biases, and beliefs into AI systems that are being trained.”