New Artificial Intelligence Research: Why can’t anyone agree on how dangerous AI is?

I’ve written a lot about AI and the debate over whether it could kill us all. But I’m still not sure where I’ll end up.

Some people with a deep understanding of advanced machine learning systems believe that the systems will become increasingly out of control, perhaps going “out of control” and threatening humanity with catastrophe and extinction. . Others have a deep understanding of how these systems work, that we have complete control over them, that the dangers do not include the extinction of humanity, and that the first group is hysterical. There are people who think it’s full of alarmists.

How do we tell who is right? We certainly don’t know.

But a clever new study by the Prediction Institute attempts to find out. The authors (Josh Rosenberg, Ezra Karger, Avital Morris, Molly Hickman, Rose Hudshire, Zachary Jacobs, and The Godfather’s Philip Tetlock) have previously worked with experts on AI and other existential risks to We asked both “super forecasters” who have a track record of Predict upcoming world events and assess the risks posed by AI.

result? The two groups had very different opinions. The experts in the study were generally much more nervous and far more likely to experience disaster than the super forecasters.

The researchers wanted to know why the opinions of these groups differed so widely. Therefore, the authors set “hostile cooperation.” They had both groups spend long hours (median 31 hours for experts and 80 hours for super forecasters) reading new material and, most importantly, discussing these issues with members of society. An opinion contrary to the moderator. The goal was to see if exposing each group to more information and the best arguments of the other group would change either of their minds.

The researchers were also interested in finding the “core,” or issues that help explain people’s beliefs, and what new information might change people’s minds. For example, one of the biggest cruxes was, “What’s going to happen to METR?” [an AI evaluator] Or will similar organizations discover evidence that AI has the ability to autonomously replicate, acquire resources, and avoid shutdowns by 2030?” The answer may be yes. If found out, skeptics (super-forecasters) say they will become even more concerned about the risks of AI. AI pessimists say they will become even more optimistic if the answer is no.

So, did everyone converge on the correct answer? …No, things weren’t supposed to go so easily.

AI pessimists have lowered the probability of a catastrophe occurring before 2100 from 25% to 20%. Optimists raised it from 0.1% to 0.12%. Both groups remained close to the starting point.

However, this report is still interesting. This is a rare attempt to bring together wise and knowledgeable people with differing opinions.While doing so, it was not solve This disagreement shed a lot of light on where the point of split came from.

Why people disagree about the dangers of AI

The paper argues that either AI will wipe out humanity, the human population will decline to less than 1 million people over a million years, or global GDP will decline to $1 trillion (less than $1 trillion). It focuses on disagreements over the potential for “collapse.” % of its current value) which he will hold for over 1 million years. At the risk of being wild, I think you can summarize these scenarios as “extinction, or at best, hell on earth.”

Of course, there are many other risks with AI that are worth worrying about, many of which we already face today.

Existing AI systems can exhibit alarming racial and gender bias. In any case, relying on them can be unreliable enough to cause problems. It can be used for bad purposes, such as creating fake news clips to deceive the public or creating pornography using the faces of people who don’t consent.

However, while these harms are certainly bad, they are clearly insignificant compared to “the AI ​​loses control and everyone dies.” The researchers decided to focus on extreme existential scenarios.

So why do people have different opinions about the likelihood of these scenarios coming to pass? It’s not because they have different access to information or lack of exposure to different perspectives. If so, adversarial cooperation, consisting of new information and massive exposure to opposing views, would have moved people’s beliefs more dramatically.

Also, interestingly, much of the disagreement about what will happen with AI in the coming years could not be explained by different beliefs. When researchers paired optimists and pessimists to compare the odds of a catastrophe, the average difference in probability was 22.7 percentage points. For the most informative “core” (where AI evaluators found that the model had very dangerous capabilities before 2030), the gap narrowed by only 1.2 percentage points.

The short-term timeline is not empty, but that is not where the main disagreement lies.

What seemed important was long term future. AI optimists generally believed that building his human-level AI would take longer than pessimists thought. As one optimist told researchers, “Language models are just models of language, not digital hyperhumanoid Machiavellians working for their own ends.” This optimist believed that reaching human-level intelligence would require a fundamental breakthrough in machine learning methods.

Many cited the need for robotics, not just software AI, to reach human levels, arguing that it would be much more difficult to achieve. Writing code and text on a laptop is another thing. It’s quite another to learn as a machine to flip pancakes, sweep tile floors, or many other physical tasks where humans outperform robots.

When the conflict deepens

The most interesting cause of the divide the researchers identified was what they called “fundamental worldview disagreements.” This is a fancy way of saying that they disagree about where the burden of proof lies in this debate.

“Although both groups agree that ‘extraordinary claims require extraordinary evidence,’ they differ on which claims are abnormal,” the researchers conclude. “Is it abnormal to believe that AI will wipe out all humans when humans have existed for hundreds of thousands of years, or is it abnormal to believe that humanity will continue to survive with AI that is smarter than humans? ”

That’s a fair question! In my experience, most non-AI laypeople consider the claim that “machines will kill us” to be more unusual. But I can see where the pessimists are coming from. Their basic idea is that the emergence of superhuman AI will be like the arrival of superhuman alien species on Earth. We don’t know if that species is going to kill us all.

However, this was not always the case with Homo sapiens. want Hundreds of thousands of years ago, when multiple intelligent species of great apes coexisted, the idea would have been to kill off all Homo erectus and Neanderthals. But we killed them all.

The emergence of smart species that are better at claiming resources for themselves tends to cause extinction of dumber and weaker species. If you hold this worldview, the burden of proof is on the optimist to show why superintelligent AI will emerge. probably won’t It will lead to disaster. Or as one pessimist in the study put it: “There are many possibilities for this situation to occur, but very few leave humans alive.”

This is not the most promising conclusion the study has reached. Disagreements caused by specific differences in opinion about what will happen in the next few years are easier to resolve. The disagreement is based not on deep and hard-to-change differences in people’s assumptions about what the next few years will be, but on what the next few years will be. About how the world works and where the burden of proof should lie.

The paper reminded me of an interview I once saw with the late philosopher Hilary Putnam. Putnam, active in the late 20th century, believed that philosophy could make progress even if there were big questions about what truth was. How does the mind work? Is there an external reality that we can grasp? — I still find it difficult to answer.

Certainly, we don’t know those answers, Putnam said. But we know more about that question. “We learn more about how difficult they are and why they are so difficult. That’s probably the lasting philosophical advance.”

That’s how I felt when I read the paper from the Prediction Research Institute. I feel like I’m not sure how much we need to worry about AI. But I think I understand a little more about why this is such a difficult question.

A version of this story was originally future perfect Newsletter. Register here!