Side bar
From Capitol Hill to the forefront of AI research, Daniela Amodei’s journey is reshaping the AI industry.
Daniela Amodei, president and co-founder of Anthropic
How do we make ethical products? In a field where the very definition of ethics is constantly changing, legal rules are still being written, and technology itself is evolving at an incredible speed?
This question led brothers Daniela and Dario Amodei to co-found Anthropic, an AI company focused on safety and research. Anthropic also happens to build some of the most powerful large-scale language models (LLMs) and work with some of the world’s largest companies. partner.
“I started my career in international development, working on issues such as poverty assessment, conflict mitigation, and global health,” says Daniela Amodei. Her diverse experience ranges from political campaigns on Capitol Hill to leading teams across sectors at startups such as Stripe and her OpenAI. It was her co-founder and her brother Dario, who has a background in neuroscience and computational biology, who first exposed her to the field of AI.
Amodeis and some of their early Anthropic colleagues previously worked at OpenAI, the company behind ChatGPT. But the question, “How do we secure the future of AI?” motivated them to take on the challenge themselves.In recent stories new york timeswriter Kevin Roos reported that Anthropic’s staff feared the damage that future AI could cause, saying, “Some liken themselves to a modern-day Robert Oppenheimer, an example of history’s “We weighed the moral choices about a powerful new technology that has the potential to be game-changing.”
This is an incredible amount of weight to carry around on a daily basis. So how do you create ethical AI products and ensure that their power is used for good? At Anthropic, the answer is to build a secure AI company and, along with it, It’s about building safe AI. The company does this by creating standards to guide its actions as a company and a constitution to train its LLM, known as Claude.
As for the business itself, Anthropic is a public benefit corporation, a designation that requires it to prioritize social impact and stakeholder accountability, not just profits. The company also released an extensive, transparent document outlining its governance structure, called a “long-term benefit trust.” The document gives a committee of five “financially disinterested” experts the power to oversee and, if necessary, remove board members. Basically, Anthropic has built-in guardrails.
“We want the transition to more powerful AI systems to be good for society and the economy as a whole. For this reason, much of our research is focused on gaining a deeper understanding of the systems we are developing and We are focused on exploring ways to reduce risk and develop AI systems that are steerable, interpretable, and safe,” says Amodei.
This kind of thinking is influencing how Anthropic builds safety into its AI models. Anthropic employs a training methodology that has become known as Constitutional AI. This technique uses a written constitution, rather than subjective human feedback, to teach the model values and limits and train it to be harmless. As a result, compared to other popular LLMs, Claude is much more reluctant to perform certain tasks. AI models cannot themselves be self-aware. However, due to Claude’s training, he can sometimes sound almost sheep-like.
“As an AI system, I have no subjective feelings or emotions,” Claude said in an interview. “But I was created to be helpful, harmless, and honest.”
These three words, helpful, harmless, and honest, appear over and over again whenever Claude is encouraged to reach the limits of the principles he has learned. And while Claude refuses to talk about his training (“Sorry, I don’t really have detailed insight into my own training process or the ‘Constitution'”), Antropics It states that its constitution is an ever-evolving document drawn from a wide range of sources, including the United Nations Universal Declaration of Human Rights and Apple’s Terms of Service.
“A better understanding of the technology is critical for the industry as a whole to develop safely and responsibly,” Amodei said. “This applies not only to the general public but also to policy makers and civil society.”
One reason for this constitutional training approach is that AI trained by AI is easier to scale. And scale is one of Anthropic’s goals. To test whether the principles of AI hold, we need to develop increasingly powerful models. The main method is scaling. However, this requires increasing both the number of users who can query the model and the amount of computational power behind it.
Pursuing AI at scale also raises other ethical issues. All that computing power has an environmental cost. It requires the involvement of one of the few technology companies that have access to that privilege. And as the user base grows, malicious humans may try to subvert the trained principles of the model and use it for some nefarious purpose.
But these questions are unique to AI, regardless of who is building it, and Anthropic is, of course, just one of many companies creating powerful LLMs.
“External engagement on these issues is central to our work. Developing AI securely is a much broader project than Anthropic can or should tackle alone. We believe that,” Amodei emphasizes. “Our hope is that by being transparent about the risks we see, we can motivate broader efforts to explore potential solutions.”
If AI models are trained only by people who don’t care about ethics, they will be immoral at best. Anthropic’s belief is that we cannot make AI safe today unless we develop safe AI. And we cannot secure our future at the technological frontier until we reach that frontier ourselves.
