From her talks at the World Economic Forum to advising President Joe Biden on AI policy, researcher Joy Buolamwini has made it her mission to enlighten the public on the promises of AI technology while weighing and educating about its risks.
Buolamwini founded the Algorithmic Justice League, a nonprofit built to raise awareness about AI harms and societal impact.
She spoke with Business Insider on the AI boom, her insights on creating the future of AI, and a deeper look into her debut book “Unmasking AI: My Mission to Protect What Is Human in a World of Machines.”
The following has been edited for clarity and length.
Business Insider: It seems we’re early in how AI could be used in the public sector, as private industry appears to be hitching bets on the technology. Is there anything significant about the timing of the book? Why was now the time to tell your story and issue warnings on AI technologies?
Joy Buolamwini: Having the book come out in the fall of 2023 was important, and I pushed for it because of the awakening in terms of people’s awareness, if not full understanding, of AI.
Because these technologies are shaping everybody’s life in many important ways, I wanted a book that was an invitation to say, “You’re part of the conversation, too. Here is the context. Here’s how this technology works at a high level. But most importantly, here’s what it means to you. Here’s what it means to future and current opportunities.”
The most important point of the book was to write something that was hopeful.
It’s not just saying, “All of this is wrong with AI. There’s nothing we can do.” We’re saying, “We have identified issues in the past, and we continue to identify issues now, but we’ve also taken steps that have made a difference.”
The fact that we had major tech companies back away from selling facial recognition to law enforcement shows that we don’t have to accept this narrative of, “The technology’s already here. There’s nothing we can do.”
Overall, it’s a book of hope to say we can create a future where artificial intelligence actually benefits those who have historically been marginalized.
The other point of the book is for no one to feel like they’re somehow immune from AI harm.
The advent of generative AI underscores that when we’re talking about AI harms, this is a conversation that requires an intersectional lens, but it’s not a conversation that’s exclusive to those who are marginalized.
When we’re talking about the impact of AI and society, there are, of course, civil rights and human rights that remain center. There are also concerns about biometric rights. This is what we’re seeing with the AI voice hoax, with facial recognition in airports. And then there are the creative rights aspects as well.
If you are living on this earth, you likely will have some intersection with civil rights, certainly biometric rights, and creative rights in some way.
I also wanted people to feel the ethical dilemmas, and in this book, I talk through my own process of gathering data and wondering, “Do we actually need to get consent?” Much of what I critique is something I’ve been a part of and wrestled with.
We have to figure this out together. This isn’t about naming and shaming. The whole point has been naming and changing.
Once we name the coded gaze, we see its impact. Once we name algorithmic bias, algorithmic harm, or algorithmic discrimination, we can then begin to address it.
BI: There’s been concern that especially last year, some of the progress on racial justice and civil rights is being walked back to make up for progress gained in 2020. Is enough progress being made to counterbalance AI encroachment and other concerted efforts to undo racial equity?
JB: I’ve been encouraged by some of the work I’ve seen with the EEOC and also the Federal Trade Commission. We’re growing some teeth as to what can be done with AI, but we also see where there’s progress, there’s attack.
That means once you’re getting at systemic change, there’s always going to be some form of resistance. I see the resistance as the friction that we are pushing toward what needs to happen in society. It was never going to be easy.
It’s important that we continue to reiterate the importance of the work that we’ve been doing. One of the things I like to point out with AI systems is that if your AI systems fail on people of color, you don’t have a robust system.
I think about people’s excitement about using AI for health, for example. We know the maternal fatality rate for women of color is double and triple depending on the context and even more depending on the region.
Sometimes, there’s an excitement to say, “We can use AI to overcome human biases,” and study after study shows that if we don’t attend to the differences and are not intentional about inclusivity, we tend to revert the progress already made.
What I want us to be especially mindful of at this time when AI is being sold as solving so many problems and when there is this attack on DEI is that we don’t back down on talking about the intersection of AI and DEI.
Also, don’t fall into the trap of making it seem as if is a niche topic.
If your AI doesn’t work on the global majority, if your AI doesn’t work for women, if your AI doesn’t work for people with disabilities, once you start expanding all of the areas we’re talking about, it comes to your AI doesn’t work for humanity.
It’s easy to say we’re building things for humanity without the specificity of which humans.
The DEI effort to say we name which humans so we can be intentional about supporting others is the way that is most robust and necessary for a given context.
It’s important to continue to acknowledge the importance of building robust AI systems. You can’t build that if you don’t include the world. It’s a bit of an oxymoron for me when I see that kind of approach.
We’ve seen the universalist understanding of AI, universalist terminology, that can sound pretty good at the high level: “We want it to work for everybody. We want it to work for all. We want it to benefit humanity!” But the vagueness actually does a disservice if you’re not willing to do the work to define that humanity and who is within and who is without and how we expand our understanding of whose stories are worthy.
Who’s data is worth protecting and compensating, and what boundaries do we put in place as so many people are looking to profit off of AI capabilities?
I have those two minds. Of course, DEI is important, and it’s not some side conversation. When I talk about the coded gaze being related to the male gaze or the white gaze, this is saying that if we don’t understand our differences and attend to them the way they need to be, then we don’t make the universal progress that is often touted as the motivation of creating these technologies in the first place.
We have a false sense of progress, and when you have a false sense of progress, you will not just have a DEI lash, you’re also going to have a tech lash, an AI lash, where people start to see that these systems and these promises aren’t actually meant for them or working for them.
I fear that for the entire AI ecosystem because it then means the beneficial uses that could have been pursued, you’re going to get resistance to that as well.
BI: Turning back to the book, the part of your book that moved me to tears was the letter you wrote to Simone Biles about putting herself and her health first as you faced a similar choice. The work of fighting for equity and ethics against the tide can be emotionally and physically draining. Any advice for people to not be weary of the fight?
JB: The biggest lesson I learned is we’re fighting together, and because we’re fighting together it means you’re not fighting alone or by yourself.
It’s so easy when it feels like the work is plenty and the workers are few to take on that plentiful work in a way that then drains and re-drains you. I’ve learned and relearned that lesson many times.
You can do both. You can take care of yourself, and you can do impactful work. Specifically, as a young woman of color, that was what I was connecting with Simone Biles on.
When you’re one of one or one of few, there’s this extra pressure we put on ourselves because of the level of scrutiny we know people in our position get.
If you’re already internally a perfectionist and externally you already know there will be a micro comb through your actions, there is another type of self-monitoring and self-presentation that’s adding to the wear and tear of the work itself.
I realized that was an unfair burden I was putting on myself in reaction to a world that would not generally position me as the expert.
But then I’ve also had great experiences, for example, the AI roundtable with President Biden or even on the book tour. I was really glad that we had a conversation with Sam Altman to show there’s a way to have conversations about the future of AI in a way that’s complementary and in a manner that actually gets at the question of our differences and why they exist.
BI: How can companies at the forefront of the AI boom not be complicit in proliferating systems that reinforce racism and other problematic worldviews? Why do you think this is the status quo in tech?
JB: As you know, I work with Olay and Procter & Gamble on the #DecodetheBias campaign, and part of what came out of that was the consented data promise. To me, that’s a proactive approach of saying, how do we build consumer AI technology in a way that’s more ethical? How do we get ahead of the process of it so we’re not doing bias whack-a-mole once the product has been released?
Having seen more of these reactive approaches, the combative approach, and the proactive approach, I want to see more companies thinking through that. What do creative rights look like in terms of consent, compensation, credit, and control if you’re making these powerful AI systems?
I was talking to Sam Altman about the climate impact of AI, especially these large language models and generative AI systems. They take so much energy to not only produce the initial model, but once you’re running queries on the existing models you’re also not just leaving a carbon footprint, but there’s a water footprint as well.
The Markup did an excellent piece talking through the water footprint of different AI systems. There’s an absolute opportunity to be proactive on the climate impact side of AI. There’s an opportunity to rethink the model of compensation, consent, control, and credit when it comes to the data pipelines that are feeding within AI systems.
One of the things we’ve been doing with the AJL that we want to see companies and governments adopt is an AI harms reporting platform — some kind of mechanism that allows you to document and respond to harm.
That takes a posture of humility to say, “We’re not going to get it all right.” It’s not that we shouldn’t be proactive and we shouldn’t try, but we should also understand that given how much this technology is proliferating, we have to be able to create an ecosystem and a process that allows us to surface these harms as they’re coming.
That is another way in which companies can be more proactive, which is to acknowledge that we’re on a journey together, which means we’re not going to get it all right at the same time.
BI: Through a lens of resource dependency, where founders and technologists who represent marginalized communities aren’t necessarily getting funding or hired to Big Tech, how might people who are building inclusively make their way into the mainstream?
JB: It’s really important to find ways to own your narrative. I found when my research was first being reported on, the ways in which I would be framed. I wouldn’t be framed based on my technical expertise. I would be framed based on my identity, and it was important for me to think through the positioning.
Yes, I’m more than happy to claim my identity as a Black woman in tech, and more than happy to say that gives me a particular perspective that has informed my work. I also have four academic degrees. I am Dr. Buolamwini, and my expertise is in algorithmic discrimination and biometric technologies.
Sometimes, it’s important for people to articulate the value of what they are doing for society in a way that includes, but is not only focused on the identity piece. That’s something I had to learn early on because I would kind of see, depending on who I was talking to, these assumptions that would be made, like “this is a woman’s piece” or “this is a Black piece” and not understand that this is a conversation that’s impacting all of us.
Even when the book was coming out, I was very specific with my book theme that I should be positioned as an AI expert which I am, not this position that’s the one with the marginalized identity within the tech community.
That’s part of the story, but that’s not the lead story. The lead story is the groundbreaking research that we’ve done and the impact of the work.
The impact on legislation and policymaking, and the impact on how people even think about the way they approach their work — that’s the most exciting part for me when we get letters to the AJL saying, “Seeing your work changed how I do mine.” “I’m in a hospital system; we’re thinking about adopting AI.” “I’m at a school; we’re thinking about doing this.” This has made me rethink that process.
That sort of cultural shaping and understanding and also setting research agendas, that you even have whole schools’ research agendas that are focused on AI discrimination, that did not exist when we were first starting out on the AJL.
All of that is part of the story, and yes, it’s cool I’m a Canadian-Ghanian-American. Yes, of course, we need more women in tech, and we need more people of color in tech and more women of color in tech, especially. I’m happy to have all of those conversations. I’m very mindful of how that conversation can be used to not even address the expertise or the impact of the work you have done if you were otherwise not marked.
So it’s a balance.
BI: What’s the likelihood for the AI wave to upend the status quo of exclusionary tech?
JB: Prediction is always interesting to me because I’m so focused on agency. What is the world do we want and then how do we go build that? We are not passive actors.
What proactive steps can we take now? What can we demand of our lawmakers in terms of the protections for biometric rights? For creative rights?
What questions can we be asking? How do we exercise our current rights? Our rights for refusal.
When you go to the airport, you can say no to facial recognition scans. Most people don’t know that, and you wouldn’t see it. Even though I study this, I really have to look for that signage.
But in that kind of situation, know that you have the right to refuse. The other part, and why I wrote the book, was to also say storytelling matters and your story matters. Sharing the story of a young grad student running into issues while working on an art project and following that curiosity, why is this happening? What does it mean? What are the wider implications? We go from the white mask to The White House.
You go from school deadlines to Davos, and you’re at the World Economic Forum talking to the world leaders and really thinking through the future of this technology. And doing it in this body, in this skin, and saying, look what happens when we allow more people into the conversation.
