Ethics for designers in the real world: Snapchat’s Juneteenth filter

Growing up a white kid in the south, Juneteenth was on my list of ‘other kids’ holidays.’ Some kids had Ash Wednesday and All Saints’ Day, some kids had the first day of deer season, and some kids had Juneteenth. Juneteenth had the added obscurity of happening outside of the school year; there were no classroom decorations, special lessons, or school plays to give me any hints as to what it was about. That’s why I knew more about Kwanzaa than I did Juneteenth.

This year I celebrated Juneteenth for the first time. A lot of white people did. A lot of companies and brands did, too. Some made Juneteenth a paid holiday. Others donated their proceeds to charities, or used their platforms to promote Black-owned business. Snapchat — and Snap, its parent company — celebrated by releasing a filter.

The backlash was immediate. The “smile to break the chains” filter wasn’t just a minor moment of tone-deafness from Snap — it was a burning dumpster in the town square. The company apologized, and took the filter down. But the damage was done.

How could this have happened? Where did Snap go wrong?

Ethics often seems like other kid’s holidays, an abstract concept you only hear about in school. Sometimes, though, you get to see how ethics impacts peoples’ lives in real time. By applying some of the ideas from Ethics for designers, we can learn a lot from Snap’s mistakes, and make our own design process more ethically rigorous.

Snap and Deontology

Oona King, Snap’s vice president of diversity and inclusion, sent an apology email to employees the weekend after Juneteenth.¹

A few passages are worth exploring in detail (emphasis is King’s).

Snap released a Lens to commemorate Juneteenth that many people felt was offensive because it prompted users to ‘smile’ to break the chains of slavery. Snap was also accused of failing to include Black perspectives in the creation of our Lens to mark Juneteenth — a date often celebrated by African-Americans to mark the end of slavery. After reviewing how the process unfolded, it’s very clear that Black Snap team members were fully involved in every stage of developing and approving the Lens and that, in hindsight, we should have developed a more appropriate Lens.

… We reviewed the Lens from the standpoint of Black creative content, made by and for Black people, so did not adequately consider how it would look when used by non-Black members of our community. What we also did not fully realize was a) that a ‘smile’ trigger would necessarily include the actual word “smile” on the content; and b) that people would perceive this as work created by White creatives, not Black creatives.

… The mischaracterization on social media — that White executives at a tech company failed, yet again, to include Black perspectives — is completely untrue. What is true is that regardless of our diverse backgrounds, we are all human, and humans make mistakes.

King’s focus throughout the letter is on the process that Snap used to create the filter. In ethics terms, this is a deontological mindset.

“Deontology” is a really unwieldy word (if I had a magic ethics wand, I would change it to something more memorable). In Ethics for designers part III, I defined deontology like this:

According to deontology, ethics is all about how you act. Deontologists search for the right set of rules to define and govern moral behavior … The deontological view is that logic and reason are all you need to determine whether actions are right or wrong.

King’s response to critics is that the process isn’t the problem. "Black Snap team members were fully involved in every stage,” she says, explaining that the people who suggested the “smile” trigger were Black — in fact, there were White team members who raised concerns. The work was seen “from the standpoint of Black creative content, made by and for Black people.”

Snap and Consequentialism

But if Snap’s process was logical and thoughtful, what went wrong? If Snap did everything right, there would be no news stories, no apology letter, no reaction videos on social media. It’s ethics in motion: in the messy world of real decision making, you can’t figure out the right course of action just by following a logical process. That’s why it’s important to understand consequentialism.

Consequentialism differs from deontology in almost every way. Again, from Ethics for designers:

Deontology says that there is a logically consistent set of rules that can determine whether some act is ethical or not. Consequentialism says “it depends.” A rule that looks ethical in one case might result in unethical outcomes in another case. Consequentialism is based on an intuition: the best action now is whatever makes the world best in the future.

Designing its Juneteenth filter, Snap followed an ethical-sounding set of rules (call them maxims if you’re feeling saucy). Chief among these was the rule to include Black perspectives in the making of new technology and creative content. That’s a great rule. Everyone should follow it. Snap’s mistake was their belief that following this rule would guarantee an ethical outcome.

If this was one isolated incident, it might be easier to chalk it up to Snap’s learning process. But Snap has been here before. In 2016, it released a filter for 4/20 that mapped Bob Marley’s face onto users. Like with its Juneteenth filter, Snap included Black perspectives in its creation, this time working with the Marley estate. And like the Juneteenth filter, the backlash was swift: it was immediately called out as digital blackface.²

Later that year, Snap released and quickly recalled a filter that painted squinted eyes and rosy cheeks onto users, distorting their face into a caricature reminiscent of racist portrayals of Asians. A Snap spokesperson took the deontological view: “Lenses are meant to be playful and never to offend.”³

When it comes to Snapchat’s filters, consequences matter.

Learning from Snap’s mistakes

Just because Snap included Black perspectives in its creative process did not guarantee their output wouldn’t be harmful to Black users. There are a few ways we can learn from this outcome to make our own design process more ethically-aware.

1. Be aware of the assumptions in your process (and correct them if they’re wrong)

Snap’s example isn’t a death knell for deontology.⁴ A defender of deontology might say, “Snap should have followed a different set of rules. That would have guaranteed ethical outcomes.”

So, what rules can we introduce to improve Snap — and, by extension, our own work?

Here’s a start: your users will be the ultimate judge of your work. Bring them into the design process — especially users who are especially at risk of being harmed, like marginalized and under-represented groups.

2. Everyone makes mistakes (virtue alone won’t save you)

King’s recognizes that Snap’s Juneteenth filter was a mistake. She also highlighted the pain of being accused of racism.

King has previously discussed her own experiences with racism as a Black Jewish woman.⁵ She was the second black woman elected to Parliament, and has made a career of championing diversity in organizations like the UK’s Channel 4 and YouTube.⁶ But what she said in her email to Snap employees is true: we’re all human, and we all make mistakes.

Nobody can be perfectly virtuous.⁷ No set of rules will work in every case. It’s important to recognize and plan for that fact. That’s why it’s important to build out your ethical toolkit: practices like ethical premortems allow you to play out the possible consequences of your decisions, no matter how unlikely. For a good starting point, check out Shannon Vallor’s Ethical Toolkit for Engineering/Design Practice.

3. Measure what matters (and hold yourself accountable)

Many large tech companies report their own diversity data: Google, Facebook, and Twitter all publish a yearly report of their staff broken down by gender and ethnicity. But not Snap. Snap’s CEO Evan Spiegel said in an interview with CNBC that “Snapchat looks like most other technology companies in terms of representation,” but that he wouldn’t release the data. Instead, Snap is “inventing” a new way to release its diversity reports.⁸

The influential management thinker Peter Drucker was an avid proponent of the power of measurement. But one of his most influential quotes of all time “if you can’t measure it, you can’t manage it,” is a misquote; he never actually said it. His take on measurement was much more nuanced:

Work implies not only that somebody is supposed to do the job, but also accountability, a deadline and, finally, the measurement of results — that is, feedback from results on the work and on the planning process itself.⁹

Measuring the ethical impact of your work, and being accountable to that measurement, is part of designing for equitable outcomes. Snap does measure the diversity of their workplace, and makes that data available to its own employees. ¹⁰ But when it comes to diversity, lack of public accountability takes all the force out of measurement.

Snap is just one case for ethically-aware design among many; we can learn a lot from analyzing what went wrong. Comparing consequentialism to deontology shows that in Snap’s case, the Juneteenth filter wasn’t just a mistake, but the result of an ethically unsound decision making process, one that didn’t anticipate the harm it would cause to users. Using tools like ethical accounting and premortems can help design teams avoid the mistakes that Snap seems to be repeating over and over.

And reading case studies like this will hopefully mean there are less of them written in the future.

Special thanks to Josh Petersel for feedback on an earlier version of this essay.