Reflections on designing equitable AI for education

Racial equity in artificial intelligence (AI) was the topic of a private roundtable discussion hosted by our colleagues at Whiteboard Advisors last week. The discussion was a timely one, given the rapid adoption of AI across education, and the launch of the new Prioritizing Racial Equity in AI Design certification by Digital Promise, in partnership with the EdTech Equity Project, in August.

CTO Dr. Amelia Kelly and CEO Dr. Martyn Farrows joined Vic Vuchic from Digital Promise and Madison Jacobs and Nidhi Hebbar from the EdTech Equity Project at the roundtable, alongside other policy, academic, and edtech experts for an in-depth conversation on the benefits and opportunities of AI, as well as how it can disproportionately penalize students of color and introduce other unfair outcomes.

Here are five important takeaways from the discussion.

1. If not designed with intention from the outset, AI in edtech can marginalize students at a massive scale.

AI holds tremendous promise in learning. Among many other applications, it can power more personalized learning paths for students and intelligent assessment tools for teachers. But it can also have unintended consequences, such as marginalizing students and increasing learning gaps.

As Vic Vuchic explained, many kids are excelling with AI-assisted education technologies, but some are also being further disadvantaged as a result of it, particularly where the AI is not able to cater to kids of different abilities and diverse backgrounds. For them, the negative implications of AI could be significant.

2. There are many ways bias can enter the AI development process

There’s a history of systemic racism that finds its way into education data.

Nidhi Hebbar of the EdTech Equity Project gave some examples of how racial bias enters each stage of AI development:

“From the very beginning, unconscious bias can impact the ideation stage of a product. Racial bias can also come into play at other stages of development, from the content and data you use to how you design the user experience to how you test your product.”

3. Designing equitable edtech products for students requires a kid-centered approach

Having an equity framework in place that covers the data collection, modeling, and infrastructure of algorithm design can help AI companies mitigate racial bias and address inequitable outcomes.

This couldn’t be more true for speech technology. In order to earn the Prioritizing Racial Equity in AI Design certification, SoapBox submitted evidence of our approach to ensure our speech technology works well for learners of color.

At the roundtable, SoapBox CTO, Dr. Amelia Kelly, spoke to the “describe the picture” data collection exercise we employ to ensure we gather authentic, spontaneous speech data from kids of all accents and dialects.

“In traditional speech data collection, if children are asked to read a sentence written by a white person, each child is going to read in the General American English (GA) vernacular the white person wrote in,” Amelia explained. “This means children who naturally speak in African American English or Latino American English will code switch to GA.”

With the “describe the picture” exercise, however, children are encouraged to speak naturally.

For example, we would show a child a picture of a cat sitting on a wall and ask them to describe what they see in the picture. This elicits responses like, “I see a cat sitting on a wall” to “I have a cat. His name is Oliver, and he likes to run up and down the stairs.”

With this approach, we collect data on children speaking in their vernacular dialect, allowing us to truly train our system and language models on different and varying pronunciations.

4. Every team member must be on board to build equitable AI

Designing equitable AI is not a one-person job: every team member must be committed to racial equity.

And an equity-centered approach to AI is an iterative process. It requires engineering teams to step into their users’ shoes and ask tough questions like “Does our AI work equally well for all users?”

Building an equitable product requires holding yourself accountable for any oversights you might have.

Dr. Amelia Kelly, CTO, SoapBox Labs

It also involves empowering teams to speak up and question their assumptions.

“Building an equitable product requires holding yourself accountable for any oversights you might have,” said Amelia. “At SoapBox, we have processes in place that allow our engineers and speech scientists to question our assumptions out loud. It’s about empowering every team member to speak up and say, ‘I don’t think this is good enough. Why don’t we explore another approach.’”

5. More awareness is needed to get edtech companies to adopt an equity-centered approach

Prioritizing equity in AI and ensuring equitable outcomes for all students is a moral imperative. But how do we incentivize best practices so edtech companies don’t see equity as a burden to their bottom line?

According to Nidhi and Madison of the EdTech Equity Project, a few key things need to happen:

1. A critical mass of edtech companies must adopt an equity-focused approach to developing solutions for others to follow suit.
2. Equitable standards need to be incorporated into the procurement processes of school systems.
3. Students and families need to be empowered to participate in organizations like the Parent Teacher Association (PTA) where some of the divisive rhetoric around equity is coming from.

Equity frameworks like EdTech Equity’s AI toolkit are a great starting point to give school and district leaders a language to talk about equity and formulate procurement policies.

“Larger school districts have been really excited about partnering with us and giving us feedback on what it actually looks like to get equity standards into the procurement process but also into the zeitgeist of the school system itself,” said Madison.

How SoapBox prioritizes racial equity

SoapBox Labs is proud to be the very first recipient of the Digital Promise and Edtech Equity Project certification for building voice technology that aims to understand every child’s voice accurately and equally, regardless of their race, background, age, or ethnicity.

To learn more about our work in equity and voice technology, contact us.