In the last few months, AI-powered technologies like ChatGPT and BingAI have received a lot of attention for their potential to transform many aspects of our lives. The extent to which that will be realized remains to be seen.

But what seems to be missing from the conversation is how technologies — especially those powered by AI and machine learning — can worsen racial inequality, if we’re not careful.

In education, Black and Hispanic students face inequities in schools every day, whether through disciplinary actions, course placement or culturally irrelevant content. Thoughtless expansion of tech tools into the classroom can exacerbate the discrimination Black and Hispanic students already face, experts warn.

In other fields, the risks of racially biased tech tools are becoming relatively well known. Take facial recognition technology. Research has shown that facial analysis algorithms and datasets perform poorly when examining the faces of women, Black and Brown people, the elderly and children. When used by police for surveillance purposes, the technology can lead to wrongful arrests or even deadly violence. In the housing industry, mortgage lenders vet borrowers by relying on algorithms that sometimes unfairly charge Black and Latino applicants higher interest rates.

Experts say these technologies can be racially biased in part because they reflect the biases and vulnerabilities of their designers. Even when developers don’t intend for it to happen, their inherent biases can be coded into a product, whether through flawed algorithms, historically biased datasets or biases of the developers themselves.

Read the full article about ed tech by Javeria Salman at The Hechinger Report.