How User Experience Research (UXR) Cam Help Build Responsible AI Applications

Today, Artificial Intelligence (AI) and, more specifically, Machine Learning are pervasive in our daily lives. From Facebook ads to YouTube recommendations, from Siri to Google Assistant, and from an automated translation of device notice to marketing personalization tools, AI now deeply permeates both our work and personal lives.

This article is the second in a series of three (catch up on the first installment here and third here) that advocate for renewed UX research efforts in ML apps.

With ML facing so many users, there is a case to approach the conception and design of ML-powered applications from a UX research perspective.

This lies on three main reasons:

1. Mental models of users haven’t caught up with how ML and AI truly work:

• UXR can uncover existing mental mentals and help design new ones that are more suited to this new tech.

2. ML and AI can have an insidious and deep impact on all users’ lives:

• UXR reveals the myriad of intended and unintended effects of apps on people’s life — and help build more ethical AI.

3. ML and AI can have disparate impacts on individuals based on their ethnicity, religion, gender, sexual orientation:

• UXR can also help address some of the sources of bias.

In this episode, we will focus on the second reason:

How to assess the impact of ML-apps on users and meet AI ethical standards?

Most ML algorithms are supposed to assist humans in their decision-making process but not make the decision themselves. However, more and more, AI systems do not content themselves in making recommendations — they make decisions. This is the case from sifting through resumes to selecting which neighborhoods to patrol for the next policy shift.

Given the scale AI systems operate, the potential impacts on specific individuals, groups, or even society, is deep and wide. While harmful human practices have always existed, they have evolved alongside social and legal guidelines that mitigate them. Not so much for AI-driven systems — yet.

Ethics in AI

Research on ethical AI has been conducted by academic researchers, industries, or governments, and they have produced a series of guidelines.

Microsoft lists 6 principles for a responsible AI: Fairness, reliability & safety, privacy & security, inclusiveness, transparency, and accountability.

And a High-Level Expert Group on Artificial Intelligence published Ethics Guidelines for Trustworthy AI for the European Union in 2019 that explains:

Develop, deploy and use AI systems in a way that adheres to the ethical principles of: respect for human autonomy, prevention of harm, fairness and explicability. Acknowledge and address the potential tensions between these principles.

While the research on these high-level principles is well engaged, there is a gap between policies and guidelines and their implementation. When should data scientists concern themselves with these considerations? How can product managers integrate them into their roadmap? What practical steps can they take to ensure their app will be responsible and trustworthy?

The Partnership for AI, a non-profit that includes industry leaders, universities, and civil society groups, acknowledges that all AI stakeholders need to be actively involved in preventing potential harm resulting from such research:

Through our work, it has become clear that effectively anticipating and mitigating downstream consequences of AI research requires community-wide effort; it cannot be the responsibility of any one group alone. The AI research ecosystem includes both industry and academia, and comprises researchers, engineers, reviewers, conferences, journals, grantmakers, team leads, product managers, administrators, communicators, institutional leaders, data scientists, social scientists, policymakers, and others.