Wizard of oz

 

Summary

The design of conversational interfaces can be experimented via a Wizard of Oz study, in which users interact with a system that they believe to be autonomous but is actually being operated by a hidden human, called the Wizard.

 

Type of pattern

  • Process

 

Type of objective

  • Trustworthiness

 

Target users

  • UX designer

 

Impacted stakeholders

  • Development teams
  • Users

 

Lifecycle stages

  • Design

 

Relevant AI ethics principles

  • Transparency & explainability

 

Context

While AI systems are increasingly affecting people’s daily life, their decision-making process is opaque to end users. The end users often do not understand how decisions are made by AI systems and are not aware of the capabilities or limitations of the AI systems. The missing explainability may lead to a lack of trust and has been identified as one of the most urgent challenges to be resolved. The existing explainable AI (XAI) solutions tackle the challenge from an AI-centred view by providing one-way flow of key information of AI solutions.

 

Problem

How can we improve the explainability of AI systems?

 

Solution

XAI can be viewed as a human-AI interaction problem and achieved by conversational interfaces for communication between users and AI systems. The design of conversational interfaces can be experimented via a Wizard of Oz study, in which users interact with a system that they believe to be autonomous but is actually being operated by a hidden human, called the Wizard. The conversation data is collected and analysed to understand requirements for a self-explanatory conversational interface.

 

Benefits

  • Improved human-AI interaction: The Wizard of Oz study learns about users’ behaviors which is helpful to improve the design of conversational interfaces.
  • Ease of use: The Wizard of Oz experiment is easy to set up.

 

Drawbacks

  • Limited by operator’s knowledge and background: Although instructions and sample answers are provided for the operator (i.e., the Wizard), the study quality is limited by operator’s knowledge and background (e.g., language).

 

Related patterns

TBD

 

Known uses

TBD