Responsible AI Pattern Catalogue

Fig.1. Overview of Responsible AI Pattern Catalogue.

Motivation

Artificial Intelligence (AI) has been transforming our society and listed as the top strategic technology in many organizations. Although AI has huge potential to solve real-world challenges, there are serious concerns about its ability to behave and make decisions in a responsible way. Compared to traditional software systems, AI systems involve higher degree of uncertainty and more ethical risk due to its autonomous and opaque decision making. Responsible AI is the practice of developing and using AI systems in a way that benefits to individuals, groups, and the wider society, while minimizing the risk of negative consequences.

The concept of responsible AI has attracted huge attention from governments, organizations, and companies. According to the 2022 Gartner CIO and Technology Executive Survey, 48% organizations have already adopted or plan to adopt AI technologies within the next 12 months while 21% of organizations have already deployed or plan to deploy responsible AI technologies within the next 12 months. Responsible AI has been widely considered as one of the greatest scientific challenges of our time and the key to unlock the market and increase the adoption of AI.

To address the responsible AI challenge, a number of AI ethics principles frameworks (e.g., Australia’s AI Ethics Principles) have been published recently, which AI systems are supposed to conform to. There have been a consensus around the AI ethics principles. A principle-based approach allows technology-neutral, future-proof and context-specific interpretations and operationalization. However, without further best practice guidance, practitioners are left with nothing much beyond truisms. For example, it is a very challenging and complex task to operationalize the the human-centered value principle regarding how it can be designed for, implemented and monitored throughout the entire lifecycle of AI systems. In addition, significant efforts have been put on algorithm-level solutions which mainly focus on a subset of mathematics-amenable ethical principles (such as privacy and fairness). However, ethical issues can occur at any step of the development lifecycle crosscutting many AI, non-AI and data components of systems beyond AI algorithms and models. To try to fill the principle-algorithm gap, further guidance such as guidebooks, questions to generate discussions, checklists and documentation templates do start to appear. Those efforts tend to be ad-hoc sets of more detailed prompts for practitioners to think about all the issues and come up with their own solutions.

Overview of Responsible AI Pattern Catalogue

We adopt a pattern-oriented approach and build up a Responsible AI Pattern Catalogue, as illustrated in Figure 1, for operationalizing responsible AI from a system perspective. In software engineering, a pattern is a reusable solution to a problem commonly occurring within a given context in software development. Many solutions contribute to multiple responsible AI principles. Rather than staying at the ethical principle or algorithm level, we focus on patterns that practitioners and broader stakeholders can undertake to ensure that responsible AI systems are responsibly developed throughout the entire lifecycle with different levels of governance. To describe the pattern, we created a template, including summary, type, objective, target users, impacted stakeholders, relevant principles, context, problem, solution, consequences, related patterns, known uses. The approach has the following characteristics.

  • Across multiple organization levels and connected – Industry/community, organization, and teams. The patterns we are introducing here are at different levels, so you can situate your practice areas in the bigger picture and see how you fit in and how different practices and patterns influence and reinforce each other from a team, organization, and industry or community level.
  • Across multiple angles – governance, process, and product. Not only should you use product patterns to directly enforce responsible AI principles within the AI product and verify or validate them, but you should also incorporate process and governance patterns to complement these efforts further:
    • Governance patterns for establishing multi-level governance for responsible AI;
    • Process patterns for setting up trustworthy development processes;
    • Product patterns for building responsible-AI-by-design into AI systems.
  • Across system life cycle and connected – Requirements, design, implementation, testing, deployment, and post-deployment monitoring. Across the lifecycle of AI systems, different patterns and practices can be applied at other times, with the outputs of one practice becoming the input of another.
  • Across the supply chain, system, and operation layer and connected. We connect most of the patterns through a system reference architecture across AI supply chain, AI system, and operation/deployment infrastructure layer.
  • Benefiting multiple connected risks. Individual responsible AI risks should be managed in silos by using risk-specific solutions. The patterns in this book often help multiple risks together to raise the responsible AI posture of the organization significantly.
  • Acknowledging drawbacks and additional risks introduced. Adopting pattern-oriented risk mitigation may introduce additional risks and costs. We recognize them by incorporating drawbacks in the patterns and connecting with other related ways to tackle the challenges further.
  • Clear differentiation of trust and trustworthiness. We recognize that the importance of gaining stakeholder trust goes beyond the objective trustworthiness of the systems. Gaining trust is about diverse and inclusive engagement, setting realistic expectations, and communicating trustworthiness evidence in a way that stakeholders can understand and meaningfully critique. We include trust and trustworthiness dimensions in our patterns.

We hope these multi-layer, multi-aspect, multi-stage, and connected patterns can help you better navigate the landscape and achieve responsible AI systems more successfully.

Fig.2. AI system stakeholders.

AI System Stakeholders

As illustrated in Fig. 2, AI system stakeholders are classified into three groups:

  • Industry-level stakeholders
    • AI technology producers: Those who develop AI technologies for others to build on top to produce AI solutions (e.g., parts of Google, Microsoft, and IBM). AI technology producers may embed RAI in their technologies and/or provide additional RAI tools.
    • AI technology procurers: Those who procure AI technologies to build their in-house AI solutions (e.g., companies or government agencies buying and using AI platforms and tools). AI technology procurers may care about RAI issues and embed RAI into their AI technology procurement process.
    • AI solution producers: Those who develop in-house and blended solutions on top of technologies and need to make sure the solutions adhere to RAI principles, standards, or regulations (e.g., parts of MS/Google providing Office/Gmail “solutions”). AI solution producers may offer the solutions to AI consumers directly or sell to others. They may use RAI tools (provided by tech producers or third parties) and RAI processes during their solution development.
    • AI solution procurers: Those who procure complete AI solutions (with some further configuration and instantiation) to use internally or offer to external AI consumers (e.g., a government agency buying from a complete solution from vendors). They may care about RAI issues and embed RAI into their AI solution procurement process.
    • AI users: Those who use an AI solution to make decisions that may impact a subject (e.g., a loan officer or a government employee). AI users may exercise additional RAI oversight as the human in the loop.
    • Investors: Those who have interests or concerns in the responsible development and use of AI, which can influence a company’s performance and risk profile.
    • AI impacted subjects: Those who are impacted by some AI-human dyad decisions (e.g., a loan applicant or a taxpayer). AI impacted subjects may contest the decision on dyad AI ground.
    • AI consumers: Individuals who consume AI solutions (e.g., voice assistants, search engines, recommender engines) for their personal use (not affecting third parties). AI consumers may care about the dyad AI aspects of AI solutions.
    • RAI governors: Those who set and enable RAI policies and controls within their culture. RAI governors could be functions within an organization in the preceding list or external (regulators, consumer advocacy groups, community).
    • RAI tool producers: Technology vendors and dedicated companies offering RAI features integrated into AI platforms or machine learning operations (MLOps) or AI for operations (AIOps) tools.
    • RAI tool procurers: Any of the preceding stakeholders who may purchase or use RAI tools to improve or check solutions’ or technology’s RAI aspects.
  • Organization-level stakeholders
    • Management teams: Individuals at the higher level of an organization who are responsible for establishing an RAI governance structure in the organization and achieving RAI at the organization level. The management teams include board members, executives, and (middle-level) managers for legal, compliance, privacy, security, risk, and sustainability.
    • Employees: Individuals who are hired by an organization to perform work for the organization and are expected to adhere to RAI principles in their work.
  • Team-level stakeholders
    • Development teams: those who are responsible for developing and deploying AI systems, including product managers, project managers, team leaders, business analysts, architects, UX/UI designers, data scientists, developers, testers, and operators. The development teams are expected to implement RAI in their development process and embed RAI into the product design of AI systems.

Governance Patterns

We identify a set of governance patterns and classify them into industry-level governance patterns, organization-level governance patterns, and team-level governance patterns (see Fig.3). The target users of industry-level governance patterns are RAI governors, while the impacted stakeholders include AI technology producers and procurers, AI solution producers and procurers, RAI tool providers and procurers. For the organization-level patterns, the target users are the management team and the impacted stakeholders are employees. The target users of team-level patterns are the development team.

Fig.3. Governance patterns for responsible AI.

Industry-level governance patterns

Organization-level governance patterns

Team-level governance patterns

Process Patterns

We identify process-oriented patterns (i.e. best practices) that can be incorporated into development processes, so the developers could consider to apply them during the development lifecycle. Fig.4 describes the software development lifecycle and the potential ethical risks and breaches corresponding to each stage, while Fig.5 presents the summarized patterns for different stages.

Fig.4. Development process lifecycle and potential ethical risk.

Fig.5. Process patterns for responsible AI system development.

Patterns for requirement stage

Patterns for design stage

Patterns for implementation stage

Patterns for testing stage 

Patterns for operation stage

Product Patterns

Product patterns provide a system-level guidance on how to design the architecture of responsible AI systems. Responsible-AI-by-design can be built into AI systems through the product patterns. Broadly, an AI system is comprised by three layers, including the supply chain layer that generates the software components which compose the AI system, the system layer which is deployed AI system, and the operation infrastructure layer that provides auxiliary functions to the AI system. Fig.6 presents the identified products patterns for each of the three layers. Those product patterns can be embedded into the AI ecosystems as product features. Fig.7 illustrates a state diagram of a provisioned AI system and highlights the patterns associating with relevant states or transitions, which show when the product patterns could take effect. Fig.8 gives a pattern-oriented responsible-AI-by-design reference architecture.

Fig. 6. Product patterns for responsible-AI-by-design architecture of an AI system.

Fig. 7. Product patterns for responsible-AI-by-design.

Fig. 8. Pattern-oriented responsible-AI-by-design reference architecture.

Supply chain patterns

System patterns

Operation infrastructure patterns

HAI interaction patterns

Principle-specific patterns

Fairness

Privacy

Explainability

Mapping RAI Pattern Catalogue to Regulations/Standards

The RAI Pattern Catalogue is mapped to EU AI Act and ISO/IEC 42001:2023 Standard.

Please refer to: mapping – Pattern Catalogue

Our Papers

Responsible AI Pattern Catalogue and Question Bank:

Software Engineering for Responsible AI:

Architecture Design for Foundation Models based AI Systems:

Contact

Qinghua Lu: qinghua.lu@data61.csiro.au