Discrimination Mitigator

Summary: Discrimination mitigation algorithms are used to address unwanted bias throughout the entire lifecycle of AI systems.

Type of pattern: Product pattern

Type of objective: Trustworthiness

Target users: Data scientists

Impacted stakeholders: RAI governors, AI users, AI consumers

Lifecycle stages: Design

Relevant AI ethics principles: Fairness

Context: In the context of AI systems, discrimination is the illegal or unfavorable treatment of an individual based on their membership in a protected group such as gender, race, or age. There are two types of discrimination that can occur in AI systems: direct discrimination happening when protected groups are explicitly used as attributes in the decision-making algorithms and indirect discrimination that occurs when protected groups are correlated with attributes in the decision-making algorithms.

Problem: How can we mitigate algorithmic discrimination in the decision-making of AI systems?

Solution: Algorithmic discrimination can be mitigated in three ways:

  • Pre-processing techniques (such as data preprocessing technique [1], optimized data pre-processing technique [2], data perturbation technique for word embedding [3]) can be used to remove the underlying discrimination by removing or adjusting data points that are biased or not representative of the population the AI system will be used on. 
  • In-preprocessing techniques (such as adding fairness constraints [4] or prejudice remover regularizer [5] for fair classification, or adding minimizing adversary as one of the learning objectives [6]) can be applied during the training of the AI model to modify the learning algorithms and remove discrimination in the decision-making process, if the modification of the learning algorithms is allowed. 
  • Post-processing techniques (such as equalized odds or equal opportunity predictor derived from a learned predictor [7], removing gender associations in word embedding [8]) are used after the AI model has been trained and deployed to adjust its output and remove any unwanted bias. 

Benefits:

  • Improved model performance: When discrimination is removed, the AI model’s performance is improved and its output is more trustworthy.
  • Better balance between fairness and accuracy: Discrimination mitigation techniques can effectively balance fairness and accuracy.

Drawbacks:

  • Lack of generality: AI models are often designed for specific tasks, which makes it difficult to extend and adapt discrimination mitigation techniques to new tasks or domains.
  • Reduced model performance: Introducing discrimination mitigation techniques can affect the overall performance of the model.
  • Compromised accuracy: Discrimination mitigation techniques may involve trade-offs in order to achieve fairness.

Related Patterns:

  • Fairness Assessor: Fairness measurement metrics are used as objectives of mitigation techniques.

Known Uses:

  • LinkedIn Fairness Toolkit (LiFT) is a library that is used to measure fairness and mitigate bias in large-scale AI workflows. A post-processing method is used to transform scores to achieve equality of opportunity in rankings.
  • Microsoft Fairlearn is a fairness toolkit that contains model assessment metrics and mitigation algorithms for AI developers to assess their AI system’s fairness and mitigate the detected bias.
  • IBM AI Fairness 360 is a python library that contains a comprehensive set of fairness metrics and algorithms (such as optimized processing, equalized odds post-processing) to mitigate bias in datasets and AI models.

References:

[1] Kamiran, F. and T. Calders, Data preprocessing techniques for classification without discrimination. Knowledge and information systems, 2012. 33(1): p. 1-33.

[2] du Pin Calmon, F., et al., Optimized Data Pre-Processing for Discrimination Prevention. 2017.

[3] Brunet, M.-E., et al. Understanding the origins of bias in word embeddings. in International conference on machine learning. 2019. PMLR.

[4] Zafar, M.B., et al. Fairness constraints: Mechanisms for fair classification. in Artificial intelligence and statistics. 2017. PMLR.

[5] Kamishima, T., et al. Fairness-aware classifier with prejudice remover regularizer. in Joint European conference on machine learning and knowledge discovery in databases. 2012. Springer.

[6] Zhang, B.H., B. Lemoine, and M. Mitchell. Mitigating unwanted biases with adversarial learning. in Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. 2018.

[7] Hardt, M., E. Price, and N. Srebro, Equality of opportunity in supervised learning. Advances in neural information processing systems, 2016. 29.

[8] Bolukbasi, T., et al., Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 2016. 29.