EXPLAINER

AI Bias and Fairness

conceptfreshness: evolvingsources not yet verified

AI systems can reproduce or amplify biases in their data and design, producing systematically different outcomes for different groups. Fairness work measures, mitigates and documents this.

Short answer: Bias can enter via data, labels, features, objectives and deployment context. Several fairness definitions exist and can conflict mathematically, so choose and justify metrics for your context, test across groups, mitigate, and keep humans accountable.

What is it?

If a system learns from historical data that reflects unfair patterns, it can repeat or magnify them — for example in hiring, lending or image recognition. Fairness work checks outcomes across groups and tries to reduce unjustified differences.

Also known as: algorithmic bias, AI fairness, bias in AI, demographic parity, equalized odds, fairness metrics, discrimination in machine learning, representational harm.

Technical definition

Sources: sampling and measurement bias, label bias, proxy variables, feedback loops and representational harms. Group-fairness criteria include demographic parity, equalised odds/opportunity and calibration; impossibility results show some cannot hold simultaneously except in special cases (Kleinberg et al., 2016; Chouldechova, 2017). Audit evidence includes disaggregated error rates (Buolamwini & Gebru, 2018, "Gender Shades"). Mitigation: data improvements, re-weighting, constraints, post-processing and process controls. Legal anti-discrimination requirements are jurisdiction-specific (AI Governance).

Entity type: concept · Freshness class: evolving.

When should I use it?

Required thinking for decisions affecting people.

Implementation patterns

Technologies and frameworks

Commonly associated: Fairlearn, AIF360, What-If Tool, disaggregated evaluation. Names are examples of what exists, not recommendations, and projects change quickly.

Practical guide

  1. Identify affected groups and harms.
  2. Choose metrics.
  3. Evaluate disaggregated.
  4. Mitigate and re-evaluate.
  5. Document and monitor.

Common failure modes

  • Optimising one fairness metric while ignoring others.
  • Removing a sensitive attribute while proxies remain.
  • Testing on non-representative groups.
  • One-time audits.

Troubleshooting

  • Large error-rate gaps between groups — Investigate data coverage and labels; apply mitigation and retest.
  • Metrics conflict — Decide which harm matters most for the use case and document the trade-off.

Contrasted with

Questions this page answers

  • What is bias in AI?
  • How do I test an AI system for fairness?
  • What is demographic parity?
  • Can an AI be unbiased?
  • What is a bias audit?

Typical goals: audit a model for bias; choose a fairness metric.

Sources and verification

Status: not independently verified. This entry was authored on 2026-10-06 from general technical knowledge, without live checking of the sources below. Treat every source as a lead to confirm (identifier, title, authorship, current version) before publication or citation.

Freshness class: evolving — Established techniques whose best practice shifts as research and tooling move. Re-review every 6 months.

Canonical sources (unverified leads):

  • Barocas, Hardt & Narayanan, Fairness and Machine Learning — fairmlbook.org
  • Buolamwini & Gebru, Gender Shades, PMLR 81 (2018) — locator to be identified
  • Mehrabi et al., A Survey on Bias and Fairness in Machine Learning (2019) — arXiv:1908.09635
Learning path: Governing AI (step 6 of 7)
← Model Cards and Datasheets (AI Documentation)OWASP Top 10 for LLM Applications →