Skip to content

Transparency and Accountability

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Transparency is a cornerstone of developing trust, specifically when AI is involved in research, government, or healthcare.

Definitions

AI transparency is the degree of openness in a system design, the data that it uses, its operational framework. Transparency includes:

  • Access to the data a system was trained upon
  • Explanation of how a model arrives at its decisions
  • List of guardrails, safeguards, and measures in place to mitigate bias

Explainable AI (XAI) focuses on describing specific sequence of steps that an AI model undertakes to arrive at a result, prediction, or response. XAI includes AI transparency and enables comprehension.

Establishing Accountability

Coalition for Health AI (CHAI) advances the responsible development, deployment, and oversight of AI in healthcare

NIST AI Risk Management Framework (AI RMF) has developed a framework to better manage risks to individuals, organizations, and society associated with AI.

Organization for Economic Cooperation and Development (OECD) AI Principles promote use of AI that is innovative and trustworthy and that respects human rights and democratic values.

World Health Organization (WHO) has released AI Ethics and governance guidelines for large multi-modal models

EU AI Act is comprehensive legislation that includes oversight of AI with a six year (2031) implementation timeline, which is already underway.

Mechanisms

Audits - regular systematic audits are essential, these includ bias audits to detect discrimination and fairness evaluations, security vulnerability checks, and performance reviews for accuracy and reliability.

Human Oversight - high risk systems require human-in-the-loop approaches which are validated by human experts before implementation

Governance Structures - clear and effective governance structures are fundamental to AI accountability. This involves defined leadership and oversight (boards), where responsibility across organizations is formalized and put into standard operating procedures.

Record-keeping / Logs - traceability and auditability require detailed records of the AI system's operation and user actions. Audit rails provide invaluable resources for incident investigation, understanding system responses, and demonstrating compliance.

Transparency

Black Box Problem

Many AI models are based on complex architectures that operate as "black boxes" meaning that their internal decision space is opaque or not comprehensible by human observers. This lack of comprehension results in acountability problems, specifically for their use in applications like health care or medical decisions.

Strategies for mitigating risk and cultivating responsible, transparent, and explainable AI

  • Promoting data diversity and bias mitigation

  • Strengthening data security and privacy-preserving AI

  • Fostering human-in-the-loop approaches w/ continuous oversight

  • Enhancing Transparency and XAI