Public Health

This work is licensed under a Creative Commons Attribution 4.0 International License.

Bias and Data Quality

Ethics: Transparency & Accountability

Standardization

The standardization of protocols and controls is critical for ensuring consistent, high-quality public health practices across different organizations and regions.

AI Protocol Analysis and Harmonization

AI can analyze multiple existing protocols to identify commonalities, differences, and best practices:

Example prompt

Analyze different measles testing protocols from various health departments
in Arizona, Texas, New Mexico, Sonora, and Chihuahua. 
Identify:
- Common required steps across all protocols
- Key differences in methodology
- Best practices that appear in multiple protocols
- Potential gaps or missing elements
- Recommendations for a standardized protocol

Natural Language Processing for Protocol Extraction

AI can extract structured information from unstructured protocol documents:

• Convert narrative guidelines into step-by-step procedures
• Identify required equipment, materials, and personnel
• Extract critical values, thresholds, and decision points
• Create standardized terminology mappings

Automated Protocol Validation

AI systems can check protocols for: - Completeness (all necessary steps included) - Consistency (no contradictory instructions) - Compliance with regulations and standards - Evidence-based practice alignment

Human-in-the-Loop (HITL) Approaches

Human-in-the-loop systems combine AI efficiency with human expertise to improve accuracy and reliability:

Key HITL Components:

• Expert Annotation: Subject matter experts review and annotate AI-generated protocols
• Iterative Refinement: AI learns from expert corrections to improve future outputs
• Decision Points: Critical decisions require human approval before proceeding
• Quality Assurance: Random sampling and human verification of AI outputs

Key Infrastructure Requirements

• Scalability: Handle increasing protocol volumes and complexity
• Interoperability: Connect with existing health information systems
• Security: Protect sensitive health information
• Reliability: 99.9% uptime for critical health operations
• Traceability: Complete audit trails for regulatory compliance

Implementation Considerations

• Change Management: Training staff on new AI-assisted workflows
• Ethical Guidelines: Ensuring AI decisions are fair and unbiased
• Regulatory Compliance: Meeting local and federal health regulations
• Performance Monitoring: Tracking AI accuracy and human satisfaction
• Continuous Learning: Regular model updates based on new evidence

Best Practices for Implementation

1. Start Small: Pilot with non-critical protocols first
2. Maintain Transparency: Document AI decision-making processes
3. Preserve Expertise: AI augments, not replaces, human judgment
4. Regular Audits: Systematic review of AI performance
5. Stakeholder Engagement: Include all users in design and testing
6. Continuous Training: Keep both AI models and humans updated

Human-in-the-loop approaches that improve accuracy and reliability, combined with robust AI infrastructure, allow for more seamless integration into public health practices while maintaining the critical human oversight necessary for healthcare decision-making.

Case Studies

Awesome Lists

Maintained on GitHub and often feature the badge: , Awesome lists contain community-maintained lists of popular and widely used software, data, and code for almost everything on the internet.

• Awesome Healthcare

• Awesome Healthcare Datasets

Diagnostics

AI has great promise for enhancing diagnostic capabilities, but have exhibited biases on race and ethnicity. There are calls to address these biases in big data and AI for health care by taking an open science approach

List of FDA approved AI medical devices: https://www.datawrapper.de/_/IBGhg/

Positive Examples

High speed internet access via wired, broadband, or satellite allows clinicians to stream data to the cloud and larger AI resources and improved access to specialty care in Underserved and Rural Areas.

Negative Examples:

• AI-enabled pulse oximeters overstate blood oxygen saturation in individuals with darker skin, exacerbating racial bias

Readings:

• https://www.medtechdive.com/news/fda-ai-medical-devices-growth/728975/

• https://nam.edu/perspectives/advancing-artificial-intelligence-in-health-settings-outside-the-hospital-and-clinic/

• https://www.nature.com/articles/s41746-018-0040-6

• https://www.digitaldiagnostics.com/products/eye-disease/lumineticscore/

Disease Surveillance & Prediction

AI-powered predictions change the landscape of population-level public health surveillance.

Machine learning (predictive AI) has been used for many years to analyze data and make predictions around disease spread and outbreak detection.

AI-systems can quickly analyze electronic health records (EHRs), social media platforms, online search queries, environmental data, genomic sequences, wearable devices, Of course. Here is the updated markdown table with public URLs for every example. Any examples for which I could not find a reliable, public URL have been removed.

Application Area	AI Technique(s) Used	Data Sources Leveraged	Specific Examples/Platforms	Key Outcomes/Impacts
Early Outbreak Detection	Machine Learning, NLP	Social Media, News Reports, Airline Travel Data, Official Health Reports	BlueDot, HealthMap	Early warning for COVID-19 by detecting initial signals before official reports; faster public health response.
Real-Time Monitoring	Machine Learning, NLP	EHRs, Lab Reports, Public Health Data, Free-text data	CDC NLP analysis for vaccine safety	Continuous tracking of disease spread and monitoring safety of vaccines by analyzing large volumes of free-text data.
Epidemic Forecasting	Machine Learning, Deep Learning	Historical Disease Data, Climate Patterns, Population Mobility, Search Queries	CDC FluSight, Dengue/Influenza models	Improved prediction accuracy for flu seasons, dengue outbreaks; better preparedness.
Identifying High-Risk Populations	Machine Learning	Demographic, Socioeconomic, Health Data, Aerial Imagery	TowerScout	Enables targeted interventions, such as identifying cooling towers from imagery to speed up response to Legionnaires' disease outbreaks.
Pathogen Genomic Surveillance	AI Algorithms, Machine Learning	Genomic Sequences of Pathogens	Nextstrain	Rapid detection of new variants (e.g., SARS-CoV-2), understanding transmissibility/virulence changes.
Syndromic Surveillance (Unconv.)	NLP, Machine Learning, Image Analysis	Online Search Queries, Social Media Posts, Wikipedia page views, Chest X-rays	AI for TB detection from X-rays	Early detection of community transmission before clinical reporting and improved speed and accuracy for TB surveillance.
Antimicrobial Resistance (AMR) Tracking	Predictive Analytics, ML	Clinical Data, Epidemiological Data, Lab Results	AI-driven AMR surveillance systems	Improved detection of AMR trends, rapid pathogen ID and resistance profiling, guidance for stewardship.
Vector-Borne Disease Prediction	Machine Learning	Environmental Data (temp, humidity, rainfall), Satellite Imagery, Historical Case Data	AI models for Malaria prediction	Prediction of high-risk areas for outbreaks, enabling targeted vector control measures.
Resource Allocation Prediction	Machine Learning	Outbreak Data, Hospital Capacity Data, Supply Chain Information	NHS A&E demand forecasting tool	Optimization of resource deployment during public health emergencies.
Misinformation Monitoring	NLP	Social Media Content, Online News	Project Heal, EPIWATCH	Identification of false narratives that could impede public health responses, enabling counter-messaging.

Readings:

• Zeng et al. (2021) Artificial intelligence–enabled public health surveillance—from local detection to global epidemic monitoring and control. Artificial Intelligence in Medicine Technical Basis and Clinical Applications

• Hattab, G. et al. (2025) The Way Forward to Embrace Artificial Intelligence in Public Health. American Journal of Public Health 115, 123_128

Resource Allocation

Perhaps the most ethically fraught application of AI in health care is on resource allocation. Decisions made by algorithms for high-risk care have exhibited racial biases.

Positive Examples:

• The UK's National Health Service (NHS) uses an AI-powered tool to forecast demand for emergency services, helping to optimize the allocation of staff, beds, and other resources to improve patient outcomes. (Read more)

Negative Examples:

• Training data based on historical spending for health care on black vs white patients resulted in an algorithm systematically biased toward spending on more on white patients than on black patients, resulting in a perpetuation and exacerbation of the health disparity.

Clinical Decision Support

AI-powered tools can assist healthcare providers in making more informed decisions, but they must be used with caution as they can also perpetuate existing biases.

Positive Examples:

• Oncology Screening: Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology
• Diabetic Retinopathy Detection: AI systems like LumineticScore can analyze retinal images to detect diabetic retinopathy, a leading cause of blindness, enabling early intervention and treatment, particularly in areas with limited access to specialists.
• Drug Discovery: Language models like Google's TxGemma are being developed to accelerate drug discovery by understanding and predicting the properties of therapeutic compounds.
• Diagnostic Conversations: Research systems like Google's AMIE are exploring the potential for AI to conduct diagnostic conversations and improve medical reasoning. (Read more)

Negative Examples:

• Skin Cancer Detection: AI models for detecting skin cancer have shown lower accuracy on darker skin tones, which can lead to missed or delayed diagnoses for patients from underrepresented racial and ethnic groups.

Mental Health

The application of AI in mental health presents both significant opportunities and serious risks, particularly concerning the well-being of vulnerable populations.

Positive Examples:

• Increased Access to Care: AI-powered chatbots and virtual therapists can help mitigate shortages of mental health providers, especially in rural and underserved areas, by offering accessible, on-demand support. (Read more)
• Potential for Effective Treatment: Early research suggests that generative AI chatbots may become a valuable tool in mental health treatment, with some studies showing promising outcomes. (Read the study)
• Positive Mental Health: AI is being explored for applications in positive psychology, focusing on well-being and flourishing. (Read the review)

Negative Examples:

• Unregulated and Unaccountable Care: The proliferation of unlicensed and unaccountable AI chatbots posing as therapists raises serious ethical concerns, as they may provide harmful advice or fail to respond appropriately in crisis situations. (Read more)
• Risk of Harm: Unregulated AI has been accused of contributing to negative mental health outcomes, including exacerbating distress, cyberbullying, and even being implicated in cases of suicide.
• Lack of Oversight: There is currently little regulatory oversight for AI therapy companies, creating a gap in ensuring the safety and effectiveness of these services. (Read more)

Further Reading

WHO Guidance: Ethics and Governance of Artificial Intelligence for Health

Focus Group on "Artificial Intelligence for Health" (FG-AI4H)