Professional Accountability, Ethics, and AI Governance in Healthcare
Deliberate Academy Editorial Team
Reviewed for accuracy and professional relevance
You're 6 lessons in — don't lose your progress.
Sign up free to save where you are and earn a verified certificate when you pass.
- Apply the GMC, NMC, and HCPC professional standards to specific AI-assisted clinical practice scenarios
- Evaluate a clinical team's response to an AI-related adverse event against the duty of candour obligations
- Identify when AI use in diagnosis or treatment planning requires patient-facing transparency under the informed consent framework
- Distinguish between AI performance bias as a technical limitation and as a professional accountability matter requiring clinical action
- Demonstrate how to build a team-level AI governance framework covering approved tools, use case boundaries, review standards, and incident reporting
The professional accountability framework that governs healthcare practice does not have a provision for AI to share responsibility. Registered healthcare professionals are accountable for their clinical decisions and their professional conduct. AI tools are not registered. They cannot be struck off, suspended, or sanctioned. When AI is involved in a clinical error, the professional conduct question focuses on the decisions and behaviors of the clinician, not the tool. This lesson addresses that framework directly, alongside the ethical obligations that AI use in healthcare creates, and the governance structures that support safe AI use at the team and organizational level.
GMC, NMC, and HCPC Standards as They Apply to AI Use
The three main professional regulators in UK healthcare have each affirmed that their existing professional standards apply to AI-assisted clinical practice. They have not created separate AI-specific standards, because the existing standards are sufficient when applied carefully to the AI context.
The GMC. Good Medical Practice requires that doctors recognize and work within the limits of their competence, make decisions based on adequate assessment of the patient, and take responsibility for their clinical decisions. These requirements apply directly to AI use. A doctor who uses an AI diagnostic tool without understanding its limitations and scope is not working within the limits of their competence. A doctor who bases a clinical decision on AI output without adequate assessment of the patient has not met the standard. A doctor who accepts an AI output and acts on it is taking responsibility for that clinical decision.
GMC guidance published in 2024 on AI in medicine explicitly states that the use of AI does not change the fundamental obligations of Good Medical Practice, and that doctors who use AI tools are responsible for ensuring those tools are used appropriately and that outputs are clinically reviewed before acting on them.
The NMC. The NMC Code requires that nurses and midwives always practice in line with the best available evidence, preserve safety, and be accountable for their decisions, actions, and omissions. The best available evidence standard applies to AI tools as much as to clinical interventions: using an AI tool whose evidence base the clinician has not assessed is not best evidence practice. The accountability standard is clear: omissions, including the omission of adequate review of AI-generated clinical outputs, carry professional accountability consequences.
The HCPC. The HCPC Standards of Proficiency require registrants to be able to practice within their scope of practice and to recognize the limits of their practice. For allied health professionals, using AI tools that extend beyond their professional scope or that generate outputs outside their competency to review is a scope of practice issue.
The Duty of Candour and AI Errors
The duty of candour is the professional and statutory obligation to be open and honest with patients when something goes wrong that causes harm or distress. In England, the statutory duty of candour under the Health and Social Care Act 2008 applies to registered healthcare providers. The professional duty of candour applies to individual registered healthcare professionals.
AI errors create specific candour considerations. When a patient suffers harm that is at least partly attributable to an error in AI-assisted clinical care, the duty of candour requires openness about what happened, including the role of AI in the clinical pathway. A patient has a right to know if an AI tool contributed to a diagnostic error that affected their care. Healthcare professionals and organizations that are opaque about the role of AI in adverse events are at risk of duty of candour failures.
The duty of candour also intersects with accountability. Recognising, recording, and reporting AI-related adverse events or near misses is both a professional obligation and a patient safety function. Learning from AI errors requires that they are acknowledged and analyzed, not attributed solely to human judgment failures that happen to have followed an AI suggestion.
An AI-assisted triage tool used in an emergency department assigns a low-priority category to a patient who subsequently deteriorates and requires intensive care. A post-incident review finds that the AI's output was accepted without independent clinical triage assessment. The department's incident report attributes the near miss to 'clinician workload and a suboptimal triage process' without mentioning AI. A senior nurse who was involved wants to know whether the report is adequate. Which statement best describes the professional and governance position?
Select one answer.
Informed Consent When AI Is Used in Diagnosis or Treatment Planning
Informed consent in clinical practice requires that patients are given the information a reasonable person would want to have about their diagnosis, treatment options, and the risks and benefits of those options, in a way they can understand. As AI becomes a more significant part of clinical pathways, the question of what patients should be told about AI use is receiving increasing attention from regulators and ethicists.
The current position is that there is no universal legal requirement in UK healthcare for patients to be specifically informed every time AI is used in their clinical pathway. However, the GMC's evolving guidance and the general principle of informed consent suggest that where AI plays a significant role in a clinical decision, particularly in diagnosis or in determining a treatment plan, patients should be informed about this in a way that supports their ability to engage with the decision.
The practical standard is one of transparency proportionate to the role of AI in the decision. AI-assisted administrative triage requires less patient-facing disclosure than AI-assisted diagnostic imaging that directly informs a cancer diagnosis. As a baseline, healthcare organizations should have published transparency statements about how AI is used in care delivery, and clinicians should be able to answer honestly if a patient asks whether AI was involved in their care.
If a patient directly asks whether AI was involved in their diagnosis or treatment planning, be honest. Saying "we use AI to help analyze imaging, but a specialist clinician reviews and takes responsibility for the diagnosis" is both accurate and reassuring. Concealment or evasion when a patient asks a direct question about their care is a trust and candour issue, not just a regulatory one.
AI Bias in Healthcare: Training Data and Underrepresented Populations
One of the most serious ethical issues in clinical AI is the differential performance of AI tools across patient populations. This is not a theoretical concern: it has been documented in published research across multiple AI clinical tools.
A landmark study of a widely used clinical algorithm found it produced significantly lower risk scores for Black patients than for white patients with equivalent clinical need, resulting in Black patients being enrolled in care management programs at a lower rate. The algorithm used healthcare costs as a proxy for healthcare needs, which systematically underestimated needs for populations who had historically had less access to healthcare.
The mechanism is structural: AI tools learn from historical data, and historical clinical data reflects historical patterns of care delivery, which have not been equitable across race, sex, socioeconomic status, and other characteristics. A diagnostic imaging AI trained primarily on images from high-income populations may perform less accurately on images from populations with higher rates of melanin-rich skin, different body composition patterns, or comorbidities that are more prevalent in those populations.
For healthcare professionals using AI tools, the practical implication is to be particularly alert to AI outputs for patients from groups that may be underrepresented in training data. This is an additional reason for clinical override when the patient's presentation does not fit the AI's output, and it is a reason to actively interrogate vendors about population diversity in their training data and validation studies.
AI bias in healthcare can cause real clinical harm by systematically directing resources, diagnoses, and treatment recommendations away from populations that already face healthcare inequalities. Healthcare professionals have an equity obligation, not just an accuracy obligation, in how they use and evaluate AI tools. Accepting an AI output that you suspect may be performing less well for a specific patient because of who that patient is would be a professional conduct failure.
AI Bias Identification and Escalation — NHS Secondary Care Cardiology Team
Context
A cardiology department at an NHS trust had been using an AI-assisted risk stratification tool for 18 months to support decisions about follow-up intensity after an initial diagnosis. A consultant who also served as the departmental clinical governance lead began noticing a pattern during case reviews: patients from a specific demographic group were consistently receiving lower risk scores than his clinical assessment suggested, and several had subsequently presented with deterioration earlier than the tool's score predicted. The pattern was not visible in individual cases but became apparent across a series of reviews.
Action
The consultant raised the observation at the departmental governance meeting, framing it as a patient safety signal rather than a definitive finding. The team agreed to apply heightened clinical scrutiny to AI outputs for the affected patient group while a formal review was conducted. The consultant submitted an incident report through the trust's clinical governance system, specifically identifying the AI tool and the suspected bias pattern. He also contacted the vendor to request subgroup performance data from their validation studies, broken down by the demographic characteristics he had identified.
Outcome
The vendor's response confirmed that the demographic group in question had been underrepresented in the original training dataset and that internal subgroup performance data showed lower sensitivity for that group compared to the headline figure. The trust paused routine use of the tool for the affected patient group pending recalibration and re-validation. The consultant's incident report became the basis for a broader review of the tool's deployment across the trust. The experience reinforced for the team that AI bias manifests in patterns across patient groups rather than as obvious failures in individual cases, and that clinical governance teams need to be actively looking for these patterns — not waiting for them to surface in adverse event reports.
Building a Safe AI Use Policy for a Clinical Team
Individual practitioners working within clinical teams have both individual accountability and collective responsibility for how AI is used in their team's practice. Building a team-level AI governance approach does not require complex bureaucracy. It requires agreement on a small number of clear principles and practices.
Approved tools. Agree as a team which AI tools are approved for use in clinical workflows, based on organizational IG clearance and the evaluation criteria covered in Lesson 5. Make this list explicit and ensure new team members are informed. Unofficial use of unapproved tools should be treated as a clinical governance concern, not a personal technology choice.
Use case boundaries. Define explicitly which clinical tasks AI tools are appropriate for and which are not. A documentation AI is appropriate for drafting clinical notes; it is not appropriate for generating clinical advice for a specific patient. An imaging AI alert is an input to clinical assessment; it is not a diagnosis.
Review standards. Agree the review standard for different types of AI output. What does adequate review of an AI-generated discharge summary look like? Who is responsible for reviewing AI-assisted coding before submission? Make these standards explicit rather than assumed.
Incident reporting. Agree that AI-related near misses and adverse events will be reported through normal clinical governance channels, and specifically that the AI component will be identified in the incident report. This creates the learning loop that supports safe AI use at team and organizational level.
Staying current. AI capabilities and regulatory expectations are evolving rapidly. Assign responsibility for monitoring guidance updates from the GMC, NMC, HCPC, MHRA, and NHS England and for communicating relevant changes to the team.
A clinical team has been using an AI-assisted risk stratification tool for six months. A team member notices that the tool consistently assigns lower risk scores to elderly patients from a specific ethnic background compared to younger patients with the same clinical parameters, and that some of those patients have subsequently deteriorated faster than predicted. What should the team do?
Select one answer.
Exercise
Your Task
Draft a one-page AI use policy for your clinical team or department. It does not need to be formally approved — the purpose is to make your team's current AI governance position explicit. Cover the five elements from this lesson: (1) which AI tools are approved for use and which are not, (2) the use case boundaries for each approved tool, (3) the review standard expected for AI outputs before clinical action, (4) how AI-related near misses and adverse events should be reported, and (5) who is responsible for monitoring regulatory and guidance updates. If your team does not currently use any AI tools, write the policy as it would apply to the ambient documentation AI your organization is most likely to adopt next.
Success looks like
- Your policy covers all five governance elements rather than only the ones that feel straightforward
- The use case boundaries are specific enough that a new team member could read them and know exactly when AI is appropriate and when clinical judgment must operate independently
- Your incident reporting section explicitly names AI as a reportable component in clinical governance events rather than treating it as an optional detail
- The policy is written in language appropriate for your clinical setting — it reflects your team's actual tools and accountability structure, not a generic template
Watch out for
- Listing only tools the team currently uses and not addressing what to do when a colleague suggests adopting a new tool — the policy should include a clear process for assessing unapproved tools
- Treating incident reporting as a governance formality rather than a patient safety function — the policy should explain why AI-specific reporting matters, not just that it is required
Hint
The most important test of a team AI policy is whether it provides clear enough guidance to resolve a genuine dilemma — a colleague asking 'can I use this AI tool for this patient task?' If the answer requires a conversation with a senior colleague every time, the policy is not yet specific enough.
- GMC Good Medical Practice, the NMC Code, and HCPC Standards of Proficiency all apply to AI-assisted clinical practice. They require that clinicians understand the limits of AI tools they use, base decisions on adequate patient assessment, and take accountability for clinical decisions regardless of AI involvement.
- The duty of candour requires transparency about AI involvement in adverse events. Patients have a right to honest answers about whether AI contributed to their clinical care, particularly where harm has occurred.
- AI bias from training data that underrepresents certain populations is a documented patient safety and equity concern. Healthcare professionals have an obligation to be alert to differential AI performance across patient groups and to apply heightened clinical scrutiny accordingly.
- Team-level AI governance requires explicit agreement on approved tools, use case boundaries, review standards, and incident reporting for AI-related events. These are clinical governance matters, not personal technology preferences.
- The professional healthcare identity in an AI-assisted world is defined by the judgment, relationship, and accountability that AI cannot provide. AI makes competent clinicians more efficient; it does not substitute for clinical competence.