Module 1 of 3 · 50 minutes
What AI actually is on your ward, and how it gets things wrong
By the end of this module you will be able to
- Distinguish a prediction tool from a generative AI tool and say how each fails
- Explain why a generative tool can be fluent, confident and wrong
- Name the four failure modes that matter most in clinical documentation
- Recognise automation bias and describe how to counter it
- State what the NMC Code asks of you when you use an AI tool
Watch: Emma and George talk it through
5 minutes. Captions are on, and the same conversation is written out in full below. The voices are computer generated.
Emma George, I'll be honest. Every second email at work now is about AI, and I still couldn't tell you what the thing is actually doing. Can you?
George I can, and it takes about a minute. First, there isn't one thing. On a ward there are at least two kinds, and they fail differently. One predicts, taking measurements already in the record and giving you a score or an alert. The other generates, taking words in and producing new words out.
Emma Give me an example of each.
George A deterioration risk score is the first kind. A scribe that listens to a consultation and drafts the note is the second. The first can be miscalibrated, wrong about a kind of patient it saw too little of. The second can invent.
Emma Invent. That's a strong word.
George It's the right one. A generative tool has learned from a huge amount of text which words tend to follow which. When it answers, it produces the most plausible continuation. It doesn't look anything up, and it has no idea whether what it produced is true.
Emma But it sounds so sure.
George Because sounding sure is what it's built to do. Smooth, confident prose is its whole output. It doesn't hesitate when it's guessing. A wrong sentence and a right sentence look identical on the screen.
Emma So how do I know which is which?
George You don't, from the text alone. You know from what you saw and heard. Which is why I want you to carry four failure modes in your head. Invention, a detail nobody said. Omission, something said that's missing. Distortion, the detail's there but changed. And misattribution, the wrong person said it.
Emma Which is the worst?
George Omission, because you can't see a gap by reading what's on the page. You only catch it if you remember what was said in the room. A missed allergy is invisible in a note that looks complete.
Emma And distortion?
George Numbers, sides and negatives. A dose that's slightly off. Left for right. No chest pain turning into chest pain. Those three get a second look from me, every time, whoever wrote the draft.
Emma Here's what bothers me. If it's right nearly every time, I'm going to stop checking. I know I will.
George Everybody does. It has a name, automation bias, and it isn't carelessness. It's what happens to a tired person given a draft that's ninety-five per cent right. The five per cent hides in the bit you stopped reading. And the better the tool, the worse it gets.
Emma That's a strange thing to teach. The better it is, the more dangerous it is.
George Not more dangerous, harder to supervise. So you use a habit. Read it as though a very fast, very confident junior had written it, someone who has never met the patient. You wouldn't sign their note unread.
Emma Fair. Now the part everyone worries about. The NMC. What does the Code actually say about this?
George Nothing about the technology, and it doesn't need to. Clause 10.4 says attribute your entries to yourself. 10.3, complete records accurately and without falsification. 6.2, keep the knowledge and skills you need for safe practice. And 19.1 and 19.2, reduce the likelihood of mistakes and allow for human factors and system failures.
Emma So if the tool wrote it and I accepted it...
George It's your entry. You adopted it. You also have to know enough about the tool to use it safely. Not how it's built. How it fails.
Emma Is the Code about to change?
George The NMC has said it will consult on a new Code from September to December 2026, and that digital and AI technologies are one of its areas. I'd be surprised if the duties got lighter. This course teaches what stands today and tells you when that might move.
Emma What about my employer? If they bring the tool in, doesn't some of this land on them?
George Some does. They share responsibility for how it's introduced, tested and supported. But your accountability to the NMC is yours. It doesn't transfer. So you want to be someone who can ask good questions about the tool, which is where we go next.
Emma So the one thing to take away?
George Two. It writes fluent text whether or not it's true. And once you accept it, it's your record.
The written material
Two very different kinds of tool
People say AI as though it were one thing. On a ward there are at least two, and they fail in different ways.
The first kind predicts. It takes measurements already in the record and produces a score or an alert: a risk of deterioration, of a fall, of readmission. It is usually narrow, trained for one job, and its typical failure is being wrong about a particular kind of patient because that patient was thin in the data it learned from.
The second kind generates. You give it words, or it listens to a conversation, and it produces new words: a summary, a letter, a draft note, an answer to a question. This is the kind arriving now in scribes and in general assistants, and it is the kind this course is mostly about.
What a generative tool is actually doing
A generative tool has learned, from a very large amount of text, which words tend to follow which. When it answers, it is producing the most plausible continuation. It is not looking the answer up, and it has no sense of whether the answer is true.
That explains the thing that surprises everyone the first time. The output reads smoothly and sounds certain whether it is right or wrong, because smoothness is what the tool is built to produce. It does not hesitate when it is guessing. A wrong sentence and a right one look exactly alike.
It also has never met your patient. It knows only what it was given in this conversation, and it fills gaps with what is usual rather than what is true for the person in the bed.
Four ways it fails in a clinical record
Invention. A detail appears that nobody said: a symptom, a medication, a plan. It sounds right because it is the sort of thing that is usually there.
Omission. Something that was said is missing. This is the dangerous one, because you cannot see a gap by reading what is on the page. You can only see it if you remember what was in the room.
Distortion. The detail is there but changed: a dose, a side, a number, or a negative turned into a positive. No chest pain becomes chest pain.
Misattribution. The record says the patient reported something their relative said, or that you did something a colleague did.
Automation bias: the failure that sits in the person
People tend to trust a suggestion from an automated system more than it deserves, especially when they are busy and the suggestion is fluent. This is called automation bias and it is well documented in human factors research far beyond healthcare.
It is not carelessness. It is what happens to a tired person at the end of a long shift who has been given a draft that is ninety-five per cent right. The five per cent hides in the part you stopped reading. The more often a tool is right, the more this matters, because the more reliable it seems, the less closely anyone looks.
The counter is a habit, not a resolution. Read the draft as though a very fast, very confident junior had written it who has never met the patient. Check what you would check from them.
What the Code says, whatever the tool
The Code does not mention any particular technology, and it does not need to. Clause 10.4 asks you to attribute any entries you make in any paper or electronic records to yourself. Clause 10.3 asks you to complete records accurately and without any falsification. Clause 6.2 asks you to maintain the knowledge and skills you need for safe and effective practice. Clauses 19.1 and 19.2 ask you to reduce the likelihood of mistakes and to take account of current evidence, including the impact of human factors and system failures.
Read together, they say this. If a tool helped write it and you accepted it, it is your entry. You must know enough about the tool to use it safely, and you must allow for the way it and you can fail.
The NMC has told registrants it will consult on a new Code from September to December 2026 and that digital and AI technologies are one of the areas it will cover. This course teaches the Code as it stands. If it changes, the duties above are very unlikely to get lighter.
Knowledge check
The knowledge check and your certificate need a free account, so that your progress and results can be saved as evidence.
The learning itself stays free and open. You are reading all of it right now without an account.
Was this module useful? Tell us in two minutes, it decides what we improve next.