Module 2 of 2 · 45 minutes
Keeping a person in charge of the machine
By the end of this module you will be able to
- Decide what an AI system may advise on and what it may control
- Explain validation before use and monitoring in use
- Describe the cyber security risk an AI system adds to plant
- Explain how the EU Machinery Regulation and AI Act affect machinery with AI
- Keep the records that show a system is under control
Work through it
1 interactive for this module, built on the WAJD Teach engine. Nothing moves until you ask it to, and every one has a written version if you would rather read it.
Watch: Emma and George talk it through
4 minutes. Captions are on, and the same conversation is written out in full below. The voices are computer generated.
Emma George, last time you told me the camera can go quietly wrong. So how do I stay in charge of it?
George Start with one question about any AI system on your site. Does it advise, or does it control?
Emma What's the difference in practice?
George A system that advises shows a person something and the person acts. A system that controls acts itself. Rejects the part, stops the line, changes a setpoint.
Emma Ours rejects parts automatically.
George Then it controls, for quality. That's a decision someone should have made deliberately. Anything new should start as advice. Moving it to control needs its own assessment. And where the action affects safety, it's part of the safety system and has to be designed and validated as one, by competent people.
Emma What usually goes wrong?
George Not a bad decision to automate. It's an advisory tool that drifts into control because people stop checking it. If the operator always accepts what the screen says, it's controlling, whatever the procedure calls it.
Emma So how do I keep it honest?
George Validate before use and watch in use. Before you rely on it, test it against cases whose true answer you know. Include the hard ones and the defects that matter most. Record the result, the conditions, and who signed it off.
Emma And afterwards?
George Challenge samples. Known good and known bad, fed through at a set interval. And track overrides. How often people overrule it, and why.
Emma What's a good override rate?
George It should never be zero. A rising rate is an early sign of drift. But a rate of zero means nobody's looking. And keep the manual method alive. If the team's forgotten how to inspect by hand, you've no fallback.
Emma Our IT manager is nervous about the supplier's remote access.
George Rightly. It's software connected to plant, usually with a route back to a supplier for updates. That makes it part of your cyber security problem. The HSE names cyber threats explicitly, and manufacturers of things like machinery and vehicles are among the sectors the EU's NIS2 Directive covers.
Emma What should I ask?
George How is it updated and by whom? What can it reach on the network? And what happens to the line if it's unavailable, or gives wrong answers on purpose?
Emma We also build machines and sell some into Europe. Anything coming?
George Two things. The EU Machinery Regulation applies from 20 January 2027 and replaces the old Directive. It addresses machinery whose safety functions rely on systems that learn. And the EU AI Act treats AI used as a safety component of a regulated product as high risk. After the July 2026 amendment, that applies from 2 August 2028.
Emma Is that UK law?
George Neither is, and Great Britain has its own machinery regulations. If you export, take advice on your own product. What I've given you is direction, not a compliance route.
Emma And what do I keep on file?
George Four records. What the system's allowed to do. How it was validated. How it's being checked. And every change made to it.
The written material
Advise or control
The most useful question to ask of any AI system on a site is whether it advises or controls. A system that advises shows a person something, and the person acts. A system that controls acts itself: it rejects the part, stops the line, changes a setpoint.
Anything new should start as advice. Moving it to control is a separate decision that needs its own assessment, and where the action affects safety it becomes part of the safety system and must be designed and validated as such by competent people. The common failure is not a bad decision to automate. It is an advisory tool that drifts into control because people stop checking it.
Validate before use, watch in use
Before a system is relied on, test it against cases whose true answer is known, including the difficult ones and the defects that matter most. Record the result, the conditions and who signed it off.
Then keep watching. Feed known good and known bad samples through at a set interval. Track how often people override it and why: a rising override rate is an early sign of drift, and an override rate of zero means nobody is looking. Give operators a simple way to stop relying on it, and a manual method they still know how to use.
- Test against known cases before use, and record it
- Challenge samples at a set interval
- Track overrides and the reasons
- Keep the manual method alive
It is also a door
An AI system is software connected to plant, usually with a route to a supplier for updates. That makes it part of your cyber security problem. The HSE explicitly includes cyber security threats among the risks to control, and manufacturers of products such as machinery, electronics, vehicles and medical devices are among the sectors the EU's NIS2 Directive covers.
Ask how the system is updated and by whom, what it can reach on the network, and what happens to the line if it is unavailable or gives wrong answers on purpose.
If you build or export machinery
Two EU laws matter to a UK firm that supplies machinery to the EU. The Machinery Regulation, (EU) 2023/1230, applies from 20 January 2027 and replaces the Machinery Directive. Among its changes, it addresses machinery whose safety functions rely on systems that learn, and puts safety components with self-evolving behaviour among the categories that need assessment involving a third party.
The EU AI Act treats AI used as a safety component of a regulated product as high risk. After the amendment of July 2026, those rules apply from 2 August 2028. Neither is UK law, and Great Britain has its own machinery safety regulations. If you export, take advice on your own product: this is a summary of direction, not a compliance route.
Knowledge check
The knowledge check and your certificate need a free account, so that your progress and results can be saved as evidence.
The learning itself stays free and open. You are reading all of it right now without an account.
Was this module useful? Tell us in two minutes, it decides what we improve next.