Module 2 of 2 · 45 minutes
Before a tool touches a customer
By the end of this module you will be able to
- Describe the safeguards for solely automated decisions about customers
- Explain how discrimination law applies to automated decisions
- Tell general information from a personal recommendation
- Explain why outputs must be explainable
- List what a firm needs in place before go live
Work through it
1 interactive for this module, built on the WAJD Teach engine. Nothing moves until you ask it to, and every one has a written version if you would rather read it.
Watch: Emma and George talk it through
3 minutes. Captions are on, and the same conversation is written out in full below. The voices are computer generated.
Emma George, we're about to let a model decide small loans on its own. What do I need to have thought about?
George Start with the decision itself. Declining a loan has a significant effect on a person. Since 5 February 2026 the law permits that to be made solely by automated means on ordinary personal data, with safeguards.
Emma Which are?
George The customer must be told. They must be able to make representations. To obtain human intervention. And to contest the decision.
Emma We'll have someone glance at the declines.
George A glance won't do. The Information Commissioner's Office says, in draft guidance, that human involvement must be active rather than tokenistic. Someone who understands the system and has the authority to change the outcome.
Emma The model doesn't use race or sex. So no discrimination risk.
George It doesn't need them. A postcode, a shopping pattern or a device type can produce different outcomes for groups defined by race, sex, age or disability. That can be indirect discrimination under the Equality Act.
Emma But the model's accurate.
George On average. A model can be accurate on average and unfair to a group. Average accuracy isn't a defence to the Equality Act or to the Consumer Duty.
Emma So I monitor.
George You compare outcomes across groups, understand why they differ, and justify the difference or remove it.
Emma Separate question. Our website assistant answers pension questions. Is that advice?
George It might be. The line between information and a personal recommendation doesn't move because a tool's doing the talking. If someone asks what should I do with my pension, and it answers with a course of action for that customer, it may have crossed the line.
Emma It has a disclaimer.
George A disclaimer doesn't change what it said. Test what it says when it's pushed. Restrict it to what you're permitted to do. And hand over to a qualified person where advice is needed.
Emma What if a customer asks why they were declined?
George You have to be able to answer in terms they can understand. And so does your senior manager when the regulator asks. A tool nobody in the firm can explain is one the firm can't govern.
Emma Give me the list before go live.
George A named owner and senior manager. A written statement of what it may and may not do. Testing against known and difficult cases. A data protection impact assessment and supplier due diligence.
Emma And for customers?
George A human route. Monitoring of outcomes by customer group. And a way to switch it off. Then keep the evidence, because we monitor outcomes needs numbers behind it.
The written material
Automated decisions about customers
Declining a loan, pricing an insurance policy or refusing a claim has a significant effect on a person. Since 5 February 2026 the UK GDPR permits such a decision to be made solely by automated means on ordinary personal data, with safeguards. The customer must be told. They must be able to make representations, to obtain human intervention and to contest the decision.
Human involvement has to be real. The Information Commissioner's Office says in draft guidance that it must be active rather than tokenistic, by someone who understands the system and has the authority to change the outcome. Where special category data, such as health data in an insurance decision, is involved, stricter conditions apply.
Discrimination does not need intent
The Equality Act 2010 applies to the provision of financial services. A model that uses a postcode, a shopping pattern or a device type can produce different outcomes for groups defined by race, sex, age or disability without ever being given those facts. That can be indirect discrimination, and the Consumer Duty separately requires a firm to check whether any group of customers is getting worse outcomes.
So monitoring is not optional. A firm needs to compare outcomes across groups, to understand why they differ, and to be able to justify the difference or remove it.
Information or advice
The line between giving information and making a personal recommendation does not move because a tool is doing the talking. A personal recommendation on a regulated product, based on a person's circumstances, is regulated advice and needs the permissions, the suitability assessment and the record that go with it.
A chatbot that is asked 'what should I do with my pension?' and answers with a course of action for that customer may have crossed the line, whatever its disclaimer says. This is the concern at the centre of the Mills Review. Inside a regulated firm, test what the tool says when it is pushed, restrict it to what you are permitted to do, and hand over to a qualified person where advice is needed.
Explain it, test it, record it
If a customer asks why, the firm has to be able to answer in terms the customer can understand, and so does the senior manager when the regulator asks. A tool nobody in the firm can explain is one the firm cannot govern.
Before a tool is allowed to affect a customer, a firm should be able to show a named owner and senior manager, a written statement of what the tool may and may not do, testing against known cases including difficult ones, a data protection impact assessment, due diligence on the supplier, a route for customers to a human being, monitoring of outcomes by customer group, and a way to switch it off.
- A named owner and senior manager
- A written statement of what it may and may not do
- Testing against known and difficult cases
- A data protection impact assessment and supplier due diligence
- A human route for customers
- Monitoring of outcomes by customer group
- A way to switch it off
Knowledge check
The knowledge check and your certificate need a free account, so that your progress and results can be saved as evidence.
The learning itself stays free and open. You are reading all of it right now without an account.
Was this module useful? Tell us in two minutes, it decides what we improve next.