Module 2 of 2 · 45 minutes
Refunds, prices and putting it right
By the end of this module you will be able to
- Monitor an agent with a human in the loop
- Act quickly when an agent gets it wrong
- Apply refund and cancellation rights
- Keep automated marketing honest
- Review complaints as a source of evidence
Work through it
1 interactive for this module, built on the WAJD Teach engine. Nothing moves until you ask it to, and every one has a written version if you would rather read it.
Watch: Emma and George talk it through
2 minutes. Captions are on, and the same conversation is written out in full below. The voices are computer generated.
Emma George, the agent's live and it seems fine. Can I leave it to run?
George No. The guidance says regularly check how it's performing. That it's delivering the right results, behaving as intended, and complying with consumer law.
Emma Checked by whom?
George A person. It says make sure there's a human in the loop, actively checking that the agent's making correct decisions. Some models can hallucinate results that are inaccurate.
Emma Suppose I find it's telling people the wrong thing.
George Act quickly. For example by refining the prompts or the workflows. Especially if it deals with large numbers of people, or with vulnerable customers.
Emma Why the hurry?
George One wrong answer from a person is one wrong answer. One wrong rule in an agent is the same wrong answer to everybody, until somebody changes it.
Emma And if I know and leave it?
George The guidance says if you don't act quickly to address problems, you may end up breaking the law.
Emma Refunds are where ours struggles.
George That's the guidance's own example. The agent needs the Consumer Rights Act 2015, for goods that are faulty. And the Consumer Contracts Regulations 2013, for someone who's changed their mind after buying at a distance.
Emma They're different?
George Different rights, different time limits. A faulty kettle and an unwanted kettle aren't the same request.
Emma We also offer sixty days to return. Longer than the law.
George Then the agent must apply it. The guidance's example of a fault is exactly that. An agent not always taking the extended returns policy into account.
Emma What about the agent writing our promotions?
George Accurate prices, including all unavoidable charges. Paid endorsements properly labelled. And any offers or price reductions genuine.
Emma One cheap way to keep an eye on all this?
George Read the complaints. A cluster of customers saying the chat told them something different is the cheapest monitoring you'll ever get.
The written material
Monitor it
The guidance says to regularly check how well an AI agent is performing: that it is delivering the right results, behaving as intended and complying with consumer law.
It warns that some AI models can misinterpret data and hallucinate results that are nonsensical or inaccurate, and it says to make sure there is a human in the loop, actively checking that the agent is making correct decisions. Regular human oversight is how mistakes are caught.
Refine it quickly
If an agent is not performing as expected, and that is leading to non-compliant or potentially non-compliant outcomes, act quickly to address the problem, for example by refining prompts or workflows.
Speed matters most where the agent interacts with large numbers of people, or with vulnerable customers. One wrong answer from a person is one wrong answer. One wrong rule in an agent is the same wrong answer to everybody until somebody changes it.
Refunds
The guidance uses refunds as a worked example. When designing and training the agent, take into account consumers' statutory rights under the Consumer Rights Act 2015, for example if goods are faulty, and the Consumer Contracts (Information, Cancellation and Additional Charges) Regulations 2013, for example if they change their mind after buying at a distance. And take into account your own terms, such as an extended returns period.
Someone with appropriate experience should regularly review exchanges with consumers, checking that the agent's decisions properly reflect the nature of the request and any differences in time limits or amounts due. A faulty kettle and an unwanted kettle are different requests with different rights.
Prices, marketing and queries
Where an agent runs marketing, the guidance says materials should give accurate information about prices, which should include all unavoidable charges, and about products. Paid endorsements should be properly labelled. Any offers or price reductions should be genuine.
Where an agent answers queries, it must respond accurately about prices, products and rights, give consumers the information they need, and not make it difficult for them to exercise their rights. Someone with appropriate experience should review how it responds, and any complaints or feedback from customers.
Knowledge check
The knowledge check and your certificate need a free account, so that your progress and results can be saved as evidence.
The learning itself stays free and open. You are reading all of it right now without an account.
Was this module useful? Tell us in two minutes, it decides what we improve next.