Decide where AI is useful
AI is most useful where participants ask open questions that structured journeys cannot anticipate. Registration, consent, and anything with a fixed correct answer are usually better handled with structured steps.
Choose providers and models deliberately
- Where data is processed and stored, and whether that meets your legal and funder requirements.
- Quality in the languages your participants use.
- Cost per conversation at your expected volume.
- How the provider handles data sent to it, including whether it may be used for training.
Build guardrails
Define what the AI may and may not talk about, and how it should respond to medical, legal, or safeguarding topics. Restrict answers to approved programme content where possible, and filter both inputs and outputs.
Tell participants
Make it clear when a participant is interacting with an automated system, and how to reach a person if one is available.
Plan fallbacks
Every AI step needs a defined path for when the service is unavailable, slow, or produces an unsuitable answer. The participant should always be able to continue.
Evaluate before launch
Build a set of realistic test questions, including difficult and out-of-scope ones, and review the answers with programme staff. Repeat the evaluation whenever the model, prompt, or content changes.
Monitor after launch
Review a sample of real conversations regularly, track fallback and escalation rates, and keep a log of AI requests and responses for audit. Treat monitoring as part of running the service, not a one-off check.