Where AI actually belongs in a service organisation
The useful applications are unglamorous, internal, and mostly invisible to customers. The visible ones are where the reputational risk sits.
Most organisations we work with have now had the conversation about where language models fit into their operation. A striking number have arrived at the same first answer: a customer-facing assistant on the website. It is the most visible option, the easiest to demonstrate, and close to the worst place to start.
It is the worst place to start because it maximises exposure and minimises control. The interaction is unbounded, the audience is the public, the failure is quotable, and the value is usually a marginal improvement on a search box that could have been fixed for a fraction of the cost.
A test worth applying
Before deploying a model anywhere, we ask three questions about the task. Is there a person who reviews the output before it has consequences? Is the cost of a wrong answer bounded? And would we be comfortable explaining the mechanism to the person affected by it?
Tasks that pass all three tend to be internal, assistive and narrow. Tasks that fail one of them tend to be the ones that reach a steering group as a strategic opportunity.
Where it earns its place
- Triage. Routing an inbound enquiry to the right team, where a human still handles it and a wrong route costs a few minutes.
- Drafting for review. Producing a first draft of a summary, a response or a record that a person edits and owns.
- Retrieval over an organisation’s own material. Finding the relevant policy or precedent, with a citation, so the person can check it.
- Structuring unstructured input. Turning free text into fields, with the original preserved and the extraction visible.
- Comparison at scale. Flagging inconsistencies across large document sets for a person to examine.
What these share is that the model is upstream of a decision rather than making one, and that a wrong output is caught by an existing process rather than by a customer.
Where it does not
Eligibility decisions. Anything with a statutory or contractual consequence. Anything where the affected person has a right to an explanation that the system cannot produce. Anything where the failure mode is a confidently worded wrong answer delivered to somebody with no way to check it.
The question is not whether the model is accurate enough. It is what happens on the occasions when it is not, and who bears that.
Design obligations
Where a model is used, the interface carries obligations that are easy to skip. Tell people that a machine produced the output. Show what it was derived from. Make correcting it as easy as accepting it — if the accept button is one click and the correction is a form, you have designed for a metric rather than for the work. And do not use uncertainty language as decoration; if the system cannot express genuine confidence, saying ‘I think’ is a costume.
None of this is a reason to avoid the technology. It is a reason to spend it where it compounds — on the internal work that is slow, repetitive and reviewed — rather than on the demonstration that is visible, unbounded and hard to withdraw once customers have seen it.
