When we talk about AI in customer support, the conversation usually centers on efficiency gains and cost savings. But there's something more fundamental at stake – something that sits at the intersection of innovation and customer loyalty.
I want to share what I've learned about responsible AI deployment in customer-facing support environments, drawn from my experiences working with organizations of various sizes across different industries.
This isn't about talking about how “great” AI is. We all know the buzz. Instead, the conversation should be about what happens when it goes wrong, and whether we've prepared for that – especially when our customers are counting on us during their most vulnerable moments.
What trust in AI actually means for support teams
Trust in AI comes into play when there's safety at stake, when there's risk, uncertainty, or ambiguity. In customer support, that's essentially our MO.
Think about it – we're the only function in every company that touches customers 24/7, nonstop, and only during those risky situations. They come to us with questions, outages, and when things don't work. Sometimes their deadlines are on the line. Sometimes their jobs are on the line. This is why trust and safety really matter in our world.
And here's something worth sitting with: trust isn't accuracy. You could upload Wikipedia into an AI agent, and it still doesn't mean people will trust the system. LLMs give fluid, fast and well-formatted responses. But it doesn't mean those responses are accurate.
The system might handle 95% of situations correctly. But what if your customer's situation is in that other 5%? How do they know? How will they escalate? Who's accountable for the outcome? Accuracy alone doesn't answer any of those questions.
Understanding intrinsic vs. extrinsic trust
There are two types of trust we need to consider when deploying AI in support environments.
Intrinsic trust develops when you can see how the system reasons and judge that the reasoning is sound. You can verify the steps it took to make a recommendation, trace the decision path, and audit something internally.
Extrinsic trust is what develops when you interact with the system again and again, see that it gives you the right information – something that really helps us. Only then do we start trusting it.
Most AI deployments assume extrinsic trust will just emerge automatically. It won't. You have to engineer it. And the useful question isn't how good the AI is. What you need to ask yourself is what happens when it fails – and you have to assume it will fail.
The black box problem in AI support systems
Aviation gives us a useful reference point here. Most are familiar with the concept of the black box – a safety engineering system with traceability and post-incident reconstruction. When something goes wrong, you can go back and understand exactly what happened.
Do we have that with AI? Do we have that with LLMs? We don't. We really don't. We don't know why a certain prediction was made. We don't have full parameter inspection. We don't have true reasoning reconstruction. So intrinsic trust in AI, as it stands today, isn't something we can rely on in customer support contexts.
Hallucinations and the three failure scenarios
Hallucinations get talked about a lot, but it's worth being specific about when they happen. An LLM hallucinates when it responds confidently in a way that isn't supported by reliable knowledge, a verified source, or the user's input.
There are three scenarios:
1. The knowledge exists but the model didn't use it
This type can be mitigated using retrieval augmented generation (RAG) models. In layman's terms, instead of relying on the training model during question-answer situations, RAG first retrieves relevant documents or knowledge and then responds based on that.
If you don't have RAG models in your business, I highly recommend implementing them.
2. The training data was wrong
We can mitigate this using good knowledge bases. If you have a good program for that, you're in better shape. We also need to develop methods for the system to forget or unlearn something that was incorrect.
3. The model extrapolated beyond what it knows
This is where reality checks have to come into play. We have to understand that plausible doesn't mean faithful. Faithful is something you can trace back to understand how reasoning came about, and with AI, we cannot do that.
Sometimes the system recreates something that seems most plausible or that it believes is the best prediction based on something it recommended before that was liked or deemed positive. That's just the reality check we have to have.
Suppressing hallucinations is possible, but it also suppresses creativity. It's a striking balance. If we start putting too many guardrails in place, we may actually create problems with generative AI.
Invest before you save
We cannot idealize AI deployment. In leadership, especially, we have to tread carefully. Naturally, we want to show savings straight away – it's very tempting, and what the C-suite expects of us. However, it's wiser and more responsible to err on the side of caution. You may need to invest before you start saving.
The only reliable way to confirm that your AI is providing correct, safe answers for customer use is application-grounded evaluation. This means investing in putting your real users (A.K.A. your best experts) to test the answers that the AI engine gives you. Yes, it costs money. But this is the investment you need to advocate for in order to create trust.
You could upload an entire medical textbook into an AI agent, and it would give you accurate answers when answering exam tests. But that doesn't mean we can get rid of doctors. We can't. Plausible explanations don't mean we can provide a safe environment for our customers.
Customer Success Collective
Being realistic about robustness
Robustness is more fragile than it looks. Research shows LLMs can be confused by simple things — a bias toward the first answer in multiple-choice formats, results that flip with slight negation.
The responsibility sits with us, the support leaders. If we don't test it, we can't trust it. AI cannot take responsibility for itself.
AI adoption fails when we overpromise before we evaluate. It fails when we measure containment instead of confidence. It fails when we replace human judgment instead of augmenting it. AI should accelerate experts without reducing accuracy and accountability.
Key testing practices for AI in support
If you don't have testing practices yet for AI in your organization, here are some good starting points:
- Prompt refinement: You can’t expect customers to know prompt engineering. To circumvent this, offer dynamic menus or predefined prompts to guide them.
- Error pattern testing: Develop a practice of analyzing errors, catching them, categorizing them, and addressing them systematically.
- Human-in-the-loop practice: Always provide an easy path for critical items to go to humans. This is non-negotiable.
You might not be able to afford all of these practices. You might not be able to implement everything at once. But pick the ones you believe are most possible for you to embed into your practice:
- Data audits: Constantly revisit what information you don't have and create it. Make sure what you do have is the most up-to-date, like maintaining your knowledge base practice.
- Confidence scoring and guardrails: This brings to mind good old intent categorization – over-index to send queries to humans when something is undefined.
- A/B testing: Test two different AI implementations simultaneously and collect customer feedback to know which one performs better.
And of course, survey customer experience continuously.
Customer Success Collective
What customers really need to know
Customers don't want to know how you've architected your AI systems. If they start trying to troubleshoot it with you, trust has already broken down.
What they need is clarity. Be open about when AI is involved, what it's allowed to do, when a human steps in, and who owns the outcome. Only humans own outcomes, so AI should never operate without proper controls and a clear transfer path to a person.
Yes, transparency builds trust, but silence erodes it.
Preparing customer support agents for AI-centric support
Agents need to provide a feedback loop. They need to see the responses the AI produces and be well aware of the most common failure scenarios, including hallucinations. Training agents on how AI fails is just as important as training them on how to use it.
Your team structure needs to support this: the ability to train, test, deploy, measure, and improve continuously. It's not an easy task, and it takes real time to build properly.
The leadership role in AI deployments
Everything here comes down to judgment, and AI can't make judgment calls. Leaders have to.
Be humble about what AI cannot do today, which, let's be honest, is a lot. Assume it will fail. Invest in testing. Drive transformation but don't skip the interim steps in pursuit of savings. Be transparent with stakeholders, who may understand AI's limits better than we do.
Cassie Kozyrkov, former Google Chief Decision Officer, has argued that the more our tools do for us, the less we understand what we're actually asking them to do. The real test for leaders, she says, isn't mastering the tools. It's mastering judgment and decision-making.
We can't outsource our thinking to machines. Customer support is the same as it was 40 years ago — we're here to deliver reliable, trustworthy service to customers during their most vulnerable moments. Technology should strengthen that mission. If it's diluting it, something has gone terribly wrong.
Customer Success Collective
Making strategic decisions about AI deployment
Think about how much control you actually have. If you have a technology team within your organization — one that deploys AI, trains it, tests and fixes problems – you can take more risks and move faster. If that team is distant, has other priorities, and the process is slow, don't take those risks.
AI deployment touches product releases, product features, and back-end administrative tasks. You can't pick a tool and go with it alone. You need to align with other organizations, account for governance requirements, brand reputation, and cost.
If you're in a mission-critical space, be as careful as possible. If your customers come to you with questions that can wait, or you can go back and fix eroded trust, you have more room to move.
But in either case, don't just pick a tool and deploy it. Think through what it will do for customers at every step, how you'll control and monitor it, and what you'll do when it fails. You need quick human intervention options and quick rollback options.
AI isn't a value-add feature — everybody expects it to be there now. Think of it as the foundational platform you build on top of, not something you bolt on. And build trust into it from the start. If you don't, customers won't have tolerance for it.
This article has been adapted from Olga's talk at the AI for Customer Support Virtual Summit 2026.
Become a CSC Insider
Thank you for subscribing
Get exclusive insights, frameworks, and strategies from customer success leaders driving real business impact.
An email has been successfully sent to confirm your subscription.
Follow us on LinkedIn
