Disclaimer: This is based on a true story.

If you ask most teams what the biggest problem in customer support currently is, most of them will say speed. We’re too slow to respond, too slow to fix issues, too slow to close tickets.

But from my perspective, the real problem in support is complexity. More playbooks, more decisions, more tools.

When complexity grows, everything feels slower. That isn’t because teams are slow. It’s because the system behind them is unclear. And complexity doesn’t move fast. It compounds.

Solving a ticket or navigating a maze?

Imagine your first day as a support engineer. You log in, you’re motivated, and you’re ready to solve problems. Your first ticket comes in. Easy, right?

You check one playbook, then another, then a third. You open three tools, then five, then seven. You check with another team, and you wait.

So here’s my question: are you solving a ticket, or navigating a maze?

That’s exactly what customer support became for us at Cardlytics. Our ticket volume was increasing, edge cases were multiplying, and the issues we were dealing with needed more context. That led to more time per ticket, more coordination and, especially, more burnout. That wasn’t because the team lacked effort. It was because the system they were operating in wasn’t designed for the complexity it had created.

So instead of the ticket or maze question, we asked something different: What if the job of support isn’t to solve tickets? What if it’s to design the system, and train your team to teach that system how tickets should be handled?

AI Across the Customer Journey ebook

Using AI in support for clarity over speed

When most people hear "AI in customer support," they immediately gravitate toward speed: faster replies, fewer tickets, more deflection. But speed alone doesn’t solve complexity, and it doesn’t create consistency. Two agents on your team might give different answers to the same question because the system behind them isn’t clear.

So what if AI isn’t about going faster? What if it’s about being clearer? Clearer rules, clearer boundaries, clearer decisions. That became our focus, and it’s why my team and I built an internal support tool called Insysio.

Insysio is an agentic support system. By agentic, I mean a system that can observe context, take action across tools, progress work end to end, and has proper guardrails to stop and hand off. That’s different from an AI assistant or a copilot.

Why full automation is often the wrong goal

After we built Insysio, a lot of people got interested and asked for a copy. Most of the teams I spoke to thought autonomy was the goal: full automation, with AI solving 100% of tickets. That’s often wrong, because autonomy isn’t a technical decision. It’s a risk decision. Are you willing to take the risk if it gets it wrong? Probably not.

Think about it this way. When you hire a support engineer, you don’t give them root access on day one. You don’t let them handle high-severity incidents on their first day. That isn’t because they aren’t smart. It’s because the risk is too high. That’s exactly how AI should be treated.

Is having a high customer support ticket volume ever a good thing?
Volume tells you something happened. It can't tell you what to do about it until you know why. The same number can mean a critical bug or a fast-growing customer base.

2 AI agent failures and what they taught us about guardrails

We deployed a lot of agents, and we got things wrong along the way. Here are two real stories.

Failure 1: Incident updates that missed the impact

We deployed our first AI agent to handle both support and incidents. At first glance, it worked. Updates were faster, but they were flatter and lacked nuance. They were technically correct, and the agent did a great job with the playbook, but the updates were operationally incomplete.

In one specific scenario, we let the agent send incident updates. Because it lacked nuance, it missed which customers had actually recovered and which hadn’t, and it sent the wrong update to customers who hadn’t. We lost trust with those customers, and our ticket volume went up.

So we asked hard questions. What did we miss in the system, the prompt and the tools? In testing, everything looked fine. But production isn’t like that. Production is messy and uneven, and we hadn’t designed the agent to understand impact.

Here’s a question worth asking yourself. Say you build an agent and its playbooks, and it does a great job on 95% of tickets. But the 5% it fails on belong to your most important customers. Is that a success? Definitely not. For us, it was a failure of the risk model we’d put together.

AI‑first support playbook
A practical framework designed to help support teams adopt AI in a focused, measurable, and sustainable way.

Failure 2: Confident guessing

This is the second issue, and one I think you should review whenever you build AI agents. You’ll probably have a master prompt, then playbooks, then tools. If you don’t design your prompt with the right guardrails, then when a customer request comes in, AI will confidently guess and send the wrong answer. That was our second biggest problem.

In both cases, the bottom line was the same. What failed wasn’t the model. It was the boundaries.

Guardrail gaps can be very specific. Say you’re a payment platform, and someone emails to say they’re struggling to implement your API, with a link in the email. Without proper guardrails, AI will go to that link, read the content and answer based on it.

Here’s one that happened to us recently. A cashback request came in for an online purchase, but the campaign only gave cashback for in-store purchases. The customer put some language in the email that made the AI think the purchase was eligible, and it issued the cashback. It was only $30, but we weren’t supposed to issue it.

Free resources worth paying for: Templates, reports and expert content – no card required – with a Customer Success Collective Insider membership.

How we redesigned the system: AI-assisted vs AI-native support

So we redesigned the system. AI now handles low-risk tickets: the most repetitive, most time-consuming ones for the team. That’s 30 to 40% of our tickets, which saves the team a lot of time.

The rest are the most complex scenarios, the ones that need a human touch, and the team handles those extremely well. As we gained confidence in the system, we started covering those complex scenarios one by one. As the team learned more about the system and about prompting, they put together better playbooks and tools to handle them. With this approach, we eliminated the risk.

There’s an important difference here between AI-assisted and AI-native support. In AI-assisted support, humans still do the work. In AI-native support, AI does the work and humans provide the guardrails.

What I see in a lot of teams is that AI handles certain tickets and escalates the rest. The team then reviews the ticket and responds. That wasn’t our approach. When there’s a ticket AI can’t handle, the team reviews it, updates the prompt, updates the playbook, then hits the "process ticket" button to let AI handle it. That let us take on the most repetitive work and improve the system every day.

How to lead your team through complex customer escalations with empathy
Is empathy a soft skill? No, it’s a strategic advantage. Learn to guide support teams through crisis and turn complaints into wins.

How customers reacted to AI-native support

Some context on where this runs. I lead two support teams at Carlytics, which manages banks’ cashback programs. One team handles high-level API and SDK implementation questions from banks, including some of the biggest. The other handles requests from banks’ customers when they don’t get their cashback.

With cashback requests, we often don’t get feedback. Customers get their cashback and move on. For the API team, once a ticket is closed, we use AI to analyze the tone of the conversation and sense whether the customer was happy with the resolution. If not, we send a couple of very personalized questions, as if they came from the support engineer, asking whether the resolution worked for them. Based on the results, we update our responses and the way we handle tickets.

Part of why we could move fast is that our volume isn’t that high. Cashback requests are about 10,000 a month, and API requests from banks are about 50 a day. But they’re very high-touch customers.

AI for Customer Support Leaders 2026 Report

How to train support engineers to be system designers

Most importantly, we trained our team. We trained our support engineers to be system designers: how to analyze a ticket, how to review a prompt, and how to put together a playbook that AI can actually understand. Instead of asking "How do I solve this?" we asked, "How should this always be handled?" Their job was no longer to close tickets. It was to teach the system how tickets should be closed.

We didn’t tell them to go and learn to code. We used Replit to vibe-code the platform, and then to build a feature where a support engineer can pick a ticket, talk to the AI and give feedback: "You made the wrong decision here. You’re supposed to say this." The AI talks back, and they align on what the answer should be.

Not everyone was comfortable with it. Two support engineers decided to move to our campaign ops team. Not everyone wants to be a coder, and not everyone is interested in how these things work. Maybe they’re scared – who knows. But most of the team enjoyed it. Because we’re adopting new technologies quickly, they’re gaining new skills, and wherever they go, they’ll be in a better place.

I had a team of 20, but unfortunately we had to scale down to four, not because of AI but for different reasons. With the training we provided, the team now feels more confident. They’re not just support engineers anymore. They’re AI support engineers who can improve the system.

How to structure prompts so AI agents don’t guess

To prevent confident guessing, we use more than two pages of guardrails in our prompt: don’t do this, don’t do that. It’s a lot of language to stop the model making mistakes, and getting there takes a lot of time.

Structure matters too. Most of you probably use Markdown when you put together prompts, with headers, titles and sections. Your prompt should be structured so AI can understand what’s high priority. A single pound symbol (#) is the highest priority for your model, and a section with two (##) means lower priority. You need to make sure you’re putting those together correctly.

We also separate the master prompt from the playbooks. The master prompt is where the guardrails go, and each playbook covers how to solve a specific ticket. You can’t put everything in the master prompt, or you’ll be paying hundreds of dollars every time you ask AI something. You have to reduce token usage when you make the API call, and structure the master prompt and playbooks so AI doesn’t hallucinate and gives the answer you need.

AI for Customer Success Certified

How long it took to build (and win over leadership)

It was a struggle. I spent seven months vibe-coding the platform, and the first three months were all by myself. When I first pitched it to leadership, they didn’t hold back. They said it was something I’d put together myself and definitely not for enterprise.

So I got more people involved. Engineers signed on, and my support team learned Replit and helped me. Around six months in, I pitched again, and this time leadership was impressed with the results, because I’d implemented backtesting. I could show where we were spending a lot of time, and how much more efficient we’d be if we let the platform answer.

To give a very specific example, a bank customer will often email something like, "We’re seeing high latency. Get on a call." That raises a lot of questions. What API? How long? Is it ongoing? AI can take care of these kinds of issues for us. I showed leadership all the scenarios like that, and that’s what convinced them.

What human in the loop really means

For us, human in the loop doesn’t mean tickets get escalated to a support engineer who then handles them. It means the team improves the system so the system can handle them. That’s how we built a sustainable system, and everyone is happy.

From my point of view, the future of support leadership isn’t building the team. It’s building AI systems that handle all customer requests.


This article has been adapted from a talk Burak gave at Customer Support Summit New York in 2026. 

Happened to miss the real thing? If you’d like to watch his full talk OnDemand, you can do so with a Pro+ membership – and catch up on hundreds of hours of exclusive thought leadership content bursting with strategies and solutions for your customer success function.

Start the conversation

Become a member of Product Marketing Alliance to start commenting.

Sign up now