How to avoid the most common and expensive AI mistakes in customer experience

As businesses try to meet rising customer expectations for speed, personalization, and efficiency, AI has become essential for scaling operations without compromising service quality.

From simple FAQ chatbots to sophisticated artificial intelligence (AI) agents that resolve issues, automate workflows, and enhance human agents, AI’s role in customer experience (CX) is rapidly evolving. 

But with so many AI solutions on the market, how do you know which one is right for your business?

In this article, we’ll break down the five key questions every business should ask before launching AI:

• Should I adopt AI?
• Should I build AI myself?
• Should I use my help desk’s AI add-on?
• Should I go with AI trained on public data?
• Should I invest in a dedicated AI solution?

By the end, you’ll have a clear framework to evaluate AI solutions, avoid costly mistakes, and choose the best approach for seamless, scalable customer support.

The limitations of traditional generative AI in customer support

Generative AI is the buzzword of the moment, and for good reason—it has made significant strides in automating certain tasks. 

However, when it comes to resolving customer support issues, generative AI as it exists today has notable limitations.

The majority of generative AI tools focus on answering FAQs by pulling information from a knowledge base. This approach, known as retrieval-augmented generation (RAG), can resolve about 20% of customer support inquiries. Many of the tools integrated into popular help desks, such as Intercom, rely on this retrieval-based technology.

But what about the other 70% of customer issues? These often require nuanced reasoning, complex logic, API integrations, password resets, troubleshooting, and more—tasks that go beyond simple text generation. 

This is where agentic AI comes in.

The future of support

Our Agentic AI is designed to operate like a human agent. It doesn’t just generate responses—it reasons, plans, and takes action. This means it can execute tasks autonomously, making it a game-changer for customer support automation.

At Forethought, our AI doesn’t just retrieve answers; it understands business logic, connects with APIs, and resolves issues without human intervention. Our second key differentiator is that our system is fully trained on a company’s specific data, making it highly effective in delivering accurate and relevant solutions.

Because it is both agentic and data-driven, our AI can assist in multiple capacities—it can fully resolve issues, support human agents in a copilot mode, and even generate valuable insights for businesses.

Agentic AI is set to redefine the customer support experience. While the term might not be widespread yet, it's only a matter of time before it becomes a key player in AI-driven automation. 

How to launch AI effectively

Customers today expect fast, personalized, and seamless support experiences. Customers expect chat responses within minutes—no one wants to wait on hold for an hour or receive an email response 24 hours later. They want help instantly.

Beyond speed, personalization is just as critical. Around 76% of customers expect businesses to know who they are, what products they use, and what issues they’ve encountered before. 

They don’t want to start from scratch every time they reach out for support. And companies are feeling this shift—87% of support teams report that customer expectations are rising year over year.

As more customer interactions move online, businesses must prioritize making these experiences fast, personalized, and scalable. The best way to achieve this? First, by optimizing and upskilling support teams. But as demand grows, scaling solely through human effort becomes impractical. That’s where AI and automation come in.

The five-step AI playbook

Before diving into AI, it’s crucial to ask five key questions. These will help determine the right approach and ensure AI adoption aligns with your business needs.

By carefully considering these five questions, businesses can make informed decisions about AI adoption and implementation, ensuring a balance between efficiency, scalability, and customer experience.

Should I adopt AI?

The short answer: Yes. Every business should consider AI adoption.

At Forethought, we surveyed thousands of support leaders to uncover the real reasons behind AI adoption and the nuances that come with it. The insights are clear—AI isn't just about deploying a chatbot that responds to customer queries. Instead, businesses should think strategically about when and why AI makes sense.

1. Predictive capabilities

One of the most valuable use cases for AI is its ability to analyze customer behavior and detect support patterns. The goal isn’t just to have a bot that responds to customers—it’s to gain insights into what’s happening within your support ecosystem. 

Understanding these patterns allows AI to provide more intelligent responses and help businesses anticipate issues before they escalate.

2. Proactive insights

AI shouldn’t just react—it should be able to identify potential problems before they arise by monitoring customer interactions and support trends. By leveraging AI for proactive insights, businesses can preemptively take action, reducing friction and improving customer satisfaction.

3. Scalability

For smaller support teams, AI can be a force multiplier, helping agents be more efficient. But for larger teams—especially those with 20 or more support agents—AI becomes essential. 

Once a company reaches this scale, the volume of interactions and data generated is significant enough for AI to deliver real value. With a large enough dataset, AI can optimize workflows, automate responses, and improve overall efficiency.

The key takeaway? AI is no longer a futuristic concept—it’s a necessity for businesses looking to scale their support operations efficiently while maintaining a high-quality customer experience.

Should I build AI myself?

The answer to this is a qualified no—there are some cases where building AI in-house makes sense, but for most companies, the pitfalls outweigh the benefits.

The challenges of building AI in-house

High customer expectations

Customers demand fast, personalized, and accurate experiences. If you build your own AI, you must ensure it meets these high expectations. While it may be easy to automate responses for common issues—like “Where’s my order?” for an e-commerce business—customer support is often far more complex. 

AI needs to handle nuances, edge cases, and unpredictable requests, requiring continuous retraining and fine-tuning to avoid issues like model hallucinations.

Long development timelines

It’s one thing to build a quick demo-quality AI—one that can answer basic questions using an off-the-shelf model like GPT. But getting to 50-60% resolution rates (where AI meaningfully reduces workload) requires significant engineering investment and a long development cycle. 

Companies often underestimate the effort required to move from a simple chatbot to an AI that can truly handle complex customer interactions.

Unexpected costs

Developing a dedicated AI system can be far more expensive than anticipated. Training a model on OpenAI or another provider can cost millions of dollars, not to mention the additional expenses for infrastructure, engineering salaries, and ongoing maintenance. For most businesses, this cost-benefit equation doesn’t make sense.

When does it make sense to build AI in-house?

Some companies—typically those operating at massive scale—have found success in building their own AI. A prime example is Klarna, which processes $80 billion in gross merchandise volume per year. 

They invested millions in AI development and saved $40 million in operational costs. While that’s impressive, it represents less than 1% of their total volume—a worthwhile savings for Klarna, but not for most businesses.

For companies of that scale, AI can deliver real impact. But for most businesses, off-the-shelf AI solutions are far more practical, cost-effective, and scalable. Instead of building from scratch, companies should leverage existing AI technologies designed specifically for customer support.

For the vast majority of businesses, the answer is no—don’t build AI yourself. Instead, investing in proven AI solutions allows companies to scale efficiently without the immense time, cost, and complexity of building AI in-house.

Should I leverage the help desk AI add-on?

This is a question we hear all the time at Forethought: Should we use the built-in AI from our help desk provider? The answer depends on what level of automation and control you need.

Not all AI is created equal, and understanding the differences between traditional chatbots, generative AI, and agentic AI is key to making the right decision.

1. Traditional decision tree-based chatbots

For over a decade, businesses have used rule-based AI chatbots, which rely on structured decision trees. These chatbots provide control—you can set up workflows, rules, and scripts—but they come at a cost. 

The experience can feel clunky for customers, and setting up complex workflows takes time. While they can scale automation, they often require a lot of maintenance and lack flexibility when handling real conversations.

2. Retrieval AI (the modern generative AI wave)

The rise of ChatGPT-like retrieval AI has led to a new generation of AI-powered help desk add-ons, such as Intercom Fin, Zendesk AI agents, and others. 

These systems use retrieval-augmented generation (RAG), meaning they pull answers from a company’s knowledge base and present them in a conversational format.

If your company has a robust knowledge base (50+ well-structured articles) and only needs simple question-answering AI, a help desk add-on may be enough. Expect 10-20% deflection rates at best.

3. The rise of agentic AI (the next evolution)

The latest and most advanced approach is agentic AI, which moves beyond simple retrieval and actually takes action. This type of AI doesn’t just answer questions—it performs tasks, automating workflows like issuing refunds, troubleshooting SaaS product issues, or handling account updates.

Unlike traditional chatbots, agentic AI operates in natural language without rigid decision trees. It combines the best of both worlds—the conversational ability of retrieval AI with the functional power of automation.

So, should you use a help desk AI add-on?

• If you only need simple question-answering AI and don’t mind limited automation, yes—a help desk AI add-on may be sufficient.
• If you want AI to take actions, automate workflows, and go beyond answering FAQs, you’ll need something more powerful—a dedicated AI vendor specializing in agentic AI.

If your goal is deeper automation and long-term AI-driven efficiency, choosing a dedicated AI solution is the way to go.

Should I go with AI built on public data?

One of the most overlooked aspects of AI in customer support is what data the AI is trained on—and it makes a significant difference in how effective your AI will be.

There are two primary ways AI models are trained for customer support:

1. AI trained on public data

Most large language models (LLMs) like GPT, Claude, and Llama are trained on a broad snapshot of the internet. They do not inherently understand your business, your customers, or your policies. 

While they can generate responses based on general knowledge, they lack the context of your specific workflows, making them less effective for personalized and accurate support.

That said, plugging a public AI model into your knowledge base can provide a baseline level of assistance. It can search through your existing articles and provide answers, but it won’t capture the depth of your customer interactions or unique business processes.

2. AI trained on your historical conversations

A more powerful approach is to train AI on your company’s historical data—including past chat logs, phone transcripts, and email exchanges. This type of AI learns from real customer interactions, business policies, and company-specific logic, leading to:

• Higher accuracy in responses
• Better resolution rates compared to AI that relies solely on public data
• More personalized customer support based on past interactions

When evaluating AI vendors, a key question to ask is: "Does this AI train on my business’s data, or is it only trained on public data?"

Security and privacy considerations

If you're leveraging AI trained on your company’s data, security and privacy become critical concerns. Businesses need to ensure customer data is protected while still benefiting from AI-driven automation.

Key security measures to look for include:

• SOC 2 compliance for general data security
• HIPAA compliance if handling healthcare data
• PCI compliance for businesses dealing with credit card transactions
• Zero-training policies with LLM providers to ensure customer data is not used for future AI training

If you want AI that delivers real business value, it should be trained on your company’s data—not just public knowledge. However, security and compliance must be a priority when making this decision. 

Should I go with a dedicated AI solution?

The final question in our AI playbook is one of the most important: Should you invest in a dedicated AI solution? The answer is a strong yes if you’re serious about security, automation, and long-term scalability.

1. Security and data privacy

If your AI is trained on your company’s data, security and compliance become critical factors. Dedicated AI vendors prioritize robust security protocols, including:

• SOC 2, HIPAA, and PCI compliance (depending on your industry)
• Zero-training policies to ensure LLM providers don’t train on your data
• Automated data redaction to protect personally identifiable information (PII)

A generic AI solution may provide basic chat functionality, but a dedicated AI partner ensures your data is protected and handled responsibly.

2. Seamless integrations for automation

One of the biggest advantages of dedicated AI vendors is deep integrations with your tech stack. As AI shifts towards agentic AI, your system must be able to connect with databases, APIs, and internal tools to take real action—resetting passwords, processing refunds, updating customer accounts, and more.

For example:

• E-commerce companies need AI that integrates with Shopify, Magento, and fulfillment systems.
• SaaS companies need AI that connects with MongoDB, Snowflake, or internal admin dashboards.
• Enterprises require AI that works with custom backend systems and CRMs like Salesforce or Zendesk.

Most generic AI solutions (like those built into help desk add-ons) don’t have the extensive integrations required for deep automation. A dedicated AI vendor ensures AI doesn’t just answer questions—it executes actions.

3. Future-proofing your AI strategy

AI is evolving rapidly, and customer service automation is just one piece of the puzzle. Beyond responding to customer inquiries, AI can:

• Act as a copilot for human agents, enhancing their productivity.
• Generate insights for better workforce management.
• Automate workflows beyond customer support (e.g., HR, finance, sales).

By choosing a vendor that is continuously innovating, businesses can future-proof their AI investment. Rather than outgrowing a basic solution and switching vendors later, it’s smarter to start with a partner that can scale with your needs.

If you want more than a basic chatbot, a dedicated AI solution is the way to go. Security, integrations, and long-term AI capabilities should be key factors in choosing the right vendor. 

AI is not just about answering questions—it’s about taking real action, automating processes, and growing with your business.

Key takeaways from the AI playbook

We’ve covered five essential questions every business should ask before adopting AI. These considerations are not just useful when working with Forethought—they are critical for evaluating any AI vendor. 

The most important takeaway? Not all AI is created equal.

Understanding the different facets of AI—from security and integrations to automation and long-term scalability—is key to making the right decision for your business. Whether you choose to build AI yourself or work with a vendor, knowing what to look for will ensure you invest in a solution that truly delivers value.