What is a chatbot?
A chatbot is a software program that holds a conversation with a person in text or voice, interprets the request, and returns an answer or takes an action. It runs inside a website widget, a messaging app, or a phone line. Most customers meet one before they ever meet a support rep.
The word covers software separated by decades of capability. Rule-based systems matched keywords to scripted replies; today’s chatbots run on large language models that generate answers from retrieved company content, which is why one term now describes tools with wildly different accuracy on the same question.
How a chatbot works
A modern chatbot runs as five layers, and a failure in any one of them looks identical to the customer: a bad answer.
The channel layer receives the message from a widget, WhatsApp, email, or a phone call and normalizes it into text. The understanding layer works out what was asked; older systems classify the message into a predefined intent, while model-based systems read the phrasing directly and keep its detail intact. The retrieval layer finds supporting material, pulling passages from help articles, policy documents, and past tickets, and when a question needs several searches chained together, agentic RAG plans and refines them.
The state layer tracks what the conversation already established, which is where AI agent memory keeps an order number alive across turns so nobody is asked for it twice. The action layer calls the systems that change something: issuing a refund, updating an address, pausing a subscription. An AI agent framework is the software that wires those layers together and decides which one runs next on any given turn.
Types of chatbots
Five architectures ship under the same label, and the type determines the failure mode you inherit.
Rule-based chatbots: Fixed decision trees where every path is authored by hand and the customer picks from buttons. Predictable, and blind to anything unscripted.
Intent-based chatbots: A classifier maps free text onto a curated list of intents, each wired to a scripted response or a small workflow. Accuracy collapses as the intent list grows.
Generative chatbots: A large language model composes the reply directly from the prompt and the conversation history. Fluent everywhere, and confidently wrong where the model has no grounding.
Retrieval-grounded chatbots: A model answers only from passages pulled out of your own documentation and records. Quality tracks content quality almost exactly.
Voice chatbots: The same stack fronted by speech recognition and synthesis, with a latency budget measured in hundreds of milliseconds.
Chatbot vs AI agent vs virtual assistant vs live chat
Buyers use these four words interchangeably, and the vocabulary matters because it sets what the tool is expected to do once it is live. An AI agent plans multi-step work and executes it inside connected systems. A virtual assistant handles broad personal tasks across many domains, from calendars to reminders to music. Live chat puts a human being on the other end of the widget in real time. A chatbot is the conversational surface itself, and the same widget can sit on top of thin scripting or a full reasoning stack, which is why the split between AI agents and traditional chatbots shows up only after the answer is delivered.
What it holds | Ownership | Who reads it | AI-retrievable | Choose it when | |
|---|---|---|---|---|---|
Chatbot | Dialogue logic, intents, grounded answers | Support or CX ops | Customers, in the channel | Yes, when grounded in retrieval | Repeat questions need an instant answer |
AI agent | Goals, tools, plans, running state | CX plus engineering | Customers and internal systems | Yes, and it also writes | The request needs steps executed end to end |
Virtual assistant | Broad skills across personal domains | Platform provider | Individual users | Partly, skill by skill | The use case is personal productivity |
Live chat | A message queue and human replies | Support team | Customers and agents | Transcripts only | The case needs human judgment now |
If most of your incoming volume is repeat questions with documented answers, a chatbot covers it. If the request requires steps taken inside a billing or order system, you need agent behaviour. Live chat stays as the floor under both.
Why chatbots matter for customer experience
Support volume arrives in a shape people handle badly: hundreds of near-identical questions about order status, password resets, and return windows, mixed with a few dozen cases that genuinely need judgment. When nothing absorbs the repetitive layer, the queue lengthens for everyone and the hard cases wait behind the easy ones.
The failure mode without a chatbot is a queue that treats every contact as equally expensive to serve. The failure mode with a badly built one is sharper: a customer reads a confident wrong answer, acts on it, and returns with a second problem the first answer created.
The tradeoff is real and rarely priced in. Every question the bot closes is a conversation your team never reads, so signals a human would have flagged (a broken checkout step, a misleading policy page) stay invisible until someone reviews transcripts on purpose.
How is a chatbot measured?
Public benchmarks do exist for the pieces underneath a chatbot. Intent classification is scored on curated datasets such as CLINC150 and BANKING77, and retrieval quality on suites like BEIR. Those taxonomies belong to the datasets: their intent labels come from the corpus, and yours come from your ticket history, so a strong dataset score transfers no promise about your refund policy. No standards body sets a figure a support team is expected to hit.
Measurement is therefore local and manual at the core. Sample real conversations weekly, label each one resolved, wrongly answered, or escalated, and read chatbot containment rate next to that accuracy review so a high containment number cannot hide a bad one. The NIST AI Risk Management Framework defines the measure function that gives this shape: what gets measured, who reviews it, and on what cadence.
How AI agents change what a chatbot is
The mechanism is a planning loop. A scripted chatbot maps one input to one output. A system built on agentic AI receives a goal, breaks it into steps, calls a tool, reads what came back, and decides what to do next, repeating until the goal is met or it hands off. That loop is the whole architectural change, and the rest follows from it.
The consequence for support is that the unit of work moves from the answer to the outcome. A chatbot that could explain the return policy becomes one that checks the order date, confirms eligibility, issues the label, and updates the ticket, which is what chatbots that execute CRM workflows do in production. It relocates the risk too: a wrong sentence is embarrassing, and a wrong write into a billing system is money you have to claw back.
What to look for in a chatbot
Judge a chatbot on the axes that decide whether it survives real volume.
Coverage comes first: how many of your actual contact reasons it can close end to end, measured against a tagged sample of last quarter’s tickets. Integration surface is binary per system, since the bot either reads an order and writes a refund through an API or it only describes the policy. Governance decides who approves a change to an answer and how that change is logged, because an edited answer ships to every customer at once.
Two compliance frameworks bite hardest here. GDPR applies because transcripts are personal data, so deletion, export, and an enforceable retention window on chat logs are engineering work. SOC 2 Type II matters once the bot holds credentials into systems that move money.
The constraint teams underestimate is write access: approval from whoever owns the billing system usually delays go-live longer than any model limitation.
Chatbots and the rest of the support stack
A chatbot is one surface among several drawing on the same content and the same systems. When a conversation escalates, agent assist picks up the thread, handing the human rep the suggested reply and the retrieved policy the bot was working from, so the customer never restarts the explanation. Where systems from different vendors have to cooperate on one case, the A2A Protocol defines how one agent passes a task to another and receives the result. The stack holds together when every layer reads the same source of truth.
What does a chatbot mean in plain terms?
Think of a chatbot as a front desk that never closes and never tires, staffed by someone who has memorized the manual and can look things up in a second, but who has only the manual to work from. Ask something the manual covers and the answer arrives immediately. Ask something outside it and everything depends on whether the desk admits the gap and passes you to a person who can help.
Take the desk away and a customer with a two-minute question stands in the same line as a customer with a two-hour problem. Both experiences get worse at once.
The tradeoff people underestimate is that a chatbot makes your written policies literal. Anything vague in the help center becomes vague in the answer, at volume, in front of everybody.
Common chatbot mistakes
Shipping without an honest exit is the first. If the system has no confidence threshold and no way to say it does not know, it will guess on every out-of-scope question, and the mechanism guarantees that guessing looks exactly like answering to the customer reading it.
Judging the bot by deflection alone is the second. A conversation the customer abandoned in frustration and a conversation the bot resolved both register as one contact that did not reach a human, so the metric climbs while satisfaction falls.
Building the flow before fixing the content is the third. Retrieval returns whatever is written, and when two help articles state different return windows, no amount of prompt tuning decides which one is correct.
Granting broad write access early is the fourth. Scoping refunds by amount, order age, and account history takes a week of policy work that teams skip, and the first bad write is what forces them to do it anyway.
Frequently Asked Questions
What is a chatbot in simple words?
A chatbot is a program you type or talk to that understands your question and replies, usually inside a website widget, a messaging app, or a phone line. Simple ones follow scripted menus. Modern ones read your wording, look up the company’s own policies and records, and can complete the task for you.
What is the difference between a chatbot and an AI agent?
A chatbot is the conversational surface a customer interacts with. An AI agent is the reasoning behind it: it takes a goal, plans several steps, calls tools, checks results, and adjusts until the task is done. Every AI agent needs a surface, and plenty of chatbots have no agent behind them at all.
Chatbot vs virtual assistant: what separates them?
A chatbot is scoped to one domain, typically a company’s own support, products, and policies, and lives in that company’s channels. A virtual assistant serves an individual person across many unrelated domains: calendars, reminders, media, smart devices. The chatbot is deep and narrow. The assistant is broad and shallow.
What are the main types of chatbots used in customer support?
Chatbot types split by architecture: rule-based decision trees, intent classifiers trained on a curated label set, generative models composing free-form replies, retrieval-grounded systems answering only from your documentation, and voice chatbots running the same stack behind speech recognition. Support teams increasingly run retrieval-grounded systems, because grounding is what keeps generated answers tied to real policy.
Do all chatbots use AI?
Chatbots do not all use AI. Rule-based systems are pure logic: a fixed tree of buttons and keyword matches with no learning involved, and many live widgets still work this way. Systems that interpret free-text phrasing, retrieve relevant passages, and compose an original reply do use machine learning models to do it.
How accurate are chatbots at answering customer questions?
Chatbot accuracy depends far more on content than on model choice. A system grounded in current, well-scoped documentation answers common questions reliably; the same model on stale or contradictory articles produces fluent errors at the same speed. Measure it by sampling and labelling real conversations weekly, since aggregate deflection numbers hide wrong answers completely.