Named entity recognition (NER)

What is named entity recognition (NER)?

Named entity recognition (NER) is a natural language processing task that finds spans inside unstructured text and assigns each one a category: a person, an organisation, a location, a date, a currency amount, or a domain-specific identifier such as an order number or a policy reference.

In customer support the categories rarely stop at people and places. A model working a billing queue is usually tuned on a dozen or more custom types, because an order ID, a tracking number, and a card number each trigger a different downstream action: a lookup, a status call, or a redaction.

How named entity recognition works

A named entity recognition system runs five stages, and each one can fail on its own.

Tokenisation splits the message into units the model can score. Contextual encoding converts each token into a vector that carries its neighbours, which is how a model separates "Apple charged me twice" from "the apple arrived bruised". Span labeling applies a tagging scheme, usually BIO, marking where an entity starts, continues, and ends. Type classification attaches the label. Normalisation resolves the raw span into a canonical value, turning "the 3rd of March" into a date field and "#55120" into a key an order API will accept. Most of the engineering effort goes into the custom types: annotate a few thousand real messages, fine-tune, then measure per type.

The model sits inside the broader natural language understanding layer, running alongside intent recognition, which decides what the customer wants while the tagger decides which objects the request points at. It also composes with text classification: one labels the whole message, the other labels parts of it.

Types of entities NER extracts, with examples

Take a chat message: "Refund for order 55120 still hasn't landed, I paid £42.60 on 3 March, card is in the name of Priya Raman." A tuned model tags four spans: an order number, an amount, a date, and a person name. Those fall into four broad groups.

• Generic entities: People, organisations, locations, dates, and money, the categories most off-the-shelf models ship with, though city names and customer surnames collide constantly.

• Domain identifiers: Order numbers, tracking codes, policy references, and SKUs, which need custom training because their formats are specific to one company.

• Sensitive identifiers: Card numbers, national identifiers, and medical record numbers, where a missed span has consequences well past a wrong answer.

• Nested and ambiguous spans: "Bank of Austin" holds a location inside an organisation, and flat taggers are forced to pick one.

Named entity recognition vs entity extraction vs intent recognition vs text classification

Buyers conflate these four because a single message passes through all of them in under a second. Entity extraction names the broader pipeline that pulls structured values out of text, including regex and lookup rules that never involve a model. Intent recognition classifies what the customer is trying to accomplish. Text classification assigns a label to the whole message, such as a contact reason, a priority, or a language. Named entity recognition works underneath all three, marking which spans of the message refer to real objects a system can look up.

If the routing decision is what you are missing, intent recognition answers it. If the answer requires a specific order, date, or amount taken from the customer's own words, that is the NER layer, and most support stacks run both models on the same message.

Why named entity recognition matters for customer experience

When entity detection is missing or weak, the visible symptom is a request for information the customer already gave. The transcript contains the order number, and the assistant asks for it again, because nothing in the pipeline turned that string into a field an API could receive. Every automated action, refund, address change, appointment reschedule, needs arguments, and entities are where the arguments come from.

The second failure is quieter. A model that finds an entity but normalises it wrongly hands the workflow a valid-looking wrong value: the customer's earlier order, a date a year off, an amount stripped of its currency. That call executes cleanly and lands in the wrong place.

The tradeoff is recall against precision. Push recall up to catch every identifier and the system masks harmless text and passes false arguments to tools; tighten precision and real identifiers slip through untagged into storage.

How is named entity recognition measured?

Three numbers, computed per entity type. Precision is the share of predicted spans that were correct. Recall is the share of true spans the model actually found. F1 is their harmonic mean, and it is the headline figure because precision and recall trade against each other.

Matching rules decide what the numbers mean. Strict evaluation counts a span only when its boundaries and its type both match the annotation, while partial or type-only matching produces friendlier figures on identical predictions, so a quoted score without its matching rule is not comparable to anything.

On CoNLL-2003, the English news corpus that has anchored this task for two decades, fine-tuned transformer models commonly report F1 in the low nineties, roughly 90 to 94, part of the benchmark saturation the Stanford HAI AI Index tracks across language tasks. Support chat looks nothing like edited newswire, so treat that as a ceiling on clean text and measure on your own annotated tickets.

How AI agents change named entity recognition

Two things changed. Large language models can extract typed entities zero-shot: give the model a JSON schema and it returns filled fields with no labelled corpus behind it, which removes the annotation bottleneck that used to gate every new entity type. The cost is determinism, latency, and price per turn, so production stacks tend to run a hybrid, with validators and pattern rules handling well-formed identifiers (card numbers, IBANs, tracking codes) and the model handling messy human spans.

The second change is consequence. In a scripted chatbot, a mislabelled entity produced a slightly wrong sentence. In an agent that calls tools, the entity is the argument, so a wrong span issues a refund against the wrong order. Extracted entities also drive intent-based search, where a recognised account or product narrows retrieval before the model writes a word. Extraction quality now sits on the same review path as the action policy, because both decide what the agent is permitted to execute.

What to look for in a named entity recognition system

Coverage first: how many entity types ship, and what it takes to add the one your business actually runs on.

Integration surface decides where detection happens. Inline detection can mask an identifier before it is written to a transcript store; batch detection after the fact leaves a window in which the raw text sat on disk.

Governance is the label schema and its owner: who approves a new type, who reviews false negatives, and whether model versions are pinned so an audit can say which model saw which conversation. Buyers handling health data ask how HIPAA compliance is evidenced end to end, and GDPR programmes ask the same of deletion, a recurring theme in compliance for regulated support.

The constraint that bites is latency: inline detection sits in the critical path of a streaming reply, and a tagger adding a few hundred milliseconds per turn forces a choice between masking early and answering fast.

Named entity recognition and PII redaction

Detection is one half of a redaction system and policy is the other. PII redaction decides what happens to a tagged span (mask, hash, tokenise, drop) and at which hop it happens, and it can only act on spans the tagger surfaced, so redaction recall is bounded by detection recall.

The trap is the second copy. Broader entity extraction pipelines write extracted values into ticket fields, analytics tables, and error logs, so an identifier scrubbed from the transcript reappears in three places nobody scrubs.

What does named entity recognition mean in plain terms?

NER stands for named entity recognition, and the short form of the idea is a highlighter with a margin note. It reads a message, highlights the pieces that refer to real things, and writes what each one is beside it: person, date, order, amount.

Without that pass, a transcript is one long string. A system can store "paid £42.60 on 3 March" perfectly and still fail the question "how much did I pay", because nothing in the sentence has been marked as money.

The tradeoff people underestimate is that the highlighter has an opinion. It was trained on somebody's idea of what counts as an entity, and your customers write about things nobody annotated: a plan name that reads like a product, a reference code shaped like a phone number. When the model meets those, it guesses, and it guesses confidently.

Common named entity recognition mistakes

Four patterns account for most of the damage.

Training on newswire and shipping to chat. Benchmark corpora are edited prose with reliable capitalisation, while support messages arrive lowercase, misspelled, and unpunctuated, and casing was one of the strongest signals the model learned.

Reporting one aggregate F1. Rare types contribute few spans to the average, so the headline score stays high while the model quietly misses the identifiers whose failures cost the most. Per-type reporting is the fix.

Treating the model as the whole control. Detection is a probability, and the surrounding system decides what happens when it is wrong: retries, error logs, and analytics exports are where unmasked identifiers usually survive, which is the practical case for AI guardrails in support automation.

Freezing the schema. Identifier formats change when a billing system is replaced, and a tagger nobody re-annotated keeps scoring well on last year's data while missing this year's order numbers.