What is a model card?
A model card is a short structured document that reports what a machine learning model was built to do, what data and metrics it was evaluated on, which groups or conditions it was tested against, and where its behavior is known to degrade. It travels with the model.
Model cards are documentation, so they change nothing about how a model behaves; they tell a reader what to test and what to watch. Most open-weight releases now publish one alongside the weights, and procurement teams increasingly ask hosted API providers for the equivalent before a contract is signed.
How a model card works
A model card is a fixed set of sections, filled in by whoever trained or fine-tuned the model and read by whoever decides whether to deploy it. The opening sections identify the artifact: architecture family, version and release date, training data cutoff, and licence terms. For a large language model they also record the maximum context window, since results measured on short prompts say little about behavior near the top of a long one.
The middle sections carry the evidence. Intended use and out-of-scope use bound the claim. Factors name the conditions results were broken out by: language, dialect, channel, customer segment. Metrics and evaluation data state what was scored and on which set. Those scores usually come from one prompting setup, often few-shot learning with a handful of worked examples, which is why identical weights can post two different numbers in two honest cards.
The closing sections hold limitations, ethical considerations, and recommendations. The loop matters as much as the sections: authored at release, re-read at procurement, revised whenever retraining changes the behavior the card described.
Types of model cards
The same format is used for four quite different artifacts, and mixing them up is how a review approves the wrong thing.
Research card: Published with an open-weight release, describing the base model, its training corpus at a high level, and academic evaluation results, usually with no deployment context.
Hosted API card: Vendor documentation for a model you cannot inspect, typically thin on training data and detailed on usage policy and rate limits.
Fine-tune card: Covers a model adapted on your own data, recording the base model, the adaptation set, and which measured behaviors moved after tuning.
Application card: Describes a deployed assistant end to end, including prompts, retrieval, and handoff rules, because the base model card explains only one component.
Model card vs system card vs datasheet for datasets vs AI impact assessment
Reviewers meet these four artifacts in the same procurement folder and assume three are duplicates of the first. A system card documents a deployed system end to end, including safeguards, retrieval, and human review. A datasheet for datasets documents a corpus: how it was collected, who appears in it, and what consent covered it. An AI impact assessment documents the risk one specific use poses to specific affected people. A model card documents a single component, the trained model itself, and it is the only one of the four written by the people who trained it.
What it holds | Ownership | Who reads it | AI-retrievable | Choose it when | |
|---|---|---|---|---|---|
Model card | One model’s intended use, evaluation, and limits | The team that trained or fine-tuned it | Buyers, risk reviewers, engineers | Yes, when published as structured text | You are judging a model before deployment |
System card | The deployed system: model, retrieval, safeguards, review | Product and safety teams | Regulators, customers, internal reviewers | Sometimes, often long prose | Behavior emerges from several parts together |
Datasheet for datasets | Collection method, composition, consent, known gaps | Data owners | Model trainers, privacy and legal | Yes, though seldom published | You need to know where the data came from |
AI impact assessment | Risk of one use to specific affected people | Risk, legal, and compliance functions | Regulators, executives, auditors | Rarely, usually a controlled document | A named deployment needs formal sign-off |
If you are choosing between models, read the model card. If you are answering for what a live support agent does to customers, the model card is one input among four, and the system card plus the impact assessment carry the rest.
Why model cards matter for customer experience
Support teams inherit a model’s blind spots as customer-facing incidents. When no card exists, the first time anyone learns the model was evaluated only on fluent written English is when a customer writes in mixed Portuguese and slang and gets a confident wrong answer. The card is where that limit is supposed to be stated in advance, in language a support lead can act on by routing those contacts to a person.
Cards also give the drift conversation a baseline. Without a recorded evaluation set and date, model drift is invisible until quality complaints accumulate, because there is nothing to re-run and compare against.
The tradeoff is real: writing and maintaining cards costs review cycles, and a team that demands one for every minor fine-tune will end up with a shelf of documents nobody revises. Fewer cards, kept current, beat complete coverage that has gone stale.
How is a model card measured?
No standards body sets a target figure a support team is expected to hit for model card completeness or quality. Cards routinely quote scores from public evaluation suites such as MMLU or HELM, and those suites score tasks drawn from their own taxonomies, which is not the taxonomy of contact reasons in your queue, so the numbers do not transfer to your resolution rate.
What you can measure is the document. Section completeness is checked against the section structure set out by Mitchell et al., which defines what a card should report: model details, intended use, factors, metrics, evaluation data, training data, quantitative analyses, ethical considerations, and caveats. Staleness is the gap between the card’s revision date and the model’s last retrain. Claim coverage is the share of card claims you have reproduced yourself on your own tickets, and it is usually the honest number.
How AI agents change model cards
An AI support agent is a composition of four parts: a model, a retrieval layer, a set of tools it can call, and a policy layer deciding what it may do unattended. The card describes one of those parts. Tool access standardized through the Model Context Protocol means identical weights with an identical card can issue refunds in one deployment and answer read-only questions in another.
That composition changes what a reviewer should conclude from a card. The card sets a ceiling on what the model is capable of and stays silent on what your configuration actually permits, so the same document supports very different risk profiles. The practical consequence is that a card is read together with the AI guardrails and the escalation rules around it, and reviewing any one of the three alone tells you little.
What to look for in a model card
Judge a card on the axes that decide whether you can operate the model safely.
Coverage first: are out-of-scope uses named explicitly, and are results broken out by language, channel, and customer segment, or reported only as one headline average. Integration surface next: does the card document function-calling behavior, latency ranges, and rate limits, since those decide whether the model fits a live queue.
Governance and ownership come third: a named owner, a version identifier, and a revision date, so you can tell which model the document describes. Security certifications belong beside it, since SOC 2 Type II, ISO 27001, ISO 42001, HIPAA, and GDPR obligations sit with the hosting arrangement. The operational constraint most teams miss is data retention, which an integration security review tends to surface before legal does.
Model cards and AI governance
A model card describes a model in isolation, and most support failures happen after the model is wired to a knowledge source. AI grounding determines whether an answer traces back to a retrieved passage, and a card reporting strong reasoning scores predicts very little about how often AI hallucination reaches a customer once retrieval is weak or the content is stale. Governance reviews that read the card and stop there have approved a component while inheriting a system.
What does a model card mean in plain terms?
Think of a model card as the rating plate on a ladder: it tells you the load it was tested at, the surface it was tested on, and what the maker will not stand behind. The ladder still works outside those conditions, sometimes fine, and nobody has checked.
Without one, a team picks a model from a leaderboard, ships it to customers, and discovers its weak spots one complaint at a time, with no record of what was ever tested. With one, the weak spots are on paper before launch and can be routed around.
The tradeoff is that a card is a claim by the people who made the thing. It narrows what you have to guess at; it does not remove the work of testing the model on your own traffic.
Common model card mistakes
Treating publication as approval is the first pattern. A card is a description authored by an interested party, and reading it does not constitute an evaluation. The mechanism of failure is substitution: the review consumes the document and skips the test run on real tickets.
Copying a base model’s card onto a fine-tune is the second. Adaptation changes exactly the behaviors the original numbers described, so the inherited card now certifies a model that no longer exists.
Omitting out-of-scope use is the third. Teams cut the section because it reads as an admission of weakness, and the omission moves the discovery of every limit into production, where a customer finds it first.
Never versioning the card is the fourth. Providers update weights behind a stable endpoint name, and a card without a revision date quietly stops describing what is being served.
Frequently Asked Questions
What goes in a model card?
A model card contains model details (version, architecture, training cutoff), intended and out-of-scope uses, the factors results were broken out by, the metrics and evaluation datasets used, quantitative results, ethical considerations, and known limitations with recommendations. The evaluation and limitations sections carry the weight; the rest is identification and context for a reviewer.
What is the difference between a model card and a system card?
A model card covers one trained model: its evaluation, its factors, and its limits. A system card covers the deployed system built around that model, including retrieval, tools, safeguards, and human review steps. Support behavior comes from the whole system, so the model card is one input into a review and never the complete picture.
Model card vs datasheet for datasets: which one do I need?
A model card and a datasheet answer different questions. The datasheet explains the corpus: collection method, composition, consent, and known gaps. The model card explains the trained artifact and how it scored. If your concern is privacy provenance or licensing, read the datasheet. If your concern is deployment behavior, read the card.
Who writes a model card?
Model cards are written by the team that trained or fine-tuned the model, usually machine learning engineers with review from legal, privacy, or risk functions. That authorship is the document’s main limitation: it is a self-report. Buyers treat it as a structured claim to verify against their own data before it informs a deployment decision.
Do fine-tuned models need their own model card?
Fine-tuned models need their own card whenever adaptation changes measured behavior, which is nearly always. The new card should name the base model, describe the adaptation dataset, and report which evaluations were re-run afterward. Inheriting the base model’s card documents a model you are no longer running, which defeats the purpose.
Are model cards legally required?
Model cards are not mandated by that name in most jurisdictions. Emerging AI regulation and enterprise procurement policies do require technical documentation covering intended purpose, evaluation, and limitations, and a well-written card supplies much of it. Regulated buyers in health, finance, and the public sector increasingly request one before any pilot begins.