Echo cancellation

What is echo cancellation?

Echo cancellation is the audio processing that removes a system's own outgoing sound from the microphone signal it captures, so the far end hears and transcribes only what the person on the near end actually said. Without it, every prompt the system plays comes back as input a fraction of a second later.

Delay is what makes echo audible. Sound leaving a speakerphone and returning through the same device's microphone typically arrives tens of milliseconds later, and a network path can push the round trip past a quarter of a second, long enough for a listener to hear a distinct second copy of the voice.

How echo cancellation works

An echo canceller runs as a loop with four stages: reference capture, delay estimation, adaptive filtering, and residual suppression.

The reference is the audio the system is about to play, whether a recorded prompt or a stream produced by speech synthesis. Delay estimation lines that reference up with the microphone stream, because the canceller can only subtract a copy it has aligned in time, and on a telephony path the alignment keeps shifting with jitter and codec buffering.

The adaptive filter then models the echo path, meaning the room, the speaker, and the handset, as a set of coefficients that update continuously through an algorithm such as normalized least mean squares. It predicts what the echo of the reference will look like at the microphone and subtracts that prediction sample by sample.

Whatever survives is usually nonlinear: distortion from a cheap speaker, a clipping amplifier, a device that resamples audio slightly off clock. A residual suppressor attenuates those leftovers and injects comfort noise so the line does not sound dead, and the cleaned stream is what finally reaches automatic speech recognition.

Types of echo cancellation

Echo has two physical origins, and cancellers divide along those origins plus the question of where the processing runs.

• Acoustic echo cancellation (AEC): Removes the echo created when a device's own speaker output is picked up by its microphone, the dominant case on speakerphones, laptops, and headsets worn loosely.

• Line echo cancellation: Handles electrical reflection at the hybrid junction between four-wire and two-wire circuits in traditional phone networks, still present wherever calls touch PSTN interconnects.

• Residual echo suppression: Attenuates the nonlinear remainder an adaptive filter cannot model, usually the last handful of decibels and the stage most likely to clip genuine speech.

• Endpoint or media-server placement: Runs either on the caller's device, which sees the true acoustic path, or in the cloud media pipeline, which sees a codec-compressed copy of it.

Echo cancellation vs echo suppression vs noise suppression vs automatic gain control

Call-quality settings ship these four together, so teams assume one switch governs all of them and tune the wrong one. Echo suppression attenuates the whole outbound channel whenever the far end is talking, which is why older speakerphones sounded half-duplex. Noise suppression attenuates persistent background sound such as fans, traffic, and open-plan chatter, using no reference signal at all. Automatic gain control normalizes level so a quiet caller and a loud one arrive at comparable volume. Echo cancellation is the only one of the four that subtracts a signal it already knows it played, which is what lets both parties speak at once.

If your agent plays audio into the same acoustic space it listens to, cancellation is the mandatory layer and the other three are cleanup. Placing gain control ahead of the canceller makes the echo path nonlinear and much harder for the filter to model.

Why echo cancellation matters for customer experience

When cancellation fails, the agent becomes its own caller. Its prompt returns through the microphone, voice activity detection registers speech, and the system concludes the caller has started talking. It stops mid-sentence, waits, hears silence, and re-prompts. Callers experience an agent that interrupts itself and asks the same question twice.

The transcript damage is quieter and more expensive. Echoed prompt audio gets recognized, so the model receives its own words as user input and answers them, which is how a call that opened with a billing question wanders into an unrelated topic in three turns. Work on reducing customer response time buys nothing when turn structure collapses.

The tradeoff is genuine. Cancellation tuned hard enough to guarantee silence during playback will also attenuate the caller's first syllable, cutting them off at the exact moment they try to interrupt.

How is echo cancellation measured?

Three measurements describe a canceller. Echo return loss enhancement, or ERLE, is the drop in echo energy between the microphone input and the canceller's output, expressed in decibels. Near-end attenuation during double-talk captures how much of the caller's own speech is destroyed while the agent is still playing audio. Convergence time is how long the filter takes to re-adapt when the echo path changes, such as a caller moving the phone from ear to speaker.

Echo cancellers are also compared competitively on shared corpora in public challenge campaigns, where every entrant processes the same recordings and per-system results are published, so independent numbers do exist for anyone willing to read the test conditions attached to them.

Teams commonly aim for roughly 20 to 40 dB of ERLE while leaving near-end speech close to untouched, and the discipline that makes such figures comparable at all is the blind shared-corpus method the NIST Speaker Recognition Evaluation has applied to speech technology since 1996.

How AI agents change echo cancellation

Voice agents lowered the tolerance for residual echo. A person hears a faint copy of their own voice and ignores it. A recognition model treats it as audio and turns it into tokens, which arrive at the language model as though the caller had spoken them, and the agent answers its own last sentence.

The reference signal also became harder to obtain. Prompts used to be files on disk with known contents. Now playback is generated while the sentence is being spoken, and systems using voice cloning to hold a consistent brand voice synthesize it token by token, so the canceller has to tap audio at the moment it leaves the playback buffer.

Latency budgets tightened alongside. Cancellation consumes part of the same few hundred milliseconds the agent has to understand and reply, which is why teams building voice agents for billing and cancellation calls count audio processing inside the response budget rather than beside it.

What to look for in echo cancellation

Judge a canceller on where it sits in the media path before you read its published decibel figure.

Coverage comes first: acoustic and line echo have separate causes, and a stack that solves one leaves the other untouched. Integration surface decides whether the canceller ever sees a clean reference, since an endpoint SDK taps the true playback signal while a cloud media server receives audio a codec has already altered. Governance matters because three parties often ship a canceller on the same call, the device, the media platform, and the carrier, and no one owns the combined result.

Regulated buyers ask how call audio is handled at this layer: whether debug captures of the microphone and reference streams sit under the same HIPAA agreement as the recording itself, and how long GDPR-relevant raw audio survives inside a tuning pipeline. The constraint teams underestimate is diagnosability, because reproducing one customer's echo complaint requires synchronized captures of both streams from that call, and most pipelines retain neither.

Echo cancellation and conversational turn-taking

Turn-taking depends entirely on the agent knowing who is speaking. Barge-in works by detecting caller speech during playback and halting the prompt immediately, so any uncancelled echo of that prompt reads as an interruption and the agent falls silent while nobody is talking.

The stricter case is speech-to-speech architecture, where audio enters the model directly with no transcript stage in between to filter oddities. Whatever the canceller leaves behind becomes part of the model's input and shapes the tone and timing of its next utterance.

What does echo cancellation mean in plain terms?

Think of echo cancellation as software that knows exactly what it just said and can therefore ignore hearing itself say it. A microphone in a room picks up everything, including the speaker sitting beside it. Because the system generated the outgoing audio, it holds a perfect copy and can subtract that copy from whatever comes back.

AEC stands for acoustic echo cancellation, the room-and-device case. The full form of the network variant is line echo cancellation, which deals with electrical reflections inside the phone system itself.

With none of it, a call becomes two voices arguing with a delayed recording of themselves, and neither side can tell who spoke last. Push it too far and the cure does the harm: the caller's opening word gets swallowed, they repeat it louder, and the call sounds worse than the original echo did.

Common echo cancellation mistakes

Skipping delay estimation is the first and most damaging. Teams tune filter length and adaptation rate while assuming the reference and microphone streams are aligned. A reference arriving late is subtracted from the wrong samples, the filter adapts toward noise, and cancellation degrades over the course of the call.

Stacking cancellers is the second. When the device, the media platform, and the carrier each run one, the second canceller adapts to a signal the first already altered, and any gain control downstream lifts the residuals back into audibility between phrases.

Tuning only on single-talk is the third. Bench tests play a prompt into a silent room, where every canceller looks excellent. Callers live in double-talk, speaking over the agent, and that is where near-end attenuation shows up.

Testing on good hardware only is the fourth. Inexpensive speakers clip and distort at volume, making the echo path nonlinear, and a linear adaptive filter cannot model it no matter how well it converged in the lab.