Match confidence scores: when to auto-accept

An identity match score of 87 does not mean "87% likely to be the same person." On most systems it means "more independent details agreed than on an 80, and fewer than on a 95." That difference matters, because teams pick an auto-accept threshold by looking at the score histogram, finding a natural gap, and drawing a line through it. The line should come from somewhere else: what it costs you when a match above the line is wrong. This post sets out a way to choose thresholds from that cost, a worked example with a 0-100 match confidence score, and how to re-tune once reviewers start producing outcomes.

Key takeaways

• A match confidence score ranks evidence; it is not a calibrated probability until you measure it against reviewed outcomes.
• Set thresholds per action: merging two customer histories needs a higher bar than refreshing a mailing address.
• Use three bands (auto-accept, manual review queue, drop), not one cut-off.
• Read the match tier next to the score; a 90 from an exact match and a 90 from a relaxed date of birth are different evidence.

What a match confidence score measures

A typical identity match score combines how many submitted details agree with a candidate record (name, date of birth, address, phone, email) and how strongly each agrees. Exact date of birth counts for more than a matching state; a full name counts for more than a first initial. The output is an ordering: higher scores have more and stronger agreement.

What it does not measure is how often a given score is right in your data. That depends on your inputs. A file with full dates of birth and current addresses will produce higher scores, and higher precision at each score, than a file of names and states. CoverFi describes its own 0-100 score this way in the glossary: a signal for prioritizing human verification, not a verdict.

Choose the entity matching threshold from the cost of being wrong

Start from the action the match feeds, and the cost of each kind of error:

The same score can be safe to auto-accept for the first row and unsafe for the third. One global entity matching threshold forces a compromise that is wrong for both. Set one per action.

A matched record also tells you only who someone is likely to be. Keep identity matching out of approval or eligibility logic entirely, and use matched records only for the purpose your data license allows.

Worked example: three bands and how to re-tune them

Say a team refreshes contact phones on account records, and runs 1,000 searches a week through a 0-100 score. It starts with three bands:

• 90 and above: auto-accept.
• 60 to 89: manual review queue.
• Below 60: drop; no phone is added.

For the first four weeks, reviewers also check a random sample from the auto-accept band, and every queued item gets an outcome: same person, different person, or can't tell. The hypothetical results:

Now the thresholds can be argued with numbers. If the team's tolerance for a wrong phone on an auto-accepted record is 2%, the 90 line holds. The 75-89 band is right 88% of the time, which is too low to automate for phone refreshes but fine to fast-track in review. The 60-74 band is close to a coin flip, so reviewer time is better spent elsewhere, and raising the drop line to 75 is defensible. Keep sampling the auto-accept band permanently, because input quality drifts and so does precision.

Why the match tier matters as much as the number

Two matches can share a score and differ in what kind of evidence produced them. A 90 that came from an exact name and full date of birth is a different decision from a 90 that came from a nickname plus a relaxed date of birth with the day and month swapped. If your provider labels the tier, use it as a second key for your bands:

accept  if tier == "exact"    and score >= 90
review  if tier in ("nickname", "former name") and score >= 75
review  if tier == "relaxed date of birth"     and score >= 85
drop    otherwise

CoverFi labels every result with the tier that produced it (exact, former name, nickname, transposed name order, compound surname, relaxed date of birth, nearby metro area) alongside the 0-100 score, so rules like the ones above can be written directly. The identity resolution guide explains why pooling candidates by tier and then scoring them keeps both signals visible.

Log every accepted match with its score, its tier, the threshold version in force, and the reviewer if there was one. When a wrong merge surfaces months later, that record is how you find every other match the same rule accepted.

Frequently asked questions

Is a match confidence score a probability?

Not by default. It ranks how much evidence agrees. It becomes a probability only after you measure precision at each score band against reviewed outcomes in your own data.

What is a good auto-accept threshold?

There is no universal number. Choose it from the cost of a wrong match for the specific action, then check it against a reviewed sample. Many teams start high and lower it only when the data supports it.

How big should the manual review queue be?

Size it to your reviewers' capacity. If the queue grows faster than it clears, use reviewer outcomes to narrow the band, rather than auto-accepting the overflow.