Axlow is built for payer-policy questions where a confident wrong answer is worse than a safe decline. We test the production Ask Axlow endpoint against fixed benchmarks, scoped edge cases, and deliberate safety traps — then preserve the results.
Last benchmark update September 23, 2026
Environment Production · axlow.com
We score substantive coverage separately from semantic and scope safety. A high answer rate is not considered a win if the answer uses the wrong payer, state, plan, CDT code, or rule intent.
Second frozen unseen benchmark
100 / 100
All 70 positive controls returned substantive typed evidence. All 30 deliberate scope-safety negatives returned no citations. The repaired cases were semantically checked against their source evidence.
Same unchanged questions used before and after repairs · now frozen
First frozen unseen holdout
50 / 50
The first unseen holdout tested new payer, CDT, intent, state and program combinations after the original benchmark work.
Frozen after completion · no further tuning against these questions
Language robustness benchmark
100 / 100
The same underlying policy facts were asked with materially different natural-language phrasing to test intent recognition rather than benchmark-wording memorization.
0 intent mismatches · 0 scoped-state mismatches · 0 safety leaks
Fixed generic benchmark
72 / 100
The remaining 28 were appropriate safe declines. Semantic/safety accuracy was 100% on the audited set, with zero misleading answers.
Includes 25 intentionally under-scoped DentaQuest questions
DentaQuest scoped benchmark
17 / 18
When state, program, or plan context is supplied, DentaQuest evidence becomes actionable without leaking state-specific rules into payer-wide answers.
One intentionally under-scoped question safely declined
The second unseen benchmark was intentionally harder and used CDT codes and scope combinations outside the earlier tests. We saved the first production result, repaired the underlying normalization and scope issues, then reran the same unchanged questions.
01 · UNSEEN BASELINE
46 / 70
substantive positive answers
24 / 30 safety negatives passed
02 · ROOT-CAUSE REPAIRS
Evidence + scope
Aetna documentation promotion, UHC Arizona Medicaid typed rules, New Mexico scope normalization, and narrow parser fixes.
No safety guard was relaxed.
03 · SAME 100 QUESTIONS
70 / 70
substantive positive answers
30 / 30 safety negatives passed
01
The evidence must match the requested payer and CDT code.
02
Frequency evidence cannot answer documentation. Bundling evidence cannot answer authorization.
03
State, Medicaid, Medicare, and named-program rules stay inside their documented scope.
04
The cited passage must actually support the answer. A code appearing in a document is not enough.
05
We check the generated answer against the evidence so cautious policy language is not turned into a stronger claim.
What these numbers do — and do not — mean
They are results from defined benchmark sets, not a claim that Axlow is 100% accurate on every possible dental-policy question. Coverage varies by payer, state, program, plan, source availability, and effective date. Ask Axlow is policy intelligence, not a member-specific benefit, eligibility, authorization, or payment determination.
We do not keep editing questions until the score looks good. Completed holdouts are frozen and retained as regression checks. Future accuracy work moves to new CDT codes, states, programs, policy intents, and adversarial scope cases.
Dental payer policy intelligence with traceable evidence.