SpeakerCard-1M: An Evidence-Grounded Speaker Card Corpus for In-the-Wild Speaker Verification

The benchmark is designed to separate retrieval, verification, and attribute grounding. Each protocol uses speaker-disjoint splits; Vox-only retrieval uses a fixed 1K-speaker English gallery, while the bilingual extension additionally reports a 144-speaker clean CN-Celeb gallery.

These are real card pairs sampled from the released test sets. In each panel the model sees one audio clip plus two text candidates and chooses A or B under a balanced forced-choice prompt. CF tests minimal trait flips with LLM-rewritten distractors; Hard uses real cards from different speakers that share coarse traits.

Why this matters: CF tests whether a model rejects an explicit single-trait contradiction. Hard tests whether it can still pick the correct speaker when the two text candidates are essentially synonymous — only the audio identity differs. CF is largely solvable from text-level reasoning (best LALM: 87.79%); Hard collapses on text alone and exposes the true cross-modal grounding gap. The dual encoder achieves 72.53% Hard vs the strongest LALM at 55.28%.

We release two trained checkpoints from the same audio–text dual encoder. The balanced recipe maximizes attribute discrimination (AC-Verify CF / Hard) while keeping retrieval competitive; the retrieval-specialized recipe continues from a retrieval-push checkpoint and pushes T2S / S2T recall further at a controlled cost to Hard. Numbers below are on the Vox-only 1K-speaker English gallery.

Cascade replaces the audio tower with the same probe → LLM verbalizer used in corpus construction, matched in BGE-M3 text–text space. The substantially larger gap on S2T (9.30 vs 25.50 at R@10) than on T2S (15.60 vs 24.80) localizes the audio tower's contribution to utterance-to-speaker aggregation rather than text retrieval per se. AC-Verify scores reported here use the style-symmetric LLM-generated counterfactual (llm_cf) protocol. SV EER columns from the paper (Vox1-O/E/H) are omitted here and reported in the manuscript.

Each model is shown one audio clip and two cards (the matched card and a counterfactual that differs in a single trait, or a mined hard-negative card), and chooses A or B with balanced A/B order. Counterfactuals are LLM-rewrites in the same surface style as the original card (the style-symmetric llm_cf protocol), so the distractor cannot be detected by style alone. CF is the per-trait average across gender, accent, age, and pitch. Higher is better.

Six of eight LALMs handle gender well (94–98%); accent shows a clear split with Qwen3-Omni and both Gemini variants above 92% versus 46–82% for the rest; age sees only Qwen3-Omni (80.93%) and Gemini 3.5 Flash (84.40%) above 80%. Pitch is the universal weak point: all eight LALMs score 49–77%, with one model below chance. Our dual encoder reaches 88.66% on pitch without conceding the coarse traits and is the only system above 90% on every per-trait column.

Real captions for cnceleb1/id00000, the same speaker referenced across the corpus. Each record exposes four text views so retrieval, verification, and interpretability experiments can choose how much state context to include. Trait phrases (gender, age, accent, pitch) are highlighted in orange; state phrases (emotion, channel, environment, rate) in teal; numeric evidence in gold.

Vox-only and Vox+CN use different training pools, validation sets, and test galleries, so the bilingual rows are reported as an extension snapshot rather than a strict same-gallery comparison.