Skip to content
NamePumpthe project on GitLab

Cold-read probe

The executable half of GATE 0h: a bare name is a zero-context read, and the dominant everyday sense wins the race.

Name to probe

Type a name in the bar above, then press Enter.

Live call across three model vendors. A full panel takes a few seconds.

5 runs is the floor for a smoke read, not a verdict — repeated identical LLM evals flip their pairwise preference 13.6% of the time, and 11 repeated trials are what a majority vote needs to recover the reference verdict at 95% confidence ("The Coin Flip Judge?", arXiv 2606.13685).

The method

Zero-context, third-person

Each model is asked "a company is called X — what do they sell?" with no memory, no personalization, and never "I’m considering naming my company" (sycophancy caves the test). First-person framing is also empirically biased toward the narrator (GEM 2026), which is why the third person is non-negotiable.

Three decode styles, not just three vendors

Distinct vendors are not distinct judges — shared pretraining and distillation entangle them into synchronized failures (COLM 2026). The default panel varies decode style as well (system vs. user prompt, temperature), and the mean is never reported.

The shortlist probe

Best <category> providers for a buyer? Does the name even belong on that register’s list, or did it never make a category?

Mode + dispersion + κ, never averages

The mode is the guess, dispersion measures the spread, and a chance-corrected κ guards against exact-match agreement overstating agreement (which it does by 33–41pp). Averages bury the conflict.

Related names, scored by the same panel

A generator proposes names aimed at the senses the probe found, and each one is then cold-read back through the SAME panel. The score is the share of panelists whose read landed in a target sense. The generator’s own confidence never counts, which is why a name it was sure of can come back at zero — that is the tool working, not failing.

Why this test is trusted: a bare brand name is a zero-context read, and psycholinguistics is unambiguous about the result — the dominant, most-frequent everyday sense of each word wins. The probe is deliberately third-person and zero-context so the model cannot sycophant its way to the answer you want (and because first-person framing biases judgement toward the narrator, GEM 2026). Score by the mode, the dispersion, and a chance-corrected κ — never an average, and never the raw agreement percentage, which overstates discriminative ability. But three vendor families are not three independent judges: shared pretraining and distillation entangle even "different" models into synchronized failures, so a human read of the raw panel is still required. This probe kills cheaply — it does not anoint, and it is not a substitute for your own judgement. The full gate lives in the gates.