Behavioral Fingerprinting Detects Model Swapping at API Relays

Core Highlights

Tomáš Brukner, a researcher at the Prague University of Economics, discovered a lightweight yet sharp detection method: by repeatedly asking a model to output random numbers between 1 and 100, one obtains a unique "behavioral fingerprint." This technique provides a low-cost, easily deployable ruler for verifying whether an API endpoint has been "model-swapped." The elegance of the approach is that it needs no special access, no proprietary telemetry and no cooperation from the service provider—only the ability to send ordinary inference requests, which any paying customer already possesses.

What makes the idea powerful is its asymmetry: the checker needs nothing the relay does not already expose, while the relay cannot easily hide its true identity without degrading the very capability the customer is paying for. In a market where middlemen sit between model owners and end users, that asymmetry is exactly what turns a curious observation into a practical auditing tool.

What Happened

Brukner queried 165 models 30 times each with random-number requests and tallied their preferred digit distributions. The results were highly distinctive: GPT-4o favored 42 and 37, Claude Sonnet 5 obsessively output 47, and Qwen3-Max answered 42 all 30 times. In other words, what is called "random" actually carries stubborn preferences across different models. These preferences stem from subtle differences in training data, sampling strategy and post-processing, and together they form an identity feature that is hard to forge.

A relay that swaps in a cheaper model would thus betray itself through a changed fingerprint, because the substitute's quirks differ from the advertised model's. The study's breadth—165 models, 30 samples each—gives confidence that the effect is systematic rather than a coincidence of a few odd cases, and that a compact probe set can reliably tell models apart in the wild.

Technical Details

The core of the method is "measuring identity through randomness." Because pseudo-random sampling is affected by seeds, temperature and distribution clipping, each model's output distribution has a stable bias that persists across sessions. Experiments show that model identity can be recognized with only about 120 requests at an error rate of roughly 10.6%, which is surprisingly small given how little information each probe carries.

Compared with approaches that require comparing weights or capturing internal logs, behavioral fingerprinting is completed entirely at the interface layer—it needs no cooperation from the service provider and touches nothing inside the model, giving it strong practicality and stealth. A caller can run the check covertly without the relay ever noticing that it is being tested, which matters precisely because a dishonest provider would otherwise disable any transparent verification it was asked to support.

Comparison with Competitors

Traditional anti-substitution schemes mostly rely on server-side signatures, response headers or proprietary protocols, where the initiative lies with the service. Behavioral fingerprinting returns the power of verification to the caller, who can judge by themselves as long as they can send requests. Unlike watermarks, which can be deliberately erased, "digit preference" is a by-product of model inference that is hard to disguise without changing capability.

Its limitation, of course, is the need for repeated sampling, and its discriminative power drops for strongly aligned models that are deliberately flattened toward uniformity, since such models have been explicitly tuned to remove exactly these telltale biases that the method relies upon. Even so, for the vast majority of production models that retain natural preferences, the technique remains a cheap and robust first line of defense against silent substitution.

Industry Impact and Use Cases

Simply put, API relays swapping models is an open secret in the industry: users think they are calling a flagship model but may actually be downgraded to a cheap one so the middleman can pocket the difference. Behavioral fingerprinting gives enterprises and individuals a "verification" ability—when connecting to a third-party gateway, model-aggregation platform or low-cost proxy, firing a set of random-number probes first reveals whether the other side delivered what was promised.

For compliance auditing, SLA fulfillment checks and model supply-chain security, this is a simple yet efficient infrastructure-level tool, and it also pushes service providers toward greater transparency about model origins. In a market where trust is hard to verify, a ten-minute probe can protect significant spending and expose vendors who quietly substitute inferior models, aligning economic incentives with honest labeling.

For procurement teams, the practical takeaway is to treat model identity as a monitored metric rather than a trusted label. Just as software supply chains are scanned for vulnerable dependencies, AI supply chains can now be scanned for substituted models, and a quick fingerprint check deserves a place in every vendor acceptance test before a contract is signed. Trust, once verified automatically, stops being a bottleneck and becomes a routine check that procurement can run without specialist help. It is a small step that could reshape how AI services are procured at scale.