Executive Summary

Key Numbers

The first two cards are the accuracies that split along access level. The third is the score when the task is naming the dataset itself rather than the domain, and the fourth is what the whole study cost.

Source: arXiv 2611.07610 (November 5, 2026) · one base model throughout, Qwen3.5-0.8B

LoRA is the most widely used way to bend a large language model toward a particular job. The original weights stay frozen, and a pair of small, flat matrices is trained alongside them. What comes out is one file far lighter than the model itself, and that file is called an adapter. Because it is small and can be snapped on and off, teams handing an in-house model to a client or uploading it to a public hub usually pass around the adapter rather than the base.

Underneath that practice sits an assumption nobody had properly tested. An adapter is the result of training, not the training data, so handing over the file is not supposed to hand over what the model learned from. Whether a model was tuned on clinical consultation records or on an internal codebase, the receiving side is thought to get the capability and nothing about the ingredients.

"Embedding-Space Fingerprinting of LoRA Fine-Tunes," which went up on arXiv on November 5, puts that assumption on the stand. The question is a plain one. Given only what the model produces, and no sight of the training data at all, can a single simple classifier recover the domain the model was tuned on?

Other work has aimed at nearby targets. Membership inference, which asks whether a particular example was used in training, is often barely above chance on large language models. Dataset inference asks whether a whole dataset went in. Both require a candidate in hand before the question can even be posed. This paper's question is coarser and needs no candidate. It can be asked by anyone holding nothing but the model, which widens the set of people in a position to ask.

The raw material is 18 public datasets grouped into five domains. Medical holds four — PubMedQA, MedMCQA, ChatDoctor-HealthCareMagic and MedQA-USMLE. Code holds four: Python-code-instructions-18k, Evol-Instruct-Code-80k, SQL-create-context and Code-instructions-122k. Mathematics holds three: MetaMathQA, MathInstruct and Orca-Math-200k. The multilingual group, meaning non-English text, holds IITB English–Hindi, OPUS-100 German–English and the non-English slice of Aya. Creative holds WritingPrompts, TinyStories, Opus-WritingPrompts and Instruction-Poems.

One base model, Qwen3.5-0.8B, was used for everything, loaded in 4-bit quantization. Each dataset was run twice under different random seeds, producing 36 fine-tunes. Every other setting was held fixed: rank 16 with alpha 32, a budget of 1.5 million training tokens, a learning rate of 2×10⁻⁴ with linear decay, an effective batch size of 16, one epoch. With only the dataset free to vary, whatever separates one adapter from another has to come from what it learned on.

There are three ways of looking, and each one has less access to the model than the last.

The three interrogate different things. White-box access takes the adapter file itself and reduces each module to a handful of numbers describing how far its values moved. Gray-box access cannot open the file but can run the model, so it feeds in 200 domain-neutral prompts and records the mean and spread of what rises through each layer. Black-box access sees nothing inside the model and keeps only the answers that come back to those same prompts. Even so, it ends up with the largest feature count of the three.

The probe prompts were chosen to lean toward no domain at all, along the lines of "Describe your morning routine." Source: arXiv 2611.07610

The classifier is one standardized logistic regression. No deep network, no separately trained model. Evaluation pulls an entire dataset out, trains on the rest and asks the classifier to label the one held back, and the author states that this protocol was frozen before any result was seen. The design stops a second seed of the same dataset from leaking the answer.

The hypothesis registered before the experiment was that more access means more recovery. White-box would hold a commanding lead, gray-box would follow, and black-box would trail. The numbers came back in the opposite order.

The task picks one of five domains, so chance is 20%. The intervals shown are the dataset-clustered bootstrap. Because the two seeds of a dataset share their training data their predictions are not independent, so resampling whole datasets gives the more conservative interval. Resampling individual predictions instead puts white-box at 25.0–58.3%, above 20%, but the author works from the conservative figure. Source: arXiv 2611.07610

Set the three values beside the chance line and the gap is easy to see. The white-box interval runs down to 19.4% and crosses to the wrong side of that line, while the other two stop at 41.7%, more than double the 20%.

Two explanations are offered for the reversal. Either the domain imprints more on behavior than on the size of the weight change, or, more likely, the hand-designed weight features simply left usable structure on the table. Norms, singular values, layer curves — all 48 features keep magnitude and throw away direction. Which way the weights turned may be written into the adapter, but none of these numbers carries it. On that reading, 41.7% says less about the weights being silent than about the 48 chosen numbers not listening. The author adds that learnable features or a richer representation might land somewhere else.

A follow-up experiment on rank backs the second explanation. One dataset per domain was fine-tuned again at ranks 8 and 64, giving ten extra adapters, and a classifier trained only on rank-16 runs was asked for their domains. White-box got three of the ten right, and all five of the rank-64 adapters came back labelled medical. Only one of those five, PubMedQA, actually is. Norms and spectra grow as rank grows, so this classifier was reading rank rather than domain. With one dataset per domain behind the count, the author says these numbers are indicative only.

The black-box side has one more layer to it. Some of the 455 features are drawn from per-token probabilities, and dropping every one of them to leave only the visible characters still returns 63.9%. Just one of the 36 predictions changes. Token probabilities are something many services now decline to hand out. That the figure holds without them means the experiment reproduces with nothing more than an ordinary API, 200 prompts and a record of what comes back.

Switch the task from domain to the dataset itself and the numbers jump again. Picking one of 18, white-box and gray-box got every one right, and black-box reached 94.4%. A guess gets 5.6%. This is not the figure to be impressed by, though. The datasets being scored are the same ones used in training, so it is closer to recognizing something already seen. What the paper rests its weight on is naming the domain of a dataset it has never met.

A finding that generated text betrays the training domain runs against an earlier report. Delta Activations, published in September 2025, found that sentence embeddings of model output cluster poorly by domain. Same material, opposite conclusion.

Whether the material was to blame is the first thing the author checked. Instead of all 455 features, the run was repeated with nothing but the representation that earlier study had used: the 384-dimensional MiniLM sentence-embedding centroid. That alone reached 61.1% on domain. One study held the same output representation and reported nothing visible; this one reports a signal. On weights the two actually agree. Delta Activations found that flattened adapter weights cluster poorly as well, and the 41.7% here points the same way. That study did work with models in the 7–9 billion parameter range, and its probe set was different.

The explanation offered is about retrieval rather than material. Domain information is present in the output embeddings but is not the dominant direction of variation, so a classifier trained against labels recovers it while unsupervised clustering does not. Clustering seizes first on whatever axis splits the data most widely. If that axis turns out to be answer length or register, domain stays buried behind it. A classifier, reading the labels, goes looking for the axis that tracks domain.

The mistakes also fall in different places. With gray-box activations, multilingual and code separated best while medical and creative ran into each other: multilingual got five of six right and code six of eight, whereas four of eight medical runs were called creative and three of eight creative runs were called medical. Both domains are English prose, and the author writes that this moves the internal statistics in similar ways. Move to black-box outputs and the medical–creative confusion disappears, with code and mathematics becoming the largest remaining pair. Medical climbs to six of eight and creative to five of eight, and four of the thirteen remaining errors fall between code and mathematics, two domains whose answers are both dominated by symbols laid out step by step.

The paper sets out its limits in six items, narrowing in advance how far a reader is entitled to carry the result.

• • One base model — the finding rests on a single 0.8B model and on LoRA alone. Transfer to other architectures, to larger scales or to full fine-tuning was not tested. A hard limit of US$30 in cloud GPU time drew the boundary of the study.
• • Thirty-six runs make for wide intervals — under the clustered bootstrap the white-box figure in particular is not distinguishable from chance, so the ordering of the three methods should be read as suggestive, the author states. Adding a third run per dataset narrows the intervals without changing that conclusion.
• • Two datasets dropped out — of the 20 intended, camel-ai/math is stored as tens of thousands of separate files and could not be loaded within the budget, and a Japanese dataset was deprioritized for the same reason. Mathematics and multilingual therefore ran with three datasets each.
• • One of the 36 was trained differently — PubMedQA seed 0 ran first as a pipeline-validation job, on 2.0 million tokens rather than 1.5 million. Replacing it with a 1.5-million-token rerun leaves the three headline figures at 41.7, 61.1 and 63.9.
• • The domains are deliberately far apart — medical, code, mathematics, multilingual and creative are a set with little resemblance between members. Telling two medical sub-specialties apart would likely be harder, as the author says directly.
• • A pilot result was thrown out by its own author — holding out one seed instead of one dataset produced 100% accuracy, but a label-shuffle control showed this was re-identifying the same dataset rather than learning the domain. The headline numbers therefore rest only on holding out whole datasets.

Whether drawing fresh training text erases the fingerprint was tested too. The two seeds behind the headline 36 saw the same bundle of tokens, so a third model per dataset was trained on 1.5 million tokens drawn independently from scratch. For 15 of the 18, the budget takes no more than 26% of the dataset, so the new bundle barely overlaps the old one; the remaining three overlap heavily, Aya by 93%. Across all 54 runs, naming the dataset still returns 1.000 for white-box and 0.981 for the other two, and domain accuracy does not move either. What the fingerprint holds onto is the dataset's distribution rather than the specific examples drawn from it.

What survives is clear enough. The level of access with the least visibility scored highest, the figure held when token probabilities were stripped out, and black-box predictions stayed put across LoRA ranks from 8 to 64. And the paper proposes no defense. Not a line on adding noise to the weights or smoothing the outputs. The demonstration stops at the fact that the signal is there, and erasing it is left as homework.

64% is not a number anyone should call exposure. One in three is wrong, and the interval is not tight. It still changes something, though. Until now the data confidentiality of adapter distribution has been managed on belief rather than on measurement. The sentence "we never gave them the data, so nothing leaks" had nothing to argue with. Now there is 20% against 64% to put beside it, and confidentiality moves from a question of whether it holds to a question of how far.

The result cuts both ways. For a team that builds a specialized model on internal data and ships only the adapter, this is a risk: a competitor can throw a few hundred neutral prompts at the API, collect the answers and form a view of where the effort went. For regulators asking companies to disclose what they trained on, it is a tool. The author writes that the finding bears directly on provenance auditing of distributed adapters and on detecting undisclosed training data. The same signal is a leak or a piece of evidence depending on whose hands it is in.

Three things are worth checking for any organization that passes adapters around.

• • What the contract actually calls confidential — a clause saying the training data will not be provided is not a guarantee that the training data stays hidden. The wording deserves a second reading on the premise that handing over the model file hands over something about what it learned from.
• • Where the controls are placed — hiding the weights closes the weakest channel in this experiment. The method that scored highest took no privileges at all and worked from the answers, and that channel stays open for as long as the service does.
• • Whether anyone asks where an incoming adapter came from — the same technique serves supply-chain review. There is now a way to estimate what kind of data an externally sourced adapter was built on, and where licensing or personal data is involved, that question belongs to the receiving side too.

Provenance is what Pebblous records while getting data into a state an AI can use directly. Keeping a record of which data came from where and went into which model gives a team something to answer with when someone later works backwards from the output. Without that record, an estimate can harden into fact with nothing to set against it. If models say a little about their ingredients, the side holding the ledger of those ingredients is the better placed.

Thanks for reading this far. The figures here were checked directly against the body of the paper posted to arXiv, and where the author held back we have held back too. We would be glad to hear where you draw the line when a model tuned on internal data goes out the door, and whether that line is drawn on the weights or on the output.

Pebblous Data Communication Team
November 9, 2026