Executive Summary

Key Figures

Source: arXiv:2608.04536

What the benchmark asks for is a bundle of columns rather than a single column, and today's methods stall around halfway through the bundle.

The Health and Retirement Study (HRS) has followed tens of thousands of middle-aged and older Americans for decades, asking about health, income, and family ties. It is one of the most heavily used datasets in gerontology, and every item comes with a codebook entry. Suppose a researcher opens it to study how social isolation shapes healthy aging. Searching the catalog for social isolation returns nothing, because no column carries that name.

What the published papers actually did looks different. Researchers picked household size, physical distance from adult children, frequency of religious attendance, and volunteering, then combined them into a single measure. Taken separately the four items look unrelated to isolation. Household size is a question about living arrangements and religious attendance is a question about faith. Put together, they become a measure that survives peer review.

A useful column may bear no resemblance to the query. It carries meaning, the paper writes, only as a complementary indicator inside a defensible measure. What retrieval systems score is the similarity between names and descriptions, and that similarity has no bearing on the answer to begin with.

Operationalization is an old procedure in the social sciences. It means turning a concept you cannot observe into items you can. The paper pulls that procedure into the middle of data discovery and states it formally. Every research question carries focal roles that have to be measured, and social isolation is one of them. An operationalization pairs each of those roles with a bundle of columns, and the paired columns, used singly or in combination, have to measure that role in a scientifically defensible way.

Why the definition is new becomes visible next to the neighboring task. Research on turning natural language into SQL has a step called schema linking. When a user asks how many flights went from LA to New York yesterday, schema linking picks out the tables and columns that correspond to the question. The user already knows what data they want. What they do not know is the name it goes by in this schema.

In OADD that premise collapses. The researcher trying to measure social isolation is not blocked by an unfamiliar column name. They are blocked because they have not yet decided what to measure it with. Both tasks take a question and return a list of columns, so they look alike from the outside, yet they diverge on everything from what the user already holds to how large the answer is.

Most recent data discovery research rests on the schema linking premise as well. Octopus scans values across scattered tables down to the cell level, and Humboldt defines metadata in an extensible form so the search surface can widen. Performance at finding relevant data inside existing data has improved steadily. What stayed in place is the assumption that relevance has already been settled. OADD is what disturbs that assumption.

Defining a new task is easier than building the answer key. Who decides which bundle of columns correctly measures social isolation? The team's answer was the published literature. An empirical study that cleared peer review is a record of someone walking the path from research question to measurement columns once, for real. Mining those records in reverse yields an answer key nobody has to invent by hand.

3.19,954 works narrowed down to 111 papers

The starting point was a bibliography of 9,954 HRS-related works as of June 2026. Of the 450 papers whose full text the team obtained, 364 were empirical analyses in which the measure was central, and narrowing further to papers that used at least two HRS columns and documented their reasoning in full left 111. The team then extracted each paper's research question while deleting the measurement method it had already settled on. Only the abstract form survived, as in whether everyday experiences of discrimination relate to immune health, with the field names actually used hidden.

Attaching columns proceeded coarse to fine. The team reconstructed how each measure was observed and what it was composed of from the paper body and supplementary material, searched for candidates using the measure name, the observed items, and codebook wording separately, then had an LLM discard obviously irrelevant candidates. In the last step it weighed the recorded measurement against codebook evidence, and left a mapping unconfirmed whenever confidence was low. Checked against 20 separate papers labeled by hand, the automated mining reached a recall of 0.942, and humans kept the automatic label as-is 94.6% of the time.

3.2When one answer runs to sixteen columns

The finished OADD-Bench holds 160 questions drawn from 111 papers, 4,682 question-column labels, and 2,032 distinct HRS identifiers. The ground-truth columns attached to a single question run from 2 at the low end to 136 at the high end, with a median of 21. That distribution says outright that this is not a problem of pinpointing one column.

The measurement components the papers actually used came to 873. One question splits into several components, and each component pulls in several columns of its own. That is also why the mean number of ground-truth columns, 29.3, sits above the median. How many columns you found and whether you filled one component completely turn out to be different questions.

Take the discrimination question above. On the discrimination side, the answer averages six items asking about courtesy received, quality of service, judgments of intelligence, threats, and treatment in medical settings. On the immune side, it combines ten blood assay fields including CD4T, CD8T, and B cell counts. Sixteen fields scattered across unrelated survey domains have to bind into a single measure before the question can be answered at all.

A question on perceptions of aging and everyday independence has the same structure. Eight items about how a person views their own aging are averaged, and five items about the ability to prepare meals, shop, use a telephone, manage medication, and handle money are summed into an impairment scale from 0 to 5. Neither side is a column you could pick out by name.

The team put three families of methods on the same benchmark: direct retrieval by keyword and embedding, schema linking borrowed from text-to-SQL, and the LLM-based OADD agent the paper proposes. The metric is Recall@5R, which allows a method to return up to five times as many candidates as the answer contains and then measures what share of the ground truth sits inside. For a question with 21 correct columns that means 105 guesses are permitted, which is a generous allowance for the retrieval side.

The direct retrieval family failed to reach a fifth of the answer even with the fivefold allowance. Scoring similarity between names and descriptions simply does not reach this kind of ground truth. The schema linking family varied widely. AutoLink, which breaks a question into multiple sub-queries and sweeps broadly, climbed to 0.401, while methods that assume the target schema is narrowly fixed hovered near 0.05. Change the nature of the problem and scores split even within one family.

The paper's own agent led at 0.465, and that number also means it missed more than half of the ground-truth columns. Counted question by question the picture sharpens. Only 50 of 160 questions, 31.3%, came back with every needed column present. AutoLink managed 43 and BGE-base 16. Shift the bar from partial credit to whether the result is usable in an actual analysis and seven questions in ten still fail.

The shape of the failure is distinctive too. The agent got at least one field right in 61.0% of measurement components, but the share where it got every field in the component fell to 37.0%. It finds the direction and does not finish the job. In the earlier example, it works out that blood assay fields are needed and brings back six of the ten. An incomplete combination does not yield a valid scale, so the gap is not a rounding issue.

Scaling the model does help. Going from nano to mini to GPT-5.5 lifted the score from 0.272 to 0.319 to 0.465. What remains is that even the largest model stayed below half. That is the basis on which the paper reads the result as something scale will not settle.

Look again at the conditions under which the agent missed half: that half was not lost for lack of documentation. HRS is among the better documented datasets, and the agent read the codebook and the column descriptions itself. What each column contains was already written down. What was not written down is which concept a given column can serve as an indicator for.

Anyone who has run a data catalog will recognize the distinction. Writing column descriptions and attaching tags is mostly the work of describing columns one at a time. It answers the question of what this column is. The knowledge that measuring social isolation requires looking at household size alongside religious attendance attaches to no single column and to no set of columns either. Until now it has lived inside papers, and inside the heads of people who have worked with the data for years.

For an organization that has understood AI-Ready Data as well-tagged data, this paper pushes the definition up a level. Translated into practical work, two things remain.

• • Is what your data can measure written down anywhere? Above column descriptions and tags, you need a layer that records which concepts these columns serve as indicators for. That is less about creating new columns than about documenting operationalizations of the data you already hold.
• • Where does that record come from? The method the paper chose is the hint. Analyses already performed on this data, meaning internal reports, dashboards, and analysis notebooks, have each walked the path from concept to columns once. Store only the results and drop the path, and the next person, along with the next agent, starts the search over.

Jagadish, a coauthor of the paper, received the 2026 SIGMOD Edgar F. Codd Innovations Award for establishing usability, the problem of making data systems something people can actually use, as a first-class design principle in data management. The question of what else to write into metadata belongs to the same lineage. It moves past whether the data is accurate to whether the party that wants to use it can see what the data makes possible. That party now includes agents.

The paper does not claim to have solved this. Building the ruler and showing where we currently stand is where its contribution ends. The remaining question passes to whoever holds the data. Whether the concept is regional income inequality or customer churn risk, if an organization measures something repeatedly, is the decision about which columns measure it written down next to the data? If it is not, that judgment has to be pulled out of somebody's memory every time.

Academic Papers

• 1.Chen, Houming; Jagadish, H. V. (2026). "From Research Questions to Columns: Operationalization-Aware Data Discovery." arXiv:2608.04536 [cs.DB].
• 2.(2026). "Octopus: A Lightweight Entity-Aware System for Multi-Table Data Discovery and Cell-Level Retrieval." arXiv:2601.02304.
• 3.(2024). "Humboldt: Metadata-Driven Extensible Data Discovery." arXiv:2408.05439.

Background

• 4.H. V. Jagadish, recipient of the 2026 SIGMOD Edgar F. Codd Innovations Award. Wikipedia
• 5.arXiv. "cs.DB Recent Submissions." Latest papers in the databases category.