Dataset Search Returned More Usable Results but Answered Fewer Questions

A page can be exactly about the data you need and still be the wrong destination. It might describe a survey without providing the data, link to a portal that needs another search, or offer an interactive chart with no machine-readable download.

For an agent asked to find a usable dataset, topical relevance is only the first test. It also needs to reach an appropriate record and an accessible data resource.

Do Agents Need Semantic Metadata? compares two agents using the same Gemini 2.5 Pro model. One searched through Google Search; the other used Google Dataset Search, a specialized ecosystem built around structured dataset information. The benchmark contained 58 queries.

The specialized route returned a higher proportion of results meeting the study's usability criteria, but answered fewer queries. The experiment compares those end-to-end search routes. It does not isolate the effect of adding schema markup to an otherwise unchanged page.

Define what a successful result lets you do

The evaluation assessed topical relevance, machine-readable data accessibility and page type. A result had to meet all three to count under the authors' FAIR-aligned criterion. That is a defined evaluation rule, not a comprehensive certification of every FAIR principle.

The reported precision was 46.4% for the Dataset Search agent and 28.0% for the general-search agent. In other words, the proportion of returned results passing the combined test was higher through the specialized route.

The coverage comparison moved the other way. The specialized agent answered 40 of 58 queries, while the baseline answered 56. A higher success rate among returned results does not tell you how many questions receive no result.

This is a familiar search trade-off. Restricting a collection can reduce unsuitable destinations while leaving less material available for other needs. Whether that helps depends on the job. A workflow requiring a downloadable dataset may value an honest empty result differently from an exploratory search where an indirect lead is useful.

The destination improved more than the topic match

The two agents had similar reported rates of highly relevant results, around 60%. The larger differences concerned accessibility and the kind of page returned.

The specialized agent more often reached dataset registry entries, while the general-search agent more often reached narrative pages or portals. A registry entry can bring together the description, provider, variables, dates and download information an agent needs to continue.

That does not mean a general web index has no way to distinguish page types, or that metadata carries no relevance information. It describes the outcomes of these systems on these queries.

A useful hypothetical comparison is a news article discussing a municipal survey versus the survey's dataset record. Both may match the query's topic. Only the latter may identify the downloadable file and its usage conditions directly. The article can remain valuable evidence while failing the agent's data-retrieval task.

Markup and the search ecosystem changed together

Google Dataset Search applies its own inclusion and filtering processes, and both search products use proprietary ranking. The same model backbone reduces one source of variation, but it does not make the indexes otherwise identical.

The researchers therefore describe an end-to-end utility comparison. The observed gain cannot be assigned entirely to schema.org markup, and the unanswered queries cannot all be attributed to one implementation mistake by publishers.

The study also required human evaluation when automated page scraping failed. That is a useful reminder that a result's usability cannot always be settled by a successful URL response or an automated extraction alone.

The narrow dataset task matters. These findings do not establish that adding generic structured data improves brand recommendations or citations in public AI search.

Inspect the route from description to data

If you publish datasets, open the page an agent or user actually lands on. Can someone identify what the dataset contains, its relevant date or version, who provides it and how to obtain a usable file? Follow the download rather than stopping at a claim that one exists.

Check that the visible record and structured metadata agree. A valid markup report cannot tell you whether the linked file is accessible or whether it is the right dataset for a question. Participation in a specialized index is useful only when the underlying record does the job.

For an agent evaluation, report precision and query coverage together. Then inspect the failures: wrong topic, wrong destination, inaccessible data and no result require different work. An agent that reaches usable records for more of your actual questions is the outcome to test.