Executive Summary

Any study that watches MCP servers actually behave hits the same wall. Most published servers are not in a state where you can simply run them and talk to them. The paper puts it plainly in its opening section: "Every behavioral MCP study must solve the same problem: most published servers cannot simply be launched and talked to."

The literature has answered that wall in several ways, and all of the answers involve selection. Reference sets and popularity lists select for servers people already use. Hand-built frames select for servers the curator could get working. MCPZoo, the largest effort of its kind, repairs servers with a multi-agent framework until they run, which turned 64,611 collected servers into 37,288 that support dynamic analysis. Each choice is sound engineering for the purpose it serves, and the paper grants as much. All four choices also erase the same quantity: how much of the published population was never alive to begin with.

So the paper turns the other way and declines to fix anything. Its sentence is short: "We measure that quantity by not solving the problem." It draws a probability sample from the registry, probes every draw exactly once with no repair, no retry and no credentials, and records an outcome for all 400, included or excluded with a reason. Letting no server fall out of the count quietly is the heart of the design.

The paper also stakes out a narrow claim for itself. In the related work section the author states first that this behavioral sample is two orders of magnitude smaller than what MCPZoo covers, and that it makes no security claims at all. The author then names the exact point where the two efforts complement each other. MCPZoo's repair pipeline exists to turn servers that do not run into servers that do, and the population that pipeline rescues is precisely what this paper measures. The same line gets drawn around Bharti's study, which tracked the registry across 120 snapshots. How fast descriptions go stale belongs to that work; this paper has two snapshots, so it uses its census only to size the population and build a sampling frame.

1.1From census to sample

The measurement has two layers. First a collector walks the official registry API to the end by cursor pagination and records self-reported metadata only: deployment model, package ecosystem, declared transport, lifecycle status, and a pinned schema revision. That layer runs no third-party code, so it extends to the whole published population. The sweep of 22 August 2026 found 24,135 servers. Of those, 7,258 are published to npm, declare stdio, and carry active status. That subset is the sampling frame.

The draw sorts the frame by registry identifier and runs a partial Fisher–Yates shuffle driven by a seeded mulberry32 generator, taking 400 without replacement. The seed, 20260819, is published, and the draw manifest records the SHA-256 of the exact frame bytes. Anyone who redraws later and finds the frame has moved will detect the mismatch instead of assuming equivalence. Each drawn package is launched over stdio with npx, and the instrument speaks the MCP wire protocol directly: it completes the initialization handshake, enumerates tools with tools/list, and checks each tool's JSON Schema against the constraints the specification requires. No credentials are ever supplied, and no tool with side effects is ever called.

Reproduction machinery carries the weight of this measurement. The prober mcp-probe, the census collector, the seeded draw script, a resumable probe runner, the aggregation scripts, the self-checking duplication code and the validity test suite are all public. Per-server outcomes for all 400 draws ship as well, so a reader can recount instead of trusting the totals. Every number in the paper regenerates from committed scripts, the published seed, and the frame hash recorded in the draw manifest.

One layer stands outside that rule. The census sweep cannot be recreated after the fact, because the population keeps moving, so each snapshot is released as the tally taken at that moment. The author attaches honest failure to this layer rather than reproduction. The collector records how many pages it walked, how many version rows it saw, and a flag for whether the sweep ran to completion. A run that was cut short is marked as a lower bound instead of being reported as if it were whole.

The provenance deserves to be stated clearly. This measurement comes from a single independent researcher, posted to arXiv on 10 September 2026, and it has not been peer reviewed. The licence is CC BY 4.0 and v1 is the only version. Both facts belong in the same sentence: no review process stands behind it, and every piece of its reproduction machinery is open.

The 400 draws split four ways. 195 servers completed the initialization handshake, which is 48.8%. 150 never started at all, 37.5%. 53 got no further because they required credentials, 13.3%. For 2 the package itself could not be fetched, 0.5%. Those four rows account for all 400, and the sentence the paper places under its table nails that point down: "No server was dropped silently."

Table 1. Outcome of every one of the 400 draws. Source: arXiv:2609.10962v1 Table 1.

2.1How much curation adds

Point the same instrument at a different sample and a different number comes back. The author assembled a 24-server frame by hand from reference servers and popular community servers, ran the same probe against it, and 16 of the 24 were included, which is 66.7%. The 17.9-point gap between the two figures is the size of the selection effect curation produces. The comparison puts a number on how much a hand-picked list lifts the start rate.

It helps to be clear about what that comparison group is. The 24-server frame carries no citation number in the paper. This is not somebody else's earlier study but the author's own hand-curated sample, measured earlier with the same instrument. Reading it as "prior work found 66.7%" gets it wrong. The size is also a limit in its own right. Both of the gaps discussed below rest on that small comparison group.

The awkward part is that curation does not push on one axis only. The share of servers that start is higher in the curated sample, and the share of tools missing safety annotations is lower there. Section 3 gives the numbers on that second axis, but the directions alone make the picture clear. Curation makes the population look healthier on one axis and sicker on the other.

That crossed direction bites immediately in practice. A bias that ran one way could be corrected with a coefficient. Here it cannot, which is why estimating the population from figures measured on a curated list does not work. The direction of the correction changes from metric to metric.

2.2More are dead than are locked

The shape of the failures differs from what the literature expected. Credential gating is the reason usually named for why public MCP servers cannot be probed, and 53 of the 400 did land in that bucket. But 150 simply never started, close to three times as many. Doors that cannot be opened for want of a key are far rarer than doorways that were never built. The paper sums the observation up this way: on the very frame from which behavioral MCP studies draw their samples, roughly two published entries in five are dead.

The underlying worry, that a listing and a working thing are not the same, has come up on this blog before. Public API training data built from only the tool links that actually ran verified graph edges by calling them. That piece counted links between tools and this one counts server handshakes, so the two sets of numbers do not sit side by side. They share a stance. Being listed and working have to be counted separately.

The 195 servers that came up advertised 2,766 tools, and 2,759 of those carried a description. That leaves seven without one, and they are worth keeping in view, because the key used later to count duplicates concatenates a tool's name with its description. Tools per server are heavily skewed: minimum 1, median 8, 95th percentile 46, maximum 300. One server advertised no tools at all.

3.1The layer machines enforce had no gaps

Structure came first. Across 2,766 tools the count of fatal JSON Schema violations was zero, and zero of the 195 servers had one. No missing schema, no wrong type, nothing broken in properties or required. To keep this from being written off as small-sample luck, the author adds one piece of support: the same result had appeared in an earlier curated frame of about 200 tools, and it held on a sample roughly fourteen times larger that was drawn rather than chosen. So the finding looks more like a structural property than a coincidence. The common belief that MCP tool schemas are frequently malformed gets no support from this measurement.

3.2The layer people fill in is more than half empty

Another layer of the same servers looks nothing like the first. The MCP specification defines four annotations that tell an agent, before it calls a tool, what kind of thing that tool is. readOnlyHint says whether it only reads, destructiveHint whether it makes changes that cannot be undone, idempotentHint whether repeating the same call is safe, and openWorldHint whether it touches the world outside. All four are optional, so nothing is compelled. No required field is missing here; a field nobody has to fill simply went unfilled. In the random sample, 1,626 of the 2,766 tools, or 58.8%, carried no annotation whatsoever.

In the 24-server hand-picked frame the same figure was 41.5%. Curation improved this number too, by 17.3 points, and for the same reason it inflated the start rate. Reference servers annotate their tools, and curated frames are full of reference servers. Hence the line the paper attaches: the lower figure should not be quoted as an ecosystem rate.

The specification settled in advance what an empty field means. The four annotations are optional, yet the schema states the value that applies when one is absent. readOnlyHint defaults to false, destructiveHint to true, idempotentHint to false, and openWorldHint to true. By the letter of the spec, a fully unannotated tool is not read-only, may make changes that cannot be undone, offers no guarantee that repeating a call is safe, and interacts with the outside world. So 58.8% describes less a hole in the information than a state in which more than half of the tools advertised by live servers fall under the specification's own conservative defaults.

The specification draws a line on the other side as well: an annotation being present is not a reason to believe it. The section governing server tools states that clients MUST consider tool annotations to be untrusted unless they come from trusted servers. An annotation is a server's own declaration about its own tool, which puts it on the same footing as the self-reported registry metadata. The official MCP blog marked the same boundary in March 2026. Annotations are not an enforcement mechanism, and if you need a guarantee that some tool cannot exfiltrate data, that job belongs to network controls or a sandbox rather than to a boolean hint. The same post notes that clients vary in how strictly they honour those conservative defaults.

The shape of the distribution is the more interesting part. Of the 194 servers that advertised at least one tool, 72 annotated every tool and 122 annotated none. Servers that annotated only some of their tools did not appear in this sample even once. Annotating is therefore not a per-tool decision but a team habit.

How the author handles this point matters as much as the measurement. All 194 servers were all-or-nothing, and the paper still does not write that no partially annotated servers exist. Seeing none of a rare category in a single sample is grounds for an upper bound rather than evidence of absence. Zero out of 194 yields a one-sided 95% upper bound of 1.53%, and that is the number the paper reports instead. The sentence after it stands out more. An earlier unreleased run over 214 servers had found four partially annotated servers, this run failed to reproduce that, and the author reported the discrepancy without explaining it. Records of that kind are what offset the weakness of a preprint that has not been reviewed. The validity section holds another one. A GitHub topic tally that an earlier draft had carried as ecosystem context became unavailable during the second sweep, so the author states that the figure is used to support no claim at all.

3.3Versions do not only lag

The 195 servers negotiated four protocol versions. 192 spoke 2025-06-18, and 2024-11-05, 2025-03-26 and 2025-11-25 accounted for one each. That last one is newer than the baseline the prober's own client advertises. Version spread in the deployed population is not a tail trailing off toward the old, then, but a distribution open at both ends, and anyone wiring tools into an agent has to think about compatibility in both directions.

The specification has moved further in the meantime. As of September 2026 the current revision is 2026-07-28, which puts the 2025-06-18 that 192 sampled servers negotiated two revisions behind. The centre of mass of the deployed population stayed where it was while the spec was revised twice. The paper does not discuss this contrast. This report checked it in the specification documents.

Here the paper turns its instrument outward. With 2,766 tool descriptions in hand from servers that were independently written and actually deployed, it can compare them against the tool descriptions that tool-use benchmarks put in front of models. Existing criticism of those benchmarks has aimed mostly at the scoring. Bhat and colleagues audited the evaluators of four tool-calling benchmark families including BFCL v4 and reported an 18.5% evaluator-human misalignment rate on 496 expert-reviewed tasks, along with score spreads of up to 18.9 points across repeated runs of one setup. This paper looks past the scoring machinery at the corpora being scored.

One sentence in the related work section locates that move. To the author's knowledge, no published work measures duplication within these corpora or compares their tool distributions against tools that are actually deployed. Plenty of work has questioned benchmarks, but the questioning concentrated on the scoring, and what the scored material is made of was hard to ask about for want of anything to compare it to. This paper could ask it only because it had 2,766 tool descriptions from real deployed servers in hand.

4.1One yardstick, held constant and pinned in code

All three corpora are measured the same way. The unit of comparison is a tool's name concatenated with its full description, and every corpus is globally deduplicated on that exact key before any similarity is computed. The paper enforces this in code rather than leaving it to discipline, for a reason rooted in experience: an earlier iteration of the work keyed deduplication per file, missed repeats that spanned files, and inflated a measured figure by fourteen points. The author wrote that mistake into the method section and then sealed it shut with code.

Three corpora go in. Real MCP is the 2,766 tools advertised by the 195 servers that came up. BFCL v4 is every tool definition in the function list of every released row, 8,726 records; seven of the twenty released files expose no function list at all, including the four multi-turn files and the memory file, and those contribute nothing to the count. UltraTool is every tool in the tools list of every English-split row, development and test together, 14,084 records. Every source file is pinned by SHA-256 in the released provenance record.

The first measurement is not near-duplication but exact duplication: how many records in each release repeat a name and a full description character for character.

Table 2. Exact-duplicate contamination of the raw releases. Source: arXiv:2609.10962v1 Table 2. The warning the paper prints directly beneath this table travels with it. The percentage is sensitive to which files are included, and a benchmark that reuses one tool across many rows produces a high rate by construction. The post-deduplication rates are considerably more stable, and the two should be read together (section 5).

The paper does not use this table to judge benchmark quality. The conclusion it draws concerns counting procedure. Any statistic computed over these files without global deduplication measures how often a benchmark repeats a task, not how many tools it contains. The figure people were calling "8,726 supported tools" was in fact the number of times the same tools reappeared across rows.

The method section singles out the exact-duplicate rate for a reason, calling it a distinct and more consequential quantity than the near-duplicate rate. Near-duplication depends on where you put a threshold, which makes it a matter of interpretation. Exact duplication counts how many times an identical string appears, which leaves no room for interpretation at all. Any tool or dashboard that reads a release file and aggregates something is already off at this layer, before anyone picks a threshold.

The other end of the table deserves equal weight. Exact duplication among real MCP tools is 10 out of 2,766, or 0.4%. Hundreds of strangers publishing their own tools almost never end up with a name and a description matching character for character. The gap between the benchmark's 68.8% and deployed tooling's 0.4% did not open over diligence. It opened over how the files are organised. One side ships a tool again for every task; the other advertises each server's tool list once. Measured with one yardstick, that structural difference lands directly in the numbers.

4.2Two documents meeting at the same integer

The claim that 85.6% comes out of file structure does not have to stay a guess, because the team that built UltraTool wrote the process into their own paper. They acknowledge that creating a tool separately for each query leaves similar tools scattered across the benchmark, and they added a manual step called Tool Merge to deal with it. Verbatim: "Due to each tool being created separately for the queries, there may be similar tools that exist in the whole benchmark. To address this issue, we manually merge groups of tools that have similar functions into a single tool." The outcome gets one sentence: "The remained toolset of UltraTool contains 2,032 distinct tools."

That 2,032 is the firmest cross-check in this report. The measurement paper never consults UltraTool's merge procedure; it simply runs global deduplication on the released files, keyed on name plus description. Counted independently that way, UltraTool's English split comes to exactly 2,032 tools after deduplication. Two documents that know nothing of each other meet at the same integer.

No matching disclosure turns up on the BFCL side. A pass through the dataset card and repository documentation found no passage explaining how much function schemas are shared and reused across test categories. That absence is not a defect by itself, yet it gains circumstantial weight next to the observation coming in the next section. UltraTool has an explanation on record, saying the merge already happened, and BFCL has no such document, while after the duplicates come out the two corpora split in exactly opposite directions.

Quoting 68.8% and 85.6% on their own to argue that a benchmark is riddled with duplicates is therefore a reading the paper explicitly rules out. These two figures have one legitimate use: as the methodological warning that aggregating over release files without global deduplication counts task repetition rather than tools. Their job is to make you ask, whenever a vendor advertises "N supported tools," what deduplication rule produced N.

Strip the exact duplicates out and the three corpora end up close in size. Real MCP holds 2,756 tools, BFCL v4 holds 2,724, and UltraTool's English split holds 2,032. The threefold gap that looked like 8,726 against 2,766 in the previous section effectively disappears, and every comparison from here runs between corpora of similar size.

Near-duplication is measured on top of that. Vectorisation is TF-IDF over word unigrams and bigrams with sublinear term frequency, fit separately per corpus and never on a shared vocabulary. Similarity is cosine. Redundancy at a threshold is defined as the share of deduplicated tools having at least one other tool at or above that cosine. The paper spells the definition out in the body for a simple reason: a different definition yields a different number.

The decisive step comes next. The rate gets split by whether a tool's near-duplicate partner lies inside the same authoring unit or outside it. For real MCP the authoring unit is the server; for both benchmarks it is the task row. One project producing several similar tools within itself and independently presented items resembling one another are entirely different events.

Table 3. Redundancy after global deduplication, by cosine threshold. Source: arXiv:2609.10962v1 Table 3.

The decomposition changes the finding. Every near-duplicate among real MCP tools sits inside a single server: the list_x, get_x and create_x families that one project naturally produces. Across independent authors the rate is 0.0% at every threshold tested. BFCL's redundancy is the opposite kind. Of its 16.7 points, 16.4 lie between independently presented tasks, and it is the only one of the three corpora whose near-duplicates survive at cosine 0.90. UltraTool comes in at 0.3%, cleaner than the tools people actually deploy.

The paper separates the claims this picture supports from the ones it does not. Two sentences mark the border.

5.1Where the author shook the result first

A figure of 0.0% across authors invites suspicion if it is left standing on its own. A separate measurement has already found that clones are common in the MCP ecosystem at the code level. Kim and colleagues extracted 87,564 tools from 7,508 MCP repositories, measured lexical and fuzzy structural similarity, and had humans inspect a sample, confirming 60% of high-Jaccard candidates and 85% of high-ssdeep candidates as genuine clones. How that result and this 0.0% can both hold becomes the question.

The author raises it as a validity threat and tests it three ways. First, measuring npm package descriptions, which exist for both started and unstarted servers, returned 0.0% in both groups at every threshold. The author notes that this test reads descriptions rather than code and has low power because both sides are zero. Second, author-family concentration by npm scope looked similar across the two groups: the largest family in the sample, seven packages, was included in full, and the next eight were excluded in full. Third and most directly, the author contributing the most servers that all started had seven servers with 125 tools between them, and the maximum cosine among those servers was 0.623, below even the lowest threshold reported. The conclusion goes only as far as finding no evidence that the filter selects clones out, and the paper adds the case it would miss. A clone that copies the implementation while rewriting the tool descriptions escapes this method and gets caught by the other one. Should that be what is happening, the gap between code-level and interface-level duplication would itself be a result worth reporting.

The remaining tests cut against the result too. After deduplication BFCL averages 1.8 tools per task row against real MCP's 14.2 per server. Removing same-unit pairs therefore takes much less away from BFCL by construction, so BFCL's cross-task figure sitting almost on top of its overall figure is partly an artefact of that asymmetry. Real MCP collapsing from 2.8% to 0.0% runs the other way and the asymmetry does not explain it. The possibility that large servers drove the rate was measured separately: capping tools per server at 10, 25 and 50 gives 0.9%, 2.2% and 3.0% at threshold 0.70, bracketing the uncapped 2.8%. The two servers advertising 300 and 122 tools did not manufacture the number. Finally, TF-IDF cosine measures lexical overlap rather than semantic equivalence, so two tools doing the same job in different words are counted separately in every corpus. All three rates are biased downward, and the paper states that it sees no reason for that bias to fall differently across corpora while acknowledging it has not shown otherwise.

On the subject of sampling, this blog has covered random sampling beating smart sampling in a data quality benchmark. Do not collapse the two conclusions into one. That piece dealt with selection techniques underperforming expectations, and this one is about a quantity that only a random draw can measure at all. The overlap is a sequence: ask how the sample was drawn before anything else.

Two complete sweeps draw the background around the measurement. The population grew from 16,548 servers on 14 July 2026 to 24,135 on 22 August, roughly 195 net new servers a day across 39 days, and both sweeps ran to completion. The composition of deployment models was recorded at the same two moments.

Table 4. Deployment model across two complete sweeps 39 days apart. Source: arXiv:2609.10962v1 Table 4, §7.

Within that window remote-only overtook package-only, growing 70.1% against 26.3%. The paper then steps back. Two snapshots cannot establish a trend, it says, and it claims none, reporting only the change between two measured endpoints. A reader who turns this table into "the trend is moving remote" has gone one notch past the source.

The methodological result points back at this paper itself. The npm-and-stdio slice a local behavioral instrument can reach grew in absolute terms from 5,804 to 7,414 servers, while its share of the population fell from 35.1% to 30.7%. An instrument that handles only stdio is measuring a shrinking minority. The paper states that this constraint binds its own behavioral layer exactly as it binds everyone else's. The same method, run again next year, will cover a smaller slice.

The measurement condition is ageing along a second axis. Inclusion here was decided by completion of the initialization handshake, and the current specification revision, 2026-07-28, removed that handshake. It drops initialize and notifications/initialized in favour of carrying the protocol version and client capabilities in a meta field on every request, and it requires servers to implement server/discover to announce supported versions, capabilities and identity. A compatibility path for servers on older revisions is written into the same specification, and all 195 sampled servers negotiated an older revision, so the measurement itself holds. Still, what 48.8% is a share of has to be read together with a revision date. This revision is also absent from the paper. Its date falls between the two sweeps, and this report checked it in the specification documents.

6.1The registry is a catalogue, not a reviewer

The gap this measurement exposes was announced in the official documentation all along. The README of the modelcontextprotocol/registry repository defines the registry as something like an app store that serves a list of servers to MCP clients. A catalogue that aids discovery, in other words, rather than a reviewing body. Even the "validation" the documentation describes goes as far as a schema format check at submission time; no functional review that checks whether a server starts or whether its tools are safe exists in the documents. A community working group operates the registry, and the API has been frozen at v0.1 since 24 October 2025. The same documentation records that the registry launched in preview on 8 September 2025, with general availability to follow. The infrastructure the paper swept twice was still in its preview stage. The design stayed the same after MCP moved to the Linux Foundation's Agentic AI Foundation on 9 December 2025. A change of governance did not change the self-reporting structure.

Observations from other layers point the same way. Padilla audited 414 internet-facing MCP servers dynamically, found 68 reportable vulnerabilities, and reported that 41.6% of confirmed servers vanished within three days between consecutive measurement rounds. That figure corroborates from the remote layer the churn implied by the 37.5% start failures this paper saw locally. Bharti's 89-day tracking of registry description staleness belongs to the same picture. Only 8.6% of servers ever rewrote a description, and the most active 5% generated 61% of all change events. Registry entries mostly stop moving the moment they are filed.

It is worth adding that the question "how many MCP servers are there" gets a different answer from every counter. A sweep of the official registry API, a third-party directory's crawl, and a vendor's announced count of active servers each count a different population. The figures this report uses in the body are the two the paper itself counted by complete sweep.

6.2The market already answered with gateways

"Do not trust the registry as given, add a verification layer" is a conclusion the market has been shipping products against for a while. MCP infrastructure in 2026 sorts itself into three tiers: public registries that aid discovery without controlling calls, enterprise registries that expose only organisation-approved servers in an internal catalogue, and gateways that enforce identity, permissions and policy at runtime. Across that category, however, no vendor publishes an uptime figure measured by probability sampling. The usual claim is a qualitative word like "verified." That gap is the ground this paper stands on.

Two cases from Korea show the opposite ends of exposure design. According to coverage of a presentation at MCP Dev Summit 2026, Naver Cloud runs more than 10,000 APIs counted across its public and internal surfaces, and its MCP gateway exposes exactly two tools to users. Semantic search picks the tool that fits each request and attaches it on the fly. At the other end sits Kakao's PlayMCP, which opened in beta on 13 August 2025 to let outside developers register servers and, as of its 1 May 2026 update, lists roughly 200 external MCP servers. Narrowing the exposed surface to an extreme, or opening it and letting it grow: each team's answer lands somewhere between those two.

The work Pebblous does in DataClinic amounts to answering "is this dataset usable" with instruments instead of impressions. This paper did the same thing to a list of tools. So the point of contact lies less in the subject of MCP than in the way an asset gets measured.

7.1In the agent era, tools enter the asset inventory

The 24,135 entries in the registry form a catalogue rather than a set of verified assets. Sample them at random and half of them do not even come up. If AI-Ready Data has meant "in a state a model can use directly," then in an environment where agents do the work, that definition has to cover tools alongside datasets. Tools then face the same questions a dataset faces. Is the schema valid? Is there an annotation that says what this means? Does it actually get called? The three measurement layers this paper stacked up, start-up and schema and annotation, have the same structure as the diagnostic layers DataClinic applies to datasets.

7.2The composition of evaluation data sets what a score means

The second half of the article lands exactly on an axis Pebblous has worked for years. Aggregate a tool-use benchmark's release files without global deduplication and the thing everyone called a tool list turns out to have been task repetition. The proposition that the composition of an evaluation dataset sets the meaning of the score gets confirmed once more, this time in the territory of agent tools.

The same paper also teaches restraint about that proposition. Inside one table, UltraTool came out cleaner than deployed tools. The conclusion is not that synthetic is bad and real is good but that no corpus can be judged before it has been measured with the same yardstick. That posture is far closer to the discipline synthetic data work requires. Lay the contrast from section 3 on top and one sentence remains. Quality holds only in the layers where it can be verified.

7.3Four things a team wiring up tools can use today

For a team about to connect external tools to an internal agent, this measurement leaves a concrete procedure. All four items follow directly from figures in this article.

• Launch them yourself before adopting them. Instead of trusting the registry listing, run your own probe to confirm start-up and handshake. The measured expectation is one in two.
• Treat an unannotated tool as the dangerous case. This is the specification's default before it is a matter of caution. With the annotations empty, the spec reads that tool as one that may make changes it cannot undo and that touches the outside world. Annotation tends to be all or nothing per server, so one review of a server settles it for every tool it offers.
• Treat registry metadata as something to verify. Transport and status declarations are self-reported, and a server that declared wrongly is still counted as it declared.
• Ask what deduplication produced "N supported tools." Section 4 shows in numbers that N can be the count of one tool repeated across rows.

7.4For anyone who measures, this seat is empty

Pebblous has spent its working life measuring whether data is ready. With the tool ecosystem adding 7,587 servers in 39 days and remote-only having overtaken package-only between the two sweeps, the seat marked "tools are an asset that has to be measured too" is still empty. Plenty of vendors sell a verified catalogue; none publishes an uptime figure from a probability sample. That is the size of the gap.

Thank you for reading to the end.

The figures in this article come from four places. The primary source, arXiv:2609.10962v1, was downloaded as full-text HTML and checked section by section and table by table, with the values in math nodes that text extraction drops restored separately. The five adjacent studies this paper cites were not taken on its summary: each arXiv page was opened directly on 14 September 2026 and the numbers re-confirmed. UltraTool's "Tool Merge" step and the figure of 2,032 were read as sentences in the original paper. Annotation defaults, the limits of trusting annotations, and the specification's revision history fall outside what the paper covers, so the MCP specification, its schema file and the official blog were opened directly on 14 September 2026. Registry policy and the Korean cases are cited by the path taken to them: official documentation, an official newsroom, and press coverage of a presentation.

Primary sources

• 1.Haseeb Mohammed Afsar. "What a Random Draw from the MCP Registry Contains, and What Tool-Use Benchmarks Contain Instead." arXiv:2609.10962v1 [cs.SE], 10 September 2026, CC BY 4.0. Single independent researcher, preprint, not peer reviewed. Tables 1–4, Figures 1–3, §3 method, §4–6 results and §8 validity threats were verified in this document. As of the check on 14 September 2026, v1 is the only version.
• 2.Haseeb Mohammed Afsar. mcp-probe: a conformance and reliability checker for Model Context Protocol servers (software and dataset). Zenodo, 2026. Concept DOI 10.5281/zenodo.21347997, code at github.com/itguruhaseeb/mcp-probe: the prober, the census collector, the seeded draw script, the aggregation scripts and the validity test suite.

Adjacent measurement studies (each re-checked at source)

• 3.Pei Chen and eight others. "Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability." arXiv:2607.11086 (2026). The collection the authors themselves call MCPZoo: 64,611 unique servers collected, and more than 37,288 made available for dynamic analysis after repair. The same paper reports that existing scanners flag 96.89% of servers as risky while fewer than half of the sampled alerts survive manual verification.
• 4.Gautam Bharti. "Registry Descriptions Go Stale Unevenly: An 89-Day Measurement of Model Context Protocol Drift, and Why Drift-Ranked Re-Auditing Under-Covers It." arXiv:2608.00997 (2026). 120 snapshots over 88.6 days across 19,099 servers; 8.6% of servers ever rewrote a description, and the most active 5% produced 61% of change events. The title says 89 days and the body says 88.6.
• 5.Nicolás Padilla. "Exposed by Design: A Dynamic Security Assessment of Internet-Facing MCP Servers at Scale." arXiv:2608.00150 (2026). 414 internet-facing servers audited dynamically, 68 reportable vulnerabilities, and 41.6% of confirmed servers gone within three days between consecutive rounds.
• 6.Taein Kim, David Jiang, Yuepeng Hu, Yuqi Jia, Neil Gong. "Evaluating Tool Cloning in Agentic-AI Ecosystems." arXiv:2605.09817 (2026). 87,564 tools extracted from 7,508 MCP repositories, with humans confirming 60% of high-Jaccard and 85% of high-ssdeep candidates as genuine clones. The abstract's opening claim, that raw tool counts may substantially overstate ecosystem diversity, meets the concern of section 4 of this article.
• 7.Vishvesh Bhat, Jay Vaghasiya, Muhammad Ahmed Mohsin, Asad Aali. "Benchmarking the Benchmarks: A Validity Audit of Tool-Calling Evaluation." arXiv:2607.02577 (2026). An evaluator audit of four families: BFCL v4, τ2-Bench, LiveMCPBench and MCP-Atlas. Evaluator-human misalignment of 18.5% on 496 expert-reviewed tasks, and a score spread of 18.9 points (57.9%–76.8%) across repeated runs of LiveMCPBench.

Benchmark corpora at source

• 8.Shijue Huang and twelve others. "Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios." arXiv:2401.17167 (2024). The original UltraTool paper. Its data construction section holds the Tool Merge step, "we manually merge groups of tools that have similar functions into a single tool," and the sentence "The remained toolset of UltraTool contains 2,032 distinct tools." This is the basis of the cross-check in section 4.2.
• 9.Berkeley Function Calling Leaderboard v4 evaluation data. Gorilla project repository, github.com/ShishirPatil/gorilla (accessed by the paper on 22 August 2026).
• 10.UltraTool English dataset. Repository github.com/JoeYing1019/UltraTool (accessed by the paper on 22 August 2026).

Specification, registry and governance

• 11.Model Context Protocol Specification, revision 2025-06-18, Server / Tools (checked 14 September 2026). The exact revision 192 sampled servers negotiated. This section carries both the rule that annotations are optional and the warning that "clients MUST consider tool annotations to be untrusted unless they come from trusted servers." The four defaults (readOnlyHint false, destructiveHint true, idempotentHint false, openWorldHint true) are stated in the ToolAnnotations comments of the same revision's schema.ts. The revision dates cited in the body, 2024-11-05, 2025-03-26, 2025-06-18 and 2025-11-25, all exist.
• 12.Model Context Protocol Specification. Versioning and 2026-07-28 Key Changes (checked 14 September 2026). As of September 2026 the current revision is 2026-07-28. SEP-2575 removed the initialize and notifications/initialized handshake, moved the protocol version and client capabilities into _meta on every request, and made server/discover mandatory for announcing supported versions, capabilities and identity. The compatibility path for older revisions is in the same document. The paper does not cover this revision, so the statements in sections 3.3 and 6 were verified by this report against the specification.
• 13.Ola Hungerford, Sam Morrow, Luca Chang. "Tool Annotations as Risk Vocabulary: What Hints Can and Can't Do." Official MCP blog, 16 March 2026. The summary that "The defaults are deliberately cautious: a tool with no annotations is assumed to be non-read-only, potentially destructive, non-idempotent, and open-world," together with the limit that annotations "aren't enforcement." The same post notes that annotation practice varies by server and that clients differ in how strictly they honour the defaults.
• 14.modelcontextprotocol/registry GitHub README (checked 14 September 2026). Defines the registry as a service that "provides MCP clients with a list of MCP servers, like an app store for MCP servers." The "validation" described there extends to schema format checking at submission, with no functional pre-review procedure. Launched in preview on 8 September 2025; the API has been frozen at v0.1 since 24 October 2025, with general availability announced for later.
• 15.Anthropic. "Donating the Model Context Protocol and Establishing the Agentic AI Foundation." anthropic.com, 9 December 2025. The transfer to the Agentic AI Foundation under the Linux Foundation. The same post records "more than 10,000 active public MCP servers" and "97M+ monthly SDK downloads across Python and TypeScript."

Industry and Korean context

• 16.CIO.com, coverage of MCP Dev Summit 2026, 18 August 2026. cio.com. The Naver Cloud gateway case ("more than 10,000 internal and external APIs, two tools exposed externally"). This article cites the coverage of the presentation rather than the presentation itself.
• 17.Kakao Newsroom. PlayMCP beta launch (13 August 2025) and status update (1 May 2026): roughly 200 external MCP servers.
• 18.NimbleBrain. "State of MCP Security." nimblebrain.ai, March 2026. A full query of the official registry's public API (8,074 records, 3,012 unique servers), with the observation that the registry lists servers without evaluating them.

Related Pebblous articles

• 19.Pebblous Blog. Public API Training Data Built From Only the Tool Links That Actually Ran: the same concern at a different layer. The unit of measurement differs, so its figures do not sit beside this article's.
• 20.Pebblous Blog. Random Sampling Beats Smart Sampling in a Data Quality Benchmark: another case on the axis of sampling design deciding the conclusion.
• 21.Pebblous Blog. The Whitelist That Filters Out 27% of the Agent Skills Found Online: the security side of registry trust. The specifics of the supply-chain incidents are covered there.
• 22.Pebblous Blog. A2A Joins MCP Under One Foundation, Enterprise Authorization Stays Outside: background on the MCP governance transfer.