Executive Summary
The data labeling market is splitting in two directions at once in 2026. At the bottom, "easy labels" (bounding boxes, simple classification, basic RLHF) are being swallowed by AI pre-labeling and collapsing into a commodity layer. At the top, only PhD-level domain experts earning $50–200 an hour remain, forming a premium layer. The first quantitative proof of this polarization is that the semi-supervised, human-in-the-loop (HITL) segment, the work that requires a human, is growing more than ten percentage points faster than the market as a whole. The structure of that restructuring, and the new gap opening inside it, is best read through one lens: data quality.
The more decisive shift is that even the judgment of finding and vetting those scarce experts is now being delegated to AI. Mercor screens candidates with a 20-minute AI video interview, Micro1 lets an AI agent pass only the top 1%, and Handshake AI sources experts at a near-zero cost from a verified credential-identity graph built over twelve years. The axis of competition has moved from "how many labels can you attach" to "how fast can you find the right person" — and the proof is that the revenue of these three vendors doubles every few months.
If the filter that picks people is an AI, then the filter's biases and errors are imprinted on the labeler pool, seep into the labels, harden into the training data, and propagate into the model's internal representations. Who, this time, verifies the experts who verify the labels? With the wages of top-tier experts and bottom-tier Global South moderators coexisting inside a single market, the question narrows to one: who, in the end, is the final guarantor of data quality?
$2.32B→$6.53B
Labeling market size
2026→2031, CAGR 22.95% · Mordor
33.15%
HITL segment CAGR
+10.2pp above the market · Mordor
$50–200
Frontier expert hourly rate
Cross-checked: Surge · Mercor · Handshake
25–150×
Top-vs-bottom wage gap
Expert vs. moderator $1.32–2
What Mechanical Turk Left Behind
Yesterday we covered the retirement of Mechanical Turk — Amazon's decision to stop accepting new customers for its 20-year-old crowdsourced labeling platform starting July 30, 2026. That piece concluded that "AI had already been quietly replacing the bottom-tier labelers." This report begins with the next question: what moved into the space the easy labels left behind?
The answer is not an empty seat but a re-stratification. Simple classification and bounding boxes were swallowed by AI pre-labeling. Industry vendors now handle roughly 80% of all labeling automatically, and humans remain only on the hard-judgment 20% at the boundary (HeroHunt). The labeling labor market splits into an upper and a lower tier as a result. Below is a repetitive-task layer at $1–12 an hour; above is a domain-expert layer at $50–200 an hour. The middle ground between them is thinning fast.
The point is that automation did not eliminate the work — it changed the nature of the work. Labels a machine can produce converge toward zero value; judgments a machine cannot make see their value soar. The space left for people narrows from "hands that attach many labels" to "expertise that is hard to verify." What followed the disappearance of the bottom-tier labeler our previous piece described was the re-stratification of the market.
One background statistic underwrites this re-stratification, worth leaving as a footnote. A 2023 study estimated that 33–46% of Mechanical Turk workers had used an LLM to complete writing tasks. The moment a real machine took over the seat where humans had been pretending to be machines was the flare that spread the judgment — "there is no reason to hand easy labels to people" — across the whole market. By 2026, with that signal fully arrived, the market had split into an upper and a lower tier. Which side pulled how far apart is what follows.
All That Remains Is the Expert
The wage table shows most sharply which way the market has split. The labeler pay compiled by the sourcing vendor HeroHunt is not a single band but three tiers. At the bottom, the bulk commodity layer runs $1–12 an hour, handling repetitive labeling and simple vision tasks through crowdsourcing. Above it, the mid-tier holds domain professionals in finance, medicine, and law at $20–85 an hour. And at the very top, the frontier layer places credentialed PhDs and experts at $85–200 or more, doing RLHF design, complex reasoning evaluation, and reinforcement-learning environment construction.
| Tier | Hourly rate | Work |
|---|---|---|
| Frontier · Expert | $85–200+ | Credentialed PhDs and experts — RLHF, reasoning evaluation, RL environment design |
| Mid-tier · Professional | $20–85 | Domain professionals in finance, medicine, law, etc. |
| Bulk · Commodity | $1–12 | Repetitive labeling, vision tasks, crowdsourcing |
Source: HeroHunt.ai, "The Ultimate AI Data Labeling Industry Overview" (2026). Combine the top two tiers into the "expert layer"; the bottom is the "automation / commodity layer."
"$50–200 an hour for PhD-level experts" is not marketing rhetoric. Surge AI pays $50–200 to frontier experts vetted on a trust score; Mercor's average expert rate is $85, with senior specialists above $200; Handshake AI's experts earn $100–125. Because three vendors independently point to the same band, this premium is a cross-verified fact.
Where the premium comes from becomes clear once you unpack the unfamiliar item in the top row of the table: "RL environment design." The work at the very top is no longer traditional labeling (attaching ground truth to data that already exists) but building, from scratch, the reinforcement-learning "environments" in which a model learns by its own trial and error. A single simple UI task costs roughly $20,000 to build; an environment that reproduces an entire complex application is priced as high as $300,000 (HeroHunt). Hand-work that "attaches" labels has turned into design work that "builds" the stage on which learning itself takes place, and the arrival of this new category, one that did not exist at all in the crowdsourcing era, is the real force pushing the top rates up. The proposition from Section 1, that automation did not eliminate the work but changed its nature, is nowhere more vivid than in the top row of the wage table.
Only the human-in-the-loop work grows on its own
This polarization also shows up in growth rates. According to Mordor Intelligence, the AI data labeling market as a whole grows at a 22.95% CAGR from 2026 to 2031. But break out the sub-segments and the semi-supervised, human-in-the-loop (HITL) segment, the work that requires human judgment, alone grows at a 33.15% CAGR, 10.2 percentage points faster than the market overall. The restructuring ("automate the easy labels; what remains is high-value work that requires a human") is being measured in the growth curve itself.
xAI's workforce restructuring compresses this shift inside a single company. In early 2025 xAI had roughly 1,500 data annotators ("AI tutors"); in September 2025 it announced that it was cutting about 500 generalist annotators (a third of the total) and shifting weight to a specialist-tutor team. The company's official explanation was that it had "decided to reduce the share of generalist roles and accelerate the expansion and prioritization of specialist AI tutors." One caveat: the accompanying figure of a "10x specialist expansion" was a plan, not a completed count.
The AI That Chooses People
Once scarce experts become the core of competitiveness, the next question naturally shifts to "how do you find those experts fast?" And the 2026 answer goes one layer further: even the judgment of finding and vetting that person is handed to AI. Where old-style crowdsourcing was procurement ("post the work and let anyone take it"), today's leading vendors have moved to talent selection: "AI vets who's qualified and passes only the top few percent." The decisive difference is that AI judges not just the sourcing of candidates but their vetting.
The three vendors take different routes. Mercor evaluates candidates in real time with a 20-minute AI video interview; Micro1's AI agent "Zara" passes only the top 1% while policing cheating. Handshake AI took a somewhat different path: because it recycles a verified identity graph built over twelve years as a university career platform (500,000 PhD candidates, 3 million master's holders), the sourcing cost of finding a new candidate is close to zero. Three sourcing models compete side by side: real-time interview (Mercor), agent vetting (Micro1), and identity graph (Handshake).
| Vendor | Sourcing / vetting method | Expert pool / hourly rate | Ownership / neutrality |
|---|---|---|---|
| Mercor | 20-min AI video interview | 30,000+ weekly active / avg. $85, senior $200+ | Independent |
| Micro1 ("Zara") | AI-agent vetting, top 1% pass | Undisclosed | Independent |
| Handshake AI | Pre-verified identity graph (12 yrs), acquired Cleanlab | 500K+ PhD candidates · 3M master's / $100–125 | Independent |
| Surge AI | Trust-score-based credential vetting | ~50,000 / $18–24, frontier $50–200 | Bootstrapped (no outside capital) |
| Scale AI | Large-scale crowd infrastructure (weak expert tier) | Outlier / Remotasks mass infrastructure | Meta 49% stake |
Sources: HeroHunt.ai, "Top 10 Data Annotators for AI Labs: 2026 Benchmark" · Sacra. Figures are the latest confirmed values as of publication (July 2026).
The proof that this shift is not spinning its wheels is in the revenue curve. As of June 2026, Mercor's annualized revenue reached $2B, doubling in four months (in July 2026 it was in talks to raise $500M at a $20B valuation — not finalized); Micro1's annualized revenue jumped from $125M to $300M; and Handshake AI's annualized gross revenue grew 349% year over year. All three are growing at a pace where "revenue doubles every three to six months." A market has opened in which the money is not in labeling capacity but in the vetting infrastructure that picks people quickly and accurately.
Neutrality as a Quality Variable
If finding experts fast is the competitive edge, why does it matter "who owns" the vendor those experts work for? In the summer of 2025, Scale AI gave the market a live answer. When Meta bought a 49% (non-voting) stake in Scale AI for roughly $14.3 billion, the competing labs that had been using Scale's data left one after another.
Google, Scale AI's largest customer, decided to wind down its contract. Per CNBC, Google was set to pay Scale AI roughly $200 million in 2025 alone (a separate report described a contract on the order of $150 million a year as of 2024 — possibly a distinct figure for a different year). OpenAI, Microsoft, and xAI joined the exodus. The reason is simple: if you keep entrusting your data pipeline to a vendor half-owned by your competitor Meta, the flow and know-how of that data could be exposed to a rival. In the aftermath of the customer flight, Scale AI laid off about 14% of its workforce (roughly 200 people) in July 2025.
What does this have to do with data quality? Everything. A data vendor's neutrality is a trust variable as important as label accuracy. If "who owns this vendor" and "is there a channel through which my data could leak to a competitor" go unverified, then no matter how high the label quality is, you cannot use that data with peace of mind. The Scale AI case is the moment the market itself proved that a vendor's ownership structure has become a new line item in data-quality risk.
It is precisely the independent vendors from the previous section that filled the void where neutrality collapsed. Vendors not owned by a competitor, ones that control their own sourcing and vetting, earn the premium. But a more fundamental question remains. Ownership can be checked in a contract. So by what do you verify the judgment of the expert that an independent vendor picked using AI?
Verifying the Verifiers
Expert labels set the ceiling on a model's performance, because a model cannot become more accurate than the labels that taught it. But if an AI filter selects those experts, the filter's biases and errors do not vanish — they flow downstream. If the vetting AI systematically favors candidates of a certain background, that bias is imprinted on the composition of the labeler pool, seeps into the labels they produce, hardens into the training data, and ultimately propagates into the model's internal representations. It is a chain that, once set, is hard to reverse.
AI vetting filter
Selects experts
Labeler pool
Bias imprinted on composition
Labels
Seeps into judgments
Training data
Hardens
Model internal representation
Propagates
The path by which the vetting filter's bias flows down from the labeler pool to the model.
Is there direct evidence that this bias exists in expert-labeler vetting? Not yet. No externally audited bias figures for AI labs' expert vetting (the Mercor / Handshake kind) have been published. Let us be clear about that. What we can reference, though, is data from adjacent domains. In general hiring AI, bias has been confirmed repeatedly: audits have found a 35% racial gap in name-recognition algorithms, 28% age discrimination in video-analysis tools, and 22% gender bias in personality-assessment algorithms, and an October 2025 Stanford study reported the qualitative finding that résumé-screening AI gives higher marks to older men.
Here we must cleanly separate fact from inference. The fact is that "bias has been confirmed repeatedly in general hiring AI." The inference is that "the same pattern could seep into expert-labeler vetting, which has no external audit system." The latter is not yet a verified fact but a reasonable concern. Yet the very absence of an audit system that could answer that concern is the heart of the problem. Who — or what — this time verifies the experts who verify the labels? The final guarantor of this chain is, for now, empty.
The moment the source of quality is anchored to an individual's reputation ("this person is trustworthy"), reproducibility and auditability collapse. If you cannot trace which filter chose that person and what that filter missed, the trustworthiness of the final data becomes an unverifiable claim. Verifying the verifiers is, therefore, the last knot in data quality.
The 2031 Market and the Open Question
The market keeps growing. Mordor Intelligence projects the AI data labeling market to grow at a 22.95% CAGR, from $1.89B in 2025 and $2.32B in 2026 to $6.53B in 2031. (Market-size definitions vary by firm: Precedence uses a longer horizon, and The Business Research Company offers a much larger figure for a generative-AI-specific category. We take Mordor as the primary reference here.) Four forces push the growth: connected and autonomous vehicles (+6.2pp), enterprise AI adoption (+5.8pp), generative-AI RLHF pipelines (+4.1pp), and AI-governance regulation (+3.7pp).
| Growth driver | Contribution |
|---|---|
| Connected & autonomous vehicles | +6.2pp |
| Enterprise AI adoption | +5.8pp |
| Generative-AI RLHF pipelines | +4.1pp |
| AI-governance regulation | +3.7pp |
Source: Mordor Intelligence. Notably, regulatory pressure is named as one of the four growth drivers — external demand for quality assurance is pushing the market.
One market, a 25–150× gap
The rise of the expert layer does not improve the lot of the bottom-tier labeler. The two extremes coexist inside the same market. While frontier experts at the top earn $50–200 an hour, at the bottom a content moderator in Kenya takes home $1.32–2 an hour. The gap is 25–150×. A survey of 144 of them found that 81% received a serious PTSD diagnosis, and some 180 former Meta / Samasource moderators filed a roughly $1.6 billion damages suit in Kenya. In Oxford Fairwork's labor audit, Remotasks scored 1 out of 10.
Legal risk seeps into the top as well. Multiple lawsuits were filed in 2025–2026 against Surge, Mercor, and Scale AI over the misclassification of labelers treated as contractors. The faster the talent arms race runs, the more the cost of failing to prove "who worked under what terms" accumulates alongside it.
To sum up: the market grows, the price of experts rises, and sourcing gets faster. Yet beneath all that speed, one thing still lags. The way data quality is finally guaranteed. Right now that method is tethered to reputation — "an AI quickly picked a trustworthy person." Not the size of the market but the structure of this guarantee is the real stake of the next restructuring.
Why Pebblous Is Watching
This market is the 2026 edition of an old question: who harvests the data? Step one further from our previous piece on the disappearance of the bottom-tier labeler, and today's market runs in the opposite direction — a talent arms race to find more expensive experts faster. The very heat of three vendors' revenue doubling every few months exposes the scaling limit of solving quality assurance through a talent-acquisition contest. The more expert sourcing is delegated to AI, the wider the gap of "who verifies the verifiers" grows.
From a data-quality standpoint this gap is especially dangerous. Expert labels set the ceiling on model performance, and when an AI filter selects those experts, the filter's bias flows all the way from the labeler pool to the model's internal representations. The moment the source of quality is anchored to an individual's reputation, reproducibility and auditability collapse. If there is no way to re-verify the trustworthiness of the expert who verifies the labels, then even the most expensive expert's labels become unauditable data.
For a data team choosing a labeling vendor, this story is a practical checklist. As the Scale AI case shows, a vendor's neutrality and sourcing transparency are themselves data-quality risks. Can you trace "who — an expert selected how, having passed what verification — made the label"? That is the new line item that must sit next to the per-label price. Checking a vendor's ownership structure, vetting method, and audit history before signing becomes the baseline of risk management.
Editor's Note. Pebblous has long worked on the problem of guaranteeing data quality through a verifiable process rather than the human eye. The market this report describes runs toward anchoring the guarantee of quality to an individual expert's reputation, and at that point a gap opens — "verifying the verifiers." We watch this market because that gap overlaps with the exact question Pebblous DataClinic targets: where do you anchor the final guarantee of quality?
References
This report was written by cross-checking the market reports, industry analyses, and press coverage below. Because market-size definitions vary by firm, we took Mordor Intelligence as the primary reference.
Market & Industry Reports
- 1.Mordor Intelligence, "AI Data Labeling Market Size, Share | Growth Trends & Forecasts 2031." Link
- 2.HeroHunt.ai, "The Ultimate AI Data Labeling Industry Overview (2026)." Link
- 3.HeroHunt.ai, "Top 10 Data Annotators for AI Labs: 2026 Benchmark." Link
- 10.Second Talent, "Top 100+ AI in Recruitment Statistics for 2026" (citing SHRM).
Primary Company Sources
- 7.Mercor, "AI Trainer Salary: Hourly Rates & Ways to Increase Pay." Link
- 8.Sacra, "micro1 revenue, valuation & funding." Link
- 9.Sacra, "Handshake revenue, valuation & funding." Link
Press Coverage
- 4.CNBC (2025-06-14), "Google, Scale AI's largest customer, plans split after Meta deal." Link
- 5.Forbes (2026-06-16), "Why Meta Paid $14.3B For Scale AI And Alexandr Wang's Data Empire."
- 6.Forbes (2026-07-09), "AI Data Labeler Mercor In Talks To Raise $500 Million At $20 Billion Valuation."
- 12.TechCrunch (2026-07-05), "Amazon will stop accepting new customers for Mechanical Turk."
Academic Research
- 11.Stanford study (2025-10), age and gender bias in résumé-screening AI — via press coverage.
※ Academic evidence (RLHF data quality, expert vs. crowd label reliability, verifier reliability, automated-hiring fairness) will be verified and reinforced in a later editing stage.