AI data companies face an unusual predicament: the better they do their job, the less their customers may need to buy the same service again.
When a model cannot assess contract risk, experts are brought in to supply that judgment. When it cannot use software, human workflows are recorded. When it cannot complete real-world tasks, actions and interactions are collected. Each of these investments narrows the capability gap between models and people. But if models eventually surpass even the best humans, whose experience will they buy next?
The current growth thesis rests on demand moving steadily upmarket, from routine annotation to expert data and tool use, and then to robotics and the physical world. Yet an upward shift in the center of demand does not mean that the supply of marketable human capability can grow without limit. High-end data may command higher prices even as the supply of useful new information narrows. The human-data business could therefore expand before it contracts. For now, this remains a hypothesis whose validity will depend on purchasing volumes, prices and the learning efficiency of models.
1. What Do AI Data Companies Actually Deliver?
In this essay, AI data companies organize people with relevant skills to perform tasks that models need to learn or be evaluated on. They capture those people’s judgments, software interactions or physical work, including both the process and its results, then process and validate the material for use in training and evaluation.
Those people may be employees, members of expert networks, crowd workers or dedicated collection teams. Their work may involve writing answers and evaluating outputs, but it can also involve operating software, teleoperating robots or demonstrating physical actions. What distinguishes this business from general labor outsourcing is that its output is structured and assessed against the requirements of model training and evaluation. If a provider merely completes a business task once for a client, it remains a conventional labor service. It enters the data business discussed here only when the process and result are converted into material from which a model can learn.
Production today can also be shared between humans and models. A model generates candidates and humans correct them. Humans design tasks and a model attempts them at scale. A system screens outputs automatically and experts adjudicate disputed cases. Assessing the value of a company therefore requires asking what new information humans contribute and whether that information improves training or evaluation.
This definition primarily covers data services involving human participation. Licensing existing data, producing purely synthetic data and building training environments follow different supply dynamics. The same assumptions about labor costs and growth cannot simply be applied to all of them.
2. Scarcity Is Moving Upward, but the Top May Be Shrinking
In a broad sense, the industry helps models “distill” human knowledge and experience. Here, distillation is a metaphor for absorbing and refining human experience. It is distinct from the technical process of knowledge distillation from a teacher model to a student model.
Demand can be pictured as a pyramid. At the base are general knowledge and basic expression. Above them comes expert judgment, followed by the process of using tools to complete work. At the top is experience gained through interaction with the physical world and by adjusting actions in response to their consequences. The closer a task is to something a model has not yet mastered and that is hard to substitute, the scarcer qualified supply will generally be and the higher the price that useful data may command.
This is not a fixed pricing hierarchy. A routine grasping action is not necessarily more expensive than a difficult mathematics problem, and different layers may coexist for a long time. The key is to separate what models have already mastered from what humans can still add. As model capabilities move upward, the uncovered tip of the pyramid may gradually narrow. Its “area” represents the range of tasks with commercial value, not a simple difference in benchmark scores.
Increasing scarcity of high-value data is not the same as an expanding market for high-value data. Once models can generate the answers of ordinary experts, fewer people will be able to contribute consistently new judgments or identify what the model has missed on the relevant tasks. This does not mean that expert professions disappear. It means that less of their labor remains suitable for purchase as an increment to model training. The top of the pyramid may become more expensive even as it becomes smaller.
The difficulty of expert knowledge lies in recognizing conditions, handling exceptions and knowing when to admit uncertainty. Tool use, meanwhile, requires preserving the work process: what to examine first, which tools to invoke, where an error is discovered and how it is corrected. Keeping only the final report often discards the judgments needed to produce it.
The first version of GDPval, released by OpenAI in 2025, contains 1,320 professional tasks across 44 occupations, with deliverables that include documents, presentations and spreadsheets. It shows that professional capabilities can be organized into specific tasks and acceptance criteria, but the number of evaluation tasks does not indicate purchasing volume or the proportion of jobs that may be replaced.
3. Where Do Data Companies Build Their Core Capabilities?
Organizing labor is the first barrier. After finding qualified workers, a provider must screen and train them, allocate tasks and adjudicate disputes so that delivery remains consistent. The recruitment, labor, review and rework cost required for each unit of acceptable data matters more than the length of a talent roster.
Technical foresight determines where to invest. What a model lacks, which kinds of data are already saturated and what forms of feedback will be needed next must all be defined with customers and tested through experiments. Move too early and there may be no buyers. Move too late and only price competition remains. Customers have more direct access to their own models’ failure records, so suppliers do not possess an inherent advantage in making these judgments.
Data processing determines whether labor can be turned into usable material. Provenance records, cleaning and deduplication, annotation, and the alignment of time and action must all serve actual training needs. Quality cannot be established by processing speed alone. Performance before and after the data is added must be compared under agreed conditions, while controlling for duplicate samples, evaluation leakage and incorrect labels.
Compliance determines whether data can be used lawfully. Participant consent, the provenance of source materials, customer usage rights, restrictions on resale, privacy and cross-border processing can all alter the boundaries of delivery. Data that is technically useful may still be ineligible for a training pipeline.
Business development must find real budgets and translate technical value into projects that can be accepted and paid for. Sample specifications, delivery schedules, exclusivity requirements and payment terms all affect returns. Repeat purchases depend on proving that the data works and keeping pace with changing demand.
These capabilities must be assessed together. Labor organization determines whether a provider can deliver. Technical judgment determines what it delivers. Processing and validation determine whether the data is effective. Compliance and commercial execution determine whether the company can earn revenue legally and repeatedly. A company’s competitiveness rests on whether it can deliver information that customers still lack, at a reasonable cost and before the window closes.
Surge AI offers a concrete comparison over time. In 2021, OpenAI explicitly credited Surge with data collection for GSM8K, a project containing about 8,500 mathematical word problems. In 2026, Surge’s research team released CoreCraft, a customer-service simulation environment containing more than 2,500 entities, 23 tools and expert-designed scoring criteria. Its public research scope had expanded from collecting questions to include building a task environment that can be run repeatedly. That expansion places different demands on technical judgment, expert organization and software engineering, but it does not reveal the composition of Surge’s revenue.
4. The Scarcer the Expertise, the More Important Its Shelf Life
For any category of task, demand may migrate along the following path. When a model cannot perform the task, buyers purchase demonstrations. Once it can handle routine cases, they purchase difficult ones. Once it can generate candidate answers, experts are asked to find errors. As its capabilities continue to improve, human work shifts further toward designing tasks, setting standards and reviewing anomalies. Expert time remains valuable, but it may no longer be sold in the same quantities or in the same form.
Anthropic’s Constitutional AI research illustrates one such change. Humans supply principles, and the model uses them to critique, revise and evaluate responses, reducing the relevant training stage’s dependence on human harmfulness labels for each individual example. This is a change in the form of supervision under a particular setup. It does not mean that human supervision can be removed entirely.
The duration of a market window must therefore be judged task by task, not across an entire profession. General legal question answering may already be commoditized, while complex judgment, new rules and new facts remain in demand. Leading on a single evaluation does not mean that a model has reliably mastered an entire field. Tasks with clear boundaries, high repeatability and abundant existing demonstrations are usually the first to lose value.
Reselling older data to models that are catching up can extend the window, provided that rights and contracts permit it. But later entrants have another route: learning from the outputs of stronger models. The DeepSeek-R1 paper used about 800,000 curated samples to fine-tune smaller models based on Qwen and Llama, creating distilled models that improved on the reported reasoning evaluations.
The case does not provide a full cost comparison with human production, nor can it be generalized to every capability. Generation, screening, verification and training still require investment. What it does show is that for some tasks, there is now an alternative to reorganizing human labor to produce answers. Open weights offer another route for reusing existing capabilities. More customers may not be enough to offset falling prices for similar data.
Distillation can propagate existing capabilities, but it cannot replace independent observation of changes in the real world. Suppliers need to distinguish between answers that can already be replicated and new experience that models have not yet acquired.
5. The Physical World Extends the Window and Raises the Bar
There is a wide gap between understanding how a person picks up an object and having a robot complete the action with its own body. Human video can reveal task order and relationships between objects, but it may not contain the control commands, contact states or corrective actions a robot needs. Transfer must also be tested again when the hardware changes.
The DROID dataset, released in 2024, contains about 76,000 demonstration trajectories and 350 hours of interaction across 564 scenes, collected by 50 people over a 12-month project. These figures do not represent full-time working hours or an industry cost benchmark, but they indicate the organizational effort required to capture diverse real-world interactions.
The value of robotics data depends on which observations, actions and feedback it preserves, and on whether it improves execution in the real world. A video of a task completed smoothly may lack information about error correction. Controlled collection can also create value by covering variations systematically. The final test should be model improvement, not merely whether the footage looks realistic or is long enough.
Different sources can also complement one another. Physical Intelligence’s π0.5 research jointly trained on robot actions, image-text material and high-level task information. NVIDIA, meanwhile, reported that a specific simulation pipeline generated 780,000 synthetic trajectories in 11 hours. That duration excludes seed demonstrations, modeling and real-world validation, so it cannot be compared with DROID’s project timeline as a measure of efficiency. It does, however, show the possibility of scaling actions that can be modeled.
One possible future structure is to use real-world data to identify gaps and calibrate errors, while using simulation, transfer and automated collection to expand repetitive experience. Greater complexity in the physical world does not mean that human collection will grow in direct proportion to the number of robots. The better models generalize from existing experience, the less repetitive human work each new task may require.
According to its public description in October 2026, Mecka brings together collection hardware, field data collection, motion tracking and 3D reconstruction, and extends its business into robot deployment. This is the company’s own description of its scope. The joint EgoVerse project in which it participated reported 1,362 hours and about 80,000 human demonstrations in the April 2026 version of its paper, and tested transfer across laboratories and robotic platforms. The research found that effective scaling depends on alignment between the data and the learning objective. Recording movement is only the starting point. Whether it can be converted into experience that benefits the model determines its value. The research findings themselves do not establish returns from the company’s deployment activities.
6. A Possible Rise and Fall, with No Known Turning Point
When demand first surges, many models are learning the same missing capabilities at once, new fields keep opening, and purchasing volume and unit value can rise together. As existing capabilities are absorbed, repeat purchases decline and the remaining problems become more expensive, but the market does not necessarily contract immediately. It may pass a peak only when new tasks and price increases can no longer offset the decline in effective purchasing volume.
The market at issue here is revenue from data services in which customers buy human experience from external suppliers. It is not the total social value of data and does not include every investment in model learning. Falling volume and rising unit prices are opposing forces, and neither needs to change linearly. A contraction scenario requires the decline in human procurement for existing tasks to eventually outweigh the offset from new tasks, new human capabilities, observations of external facts and price increases. The balance between in-house production and outsourcing will also affect the market available to suppliers. At present, there are not enough statistics to determine the year or scale of any turning point.
The history of human computers offers a useful reference. NASA has documented how some of these workers moved into programming and other roles as electronic computers developed. Society needed ever more computation, but it no longer needed manual computation to grow at the same rate. This does not prove that the AI data industry must follow the same path. It does show that growth in total demand can coincide with the decline of one form of human supply.
The strongest counterargument must also be preserved. With the help of tools, humans may open up more tasks. New facts will continue to emerge, and the physical world may require large amounts of field experience for a long time. If these increments consistently outpace the speed at which models absorb and generalize, or if critical feedback remains difficult to verify automatically, the human-data collection market may continue to grow. Whether new fields merely extend the window or expand the market on a lasting basis will have to be answered by subsequent purchasing behavior and learning efficiency.
7. Once Models Surpass Humans, Whom Will They Learn From?
Take the end state one step further. If models reliably surpass the best humans in solving problems, making judgments and performing relevant tasks, and if they can generate and verify useful samples more cheaply, then the business of organizing human capability demonstrations may indeed lose most of its value. Renaming the business from “data” to “experts” or “embodied AI” does not avoid the question.
Yet surpassing human capability is not the same as knowing everything about the world. No matter how capable a model is, it cannot determine the current temperature of a particular piece of equipment through reasoning alone. At least three distinct needs remain: what humans already know, what has newly happened in the world and what outcomes humans want. The first can be absorbed. The latter two still require observation, interaction, preference expression and authorization, but they may not require large amounts of human labor, and independent data companies may not be the ones to provide them.
One route to further progress is self-play and learning with verifiable feedback. In games with explicit rules and unambiguous outcomes, AlphaZero used self-play to achieve capabilities beyond those of humans without continually purchasing demonstrations from stronger players. AlphaProof combined a pretrained model, reinforcement learning and the Lean proof system, using successfully verified proofs for further training. Verifying an answer does not always require a human who is better at solving the problem than the model.
Another route is learning from reality. A model proposes a material formulation and a laboratory tests it. A robot attempts an action and sensors record the outcome. An agent performs a task and a system reports whether it actually succeeded. Learning material gradually shifts toward experience generated by machine exploration. Real-world observation may still depend on human field sampling, but what is being purchased in that case is an external fact. The cognitive ability of the person collecting it does not need to exceed the model’s. More of the cost will shift toward equipment, environments, compute, execution and reliable feedback, while simulation gaps, long-cycle outcomes and safety constraints will continue to limit the pace of progress.
Recursive self-improvement, or RSI, describes a process one level higher. AI improves algorithms, training methods and computational efficiency, helping to produce stronger successor systems, which then continue the process of improvement. RSI can be combined with self-play, but it is not the same thing. AlphaEvolve demonstrates one possible component of such a path: models propose programs, automated evaluators verify and score them, and the system finds improvements to computing infrastructure and AI training. It does not, however, prove that stronger successor systems have already formed an autonomous, continuously accelerating recursive loop.
All of these paths depend on reliable verification. Generating more content, or having several models agree with one another, is not enough to prove that capability has improved. Formal proofs, execution results, experiments and real-world feedback allow learning to avoid circling entirely within a model’s own judgments. Data will remain important, but future spending growth may be directed toward machine-acquired experience rather than purchased human experience.
8. Data Companies May Survive, but Not Every Business Will
During the transition, the industry may consist of a small number of integrated platforms alongside many specialist teams. Platforms can spread costs by coordinating complex operations, using technical tools and delivering compliant data. Specialist teams can survive through depth in particular tasks. Low-cost execution providers will face pressure from automation and pricing. A shrinking outsourced human-data market is not inconsistent with growth among market leaders. Gains in share can temporarily offset market decline, but not indefinitely.
Over the longer term, the key distinction will be who continues to control information and feedback that models cannot acquire cheaply. Companies may preserve value if they own access to distinctive environments, clearly authorized and continuously updated data sources, or capabilities that reduce the cost of experimentation and verification. Companies that merely repackage similar labor into deliverables are more vulnerable to replacement by models, platforms or customers’ in-house operations. The ability to organize labor can support a transition, but it does not automatically become a moat in software, hardware or experimental facilities.
Beyond Surge’s task environments, APEX-Agents, released by Mercor’s research team, contains 480 cross-application tasks designed by professionals in investment banking, consulting and law, together with files, reference outputs, scoring criteria and execution-based evaluation infrastructure. Anthropic’s engineering article on agent evaluation similarly considers tasks, execution traces, outcomes and multiple scoring methods together. These projects show how expert labor can extend from providing answers to defining and verifying tasks, but public research does not establish sustainable revenue or software-like margins.
Moving into training environments, verification and automated collection is an expansion into adjacent businesses. It cannot be counted directly as part of the outsourced human-data market described above. The competitive set also changes. Model developers possess internal failure records. Application providers and robotics deployment companies own field feedback. Cloud and simulation platforms control compute and execution environments. Even if spending in these areas continues to grow, the revenue will not necessarily flow to today’s data suppliers. Each new business must still be evaluated on its costs, barriers and customer relationships.
If field collection and verification also become highly automated, the original labor-intermediation layer will lose still more value. Acquiring information will continue to carry a cost, but who bears that cost and who earns the revenue will reflect a different industry structure. Calling all of these new businesses “data” does not prove that the old businesses can keep growing.
The indicators worth tracking next are how many new human samples a given capability requires, how repeat purchases and unit prices for expert data change, whether new fields can absorb the demand lost in older ones, and whether revenue actually comes from human demonstrations or from continuous feedback and execution environments. The test of durability is what a company can still provide tomorrow after the model has learned today’s material.
The pyramid of human experience may not grow upward forever. Models may gradually move beyond it and into domains that humans have not yet explored. Data will still matter then. Finding human teachers for models may no longer matter as much.