A newborn sea turtle has a few days of stored energy to reach the safety of the open sea. It cannot afford to waste a minute on the beach. So the marvelous processes of evolution led to sea turtle hatchlings following a simple rule: Emerge at night and crawl toward the brightest horizon.
For most of turtle history, this evolutionary feature led to most baby sea turtles finding the sea successfully—on a natural beach, the brightest direction is the open sky above the sea. Humans changed that. Coastal development brought streetlights, hotels, homes, and other sources of light that weren’t the sky.
But no one told the hatchlings. They still follow the same instinct, and this year on Florida’s Anna Maria Island, volunteers recorded more than 3,000 of the tiny turtles heading the wrong way; 55 were found dead on a road in August. Biologists call this an evolutionary trap: A sudden change in the environment breaks the link between a cue and the reality it used to signal. Behavior that once worked—crawling toward the brightest horizon—starts to fail.
Thanks to AI, a wide variety of human systems are now experiencing their own versions of the evolutionary trap. These systems are being used as intended, but AI has changed the environment they operate in and is putting them under a load they were never built to carry. And the cue that organizations have long relied on to tell them they are making progress—a rising volume of output—no longer means what it once did.
The sea turtle hatchlings are crawling to their doom. Here are three fundamental issues that business leaders need to understand if they want to avoid the same fate.
1. Volume: The end of the natural limit
Human systems in business, government, and academia are being placed under enormous strain. On one leading hiring platform, the average job ad now draws around 254 applicants, and the number of applications each recruiter has to handle is up 412%. In May, the U.K.’s Information Commissioner’s Office had to issue new guidance to help public authorities cope with a rising volume of freedom of information requests generated with AI. And in September, arXiv, the preprint server where much of the world’s physics, mathematics, and computer science research first appears, received 40,363 submissions—a record, double the figure from two years earlier.
It may be hard to remember now, lost as it is in the mists of the hoary past, but producing things like job applications, research papers, and formal requests for information used to take real time and serious effort. That put a natural constraint on the volume and speed of production; as arXiv’s announcement put it, there used to be “an easily discernible, practical limit” to how fast submissions could be produced.
AI is destroying that limit by drastically lowering the marginal costs of production. It is also allowing many new producers to enter the market, because it significantly lowers the knowledge required to produce things like papers or government applications and complaints.
All that production still needs to be processed—someone has to read those job applications, evaluate those papers, respond to those requests for hitherto unpublished information. And while AI can help with this, it cannot do so at anywhere near the same rate as it can help with production, because while producing something requires only that it seem right, reviewing it means deciding that it is right.
So the final judgment—of math proofs, of job applications, of government claims—stays with human beings. And this creates a radical asymmetry. The production side has been supercharged with AI while the review side is ultimately stuck at the human pace.
2. Signaling: When effort stops meaning anything
Nobody facing a stack of job applications has the time to examine each one from scratch. They rely on signals to decide what deserves attention. But how to separate signal from noise? According to Michael Spence, who shared the 2001 Nobel Prize in economics for his work in signaling theory, signals carry information when they are expensive to send. So for example, a cover letter clearly written for a particular role worked as a signal to pay closer attention to the applicant because writing that bespoke letter took time and effort.
AI will completely destroy that kind of signaling effect. AI makes it straightforward for everyone to generate bespoke cover letters, so such letters can no longer be used to differentiate between applicants. A serious candidate becomes hard to tell apart from someone who auto-applied to a thousand jobs overnight.
None of this means AI-assisted submissions are necessarily bad. In fact, the problem arises precisely because they aren’t—it is because some AI-assisted submissions are good, and all of them are polished, that the reviewers of applications can no longer use conventional signals to quickly sort applications. And that leaves two expensive options: Either get a human being to check everything, which takes time and money; or leave the sorting to AI.
Attractive as it might be to fight fire with fire, this latter approach comes with its own issues. One of the biggest problems is that AI screeners tend to favor résumés written by AI. One study simulated hiring across 24 occupations and found that candidates who used the same AI model as the employer were 23% to 60% more likely to be shortlisted than equally qualified people who wrote their own résumés. This approach to sorting, then, ends up rewarding precisely the kind of polish that has stopped meaning anything.
AI screeners can also inherit old prejudices. Researchers found that three state-of-the-art AI models favored résumés with white-associated names 85% of the time, and a federal court has allowed a discrimination lawsuit over Workday’s AI screening tools to proceed. For the moment, then, AI cannot be relied on to separate signal from noise in the output it has made so easy to produce.
3. Understanding: Answers no one can follow
In October, OpenAI published AI-generated solutions to more than 370 outstanding math problems. This is an exciting and remarkable development. But as the mathematician Daniel Litt said, “to extract understanding from [the OpenAI results] there will be a huge amount of human labor involved.”
Litt is pointing at something businesses should take very seriously: Knowledge is not the same as understanding, and AI is producing the first far faster than humans can supply the second. Software engineers already have a name for the result: comprehension debt, the growing gap between how much code exists and how much of it anyone actually understands.
Does that gap matter if the output is right? Sometimes it doesn’t. A forecast that proves accurate is useful whether or not anyone knows how it was made. The trouble comes when circumstances change. Without understanding, a business can keep acting on a model’s confident answers long after the conditions that made them reliable have gone. Accountability is the other problem. A company remains answerable for the decisions it hands to machines, and when a regulator or a court asks why something happened, someone has to be able to explain it.
There is also a subtler cost. As Terence Tao, one of the world’s greatest mathematicians, has argued, quite often it is the process of reaching a solution, rather than the solution itself, that advances understanding. A company that lets AI do the working-out may still get its answers, but it skips the work that would have led to its next breakthrough idea.
The new scarcity
Until recently, knowledge work was constrained by the amount of time and effort it took humans to produce. AI has removed this constraint and replaced it with a different one. What is scarce now is not the ability to create output but the capacity to read and review everything that AI can help produce.
Job applications, research papers, and information requests are particularly visible signs of this shift because they cross institutional boundaries and are easy to count. But the same shift is underway inside organizations that do knowledge work. Every AI-assisted memo or slide deck an employee produces creates additional work for a colleague or boss who has to determine whether it’s useful.
No organization can give careful attention to everything its people can now produce, so leaders have to decide where that attention goes. This is, in part, a design problem.
Greenhouse, the hiring platform, now lets job seekers flag just one application per month as their top priority. The flag carries information for the same reason a tailored cover letter once did: It costs the sender something. This time the cost is one that AI cannot reduce, because an applicant who has used this month’s flag has no other to use. The company’s CEO, Daniel Chait, says candidates who use it are hired at roughly five times the rate of everyone else.
The same logic applies inside an organization. When effort no longer limits how many documents and how much code employees can hand one another for review, a limit has to be designed in, forcing people to choose which pieces of their own output most deserve attention.
Limits of this kind help separate signal from noise. But whatever gets through still has to be assessed by someone who understands the work well enough to tell the good from the merely plausible. And that understanding comes from personal experience with the work in question. An organization that hands a large part of its reasoning to AI models will find, over time, that it has fewer people able to judge what the machines produce.
The hatchlings on Anna Maria Island crawl toward the brightest horizon because, for most of their history, the brightest horizon was the one over the sea. Organizations have long treated growth in output as a sign of progress. But AI is destroying the value of that signal, and an organization that keeps steering by it is heading for the streetlights instead of the ocean. Changing course will require new signals, and people who are sufficiently immersed in the work to be able to read them and judge them.








