Killing the Red Queen
ctrl+f [tl;dr] if all you want is the thesis.
0. Introduction
Alice never could quite make out, in thinking it over afterwards, how it was that they began: all she remembers is, that they were running hand in hand, and the Queen went so fast that it was all she could do to keep up with her: and still the Queen kept crying “Faster! Faster!” but Alice felt she could not go faster, though she had not breath left to say so.
The most curious part of the thing was, that the trees and the other things round them never changed their places at all: however fast they went, they never seemed to pass anything. I wonder if all the things move along with us? thought poor puzzled Alice. And the Queen seemed to guess her thoughts, for she cried, “Faster! Don’t try to talk!”
And they went so fast that at last they seemed to skim through the air, hardly touching the ground with their feet, till suddenly, just as Alice was getting quite exhausted, they stopped, and she found herself sitting on the ground, breathless and giddy.
The Queen propped her up against a tree, and said kindly, “You may rest a little now.”
Alice looked round her in great surprise. “Why, I do believe we’ve been under this tree the whole time! Everything’s just as it was!”
“Of course it is,” said the Queen, “what would you have it?”
“Well, in our country,” said Alice, still panting a little, “you’d generally get to somewhere else if you ran very fast for a long time, as we’ve been doing.”
“A slow sort of country!” said the Queen. “Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”
In a red queen race, participants are forced to expend ever-greater amounts of energy simply to maintain their relative positions. Millions of years ago, a proto-cheetah is hunting proto-gazelles, and only the fastest proto-gazelles tend to survive long enough to breed. Over time, the proto-gazelle population becomes faster, and only the fastest proto-cheetahs tend to be successful in bringing them down. The slower proto-cheetahs starve, and thus the proto-cheetah population becomes faster, and the cycle continues.
Fast forward to today, and cheetahs and gazelles now spend a shocking proportion of their caloric budget on growing and maintaining absurdly over-the-top musculature, only to kill and escape (respectively) at roughly the same rates as their much slower ancestors.
In general, once a red queen race has begun, things will continue to ratchet upward until physical limits are hit—
(e.g. trees racing upward for greater access to sunlight eventually reach a point where they can’t grow higher without being blown over by the wind or tearing loose from shallow soil or exhausting all nutrients within reach)
—or until one side forces the other to extinction, or until some other force intervenes, superseding the competition.
I. Standing Out
Imagine the state of mind of a college admissions committee member in, say, the year 1960. Only around 45% of graduating high schoolers apply for college at all; most move directly into the work force, or go into apprenticeships or trade schools.
As you review your handful of candidates, your mental checklist might look something like this:
There just weren’t all that many considerations that were relevant in 1960. It was a slower race back then, relatively speaking. If you had only one slot available for this group, it would go to Elliot, based on the fact that Elliot has alumni parents and a solid letter of recommendation and a high GPA. If you have two slots, it might be a bit of a tough call between Alexis and Finley, depending on how much your school cares about legacy.
Eventually, though, everybody [who is applying to college at all] figures out that those are the things you’re looking for, and by 1980 your field of applicants now looks like:
So you do the obvious thing: you find new distinguishing criteria. After all, you’re looking for the best possible candidate. So perhaps you see who’s getting straight A’s and also is an Eagle scout. Or who’s getting straight A’s and also can play the cello. Or who’s getting straight A’s and also winning athletic championship trophies, or volunteering on weekends, or being published in local poetry anthologies.
Over time, the criteria accumulate, until the chart in 2020 starts to look like this:
The key thing to recognize is that the relative positions of all of these applicants haven’t changed much, if at all. No matter the system, the students with the highest combination of [aptitude], [motivation], [support], and [luck] are going to float toward the top. The very best students are still (mostly) getting into the most competitive schools, and the next tier still (mostly) get their choice among all the other schools.
More people are going to college overall, so some people who would've gone straight to work in 1960, or headed off to a trade school, are going to state schools or community college instead. But there are also more colleges, with more slots. Minus a small amount of noise, everyone is still ending up in pretty much the same place in the hierarchy, just like the cheetahs and the gazelles.
But the cost of purchasing that place has gone way up. The most capable-and-lucky-and-motivated-and-supported people in 1960 had to jump through two or three hoops. Their 2020 counterparts have to jump through a baker’s dozen. The red queen race has consumed all of the slack and spare energy lying around in the system—each individual applicant on the list has hopes of getting ahead, standing out, and making an impression, and many individuals will act on those hopes by putting forth more effort on the margin, and the ultimate effect is that the bar just keeps rising.
Every time a new innovation proves successful in the getting-into-college game, it creates pressure for everyone else to adopt that innovation in the decade following. And each time that a given innovation reaches saturation, such that everyone is doing it, some new discriminator becomes relevant, and a new layer of obligation gets laid down—but without removing any of the previous obligations.
Thus, high school seniors (and juniors, and sophomores, and freshmen, and even middle schoolers and elementary schoolers, as the selection pressure creeps ever outward) spend an enormous amount of their time and energy on building a portfolio of impressive-seeming accomplishments—
—and end up, more or less, right where they would have if their teachers had put their heads together and crowdsourced a few predictions, no extraordinary feats required.
II. Perverse Incentives
There's a game I liked to play with my sixth grade students. I would get a long, thin, capped PVC pipe, theatrically weighted down to the floor with chains and heavy objects. I would assemble my students in two rows, facing each other, their index fingers extended along a central line.
I would tell my students that this was a "helium stick," and that, if they weren't careful, it would try to fly up into the ceiling. Then I would slide the stick down the center of the double row, such that it was resting on each student's index fingers.
The rules of the game were simple:
Do not grip the stick; it has to be resting on your fingers.
Do not allow your fingers to break contact with the stick; they must always always always be touching.
Lower the stick to the ground.
Invariably, the stick immediately begins to rise, panic and vitriol in its wake.
The helium stick game is not exactly a red queen race (though it is a member of the larger class of Molochian dynamics of which red queen races are a subset). I mention it because it elegantly illustrates the core problem of red queen races: the local and short-term incentives are misaligned with the global and long-term good.
Cheetahs and gazelles do not want to run fast; both would be better off overall if the race to survive required less investment in musculature. High schoolers do not want to burn all of their free time jumping through hoops to get into good colleges. My students did not want the stick to fly up toward the ceiling.
But! If a given student stops exerting upward pressure on the stick, their fingers fall out of contact with it, and they receive a game penalty. Thus, even knowing it was contra to their overall goal, each student “had no choice” but to keep pushing upwards a teeny tiny bit to stay in contact, which caused the stick to rise, which lifted it off of their neighbors’ fingers, which caused those neighbors to have to push upwards more, which caused the stick to rise further...
The college application process is a particularly crisp example of this dynamic, but it’s just one among many. College itself is largely a red queen race, at this point—it’s pretty terrible that the best half of jobs/careers/lifestyles are gated behind a paywall that requires you to spend many thousands of dollars and burn four of the potentially-best years of your youth—
(Especially since only a small minority of those jobs/careers/lifestyles actually meaningfully benefit from the process, and almost none relative to simply getting started four years sooner, and having four additional years of on-the-ground training, practice, and experience.)
—but when offered a choice between a potential employee who has The Official Stamp of Approval and a potential employee who doesn’t, who’s going to pick the latter? Outside of a few tiny subcultures like Bay Area tech startups (where having gone unnecessarily to college is sometimes treated as a minor mark of shame), there’s no legible benefit to taking the less-credentialed candidate, if the more-credentialed candidate is just as easy to get.
Thus, unless you expect literally none of the people you’re competing with to have gone to college, you have a strong incentive to go to college, regardless of whether it’s actually helpful for the work you expect to do.
(My own formal education in education added literally zero useful tools or concepts to my teaching toolkit, 70% of which came from an afterschool job teaching Tae Kwon Do as a teenager, and the other 30% of which I developed in contact with actual middle schoolers in actual classrooms.)
And so, before long, “everyone” “has to” go to college, even though the median and modal result is a substantial waste of time and energy and money for no relative gain. College essentially becomes High School II—
(Including all sorts of compulsory horizon-broadening material for which many students have no intrinsic interest, and which they therefore cram and shortcut their way around, rather than engaging with it in the ways that would in fact cause their horizons to be meaningfully broadened.)
—and meanwhile the rot creeps even further, into postsecondary education.
A few other examples of red queen races that have taken off or noticeably accelerated in my lifetime:
Modern-day purity tests. If there’s one last slot at your conference, and you’re debating which of two highly qualified speakers to invite, and one of them is currently in hot water over an off-color joke they retweeted back in 2008 and the other is not, or if both of them are women of color but one of them is also LGBTQ, or if both of them are advocates for certain kinds of social reform but one of them is an advocate for four disadvantaged groups and the other for only two…
Liability, litigiousness, and the creep of bureaucracy and regulation. Everyone races to be the least vulnerable to legal reprisals and the least blame-able in the event of disaster, until friends honestly and sincerely recommended to me that I not allow people to jump on the trampoline at my house until they had signed a waiver.
The race by advertisers to catch your attention while you click around on the internet or walk around in public. Advertisements grow louder and brighter and more intrusive, competing for their share of a mostly-constant maximum potential amount of attention-from-the-audience.
III. Unnatural Selection
The root of the problem (according to me) is unbounded optimization.
The best colleges aren’t looking for students who are good enough. They’re looking for the absolute cream of the crop. Advertisers competing for your finite attention are in a zero-sum game; they need to beat all of the other things that might pull your eyes away. A gazelle that isn’t fast enough gets eaten; a cheetah that isn’t fast enough starves.
By default, a lot of processes are unbounded in this way. It’s much easier to build a thing that points in a direction than to build a thing that homes in on a target. “More” is a more basic and fundamental instruction than “at least that much.”
(Another angle on this dynamic in my previous essay.)
But it is possible to satisfice, rather than optimize. It’s possible to create systems that say “enough is enough.”
There are two primary strategies for defusing red queen races which I encounter frequently in the wild, and neither seems (quite) sufficient to me.
The first is “unilaterally refuse to run in the race.” For example, don’t actually apply to college—go to a coding bootcamp instead, or use the money you would have spent on college to hire a tutor, or “just go learn a trade.”
This is not a strawman—I do in fact actually hear this advice bandied about a lot. But it seems to me that it will only ever be truly workable for a tiny privileged fraction of society—
(The gazelles who are beefy enough to just stare down a cheetah, metaphorically speaking, and therefore don’t have to run.)
—and it doesn’t solve the problem for the rest of us. Plus, with enough pressure, the “out” becomes its own red queen race quickly enough. Coding bootcamps in 2018 were already noticeably more expensive, more exclusive, and less effective than they were in 2012; I’m not sure if they were viable at all by 2024.
The second class of solution is collective action in resistance, à la “if we all boycott the SAT, they can’t use it as a filter.”
This seems to me to be fragile, for the same reasons that the helium stick floated up to the ceiling. There are tremendous incentives to defect when that sort of action is proposed, and even if the first-order incentives don’t deter you, the larger the group involved, the more you should anticipate that other people will defect, and therefore the more sense it makes for you to defect self-protectively.
Coordination schemes like unions do succeed a nonzero amount of the time, but mostly where there are tight-knit groups with strong identity and lots of explicit policing from other members; that sort of strategy is unlikely to (e.g.) cause tens of millions of Americans to be able to avoid a college experience they don’t actually want and don’t straightforwardly need.
More importantly, neither [refusing to play] nor [colluding with the other players] does anything about the selection pressure itself.
My proposed solution: taxes.
Not in the literal sense, as in “money collected from individuals and groups and then spent by the government.” But I think that “taxes” is a good handle for a class of sacrifices that people already understand, and are at least provisionally willing to make. I think that framing [the suggestion I’m about to make] as a tax is the way you would go about convincing people that it’s worthwhile and feasible.
In the case of college applications, what drives the red queen race is elite institutions’ desire to find and attract only the very, very best. In an ordinary year, Harvard accepts around 2,000 students, and rejects close to 60,000. They don’t want to waste one of those precious slots on someone who’s merely excellent.
(The answer to “why?” is complicated, but the short version is that Harvard wants the self-reinforcing feedback loop of being the best and therefore producing the best and therefore being the best and therefore attracting the best and therefore being the best and therefore producing the best…)
The problem is, the cost of Harvard’s desire for perfection isn’t paid by Harvard. It’s paid for by everyone else, who has to run faster and faster in the red queen race that results.
To defuse and disincentivize red queen races, a society needs to tax the group that benefits from the overall dynamic—the group that creates the fitness reward for each new innovation, and thereby generates the pressure which drives the treadmill.
In the case of college applications, the beneficiary is the college itself, and the benefit is increased discriminatory power. By looking deeper and deeper into every aspect of an applicant's life—by making more and more of an applicant's life fair game, and thus under pressure to be optimized for impressiveness—colleges gain the ability to be more sure that they have found the absolute best candidates, along the axes they care about.
Where just looking at GPA and SAT scores would leave them with a pile of undifferentiable high achievers, looking at GPA and SAT scores and extracurriculars and athletics and demographics and community service and intersectionality and so on and so forth allows them to make ever-finer distinctions (including separating the able-to-endure from the unwilling-to-jump-through-hoops).
The proposed tax, then, is this: limit the ability of colleges to be that confident, about distinctions that fine. Colleges would be asked, by the overall culture, to make a sacrifice of a fraction of a percentage point in the quality of their admissions filters, to pay for the ability of everyone else to relax.
(Or, not asked, actually, but rather compelled by legislation.)
Such practices already exist, to some degree—think of how some applications are “blinded,” so that admissions staff can avoid introducing unconscious bias against certain classes of names. We already recognize, in at least some cases, that we must sometimes plug our ears and pretend we don’t know things (even though we would benefit locally from knowing them), because we would all suffer, globally, from a policy of acknowledging them. Easy examples besides blinded applications include:
Facts which are deemed inadmissible in court
Private medical information
The sealed records of minors who committed crimes and served in juvenile detention
Things like sexual orientation and religious persuasion, which are banned from consideration in most workplaces
(More contentious and less universally agreed-upon examples include things like “what if we don’t cancel people over a borderline racist Halloween costume they wore twenty years ago, when social mores were different?” Individuals and groups can agree to not-count information that is plausibly relevant, and which would allow them to better distinguish the moral superheroes from the morally ordinary, because they can see the world that would result from unlimited scrutiny, and do not feel comfortable with it.)
In the case of college applications, what this would look like is creating a whitelist of “things we allow public institutions to use as discriminators,” with the items on that whitelist carefully selected so as to ameliorate (as much as possible) the red queen race dynamics. It would exclude categories of information which will tend to drive applicants to exhaustion—such as, for instance, any consideration of extracurriculars beyond a student’s chosen most-impressive first two.
To be clear: this is a very real cost, to the colleges. It is genuinely a tax, no less so than taxes of money. The ability to distinguish the 99.9th percentile applicant from the 99.99th percentile applicant is highly valuable, and losing that is a real hit.
But—I claim—a sufficiently mature society is capable of taking a long enough view to see that the alternative is a massive and grotesquely wasteful signaling game. We accept and treat-as-normal the current state of college applications because we've been steeping in it for decades—if we were teleported back to 1960 and given a clear choice between having colleges be 1% less adroit at distinguishing their applicants or having the upper quartile of our high schoolers spend 20 and 30 hours per week outside of school in pursuit of the perfect portfolio, we would have no trouble identifying which of these costs is lesser, and which of these worlds is better.
A few related thoughts about the specific example of college applications, before stepping back out to the general case:
First, while the proposed tax is indeed a real cost, and I don’t want to minimize that, it’s not quite as much of one as it might seem at first blush, because college applications are already “noisy” and much of what goes on in them is Goodharting and cargo culting. It’s just actually not that easy, when looking at the top 1% of applicants, to distinguish between genuine enthusiasm and diligent fakery, or to sniff out subtle lies and exaggerations (especially when parents, teachers, and coaches are often interested in colluding with their students to outwit the admissions department). The science of assessment is still in its infancy, and a genuinely mature society would be honest enough to admit that what is being lost isn’t quite
[the ability to distinguish the 99.9th percentile from the 99.99th]
…so much as it is something like a sixty-percent chance of successfully doing so in any given case.
(Numbers made up, but still.)
Second, while this is even more of a side note and should possibly be its own essay, much of the bloat of college applications in particular comes from the fact that our current society is trending toward de facto banning straightforward objective assessments, and colleges have had to scramble to cobble together politically correct alternatives. A society sufficiently sane to curtail colleges’ information seeking would presumably also be sane enough to just let people use aptitude tests—not only for college, but for avoiding college and going straight into one’s field of interest, as well.
Third, it is absolutely true that, in this world, colleges and students both would be incentivized to sneak around the regulations. The claim is not “no one would develop stealth ways to signal marginal impressiveness” or “no one would feel pressure to pack their schedule.”
But there is tremendous value in creating any protected space at all. In changing the default expectation, and putting hard common-knowledge limits on what is reasonable and permissible to ask of students.
And in this imagined saner society, everyone knows exactly why this tax is levied, and so there would be at least some people in every admissions department who are intrinsically and morally motivated to serve as a check on the self-interest of both students and the college as a whole (just as with our current ethics there are at least some people in every admissions department who are intrinsically and morally motivated to ensure that people can’t just buy their way into whatever school they like).
The point is not to eradicate the incentives entirely, but rather to create counterbalancing incentives that give us a little bit of breathing room, and make driving-the-red-queen-race-forward a less overtly appealing option in every individual case.
IV. Are such interventions even possible?
Yes. Cultures differ wildly on all sorts of axes, and this is one of them. Additionally, culture does in fact change, sometimes even quickly. Justifying “this isn’t just a complete non-starter” is its own essay, but I wanted to at least assert that you can in fact get there, from here.
(Which is not to say that it’s easy.)
V. [tl;dr]
The general claim, then, is this:
0. Of red queen races in human society, many (most? all?) take the form of a runaway escalation of obligations, primarily driven by an increase in the scope and quantity of data being used to make comparisons between people. As more people max out the known metrics, more metrics are added (but none are taken away). Meanwhile, despite working harder and harder, people’s relative positions tend not to improve.
1. To short-circuit this pattern, a society would do well to put an effective cap on discernment, making some information inadmissible and sacrificing a small degree of precision in judgment.
2. Because this is an area in which people are not currently accustomed to making sacrifices for the collective good, the recommended strategy for explaining, popularizing, and implementing such caps is to lean heavily on the metaphor of taxes.
Anywhere that competition is driving us to ruin, it is possible for us to look ahead, see the path we’re racing down, and choose to do something else.
This is not true for cheetahs and gazelles, and there is a certain kind of cynical human who will scoff that it’s not true for us, either. And certainly there are plenty of examples of this sort of thing failing.
But there are examples of it working, too. It seems worth it to at least try to take advantage of the edge we have over blind processes like natural selection.
One final note, which hopefully will become its own essay someday but for the moment is worth stating in brief:
It seems to me that a major prerequisite of the success of any such initiative is either a sufficient density of long-view reasonableness (such that the majority of the population can be expected to grok that it's better to make a painful sacrifice now than pay a much larger, increasing, and ongoing cost forever), or something like robust fellow-feeling.
In other words, people will either be motivated by enlightened self-interest or, if their self-interest fails to be particularly enlightened, by regular run-of-the-mill human emotion. In particular, while many people are willing to make even quite substantial sacrifices for their ingroup, they are utterly unwilling to offer the slightest concession to their outgroup (c.f. COVID-19 and masks).
If you feel like the people benefitting from this kind of intervention are a part of your tribe and within your circle of concern, the intervention “pays for itself.” If not, though—if the taxed organizations feel resentful and put-upon rather than noble and dutiful—then they will work tirelessly to subvert regulation in favor of their own self-interest. Absent sufficient long-view reasonableness to make a compelling case in lay game-theoretic terms, I recommend that any attempts to try curtailing discernment start with a subgroup where people already feel substantially responsible for and warm toward one another.
Thank you for yet another very enlightening post. I actually have an example from my job that shows it is indeed very much possible to act against these red queen races. I teach at in a university program in computer science, which next year will accept 590 new students (up from 550 this year) but gets around 3000 applicants. In practice, 245 of the positions are reserved for Dutch-speaking applicants but the number of those applicants is below the maximum, so it is more like almost 3000 candidates competing for around 345 positions. Since this is a European university, the main selection is done through a "cognitive skills test" (which IMHO is already much better than whatever it is American universities are doing, but I digress). Until recently, the procedure was to simply rank all the candidates according to their test scores and pick the best ones. But this created a kind of red queen race, in that some students studied *very very* hard for exactly the kind of questions they would get at this entrance exam. Our data on the actual success rates of these students showed that beyond a certain threshold, the test scores were essentially noise and had zero predictive power on the actual success rates of these applicants in their studies. So eventually some people recognized that this was creating extra work for everyone and fostered a very competitive atmosphere compared to before (when the selection was less strict).
So this year the admission committee made a heroic decision to kill the red queen by changing the procedure to instead take all students who pass the threshold and do a random selection from those. (Well not quite: there candidates are divided into three buckets and the first bucket has basically guaranteed entrance, and the second bucket is selected randomly.) When I first heard about this I was a bit offended that we would resort to a lottery for such an important decision, but after thinking about it I now believe it was a very good and brave thing to try. I already heard that the aerospace program might introduce the same policy. Let's hope other universities take notice and follow suit.
Here's the link with all the information about our selection procedure, should anyone be interested: https://www.tudelft.nl/en/onderwijs/opleidingen/bachelors/computer-science-and-engineering/bachelor-of-computer-science-and-engineering/from-application-to-enrollment/selection-procedure
What I wish for is people starting more universities which are aiming at supplying excellent educations, but the incentives don't encourage this.