Optimization is often immoral.
(shortpost)
If you run a business like a grocery store or a small restaurant, you often can find people who will work for less and less and less money, trading on the desperation of the poor and downtrodden. You’ve got a guy stocking your shelves for $9/hr, and you can probably find someone who’ll do it for $8/hr, or $7/hr, and hey, presto—more money in your pocket.
If you’re interviewing people for a coveted position (say, at an elite university or a popular startup), you probably can just keep on stacking hoops ad infinitum, until you find the people who are most willing to put up with a bullshit selection process that involves having twenty-seven extracurriculars and perfect grades and being willing to wait on a list for four weeks while you see whether you can find someone even better.
But you probably should not do these things, because they are mean and bad. They are the race to the bottom, happening in real time—what Moloch feels like, from the inside, is “well, why wouldn’t I go ahead and take advantage of the fact that I can? Obviously I want the cheapest labor that gets the job done; obviously I want the absolute best candidate that I can find.”
Notice yourself sneaking an eleventh sheep onto the common field (metaphorically speaking). Notice that there is slack in the system, and that you’re taking it away.
(“But if I don’t, others will.” Maybe true. Not a justification.)
There are some situations that call for true optimization, but they’re few and far between—far fewer and far farther than people let themselves believe, self-servingly. I think I really do want the literal best designing and maintaining my nation’s nuclear power plants. I don’t think I want so much selection pressure on entry-level white-collar jobs that it drives an entire generation to anxiety, depression, and burnout.
Optimization is often immoral. Satisfice, instead—set a reasonable bar that gets you what you want, and do not discriminate above that bar. Take the first guy that clears it, or collect all the people who clear it and then run a lottery, or something that doesn’t drive us further and further into the red queen race.
Also worth noting, since I've been learning a lot about how transformers and LLMs work the past few days and am seeing everything in relation to that (https://www.reddit.com/r/PhilosophyMemes/s/zsZjEkcg2B): at the end of a run, the transformer gets a probability distribution over all possible tokens of which comes next. Using a "greedy" selection algorithm, where it always picks the absolute highest most likely word to go next, actually doesn't give as good results in practice as using some other algorithm to pick from among the top choices. The "temperature" parameter modulates how far from the highest probability token it will go
https://forum.effectivealtruism.org/posts/B6d8Wzk4gNzHsXvdi/ai-safety-is-extremely-bottlenecked-on-grantmakers
"hiring one fewer grantmaker usually means those millions will just sit in an account for another year rather than being deployed to useful ends. And when a strong candidate turns down a CG offer, the result is often not “a slightly-less-good grantmaker," it’s just one fewer grantmaker. We routinely close rounds with fewer hires than we'd planned for."
Just an example of this dynamic, imo. See also the comments on that thread.